Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception in Process Pool #92

Open
wahello opened this issue Apr 21, 2020 · 2 comments
Open

Exception in Process Pool #92

wahello opened this issue Apr 21, 2020 · 2 comments

Comments

@wahello
Copy link

wahello commented Apr 21, 2020

When I use jsons in main process is ok, while json used in process pool raise Exception.
@ramonhagenaars
Copy link
Owner

Hi @wahello. I'm gonna need more information to reproduce your problem. Could you provide me the following information:

  • which Python version you were using
  • which version of jsons you were using
  • a piece of code that calls a jsons function that caused the Exception
  • the complete stack trace

@wahello
Copy link
Author

wahello commented Apr 27, 2020

python version 3.7
jsons version 1.1.1

{'md5': 'aeb8399610e8a4fadee106aa590b294f', 'domain_id': '155', 'url': 'https://www.jianshu.com/nb/8834947', 'title': '随笔', 'domain': 'www.jianshu.com', 'category_id': '3', 'http_status': None, 'spider_status': '0', 'visits': '1', 'size': None, 'addtime': '2020-01-11 09:34:29', 'updatetime': '2020-01-11 09:34:29'}

Stack trace is

Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 470, in _handle_results
    task = get()
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py", line 251, in recv
    return _ForkingPickler.loads(buf.getbuffer())
@dataclass
class DomainTaskModel():
    id: int
    domain_id: int
    domain: str
    title: Optional[str]
    url: str
    category_id: Optional[int]
    http_status: Optional[int]
    spider_status: Optional[int]
    visits: Optional[int]
    size: Optional[int]
    md5: str
    addtime: Optional[datetime]
    updatetime: Optional[datetime]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants