Skip to content

Commit b91eb8e

Browse files
Alexandre LandeauAlexandre Landeau
Alexandre Landeau
authored and
Alexandre Landeau
committedMay 12, 2023
changed default number of processes to prevent bug on MacOS and python 3.8
1 parent eee133d commit b91eb8e

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed
 

‎python-lib/spacy_tokenizer.py

+2-1
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,8 @@ class MultilingualTokenizer:
122122

123123
DEFAULT_BATCH_SIZE = 1000
124124
MAX_NUM_CHARACTERS = 10 ** 7
125-
DEFAULT_NUM_PROCESS = 2
125+
# Set to 1 to prevent pickling issues when spawning multiple processes on MacOS
126+
DEFAULT_NUM_PROCESS = 1
126127
DEFAULT_FILTER_TOKEN_ATTRIBUTES = {
127128
"is_space": "Whitespace",
128129
"is_punct": "Punctuation",

0 commit comments

Comments
 (0)