Sourcery refactored master branch #1
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Comment on lines
-38
to
-39
| for _ in dataset: | ||
| pass |
There was a problem hiding this comment.
Function read_unformated refactored with the following changes:
- Remove redundant pass statement (
remove-redundant-pass) - Remove nested block which has no effect (
remove-empty-nested-block)
Comment on lines
-48
to
-49
| for _ in dataset: | ||
| pass |
There was a problem hiding this comment.
Function read_formatted_as_numpy refactored with the following changes:
- Remove redundant pass statement (
remove-redundant-pass) - Remove nested block which has no effect (
remove-empty-nested-block)
Comment on lines
-107
to
+103
| times[read_func.__name__ + " after write_array2d"] = read_func(feats, tmp_dir) | ||
| times[f"{read_func.__name__} after write_array2d"] = read_func(feats, tmp_dir) |
There was a problem hiding this comment.
Function benchmark_array_xd refactored with the following changes:
- Use f-string instead of string concatenation [×3] (
use-fstring-for-concatenation)
Comment on lines
-81
to
+96
| print(func.__name__, str(kwargs)) | ||
| times[func.__name__ + " " + " ".join(str(v) for v in kwargs.values())] = func(dataset, **kwargs) | ||
| print(func.__name__, kwargs) | ||
| times[ | ||
| f"{func.__name__} " + " ".join(str(v) for v in kwargs.values()) | ||
| ] = func(dataset, **kwargs) | ||
|
|
||
|
|
||
| print("shuffling dataset") | ||
| dataset = dataset.shuffle() | ||
| print("Second set of iterations (after shuffling") | ||
| for func, kwargs in functions_shuffled: | ||
| print("shuffled ", func.__name__, str(kwargs)) | ||
| times["shuffled " + func.__name__ + " " + " ".join(str(v) for v in kwargs.values())] = func( | ||
| dataset, **kwargs | ||
| ) | ||
| print("shuffled ", func.__name__, kwargs) | ||
| times[ | ||
| f"shuffled {func.__name__} " | ||
| + " ".join(str(v) for v in kwargs.values()) | ||
| ] = func(dataset, **kwargs) | ||
|
|
There was a problem hiding this comment.
Function benchmark_iterating refactored with the following changes:
- Remove unnecessary call to
str()withinprint()[×2] (remove-str-from-print) - Use f-string instead of string concatenation [×3] (
use-fstring-for-concatenation)
Comment on lines
-34
to
+36
| title += " " + metric_name + " |" | ||
| title += f" {metric_name} |" | ||
| lines += "---|" | ||
| value += val_str + " |" | ||
| value += f"{val_str} |" |
There was a problem hiding this comment.
Function format_json_to_md refactored with the following changes:
- Use f-string instead of string concatenation [×3] (
use-fstring-for-concatenation)
| @@ -16,6 +16,7 @@ | |||
| """Amazon Customer Reviews Dataset --- US REVIEWS DATASET.""" | |||
|
|
|||
There was a problem hiding this comment.
Lines 101-101 refactored with the following changes:
- Use f-string instead of string concatenation [×2] (
use-fstring-for-concatenation)
Comment on lines
-179
to
+180
| for i, row in enumerate(reader): | ||
| yield i, row | ||
| yield from enumerate(reader) |
There was a problem hiding this comment.
Function AmazonUSReviews._generate_examples refactored with the following changes:
- Replace yield inside for loop with yield from (
yield-from)
| @@ -16,6 +16,7 @@ | |||
| """AmbigQA: Answering Ambiguous Open-domain Questions""" | |||
|
|
|||
There was a problem hiding this comment.
Lines 43-46 refactored with the following changes:
- Use f-string instead of string concatenation [×2] (
use-fstring-for-concatenation)
Comment on lines
-102
to
+100
| features_dict.update(detail_features) | ||
| features_dict |= detail_features |
There was a problem hiding this comment.
Function AmbigQa._info refactored with the following changes:
- Merge dictionary updates via the union operator (
dict-assign-update-to-union)
Comment on lines
-107
to
+109
| path_dict = dict() | ||
| path_dict = {} | ||
| for round_tag in ["R1", "R2", "R3"]: | ||
| path_dict[round_tag] = dict() | ||
| path_dict[round_tag] = {} |
There was a problem hiding this comment.
Function ANLI._split_generators refactored with the following changes:
- Replace dict() with {} [×2] (
dict-literal)
| for idx, line in enumerate(open(filepath, "rb")): | ||
| for line in open(filepath, "rb"): |
There was a problem hiding this comment.
Function ANLI._generate_examples refactored with the following changes:
- Remove unnecessary calls to
enumeratewhen the index is not used (remove-unused-enumerate)
| @@ -15,6 +15,7 @@ | |||
| """AQUA-RAT (Algebra Question Answering with Rationales) Dataset""" | |||
|
|
|||
There was a problem hiding this comment.
Lines 56-60 refactored with the following changes:
- Use f-string instead of string concatenation [×6] (
use-fstring-for-concatenation)
| features = {} | ||
|
|
||
| features["tweetID"] = datasets.Value("int64") | ||
| features = {"tweetID": datasets.Value("int64")} |
There was a problem hiding this comment.
Function ArCov19._info refactored with the following changes:
- Merge dictionary assignment with declaration (
merge-dict-assign)
| @@ -15,6 +15,7 @@ | |||
| """Arabic Billion Words Corpus""" | |||
|
|
|||
There was a problem hiding this comment.
Lines 44-53 refactored with the following changes:
- Use f-string instead of string concatenation [×10] (
use-fstring-for-concatenation)
Comment on lines
-138
to
+140
| out = re.findall(r"" + pattern, sample.group(0), re.MULTILINE | re.DOTALL) | ||
| out = re.findall(f"{pattern}", sample.group(0), re.MULTILINE | re.DOTALL) |
There was a problem hiding this comment.
Function ArabicBillionWords._extract_tags refactored with the following changes:
- Use f-string instead of string concatenation (
use-fstring-for-concatenation)
Comment on lines
-119
to
+124
| if self.config.name == "full" or self.config.name == "ptpt": | ||
| if self.config.name in ["full", "ptpt"]: | ||
| train_paths.append(os.path.join(data_dir, "assin-ptpt-train.xml")) | ||
| dev_paths.append(os.path.join(data_dir, "assin-ptpt-dev.xml")) | ||
| test_paths.append(os.path.join(data_dir, "assin-ptpt-test.xml")) | ||
|
|
||
| if self.config.name == "full" or self.config.name == "ptbr": | ||
| if self.config.name in ["full", "ptbr"]: |
There was a problem hiding this comment.
Function Assin._split_generators refactored with the following changes:
- Replace multiple comparisons of same variable with
inoperator [×2] (merge-comparisons)
| rest = "[" + rest | ||
| rest = f"[{rest}" |
There was a problem hiding this comment.
Function Atomic._generate_examples refactored with the following changes:
- Use f-string instead of string concatenation (
use-fstring-for-concatenation)
| for idx, line in enumerate(f): | ||
| for line in f: |
There was a problem hiding this comment.
Function BabiQa._generate_examples refactored with the following changes:
- Remove unnecessary calls to
enumeratewhen the index is not used (remove-unused-enumerate) - If else clause is always executed move code to same level as loop (
useless-else-on-loop)
This removes the following comments ( why? ):
# After last line
| if line == "" or line == "\n": | ||
| if line in ["", "\n"]: |
There was a problem hiding this comment.
Function Bc2gmCorpus._generate_examples refactored with the following changes:
- Replace multiple comparisons of same variable with
inoperator (merge-comparisons)
Comment on lines
-133
to
+143
| yield f"{file_idx}_{line_idx}", { | ||
| "fname": fname.name, | ||
| "char": chars, | ||
| "char_type": char_types, | ||
| "is_beginning": is_beginnings if split == "train" else [0 for i in range(len(chars))], | ||
| } | ||
| yield ( | ||
| f"{file_idx}_{line_idx}", | ||
| { | ||
| "fname": fname.name, | ||
| "char": chars, | ||
| "char_type": char_types, | ||
| "is_beginning": is_beginnings | ||
| if split == "train" | ||
| else [0 for _ in range(len(chars))], | ||
| }, | ||
| ) |
There was a problem hiding this comment.
Function Best2009._generate_examples refactored with the following changes:
- Replace unused for index with underscore (
for-index-underscore)
Comment on lines
-75
to
78
| name = "%s_to_%s" % (language_pair[0], language_pair[1]) | ||
| name = f"{language_pair[0]}_to_{language_pair[1]}" | ||
|
|
||
| description = f"Translation dataset from {language_pair[0]} to {language_pair[1]} or {language_pair[1]} to {language_pair[0]}." | ||
|
|
There was a problem hiding this comment.
Function BianetConfig.__init__ refactored with the following changes:
- Replace interpolated string formatting with f-string [×2] (
replace-interpolation-with-fstring)
Comment on lines
-141
to
-148
| result = ( | ||
| yield ( | ||
| sentence_counter, | ||
| { | ||
| "id": str(sentence_counter), | ||
| "translation": {lang1: x, lang2: y}, | ||
| }, | ||
| ) | ||
| yield result |
There was a problem hiding this comment.
Function Bianet._generate_examples refactored with the following changes:
- Inline variable that is immediately yielded (
inline-immediately-yielded-variable)
| folder = l1 + "-" + l2 | ||
| folder = f"{l1}-{l2}" |
There was a problem hiding this comment.
Function BiblePara._generate_examples refactored with the following changes:
- Use f-string instead of string concatenation [×2] (
use-fstring-for-concatenation)
Comment on lines
-125
to
+130
| {k: os.path.join(dl_path, "bigPatentData", k + ".tar.gz") for k in split_types} | ||
| { | ||
| k: os.path.join(dl_path, "bigPatentData", f"{k}.tar.gz") | ||
| for k in split_types | ||
| } | ||
| ) | ||
|
|
There was a problem hiding this comment.
Function BigPatent._split_generators refactored with the following changes:
- Use f-string instead of string concatenation (
use-fstring-for-concatenation)
Comment on lines
-148
to
164
| "train": "https://archive.org/download/biomrc_dataset/biomrc_large/dataset_train{}.json.gz".format( | ||
| setting | ||
| ), | ||
| "val": "https://archive.org/download/biomrc_dataset/biomrc_large/dataset_val{}.json.gz".format( | ||
| setting | ||
| ), | ||
| "test": "https://archive.org/download/biomrc_dataset/biomrc_large/dataset_test{}.json.gz".format( | ||
| setting | ||
| ), | ||
| "train": f"https://archive.org/download/biomrc_dataset/biomrc_large/dataset_train{setting}.json.gz", | ||
| "val": f"https://archive.org/download/biomrc_dataset/biomrc_large/dataset_val{setting}.json.gz", | ||
| "test": f"https://archive.org/download/biomrc_dataset/biomrc_large/dataset_test{setting}.json.gz", | ||
| } | ||
|
|
||
| elif self.config.biomrc_version == "small": | ||
| urls_to_download = { | ||
| "train": "https://archive.org/download/biomrc_dataset/biomrc_small/dataset_train_small{}.json.gz".format( | ||
| setting | ||
| ), | ||
| "val": "https://archive.org/download/biomrc_dataset/biomrc_small/dataset_val_small{}.json.gz".format( | ||
| setting | ||
| ), | ||
| "test": "https://archive.org/download/biomrc_dataset/biomrc_small/dataset_test_small{}.json.gz".format( | ||
| setting | ||
| ), | ||
| "train": f"https://archive.org/download/biomrc_dataset/biomrc_small/dataset_train_small{setting}.json.gz", | ||
| "val": f"https://archive.org/download/biomrc_dataset/biomrc_small/dataset_val_small{setting}.json.gz", | ||
| "test": f"https://archive.org/download/biomrc_dataset/biomrc_small/dataset_test_small{setting}.json.gz", | ||
| } | ||
|
|
||
| else: | ||
| urls_to_download = { | ||
| "test": "https://archive.org/download/biomrc_dataset/biomrc_tiny/dataset_tiny{}.json.gz".format( | ||
| setting | ||
| ) | ||
| "test": f"https://archive.org/download/biomrc_dataset/biomrc_tiny/dataset_tiny{setting}.json.gz" | ||
| } | ||
|
|
There was a problem hiding this comment.
Function Biomrc._split_generators refactored with the following changes:
- Simplify conditional into switch-like form [×7] (
switch) - Replace call to format with f-string [×7] (
use-fstring-for-formatting)
Comment on lines
-128
to
-135
| result = ( | ||
| yield ( | ||
| sentence_counter, | ||
| { | ||
| "id": str(sentence_counter), | ||
| "text": row, | ||
| }, | ||
| ) | ||
| yield result |
There was a problem hiding this comment.
Function Cc100._generate_examples refactored with the following changes:
- Inline variable that is immediately yielded (
inline-immediately-yielded-variable)
| url = my_urls + "en_XX-" + self.config.language_code + ".tsv.xz" | ||
| url = f"{my_urls}en_XX-{self.config.language_code}.tsv.xz" |
There was a problem hiding this comment.
Function CcalignedMultilingual._split_generators refactored with the following changes:
- Use f-string instead of string concatenation [×3] (
use-fstring-for-concatenation)
Comment on lines
+205
to
+215
| elif reverse: | ||
| yield id_, { | ||
| "translation": {lc: data[0].strip(), "en_XX": data[1].strip()}, | ||
| "LASER_similarity": data[2], | ||
| } | ||
|
|
||
| else: | ||
| if not reverse: | ||
| yield id_, { | ||
| "translation": {"en_XX": data[0].strip(), lc: data[1].strip()}, | ||
| "LASER_similarity": data[2], | ||
| } | ||
| else: | ||
| yield id_, { | ||
| "translation": {lc: data[0].strip(), "en_XX": data[1].strip()}, | ||
| "LASER_similarity": data[2], | ||
| } | ||
| yield id_, { | ||
| "translation": {"en_XX": data[0].strip(), lc: data[1].strip()}, | ||
| "LASER_similarity": data[2], | ||
| } |
There was a problem hiding this comment.
Function CcalignedMultilingual._generate_examples refactored with the following changes:
- Merge else clause's nested if statement into elif (
merge-else-if-into-elif) - Swap if/else branches (
swap-if-else-branches)
| self.split_file = os.path.join(directory, name + ".json") | ||
| self.split_file = os.path.join(directory, f"{name}.json") |
There was a problem hiding this comment.
Function CfqConfig.__init__ refactored with the following changes:
- Use f-string instead of string concatenation (
use-fstring-for-concatenation)
Comment on lines
-95
to
+99
| if split == "train": | ||
| batches = ["data_batch_1", "data_batch_2", "data_batch_3", "data_batch_4", "data_batch_5"] | ||
|
|
||
| if split == "test": | ||
| batches = ["test_batch"] | ||
|
|
||
| elif split == "train": | ||
| batches = ["data_batch_1", "data_batch_2", "data_batch_3", "data_batch_4", "data_batch_5"] |
There was a problem hiding this comment.
Function Cifar10._generate_examples refactored with the following changes:
- Simplify conditional into switch-like form (
switch)
Sourcery Code Quality Report✅ Merging this PR will increase code quality in the affected files by 0.77%.
Here are some functions in these files that still need a tune-up:
Legend and ExplanationThe emojis denote the absolute quality of the code:
The 👍 and 👎 indicate whether the quality has improved or gotten worse with this pull request. Please see our documentation here for details on how these metrics are calculated. We are actively working on this report - lots more documentation and extra metrics to come! Help us improve this quality report! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Branch
masterrefactored by Sourcery.If you're happy with these changes, merge this Pull Request using the Squash and merge strategy.
See our documentation here.
Run Sourcery locally
Reduce the feedback loop during development by using the Sourcery editor plugin:
Review changes via command line
To manually merge these changes, make sure you're on the
masterbranch, then run:Help us improve this pull request!