Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Testing suite performance improvement #177

Open
CanBerker opened this issue Jan 21, 2019 · 1 comment
Open

Testing suite performance improvement #177

CanBerker opened this issue Jan 21, 2019 · 1 comment
Labels
On hold

Comments

@CanBerker
Copy link
Collaborator

With ForLetClauses1 and ForLetClauses2, our testing system is struggling performance wise if parallelize uses big ranges (eg. 1 to 10000). In the console application, results are fetched from spark and truncated to print only the first 100, which is fine. The testing system however, fully reads the results with a series of next() calls and without any truncation. I think this needs to be made more efficient. if these tests are to be put in the system (they take 1-2 minutes each).

Originally posted by @CanBerker in #174 (comment)
@CanBerker
Copy link
Collaborator Author

Testing suite now uses spark as well with the following PR. Performance however, is not measured/compared.
#201

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
On hold
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ghislainfourny @CanBerker