Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak in import job? #31

Open
brson opened this issue Sep 12, 2022 · 6 comments
Open

Memory leak in import job? #31

brson opened this issue Sep 12, 2022 · 6 comments

Comments

@brson
Copy link
Collaborator

brson commented Sep 12, 2022

There's probably a memory leak in the import job. Recently the entire machine came down after the import process was killed, and just now I see it is using 30% of the machine's memory. It should be using just a low constant amount of memory.

@brson
Copy link
Collaborator Author

brson commented Sep 12, 2022

I just don't see anything in the code that could leak. There's no caching, no collections, tasks are all joined. Maybe one of the chain clients has a leak.

Probably need to use a heap profiler.

Other jobs use low memory.

@brson
Copy link
Collaborator Author

brson commented Sep 12, 2022

I know people have used massif for heap profiling rust.

@brson
Copy link
Collaborator Author

brson commented Sep 28, 2022

Import job was taking 50% of ram when I logged in.

@brson
Copy link
Collaborator Author

brson commented Oct 15, 2022

Memory usage looks much better after some dependency upgrades.

@brson
Copy link
Collaborator Author

brson commented Oct 27, 2022

Memory usage for the import job was at a steady low number, but today I noticed that Arbitrum fell behind, and memory usage is at 27%.

Is there some memory being held by import jobs that run forever? I don't know what it would be.

@brson
Copy link
Collaborator Author

brson commented Oct 27, 2022

Memory usage immediately rises to 20% again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant