Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API: sync past blocks incrementally #331

Open
sohkai opened this issue Jun 27, 2019 · 0 comments
Open

API: sync past blocks incrementally #331

sohkai opened this issue Jun 27, 2019 · 0 comments
Labels
component: api enhancement New feature or request

Comments

@sohkai
Copy link
Contributor

sohkai commented Jun 27, 2019

We currently fetch all past blocks in one shot, leaving us no way to tell a user how long they should wait and how far they have synced.

Given how the events caching works for externals, this is also problematic because we wait for all contracts' past events to load before committing to the app's cache.

From #206:

Something we could also do is detect if we're syncing a large amount of blocks (e.g. >100k) and then successively call eth_getlogs with to and from block numbers so that we don't have to wait for the entire operation to finish at once before we process any results.

As of today, loading the governance.aragonproject.eth org's voting app requires >5min to sync ~1million blocks, with the eth_getlogs sending 20kb of data all at once at the end. We could instead split this operation into 100k-1M block chunks to start loading data faster for the user

@sohkai sohkai added the enhancement New feature or request label Jun 27, 2019
@stale stale bot added the abandoned label Dec 24, 2019
@aragon aragon deleted a comment from stale bot Dec 27, 2019
@stale stale bot removed the abandoned label Dec 27, 2019
@sohkai sohkai changed the title API: Sync past blocks incrementally API: sync past blocks incrementally Jun 9, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: api enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant