Figure out how to get unbanned on optimism

[brson]

all_chains has both Heco and Optimism commented out because we've been previously banned from their public RPCs. Figure out how to avoid this problem.

[mslipper]

Hey there - Matt from Optimism here. Happy to work with you to unban you from our RPC. Has an IP been banned, or are you just being rate limited?

[brson]

@mslipper thank you for the generous offer. It's been a while since we turned off optimism and I didn't save the logs, don't remember what the errors were.

I've just now turned optimism back on and so far its getting successful responses. Probably we were rate limited and didn't handle it right. At the moment it looks like we have a 250ms delay between block requests on optimism.

It'll take a few hours before any optimism numbers go live on the site.

[brson]

@mslipper we've recently updated our "courtesy delay" logic we use to pace our requests, to keep a more accurate pace and hopefully stop falling behind block production on so many chains.

On optimism, using the mainnet.optimism.io RPC endpoint, we were aiming for 250ms between block requests and got IP-blocked. Do you know what the rate limit is on that endpoint?

[brson]

This is the error we started receiving from optimism:

2022-10-15T19:08:54.167267753+00:00 ERROR realtps_import::jobs - error: Deserialization Error: invalid type: null, expected u64 at line 1 column 226. Response: {"jsonrpc":"2.0","error":{"code":-32016,"message":"Your IP has exceeded its requests per second capacity. To increase your rate limits, please sign up for a free Alchemy account at https://www.alchemy.com/optimism."},"id":null}

[brson]

We also got blocked by optimism's rpc at a 500ms pace.

[brson]

I wonder if some of the non-block requests we are making, which are a lot rarer than the block requests, but aren't as carefully paced, are getting us penalized.

[brson]

With our changes to our delay logic, I've updated the "block pace" for most chains to 500ms, and optimism to 1000ms.

At 1000ms I don't see optimism blocking us yet.

[brson]

I'm still having a tough time keeping a block pace that both keeps up with the optimism chain and doesn't get ip blocked. A "block pace" of anything less than 1s gets us blocked, so that's what I have set. (edit: now 2s) And now the "rescan delay", the time between import runs, is at (edit: 15s).

I'm surprised that I've had a hard time keeping up as optimism appears to produce blocks every ~30s or so. Maybe we've had some network problems that caused us to fall behind.

[brson]

Ok, now that I look at the actual optimism block data we are receiving, I see that we receiving many blocks of 1 tx, with identical timestamps; the optimism explorer though reports ~30 second "batches" containing many txs.

Presumably these fast 1tx blocks are an ethereum-compatibility thing, and there is an optimism specific rpc we can be calling to get slower batches.

As is, with 1s rate-limiting, whether we can actually follow the chain probably depends on the tx volume.

[brson]

Well I don't see any alternative RPC APIs for optimism, so it's not clear what a "batch" is.

I do see that the official docs say not to use their public RPC for production, and to use https://www.alchemy.com/optimism instead.

I suspect that we must use a private RPC to keep up with optimism.