Ci mock run

[jurraca]

Adding a mock, end-to-end run of the map process.

I would like to use actual downloaded RPKI data instead of a already validated data which would require a new flag just for this. However, having issues using actual RPKI data. I added two repositories:

• a subset of the rpki.arin.net repo, with intermediate certs, CRL and MFT files.
• a full set from rpki.sunoaki.net.

All these end up as invalid with the error cannot find a local certificate issuer. I think I have some gaps in understanding process here.

• the rpki-client validation enforces the offline flag, so it should look for certs locally.
• The certs are in the repository path, and afaict rpki-client looks for the certs there.
• the certificates are valid, with their valid_until date set far into the future.
• I'm not sure if you need the full recursive path -- with the (nested) ARIN repo, it looks like yes, which is why i added the whole path and chain of certs, but for sunoaki this should not be an issue. All end up as invalid.

Curious if you have an idea @fjahr if not I'll ask the rpki-client team.
Thanks.

[fjahr]

I not a big fan of adding all these files to the repository, it's pretty noisy. Can't we add a dummy reproduction data repository that the CI clones and uses instead? I suggested this previously but not sure if you overlooked it.

I think I didn't see Routeviews data in here yet, I would like to have that too so that code is covered as well. I think the easiest way to do this is to get an actual reproduction run, trim the data of each of the data sources by deleting parts of it (a lot). Then let the CI run a normal reproduction run with the same timestamp and evaluate the hash result to be what we expect.

I think the data in a separate repo can be valuable in other contexts too, like doing a workshop where you can't be sure the participants have a lot of time and online connectivity. We could also add more reproduction runs to the repo that hit different edge cases without poluting the repo here with thousands of files.

For the error you describe: You think you removed the /ta folder which includes the actual trust anchors. Without these rpki-client can't validate the ROAs and since the repro run is offline it doesn't try to use the TALs to download them again.

[fjahr]

As mentioned in the other message, I would prefer if we would run a reproduction run with the actual static timestamp and then evaluate the hash to be what we expect. With the dynamic date the hash wouldn't be deterministic and I think we might be missing on some potential coverage of our deteriminism.