Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmark for JITgen #2260

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Add benchmark for JITgen #2260

wants to merge 5 commits into from

Conversation

georgwiese
Copy link
Collaborator

@georgwiese georgwiese commented Dec 18, 2024

Adds the benchmark I used for measuring the witness generation JIT in #2242.

@georgwiese
Copy link
Collaborator Author

Maybe it's late, but I don't know how to run this:

$ cargo bench --bench jit_witgen_benchmark
    Finished `bench` profile [optimized] target(s) in 0.48s
warning: the following packages contain code that will be rejected by a future version of Rust: mathru v0.6.10
note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`
     Running benches/jit_witgen_benchmark.rs (target/release/deps/jit_witgen_benchmark-a4f413cc0aba77fb)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

@pacheco
Copy link
Collaborator

pacheco commented Dec 18, 2024

i think you have to add the bench to the Cargo.toml: check riscv/Cargo.toml for executor_benchmark

@georgwiese
Copy link
Collaborator Author

That worked, thanks!

@georgwiese georgwiese marked this pull request as ready for review December 18, 2024 20:41
@georgwiese
Copy link
Collaborator Author

georgwiese commented Dec 18, 2024

Example run:
POWDR_JIT_OPT_LEVEL=1 cargo bench -p powdr-pipeline --bench jit_witgen_benchmark

On my machine, one iteration takes around 12.5s.

However, it seems like in CI, this is closer to 55s. On our server, the time for one iteration is around 62s.

Seems to be mostly compilation time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants