This is an (un)fair benchmark of most mature (version >= 1) pico-framework available for the Ruby programming language.
With pico i intend very tiny routing Web framework, with almost no dependencies but for Rack.
In the micro-service hyping era, exposing fast, plain HTTP routes avoiding the heavyweight of bold frameworks is a desirable feature.
Considering a plain Rails 6.0 app recorded a throughput of 7000 req/sec, you can understand why micro-frameworks are a good option.
Here are the list of the pico-frameworks included in the benchmark:
- Sinatra: is one of the first micro-frameworks for ruby, the most feature complete of the pack
- Roda: born form the ashes of Cuba a performant tree-routing framework that can be extended via plug-ins
- Rack-App: a performant pico framework dependent on Rack only
- NyNy: a tiny Web framework, dependent from ActionPack
- Grape: an opinionated framework, with several dependencies
- Camping: proud to be a mere 4KB Web framework (the core part)
- Syro: another, Cuba inspired, simple router for web applications.
I also included a plain rack application to see how much each solution diverge from the raw metal.
The "application" i tested is barely minimal: it is the HTTP version of the "Hello World" example.
I registered these benchmarks with a MacBook PRO 15 mid 2015 having these specs:
- OSX Mojave
- 2,2 GHz Intel Core i7 (4 cores)
- 16 GB 1600 MHz DDR3
- Ruby 2.6.3
All of the pico framework run over the mighty Puma application server.
I used wrk as the loading tool. I measured each application server three times, picking the best lap:
wrk -t 4 -c 100 -d30s --timeout 2000 http://0.0.0.0:9292/<app-name>
bundle exec puma -w 8 -t 1 --preload -e production config.ru
Here are the benchmarks results ordered by increasing throughput, along with the runtime dependencies footprint (measured by lapidarius gem).
| App Server | Throughput (req/s) | Runtime deps. |
|---|---|---|
| Grape | 16933.75 | 18 |
| Sinatra | 20322.81 | 4 |
| Camping | 23473.66 | 2 |
| NyNy | 30839.99 | 2 |
| Rack-App | 33372.97 | 1 |
| Syro | 43067.48 | 2 |
| Roda | 43116.18 | 1 |
| Rack | 43863.58 | 0 |
After have inspected the tested framework i dare to categorize them within three different groups:
Minimal libraries built on top of Rack APIs, offering (in some cases) identical performance, but leaving the burden (freedom?) of more complex features to the developer.
Syro and NyNy fall within this group.
More advanced routers that offers desirable features (i.e. filters, streaming) aside from a pretty routing interface.
Rack-App and Camping falls within this group.
Libraries that add to the advanced routing features, extensibility via plug-ins/contributions.
Sinatra, Grape and Roda falls within this group.
I admit that when i need raw performance over few endpoints i stick with raw Rack: it is pretty flexible and leave you writing less code than you think to get things done.
When i need more features i stick with Roda, for the following reasons:
- although it is feature-complete it only depends on Rack
- is fast, very close to plain Rack
- it integrates perfectly with Sequel, also from the Ruby-hero Jeremy Evans.