Blog

Performance Comparison of Ruby Frameworks: Sinatra, Padrino, Goliath, and Ruby on Rails

Eugene Melnikov

ruby-frameworks-1

The main goal of this article was to find the best framework for a very basic but highly loaded Ruby application. This is the updated version of the comparison that was first posted in Jun 2013. Now we ran all the tests again, using the latest versions of Sinatra, Padrino, Goliath, and RoR. Unfortunately, the Espresso framework that we had tested last time disappeared from all the repositories, so it is no longer included.

We used the following test infrastructure and settings: ApacheBench 2.4.3 ab -n 1000 -c 100 localhost, a 2.3 GHz Intel Core i5 Sandy Bridge CPU, 4 GB of 1333 MHz DDR3 RAM, a 120 GB Intel 330 SSD, OS X Mavericks 10.9.1, Ruby 2.0.0p247, and MySQL 5.6.14 in a development environment.

All the tables below contain data on the time it takes to process 1,000 requests.

 

Sinatra 1.4.4

Web Server
No Views and DB
Views (Slim)
Views (Slim) + MySQL (Sequel)
WEBrick 1.6.1
3.663 s
8.594 s
8.931 s
Thin 1.6.1
0.626 s
5.730 s
7.159 s
Unicorn 4.8.1
1.651 s
8.240 s
8.235 s

 
Let’s start with Sinatra. It turned out that Unicorn works faster with MySQL and Views than with Views only, but since the difference is very small (0.005 s), this might be due to an error.

To eliminate any inaccuracies, we re-checked all the controversial results many times and got pretty stable values. So, you should be extremely careful when building high-load applications, because any small changes may slow them down. We recommend that you always run benchmark tests to the Web server you want to use in production.

Since Thin demonstrated the best results while keeping the application very simple, we decided to use it for the rest of the tests.
 

Padrino 0.11.4

Cache
No Views and DB
Views (Slim)
MySQL (Sequel)
Views (Slim) + MySQL (Sequel)
No cache
0.729 s
6.663 s
1.388 s
7.372 s
Memory
0.662 s
6.218 s
1.170 s
7.140 s

 
Although the Padrino framework is built on Sinatra with a lot of helpful features, like in Rails, its performance is still close to that of a clean Sinatra application. This is very good news for us.
 

Goliath 1.0.2

No Views and DB
Views (Slim)
MySQL (Sequel)
Views (Slim) + MySQL (Sequel)
2.075 s
7.477 s
4.314 s
10.184 s

 
The results for Goliath look a bit strange, because we expected this framework to be the fastest, thanks to its own HTTP server. If you test it and get different results, please do not hesitate to mention that in the comments and/or propose your version of the benchmark test.
 

Rails 4.0.2

No Views and DB
Views (Slim)
MySQL (Sequel)
Views (Slim) + MySQL (Sequel)
1.539 s
1.790 s
2.248 s
2.501 s

 
We also decided to test Rails 4.0.2. For these tests, we enabled caching classes in development.rb and disabled sessions and protection from forgery. We also used the Sequel database toolkit instead of ActiveRecord to make the results more trustworthy. As you can see, they turned out to be close to Goliath’s, but slightly better.

You can take a look at the applications I created for these tests to make sure they were really similar. Hope this article helps you to choose the right Ruby framework. Feel free to leave your comments below.

p.s. Special thanks to Alexander Kuntsevich, our Ruby developer, for his assistance.


Download a more detailed version of the comparison (with 10 diagrams) here:

Performance Comparison of Ruby Frameworks: Goliath, Ruby on Rails, Sinatra, Padrino, and Espresso


About the author

Eugene Melnikov is a senior developer at Altoros with 5+ years of experience in Ruby, Rails, AngularJS, and Node.js. His professional interests also include сloud computing and automated testing. You can find him on GitHub.

18 Comments
  • Ed

    Did you make any attempt to test these frameworks in production mode, the performance implications in rails at least are huge…

    • Eugene Melnikov

      > Ed> Did you make any attempt to test these frameworks in production mode, the performance implications in rails at least are huge…

      Ed, most likely, the results at large scale will be similar, more or less. Though, obviously, they may differ in the production mode (depending on configuration, load, etc.), ratios will probably stay the same. That’s what matters most when comparing performance of several products, isn’t it? 😉 We’re also working on updating this benchmark with more accurate figures and diagrams, so stay tuned.

      • beyondludus

        I believe performance of Goliath is also dramatically improved if the “production” environment is used.

        • Andres Colon

          Would like to see this comparison.

    • brauliobo

      Agree, it makes no sense to benchmark on development.

  • Owen

    So Sinatra with Slim and MySQL took 7.159 s and Rails took 2.501 s? Is that correct? Unless I’ve misread something I am extremely surprised that Rails beat Sinatra.

    • Eugene Melnikov

      Hi Owen. There is nothing strange about these results. A lot depends on how you run the applications and how you configure the Web server for RoR. If you would like to repeat the tests, you can find the applications we used here: https://github.com/melnikaite/ruby-frameworks-comparison/branches

  • Hey you can test latest padrino 0.12.2 and Rails 4.1.4

    • Eugene Melnikov

      Hi Zephiro,

      Surely there would be some changes, but these versions are just a little bit different from those described in the post. I think that we should better wait for some major updates and then repeat the tests again.

      • Darío Javier Cravero

        Hi Eugene,

        Thanks for your benchmark :).

        Padrino core dev here. The gap from 0.11.4 to 0.12.3 (released yesterday) is quite big. We’ll get our versioning right as we reach 1.0.

        Regards,
        Darío

        • Eugene Melnikov

          Hi Dario,

          Thanks for the information. We’ll be following new Padrino releases closely.

          • Jerome Lanteri

            Hi Eugene,
            i read your comment and trust you… so 2 years after, now rails-5 also, were are the new benchmark tests ? you said something, where are the act of promises ?
            show us please.

          • Captain Bats

            Jerome, you are free to run your own benchmarks if you need them, no need to wait for someone else to do it and document it for you.

          • Jerome Lanteri

            you totaly miss-understand the message.
            I try again for you specially (hope you will appreciate):
            The article talk about tests when rails will be update. Tehn, i commented for show that the credibility of the writer of this article is (in facts) engaged and now, it is also passed and contents/promises has to be relative to the facts of acts. It is an objective comment.
            But… yes, thank you “Captain Bats”, do you really think that i don’t know i can do my own tests ? or do you think now that in fact, you not understood my comment ?
            hope this would help you to understand better (it looks like you not read the end: “you said something, where are the act of promise ?”).
            I think Captain Bats that if you read all the comment you should be able to easily understand and have an answer who should be effective. good luck…

  • Dezso Papp

    Great to see Padrino (my fav) on top. Yet, you have misstyped its name in the first paragraph (Podrino) 😉

    • Olga Kurylionak

      Corrected. Thanks for noticing!

  • Eugene Melnikov

    New comparison is here https://blog.altoros.com/performance-comparison-of-ruby-frameworks-app-servers-template-engines-and-orms-q4-2016.html

    • Phil

      Excellent report

Benchmarks and Research

Subscribe to new posts

Get new posts right in your inbox!