Performance Comparison of Ruby Frameworks: Sinatra, Padrino, Goliath, and Ruby on Rails

by Eugene MelnikovFebruary 7, 2014
This blog post shares the performance results of four popular Ruby frameworks to find the best fit for a highly loaded Ruby application.

Evaluation results

ruby-frameworks-1The main goal of this article was to find the best framework for a very basic, but highly loaded Ruby application. This is the updated version of the comparison that was first posted in June 2013. Now, we ran all the tests again, using the latest versions of Sinatra, Padrino, Goliath, and Ruby on Rails (RoR). Unfortunately, the Espresso framework that we had tested last time disappeared from all the repositories, so it is no longer included.

We used the following test infrastructure and settings: ApacheBench 2.4.3 ab -n 1000 -c 100 localhost, a 2.3 GHz Intel Core i5 Sandy Bridge CPU, 4 GB of 1333 MHz DDR3 RAM, a 120 GB Intel 330 SSD, OS X Mavericks 10.9.1, Ruby 2.0.0p247, and MySQL 5.6.14 in a development environment.

In the comparative tables below, you will find performance results illustrating how much time it takes for the solutions to process 1,000 requests.

 

Sinatra 1.4.4

Let’s start with Sinatra. It turned out that Unicorn works faster with MySQL and Views than with Views only, but since the difference is very small (0.005 sec), this might be due to an error.

Web server
No Views and DB
Views (Slim)
Views (Slim) + MySQL (Sequel)
WEBrick 1.6.1
3.663 sec
8.594 sec
8.931 sec
Thin 1.6.1
0.626 sec
5.730 sec
7.159 sec
Unicorn 4.8.1
1.651 sec
8.240 sec
8.235 sec

To eliminate any inaccuracies, we rechecked all the controversial results many times and got pretty stable values. So, you should be extremely careful when building high-load applications, because any small changes may slow them down. We recommend to always run benchmark tests on the web server you want to use in production.

Since Thin demonstrated the best results, while keeping the application very simple, we decided to use it for the rest of the tests.

 

Padrino 0.11.4

Although the Padrino framework is built on Sinatra with a lot of helpful features, like in RoR, its performance is still close to that of a clean Sinatra application. This is very good news for us.

Cache
No Views and DB
Views (Slim)
MySQL (Sequel)
Views (Slim) + MySQL (Sequel)
No cache
0.729 sec
6.663 sec
1.388 sec
7.372 sec
Memory
0.662 sec
6.218 sec
1.170 sec
7.140 sec

 

Goliath 1.0.2

The results for Goliath look a bit strange, because we expected this framework to be the fastest, thanks to its own HTTP server.

No Views and DB
Views (Slim)
MySQL (Sequel)
Views (Slim) + MySQL (Sequel)
2.075 sec
7.477 sec
4.314 sec
10.184 sec

If you test it and get different results, please do not hesitate to mention that in the comments and/or propose your version of the benchmark test.

 

RoR 4.0.2

We also decided to test RoR 4.0.2. For these tests, we enabled caching classes in development.rb and disabled sessions and protection from forgery. We also used the Sequel database toolkit instead of ActiveRecord to make the results more trustworthy. As you can see, they turned out to be close to Goliath’s, but slightly better.

No Views and DB
Views (Slim)
MySQL (Sequel)
Views (Slim) + MySQL (Sequel)
1.539 sec
1.790 sec
2.248 sec
2.501 sec

You can take a look at the applications we created for these tests to make sure they were really similar. We hope this article helps you to choose the right Ruby framework. Feel free to leave your comments below.

 

Further reading

 


The post was written by Eugene Melnikov with assistance of Alexander Kuntsevich.