Loading..
Processing... Please wait...

Product was successfully added to your shopping cart.



Magento 1.9.3.2 vs 2.1.4 Performance Benchmark

Today I will see what is faster: Magento CE 1.9 or Magento CE 2.1. I was inspired by this StackExchange question and by the Amasty team benchmark. Amasty compared Magento 1.9.2.2 vs Magento 2.0.0. They found out that 1.9 is faster than 2.0. Now it is 2017 and Magento 2.1.4 and 1.9.3.2 just came out so I decided to see if 1.9 still beats 2.1.

Disclaimer: I will be comparing Community Editions (CE). I am aware that Magento released the official Magento 2.0 Site Performance and Scalability Optimizations white paper where they show Magento Enterprise Edition (EE) 1.14.2 is slower than Magento EE 2.0. We will be talking about CE here.

I don't want to wait show me results now!

 

Setup

My setup is simple and could be replicated by others. I will use Amazon AWS EC2 instance that I setup in this guide. Here is the details:

 

 

Magento 2.1.4

The settings that I will use for Magento 2.1:

  • Production Mode
  • Theme - Luma
  • Merge Javascript files - Yes
  • Enable Javascript bundling - Yes
  • Minify Javascript files - Yes
  • Merge CSS files - Yes
  • Minify CSS files - Yes

 

 

Magento 1.9.3.2

The settings for Magento 1.9.3.2:

  • Default install with Sample data 1.9.2.4
  • Theme: rwd/default
  • All cache enabled
  • Flat Catalog enabled for products and categories

 

Test Scenario

I will be using JMeter as load testing software. Scenario as follows: 50 concurrent users with ramp-up period of 10 seconds repeating 20 times. That comes down to 1000 requests. I will be using Mac Numbers 2D-Line chart so it will be linear extrapolation of CSV load test data.

I will be testing homepage, cart page with no items and customer login page. I won't be testing product/category pages as they might be subject to theme or data difference.

 

NO Full Page Cache

First we will test with Magento 2.1 Full Page Cache (FPC) off. I want to make 1.9 and 2.1 even because Magento CE 1.9.x doesn't have FPC by default.

 

1. Homepage

 

Magento 1.9.3.2 vs 2.1.4 NO FPC Homepage Magento 1.9.3.2 vs 2.1.4 NO FPC Homepage

Magento 1.9 is obviously faster. Here is some numbers:

Magento 1.9.3.2 vs 2.1.4 NO FPC - Homepage Magento 1.9.3.2 vs 2.1.4 NO FPC - Homepage

 

 

2. Cart Page

Magento 1.9.3.2 vs 2.1.4 NO FPC Cart Page No Items Magento 1.9.3.2 vs 2.1.4 NO FPC Cart Page No Items

 

Magento 1.9.3.2 vs 2.1.4 NO FPC Cart Page No Items - Response Times Magento 1.9.3.2 vs 2.1.4 NO FPC Cart Page No Items - Response Times

 

 

Yes Full Page Cache

No we will turn on Magento 2.1.4 FPC and will install the free Zoom FPC for Magento 1.9.3.2.

Zoom FPC is an abandoned project but with a little magic you can make it work with 1.9.

For 2.1.4 I will be using built-in FPC:

Magento 2.1.4 built-in FPC Magento 2.1.4 built-in FPC

I will be using built-in FPC as I have shown here there is hardly a difference between Varnish/built-in/Redis FPC.

 

1. Homepage

Magento 1.9.3.2 vs 2.1.4 FPC Homepage Magento 1.9.3.2 vs 2.1.4 FPC Homepage

 

Magento 1.9.3.2 vs 2.1.4 - FPC - Homepage Magento 1.9.3.2 vs 2.1.4 - FPC - Homepage

We see that Magento 1.9 with FPC beats Magento 2.1.4 FPC easily. Does it true for cart page? Let's see.

 

2. Cart Page

Magento 1.9.3.2 vs 2.1.4 FPC Cart Page No Items Magento 1.9.3.2 vs 2.1.4 FPC Cart Page No Items
Magento 1.9.3.2 vs 2.1.4 - FPC - Cart Page No Items Magento 1.9.3.2 vs 2.1.4 - FPC - Cart Page No Items

 

3. Customer login page

Magento 1.9.3.2 vs 2.1.4 Customer Login Page FPC Magento 1.9.3.2 vs 2.1.4 Customer Login Page FPC

 

Magento 1.9.3.2 vs 2.1.4 Customer Login Page FPC Magento 1.9.3.2 vs 2.1.4 Customer Login Page FPC

Takeaway

Magento 1.9.3.2 beats Magento 2.1.4 with either FPC on or off. For Magento 1.9 FPC I was using this excellent free extension Zoom FPC.

Magento 1.9.3.2 vs Magento 2.1.4 Benchmark Magento 1.9.3.2 vs Magento 2.1.4 Benchmark

 

Do you have a different benchmark results? Share it comments!

Do you know that Magento 2 TTFB can easily be hurt by poorly coded 3rd party extension - Wyomind is an example of it.

 

 

 

If you find this post interesting do not hesitate to sign up for our newsletter and join the 721 people who receive Magento news, tips and tricks regularly.

Thank You!

 

 

8 thoughts on “Magento 1.9.3.2 vs 2.1.4 Performance Benchmark”

  • Konstantin Gerasimov

    zoom fpc is faster than lesti

  • Dennis

    Konstantin,

    Is there a reason you've used Zoom FPC instead of Lesli FPC which is much more popular and free as well?

  • Konstantin Gerasimov

    @developer - I explained how I get these results. You are welcome to repeat the steps I took and report the results you get.

  • developer

    How you get this resuls? I've tested magento 2.1.6 without varnish with sample data products and first bit on home page I got in 90 ms. I.m testing on my home PC with HDD and and fx8230 and you are testing on server

  • Konstantin Gerasimov

    @Matt magento 2 doesn't require fpc when developing - what a non sense!

    I am testing TTFB - and that is the only thing that matters in my benchmark.

    "Both solutions tested do not indicate whether the FPC is actually working" - yes they do - just compare TTFB to the ones without FPC.

  • Matt Johnson

    Im seeing a few more test methodology / setup issues:

    Magento 1 CE doesnt require an FPC, It's design to work without one (Though can be improved with one)

    Magento 2 requires Varnish Cache in production (And FPC when developing), Magento 2 isnt designed to work without a caching layer.

    Your tests are only requesting the HTML document (not any static assets) so you are not seeing an actual users page load and the settings around minification, combining files are not doing anything as these files are not being requested.

    Both solutions tested do not indicate whether the FPC is actually working, Most FPC implementations are failures as there are no metrics on what's missing cache.

  • Konstantin Gerasimov

    Steve,

    with a little hacking you can make Zoom work with 1.9 no problem
    you can patch 1.9 to work with php7, it is still 1.9

    where does c4 come from? I am using c3 as it is stated in the article. It is also stated that i run m1 and m2 on the same aws box. I keep m1 and m2 in separate folders and with a simple nginx edit i change between them.

    i have done NO configuration to neither M1 nor M2. out of the box installs as I have stated.

  • Steve Holdoway

    Problems with this...

    - Zoom is no longer supported, and only purports to support Magento up to v. 1.6
    - PHP 7 is not supported for Magento 1.9
    - c4 instances are faster than c3, and tend to use a different infrastructure
    - where are you running the tests from? If locally then there's added stress from the testing and no exervising of the network
    - configuration. What have you done to speed up database, php, system?

    For me the last is the most important. It's clear that the bottlenecks that you're seeing are outside the variables of the test.