Loading..
Processing... Please wait...

Product was successfully added to your shopping cart.



Magento 2 and 1 Million Products

Can Magento 2 handle 1 million products?

 

I came across that question many times. It got me interested and I started searching the web. I didn’t find an answer so I decided to set up my own experiment.

 

In this article I will install the latest Magento 2.2.2 (M2), load it with 1 million products and look at the performance. I will describe the procedure in details and will provide specifics so anybody could replicate and prove my findings.

Preconditions

 

Here is what I will be running:

 

  1. Magento Open Source 2.2.2.

  2. Debian 8 server with Linode 12G plan: 12G RAM, 6 CPU cores, 196Gb SSD.

 

I will be using Magento 2 performance toolkit to generate data for testing. It comes with M2 and it helps with testing the impact of various 3rd-party extensions and customizations.

Load 1M products

 

M2 performance toolkit has different profiles:

 

  1. Small profile: 800 simple products, 30 categories, 200 customers, 80 orders.

  2. Medium profile: 24k simple products. 300 categories, 2000 customers, 50000 orders.

  3. Large profile: 300k simple products, 3000 categories, 5000 customers, 100k orders.

  4. Extra large profile: 600k simple products, 6000 categories, 10k customers, 150k orders.

 

Profiles are defined as XML files and are stored in setup/performance-toolkit/profiles/ce/ folder inside Magento root. Here is an example of medium profile:

 

Medium Profile Medium Profi

We would edit extra large profile XML to set the maximum number of simple products to 1 million:

-- <simple_products>600000</simple_products>

++ <simple_products>1000000</simple_products>

 

Then we would run an import process:

 

php bin/magento setup:perf:generate-fixtures \ setup/performance-toolkit/profiles/ce/extra_large.xml

 

It would take a while to complete. I had to run it two times because the first time it crashed MySQL server:

 

MySQL crash MySQL cr

The second time it run out of memory:

 

Out of memory Out of memory

I then manually reindexed it:

 

php bin/magento indexer:reindex

 

Then again it failed on Catalog Rule Product reindex:

 

Reindex failed Reindex failed

It seemed the server just could not handle the testing data that big.

 

I decided to cut down the number of catalog price rules from 20 to 2, categories from 6000 to 60, customers from 10000 to 100 and orders from 16000 to 1500. I made certain edits to extra_large profile and restarted setup:perf:generate-fixtures.

 

This time it went OK and now I have 1,400,000 products in Magento 2:

 

1 million SKUs 1 million SKUs

Speed Measurements

 

Now it is time to feel 1 million products. I have to say I put store in production mode and enabled all cache options.

 

We will be measuring server response time or time to first byte. Full page load time depends on network capacity and can be different from computer to computer.

 

Admin Panel

Backend seems fast. TTFB of Dashboard page is 1.37 sec:

 

Dashboard-TTFB Dashboard-TTF

 

Catalog > Products takes 2.12 sec:

 

Catalog-TTFB Catalog-TTFB

 

Catalog > Categories takes a bit more, 3.43 sec:

Catalog-Category-TTFB Catalog-Category-TTFB

Frontend Performance

 

Storefront is not so fast. Of course Full Page Cache helps but for the sake of an experiment I will turn it off:

 

FPC is off FPC is off

 

We are using the default Magento 2 Luma theme.

 

Homepage responds in 846 ms:

 

Homepage-TTFB Homepage-TTFB

 

Category page responds in 10,03 sec:

 

Category-TTFB Category-TTFB

 

Adding a configurable product to a cart takes 1.34 sec which is alright:

 

Add to cart TTFB Add to cart TTFB

 

Takeaway

 

Magento 2 can easily handle more than 1 million products. Frontend speed is bearable and backend seems fast enough.

 

Reindex process does take time, in my case it was around an hour for all indexers to finish.

 

PS: Check out the article where I loaded WooCommerce with 1 million products and compared its performance to Magento 2.

  

Struggle with slow Magento! We can help! Fill out the form below to get a free quote.

 

Other articles you might be interested in:

  1. Magento 2 TTFB (Time To First Byte) Optimization
  2. Magento 2 hardware requirements (Updated 2018)
  3. Speed up Magento 2 and 1 - The Ultimate Guide (Updated 2018)
  4. WooCommerce and Magento 2 - 1 million products benchmark
  5. Magento TTFB optimization to reduce time to first byte

  

If you find this post interesting do not hesitate to sign up for our newsletter and join the 877 people who receive Magento news, tips and tricks regularly.

Thank You!

 

6 thoughts on “Magento 2 and 1 Million Products”
  • Ravindra
    Ravindra October 1, 2018 at 9:48 am

    You are post a very nice blog. I got my information through your blog. please keep it up.I like it.

  • Konstantin Gerasimov
    Konstantin Gerasimov August 14, 2018 at 4:45 pm

    @R Raut - Hello! I believe there is no actual limit but the more products you have the longer the reindex process is going to take.

  • R Raut
    R Raut August 12, 2018 at 9:39 pm

    Hello,
    Great article. By the way whats the limit of Magento 1 to handle products? I am not happy with interface of Magento 2.

    Best regards..

  • Ahmed
    Ahmed February 11, 2018 at 9:05 pm

    Hi Konstantin :)

    Your experiment was pretty dope.
    Gonna wait for the next one magento2 vs woocommerce :)

    Thanks

  • Konstantin Gerasimov
    Konstantin Gerasimov February 9, 2018 at 5:44 pm

    @san thank you, I will try it.

  • san brown
    san brown February 9, 2018 at 3:07 pm

    To speed up the front end you should try ElasticSuite https://github.com/Smile-SA/elasticsuite Search frontend. I've got 2.1M products running on a beefier VM (128gb ram,24vcpu/512gb SSD) with php 7.2 and it is not so slow with some tuning of mysql/elasticsearch/php/varnish/redis!