Loading..
Processing... Please wait...

Product was successfully added to your shopping cart.



Magento 2 and 1 Million Products

Can Magento 2 handle 1 million products?

 

I came across that question many times. It got me interested and I started searching the web. I didn’t find an answer so I decided to set up my own experiment.

 

In this article I will install the latest Magento 2.2.2 (M2), load it with 1 million products and look at the performance. I will describe the procedure in details and will provide specifics so anybody could replicate and prove my findings.

Preconditions

 

Here is what I will be running:

 

  1. Magento Open Source 2.2.2.

  2. Debian 8 server with Linode 12G plan: 12G RAM, 6 CPU cores, 196Gb SSD.

 

I will be using Magento 2 performance toolkit to generate data for testing. It comes with M2 and it helps with testing the impact of various 3rd-party extensions and customizations.

Load 1M products

 

M2 performance toolkit has different profiles:

 

  1. Small profile: 800 simple products, 30 categories, 200 customers, 80 orders.

  2. Medium profile: 24k simple products. 300 categories, 2000 customers, 50000 orders.

  3. Large profile: 300k simple products, 3000 categories, 5000 customers, 100k orders.

  4. Extra large profile: 600k simple products, 6000 categories, 10k customers, 150k orders.

 

Profiles are defined as XML files and are stored in setup/performance-toolkit/profiles/ce/ folder inside Magento root. Here is an example of medium profile:

 

Medium Profile Medium Profi

We would edit extra large profile XML to set the maximum number of simple products to 1 million:

-- <simple_products>600000</simple_products>

++ <simple_products>1000000</simple_products>

 

Then we would run an import process:

 

php bin/magento setup:perf:generate-fixtures  setup/performance-toolkit/profiles/ce/extra_large.xml

 

It would take a while to complete. I had to run it two times because the first time it crashed MySQL server:

 

MySQL crash MySQL cr

The second time it run out of memory:

 

Out of memory Out of memory

I then manually reindexed it:

 

php bin/magento indexer:reindex

 

Then again it failed on Catalog Rule Product reindex:

 

Reindex failed Reindex failed

It seemed the server just could not handle the testing data that big.

 

I decided to cut down the number of catalog price rules from 20 to 2, categories from 6000 to 60, customers from 10000 to 100 and orders from 16000 to 1500. I made certain edits to extra_large profile and restarted setup:perf:generate-fixtures.

 

This time it went OK and now I have 1,400,000 products in Magento 2:

 

1 million SKUs 1 million SKUs

Speed Measurements

 

Now it is time to feel 1 million products. I have to say I put store in production mode and enabled all cache options.

 

We will be measuring server response time or time to first byte. Full page load time depends on network capacity and can be different from computer to computer.

 

Admin Panel

Backend seems fast. TTFB of Dashboard page is 1.37 sec:

 

Dashboard-TTFB Dashboard-TTF

 

Catalog > Products takes 2.12 sec:

 

Catalog-TTFB Catalog-TTFB

 

Catalog > Categories takes a bit more, 3.43 sec:

Catalog-Category-TTFB Catalog-Category-TTFB

Frontend Performance

 

Storefront is not so fast. Of course Full Page Cache helps but for the sake of an experiment I will turn it off:

 

FPC is off FPC is off

 

We are using the default Magento 2 Luma theme.

 

Homepage responds in 846 ms:

 

Homepage-TTFB Homepage-TTFB

 

Category page responds in 10,03 sec:

 

Category-TTFB Category-TTFB

 

Adding a configurable product to a cart takes 1.34 sec which is alright:

 

Add to cart TTFB Add to cart TTFB

 

Takeaway

 

Magento 2 can easily handle more than 1 million products. Frontend speed is bearable and backend seems fast enough.

 

Reindex process does take time, in my case it was around an hour for all indexers to finish.

 

PS: Check out the article where I loaded WooCommerce with 1 million products and compared its performance to Magento 2.

  

If you find this post interesting do not hesitate to sign up for our newsletter and join the 1312 people who receive Magento news, tips and tricks regularly.

Thank You!

 

10 thoughts on “Magento 2 and 1 Million Products”
  • Uelder Martins da Silva June 3, 2019 at 4:19 pm

    Very interesting post. We are trying to set up a store with 12M sku and I can share the performance results with you guys.
    At this moment we are trying to find a way to display the products in list with the filters to the users find the products by the attributes.

  • Antonella Morales December 10, 2018 at 11:11 am

    @Guy I think it can, how would you display them on the frontend though? With Read more button?

  • Guy Pineda December 6, 2018 at 10:46 pm

    The testing you did was interesting but can Magento 2 handle over 300 different items added to one cart? This is a regular occurring problem for us. Can the Performance Toolkit test for this scenario?

  • Farhan Islam December 4, 2018 at 7:44 am

    Glad to see someone doing inside outside of Magento. there are some extreme examples from my client list those running with over 8 million products on site and site response time or first byte is less than 879ms.

    The only thing that effects the performance is how you actually control the data and how well you had defined.

  • Ravindra October 1, 2018 at 9:48 am

    You are post a very nice blog. I got my information through your blog. please keep it up.I like it.

  • Konstantin Gerasimov August 14, 2018 at 4:45 pm

    @R Raut - Hello! I believe there is no actual limit but the more products you have the longer the reindex process is going to take.

  • R Raut August 12, 2018 at 9:39 pm

    Hello,
    Great article. By the way whats the limit of Magento 1 to handle products? I am not happy with interface of Magento 2.

    Best regards..

  • Ahmed February 11, 2018 at 9:05 pm

    Hi Konstantin :)

    Your experiment was pretty dope.
    Gonna wait for the next one magento2 vs woocommerce :)

    Thanks

  • Konstantin Gerasimov February 9, 2018 at 5:44 pm

    @san thank you, I will try it.

  • san brown February 9, 2018 at 3:07 pm

    To speed up the front end you should try ElasticSuite https://github.com/Smile-SA/elasticsuite Search frontend. I've got 2.1M products running on a beefier VM (128gb ram,24vcpu/512gb SSD) with php 7.2 and it is not so slow with some tuning of mysql/elasticsearch/php/varnish/redis!