Can Magento 2 handle 1 million products?
I came across that question many times. It got me interested and I started searching the web. I didn’t find an answer so I decided to set up my own experiment.
In this article I will install the latest Magento 2.2.2 (M2), load it with 1 million products and look at the performance. I will describe the procedure in details and will provide specifics so anybody could replicate and prove my findings.
Preconditions
Here is what I will be running:
Magento Open Source 2.2.2.
Debian 8 server with Linode 12G plan: 12G RAM, 6 CPU cores, 196Gb SSD.
I will be using Magento 2 performance toolkit to generate data for testing. It comes with M2 and it helps with testing the impact of various 3rd-party extensions and customizations.
Load 1M products
M2 performance toolkit has different profiles:
Small profile: 800 simple products, 30 categories, 200 customers, 80 orders.
Medium profile: 24k simple products. 300 categories, 2000 customers, 50000 orders.
Large profile: 300k simple products, 3000 categories, 5000 customers, 100k orders.
Extra large profile: 600k simple products, 6000 categories, 10k customers, 150k orders.
Profiles are defined as XML files and are stored in setup/performance-toolkit/profiles/ce/ folder inside Magento root. Here is an example of medium profile:
We would edit extra large profile XML to set the maximum number of simple products to 1 million:
-- <simple_products>600000</simple_products>
++ <simple_products>1000000</simple_products>
Then we would run an import process:
php bin/magento setup:perf:generate-fixtures setup/performance-toolkit/profiles/ce/extra_large.xml
It would take a while to complete. I had to run it two times because the first time it crashed MySQL server:
The second time it run out of memory:
I then manually reindexed it:
php bin/magento indexer:reindex
Then again it failed on Catalog Rule Product reindex:
It seemed the server just could not handle the testing data that big.
I decided to cut down the number of catalog price rules from 20 to 2, categories from 6000 to 60, customers from 10000 to 100 and orders from 16000 to 1500. I made certain edits to extra_large profile and restarted setup:perf:generate-fixtures.
This time it went OK and now I have 1,400,000 products in Magento 2:
Speed Measurements
Now it is time to feel 1 million products. I have to say I put store in production mode and enabled all cache options.
We will be measuring server response time or time to first byte. Full page load time depends on network capacity and can be different from computer to computer.
Admin Panel
Backend seems fast. TTFB of Dashboard page is 1.37 sec:
Catalog > Products takes 2.12 sec:
Catalog > Categories takes a bit more, 3.43 sec:
Frontend Performance
Storefront is not so fast. Of course Full Page Cache helps but for the sake of an experiment I will turn it off:
We are using the default Magento 2 Luma theme.
Homepage responds in 846 ms:
Category page responds in 10,03 sec:
Adding a configurable product to a cart takes 1.34 sec which is alright:
Takeaway
Magento 2 can easily handle more than 1 million products. Frontend speed is bearable and backend seems fast enough.
Reindex process does take time, in my case it was around an hour for all indexers to finish.
PS: Check out the article where I loaded WooCommerce with 1 million products and compared its performance to Magento 2.
If you find this post interesting do not hesitate to sign up for our newsletter and join the 1433 people who receive Magento news, tips and tricks regularly.
Thank You!
I got a requirement where my client having 140 million products. I'm bit confused can I do that project with Magento Open source v2.4.x? Will it handle?
Very interesting post. We are trying to set up a store with 12M sku and I can share the performance results with you guys.
At this moment we are trying to find a way to display the products in list with the filters to the users find the products by the attributes.
@Guy I think it can, how would you display them on the frontend though? With Read more button?
The testing you did was interesting but can Magento 2 handle over 300 different items added to one cart? This is a regular occurring problem for us. Can the Performance Toolkit test for this scenario?
Glad to see someone doing inside outside of Magento. there are some extreme examples from my client list those running with over 8 million products on site and site response time or first byte is less than 879ms.
The only thing that effects the performance is how you actually control the data and how well you had defined.
You are post a very nice blog. I got my information through your blog. please keep it up.I like it.
@R Raut - Hello! I believe there is no actual limit but the more products you have the longer the reindex process is going to take.
Hello,
Great article. By the way whats the limit of Magento 1 to handle products? I am not happy with interface of Magento 2.
Best regards..
Hi Konstantin :)
Your experiment was pretty dope.
Gonna wait for the next one magento2 vs woocommerce :)
Thanks
@san thank you, I will try it.
To speed up the front end you should try ElasticSuite https://github.com/Smile-SA/elasticsuite Search frontend. I've got 2.1M products running on a beefier VM (128gb ram,24vcpu/512gb SSD) with php 7.2 and it is not so slow with some tuning of mysql/elasticsearch/php/varnish/redis!