Loading..
Processing... Please wait...

Product was successfully added to your shopping cart.



Magento 2 Migration - The Ultimate Guide (Updated 2018)

Magento 1 to Magento 2 Migration Service

The day has come.

The Magento team (now part of the Adobe family) has officially announced that support for version 1 will end in June of 2020.

What does this mean to a small business owner?

If you happen to run a Magento 1 website after June ‘20, you’re on your own with anything your website may require.

Official support is ending, so don't expect any more official patches or security updates.

The time to upgrade to Magento 2 has come.

I’m going to help you make the transition.

This guide will walk you through migrating a stock Magento 1 installation (with sample data) to a fully functional Magento 2 store.

I’ll outline possible issues that may arise during or after migration and suggest fixes.

How will this be helpful?

Except for highly customized shops, a typical Magento 1 store is not much different from a stock installation.

By following this tutorial, you’ll be able to perform a Magento 2 migration service yourself, saving your business thousands of dollars in developer fees.

P.S.: Make sure to backup your database and files before you attempt to move to M2. There may be issues, and it is important to have a Plan B just in case.

Also, consider test-driving the migration process on a staging environment first. Make a copy of your website somewhere (on a different server preferably) and practice your update steps there.

The 5-Step Magento 1 to Magento 2 Migration Process

  1. Stock M2 installation

  2. Data migration

  3. Theme migration

  4. Migrating extensions

  5. Test, test, test

Step 0: Software and Hardware Requirements

Magento 2 will run poorly on outdated software, and it won’t work on PHP 5.

Your server should have the following programs installed:

  • PHP: 7.0.13+ or 7.1.x

  • MySQL: 5.6, 5.7 or MariaDB 10.0,10.1,10.2 or Percona 5.7

  • The following PHP extensions: bc-math, ctype, curl, dom, gd, intl, mbstring, mcrypt, hash, openssl, PDO/MySQL, SimpleXML, soap, spl, libxml, xsl, zip, json, iconv

There are also certain hardware recommendations:

  • At least 2G of RAM minimum

  • 24GB+ SSD (Solid State Drive)

M2 won’t run well with less than 2G of RAM, and SSD technology will cause it to run much faster.

For more on the topic, read my previous post.

Step 1: Stock Magento 2 Installation

Transitioning to M2 starts by installing a fresh copy of the latest Magento 2 software.

Let’s head over to the official website and grab the latest version of M2 Open Source.

As of time of this writing, that’s 2.2.5.

Download The Latest Magento 2 Download The Latest Magento 2

Once downloaded, install it on a basic VPS (virtual private server) with 2G of RAM and 24G of SSD space.

I am using Nginx and PHP-FPM 7.1.

It’s easier than ever to pair Nginx with Magento 2 since the latter comes with a premade configuration file, nginx.conf.sample.

When everything is ready, let’s open up a browser and start the installation process.

Installing Magento 2: Step 1 Installing Magento 2: Step 1

The readiness check failed since I had quite a few PHP extensions missing.

Missing Extensions While Installing Magento 2 Missing Extensions While Installing Magento 2

I went ahead and installed the missing software with the following command (I was using Debian 9):

apt-get install php7.1-curl php7.1-mcrypt php7.1-bcmath

Finally, step 6:

Installing Magento 2: Step6 Installing Magento 2: Step6

Press the “Install Now” button to begin.

Install Log Install Log

Before long, the “Success” message will appear. For this example, it took me about half a minute.

Success! Success!

Now that we have a fresh Magento 2 store, it’s time to migrate the data from the old M1 site.

In our case, the old Magento 1 site is not actually that old.

It’s just a fresh Magento 1.9.3.9 install loaded with sample data.

Magento 1 with Sample Data Magento 1 with Sample Data

You can download it from the official website.

Magento 2 Data Migration

The Magento team has simplified the migration process with an automatic Magento Migration Tool (MMT).

This program will transfer the data from your old M1 installation to the new M2 store.

Notice: You will need SSH (Secure Shell) access to perform a migration. Obtain it from your system administrator or your hosting support team.

Install the MMT from repo.magento.com using a PHP package manager software called composer.

If you haven’t installed composer, go ahead and do so now. Visit the Getting Started page on the official website for more information.

The Magento Migration Tool installation will place the files into a Magento 2 folder, which I’d setup in the previous step.

The following commands should do the trick, so run them inside the folder:

composer config repositories.magento composer https://repo.magento.com
composer require magento/data-migration-tool:2.2.5

Take note of the 2.2.5 at the end of the second command. That is our M2 platform version.

If you’re migrating to a different version, change the command accordingly.

Upon running the composer require, the system will ask you for your authentication keys.

Get these by heading over to My Profile on Magento Marketplace website.

My Profile Magento Marketplace My Profile Magento Marketplace

Click on Access Keys under My Products menu.

Access Keys Magento Marketplace Access Keys Magento Marketplace

Create a new Access key if necessary.

Then use the Public Key as a username and the Private Key as a password.

Once you enter that information, you should be able to install the data migration tool successfully.

root# composer require magento/data-migration-tool:2.2.5
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 1 install, 0 updates, 0 removals
  - Installing magento/data-migration-tool (2.2.5): Downloading (100%)         
Writing lock file
Generating autoload files

Next, we’ll need to configure the tool.

Go to the Magento 2 root folder and find the following directory:

vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.3.9

Copy config.xml.dist file to config.xml.

Substitute 1.9.3.9 with your site’s Magento 1 version and change opensource-to-opensource to opensource-to-commerce if you are migrating from the Community Edition to Enterprise (now known as Commerce).

Now open config.xml and find the code block below:

<source>
  <database host="localhost" name="magento1" user="root" />
</source>
<destination>
  <database host="localhost" name="magento2" user="root" />
</destination>

Then enter Magento 1 and Magento 2 database credentials.

Include password attribute like this:

<source>
  <database host="localhost" name="magento1" user="magento1" password="magento1"/>
</source>
<destination>
  <database host="localhost" name="magento2" user="magento2" password="magento2"/>
</destination>

Inside the <options> tag is another mandatory field to specify - <crypt_key>:

<options>
……
<crypt_key>689e13e5b98a2d82da67a7042b582f9e</crypt_key>
…
</options>

You can find your M1 crypt_key within the app/etc/local.xml file inside your old site’s root folder.

<config>
    <global>
        <install>
           <date><![CDATA[Wed, 12 Sep 2018 21:29:35 +0000]]></date>
        </install>
        <crypt>
            <key><[CDATA[<strong>689e13e5b98a2d82da67a7042b582f9e</strong>]]></key>
        </crypt>
        <disable_local_modules>false</disable_local_modules>
        <resources>

Settings Migration

Now, we start the actual data migration.

We first migrate settings, so head over to the M2 root folder and run:

php bin/magento migrate:settings vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.3.9/config.xml

Make sure to specify the correct path to your config.xml.

In our case, there were no errors, so the settings migration went fine:

# php bin/magento migrate:settings vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.3.9/config.xml

[2018-09-14 18:22:22][INFO][mode: settings][stage: integrity check][step: Settings Step]: started
100% [============================] Remaining Time: < 1 sec
[2018-09-14 18:22:22][INFO][mode: settings][stage: integrity check][step: Stores Step]: started
100% [============================] Remaining Time: < 1 sec
[2018-09-14 18:22:22][INFO][mode: settings][stage: data migration][step: Settings Step]: started
100% [============================] Remaining Time: < 1 sec
[2018-09-14 18:22:24][INFO][mode: settings][stage: data migration][step: Stores Step]: started
100% [============================] Remaining Time: < 1 sec
[2018-09-14 18:22:24][INFO][mode: settings][stage: volume check][step: Stores Step]: started
100% [============================] Remaining Time: < 1 sec
[2018-09-14 18:22:24][INFO][mode: settings][stage: volume check][step: Stores Step]: Migration completed

Of course, this is a simple case and your store may be more complex than a stock installation.

If so, consider changing the following default setting in the XML configuration file:

vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.3.9/config.xml:

<settings_map_file>etc/opensource-to-opensource/settings.xml.dist></settings_map_file>

Copy this to settings.xml before editing.

And don’t forget to set the right filename in config.xml!

<settings_map_file>etc/opensource-to-opensource/settings.xml</settings_map_file>

Now open up settings.xml and study its contents, but it’s pretty self-explanatory.

If you need to ignore a setting, you do:

<ignore>
   <path>path/to/setting</path>
</ignore>

To rename:

<rename>
            <path>crontab/jobs/system_backup/schedule/cron_expr</path>
            <to>crontab/default/jobs/system_backup/schedule/cron_expr</to>
</rename>

You should be able to set up settings.xml any way you need.

Initial Data Migration

Now, we’re ready to transfer orders, customers and the actual store data. Do this by inputting the following command:

# php bin/magento migrate:data vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.3.9/config.xml

When I ran it, I got the “Integrity Check failed” message along with a bunch of errors:

# php bin/magento migrate:data vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.3.9/config.xml

[2018-09-17 20:40:46][INFO][mode: data][stage: integrity check][step: Data Integrity Step]: started
100% [============================] Remaining Time: < 1 sec
[2018-09-17 20:40:47][ERROR]: Foreign key (FK_CATALOG_EAV_ATTRIBUTE_ATTRIBUTE_ID_EAV_ATTRIBUTE_ATTRIBUTE_ID) constraint fails on source database. Orphan records id: 121,164 from `catalog_eav_attribute`.`attribute_id` has no referenced records in `eav_attribute`
[2018-09-17 20:40:47][ERROR]: Foreign key (FK_CAT_PRD_ENTT_INT_ATTR_ID_EAV_ATTR_ATTR_ID) constraint fails on source database. Orphan records id: 121 from `catalog_product_entity_int`.`attribute_id` has no referenced records in `eav_attribute`
[2018-09-17 20:40:47][ERROR]: Foreign key (FK_EAV_ENTITY_ATTRIBUTE_ATTRIBUTE_ID_EAV_ATTRIBUTE_ATTRIBUTE_ID) constraint fails on source database. Orphan records id: 121,163,164 from `eav_entity_attribute`.`attribute_id` has no referenced records in `eav_attribute`
…..
…..
…..

Let’s fix them one by one.

The first error, “Foreign key constraint fails,” is actually a pretty simple issue:

[2018-09-18 18:31:18][ERROR]: Foreign key (FK_CATALOG_EAV_ATTRIBUTE_ATTRIBUTE_ID_EAV_ATTRIBUTE_ATTRIBUTE_ID) constraint fails on source database. Orphan records id: 121,164 from `catalog_eav_attribute`.`attribute_id` has no referenced records in `eav_attribute`

We just have to delete any orphaned records from the specified tables:

DELETE FROM catalog_eav_attribute WHERE attribute_id IN (121, 164);

The second error was “Field is not mapped”:

[2018-09-18 18:56:57][INFO][mode: data][stage: integrity check][step: EAV Step]: started
100% [============================] Remaining Time: < 1 sec
[ERROR]: Source fields are not mapped. Document: customer_eav_attribute. Fields: is_used_for_customer_segment

Here's the manual way to do this, but if you'd like to save some time, there's also an automated method (scroll down a bit).

1. Manual Way

To fix this one, just exclude the fields in question from the map-eav.xml file:

 <options>
        <map_file>etc/opensource-to-opensource/1.9.3.9/map.xml.dist</map_file>
        <eav_map_file><strong>etc/opensource-to-opensource/map-eav.xml.dist</eav_map_file>

Since it’s under the EAV step, the correct file is map-eav.xml.dist.

You can choose to rename map-eav.xml.dist to map-eav.xml, or leave it as it is, but if you do rename it, don’t forget to change the config.xml file as well.

Let’s open up the map-eav XML file and add the following lines inside the <field_rules> </field_rules> tags:

<field_rules>
….
            <ignore>
                <field>customer_eav_attribute.is_used_for_customer_segment</field>
            </ignore>
…..
</field_rules>

These lines tell the migration tool to ignore the unmapped field.

After modifying any XML file, make sure to add an “-r” (reset) switch to the migrate:data command so the changes are picked up:

php bin/magento migrate:data  -r vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.3.9/config.xml

The next issue involves a message saying that the “Source documents are not mapped.

You can take care of this issue manually, as I’ve done below, but there’s also an automated method that I’ll describe next.

[2018-09-18 19:21:52][INFO][mode: data][stage: integrity check][step: Map Step]: started
100% [============================] Remaining Time: < 1 sec
[2018-09-18 19:21:55][ERROR]: Source documents are not mapped: paybox_question_number,social_facebook_actions,strikeiron_tax_rate

By “documents” the tool means the DB tables.

We have to work with the XML map file since it falls under the Map step:

<options>
        <map_file>etc/opensource-to-opensource/1.9.3.9/map.xml.dist</map_file>

Place ignore statements in that file like so:

<document_rules>
…...
            <ignore>
                <document>paybox_question_number</document>
            </ignore>
            <ignore>
                <document>social_facebook_actions</document>
            </ignore>
            <ignore>
                <document>strikeiron_tax_rate</document>
            </ignore>
….
</document_rules>

You get the idea; we’re just adding ignore statements to the XML map file.

If it’s a document, we ignore the corresponding table, and if it’s a field, we ignore a table field.

Since this will require a bit of copying and pasting, it may take some time to complete.

For me, it took around half an hour to finally pass the integrity check.

2. Automated Way

This magic option will save you tons of time.

[NOTICE]: You can use --auto or -a option to ignore not mapped differences between source and destination to continue migration

Finally, I was able to successfully migrate all the data:

php bin/magento migrate:data  -r -a  vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.3.9/config.xml
[2018-09-19 17:48:50][INFO][mode: data][stage: integrity check][step: Data Integrity Step]: started
…..
…..
…..
[2018-09-19 17:49:11][INFO][mode: data][stage: volume check][step: Tier Price Step]: started
100% [============================] Remaining Time: &lt; 1 sec
[2018-09-19 17:49:11][INFO][mode: data][stage: data migration][step: SalesIncrement Step]: started
100% [============================] Remaining Time: &lt; 1 sec
[2018-09-19 17:49:11][INFO][mode: data][stage: volume check][step: SalesIncrement Step]: started
100% [============================] Remaining Time: &lt; 1 sec
[2018-09-19 17:49:11][INFO][mode: data][stage: data migration][step: PostProcessing Step]: started
100% [============================] Remaining Time: &lt; 1 sec
[2018-09-19 17:49:11][INFO][mode: data][stage: data migration][step: PostProcessing Step]: 
<strong>Migration completed</strong>

Now, if we open the newly migrated Magento 2 site, we’ll see something like this:

Magento 2 After Data Migration Magento 2 After Data Migration

Our Magento 1 categories and images are missing, so we’re not quite done yet.

Migration of Media Files

Expect to manually transfer a few aspects of your site, such as images.

Important: If you store your media in a database (as Magento 1 can do), it will automatically get transferred to M2.

Just Sync it before data migration:

  • Magento 1 backend menu > System > Configuration > Advanced > System

  • Click on a Synchronize button

Synchronize Media Synchronize Media

In this example, I’ve stored the media in a file.

To copy the data over to Magento 2, we simply copy the M1:media folder over to M2:pub/media:

cp -r /path/to/magento1/media /path/to/magento2/pub

After that, we have to reindex the Magento 2 store with the following command:

php bin/magento indexer:reindex

Then recompile it:

php bin/magento deploy:mode:set production

Now, our website should have all our M1 products, along with their images:

Magento 2 With Migrated Products Magento 2 With Migrated Products

Magento 2 Theme Migration

You’ll save a lot of time by going with a premade Magento 2 theme.

You can port your existing M1 theme, but if you do that, prepare to dive deep into M2 internals.

In most cases, you are better off hiring a skilled Magento 2 developer.

A few things to keep in mind:

  • The XML structure of M2 is different than that of M1, so you cannot simply copy and paste the layout files.

  • M2 replaces the prototype JS framework of M1 with jQuery.

  • CMS syntax is similar, so you can make CMS pages work with a little effort.

  • I recommend basing your newly created M2 theme off the default Luma theme. This will give you a strong framework to work with.

  • M2 features three running modes: default, developer, production. Use developer mode to develop, and then production mode once the site goes live.

Magento 2 Extension Migration

M1 plugins won’t work with M2.

There is no reliable (at least to my knowledge) Magento 2 code migration tool that magically turns a M1 module into a M2-compatible extension.

You’ll have to do most of the work manually.

My advice would be to make a list of extensions you’d like to have on your new M2 site.

Then either hire a Magento 2 developer (one who offers a Magento migration service) to refactor your extensions or find alternatives for them yourself.

When Things Go Wrong

The Magento 2 migration process isn’t always smooth, and many things may not go as planned.

Below, you’ll find some of the most common issues you might come across while moving to M2.

1. Empty Category / Landing Page

After migration, you may find that some pages look like this:

Magento 2 Empty Page After Migration Magento 2 Empty Page After Migration

Typically, this happens due to incorrect CMS syntax.

For example, a page can load static blocks with the following code:

This works with Magento 1 but not Magento 2.

You will need to change type=”cms/block” to class=”Magento\CMS\Block\Block”:

Those of you who are familiar with a MySQL command line could just run:

UPDATE cms_block SET content=replace(content,'type="cms/block"','class="Magento\\Cms\\Block\\Block"');

This will update all static blocks.

2. Missing Catalog Images

This issue might be hard to spot.

If your newly migrated M2 store has broken product images on all category pages, or just a few of them, there’s an easy solution.

Just regenerate the image cache with this simple command:

php bin/magento catalog:image:resize

This will resize the product images and put them in the correct cache folders.

The command may take some time to run, so be prepared to wait.

3. “No Such Entity” Error

This error message may appear in various places, including when you’re reindexing your store, performing backend activities, or on some of the frontend pages.

From my experience, this error is usually related to corrupted store tables: store, store_group or store_website.

Login to the MySQL command line tool (or use phpMyAdmin) and inspect those tables.

Do all group_ids found in the store also exist in store_group?

Is this true for the website_id field and store_website table?

Here are a few helpful SQL queries:

SELECT * FROM store JOIN store_group USING (group_id) WHERE default_store_id IS NULL;

SELECT * FROM store JOIN store_website USING (website_id) WHERE default_group_id IS NULL;

If any of the above queries return non-empty results, you have a corrupt table.

Dive deeper and delete hanging records.

4. Double .html.html in Product Urls

Some users report having .html.html in almost all product urls after they move to Magento 2.

That happens because they have .html suffix in either url_key or url_path product attributes.

Here is how to get rid of it.

Warning: Your wrong .html.html urls might have already been indexed by google. To avoid loosing any traffic make sure you put .html.html -> .html redirect in place after you apply my fix.

Login to DB and issue the following commands:

UPDATE catalog_product_entity_varchar 
 SET value=replace(value,'.html','') WHERE attribute_id IN 
(SELECT attribute_id FROM eav_attribute 
    WHERE attribute_code in ('url_key','url_path') AND entity_type_id = 
       (SELECT entity_type_id FROM eav_entity_type WHERE entity_type_code='catalog_product'))
DELETE FROM url_rewrite WHERE target_path LIKE "%.html.html%";
UPDATE IGNORE url_rewrite SET request_path = REPLACE(request_path, '.html.html', '.html');

Then reindex the database:

php bin/magento indexer:reindex

And .html.html should now be replaced with a single .html.

Magento 2 Delta Migration

After the initial data migration, and theme, customizations and extensions migrations, you are ready to go live.

The final step is to transfer any new orders, customers (and sometimes products) created in your production Magento 1 store since you first migrated data to Magento 2.

Obviously, starting from scratch isn’t an option since you’d lose all of your customizations.

That’s when delta migration feature comes handy.

Data migration tool allows you to migrate only specific changes.

You can run these incremental updates as often as you want.

The command is simple:

php bin/magento migrate:delta /path/to/config.xml

In our sample 1.9.3.9 -> 2.2.5 case, that would be:

php bin/magento migrate:delta vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.3.9/config.xml

Things to keep in mind about delta migration

There are a few things to remember while loading incremental updates:

  1. Delta configuration doesn’t track every change that the Magento 1 database might have.

  2. After an initial migration, do not remove or add products, orders, customers from and to your new Magento 2 database. This may break the delta migration process.

  3. Do not change inventory levels in either of the databases (M1 and M2), or it may not be picked up by the delta script.

Test, Test, Test

I can’t emphasize it enough - thoroughly test your newly created Magento 2 before going live!

Make a list of ALL the features of your current Magento 1 and check them one by one.

Make sure all the payment methods work and the money is captured.

Test any complex shipping setups to make sure they work like they’re supposed to.

This may seem like common sense, but you would be surprised at the amount of mistakes you’d spot through regular check-ups.

At the end of the day, you want your shop to be stable, user-friendly and profitable.

You can’t expect any profits from a website that barely works.

A Few Final Words

Going from Magento 1 to Magento 2 is no easy task, but with enough planning and attention to detail, you can pull it off with little-to-no downtime.

One important piece of advice I’d like to share:

Test drive your migrated Magento 2 store.

Test every bit of it: checkout, customer registration, product management.

Let me say a few words about load testing. This is when you put your new website under a simulated amount of average traffic.

How many people visit your Magento 1.x store a minute?

10, 50, 100?

Go ahead and see what happens to your Magento 2 site when 100 visitors hit it.

Does it become unbearably slow?

Do all the functions still work?

Can you checkout?

You want to make sure the site works well under every day traffic.

Once you’ve made sure all of that is in order, the next thing would be to pull the plug, make a DNS switch, and see your M2 site go live.

As always, you are very welcome to share your migration experience in the comment section.

All the best!

  

Need help with a Magento site? Hire me!

 

Other articles you might be interested in:

  

If you find this post interesting do not hesitate to sign up for our newsletter and join the 879 people who receive Magento news, tips and tricks regularly.

Thank You!