Magento 2 Data Sync Overview

The Klevu Search Magento 2 extension needs to regularly synchronise Product, Category and CMS data with Klevu.

There are three different ways this can be achieved:

  1. Via the native Magento cron system (default).
  2. Configure a cron on your own server (recommended).
  3. You provide us with an XML Feed URL, which you maintain (full control).
Note: Order data is separate and is automatically synchronised with Klevu via the native Magento Cron, every day at 2AM local time.

Native Magento Cron (default)

This is the default method because it is the simplest, does not require any additional work on your part and will behave as expected for the majority of smaller stores.

The frequency at which Klevu synchronises any changes to your Products, Categories and CMS data can be configured by accessing the Store Configuration > Klevu > Data Sync Settings:

magento-data-sync-frequency

In the case of larger stores with more products, the process of synchronising data with Klevu can take a long time and cause delays to the other scheduled tasks you have configured within your Magento instance. Furthermore, those other scheduled tasks can also delay the Klevu sync from running, causing delays in your search results showing the most recent product information.

To avoid this scenario you can instead configure an External Cron which runs independently of the native Magento Cron, which is detailed below.


External Cron (recommended)

This is the recommended approach for most stores.

Instead of relying on the native Magento cron, which can be unpredictable depending on the other extensions you have installed, you will instead have your systems administrator configure server-level scheduled cron tasks which will synchronise your Products, Categories and CMS data with Klevu according to a schedule you define.

1. First, disable synching of Klevu data via the native Magento Cron by accessing Store Configuration > Klevu > Data Sync Settings and specifying Frequency: Never.

2. Next, speak to your systems administrator and have them configure a new set of system scheduled cron tasks by referencing the example below:

*/5 * * * * /usr/bin/php /path/to/bin/magento klevu:syncstore:storecode default >> /path/to/var/log/Klevu_Cron.log

With the above system crontab configuration, the Klevu command klevu:syncstore:storecode will run for store view ‘default’ every five minutes, regardless of what other scheduled tasks are configured on your server or within your Magento instance. You can confirm the scheduled task is working correctly by monitoring the log file var/log/Klevu_Cron.log, which will be populated with the output of the Klevu sync command.

If you have multiple websites or store views, the process is the same but repeated once per store view. If you find you encounter memory issues with this approach, your systems administrator can stagger the timing of these commands to ensure they don’t all run at the same time.

For example the following will synchronise store views ‘default’, ‘french’ and ‘german’ at slightly different times each hour.

0,10,20,30,40,50 * * * * /usr/bin/php /path/to/bin/magento klevu:syncstore:storecode default >> /path/to/var/log/Klevu_Cron_default.log

3,13,23,33,43,53 * * * * /usr/bin/php /path/to/bin/magento klevu:syncstore:storecode french >> /path/to/var/log/Klevu_Cron_french.log

7,17,27,37,47,57 * * * * /usr/bin/php /path/to/bin/magento klevu:syncstore:storecode german >> /path/to/var/log/Klevu_Cron_german.log
Note: When using the above commands you do not need to run the klevu:syncdata command.

You don’t need to worry about any of these tasks overlapping one another. The Klevu extension maintains ‘lock files’ for each process to ensure only one can run at any given time. You can learn more about this here: Magento 2 Lock Files.

Magento Commerce Cloud

If you are a Magento Cloud customer, you can configure the above cron jobs by adding them to your .magento.app.yaml file. Please read Magento DevDocs: Add custom cron jobs to your project for more information and have your systems administrator adapt the above instructions for your own implementation.


XML Feed URL (full control)

If for any reason you are not happy with the above options, you can also opt to take full control over the data sync by providing us with a URL where you will host an XML product feed, which you keep up to date according to your own business logic.

Klevu has an internal scheduling system which we can configure to download your feed at regular intervals and synchronise your data that way. If you would like to explore this option, please contact support@klevu.com.