Magento 2 Lock Files

From time to time the Klevu Search extension will synchronise data between Magento and Klevu in order to receive the most up to date information about your store’s products. This process happens during a scheduled task (a cron job) which runs regularly according to your configuration settings.

What is a Lock File?

While this process is running, a ‘lock file’ is created within your Magento instance, which is used as a kind of flag to know that a process is already running, to prevent the same process starting again whilst one is already running. Once the sync process has completed, the Klevu extension will remove the lock file, allowing a subsequent process to perform another sync. It looks something like this:

  1. A new process is requested, no lock file exists so synchronisation begins and a new lock file is created.
  2. After a few minutes, another sync process is attempted, but is rejected since a lock file already exists.
  3. The original sync process completes and the lock file is removed.
  4. Yet another process is requested, but this time there is no lock file so all of the above is repeated.

In this manner, your Magento instance can efficiently process data continuously whilst ensuring tasks are not being duplicated or repeated, with a regular stream of updates being submitted to Klevu.

Stale Lock Files

Under normal circumstances, the above process works harmoniously with lock files being created and deleted. Problems can occur when, for any number of reasons, a sync process does not complete and the lock file is not deleted. In this scenario, no future data syncs will be allowed since there is an orphaned or stale lock file. When this happens, you may notice that products which have been recently updated are not being reflected in Klevu Search results.

The Klevu Search extension will attempt to detect and resolve this scenario for you, but sometimes human intervention is required to check the existence of these lock files. Within our Klevu Search configuration under Stores > Configuration > Klevu > Search Configuration > Data Sync Settings, you will find a section called ‘Cron Status’.

magento-lock-files

Here we can see the existence of two lock files: `default_klevu_running_index.lock` and `klevu_running_index.lock`. If the date of either of these lock files is a particularly long time in the past, then something may be wrong and you should click on ‘Clear Klevu Lock File’ to delete them and allow new sync processes to start.

In the same configuration area, we also have a setting available to automatically detect and delete lock files older than a particular age, to help you automate this process.

How or Why are Lock Files Orphaned?

A common cause is the use of Object method instead of Collection Method, which causes many products to be individually loaded and uses a lot of server memory, until the point it runs out and the process dies half way through. As a result the lock file it created would never be removed.

There are also other more general memory issues, such as when your server is under heavy load for unrelated reasons during a Klevu sync, and the server itself dies or has to end certain processes. This can result in these lock files remaining and becoming orphaned.

The solution to these problems is to explore using Collection Method for sync, staggering your scheduled tasks so that Klevu sync does not run at the same time as other heavy processes, or increasing the available CPU or RAM capacity of your server.