Jump to content

file update collisions with lazycron?


Ken Muldrew
 Share

Recommended Posts

I had an odd bug that I haven't been able to replicate and I'm wondering if it's possible for some code run by an AJAX call to simultaneously try to overwrite a file that is also being updated by a lazycron routine. 

In order to get the website to load in a reasonable time, I read in a bunch of asset data from a static json file rather than using page->children(). Every couple of hours, lazycron reads through all the assets and updates the file. When users add new assets, though, the code that creates a new asset page also appends the record to the end of the json file so that it will be visible right away. This usually works fine but the other day a user added six assets in a relatively short period and the json file got corrupted. Due to competing demands for my time, and the panic of my boss over the empty screen, I revived the website quickly without saving the corrupt file for later analysis. I have tried to replicate the error on a staging site but with no success. 

Does anyone think that a collision is at all likely in this scenario?
Is there a way to either suspend the execution of a lazycron routine, or to suspend access to a particular file, in code? 

Link to comment
Share on other sites

You could employ PHP's flock function both in the code that updates your JSON file and the lazy cron script, though you have to at least consider how it will affect execution time for concurrent requests that have to wait until existing locks have been released. Rule of thumb: lock late, release early, i.e. try to perform all processing before you lock the file, then lock, update, release.

  • Like 2
Link to comment
Share on other sites

Many thanks, BitPoet. I wasn't even aware of that function. It may be a while before I see if it solves the problem, but for what it's worth, here is how I implemented locking in lazy cron and the script called from AJAX:

excerpt from lazy cron:

        $json = json_encode($mapData);
        $fp = fopen('site/templates/api.json', 'r+');
        if (flock($fp, LOCK_EX)) { // lock the file
            fwrite($fp, $json);
            flock($fp, LOCK_UN); // unlock the file
            //$log->save('user_activities','api.json updated with ' . count($mapData['locations']) . ' records');
        } else {
            // flock() returned false, no lock obtained
            $log->save('user_activities',"Could not lock api.json. Update failed");
        }
        fclose($fp);
        
excerpt from form.php:

                    // lock api.json while reading/writing to prevent collision with lazycron
                    $handle = fopen('api.json', 'r');
                    while (!flock($handle, LOCK_SH)) { usleep(1); }
                    $inp = fread($handle, filesize('api.json'));
                    $tempArray = json_decode($inp, true); // convert json to php array
                    array_push($tempArray['locations'], $mapData['locations']); // add new data to end of array
                    $jsonData = json_encode($tempArray); // convert back to json
                    file_put_contents('api.json', $jsonData); // re-write the file 
                    flock($handle, LOCK_UN);
                    fclose($handle);
                    
 

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...