Locks and Queues

[SteveB]

Mostly about managing a queue and locking data while it's being updated.

I'm wondering if anyone has a different idea about this or feels uncomfortable about employing Page, Template, etc. instead of a module. I like it this way because it's so adaptable but maybe someone has a more appealing way.

Scenario:

The site uses lots of page fields and most of its pages are indexed by ElasticSearch. When a page is indexed, page fields are (usually) represented in the index by the title field of the referenced page. ElasticSearch automatically re-indexes a page whenever it is saved. That's great but if A points to B and we save B after renaming it, B is re-indexed but A still has the old title in its search index. 

Solution:

To handle that I have code hooked after Pages::saveReady make a PageArray of pages we'd like to re-index. Usually just a few but in rare instances a few thousand. Some care is taken with the querying in case we have a lot.

To queue that work I'm using a page with a page field to represent the queue. You use a urlSegment to pass it a command (similar to a process page). Setting aside some details about access, the hooked code adds pages to the page array. Cron calls the queue page periodically, code in the template file does the first N pages, removes them from the page field and updates the page. 

So far so good and it has worked very nicely but two things are asynchronously fetching and updating that page field without any locking. They're quick but if we're not lucky updates will be missed or repeated.

So I wrote some little functions to add, remove and test Page::statusLocked on the queue page. Docs say it's not enforced by the core (but is checked by Process modules). Both the hooked code and the queue code will respect the lock, wait and retry for a few seconds. After updating the page they release the lock. To avoid problems with stuck locks, the function that waits and retries will only wait so long before unlocking the page. It waits longer than we'd ever need to keep something locked in normal operation.

Thanks. Details on request, trying not to bury the main points.

[horst]

Sounds very interesting!

I'm specially interested to know how the lock is respected. Is it only in your own functions and process module, or do all Process Modules respect it? (If yes to all ProcessModules, where can I view this?)

[SteveB]

Sorry Horst, it's just 3 functions to make working with the page status attribute more convenient. I'd say these are work in progress but here's what they look like now.

Edit: Skip ahead... These fns were the wrong approach.

//with default args will wait up to 4s for locked page to unlock. Returns false or the page.
function claimPage($page, $tries=5, $ms=100){
	if($page->hasStatus(Page::statusLocked)) $page = waitForPage($page, $tries, $ms);
	if(! $page) return false;
	$page->addStatus(Page::statusLocked);
	$page->save();
	return $page;
}
//if page is locked, wait and recheck. Returns false or the page.
function waitForPage($page, $tries=5, $ms=100){
	$t = 500;
	while($page->hasStatus(Page::statusLocked) && $tries--){
		$t += $ms;
		usleep($ms);
		$this->pages->uncache($page);	//so we get current status
	}
	if (! $tries && $page->hasStatus(Page::statusLocked)){
		$this->releasePage($page);	//we're expiring the lock afer a decent wait...
		return false;	//but honoring the lock
	}
	return $page;
}
//unlock and save 
function releasePage($page){
	$page->removeStatus(Page::statusLocked);
	$page->save();
	return $page;	
}

 

[netcarver]

Hi SteveB,

I'm pretty sure the code you posted above will not solve your problem. What happens if...

• We start off with page P not having a locked status.
• Thread A then gets scheduled, and runs the code in claimPage(P). It gets halted just before the line $page->save(); gets executed. Execution is then passed to thread B which is also trying to claim the page - so, it too, runs the claimPage(P) method - but this time to completion (saving P with the flag) and gets page P back. Eventually B is halted and focus switched back to thread A which picks up where it left off, and saves P again (with the locked flag) and gets page P back from the routine.

Both threads will think they have successfully claimed the page, yet neither of them had to waitForPage(P);

[SteveB]

Whoops. Reminds me why I used use mysql update for this kind of thing. Was trying to stick with pages etc. Will rethink. Thanks!

[netcarver]

As a possible alternative, take a look at using PHP's flock() call with the LOCK_EX flag (either in blocking or non-blocking mode.) I've done that before - it's portable - and it works well. Take a look at the examples and comments on that page, you might spot something that fits your needs.

[SteveB]

Okay, using flock now. I ended up making a class to manage locks on PW pages. The lock file goes in with the page's other file assets. Automatic unlock on destruct. Logging if you want it.

That got me thinking about adding this support for locking to the Page class as described here but it's not really necessary.