Execute slow tasks without waiting

[Robin S]

Sometimes you need to execute a slow task after some event occurs in the PW admin, and normally you have to wait for this task to finish before you can continue using the admin. This is because PHP is "blocking", meaning that while one thing is executing nothing else can execute.

There are potentially lots of different kinds of tasks that could be slow, but just as an example suppose you want to generate resized variations of images on a page, and there are a lot of images. You might have a hook like this so that any non-existing variations are created when the page is saved:

$pages->addHookAfter('saveReady', function(HookEvent $event) {
	/** @var Page $page */
	$page = $event->arguments(0);

	// When a gallery page is saved
	if($page->template == 'gallery') {
		// Create an image variation for each image
		foreach($page->images as $image) {
			$image->size(1200, 1200);
		}
	}
});

When you save a gallery page in the PW admin, the admin will be unresponsive and will only load again after all the variations have been created.

I wanted to find a way for slow tasks to be triggered by events in the PW admin and for the website editor not to have to wait for the task to finish before continuing with other work in the admin. Inspired by this StackOverflow answer I came up with the following solution that seems to work well. Using the image variations task above as an example...

First we make use of the URL hooks feature to set up a URL that can trigger tasks to run when it is loaded:

// A URL that will trigger tasks when loaded
$wire->addHook('/run-task/', function($event) {
	$input = $event->wire()->input;

	// A simple check to avoid unauthorised access
	// You could implement more advanced checks if needed
	if($input->post('key') !== 'cTdPMBQ7x8b7') return false;

	// Allow the script to keep running even though we have set a short WireHttp timeout
	ignore_user_abort(true);

	// The "create variations" task
	if($input->post('task') === 'create-variations') {
		$page_id = (int) $input->post('page');
		$p = $event->wire()->pages->get($page_id);
		// Create an image variation for each image
		foreach($p->images as $image) {
			$image->size(1200, 1200);
		}
		return true;
	}

	return false;
});

Then in the Pages::saveReady hook we use WireHttp to load that URL and post parameters that define what task to run and anything else needed for the task (in this case the ID of the page that has been saved).

$pages->addHookAfter('saveReady', function(HookEvent $event) {
	/** @var Page $page */
	$page = $event->arguments(0);

	// When a gallery page is saved
	if($page->template == 'gallery') {
		// Load the /run-task/ URL using WireHttp
		$http = new WireHttp();
		// Set a short timeout so we don't have to wait until the script finishes
		// Timeout values shorter than 1 second can be tried once a core issue is fixed
		// https://github.com/processwire/processwire-issues/issues/1773
		$http->setTimeout(1);
		$url = $event->wire()->config->urls->httpRoot . 'run-task/';
		$data = [
			'key' => 'cTdPMBQ7x8b7',
			'task' => 'create-variations',
			'page' => $page->id,
		];
		$http->post($url, $data, ['use' => 'curl']);
	}
});

By doing it this way the task runs in a separate request and the website editor doesn't have to wait for it to finish before they can continue working in the PW admin.

[szabesz]

Thanks for sharing! As always, your contribution to ProcessWire and its community rocks!

[kongondo]

Thanks for this @Robin S. 

Maybe this technique can be used to achieve a non-blocking SSE? @bernhard, what do you think? I haven't thought it through; just wondering ATM.

For reference:

 

[bernhard]

Hey @kongondo thx. Not sure what you mean exactly. I already have a working solution. The key is to create the SSE stream before the session is started: https://github.com/baumrock/RockFrontend/blob/7ea19d668c5a2acc74777374b51535d42aa56381/RockFrontend.module.php#L121

That way the script from the SSE endpoint does not block the server for regular requests of the user. I agree that it sounds similar but I think it's two different things and it's good to have two different solutions for it 🙂 

[kongondo]

6 hours ago, bernhard said:

Not sure what you mean exactly. I already have a working solution. The key is to create the SSE stream before the session is started: https://github.com/baumrock/RockFrontend/blob/7ea19d668c5a2acc74777374b51535d42aa56381/RockFrontend.module.php#L121

Aha! Thanks @bernhard. I wasn't aware you'd found a non-blocking solution. Thanks!

[WilliamBruce]

On 6/30/2023 at 8:49 AM, Robin S said:

Sometimes you need to execute a slow task after some event occurs in the PW admin, and normally you have to wait for this task to finish before you can continue using the admin. This is because PHP is "blocking", meaning that while one thing is executing nothing else can execute.

There are potentially lots of different kinds of tasks that could be slow, but just as an example suppose you want to generate resized variations of images on a page, and there are a lot of images. You might have a hook like this so that any non-existing variations are created when the page is saved:

$pages->addHookAfter('saveReady', function(HookEvent $event) {
	/** @var Page $page */
	$page = $event->arguments(0);

	// When a gallery page is saved
	if($page->template == 'gallery') {
		// Create an image variation for each image
		foreach($page->images as $image) {
			$image->size(1200, 1200);
		}
	}
});

When you save a gallery page in the PW admin, the admin will be unresponsive and will only load again after all the variations have been created.

I wanted to find a way for slow tasks to be triggered by events in the PW admin and for the website editor not to have to wait for the task to finish before continuing with other work in the admin. Inspired by this StackOverflow answer I came up with the following solution that seems to work well. Using the image variations task above as an example...

First we make use of the URL hooks feature to set up a URL that can trigger tasks to run when it is loaded:

// A URL that will trigger tasks when loaded
$wire->addHook('/run-task/', function($event) {
	$input = $event->wire()->input;

	// A simple check to avoid unauthorised access
	// You could implement more advanced checks if needed
	if($input->post('key') !== 'cTdPMBQ7x8b7') return false;

	// Allow the script to keep running even though we have set a short WireHttp timeout
	ignore_user_abort(true);

	// The "create variations" task
	if($input->post('task') === 'create-variations') {
		$page_id = (int) $input->post('page');
		$p = $event->wire()->pages->get($page_id);
		// Create an image variation for each image
		foreach($p->images as $image) {
			$image->size(1200, 1200);
		}
		return true;
	}

	return false;
});

Then in the Pages::saveReady hook we use WireHttp to load that URL and post parameters that define what task to run and anything else needed for the task (in this case the ID of the page that has been saved).

$pages->addHookAfter('saveReady', function(HookEvent $event) {
	/** @var Page $page */
	$page = $event->arguments(0);

	// When a gallery page is saved
	if($page->template == 'gallery') {
		// Load the /run-task/ URL using WireHttp
		$http = new WireHttp();
		// Set a short timeout so we don't have to wait until the script finishes
		// Timeout values shorter than 1 second can be tried once a core issue is fixed
		// https://github.com/processwire/processwire-issues/issues/1773
		$http->setTimeout(1);
		$url = $event->wire()->config->urls->httpRoot . 'run-task/';
		$data = [
			'key' => 'cTdPMBQ7x8b7',
			'task' => 'create-variations',
			'page' => $page->id,
		];
		$http->post($url, $data, ['use' => 'curl']);
	}
});

By doing it this way the task runs in a separate request and the website editor doesn't have to wait for it to finish before they can continue working in the PW admin.




Thanks for sharing  🙂 As a college student, I often struggle with essay writing and meeting deadlines. Fortunately, I stumbled upon the site specifically their free essay examples on perception. read here valuable resource not only teaches me how to write an essay effectively but also provides inspiration and guidance to ensure the timely completion of my assignments. I highly recommend it to fellow students.

Thanks for sharing 🙂