Jump to content
mtwebit

Tasker

Recommended Posts

Tasker is a module to handle and execute long-running jobs in Processwire. It provides a simple API  to create tasks (stored as PW pages), to set and query their state (Active, Waiting, Suspended etc.), and to execute them via Cron, LazyCron or HTTP calls.

Creating a task

$task = wire('modules')->Tasker->createTask($class, $method, $page, 'Task title', $arguments);

where $class and $method specify the function that performs the job, $page is the task's parent page and $arguments provide optional configuration for the task.

Executing a task

You need to activate a task first

wire('modules')->Tasker->activateTask($task);

then Tasker will automatically execute it using one of its schedulers: Unix cron, LazyCron or TaskerAdmin's REST API + JS client.

Getting the job done

Your method that performs the task looks like

public function longTask($page, &$taskData, $params) {
  ...
}

where $taskData is a persistent storage and $params are run-time options for the task.

Monitoring progress, management

The TaskerAdmin module provides a Javascript-based front-end to list tasks, to change their state and to monitor their progress (using a JQuery progressbar and a debug log area). It also allows the on-line execution of tasks using periodic HTTP calls performed by Javascript.

TaskerAdmin.png.e535554a45673edd5dd1049b910585fe.png

 

Monitoring task progress (and log messages if debug mode is active)

TaskMonitor.png.912e5949fab4d8a61f39262e2e682ddd.png

 

Task data and log

TaskData.png.e5bd86b403dad7b2718cf4fdd39ec175.pngTaskLog.png.1795f8a454cba4d9756877e77b5ed820.png

Detailed info (setup, task dependencies, time limits, REST API etc.) and examples can be found on GitHub.

This is my first public PW module. I'm sure it needs improvement :)

 

  • Like 15

Share this post


Link to post
Share on other sites

hi @mtwebit welcome to the forum and thanks for that module! i've created something similar for one of my projects but it's custom and not built into a module so i think there's potential for what you are doing :)

some ideas/questions/feedback:

  • use InputfieldMarkup to structure your custom code and make it look&feel more native. see my new tutorial here: 
  • do you know the wirequeue module?
  • would be great to have an easy way to integrate this into other modules or custom pages. maybe it is already possible - i'll have a look when i have more time :)

This is how my progressbar looks like using InputfieldMarkup:

2017-11-09 16_20_28-Edit Page_.png

  • Like 4

Share this post


Link to post
Share on other sites

Thanks for the feedback. I'll definitely have a look at InputfieldMarkup and wirequeue.

It is possible to integrate the UI elements into other pages. There's a renderTaskList() method to perform this:

    if(wire('modules')->isInstalled("TaskerAdmin")) {
      $out .= '<h3>Tasks</h3>';
      $out .= wire('modules')->get('TaskerAdmin')->renderTaskList($selector);
    }

It can render a task list or a detailed task monitoring div including JS code to show the progressbar or even to execute the task when displaying the page.
It needs improvement, however, as its links point to the TaskerAdmin page atm and the list UI is not really customizable.

Share this post


Link to post
Share on other sites

I'm doing something similar (far from being as awesome as your module) using only some hobbled up pages and templates, and it's getting out of hand. This looks worthwhile to dig around in. I'll try it out and try to give helpful feedback. Thanks for sharing. :) 

Share this post


Link to post
Share on other sites

Thank you @mtwebit for that module! 

But I have a trouble with processwire setup with two site folders: "site" and "site-users". Tasker module is installed  in site-users/modules folder.

How to force processwire to load that modules instead of site/modules? Because runByCron.sh can't find Tasker module:

require "index.php";

if (!wire('modules')->isInstalled('Tasker')) {
echo 'Tasker module is missing.';
exit;
}

 

 

Nikolay.

 

 

SOLVED: By addition of $_SERVER['HTTP_HOST']=$siteDomain; before index.php

Edited by nicolant

Share this post


Link to post
Share on other sites

I tested this module sometime ago before the release of DataSet and I had to fix the hardcoded admin url. I will sent you the new pull-request this afternoon concerning the following fix/change, feel free to merge it or not.

 

  • install / uninstall written: fields and template created on installation / removed on uninstall
  • fix: corrected admin page link in module and js file
  • process module link moved under setup (can be moved later by the user without messing the admin url)
  • about, tips markup improvements

 

@mtwebit do you have an idea from where I could start to begin to port this module under Windows as the PCNTL extension isn't supported by this last OS ? 

I have already in mind implementing pthread to consume a Messaging Queueing System or something like that.

 

And for people who wonder how it works with DataSet, there are a small screencast (shortened) importing a CSV DataSet :

 

tasker.thumb.gif.e4a2d8b1b19b83b1f23cb546418c8570.gif

 

Edited by flydev
Question about Windows
  • Like 4

Share this post


Link to post
Share on other sites

@nicolant could you try !$this->wire('modules')->isInstalled... instead?

Share this post


Link to post
Share on other sites
On 1/6/2019 at 9:59 AM, flydev said:

 

@mtwebit do you have an idea from where I could start to begin to port this module under Windows as the PCNTL extension isn't supported by this last OS ? 

I have already in mind implementing pthread to consume a Messaging Queueing System or something like that.

Thanks for the screencast, @flydev. I really appreciate it.

Regarding the pcntl on Windows... Unfortunately, I don't have experience in this field. Windows has no signals (as far as I know) so I'd use a virtualization tool to overcome this limitation. You can try an API/kernel virtualization (e.g. Linux subsys for Windows) or use Docker.

Also thanks for the pull req. I'll check your suggestions.

Share this post


Link to post
Share on other sites
16 minutes ago, mtwebit said:

You can try an API/kernel virtualization (e.g. Linux subsys for Windows) or use Docker.

Ok that's an idea, I will take a look.

 

About the PR, I didn't had the time the other day, will try to re sent it today.

Thanks again for this great module.

Share this post


Link to post
Share on other sites

I've uploaded a new version (0.9.5) to GitHub.

It tries to handle DB connection loss errors and it has a basic profiler to optimize your import routines. See the wiki for more details.

Note: TaskerAdmin needs some fixes in its JS-based task executor. Don't use this feature atm. (Cron is always the preferred task execution method.)
 

  • Like 4

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By MoritzLost
      Process Cache Control
      This module provides a simple solution to clearing all your cache layers at once, and an extensible interface to perform various cache-related actions.
      The simple motivation behind this module was that I was tired of manually clearing caches in several places after deploying a change on a live site. The basic purpose of this module is a simple Clear all caches link in the Setup menu which clears out all caches, no matter where they hide. You can customize what exactly the module does through it's configuration menu:
      Expire or delete all cache entries in the database, or selectively clear caches by namespace ($cache API) Clear the the template render cache. Clear out specific folders inside your site's cache directory (/site/assets/cache) Clear the ProCache page render cache (if your site is using ProCache) Refresh version strings for static assets to bust client-side browser caches (this requires some setup, see the full documentation for details). This is the basic function of the module. However, you can also add different cache management action through the API and execute them through the module's interface. For this advanced usage, the module provides:
      An interface to see all available cache actions and execute them. A system log and logging output on the module page to see verify what the module is doing. A CacheControlTools class with utility functions to clear out different caches. An API to add cache actions, execute them programmatically and even modify the default action. Permission management, allowing you granular control over which user roles can execute which actions. The complete documentation can be found in the module's README.
      Plans for improvements
      If there is some interest in this, I plan to expand this to a more general cache management solution. I particular, I would like to add additional cache actions. Some ideas that came to mind:
      Warming up the template render cache for publicly accessible pages. Removing all active user sessions. Let me know if you have more suggestions!
      Links
      https://github.com/MoritzLost/ProcessCacheControl ProcessCacheControl in the Module directory CHANGELOG in the repository Screenshots


    • By Macrura
      PrevNextTabs Module
      Github: https://github.com/outflux3/PrevNextTabs
      Processwire helper modules for adding page navigation within the editor.
      Overview
      This is a very simple module that adds Previous and Next links inline with the tabs on the page editor. Hovering over the tab shows the title of the previous or next page (using the admin's built in jqueryUI tooltips.)
      Usage
      This module is typically used during development where you or your editors need to traverse through pages for the purpose of proofing, flagging and/or commenting. Rather than returning to the page tree or lister, they can navigate with these links.
      Warnings
      If you are using PW version 2.6.1 or later, the system will prevent you from leaving the page if you have unsaved edits.
      For earlier versions, to avoid accidentally losing changes made to a page that might occur if a user accidentally clicks on one of these, make sure to have the Form Save Reminder module installed.
      http://modules.processwire.com/modules/prev-next-tabs/
    • By Gadgetto
      SnipWire - Snipcart integration for ProcessWire
      Snipcart is a powerful 3rd party, developer-first HTML/JavaScript shopping cart platform. SnipWire is the missing link between Snipcart and the content management framework ProcessWire.
      With SnipWire, you can quickly turn any ProcessWire site into a Snipcart online shop. The SnipWire plugin helps you to get your store up and running in no time. Detailed knowledge of the Snipcart system is not required.
      SnipWire is free and open source licensed under Mozilla Public License 2.0! A lot of work and effort has gone into development. It would be nice if you could donate an amount to support further development:

      Status update links (inside this thread) for SnipWire development
      2020-03-21 -- SnipWire 0.8.5 (beta) released! Improves SnipWires webhooks interface and provides some other fixes and additions 2020-03-03 -- SnipWire 0.8.4 (beta) released! Improves compatibility for Windows based Systems. 2020-03-01 -- SnipWire 0.8.3 (beta) released! The installation and uninstallation process has been heavily revised. 2020-02-08 -- SnipWire 0.8.2 (beta) released! Added a feature to change the cart and catalogue currency by GET, POST or SESSION param 2020-02-03 -- SnipWire 0.8.1 (beta) released! All custom classes moved into their own namespaces. 2020-02-01 -- SnipWire is now available via ProcessWire's module directory! 2020-01-30 -- SnipWire 0.8.0 (beta) first public release! (module just submitted to the PW modules directory) 2020-01-28 -- added Custom Order Fields feature (first SnipWire release version is near!) 2020-01-21 -- Snipcart v3 - when will the new cart system be implemented? 2020-01-19 -- integrated taxes provider finished (+ very flexible shipping taxes handling) 2020-01-14 -- new date range picker, discount editor, order notifiactions, order statuses, and more ... 2019-11-15 -- orders filter, order details, download + resend invoices, refunds 2019-10-18 -- list filters, REST API improvements, new docs platform, and more ... 2019-08-08 -- dashboard interface, currency selector, managing Orders, Customers and Products, Added a WireTabs, refinded caching behavior 2019-06-15 -- taxes provider, shop templates update, multiCURL implementation, and more ... 2019-06-02 -- FieldtypeSnipWireTaxSelector 2019-05-25 -- SnipWire will be free and open source Plugin Key Features
      Fast and simple store setup Full integration of the Snipcart dashboard into the ProcessWire backend (no need to leave the ProcessWire admin area) Browse and manage orders, customers, discounts, abandoned carts, and more Multi currency support Custom order and cart fields Process refunds and send customer notifications from within the ProcessWire backend Process Abandoned Carts + sending messages to customers from within the ProcessWire backend Complete Snipcart webhooks integration (all events are hookable via ProcessWire hooks) Integrated taxes provider (which is more flexible then Snipcart own provider) Useful Links
      SnipWire in PW modules directory SnipWire Docs (please note that the documentation is a work in progress) SnipWire @GitHub (feature requests and suggestions for improvement are welcome - I also accept pull requests) Snipcart Website  
      ---- INITIAL POST FROM 2019-05-25 ----
       
    • By horst
      Croppable Image 3
      for PW 3.0.20+
      Module Version 1.2.0
      Sponsored by http://dreikon.de/, many thanks Timo & Niko!
      You can get it in the modules directory!
      Please refer to the readme on github for instructions.
       
      - + - + - + - + - + - + - + - + - + - NEWS - 2020/03/19 - + - + - + - + - + - + - + - + - + - 
      There is a new Version in the pipe, that supports WebP too: 
       
      - + - + - + - + - + - + - + - + - + - NEWS - 2020/03/19 - + - + - + - + - + - + - + - + - + - 
       
       
      -------------------------------------------------------------------------
       
      Updating from prior versions:
       
      Updating from Croppable Image 3 with versions prior to 1.1.7, please do this as a one time step:
      In the PW Admin, go to side -> modules -> new, use "install via ClassName" and use CroppableImage3 for the Module Class Name. This will update your existing CroppableImage3 module sub directory, even if it is called a new install. After that, the module will be recogniced by the PW updater module, what makes it a lot easier on further updates.
      -------------------------------------------------------------------------
       
      For updating from the legacy Thumbnail / CropImage to CroppableImage3 read on here.
       
      -------------------------------------------------------------------------
       
×
×
  • Create New...