Jump to content
horst

Wire Queue, basic implementation of simple queues

Recommended Posts

Wire Queue

Wire Queue is a module that allows easy creation and usage of Queues in ProcessWire.

It is based upon a basic parent module (WireQueue) that should have one or multiple StorageHandler modules installed too. This beta release contains a simple plain text storage module, WireQueueTextfile, and a simple Sqlite3-DB storage module, WireQueueSqlite3.

The base module creates the needed:

  • FIELDS (wire_queue_type, wire_queue_state, wire_queue_storage_options)
  • TEMPLATES (wire-queue-container, wire-queue-storage, wire-queue-tools)
  • PAGES (wire-queues = container for all Queuepages, wire-queue-storages = container for StoragetypePages)
  • ROLE (wire-queue-admin)

Each storage module creates one page under wire-queue-storages.

New Queues can be created in the backend by adding a new page under "Wire Queues". After creation one have to select a Storage type from availables list and publish the page. After that, there are some buttons available to start / pause / and close the queue.

Putting and getting data to and from the queue is done via API calls. First you have to get the page that holds the queue object.

// get and validate the queue handle
if($queue = $pages->get(SELECTOR_TO_DESIRED_PAGE)->wireQueue()) {
    // wrap your data into an array and pass it into the queue
    $success = $queue->addItem($data);
    ...
}
// get and validate the queue handle
if($queue = $pages->get(SELECTOR_TO_DESIRED_PAGE)->wireQueue()) {
    $data = $queue->getItem();
    ...
}

Thats basically all what you want from a queue. Following are a few conveniences:

  • $queue->getPage()->title gives you the title of the queue, ($queue->getPage() returns the page object)
  • $queue->className() shows the StorageType of the queue
  • $queue->getState() and $queue->getStateStr() returns the current state of a queue:
    1 = new / empty
    2 = enabled / running
    3 = paused
    4 = closed / archived
  • $queue->itemCount() gives the total number of all pending items in the queue

Here is code that gives an overview of all Queues in the system:

$container = $pages->get('template=wire-queue-container');
$bodyContent = "<h1>$container->title</h1>";
$bodyContent .= "<p>There are currently {$container->numChildren} Queues defined:</p>";
$bodyContent .= "<ul>";
foreach($container->children() as $p) {
    if(! ($queue = $p->wireQueue())) continue;
    $bodyContent .= "<li>{$queue->getPage()->title}<ul>";
    if(!$queue->ready2use()) {
        $bodyContent .= "<li>{$queue->className}</li>";
        $bodyContent .= "<li>This Storagetype is not ready to use! The System seems not to provide all requirements.</li>";
        $bodyContent .= "</ul></li>";
        continue;
    }
    $bodyContent .= "<li>{$queue->className}</li>";
    $bodyContent .= "<li>{$queue->getStateStr()} ({$queue->getState()})</li>";
    $bodyContent .= "<li>Currently are {$queue->itemCount()} items pending!</li>";
    $bodyContent .= "</ul></li>";
}
$bodyContent .= "</ul>";

Following is a screenshot of the backend

wire-queue-01.gif

The module is available in the modules directory: http://modules.processwire.com/modules/wire-queue/
Or you get it from Github: https://github.com/horst-n/WireQueue.
.
.
.
The Sqlite3 storage handler not only let you push and pull data items to and from it, it also can keep track of the current state of a record. If you use multiple / different workers for pulling and processing the data, you can store an ID for them too.
 
This is how the DB Table looks like:
pw_wirequeue_sqlite3_db-screen.jpg
 
The Wire Queue Sqlite3 storage handler provides the methods

  • addItem($arrayData)      // same as WireQueueTextfile
  • getItem($worker = null)   // same as WireQueueTextfile, (but the textfile storage cannot support $worker!)
  • updateItemState($id, $state, $worker = null)  // this you can use for further processing, if you want

 .
addItem($arrayData)
 
$arrayData is a, maybe associative, array containing all your data for one record. The method stores it under the next free id under data in the Sqlite-DB-file and sets the state to 0. The field worker is empty for new added records.

Following is a basic working example for pushing data records into a queue:

// you have created a queue in PW, the ID of the page is 1420 for example
// here is some skeleton code for an importer that uses this queue

// get and validate the queue handle
if(! ($queue = $pages->get('id=1420')->wireQueue())) exit(); // we could not get the Queuepage

// now start to scan / read data for your imports, wrap each record into an array and put it into the queue
foreach($pages->find(YOURSELECTOR) as $p) {
    $data = array($p->field1, $p->field2);
    $queue->addItem($data);
}

.

.

getItem($worker = null)
 
$worker is an integer that you can define to suite your needs. If you don't use or don't want identify multiple workers, just ommit it. The method pulls one pending record from the queue, changes the state from 0 to 1, and returns an associative array with the keys id and data. array('id' => integer, 'data' => array)
You will need the id if you further want to use the queue to keep track of processing steps.
You must pull your stored $data from $array['data'] and use the id for further storing the state.

.

updateItemState($id, $state, $worker = null)
 
$id identifys the record a worker has processed, for $state you can define and use every integer you like, but not 0 or 1. If you also want to store altered data, and not only different states, you can use updateItem($id, $state, $worker = null, $data = null) instead of updateItemState().

.

Here is a working example with a bit pseudo code how workers can get a pending record to process it further and store back the result of the process:

// you have created a queue in PW, the ID of the page is 1420 for example
// here is some skeleton code for an importer that uses this queue

// get and validate the queue handle
if(! ($queue = $pages->get('id=1420')->wireQueue())) exit(); // we could not get the Queuepage

// we do not use different workers here in that example, so we do not pass an worker id here
$tmp = $queue->getItem();   // get a record from the queue
$record_id = $tmp['id'];    // store the record id
$data = $tmp['data'];       // get the data array

// process the $data ... 
// and get the result of the process, in this example true or false
$result = processMyRecord($data);

// as new records have a state = 0, fetched records have a state = 1, 
// you may define a state of 2 for successful processed records, and higher ones for failed records
$state = true === $result ? 2 : 3;
$queue->updateItemState($record_id, $state);

.

.

getItem4FurtherProcessing($state, $worker = null)
 
The $state you pass to the method is the state you want get the record for. If there is one pending, its state will be set +1 and the id and data is passed back to you in an associative array: array('id' => integer, 'data' => array).

---------

Here is a pseudo code example how (multiple) worker scripts may batch process queue records with the sqlite storage handler

// on the server in this example, everyscript will timeout / die after 180 seconds
// we start a timer
$time = Debug::timer();

// we use different instances of workers that pull and process records from the queue, 
// so additionally to the processings states, we also want to store the worker ids
$worker = 2000;

// now start to process items by pulling one after the other from the queue
while(150 > Debug::timer($time)) {
    $tmp = $queue->getItem($worker);   // get a record from the queue
    if(!$tmp) continue;                // maybe currently are no pending records available
    $record_id = $tmp['id'];           // store the record id
    $data = $tmp['data'];              // get the data array
    $result = processMyRecord($data);  // process the data and get back the result
    $state = true === $result ? 2 : 3; // define an integer for the new state, according to the result of processing the data
    $queue->updateItemState($record_id, $state, $worker);
}

// we are close to the timeout for the script, so finish it with calling it itself to get a fresh run
$session->redirect('./');
  • Like 10

Share this post


Link to post
Share on other sites

Just to clarify: the further processing does not have to be done from within Processwire, nor does it to be done by PHP. You can use PW to only collect the data, maybe in the example Sqlite storage handler. To further process the data records, you can use every software that can communicates to a Sqlite DBfile.

If you need different fields in a Sqlite table, just copy the WireQueueSqlite3 module, - change the file- and class name to be no duplicates of the distributed storage handler, - and, at least, change the fielddefinitions for the DB table. But you can change every thing you want.

If you want or need to use other storage types and setup a new WireQueueSomething module, - please feel free to submit / commit it. :)

Share this post


Link to post
Share on other sites

hi horst,

thank you for your module. could be that i need it one day, though i have to say it was all new to me and your explanation in the other thread was helpful for me: https://processwire.com/talk/topic/11927-wirequeue-needs-testing/?p=111287 :)

thank you also for the screencasts that helped me a lot understanding what it is all about.

the further processing must not be done from within Processwire...

i think you mean "does not have to be done" as "must not be done" = "darf nicht innerhalb pw gemacht werden", right?

could you please explain why you used TXT/SQLite for data storage? why not use a mysql table from within pw? thank you :)

Share this post


Link to post
Share on other sites

@BernhardB: thanks for the language explanation, I have corrected it above in my post! :)

I haven't used MySQL tables, because my main motivation was to have a storage that not belongs to the PW-DB. I think this is useful in regard of Backup / Restore processes, if one have not mass temporary records with it. Filebased storage is portable and easy to use (TXT). SQLite is also portable but has DB support what lets you simply push, pull and update records. For example, think of sending newsletters to 5k recipients, SQLite is very convenient to handle this.

Another usecase what I have had: A website collected infos over a time of two weeks and the (post) processing should be done afterwards. Here we simply have implemented a download link for the customer into the editpage, so that he could store the textfile (CSV style), locally on his computer for further processing with excel.

So, I believe there will be also usecases where a MySQL table is a better storage. For all storage types that will need initial config params, I have implemented a simple Textareafield into the queue page, what can be used to take that settings. (one can use it as key = value pairs each in its own row, like in some other modules, maybe. But it is up to the developer if he like to use that simple approach or if he implements something other). The Textareafield is collapsed by default for admins, and hidden for others. (Whereas the queue pages require minimum the role "wire-queue-admin" or superuser for everything else than viewing the state or count).

So, hopefully someone will make a mysql storage handler and commit it here or at github. :)

(or another DB)

Edited by horst
  • Like 1

Share this post


Link to post
Share on other sites

Hey Horst, 

Looks really impressive, but I'm wondering how this module differs from the queuing done in IftRunner combined with Page Actions. I'm comparing both at the moment for a rather large project we are working on. One thing I see it that you have different queues whereas IftRunner only has one queue to store them all. Any other differences that you are aware of?

Share this post


Link to post
Share on other sites

Hi Arjen,

I haven't used IftRunner and doesn't know it. So, sorry, I cannot answer anything in regard of comparing. If you already know IftRunner and it does support what you need, I would stick with it. WireQueue is new and not (widely) tested currently. In fact, currently it is only tested by me in a few use cases. Before using it in a huge project, you definetly should do thouroughful testing.

With a quick look through the repository, it seems that queuing in IftRunner is done internally, means, it is mapped to page actions, you do not have to populate it manually, but you also cannot use it externally, without page actions. But not really sure on this, - it was my first read here.

Share this post


Link to post
Share on other sites

Thanks for responding. We are going to test it thoroughly. That's why I'm looking for all options. Pity you haven't used it yet. IftRunner with (Page) Actions are really simple as a concept, but quite powerful. Antti wrote about their use case a while ago.

  • Like 2

Share this post


Link to post
Share on other sites

...  Pity you haven't used it yet. IftRunner with (Page) Actions are really simple as a concept, but quite powerful. Antti wrote about their use case a while ago.

Thanks for the link to Anttis writeup.

------------------

Following is a description for what the SQLite handler was used in the past before I built WireQueue around it.

One use case was for an award winning Cinemaportal where they send up to 2x 13k emails in different newsletters every week. I should embed the WireMailSMTP module there and I suggested that we should use SQLite files for it.

This is running a half year in production now without any problems. At the beginning we have had a testing for 2 weeks where we slightly modified and tweeked some little parts. On that server where the website is hosted were running many many different programs and services, so, that for unknown reasons, sometimes the SQLite DB stopped working unexpected. But SQLite has a sort of transaction feature (Sqlite-Journal files), so that no recipient got lost. On every run we retrieve 500 pending recipients.

The only thing what could happen there was, that, when immediately after successful sending a mail and the SQLite crashed before it could update the record in the dbfile AND in the journal file, this single recipient will be picked up again with the next batch, what results in sending him the newsletter twice. So, this (only) happened a few times per week, (3-10) on 10k mails, = 0,1 percent. And as this may be a bit specific to that explicit server, as I haven't encountered it with other use cases, it seems to be much more secure to use SQLite with its transaction / rollback functionality as MySQL with MyISAM! Don't know about the differences when using MySQL and InnoDB. But we / you can build storage handlers for every DB we / you like. :)

So, what I have done with building WireQueue, is implementing it for easy usage and monitoring in PWs backend. post-1041-0-54734100-1455109217_thumb.pn

The previous used fieldsdefinition of the SQLite table has changed from userid, email, sent, tries to data, state, worker, but everything else of the SQLite part is identical in WireQueue.

Hope this helps a bit.

Edited by horst
added info about use cases for SQLite
  • Like 3

Share this post


Link to post
Share on other sites

hi horst,

doing first tests with your module: how can i re-open a queue after closing/archiving it?

thank you

Share this post


Link to post
Share on other sites

With the UI, you cannot! Archiveing means what it says. :)

If you want to stop using it, with the possibility to reuse it later, please use pause.

I'm not sure on this, as it isn't meant to do this by design, but you may try to cahnge the fieldvalue by API:

// get the desired queue page by its id
$queue = $pages->get("id=1234");

// get the fieldname that stores the state
$fieldname = WireQueue::WIRE_QUEUE_FIELD2;

// change value from 4 (archived) to 3 (paused), or 2 (active)
$queue->$fieldname = 3; // or 2
$queue->save();

BUT: Without warranty. I'm not sure if this has any sideeffects, its not tested!

Share this post


Link to post
Share on other sites
$pages->get(SELECTOR_TO_DESIRED_PAGE)->WireQueue() 

You have typos in your first two examples: it should be wireQueue and not WireQueue

ok no problem, but i think there should at least be a warning that this is an irreversible action :)

Share this post


Link to post
Share on other sites
$pages->get(SELECTOR_TO_DESIRED_PAGE)->WireQueue() 

You have typos in your first two examples: it should be wireQueue and not WireQueue

ok no problem, but i think there should at least be a warning that this is an irreversible action :)

Thanks for pointing to the typos. I corrected it.

  • Like 1

Share this post


Link to post
Share on other sites

hi horst,

i need a little more help, sorry. i don't really get the concept. for example when i have a queue to resize images, how would i setup this? when i have a queue "resizeimages" i could add all resize requests to this queue and then run the resize for every item. so far so good, but what if i have different users with images from different pages? user A adds 100 images to queue "resizeimages" and user B adds 100 images to the same queue. the queue now has 200 images in it if that happens at the same time. how does the queue know which images belong to wich user?

the "worker" is empty for new entries so i can't use this. or should i do an additem() and right after that an updateitemstate(...worker...)?

or am i completely on the wrong track with what i want to do?

thank you

edit: ok i don't think that would work because there would still be all items in the queue and there's no kind of "selector" to get only the items from one worker, right? so would i have to create a queue for every page where i want to resize images? i didn't try it so far, but i think that sould be possible via standard api creating new pages and setting the appropriate values?

Share this post


Link to post
Share on other sites

Sorry, I cannot follow. A queue only stores the data you put into it, nothing more. If you need references to whatever, put it into your data. That's the only way to have your needed data together. But this has nothing to do with the queue. Once again, the queue is nothing else than a storage, a DB table, with exactly 1 field for your data. Therefore, (I think you do this already), you have to wrap your data into an array. If you need more data-information as you currently have in it, just put it into the data array too. :)

Share this post


Link to post
Share on other sites

hmmm, ok i see you really didn't get me :) i was writing a more detailed answer but while i was writing i found a better solution for my usecase. i think i wanted to use the queue in a way it is not built for (but not the way you understood me, but thats no factor any more) ;)

Share this post


Link to post
Share on other sites

hi horst, me again ;)

just came across https://processwire.com/api/variables/log/ and as i read the docs i thought it is quite similar to txt-queues. i think one could easily create simple queues this way and would be interested in your opinion.

creating a new queue would be as simple as $log->save('examplequeue', 'mydata');

fetching lines from the end of the file is also easy (http://kongondo.github.io/ProcessWireAPIGen/master/source-class-WireLog.html#196-229) and could maybe modified by ryan with an option go get lines from the beginning of the file and not only the end. this functionality is already built into the filelog class http://kongondo.github.io/ProcessWireAPIGen/master/source-class-FileLog.html#164-216

only thing missing would be a method to delete rows from the beginning or end of the file, but that would also be really easy.

edit: don't know about the "workers", didn't get the concept...

Share this post


Link to post
Share on other sites

edit: don't know about the "workers", didn't get the concept...

Workers are separate processes, which run alongside your webserver stack on the host system. They can process things they get out of the queue without being reliant on the Request->Process->Response cycle of the webserver.

E.g. You want to have a way for users to request a big report on your website. As a big report sounds like, it take some time to generate the pdf for it (say ~1 Min). You wouldn't want your user to sit around and watch the nice browser spinner, waiting for the next page to pop up. You can now write the request in a queue and instantly return a message stating that the report will be generated and emailed to the user when ready.

To generate that report you may know the option of using a cron to trigger website logic, without an actual user requesting a website, but that's also flawed with things like timeouts and such, also it's only starting in intervals. That's where workers come in.

These are php (and other script) processes, which often run as services like e.g. your apache server does. These scripts contain some way to not exit early (e.g. infinite while loop) – which is the biggest difference to website request – and periodically check their queue if there's work to do. If your report request is in it, they generate the pdf, email it and if done delete the queue item. If there are other reports to do they'll move on to them, otherwise they sleep until another queue item is available. 

  • Like 5

Share this post


Link to post
Share on other sites

@horst

Thanks for this module, I've been hammering it pretty hard for the past month and have just created a pull-request over on github for your consideration. I've been having a few reliability problems with the textfile driver's itemCount() method. I also wanted a fast way to purge a queue without having to call getItem() repeatedly.

The itemCount() method sometimes returns an incorrect count (anywhere from 0 to the true number of items in the queue). This happens even when there are multiple thousands of jobs queued up and it makes it unreliable as part of a conditional statement. The explanation is given in the commit that fixes the issue as part of the pull request.

Thanks for looking!

  • Like 6

Share this post


Link to post
Share on other sites

Many thanks @netcarver.

I must admit that I have written the Textfile-driver only for this module to have a basic driver. Other as the SQLite-driver, it has not have much testing. I'm glad to hear that you have done this and you contribute to the module. I have you added as colaborator to it, so that you can change / add other improvements / enhancements as you like. ^-^;)

  • Like 3

Share this post


Link to post
Share on other sites

Thanks for adding me as a colaborator, @horst.

I've pushed a new dev branch to the repository that contains,

  • Bugfixes to the textfile driver's itemCount() method and small cleanups for edge cases.
  • New methods isEmpty(), purgeItems() and getItems() added to the base storage class and fully implemented in the textfile driver. The SQLite driver currently has stub implementations of these that need filling out.
  • The addition of a Redis driver for super-fast in-memory queues (implements the full base class interface.)

If anyone wants to give this a try before horst gets around to merging into the master branch, feel free to grab the zip (repository here.)

  • Like 5

Share this post


Link to post
Share on other sites

Horst and I would like to announce the release of v1.0.0 of the WireQueue suite of modules.

There are now drivers for textfiles, redis and SQLite and the interface has been extended to include the isEmpty(), purgeItems() and getItems() methods.

Enjoy!

  • Like 12

Share this post


Link to post
Share on other sites

I think I may have found a small problem with WireQueueTextfile, although it could be due to this PHP bug?

The array is being serialized in the addItem() function, but in my case an element's value is sometimes a string containing newline characters "\r\n". I believe WireQueueTextfile saves each array to one line, but these characters are causing the serialized string to be split onto multiple lines, which I'm assuming causes problems for WireQueue.

Would it make sense to somehow sanitize these values (outside of WireQueue) before adding them to the queue?

Share this post


Link to post
Share on other sites

Could you try replacing addItem() and getItem() with the following...

    public function addItem($arrayData) {
        if(!$this->_addItem()) return false;
        if(2 != $this->getState()) return false;
        if(!$fp = @fopen($this->getFilename(), 'ab')) return false;
        if(flock($fp, LOCK_EX)) {
            $arrayData = serialize($arrayData);
            $data = str_replace("\r\n", '\r\n', $arrayData) . "\n";
            $res = fwrite($fp, $data);
            fflush($fp);
            flock($fp, LOCK_UN);
            fclose($fp);
            return $res == strlen($data);
        }
        fclose($fp);
        return false;
    }

    public function getItem($worker = null) {
        if(!$this->_getItem()) return false;
        if(2 != $this->getState()) return false;
        if(!$fp = @fopen($this->getFilename(), 'rb+')) return false;
        if(flock($fp, LOCK_EX)) {
            $line = trim(fgets($fp));
            if(!$line) {
                flock($fp, LOCK_UN);
                fclose($fp);
                if(0 == $this->itemCount()) return null;
                return false;
            }
            // we have the first entry, now write all following data into a buffer
            $fpTmp = @fopen('php://temp/maxmemory:' . intval(1024 * 1024 * 5), 'rb+');
            while(!feof($fp)) fwrite($fpTmp, fread($fp, 4096));
            fseek($fp, 0, SEEK_SET);
            ftruncate($fp, 0);
            fseek($fpTmp, 0, SEEK_SET);
            // write back buffer into file
            while(!feof($fpTmp)) fwrite($fp, fread($fpTmp, 4096));
            fclose($fpTmp);
            fflush($fp);
            flock($fp, LOCK_UN);
            fclose($fp);
        }
        return unserialize(str_replace('\r\n', "\r\n", $line));
    }

Untested - YMMV!

  • Like 2

Share this post


Link to post
Share on other sites

No luck with that unfortunately, still lots of fragmented lines. Do the instances of '\r\n'  need to be wrapped in double quotes? I can't work out the logic here!

I should also note that these strings can include \r\n but are also sometimes littered with instances of \r or \n individually.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By d'Hinnisdaël
      Happy new year, everybody 🥬
      I've been sitting on this Dashboard module I made for a client and finally came around to cleaning it up and releasing it to the wider public. This is how it looks.
      ProcessWire Dashboard

      If anyone is interested in trying this out, please go ahead! I'd love to get some feedback on it. If this proves useful and survives some real-world testing, I'll add this to the module directory.
      Download
      You can find the latest release on Github.
      Documentation
      Check out the documentation to get started. This is where you'll find information about included panel types and configuration options.
      Custom Panels
      My goal was to make it really simple to create custom panels. The easiest way to do that is to use the panel type template and have it render a file in your templates folder. This might be enough for 80% of all use cases. For anything more complex (FormBuilder submissions? Comments? Live chat?), you can add new panel types by creating modules that extend the DashboardPanel base class. Check out the documentation on custom panels or take a look at the HelloWorld panel to get started. I'm happy to merge any user-created modules into the main repo if they might be useful to more than a few people.
       Disclaimer
      This is a pre-release version. Please treat it as such — don't install it on production sites. Just making sure 🍇
      Roadmap
      These are the things I'm looking to implement myself at some point. The wishlist is a lot longer, but those are the 80/20 items that I probably won't regret spending time on.
      Improve documentation & add examples ⚙️ Panel types Google Analytics ⚙️ Add new page  🔥 Drafts 🔥 At a glance / Page counter 404s  Layout options Render multiple tabs per panel panel groups with heading and spacing between ✅ panel wrappers as grid item (e.g. stacked notices) ✅ Admin themes support AdminThemeReno and AdminThemeDefault ✅ Shortcuts panel add a table layout with icon, title & summary ✅ Chart panel add default styles for common chart types ✅ load chart data from JS file (currently passed as PHP array) Collection panel support image columns ✅ add buttons: view all & add new ✅
    • By Pip
      Hi everyone!
      I'm trying out the Login/Register module for my site. Noted that the module assigns the newly registered user to login-register role. 
      Once you modify the login-register role's permissions, particularly adding page-edit, the new member role will be set to guest. 
      Thing is I'd like to grant my new users the power to create their own pages. Any advice? 
      Thanks. 
    • By Gadgetto
      SnipWire - Snipcart integration for ProcessWire
      Snipcart is a powerful 3rd party, developer-first HTML/JavaScript shopping cart platform. SnipWire is the missing link between Snipcart and the content management framework ProcessWire.
      With SnipWire, you can quickly turn any ProcessWire site into a Snipcart online shop. The SnipWire plugin helps you to get your store up and running in no time. Detailed knowledge of the Snipcart system is not required.
      SnipWire is free and open source licensed under Mozilla Public License 2.0! A lot of work and effort has gone into development. It would be nice if you could donate an amount to support further development:

      Status update links (inside this thread) for SnipWire development
      2020-07-03 -- SnipWire 0.8.7 (beta) released! Fixes some small bugs and adds an indicator for TEST mode 2020-04-06 -- SnipWire 0.8.6 (beta) released! Adds support for Snipcart subscriptions and also fixes some problems 2020-03-21 -- SnipWire 0.8.5 (beta) released! Improves SnipWires webhooks interface and provides some other fixes and additions 2020-03-03 -- SnipWire 0.8.4 (beta) released! Improves compatibility for Windows based Systems. 2020-03-01 -- SnipWire 0.8.3 (beta) released! The installation and uninstallation process has been heavily revised. 2020-02-08 -- SnipWire 0.8.2 (beta) released! Added a feature to change the cart and catalogue currency by GET, POST or SESSION param 2020-02-03 -- SnipWire 0.8.1 (beta) released! All custom classes moved into their own namespaces. 2020-02-01 -- SnipWire is now available via ProcessWire's module directory! 2020-01-30 -- SnipWire 0.8.0 (beta) first public release! (module just submitted to the PW modules directory) 2020-01-28 -- added Custom Order Fields feature (first SnipWire release version is near!) 2020-01-21 -- Snipcart v3 - when will the new cart system be implemented? 2020-01-19 -- integrated taxes provider finished (+ very flexible shipping taxes handling) 2020-01-14 -- new date range picker, discount editor, order notifiactions, order statuses, and more ... 2019-11-15 -- orders filter, order details, download + resend invoices, refunds 2019-10-18 -- list filters, REST API improvements, new docs platform, and more ... 2019-08-08 -- dashboard interface, currency selector, managing Orders, Customers and Products, Added a WireTabs, refinded caching behavior 2019-06-15 -- taxes provider, shop templates update, multiCURL implementation, and more ... 2019-06-02 -- FieldtypeSnipWireTaxSelector 2019-05-25 -- SnipWire will be free and open source Plugin Key Features
      Fast and simple store setup Full integration of the Snipcart dashboard into the ProcessWire backend (no need to leave the ProcessWire admin area) Browse and manage orders, customers, discounts, abandoned carts, and more Multi currency support Custom order and cart fields Process refunds and send customer notifications from within the ProcessWire backend Process Abandoned Carts + sending messages to customers from within the ProcessWire backend Complete Snipcart webhooks integration (all events are hookable via ProcessWire hooks) Integrated taxes provider (which is more flexible then Snipcart own provider) Useful Links
      SnipWire in PW modules directory SnipWire Docs (please note that the documentation is a work in progress) SnipWire @GitHub (feature requests and suggestions for improvement are welcome - I also accept pull requests) Snipcart Website  

       
      ---- INITIAL POST FROM 2019-05-25 ----
       
    • By Sten
      Hello
      Till now I hacked something with the twig template but it works no more with new PW versions so I look forward to create a module. I am working on a site in multiple languages : French, English, Italian, German, Spanish, Portuguese, Hebrew, Russian. The new posts are entered in any language with a field for language. Till now, I got twig files to get the translations with constants defined for each part of the pages.
      So I'd like to create a module to include theses files added according to the url /fr/en/...
      Have you some observations to do before I begin about the direction to take ?
      Thank you
    • By ukyo
      Mystique Module for ProcessWire CMS/CMF
      Github repo : https://github.com/trk/Mystique
      Mystique module allow you to create dynamic fields and store dynamic fields data on database by using a config file.
      Requirements
      ProcessWire 3.0 or newer PHP 7.0 or newer FieldtypeMystique InputfieldMystique Installation
      Install the module from the modules directory:
      Via Composer:
      composer require trk/mystique Via git clone:
      cd your-processwire-project-folder/ cd site/modules/ git clone https://github.com/trk/Mystique.git Module in live reaction with your Mystique config file
      This mean if you remove a field from your config file, field will be removed from edit screen. As you see on youtube video.
      Using Mystique with your module or use different configs path, autoload need to be true for modules
      Default configs path is site/templates/configs/, and your config file name need to start with Mystique. and need to end with .php extension.
      Adding custom path not supporting anymore !
      // Add your custom path inside your module class`init` function, didn't tested outside public function init() { $path = __DIR__ . DIRECTORY_SEPARATOR . 'configs' . DIRECTORY_SEPARATOR; Mystique::add($path); } Mystique module will search site/modules/**/configs/Mystique.*.php and site/templates/Mystique.*.php paths for Mystique config files.
      All config files need to return a PHP ARRAY like examples.
      Usage almost same with ProcessWire Inputfield Api, only difference is set and showIf usage like on example.
      <?php namespace ProcessWire; /** * Resource : testing-mystique */ return [ 'title' => __('Testing Mystique'), 'fields' => [ 'text_field' => [ 'label' => __('You can use short named types'), 'description' => __('In file showIf working like example'), 'notes' => __('Also you can use $input->set() method'), 'type' => 'text', 'showIf' => [ 'another_text' => "=''" ], 'set' => [ 'showCount' => InputfieldText::showCountChars, 'maxlength' => 255 ], 'attr' => [ 'attr-foo' => 'bar', 'attr-bar' => 'foo' ] ], 'another_text' => [ 'label' => __('Another text field (default type is text)') ] ] ]; Example:
      site/templates/configs/Mystique.seo-fields.php <?php namespace ProcessWire; /** * Resource : seo-fields */ return [ 'title' => __('Seo fields'), 'fields' => [ 'window_title' => [ 'label' => __('Window title'), 'type' => Mystique::TEXT, // or InputfieldText 'useLanguages' => true, 'attr' => [ 'placeholder' => __('Enter a window title') ] ], 'navigation_title' => [ 'label' => __('Navigation title'), 'type' => Mystique::TEXT, // or InputfieldText 'useLanguages' => true, 'showIf' => [ 'window_title' => "!=''" ], 'attr' => [ 'placeholder' => __('Enter a navigation title') ] ], 'description' => [ 'label' => __('Description for search engines'), 'type' => Mystique::TEXTAREA, 'useLanguages' => true ], 'page_tpye' => [ 'label' => __('Type'), 'type' => Mystique::SELECT, 'options' => [ 'basic' => __('Basic page'), 'gallery' => __('Gallery'), 'blog' => __('Blog') ] ], 'show_on_nav' => [ 'label' => __('Display this page on navigation'), 'type' => Mystique::CHECKBOX ] ] ]; Searching data on Mystique field is limited. Because, Mystique saving data to database in json format. When you make search for Mystique field, operator not important. Operator will be changed with %= operator.
      Search example
      $navigationPages = pages()->find('my_mystique_field.show_on_nav=1'); $navigationPages = pages()->find('my_mystique_field.page_tpye=gallery');
×
×
  • Create New...