horst

@flydev: many thanks for your contribution, I have added it to the module by following your perfect guidance step by step. Very Nice. So, I have not tested it because I have no server with a Self-Signed Certificate at hand. But I very likely trust someones code who is able to contribute PHP code backwards compatible from PHP 5.6 to PHP 3.0! $version=explode(".",function_exists("phpversion") ? phpversion() : "3.0.7");

I have had a read about the way(s) one can embed it into CSS, where base64-encoded was title as sub-optimal as SVG is textbased (XML) and not binary data. They suggested this way: background-image: url("data:image/svg+xml;utf8,%3Csvg xmlns='http%3A//www.w3.org/2000/svg' width='120px' height='120px' viewBox='0 0 379 379'%3E %3Cg transform='matrix(1.25,0,0,-1.25,40,345)' id='g10'%3E %3Cg transform='scale(0.08,0.08)' id='g12'%3E %3Cpath id='path14' style='fill%3Argb(0,0,0)' d='m 1517.6,1767.02 c 138.62,0 251.01,-112.38 251.01,-251.02 0,-138.62 -112.39,-251.01 -251.01,-251.01 -138.64,0 -251.02,112.39 -251.02,251.01 0,138.64 112.38,251.02 251.02,251.02 z m -1.59,1264.99 C 679.137,3032.01 0.644531,2353.89 0.015625,1517.16 L 0,1517.16 0,749.051 388.961,403.559 l 0,1127.031 c 0,622.48 504.602,1127.06 1127.049,1127.06 622.45,0 1127.03,-504.58 1127.03,-1127.06 0,-622.41 -504.58,-1127.031 -1127.03,-1127.031 l -206.22,0 0,552.769 206.22,0 c 317.15,0 574.32,257.102 574.32,574.262 0,317.2 -257.17,574.32 -574.32,574.32 -317.16,0 -574.291,-257.12 -574.291,-574.32 L 941.719,0 1516.01,0 c 837.26,0 1516.01,678.73 1516.01,1515.99 0,837.28 -678.75,1516.02 -1516.01,1516.02' /%3E %3C/g%3E %3C/g%3E %3C/svg%3E") --- If you like and are brave, just drop the data uri into your browsers URL bar and press enter: data:image/svg+xml;utf8,%3Csvg xmlns='http%3A//www.w3.org/2000/svg' width='120px' height='120px' viewBox='0 0 379 379'%3E %3Cg transform='matrix(1.25,0,0,-1.25,40,345)' id='g10'%3E %3Cg transform='scale(0.08,0.08)' id='g12'%3E %3Cpath id='path14' style='fill%3Argb(0,0,0)' d='m 1517.6,1767.02 c 138.62,0 251.01,-112.38 251.01,-251.02 0,-138.62 -112.39,-251.01 -251.01,-251.01 -138.64,0 -251.02,112.39 -251.02,251.01 0,138.64 112.38,251.02 251.02,251.02 z m -1.59,1264.99 C 679.137,3032.01 0.644531,2353.89 0.015625,1517.16 L 0,1517.16 0,749.051 388.961,403.559 l 0,1127.031 c 0,622.48 504.602,1127.06 1127.049,1127.06 622.45,0 1127.03,-504.58 1127.03,-1127.06 0,-622.41 -504.58,-1127.031 -1127.03,-1127.031 l -206.22,0 0,552.769 206.22,0 c 317.15,0 574.32,257.102 574.32,574.262 0,317.2 -257.17,574.32 -574.32,574.32 -317.16,0 -574.291,-257.12 -574.291,-574.32 L 941.719,0 1516.01,0 c 837.26,0 1516.01,678.73 1516.01,1515.99 0,837.28 -678.75,1516.02 -1516.01,1516.02' /%3E %3C/g%3E %3C/g%3E %3C/svg%3E

This thread is closed now. All further discussions, questions moved to the release thread under this forums module section: https://processwire.com/talk/topic/12151-wire-queue-basic-implementation-of-simple-queues/

Which version was that? I use PHPed and hadn't had that, but also aren't aware of that license "feature".

There were also a proof of concept by @netcarver here: http://modules.processwire.com/modules/crypto-ppp/ Maybe this can become useful at some point.

Thanks for the link to Anttis writeup. ------------------ Following is a description for what the SQLite handler was used in the past before I built WireQueue around it. One use case was for an award winning Cinemaportal where they send up to 2x 13k emails in different newsletters every week. I should embed the WireMailSMTP module there and I suggested that we should use SQLite files for it. This is running a half year in production now without any problems. At the beginning we have had a testing for 2 weeks where we slightly modified and tweeked some little parts. On that server where the website is hosted were running many many different programs and services, so, that for unknown reasons, sometimes the SQLite DB stopped working unexpected. But SQLite has a sort of transaction feature (Sqlite-Journal files), so that no recipient got lost. On every run we retrieve 500 pending recipients. The only thing what could happen there was, that, when immediately after successful sending a mail and the SQLite crashed before it could update the record in the dbfile AND in the journal file, this single recipient will be picked up again with the next batch, what results in sending him the newsletter twice. So, this (only) happened a few times per week, (3-10) on 10k mails, = 0,1 percent. And as this may be a bit specific to that explicit server, as I haven't encountered it with other use cases, it seems to be much more secure to use SQLite with its transaction / rollback functionality as MySQL with MyISAM! Don't know about the differences when using MySQL and InnoDB. But we / you can build storage handlers for every DB we / you like. So, what I have done with building WireQueue, is implementing it for easy usage and monitoring in PWs backend. The previous used fieldsdefinition of the SQLite table has changed from userid, email, sent, tries to data, state, worker, but everything else of the SQLite part is identical in WireQueue. Hope this helps a bit.

Your link points to a 404 for me

Hi Arjen, I haven't used IftRunner and doesn't know it. So, sorry, I cannot answer anything in regard of comparing. If you already know IftRunner and it does support what you need, I would stick with it. WireQueue is new and not (widely) tested currently. In fact, currently it is only tested by me in a few use cases. Before using it in a huge project, you definetly should do thouroughful testing. With a quick look through the repository, it seems that queuing in IftRunner is done internally, means, it is mapped to page actions, you do not have to populate it manually, but you also cannot use it externally, without page actions. But not really sure on this, - it was my first read here.

Tested: ImageAnimatedGif works well with PW 3.0+.

Successfully tested with PW 3.0 today

Successfully tested with PW 3.0 today!

This is a pure info icon. It (only) recommends you that you have $config->debug set to true! So, maybe not that useful in develoment mode, but later, if you use a small set of admin links for editors too, it is useful to be rmembered to turn out debug mode when finished something on a production site.

@BernhardB: thanks for the language explanation, I have corrected it above in my post! I haven't used MySQL tables, because my main motivation was to have a storage that not belongs to the PW-DB. I think this is useful in regard of Backup / Restore processes, if one have not mass temporary records with it. Filebased storage is portable and easy to use (TXT). SQLite is also portable but has DB support what lets you simply push, pull and update records. For example, think of sending newsletters to 5k recipients, SQLite is very convenient to handle this. Another usecase what I have had: A website collected infos over a time of two weeks and the (post) processing should be done afterwards. Here we simply have implemented a download link for the customer into the editpage, so that he could store the textfile (CSV style), locally on his computer for further processing with excel. So, I believe there will be also usecases where a MySQL table is a better storage. For all storage types that will need initial config params, I have implemented a simple Textareafield into the queue page, what can be used to take that settings. (one can use it as key = value pairs each in its own row, like in some other modules, maybe. But it is up to the developer if he like to use that simple approach or if he implements something other). The Textareafield is collapsed by default for admins, and hidden for others. (Whereas the queue pages require minimum the role "wire-queue-admin" or superuser for everything else than viewing the state or count). So, hopefully someone will make a mysql storage handler and commit it here or at github. (or another DB)

Hi and welcome 3dots. You are on the right track, but not fully. Some hints may be: To build a form for the front end, you also can use the PW API: https://processwire.com/talk/topic/2089-create-simple-forms-using-api/ How to create a page is in the DOCs or in the forums many times, I believe: https://processwire.com/talk/topic/352-creating-pages-via-api/ Follow SOmas tut with the simple forms creation and do not forgett to use the validation and sanitizer. Have a look at the Docs and at the cheatsheet, too And have fun If you have any further questions after reading and testing, come back and ask.

This has got some small fixes and updates and is officially released now: https://processwire.com/talk/topic/12151-wire-queue-basic-implementation-of-simple-queues/

Successfully tested ALIF with PW 3.0 and set the version to 1.0.0 stable now.

Successfully tested PIA with PW 3.0 and set the version to 1.0.0 stable now.

Just to clarify: the further processing does not have to be done from within Processwire, nor does it to be done by PHP. You can use PW to only collect the data, maybe in the example Sqlite storage handler. To further process the data records, you can use every software that can communicates to a Sqlite DBfile. If you need different fields in a Sqlite table, just copy the WireQueueSqlite3 module, - change the file- and class name to be no duplicates of the distributed storage handler, - and, at least, change the fielddefinitions for the DB table. But you can change every thing you want. If you want or need to use other storage types and setup a new WireQueueSomething module, - please feel free to submit / commit it.

Wire Queue Wire Queue is a module that allows easy creation and usage of Queues in ProcessWire. It is based upon a basic parent module (WireQueue) that should have one or multiple StorageHandler modules installed too. This beta release contains a simple plain text storage module, WireQueueTextfile, and a simple Sqlite3-DB storage module, WireQueueSqlite3. The base module creates the needed: FIELDS (wire_queue_type, wire_queue_state, wire_queue_storage_options) TEMPLATES (wire-queue-container, wire-queue-storage, wire-queue-tools) PAGES (wire-queues = container for all Queuepages, wire-queue-storages = container for StoragetypePages) ROLE (wire-queue-admin) Each storage module creates one page under wire-queue-storages. New Queues can be created in the backend by adding a new page under "Wire Queues". After creation one have to select a Storage type from availables list and publish the page. After that, there are some buttons available to start / pause / and close the queue. Putting and getting data to and from the queue is done via API calls. First you have to get the page that holds the queue object. // get and validate the queue handle if($queue = $pages->get(SELECTOR_TO_DESIRED_PAGE)->wireQueue()) { // wrap your data into an array and pass it into the queue $success = $queue->addItem($data); ... } // get and validate the queue handle if($queue = $pages->get(SELECTOR_TO_DESIRED_PAGE)->wireQueue()) { $data = $queue->getItem(); ... } Thats basically all what you want from a queue. Following are a few conveniences: $queue->getPage()->title gives you the title of the queue, ($queue->getPage() returns the page object) $queue->className() shows the StorageType of the queue $queue->getState() and $queue->getStateStr() returns the current state of a queue: 1 = new / empty 2 = enabled / running 3 = paused 4 = closed / archived $queue->itemCount() gives the total number of all pending items in the queue Here is code that gives an overview of all Queues in the system: $container = $pages->get('template=wire-queue-container'); $bodyContent = "<h1>$container->title</h1>"; $bodyContent .= "<p>There are currently {$container->numChildren} Queues defined:</p>"; $bodyContent .= "<ul>"; foreach($container->children() as $p) { if(! ($queue = $p->wireQueue())) continue; $bodyContent .= "<li>{$queue->getPage()->title}<ul>"; if(!$queue->ready2use()) { $bodyContent .= "<li>{$queue->className}</li>"; $bodyContent .= "<li>This Storagetype is not ready to use! The System seems not to provide all requirements.</li>"; $bodyContent .= "</ul></li>"; continue; } $bodyContent .= "<li>{$queue->className}</li>"; $bodyContent .= "<li>{$queue->getStateStr()} ({$queue->getState()})</li>"; $bodyContent .= "<li>Currently are {$queue->itemCount()} items pending!</li>"; $bodyContent .= "</ul></li>"; } $bodyContent .= "</ul>"; Following is a screenshot of the backend The module is available in the modules directory: http://modules.processwire.com/modules/wire-queue/ Or you get it from Github: https://github.com/horst-n/WireQueue. . . . The Sqlite3 storage handler not only let you push and pull data items to and from it, it also can keep track of the current state of a record. If you use multiple / different workers for pulling and processing the data, you can store an ID for them too. This is how the DB Table looks like: The Wire Queue Sqlite3 storage handler provides the methods addItem($arrayData) // same as WireQueueTextfile getItem($worker = null) // same as WireQueueTextfile, (but the textfile storage cannot support $worker!) updateItemState($id, $state, $worker = null) // this you can use for further processing, if you want . addItem($arrayData) $arrayData is a, maybe associative, array containing all your data for one record. The method stores it under the next free id under data in the Sqlite-DB-file and sets the state to 0. The field worker is empty for new added records. Following is a basic working example for pushing data records into a queue: // you have created a queue in PW, the ID of the page is 1420 for example // here is some skeleton code for an importer that uses this queue // get and validate the queue handle if(! ($queue = $pages->get('id=1420')->wireQueue())) exit(); // we could not get the Queuepage // now start to scan / read data for your imports, wrap each record into an array and put it into the queue foreach($pages->find(YOURSELECTOR) as $p) { $data = array($p->field1, $p->field2); $queue->addItem($data); } . . getItem($worker = null) $worker is an integer that you can define to suite your needs. If you don't use or don't want identify multiple workers, just ommit it. The method pulls one pending record from the queue, changes the state from 0 to 1, and returns an associative array with the keys id and data. array('id' => integer, 'data' => array) You will need the id if you further want to use the queue to keep track of processing steps. You must pull your stored $data from $array['data'] and use the id for further storing the state. . updateItemState($id, $state, $worker = null) $id identifys the record a worker has processed, for $state you can define and use every integer you like, but not 0 or 1. If you also want to store altered data, and not only different states, you can use updateItem($id, $state, $worker = null, $data = null) instead of updateItemState(). . Here is a working example with a bit pseudo code how workers can get a pending record to process it further and store back the result of the process: // you have created a queue in PW, the ID of the page is 1420 for example // here is some skeleton code for an importer that uses this queue // get and validate the queue handle if(! ($queue = $pages->get('id=1420')->wireQueue())) exit(); // we could not get the Queuepage // we do not use different workers here in that example, so we do not pass an worker id here $tmp = $queue->getItem(); // get a record from the queue $record_id = $tmp['id']; // store the record id $data = $tmp['data']; // get the data array // process the $data ... // and get the result of the process, in this example true or false $result = processMyRecord($data); // as new records have a state = 0, fetched records have a state = 1, // you may define a state of 2 for successful processed records, and higher ones for failed records $state = true === $result ? 2 : 3; $queue->updateItemState($record_id, $state); . . getItem4FurtherProcessing($state, $worker = null) The $state you pass to the method is the state you want get the record for. If there is one pending, its state will be set +1 and the id and data is passed back to you in an associative array: array('id' => integer, 'data' => array). --------- Here is a pseudo code example how (multiple) worker scripts may batch process queue records with the sqlite storage handler // on the server in this example, everyscript will timeout / die after 180 seconds // we start a timer $time = Debug::timer(); // we use different instances of workers that pull and process records from the queue, // so additionally to the processings states, we also want to store the worker ids $worker = 2000; // now start to process items by pulling one after the other from the queue while(150 > Debug::timer($time)) { $tmp = $queue->getItem($worker); // get a record from the queue if(!$tmp) continue; // maybe currently are no pending records available $record_id = $tmp['id']; // store the record id $data = $tmp['data']; // get the data array $result = processMyRecord($data); // process the data and get back the result $state = true === $result ? 2 : 3; // define an integer for the new state, according to the result of processing the data $queue->updateItemState($record_id, $state, $worker); } // we are close to the timeout for the script, so finish it with calling it itself to get a fresh run $session->redirect('./');

It is an awesome video, so, also with music. Have had a nice looking. Thanks man! Will also try the module in the next time.

I assume this will not work with a simple proxy as it is not that easy to trick google / youtube.

I'm on mobile and not answering directly to your question, but want mention that instead of the forums search box, you should use google with site:processwire.com/talk as additional search tag.

I'm really happy with NuSpheres PhpED on Windows! If you create an project it also knows more of it as you or I ever will, It has all sorts of Tooltips and Popuplists with classes, properties, functions, that also contain links to the files wheere they live, etc. It has a pane called Code Navigator which shows this list(s) of the current opened file with synchronized focus, means: it highlights that method where your cursor in the editor pane currently is. It has an own live debugger that can be embedded as a php extension in apaches webserver, also on remote machines, that can be controlled with little browser extensions or else. It supports remote editing via sftp or ftpes, ssh, terminals, it has a DB client that fastly displays information of mysql or sqlite databases and it has much more what I currently have not tested. Its memory consumption is not low but around 0.5 GB when I have opened 1-3 projects simultaneous. Downside is, it is not free. But it is worth its costs.

What do you mean with "this"?

Very nice! - ähm but unfortunately not my colors. Ough. Please can we have a less colorful theme or a theme switcher with one less colorful, once all other is running? please, please, please, ...