Module Wire Queue, basic implementation of simple queues

[netcarver]

1 hour ago, ZGD said:

Do the instances of '\r\n'  need to be wrapped in double quotes? I can't work out the logic here!

I don't think so, no. The idea above was to replace any embedded ASCII carriage-return+linefeed pairs with a literal sequence '\' + 'r' + '\' + 'n' on save of the data. On reading the data the, now-literal, sequence is turned back into an embedded ASCII pair.

If the ASCII linefeed and return characters can occur anywhere - not just next to each other - then you need to replace them individually. Try doing this instead...

protected function encodeData($data) {
    $data = serialize($data);
    $data = str_replace("\r", '\r', $data);
    $data = str_replace("\n", '\n', $data);
    return $data;
}

protected function decodeData($data) {
    $data = str_replace('\r', "\r", $data);
    $data = str_replace('\n', "\n", $data);
    $data = unserialize($data);
    return $data;
}

public function addItem($arrayData) {
    if(!$this->_addItem()) return false;
    if(2 != $this->getState()) return false;
    if(!$fp = @fopen($this->getFilename(), 'ab')) return false;
    if(flock($fp, LOCK_EX)) {
        $data = $this->encodeData($arrayData) . "\n";
        $res = fwrite($fp, $data);
        fflush($fp);
        flock($fp, LOCK_UN);
        fclose($fp);
        return $res == strlen($data);
    }
    fclose($fp);
    return false;
}

public function getItem($worker = null) {
    if(!$this->_getItem()) return false;
    if(2 != $this->getState()) return false;
    if(!$fp = @fopen($this->getFilename(), 'rb+')) return false;
    if(flock($fp, LOCK_EX)) {
        $line = trim(fgets($fp));
        if(!$line) {
            flock($fp, LOCK_UN);
            fclose($fp);
            if(0 == $this->itemCount()) return null;
            return false;
        }
        // we have the first entry, now write all following data into a buffer
        $fpTmp = @fopen('php://temp/maxmemory:' . intval(1024 * 1024 * 5), 'rb+');
        while(!feof($fp)) fwrite($fpTmp, fread($fp, 4096));
        fseek($fp, 0, SEEK_SET);
        ftruncate($fp, 0);
        fseek($fpTmp, 0, SEEK_SET);
        // write back buffer into file
        while(!feof($fpTmp)) fwrite($fp, fread($fpTmp, 4096));
        fclose($fpTmp);
        fflush($fp);
        flock($fp, LOCK_UN);
        fclose($fp);
    }
    return $this->decodeData($line);
}

Sorry, I don't really have time to test this out - you'll have to play with the encoding. You may find that doing the serialisation at the end of the encode (and unserialisation at the start of decode) might do it.

HTH

[ZGD]

Ah I understand now, I hadn't realized single quotes were used for literal strings. All makes sense and the above code seems to do the trick, thanks once again!

[netcarver]

@ZGD

The above way of encoding/decoding is only meant as an example - it will put carriage-returns and linefeeds anywhere there is a literal '\r' or '\n' in the data. So, if you have anything like a Windows path (C:\regarding\my\new\nested\filesystem) you could get 

C:
egarding\my
ew
ested\filesystem

out.

You will need to pick substitution tokens (particularly for the "\n" newlines) that will not be in your data stream. I think you will get away without substituting the "\r" carriage returns.

[Peejay]

Hi, 

I want to test this module but I have multiple errors with an install on the latest master version of Processwire (3.0.61):

• https://github.com/horst-n/WireQueue/issues/2 (the solution of @netcarver worked!)
• Next I have an error on line 533 in de WireQueue.module ( Call to undefined function wireMkdir()) : I changed it "wireMkdir()" to mkdir()
• When I want to make a Queue and I select a wire_queue_type and save it I get this error:
   Error saving field "wire_queue_type" - Page 1031 is not valid for wire_queue_type (Page 1031 does not have required parent 1016)

Can someone help me with this, or is there a new version on his way?