Module ProcessEmailToPage

[huseyin]

I recieve SPAM emails with email addresses like **name**@domain.com.

For such emails the the module throws error messeges and stops reading further. No emails after this SPAM are downloaded.

The error shown in Tracy Debugger is like this : 

Unknown Selector operator: '=**' -- was your selector value properly escaped? field='email', value='DialVision**@carsvasite.us', selector: 'email=**DialVision**@carsvasite.us, sort=sort, parent_id=29, templates_id=3' (in /wire/core/Selectors.php line 419)

How can i bypass this error...

[MarkE]

On 1/28/2020 at 3:04 PM, huseyin said:

Unknown Selector operator: '=**' -- was your selector value properly escaped?

This doesn't look like a module issue. Try using single quotes around the double asterisks: '**' not **

i.e. (in your example) email='**DialVision**@carsvasite.us'

EDIT: Correction - in my off-the-cuff response I hadn't realised that this is in the code, not set in options. See the link in the next post.

[MarkE]

It's a shame that this module is no longer supported. I have an app that uses it and it works well. I'm not sure that the use of Flourish is a problem as it only uses a few classes, which presumably could be modified if they cause issues  in the future with new PHP versions. I can't find a better alternative at the moment.

However, there are a couple of problems I have had with the module:

• html tags such as <ol> cause problems
• The additional "notify users" for categories does not work well. Firstly, any additional users in one category are applied to other categories and, secondly, only one set of From, Subject and Body is allowed.

I have attempted to fix these at https://github.com/MetaTunes/ProcessEmailToPage. Please feel free to use (and to fix any further bugs!). I'm not promising super support, but will look into any genuine bugs.

Another point to note re multiple categories - all the email addresses must use the same password as this is set at an overall level. I have not attempted to fix this.

EDIT: updated code to fix the issue reported by @huseyin - let me know if it works.

[adrian]

@MarkE - thanks for taking up the torch and trying to get some things fixed. Perhaps you could could get in touch with @Pete and get access to push changes to the main repo, or perhaps have the modules directory point to your forked version?

Again, sorry I can't keep working on it, but I've never actually used it, so that makes it hard to justify the time, not to mention that I really don't know what is and isn't needed because I don't see the problems as they arise.

[Pete]

@MarkE I may finally be using this module on a project many years after last being involved with it. Did you want access to the main repo to merge your changes into it?

My needs may be somewhat different - updating a trip diary (single page, many entries) over the course of a trip - so I'd likely be using a modified version anyway but whilst I'm looking back at this topic I thought I'd ask ? 

[Pete]

I think I've actually already spotted some workflow flaws that mean I couldn't use it for this project actually, but the offer of access/managing the repo still stands.

[adrian]

@Pete - in case it helps, I had to do something with processing incoming emails recently and given the abandones status of the flourish library, I went with pure PHP. 

$inbox = imap_open('{imap.gmail.com:993/imap/ssl/novalidate-cert}', 'me@gmail.com', 'password');
$msgnos = imap_search($inbox, 'UNSEEN');

if($msgnos) {
    foreach($msgnos as $msgno) {
        
        imap_setflag_full($inbox, $msgno, "\\Seen \\Flagged", ST_UID);
        
        $header = imap_headerinfo($inbox, $msgno);
        
        $bodyText = imap_fetchbody($inbox, $msgno, 1.2);
        if(!strlen($bodyText) > 0) {
            $bodyText = imap_fetchbody($inbox, $msgno, 1);
        }
        $timestamp = strtotime($header->date);
        $toObj = $header->to[0];
        $fromObj = $header->from[0];
        $to = $toObj->mailbox.'@'.$toObj->host;
        $from = $fromObj->mailbox.'@'.$fromObj->host;
        $u = $users->get('email='.$from);
        if($u->id) {
            // process contents $bodyText here
        }
    
    }

}

I had this all ready to use and then decided to actually switch to using Mailgun's incoming callback feature - if you have access to it, I think it's much nicer because they will ping your site via a webhook callback, rather than you needing to regularly ping your email via imap.

[MarkE]

@Pete - the module has been working trouble-free for the last 2 years. I’ve pretty much forgotten the details, but will look into it when I have a mo. Funnily enough, I may have another app when I might want to use it - but I’ll look at @adrian’s solution as well. Mailgun would be good for the first app also, but not the second, so that’s a useful lead too, thanks.  

[tires]

By visiting the hash URL to checking the mail manually i get the following error message:

Deprecated: strpos(): Non-string needles will be interpreted as strings in the future. Use an explicit chr() call to preserve the current behavior in /mydomain/mysite/site/modules/ProcessEmailToPage/flourishlib/fMailbox.php on line 505

How can i avoid this?

[adrian]

@tires - the Flourish library is no longer maintained, so this module really needs to be rebuilt so that it doesn't rely on it. It's quite easy with imap_open https://www.php.net/manual/en/function.imap-open.php but I'm afraid I really can't contribute more time to this module - I've never actually used it myself and I don't think it has a huge user base, but I am sure Pete would accept a PR if you'd be willing to implement imap_open.

 

 

[MarkE]

I use it on one site and it is quite neat, but I have yet to move that site to php8, so thanks for the heads-up @tires

[tires]

57 minutes ago, adrian said:

@tires - the Flourish library is no longer maintained, so this module really needs to be rebuilt so that it doesn't rely on it. It's quite easy with imap_open https://www.php.net/manual/en/function.imap-open.php but I'm afraid I really can't contribute more time to this module - I've never actually used it myself and I don't think it has a huge user base, but I am sure Pete would accept a PR if you'd be willing to implement imap_open.

It would be a pity if the module was not continued!
I'm afraid I can't contribute in any helpful way. Unfortunately i lack the skills ...

[adrian]

In case someone feels like contributing by replacing Flourish with imap_open, this code should get you started:

$inbox = imap_open('{imap.gmail.com:993/imap/ssl/novalidate-cert}', 'address@domain.com', 'password');
$msgnos = imap_search($inbox, 'UNSEEN');

if($msgnos) {
    
   foreach($msgnos as $msgno) {

       imap_setflag_full($inbox, $msgno, "\\Seen \\Flagged", ST_UID);

       $header = imap_headerinfo($inbox, $msgno);

       $bodyText = imap_fetchbody($inbox, $msgno, 1.2);
       if(!strlen($bodyText) > 0) {
           $bodyText = imap_fetchbody($inbox, $msgno, 1);
       }
       $timestamp = strtotime($header->date);
       $toObj = $header->to[0];
       $fromObj = $header->from[0];
       $to = $toObj->mailbox.'@'.$toObj->host;
       $from = $fromObj->mailbox.'@'.$fromObj->host;

   }

}

 

[MarkE]

On 10/18/2022 at 9:58 PM, MarkE said:

I use it on one site and it is quite neat, but I have yet to move that site to php8

I've now moved that site to php 8 and the module has stopped working. Any insights would be great. Meanwhile I will try and diagnose.

[MarkE]

1 hour ago, MarkE said:

Meanwhile I will try and diagnose.

The problem (or, at least, on of them) is line 505 of fMailbox.php. Since php 8.0.0, providing a non-string as a needle to strpos is no longer supported. $filter has a default value of null. 

(strpos($header, $filter) !== FALSE) evaluates to true and should be replaced. The following works in my case, but further fixes may be required if the $filter is actually set.

if ($filter && strpos($header, $filter) !== FALSE)

 

[adrian]

@MarkE - I really do think migration to use imap_open as per my above post would be a better option given the status of the flourish library.

[MarkE]

Just now, adrian said:

I really do think migration to use imap_open as per my above post would be a better option given the status of the flourish library.

You are almost certainly right. I needed a quick fix! I'm using POP3 at the moment. I'll try your suggestion when I have a bit more time. Ta.

[MarkE]

Hi @adrian, @Pete & @tires and anyone else interested in this module. As I said above, I was using this on one site and am now looking at using it more widely as it is very useful for sending emails to a group: the email is saved to a page then forwarded (via LazyCron) to a list, a record thus being maintained of all such emails.

Generally it has worked well without the suggested imap upgrade. However, I recently encountered a problem. The email import is triggered by a hook on Page render. My new pagebuilder module (not released) uses multiple PW pages per displayed page. I think the rapid-fire execution caused the emails to be picked up multiple times without the server deleting them. I tried replacing the hook, without success, with a LazyCron hook. Instead I have just added a session var so that the import cannot happen more than once a minute.

I'm afraid my technical understanding of the cause of this problem is (very) limited, but I wonder whether it would also be an issue with imap...

[MarkE]

Another issue for anyone else using this module: Line 625 of ProcessEmailToPage.module has 

$attachment['filename'] = $this->sanitizer->pageName($attachment['filename'], true);

I don't know why the pageName sanitizer is being used to sanitize a file name, but this gives an incorrect result if the file name has brackets in it. In my version, I have changed pageName to fileName.