Module ProcessEmailToPage

[tires]

Great Module!

But i have problems with the {pagetitle} in the "notify subject" field, even in the latest version.
When i recieve the notify email the word "{pagetitle}" itself is in the subject.
The same shortcode in the Notify Body works fine.

By the way, is there a way to use other shortcodes here and in the Notify Body such as {pagelink}, {author}, {publishdate} and so on?

 

 

 

[adrian]

7 minutes ago, tires said:

When i recieve the notify email the word "{pagetitle}" itself is in the subject.

Try the new version just committed.

7 minutes ago, tires said:

By the way, is there a way to use other shortcodes here and in the Notify Body such as {pagelink}, {author}, {publishdate} and so on?

Not currently, but it should be pretty easy to implement - do you feel like having a go at enhancing the replaceShortcodes() method to handle this? PRs greatly appreciated!

[tires]

Sorry for the late answer!

THANK YOU for the reply an debugging of the module!

On 7/2/2019 at 5:51 PM, adrian said:

Not currently, but it should be pretty easy to implement - do you feel like having a go at enhancing the replaceShortcodes() method to handle this? PRs greatly appreciated!

I will take a look  ...

 

[tires]

I got a new problem with the new version.

One of the mails on the server creates a new article each time the mails are downloaded.
Because of a cronjob there are now a bunch of articles with the same content.

How is the way the module looks if there is already an article with this title etc.?
Or is there a way to delete the mails after download?

Any ideas?

[adrian]

@tires - it should already delete messages after the page has been created: https://github.com/Notanotherdotcom/ProcessEmailToPage/blob/548c9bff1d40aee7bf049032f78b301602517772/ProcessEmailToPage.module#L716

Perhaps you can debug why this isn't working?

[tires]

I just see when i am logged in there is an error message:

Quote

Error: Call to a member function is() on null (line 724 of /abc/xyz/web/site/modules/ProcessEmailToPage/ProcessEmailToPage.module)

This error message was shown because: you are logged in as a Superuser. Error has been logged.

In line 724 in this file there is this code
 

$published = $page->is(Page::statusUnpublished) ? 'unpublished' : 'published'; 

What else can i do to debug?

By the way, these are my module settings:

[adrian]

@tires - that error should be fixed in the latest version - not sure if that helps your issue with deletion of the email / duplicate copies of page creation though.

[tires]

Ok, it is finally working with the Version 1.3.3!

But there is an interesting bug: if the from email contains a period like "my.mail@email.com" the module don't save the mail as a page!
It takes me a few minutes to find it out.

 

 

[adrian]

@tires - could you please investigate where those "period containing" email addresses get ignored? Is the email being read, but not processed, or is it not read at all? Does it help if the "Only Users" checkbox is unchecked?

[tires]

@adrian I can see the "period containing" mails on the mailserver, they being read and deleted with the cronjob. But they don't appear as page.

I tried every setting. It neither work with "whitelist" nor with "only users" ...

 

 

[adrian]

@tires - I just tested here with a period containing email address and it worked as expected.

I think you'll need to debug at your end. Start at this line: https://github.com/Notanotherdotcom/ProcessEmailToPage/blob/6b192fc6e01e50217e8be2b0e48c8745824630ed/ProcessEmailToPage.module#L470 and make sure the message is in that array. Then go inside the foreach and figure out what rule in there is preventing the message from creating a new page which happens here: https://github.com/Notanotherdotcom/ProcessEmailToPage/blob/6b192fc6e01e50217e8be2b0e48c8745824630ed/ProcessEmailToPage.module#L522

Sorry, I think Pete is no longer supporting this module and I don't have much time to help out anymore either I'm afraid.

Reality is that the flourish library that this module uses is also abandoned, so this module really needs to be rewritten with an alternate mail library. That said, it still works for me, so you should be able to figure out where the problem is that is affecting you.

[tires]

2 hours ago, adrian said:

 - I just tested here with a period containing email address and it worked as expected.

This is really strange!
I sent my (exactly same) test mail from the same email provider one from the "non period containing email" without any problems and one from the "period containing email" with the described errors.

I dont know, if i am able to debug the module.
But i will have a look.

[tires]

Ok, i compared the sourcecode of the "period containing" and the "non period containing" emails more closer.

It seemed to be some "X-Spam" infos in the mail head that might be the problem.

I also tried a nother "period containing" mailaddress without any problems.

[tires]

Still some problems.

Some mails from the same sender are saved as page so are not!

The only difference i can see in the sourcecode of the mail is the "x-spam-status".

This is not saved as page:

X-Spam-Status: No, hits=-6.6 required=9999.0 tests=BAYES_00 autolearn=disabled
	version=3.4.0

This is saved as page:

X-Spam-Status: No, hits=-1.9 required=9999.0 tests=BAYES_20 autolearn=disabled
	version=3.4.0

Strange, isn't it?

[tires]

Little addition for the module.

I wanted to add a link to the saved article in the Notify mail.

So i changed:

211 'notes'=>"Possible shortcodes:\n{editlink} to insert a link to edit the page\n{pagelink} to insert a link to the page\n{pagetitle} for the title of the new page\n{publishstatus} for 'unpublished' or 'published'"),

and added:

723 $text = str_replace('{pagelink}', $page->httpUrl, $text);

 

PS. There are to many "s" in all "Posssible"s in the module ...

[dab]

I'm having problems adding a second category to check for emails & save to pages no PW errors (or php errors).

On "save" no new Email Category is added.

Anyone else encountering difficulties? I'm using ProcessEmailToPage v. 1.3.3, PW 3.0.123 & php PHP 7.2.

Thanks Nick

[formulate]

Not sure if this is dead or if anyone is still working on it. I have a problem where emails sent from Outlook only work about 20% of the time. The other 80% the module just leaves sitting on the mail server untouched. I can't seem to find any reason why. If I send the exact same email from a non-outlook client, the module picks it up no problem. Any ideas why this may be the case?

[adrian]

@formulate - I think Pete ran out of steam on this years ago, which is when I picked it up, but I haven't actually ever used it myself so my motivation to continue is not really there I'm afraid. The module really needs a complete overhaul and most importantly needs a replacement for the flourish library which is no longer maintained.

Do you feel like investigating alternatives for Flourish and perhaps taking over the module?

[formulate]

Rather than take over, I'd probably just do something from scratch. However, I'm extremely short on time at the moment. Heck, I don't even have time to be trying to debug this issue my client is having. Embarrassingly, at the moment I'm just processing his emails by hand as it's faster for me to just manually input his 2 or 3 emails a week than it is to try and fix this thing. Obviously not a long-term solution, but something I can afford to do with my limited time.

[MarkE]

Hi @formulate - I'm using the module, without any problems, from Outlook and other email clients. So I'm not sure it's anything about Outlook per se. Have you checked what are the restrictions on sending addresses?

[formulate]

On 11/26/2019 at 1:33 AM, MarkE said:

Hi @formulate - I'm using the module, without any problems, from Outlook and other email clients. So I'm not sure it's anything about Outlook per se. Have you checked what are the restrictions on sending addresses?

No restrictions on the address (I've experimented a lot with the module restrictions) and I have it whitelisted in the SPAM filter. It's weird that from that same address, some emails work fine and some don't. I would expect that it would be all or nothing.

At any rate, I may work on switching the module over from the Flourish library to the newer/better/maintained Fetch library.

Another approach I'm toying with is to just have the email address pipe to a script on the server that bootstraps PW's index.php and simply creates the page. No need to mess around with checking an email account, etc.

[tires]

I got the error

Fehler: Exception: Can’t save page 0: /artikel/: It has an empty 'name' field (in /xyz/wire/core/PagesEditor.php line 424)

#0 /xyz/wire/core/Pages.php(417): ProcessWire\PagesEditor->save(Object(ProcessWire\Page), Array)
#1 /xyz/wire/core/Wire.php(386): ProcessWire\Pages->___save(Object(ProcessWire\Page), Array)
#2 /xyz/wire/core/WireHooks.php(723): ProcessWire\Wire->_callMethod('___save', Array)
#3 /xyz/wire/core/Wire.php(442): ProcessWire\WireHooks->runHooks(Object(ProcessWire\Pages), 'save', Array)
#4 /xyz/wire/core/Page.php(2584): ProcessWire\Wire->__call('save', Array)
#5 /xyz/site/modules/ProcessEmailToPage/ProcessEmailToPage.module(619): ProcessWire\Page->save(NULL)
#6 /xyz/wire/ 

I think it is because i tried to load an email with arabic characters in the subject.

But even after i deleted this mail the error ist displayed ...

Any ideas?
Thanks!

[tires]

What can i do?
The website isn't working for days now ...

 

[adrian]

@tires - sorry this module really isn't seeing much love these days. Pete is long gone and I have never needed to use it so it's hard for me to continue supporting it, especially given that it really needs a new mail library among other things. To get your site working again, I think you probably just need to delete the email with the arabic characters in the subject that is causing the problem. If that's not available, I think you should be ok.

It probably shouldn't be too hard to fix the bug though either but we'd need to know what $message['headers']['subject'] is returning - actually, dumping $arr just after line 536 would be best. 

[tires]

Damn it!
It was just a problem with the email account!

I only got access to the mail account with a webmailer and the mail was just flagged as "deleted" but really removed from the server.
Now roundcube deleted it ... and everything is working!