Module ProcessEmailToPage

[martind]

Yes, I've tried it without ==blog== written in also hoping it would just write the the body field but nothing. It's very odd, it successfully brings the title across. And as the title is coming across it would make sense that the connection to the mail server is successful. The mail server is on a separate box but connection appears to succeed.

I did have the module stored in the root of the modules directory, but since then I've uninstalled the module, moved the files to /site/modules/ProcessEmailToPage and reinstalled.

Is there anything else I can try as I really like the idea of the module.

had the same issue. it works if you make following change in ProcessEmailToPage.php on line 271:

//$message['html'] = str_replace(' ', '', $message['html']);
// change to
$message['html'] = str_replace(' ', '', $message['text']);

[davo]

had the same issue. it works if you make following change in ProcessEmailToPage.php on line 271:

//$message['html'] = str_replace(' ', '', $message['html']);
// change to
$message['html'] = str_replace(' ', '', $message['text']);

Bingo! Yes it does indeed. Thank you. I'm assuming this is because my email was encoded as plain text and not html

[adrian]

Hi Pete,

Just playing with this module for the first time - looks fantastic - thank you.

Just a couple of things:

1. there are lots of undefined variable and undefined index notices on the module config page that would be nice to have cleaned up.
    
2. You still have "live" in the js file. I like the approach that Wanze took with live vs on in the batcher module depending on the version of jquery.
    
3. Also, the "Add another category" button seems to be missing - I am guessing this is a class issue with Ryan's new default admin theme that I am testing it with.

Thanks a lot for looking into these.

[martind]

is this great module working ok with the latest dev version 2.3.11 with default admin-theme?

the problem i see is, that on module-admin page following fields can`t get saved: "publishDefault":"","emailAddress":"","emailCategory":0,"emailTemplate":"","categoryData":""

[martind]

ok, for my case it was possible to fix it by adding the categoryData array manually to the modules json via mysql. something like

[{\"emailAddress\":\"my@me.com\",\"emailCategory\":\"1013\",\"emailTemplate\":\"46\"}]

[itsberni]

Thanks for this great module, pete.

Unfortunately i've got some problems with the setup.

In my testenviroment ( pw 2.4 ) there is no possibility to save email categories. The button "add new category" simply lacks.

Could you give me any advice, what i supposed to do.

Thanks a lot!

[adrian]

I have attached a revised version of the module. I know Pete is pretty busy with other things at the moment, so I thought I'd try to help out.

I haven't tested this thoroughly, but I think it should take care of things. It also includes a few other fixes, like adding delete buttons for each category and also fixing a bunch of PHP warnings.

If you could test and let me know if everything is working fine and then I'll submit a pull request for Pete.

My forked version

[itsberni]

Thanks adrian,

now the button is visible. I can setup an email-categorie but now a server error occurs, when i´m calling the Cron-Job url

Error: Exception: There was an error connecting to the server (in /kunden/...xxx....site/modules/ProcessEmailToPage/flourishlib/fMailbox.php line 981)

I´ve tried serveral email-server setup combinations without any success ( imap/pop3 over secure or unsecured ports - 143 or 995 ).

Are there anny known fixes or functioning setups?

[adrian]

Glad that sorted out the category adding issue. I'll submit that pull request to Pete.

As for the server connection issue - I am really not sure, but I don't think it is related to this module. Perhaps there are some issues with the Flourish library (included in this module) and your pop/imap server, or a firewall issue.

Maybe someone who has been using the module more might have a better idea.

[adrian]

Ok, I decided to do some testing at my end on the connection issue. I was trying with a gmail account using imap and found I needed to force a secure connection:

http://flourishlib.com/docs/fMailbox

It seems to me like this module should have an config option to define this true/false, but for the moment I just changed this line in the .module file from:

$mailbox = new fMailbox($this->emailType, $this->emailHost, $category->emailAddress, $this->emailPassword, $this->emailPort);

to:

$mailbox = new fMailbox($this->emailType, $this->emailHost, $category->emailAddress, $this->emailPassword, NULL, TRUE);

I was having connection or username/password issues until I made this change, but now it works perfectly and pages are created as expected.

Can you let me know if that works for you and if it does I'll add a config setting for it and add that to the pull request to Pete.

[adrian]

Just added a new forceSecure option to the config. Also made sure that if port is left blank, that it sends NULL, rather than blank which seems to be a requirement.

Also added a fix for the HTML vs text email issue reported by @martind

All these will go to another PR for Pete in a few minutes.

Please let me know if the attached version works for you now.

My forked version

[itsberni]

adrian - sorry for my delayed answer. the time delay...i had to sleep ;-)

great job, thank you - i think it seems to work now. but i have to continue testing, because there is still an error occurring after the cron-call:

Error: Call to a member function add() on a non-object (line 345 of /kunden/.../pw/site/modules/ProcessEmailToPage/ProcessEmailToPage.module)

i think the reason for this could be an incorrect setup of the template ( missing/wrong fields ?! ).

my email contains only the subject and a jpg-file. after the cron-call the jpg has been downloaded in pw-folder ( root ). from the subject the module builds a child of the category i´ve chosen in the modules - setup ( as it should, i think ). but although i added an image field to the template, the image won´t appear in the "category". there is only the title ( email-subject ).

[adrian]

I am curious as to the name of your images field. That error you are getting makes me think you don't have a field called "images". The module currently has the field hardcoded as "images" so clearly that needs to be made flexible so that other fields can be used. Not sure the best approach for this though. If there is only one image field in the selected template, it's easy, but if there is more than one we'd have to either just choose the first, or ask what field to use in the config settings, which might be best. Also, if there is more than one image attached, and the chosen field is set to only store one image, then there is also an issue. Also I see that there isn't anything to determine if the attachment is in fact an image which is something that should also be checked, and if it's not should the file be added to a files field on the page?

Anyway, please check that your images field is in fact named "images" and see if that helps. I am curious about nothing showing up in the body of your page though. Do you have a "body" field in the selected template? Sorry, I see that you didn't enter any text in the body of the email, so this is expected. Regardless, I think this is something else that needs to be made configurable - at the moment the body field must be named "body" as this is the hardcoded name.

EDIT: I haven't narrowed down why yet, but if I send the email from Mac Mail, no attachments get converted to images, but if I send them from gmail from within the web interface, everything works perfectly - maybe there is some limitation with flourish and its ability to detect attachments in some case, or maybe it is the crazy way Macs attach files

ANOTHER UPDATE: I just sent an email with an attached image from MS Outlook and it also works perfectly, so I am beginning to think there really is something amiss with attachments and the Flourish/Mac Mail combination. Can anyone else confirm this with the version of the module attached to my previous message?

AND ANOTHER: I have added support for inline attachments and this seems to be helping with Mac Mail. I had also been using this setting:

defaults write com.apple.mail DisableInlineAttachmentViewing -bool yes

on my Mac, but reversing that (change yes to false), along with the new inline support and images are now being added. The one outstanding problem I can see at the moment is that if a Mac Mail message also has text in it, the image won't work, so test without anything but the image for now if you are using Mac Mail.  Turns out this needed "related", not "inline". I have added support for this in the new version attached this morning.

One other thing I think might be a nice addition to this module is support for embedding images into the body field if they were embedded inline in the email. I need sleep now, but might look into this tomorrow!

My forked version

Edited September 20, 2014 by adrian

[adrian]

Ok, I have just added support for images that are embedded into the text within an email. These images are now added to the page's images field and then embedded into the body RTE field, just as they appeared in the email.

Minimal testing so far, but seems to work great even with multiple images. Would appreciate some testing of the attached version before I submit to Pete.

My forked version

ProcessEmailToPage.zip

[itsberni]

great job, adrian!

i´ve tested it with an attached file and as an embedded image. both works ( tested with outlook for mac and iPhone ).

but let me know your thoughts about embedding the image into the body. in my opinion ( when the image was sent embedded ) the image has to be treated separately ( over the field images e.g. ), then the developer can decide what happens with the image in the FE.

what do you think?

Another question: when the mail was sent over iphone there is same strange behaviour with the folder ( assets/files/1283/ ) the image is saved in.

you can see it in the attaced pic.

[adrian]

Thanks for the feedback. I may not have time to look into the ios issue today, but will get to it as soon as I can. Could you please let me know the actual filename of that image in the 1283 folder?

Regarding the behaviour of images and whether they should automatically end up in the body field or not. My thought was that if they were attached images, they should just end up in the images field and not embedded into the body field, but if they were embedded in the email text (html), then they should be embedded in the body field, just as they were in the email. Perhaps I could add an option to control this behaviour. The complication to all this seems to be that Mac Mail always sends images inline within the message body, which being relatively new to Mac, is still a mystery to me. I think we can end up with something pretty robust here, but it might take some testing with different email clients and some tweaking to get everything always working as expected.

[Pete]

Thanks for all the updates adrian - I've commited the latest updates on Github and am looking forward to your latest addition for embedded images.

I wonder if it might be worth making the embedded images feature optional though? I'm thinking of a case where this could be used in a company where people have images in their email signatures and these would come through as well - just a thought.

[adrian]

Hey Pete - no problem. Thanks for committing those initial bug fixes. 

Good point about the email signatures for sure. The issue I have is how Mac Mail seems to embed all attachments. Maybe someone else can enlighten me as to firstly why this is a good idea and how we can use flourish to determine the difference between signature images which should really be html with a linked image, vs an image that was attached to the message. Maybe it's a matter of playing around more with flourish's inline vs related options. I'll look into it more later.

[itsberni]

Adrian.

There´re two images saved in this folder 1283. One named foto.jpg and one thumb named foto.0x100.jpg

[adrian]

Good to know. So that image named "1283" in the images folder is something extra, rather than an image that you emailed that was somehow corrupted then?

[itsberni]

no - there was only one foto submitted. that "image" is no image. i think it´s some kind of waste due to the email-sending process ;-)

[adrian]

Ok, this is a really quickly assembled update. It adds an option to determine whether embedded images will be added to the body field of the page or not. This seems to work great but would appreciate you checking it, especially from different devices / email clients. At the moment this setting is site-wide, but it might potentially make more sense to have it for each email address / category.

The other change will hopefully fix that ghost image you had, although I haven't been able to replicate it at my end, so haven't really been able to test. Please let me know if it fixes it for you.

My forked version

[Pete]

Cheers adrian - I'll check it out later unless someone beats me to it.

The problem I found is that even just between Thunderbird and Outlook signatures are treated a bit differently and there's no code in the source of the email to say when a signature starts in an email unfortunately. I honestly don't think that until someone big enough like Google (joining forces with Microsoft and possibly supermom) redesigns how emails should work from the ground up we'll ever see a satisfactory standard for handling email signatures

[adrian]

Hey Pete - agreed signatures are currently horrible, especially when people insist on embedding images in their signature, rather than writing them in HTML with a linked image, often resulting in every email from that person having the image show up as an attachment depending on the combination of the sending and receiving email client and settings.

My biggest beef is Mac Mail's displaying of images/pdfs inline in the body even if they were added as attachments. Just my opinion, but I think Outlook and others got this right, but I am ranting now

[mr-fan]

First thank you very much for this!

Read the whole treat....may i've missed it.

get a connection error for wrong PW and Username....

All settings are done so far - where i have to set the username? this one is missing to set since it is not the emailadress it is like PW: mypass USER: webxxxp1

regards mr-fan

***Realy great module if it works - it would complete/extend my Recipe HTML App for my wife to collect there cooking power digital