newsletter system with PW

[diogo]

I want to collect some emails with a very simple form (we want to collect only the email) for a future newsletter. I was configuring Mailchimp because i thought it would make things easier for me and the client... it doesn't.

I could just collect the emails directly to the database, but since i'm building the site on PW i thought i could use it's powers to do it. The person that will manage this is not tech savvy at all, and would be nice to keep everything inside the admin area and using the same kind of logic.

i thought of to ways:

1. creating users with the role guest (or equivalent, newsletter for instance) and a random password that won't be revealed to them.

2. creating a page "emails", with children pages where each one is an email. I would create this pages through the API every time the form is submitted.

I'm tending for the second choice and the plan is:

1. collect the email from the input field. Sanitize it, and create a page with the email as title. The name of the page could be the email without the @ for instance.

2. confirm if the page is there with $pages->find("title=$email").

3. if yes, display confirmation and email confirmation to the user using php mail().

this is only for collecting. I still have to think how i can make the newsletter also in PW.

It makes sense to go this way? Or am I complicating?

[apeisa]

Just my 2 cents quickly from mobile: newsletters are very complicated if done well. I would definitely go with mailchimp or madmimi etc and integrate to their api.

[diogo]

Thanks Apeisa! When configuring mailchimp it was annoying me that it was sending way to many confirmation and thank you emails for a simple email subscription, I designed and customized all the messages though. The reason why I gave up on from it, was that, apparently, you can't customize the error messages, and they are automatically translated and full of errors. I will try madmimi later.

For now I'm quite happy with my efforts to collect the emails, here is what I have:

            <?php echo $pages->get(1011)->text;  
               if($input->post->submit){ 
                   $email = $sanitizer->email($input->post->email);
                   if(!$email) {
                       echo "<p>" .$input->post->email . " is not a valid email</p>";
                   }
                   elseif($pages->get("title=$email") instanceof NullPage) {
                       $p = new Page();
                       $p->template   = $templates->get("person");
                       $p->parent     = $pages->get(1015);
                       $p->title      = $email;
                       $p->save(); 
                       echo "<p>your email was saved </p>";
                   }
                   else {
                       echo "<p>" . $email . " exists already in our database</p>";
                   } 
               } else { ?> 
               <?php echo $pages->get(1011)->emailform; ?>
           <!-- Begin Signup Form -->
               <form action="<?php echo $page->url; ?>" method="post" id="signup">
                   <input type="text" name="email" class="email" id="email" />
                   <br />
                   <input type="submit" value="senden" name="submit" class="send" />
               </form>
           <!--End signup form-->
           <?php }; ?>

Besides this I'm validating the email with jquery before submiting. It's working, but am I doing something unsafe or it's ok?

[apeisa]

Code looks fine for me. You are using sanitizer and checking for duplicates.

When configuring mailchimp it was annoying me that it was sending way to many confirmation and thank you emails for a simple email subscription, I designed and customized all the messages though. The reason why I gave up on from it, was that, apparently, you can't customize the error messages, and they are automatically translated and full of errors.

There might be double opt-in required when using their forms. But if you use API or even manual importing of emails, then those emails are not required.But for the hard part, like crafting and sending newsletters, handling bounces & unsubscribes etc I would definitely use 3rd party service. One great one to add to that list is Campaign Monitor.

All those three have an API, so you can integrate it much more deeply into your PW site if you want to (and have time).

[diogo]

Thanks again Apeisa. I will add this one to my list.

[vknt]

I use Mailchimp for newsletters. As Apeisa said, when using the API, confirmation mails are not necessary.

I save the users in PW and sync them to Mailchimp. This is the piece of code I use:

<?php
        /**
        This Example shows how to run a Batch Subscribe on a List using the MCAPI.php 
        class and do some basic error checking or handle the return values.
        **/
require_once 'libs/mailchimp/MCAPI.class.php';
require_once 'libs/mailchimp/config.inc.php'; //contains apikey
        
        $api = new MCAPI($apikey);
        
        foreach ($page->children as $subscriber) {
		$name = $subscriber->title;
		$email = $subscriber->visitor_email;
		$batch[] = array('EMAIL'=>$email, 'NAME'=>$name);
	}
              
        $optin = false; //no, don't send optin emails
        $up_exist = true; // yes, update currently subscribed users
        $replace_int = false; // no, add interest, don't replace
        
        $vals = $api->listBatchSubscribe($listId,$batch,$optin, $up_exist, $replace_int);
        
        if ($api->errorCode){
            echo "Batch Subscribe failed!\n";
        	echo "code:".$api->errorCode."\n";
        	echo "msg :".$api->errorMessage."\n";
        } else {
        	echo "added:   ".$vals['add_count']."\n";
        	echo "updated: ".$vals['update_count']."\n";
        	echo "errors:  ".$vals['error_count']."\n";
        	foreach($vals['errors'] as $val){
        		echo $val['email_address']. " failed\n";
        		echo "code:".$val['code']."\n";
        		echo "msg :".$val['message']."\n";
        	}}
        
        ?>

[diogo]

Thanks a lot vknt sounds like a good choice. I will finish my subscribing process, and after I will try your code.

From where do you call this file?

[vknt]

Sorry for the late answer, I'm calling the code from within the template file of a page that I use to sync to mailchimp among other things, but you could have it in the page template of you form. Doesn't really matter. As long as you make sure to include the necessary classes and fill in the API key.

[diogo]

I want to share with you guys the final solution that I implemented for this newsletter.

So, I decided not to use any system like Mailchimp or Campaign Monitor (thanks for these and other suggestions, anyway). I wanted an extremely simple system, and decided to built it with PW instead.

Collecting the emails is done in a very simple way: when the user inputs the email, it is sanitized and, if it doesn't exist, a new page with the email as title is created under a parent called "people". Like this I get a nice list of emails in the page tree. After this, the user gets a success message, and is sent a welcome email.

$email = $sanitizer->email($input->post->email);
  if(!$email) {
  echo $pages->get(1023)->not_email;
  }
  elseif($pages->get("title=$email") instanceof NullPage) {
	 $p = new Page();
	 $p->template = $templates->get("person");
	 $p->parent = $pages->get(1015);
	 $p->title = $email;
	 $p->save();

	 echo $pages->get(1023)->confirm;
		//send the email
		$to = $email;
		$subject = 'Willkommen';
		$message = <<<EOT ...(here goes the html of the welcome email)...

This email (and also the future ones) has a link for an unsubscribing page that looks for the imputed email and deletes it:

$pages->get("title=$email")->delete();
  if($pages->get("title=$email") instanceof NullPage) {
  echo $page->byebye;
  } else {
  echo $page->error;
  }

The newsletter itself is also very simple. In this case we decided that periodically, the administrator will choose some of the posts from the blog to build the emails. So, I created a page "messages" that accepts children pages with a template that has only a date field and a pages select field. To send a message, the administrator creates a new page under "messages", chooses the blog posts that wants to include, publishes it and presses view. Here she is presented with a sample of how the email looks like, and a button "send" on the bottom. The email is sent, and she is redirected to the PW admin with a success message.

Again, I kept it very simple because we wanted limited functionality, and maybe it needs some adjustments. But It could be much more complete, with the alternative to write a text and put images instead of only choosing the blog posts, for instance. And maybe even turned into a module.

[ryan]

Looks like a great solution, thanks for posting Diogo. A couple of suggestions I'd have would be:

1. For all your $pages->get("title=$email"), I'd suggest making them as specific as possible since they can be initiated by a user. Specifically, I'd make it "title=$email, template=person, parent=/subscribers/". While not technically necessary, it just puts in another layer of security at very little expense. Who knows, maybe you'll use email addresses for titles somewhere else?

2. You may want to consider changing your $page->delete() to a $pages->trash($page), just to give you the opportunity to review who unsubscribed before emptying.

[diogo]

Thanks for the tips Ryan! I will make those changes

edit: hm, I think I won't implement the second tip. For a matter of privacy, I think an unsubscribed email should be immediately deleted.

[ffub]

I have thought a bit about newsletter integration and how an emailer setup would work from within ProcessWire, either by building something natively or integrating with Campaign Monitor / Mailchimp.

I have a couple of sites for organisations with medium-sized memberships. These need to email their members from time to time so I built a dead simple module to do this which supports a single email template. You can take a look here. Please feel free to build upon this, or just take the basic idea and run with it.

https://github.com/ffub/ProcessMailer

These are the benefits I can see from building a complete web mailer in PW:

• Ease of use
  
  • My clients already know how to use the PW backend
    
  • All of them find Campaign Monitor and MailChimp very hard to understand and use, especially the latter
    
  • Content management all in one place. They can manage their web presence all from the backend of their site
    

  [*]Tight integration with user database. You can easily create a mailing lists with PW’s awesome selector engine, eg: $users->find(“roles=5810, newsletters=news|offers”)

  [*]Create email templates based on the PW templating and field system

  • Multiple email templates
    
  • Familiar API
    
  • Keep your templates alongside your front end templates
    
  • Use templates for system messages
    

  [*]Because sent mails would be stored in the system as Pages and the templates would be on the server,

These are the negatives

• Your server will get blacklisted! Preventing users from marking your email as spam is a pretty herculean task. If you use the web server the site is hosted on to send the email then a blacklisting will also affect the system messages send out by all the sites on that server.
  
• You can work around this somewhat by allowing the web mailer to use a separate SMTP host in its settings. Hiring an SMTP host for bulk mailing will typically cost you more than just using Campaign Monitor or MailChimp. Hosts such as Gmail will only let you send up to 500 messages and will frown upon sending bulk mailings through their service.
  
• Performance: Sending multiple HTML emails can be very slow. I’m not sure how much offsetting the sending to an SMTP server improves this, but it’s definitely a concern on a web server.
  
• Bounces: You would have to setup the mailer on your server to route these to ProcessWire. Even then, it’s quite a job working out which are actual bounces and which are Out of Office messages or genuine replies.
  
• Metrics: Email analytics is a task best suited to the big boys.
  

How would you build an emailer in processwire?

I think it would actually be pretty simple to build a pretty powerful Mailer.

• Lists would be created by choosing a role, or using a selector for more advanced ones.
  
• Templates would be stored alongside regular page templates and you’d use the familiar $page API within them. Eg $page->subject. The rich text editor field might need some trickery to make sure all links and image paths are absolute URLs including the hostname.
  
• Campaigns or Messages would be created as pages. You could have draft and sent statuses or parents. It would be possible to view them on the front end too.
  
• Your email templates could easily pull in data via the Page field type, allowing you to sent out emails for events and the like with ease.
  
• Text versions could either be based on an alternative template (multiple files per template may be tricky) that strips out any HTML or by running the HTML template through Lynx on the server, which makes for a unix/cygwin dependency. If you used Textile or Markdown for your body field rather than TinyMCE this would be less of an issue to work around.
  
• An analytics field in the settings would let you enter a Google Analytics ID. Links could then be tracked with Google's campaign tracking
  

I’m certainly keen to look into the templating side of this over Spring as I’d like to use HTML templates to send some of my notification messages.

thanks,

Stephen

[Pete]

Just a very quick one from me on the subject - I do love the idea of pulling together emails using content from other pages and templates - simply have a basic template with an introduction field, then a latest news PageField that can select multiple pages and use their contnt, then maybe another text box and so on... you're limited only by your imagination here!

What I wouldn't then do is send the email from my own server, due to the blacklisting reasons ffub mentions above. It's a right pain in the arse to get yourself de-listed and all it takes is for a few users to click "Junk" instead of "Trash" (that was an issue years ago so I hope most free email providers have clarified their terminology by now) and you'll be fighting an uphill battle in no time, which is never fun.

What I would do personally is use PW to pull together the content and then use a service that allows you to import contents via the API. I know MailChimp can do that and with a basic shell template in the dedicated email service, I don't see why you shouldn't be able to then just inject the HTML you've created from your template in PW into something like MailChimp.

Then you have the best of both worlds.

And a slight headache from programming a module to shunt the parsed email contents over to the email service, but worth it in the long run

You could really go to town with the programming if you wanted to and simply never have your user see the 3rd party interface - I know the MailChimp API lets you pull subscriber lists and stats, and activity reports etc - I find their interface confusing so it would be great to just pull in the information you need.

And all of this under its own tab in the PW admin.

Well, I can dream can't I?

I do like the idea of sending email and handling it all from within PW but I've just had too many bad experiences with subscribers accidentally/intentionally getting a server blacklisted.

[ffub]

You're right, Pete. I think splitting this into a few modules which handle different levels of integration with those third party services would be most sensible.

• A basic email templating module that adds support for creating HTML emails from PW using templates. This would provide a simple API for both sending these emails from other modules and templates and for retrieving their HTML output.
  
• A list manager for creating mailing lists based on criteria such as certain fields, a selector on $users or roles. Each list could have some associated settings such as default from address, etc. The lists are synced with either CM or MailChimp. The module would need support for websockets to receive unsubscribe requests from those services.
  
• A webmailer module that allows you to create emails. This would require the two above modules and feed the HTML for the emails to CM or MailChimp.
  

I'll have think about the first of these over the weekend.

Stephen

[Christophe]

Hello,

There seems to be so many topics about newsletters that I don't really know where to post.

I was thinking of perhaps using the Newsletter plugin with Seo Panel one day, for some particular use cases.

I've just found mailerlite, while reading a blog article/interview. It seems to be already integrated with some well-known and less known CMSs.

It could be interesting to also have something like:

"MailerLite plugin for ... is here. Download it and receive a 50% discount for subscription. Boost your email marketing!"

Just wanted to share what I've found . Their services are apparently very affordable.

Updates:

https://www.mailerlite.com/international

http://docs.mailerlite.com/

[Christophe]

Updates: a "Forever Free Plan" and affordable "Small Business Plans" now -> www.mailerlite.com/pricing

And tinyletter.com and sendy.co

[seddass]

I don't have good memories with sendy.co. It covers the basics needs in affordable price but it is not reliable enough. Just my two cents. 

[videokid]

I have good experiences with sendy.co...

I use the api to subscribe users & email creation goes automatic for one site I did... I'm pretty happy with the result... 

BTW sendy 2.x has a template option now, which is a big step forward.

[Frank Vèssia]

I personally use Horst module wireMailSMTP configured with Amazon SES and a simple code + cronjob.
 

<?php
// some config variables

//these two variables should match the cronjob setting.
$hour_interval  = 9; // interval time in hours the cronjob is working, ex.: from 14:00 to 23:00
$cron_minute    = 5; // cronjob execution every tot minute.

$daily_sends    = ($hour_interval * 60) / $cron_minute;
$max_days       = 5; // for how long I want to send the newsletter in a week ? max 7 days

// this variable tells me how many times I have to send the newsletter based on the previous variables.
$total_sends    = $daily_sends * $max_days;

$total_users    = $users->find("privacy_newsletter=1,roles!=1016,limit=2")->getTotal();

// this limit tells me how many users I have to select per send to complete all the newsletter in the interval I configured previously ($max_days + cronjob settings)
$limit          = round( $total_users / $total_sends );
 
/********  INIT THE MAIL **********/
 
$mail = wireMail();
 
/********  IF ALL CICLES ARE COMPLETED RESET THE MAIL SENT LOG **********/

// 'cicles' field is an integer incremented every newsletter send
if($page->cicles >= $total_sends){
  $mail->sentLogReset();
  // reset cicles variables
  $page->of(false);
  $page->cicles = 0;
  $page->save();
  $page->of(true);
}
 
/********  SET THE START FOR USER SELECTION **********/

$start = $page->cicles * $limit;
 
/********  USER SELECTION **********/
 
$members = $users->find("privacy_newsletter=1,roles!=1016,sort=created,start=$start,limit=$limit");
$to = $members->explode('email');
$mail->to($to);
 
/********  SET THE NEWSLETTER CONTENT **********/
 
$mail->subject("LATEST NEWS");
$mail->bodyHTML($page->newsletter->last()->bodyhtml);
// 'newsletter' field is s repeater with just a textarea field, the current newsletter is the latest item of the page.
 
/********  SEND NEWSLETTER **********/
 
$numSent = $mail->sendBulk(true)->useSentLog(true)->send();
$page->of(false);
$page->cicles++;
$page->save();
$page->of(true);
 

I just created a page with this template and call the url from cronjob.

I'm still optimizing some steps but right now it's working well . Next step is create the cronjob directly from php using some libraries i'm studying so I don't have to check my variables and the cronjob.

I send 2-3k mails per day and using Amazon is really cheap compared to other systems.