Progress report for development of my GroupMailer module

[Gadgetto]

Under MODX my Newsletter Add-On manages all mailings in its own resource containers. This is comparable to a PW page, which serves as a container for subpages.

My newsletter module GroupMailer will also provide an administration console where you can start, stop and generally manage mailings + allows you to watch the sending status.

What would you recommend as an experienced PW user? How should the mailings be managed in the PW module? I want to follow the PW paradigms as much as possible and also offer the greatest possible flexibility.

Here is a screenshot from the MODX version:

Greetings, and a Happy New Year!

Martin

[flydev]

I would see something like you show us in the screenshot in a Process Module. I remember a lot people discussed about NewsLetter module here in the forum where you can grab insights from them.

Also there are already some modules which could be taken as idea / or connected to this module. Like sending Page via mail, members registration etc. You will find all those modules and discussion with the link below :

https://www.google.com/search?q=site:processwire.com/talk/ newsletter

and 

 

9 minutes ago, Gadgetto said:

allows you to watch the sending status.

For this task, I would see Tasker implemented (the only one drawback, and a good one, it work only under Unix based system, Windows do not support the PCNTL extension)

https://github.com/mtwebit/Tasker/wiki

 

I am watching closely now your thread as I will need a gud NewsLetter module in the next months ?

 

[bernhard]

RockGrid is perfect for such scenarios:

• You get nice, condensed, sortable, paginated tables
• You can style cells easily (like statuses for sent, pending, draft etc)
• You have a batcher built in, making it possible to send x mails at a time, running via AJAX so you don't get server timeouts

The main reason why I released it under the MIT license was that it can be built into other modules easily.

5 hours ago, Gadgetto said:

I want to follow the PW paradigms as much as possible

That would be a process module and MarkupAdminDataTable and LOTS of custom code...

[dotnetic]

Please take also a look at existing solutions. Maybe they can act as a starting point: 

 

[gebeer]

Great to see someone working on a Newsletter module again! Something that is definitely missing in PW.

Ryan made a remark here in the last paragraph that he is planning on releasing a ProMailer module. Looking forward to that also.

Like @bernhard said, a process module would be the way to go about it, I guess.

@horst has released a basic queue module back in 2016 that might come in handy for batch sending.

 

[bernhard]

1 hour ago, gebeer said:

@horst has released a basic queue module back in 2016 that might come in handy for batch sending.

RockGrid has this built in, so it would be a lot easier to just use RockGrid instead of using DataTables + Queue.

Only difference is that horsts queue works on the server side and RockGrid batcher works on the client side.

Sending emails is another interesting decision to take: do you want to send them via the server or via a transactional service? This would need different implementations..

[Gadgetto]

This is what I have so far (very early state, please do not pay attention to the column values)

I have decided to use ProcessWire modules wherever possible. I also want to avoid dependency on third-party modules - there are so many out there which are abandoned or no longer supported. The grid in the screenshot is an MarkupAdminDataTable combined with parts of a PageLister.

As this will be a live-view, where you can watch your mailings flying out, I'll refresh part of the columns via Ajax.

As I said, we have already developed a very powerful multiprocessing sending engine for the mailing module. So we are good to go here.

In general, I'd like to orient the Module as closely as possible to the ProcessWire look and feel and paradigm. The UIKit Admin Theme will be primarily supported.

[Gadgetto]

45 minutes ago, bernhard said:

Sending emails is another interesting decision to take: do you want to send them via the server or via a transactional service? This would need different implementations..

I'll support both methods (probably "sending via server" in the first place).

[gebeer]

3 hours ago, bernhard said:

Only difference is that horsts queue works on the server side and RockGrid batcher works on the client side.

Client side is fine for not so long running tasks. But for queus that take longer you might not want to sit and wait until they're finsished. So I guess server side is the way to go here.

 

2 hours ago, Gadgetto said:

I'll support both methods (probably "sending via server" in the first place)

Great!

[Gadgetto]

7 minutes ago, gebeer said:

Client side is fine for not so long running tasks. But for queus that take longer you might not want to sit and wait until they're finsished. So I guess server side is the way to go here.

 

Great!

Our multi-processing engine runs server side. I don't see any advantages for running this client side. I'd like to start sending and then close browser. 

[bernhard]

19 minutes ago, gebeer said:

Client side is fine for not so long running tasks. But for queus that take longer you might not want to sit and wait until they're finsished. So I guess server side is the way to go here.

9 minutes ago, Gadgetto said:

I don't see any advantages for running this client side. I'd like to start sending and then close browser. 

That's why I mentioned it. That might be a feature or a drawback - depending on the situation.

 

[horst]

4 hours ago, Gadgetto said:

I'll support both methods (probably "sending via server" in the first place).

We have the wiremail class. Is this supported too? 

Im asking, because there already exist some modules to use different services, what would be cool if this can be used with your mailer app too. (If one need another service, just switching one module and its done)

[Gadgetto]

1 hour ago, horst said:

We have the wiremail class. Is this supported too? 

Im asking, because there already exist some modules to use different services, what would be cool if this can be used with your mailer app too. (If one need another service, just switching one module and its done)

Sure, I'll setup on WireMail class so we can use all available extensions and services. And it can be "easily" extended.

[Gadgetto]

Here is another status report for development (early state) of my GroupMailer module:

I've setup the dashboard to list all GroupMailer messages found in pages tree. The question is how messages (pages) should be managed in general. I plan to make the whole thing as flexible as possible. You can create messages wherever you want, in the root - directly under "Home" or within a container page.

In the upper area of the dashboard you can select the desired container and all messages of this container will be listed.

Messages are identified by the template "groupmailer-message". A field containing meta data for the message (dispatch status, number of recipients, number of mails sent, etc.) is attached to this template. The Messages grid will show these values.

The Messages grid itself is a rewrite of the ProcessPageLister module matching the requirements of GroupMailer. It will provide all features of the original PageLister module + specific GroupMailer functions.

It will be a live view of the current state of all Newsletters (Messages). That means you can watch how mails are sent out and immediately stop/restart etc. the sending process.

Here is a screenshot of the current state:

(The columns are not the final ones and will be adjusted according to my needs.)

What do you think? Am I on the right way or do you have any hints?

[dotnetic]

@Gadgetto Just a few quick thoughts:

• The overview should show the subscriber list (which group/s is the mail being sent to), or is this another page, which just shows the contents of a newsletter/mail?
• Does the module generate the HTML and text for the email?
  If so, the contents should be HTML a la 95 / aka Word HTML and also use inline-styles.
• Does the module take care of the subscribing process (for example double-opt-in subscription)?
• I dislike the idea, that the contents/messages have to have a special template. To be real flexible, I could use my own templates and pull data from them. 
  So there could be options for a message/newsletter content:
  Select a container page (only pages with template groupmailer-message will be used), use a frontend template (which could only be accessible if user is logged in)  or provide your HTML here (CKEditor field)

[Gadgetto]

33 minutes ago, jmartsch said:

The overview should show the subscriber list (which group/s is the mail being sent to), or is this another page, which just shows the contents of a newsletter/mail?

The page in the screenshot shows the GroupMailer dashboard (the main page of the module). This will be a live-view of all your newsletters/messages/mailings, where you can watch your mailings flying out.

The subscribers overview will be another page!

33 minutes ago, jmartsch said:

Does the module generate the HTML and text for the email?
If so, the contents should be HTML a la 95 / aka Word HTML and also use inline-styles.

Yes, GroupMailer will be able to create full Emails including Subject and Body. How this will be achieved is another thing ... Simply using the CKEditor won't produce nice formatted mails which renders similar in all clients.

I'm thinking of a kind of template system whose areas represent content blocks. These content blocks can then be filled by any fields you like. How exactly this will be implemented I still have to think about.

Inline-styling will be built in (also had this in my GoodNews extra for MODX). 

33 minutes ago, jmartsch said:

Does the module take care of the subscribing process (for example double-opt-in subscription)?

Yep! This will be a sub-module. I know there are already some other modules available that use subscription models. But I want to avoid dependencies on other modules, because some/many of them are badly maintained or abandoned. 

33 minutes ago, jmartsch said:

I dislike the idea, that the contents/messages have to have a special template. To be real flexible, I could use my own templates and pull data from them. 
So there could be options for a message/newsletter content:
Select a container page (only pages with template groupmailer-message will be used), use a frontend template (which could only be accessible if user is logged in)  or provide your HTML here (CKEditor field)

You won't be bound to the pre-installed template. Using your own templates you will need to add a special GroupMailer field (pre-installed) which will hold the Message meta-data needed by the engine.

In general I'd like to keep the development in such a way that I will release an executable version very early and take into account opinions as well as tips and suggestions from experienced developers like you. 

Fact is, I still have a long way to go. In particular, the incredible flexibility of ProcessWire requires a major rethink and increases the development effort to keep the flexibility.

[dotnetic]

4 minutes ago, Gadgetto said:

I know there are already some other modules available that use subscription models. But I want to avoid dependencies on other modules, because some/many of them are badly maintained or abandoned. 

But to save yourself from work, you could use one of them as a starting base (and extend them by yourself), because they may be abandoned, but that does not mean, they aren't functional.

4 minutes ago, Gadgetto said:

I will release an executable version very early and take into account opinions as well as tips and suggestions from experienced developers like you. 

Release early and release often is the way to go! Good choice ?

[Gadgetto]

Just now, jmartsch said:

But to save yourself from work, you could use one of them as a starting base, because they may be abandoned, but that does not mean, they aren't functional.

That's my plan. Do you know good modules for subscription management?

2 minutes ago, jmartsch said:

Release early and release often is the way to go! Good choice

It will take some time, but I will publish GroupMailer on GitHub. Active participation will then be possible.

[dotnetic]

Maybe try https://github.com/justb3a/processwire-newslettersubscription. @justb3a's stuff has normally very good quality.

[Gadgetto]

Here is another progress report for development (early state) of my GroupMailer module:

• The dashboard and the message lister are nearly finished.
• Behind the scenes I created a custom field which holds all MessageMeta data. This special field can be attached to each template you like and will immediately make all corresponding pages a GroupMailer message. This allows to maintain the extreme flexibility of ProcessWire.

More to come ...

[lpa]

Hi, 

What might be the status of GroupMailer? Is it published somewhere or was it ever finnished?