PW 2.8.32, current work, and PW usage

[ryan]

This week we’ve got ProcessWire 2.8.62 (corresponding to 3.0.62 master), a look at some current work-in-progress, and some nice momentum in ProcessWire usage and market share.

https://processwire.com/blog/posts/new-2.8-version-current-projects-and-pw-usage/

[Robin S]

@ryan, you mentioned Pro module updates in this week's post. When you're working on the pro modules it would be great if you could give the pro module documentation some attention.

The documentation that does exist is only in the pro module subforums and the forum post format doesn't provide a good interface for browsing or searching individual module methods or for formatting the documentation and code examples. It is much inferior to the current core API reference docs, which doesn't seem right considering the pro modules are premium commercial products.

Keeping the documentation in the pro forums also has the effect that if a pro module purchaser does not renew their support/upgrade license then they are cut off from the documentation. Not that cost of ongoing support/upgrades is unreasonable (your pro modules are fantastic value for money) but I think the essential documentation should always be available to pro module purchasers.

A third-party pro module was launched today and the presentation of its features and documentation is excellent. This got me thinking about the issue.

[szabesz]

13 hours ago, Robin S said:

I think the essential documentation should always be available to pro module purchasers

+1 But why not the "full documentation"? I purchased ListerPro without actually knowing what I'm gonna buy, I was just wild guessing. I do like what I have but was surprised by the low number of Actions it comes with, for example. I've also been thinking about ProFields, but descriptions and proper examples of the usage of those fields are scarce, so I cannot really see if I really need them or all the features I need can be done with the built in core fields.

Speaking of ProFields, they are listed here: https://processwire.com/talk/store/

But there is this one here: http://modules.processwire.com/categories/premium/ which is rather outdated as far as I can see.

And there are the old video demonstrations, also rather outdated.

[bernhard]

Hi @ryan

thanks as always for your work and all the progress! may i ask about one special feature here that lots of us are waiting for since < 2016 and that was on the roadmap of 2016 and 2017 and still there are no news about it...

I'm talking about client side image resizing...

Roadmap 2016

Roadmap 2017

Please don't get me wrong - the intention of this posting is really not to point to you with my fingers (like "but you said you will do that..."). and of course, i could have tried to implement it on my own... but i'm not sure how much sense that would make. first of all, since it is on the roadmap for such a long time, it is risky to put effort in something that maybe solved by the core one week later and likely much better. secondly i'm not that familiar to javascript and last but not least, i think this should really be built into the core image field because i think that is a main feature of a content management software!

 

if you don't think that you can make that happen soon it would be nice to know. maybe we could find someone in the forum that has the abilities to do it. i would also be happy to sponsor development, or at least a part, if someone else would be willing to join.

ps: i know that we already have media manager and jquery file upload, but i don't think that this is a good solution. processwire is so easy and flexible in almost all situations that it just feels too bad to not being able to just put an imagefield on one template and make the client upload several large images and display them as a gallery... 

would really be happy to hear your opinion about that and of course, also the opinion of all the others

[ryan]

Speaking for my modules only, I'm marketing these as turn-key modules, not as API tools. With the exception of the ProfilerPro module and Likes modules, the Pro modules that I develop were never intended as primarily API programming modules. They are modules that, outside of output generation, are meant to be largely turn-key, without little or no coding needed. It is true that you can use them at the coding level to do lots of things, they have an API and hooks available, and we could write an encyclopedia worth of content to cover it all. And it's also true that there are a few people, perhaps 5% of the users, that pursue this kind of stuff with these modules (especially Robin S., Adrian, LostKobrakai). I think that's great, and I'm happy to support that for those that are interested in using them that way, and have been for a long time via the support boards. This is a welcome and valuable audience for these modules, but not the primary or intended audience.

The intention with most of the Pro modules is to provide things you can install and start using, and not have to code around. The documentation is consistent with the audience and intention of the modules. For those that want to get into the code side, I welcome it, but prefer to support them via the support board and focus in specifically on their needs. Though in the ProFields modules we do cover a lot of API side stuff too. The code of the modules is also well documented with phpdoc as well, making it handy in an IDE environment. If I develop a module intended primarily for API usage in the future, then of course the approach would be different. But because that doesn't describe my current Pro modules, I prefer to support people individually when they want to pursue unique needs that might require additional code, as everyone's needs are slightly different. 

Quote

A third-party pro module was launched today and the presentation of its features and documentation is excellent. This got me thinking about the issue.

That's beautifully put together– joshuag is an amazing designer and front-end framework developer, and there's no way I could ever compete with his skills in those areas. Sorry about that, but that's just reality. I can make great modules, but I'm far more developer than designer and marketer. I gave up trying to be a designer long ago. Also should mention though that what he's marketing clearly has a larger API-side than what's intended for any of my modules. And I agree he's outlined everything really nicely.

Quote

I purchased ListerPro without actually knowing what I'm gonna buy, I was just wild guessing. 

I feel that I covered everything about the product pretty exhaustively. What do you think is lacking? https://processwire.com/api/modules/lister-pro/

Quote

I do like what I have but was surprised by the low number of Actions it comes with, for example.

Actions are just one small part of ListerPro, but it comes with seven actions, including an action that's a template for creating your own. Also why the surprise (?), the ListerPro page outlines all of the seven actions here. 

Quote

 I've also been thinking about ProFields, but descriptions and proper examples of the usage of those fields are scarce, so I cannot really see if I really need them or all the features I need can be done with the built in core fields.

Have you seen this? https://processwire.com/api/modules/profields/

Quote

But there is this one here: http://modules.processwire.com/categories/premium/ which is rather outdated as far as I can see.

I don't use this one very often, and may delete it. It started back before we had an on-site store, so isn't really needed anymore, as it's been replaced by our store. 

Quote

And there are the old video demonstrations, also rather outdated.

They are not outdated. As far as the information goes about the modules, they are fully relevant and up-to-date. Though I should mention I always try to provide more than I market, and feel this is a good thing. I like for people to feel that they got something more and better than what they expected, with everything I do. 

 

 

[ryan]

Quote

I'm talking about client side image resizing...

Most of our image/file features have been developed by the community rather than me, and I'm guessing this will follow a similar path. It doesn't seem to come up very often, but since you've mentioned it here maybe we'll have to bring more focus to it. I'll try to get a closer look this week to get a better idea of when we might be able to get more momentum going here. I do agree it would be a nice thing to have sooner rather than later. Also, since you quoted both 2016 and 2017 roadmap, it's important to note that this is a roadmap. It is a list of goals, it is not a contract of promises. We never expect to be able to accomplish all the goals, but feel it's important to have them nevertheless. Always good to aim for more rather than less. What doesn't get completed in one year still stays on the roadmap for the next year, unless it's determined it shouldn't be for some reason.

[szabesz]

41 minutes ago, ryan said:

I feel that I covered everything about the product pretty exhaustively. What do you think is lacking? https://processwire.com/api/modules/lister-pro/

First of all, thank you for the reply! Maybe it's just me who found it hard to get a proper idea what to expect. Also, while writing these words I have just realized that there is http://demo.processwire.com/ and by logging in I see that ListerPro (and ProCache) is used there, so I could have checked it out, but maybe other Pro modules could be demonstrated this way and the Store should point to the demo site.

[teppo]

1 hour ago, ryan said:

Speaking for my modules only, I'm marketing these as turn-key modules, not as API tools. [...] It is true that you can use them at the coding level to do lots of things, they have an API and hooks available, and we could write an encyclopedia worth of content to cover it all. And it's also true that there are a few people, perhaps 5% of the users, that pursue this kind of stuff with these modules (especially Robin S., Adrian, LostKobrakai).

This may be just my impression, but I think that this side of your modules, particularly FromBuilder and ProCache, would be worth exploring a bit further. I find myself extending both modules quite often for different client needs, and in my opinion third party extensions to these modules could bring in notable extra value. That, at least, is how it works for their WordPress counterparts.

(Not saying that you should start mimicking Ninja Forms or Gravity Forms etc. but I do think that they have handled the extension thing pretty well.)

1 hour ago, ryan said:

I don't use this one very often, and may delete it. It started back before we had an on-site store, so isn't really needed anymore, as it's been replaced by our store. 

Please don't. While Pro modules are better featured in other places, this is a good way for third party commercial module authors to get some extra visibility. That alone is a very good reason to keep this category around and even develop it further 

(Oh, and sorry if I misunderstood and you just meant that you might remove your own Pro modules from there. That's fine by me, though it'd be a shame; I like the idea of being able to quickly check out what the commercial module market for ProcessWire looks like.)

1 hour ago, ryan said:

It doesn't seem to come up very often, but since you've mentioned it here maybe we'll have to bring more focus to it. [...] I'll try to get a closer look this week to get a better idea of when we might be able to get more momentum going here. I do agree it would be a nice thing to have sooner rather than later.

Thanks for considering this. Perhaps you're referring mainly to new users bringing this up "out of the blue", but I for one tend to avoid bringing up topics that are, for an example, already listed on the roadmap. I'm probably not the only one who thinks like this, which might be one of the reasons why it doesn't come up that often

[Robin S]

10 hours ago, ryan said:

It is true that you can use them at the coding level to do lots of things, they have an API and hooks available, and we could write an encyclopedia worth of content to cover it all.

IMO having comprehensive documentation is something that can only add value to the Pro modules, and takes nothing away from them. I get that many users will only interact with the modules via the admin interface and would have no desire to browse through all the methods - they can ignore that documentation entirely. But it's nice to have thorough documentation for those who do have a need for it.

I'm not proposing that you necessarily spend a lot of time writing up documentation beyond what's already present in the source. The tools already exist to extract and format those phpdoc comments as online documentation as it powers the current API reference docs. And the API Explorer module can do this for modules already. The reason I prefer online docs to the API Explorer is that it's so useful to harness the power of Google to find what you're looking for. It's quicker and more powerful to open a new browser tab and do a search from the address bar (I have a keyword shortcut for "site:processwire.com/api/ref/+%s") than navigating through the API Explorer interface. And of course not everyone has API Explorer.

Could the Pro modules be stored in private GitHub repos and indexed the same way as the API reference? Maybe this module from @justb3a could be useful for the authorisation?

[ryan]

Quote

I'm talking about client side image resizing...

I started experimenting this morning. It's not all that difficult to get going. Proof-of-concept up and running already, so should have this functional on the core dev branch this week hopefully. (note: works for JPGs only)

[LostKobrakai]

@ryan This might be a handy addition to the topic as well: https://gist.github.com/LostKobrakai/ddb11c64dcb0f1bf1c8eed4dd534a868 (prove-of-concept level)

Chunked uploads would allow for file uploads beyond any php set limits even for files, which we obviously cannot shrink client side.

[Pixrael]

and maybe: https://github.com/meltingice/CamanJS or https://github.com/girliemac/filterous-2 too

[LostKobrakai]

These are certainly nice projects, but I doubt a majority of users would need them, so they're probably a better fit for 3rd party modules than the core.

[ryan]

Quote

(note: works for JPGs only)

Working with PNG and GIF now too. 

Quote

Chunked uploads would allow for file uploads beyond any php set limits even for files, which we obviously cannot shrink client side.

Looks really interesting! I'm impressed with how far you've taken this already. Is this a fairly common need? (chunked uploads). I'm not sure I've come across a case where I would have used this on a site before, but I imagine there are cases this could be a real life saver. 

Quote

and maybe: https://github.com/meltingice/CamanJS or https://github.com/girliemac/filterous-2 too

Really cool libraries! Probably too much for our needs in the admin, but seems like a lot of great potential for modules here. 

 

[cb2004]

15 minutes ago, ryan said:

Looks really interesting! I'm impressed with how far you've taken this already. Is this a fairly common need? (chunked uploads). I'm not sure I've come across a case where I would have used this on a site before, but I imagine there are cases this could be a real life saver.

 

This would certainly come in handy for me on a few projects. I do work for charities and set them up with hosting that is free, but there are a lot of restrictions put in place naturally. +1 from me.

[mr-fan]

client side image resizing is definitly a common need - if you work with non proffesional endusers that wanna trow images from their digcams online...;)

[bernhard]

1 hour ago, ryan said:

Working with PNG and GIF now too.

awesome

1 hour ago, ryan said:

Quote

llow for file uploads beyond any php set limits even for files, which we obviously cannot shrink client side.

Looks really interesting! I'm impressed with how far you've taken this already. Is this a fairly common need? (chunked uploads). I'm not sure I've come across a case where I would have used this on a site before, but I imagine there are cases this could be a real life saver. 

i think this would be a great addition as well and would like to support lostkobrakai here. i already stumbled over this problem here: 

and it came up some other times as well: 

i ended up needing my client to upload everything via FTP or dropbox and pasting the links into a textfield. i wanted to develop a module that scans the directory and adds the files automatically to a file field but all of this is a really bad solution regarding UX. i almost had to build a video-platform once where this would also have been a very welcome feature. otherwise i would have had to use some 3rd party uploader with all the problems (different design, maybe opening security holes, ...).

especially regarding security i would feel much more comfortable when having a built in solution rather than adding any 3rd party hack...

ps: it may not be a very common need, but it can be a fundamental one

[LostKobrakai]

4 hours ago, ryan said:

I'm not sure I've come across a case where I would have used this on a site before, but I imagine there are cases this could be a real life saver. 

Basically every time you need to handle files bigger than say 100-200MB, but you don't want to / can't give php more memory.  Also timeouts

4 hours ago, ryan said:

I'm impressed with how far you've taken this already.

Given more free time I would've created a bit more fleshed out PR to the core, but it's currently not possible. The biggest thing to improve here would be some kind of cleanup, so chunks of unsuccessful uploads do not sit there forever. Keeping them for a short timeframe is nice, because it'll speed up second tries.

[LostKobrakai]

Maybe processwire could harness tus for chunked uploads. Integrating the existing js/php components into processwire might be simpler than writing a custom backend for resumable.js.

[Christophe]

tus and the Uppy File Uploader (an official implementation by the same people) seem really interesting.

As well as resumablejs and transfer.sh (mentioned in a linked topic).

[szabesz]

On 3/5/2018 at 7:35 PM, Christophe said:

tus and the Uppy File Uploader (an official implementation by the same people) seem really interesting.

FYI:

https://uppy.io/blog/

"Today, after 3 years of development, we are launching version 1.0 of Uppy, our file uploader for web browsers."

"Saves battery and data plan by letting users pick files from Webcam, Dropbox, Google Drive and Instagram, while letting servers do the heavy lifting via Companion"