Modules Manager

[Soma]

Modules Manager for ProcessWire2.+

Module Manager enables you to browse the modules directory on modules.processwire.com, download, install or update them.

Requires

 

• the JqueryDataTables module to be installed before you can install Modules Manager.
• "allow_url_fopen" to be enabled in your php.ini.
• "openssl" PHP extension needs to be installed on your server.
• PHP to have read/write access to the /site/modules/ directory

For further informations contact the readme seen on modules directory or github.

 

Modules Directory: http://modules.processwire.com/modules/modules-manager/
Github Project: https://github.com/s.../ModulesManager

[Pete]

I don't like this...

I LOVE IT

I understand that this is a work in progress, but I think it would make sense if this were eventually merged with the actual modules page in the PW admin once it's tested.

[Soma]

Thanks Pete. I'm not sure this should replace the PW module page, I see it more as a nice extension to it.

So the current modules page is nice for installing and have a overview of ALL modules and still allow for manual downloading and installing. The modules manager is just for downloading and seeing what modules are available or already installed and to update them. Many things are still unconsidered and untested (only locally), but figured it would be a nice practice to see how far it can get with given new module directory.

[Pete]

Fair enough

I just noticed the Newsletter Tool link in your menu too - what's that?

[Soma]

Fair enough

I just noticed the Newsletter Tool link in your menu too - what's that?

Just a simple Newsletter-Tool of a co-worker iframed.

[nik]

This one's just great, Soma! Had to give it a try straight away.

Also on my local install I had to adjust the /site/modules/ directory to have write permission using php.

Same goes for site/assets/ as well. Without write permissions downloading the module zip file fails. This actually revealed a bug too, giving this:

Error Using $this when not in object context (line 278 of /Users/nik/Sites/test/site/modules/ModulesManager/ModulesManager.module)

Looks like you're using $this inside a static method (downloadFile()). Changing $this->error(...) to wire()->error(...) does the trick (don't know if this is the best way, though). There seems to be a few of these actually in the same method.

When installed you'll have a new admin page under "Setup", feel free to move it to wherever you like.

This left me a bit confused at first as there was no new page under "Setup" after installation. But then I found it under "Modules". So, either that's what you intended and just documented it wrong or maybe page with id 21 isn't always "Setup". Either way, something should be fixed here as well .

Anyway, good work so far!

[ryan]

I don't like this...

I LOVE IT

I understand that this is a work in progress, but I think it would make sense if this were eventually merged with the actual modules page in the PW admin once it's tested.

I agree with everything Pete said! Tested out here and worked beautifully. Such an easy way to install modules. Great work Soma!

I was actually experimenting with something similar here this morning before work. But my version was limited to pasting in the ZIP file URL and having it install from that. Yours is so much farther along. So let me know what I can do to assist or collaborate here. I agree that it would make sense to have this buit in as a core functionality once stable.

A few minor things to mention:

1. It installed under /processwire/modules/modulesmanager/ rather than /processwire/setup/modulesmanager/. So I moved it to Setup. Your instructions mentioned it would install on Setup, so just wanted to mention this in case anyone else tries it and can't find the Modules Manager.

2. Your uninstall function is named "deinstall" rather than "uninstall". As a result, it won't get executed when the module is uninstalled.

3. Rather than using mkdir() and $this->config->chmodDir, you can use the wireMkdir() function if you want to (which pulls from $config->chmodDir on it's own).

4. There are some chmod(0777)s in there. Should this use $config->chmodDir instead?

5. For lines like this (below), I'd suggest comparing the page name rather than the title, just because I can see tweaking headlines, but not changing page names. Though what we'll probably do is just setup the web service to have an option to exclude modules that aren't auto-installable, so that you don't have to perform the check at all.

if(in_array('Language Packs',$categories)) continue;
if(in_array('Admin Themes',$categories)) continue;

[Soma]

Sorry, thanks for mention it, yeah it's under module page. I'll update so no confusion, lol I forgot to change the id.

Am working on some things at the moment and coming back later. Thanks for the feedback so far.

[Martijn Geerts]

Wonderful, I should say that every time! The quality of modules here are so high, never seen it elsewhere on CMS land. Think it has to to with the simplicity & the flexibility of the core. This is without doubt beautiful work Soma ! You're on a killing Spree!

[Soma]

I just commited an update with some things corrected and improved. Also changed install parent to "Setup". Not necessary now, but also added apikey and remoteurl config.

Thanks Nik for the feedback. I changed the function to private and not static, so it should be ok. I haven't really tested it before, but should work now and throw some infos that if the dirs are not writeable.

Thanks Ryan, very helpful infos. I removed the chmod 0777 as it was to test. And I use now wireMkdir. Good to know. Also corrected the deinstall typo!

Point #5. Good point. I'll try to use the array keys for name => title. Will let it for now.

As for the work on it and future. I'm pretty sure at some point I would love to get any help (any time), I would love someone with more experience in all those advanced stuff could help. Especially if it should be a core implementation once, I think you would be the person #1 to help and take it over. But I also think there's so much about it that can or should be done my head already explodes.

[Soma]

Just made an update with some small improvements. Added also link to each module for its page on http://modules.processwire.com.

I have now also added the module to the modules directory http://modules.processwire.com/modules/modules-manager/ so it can now also be updated using the Modules Manager. I just tried and it's possible so it updates itself!

[diogo]

I love this module Soma!! Everything seems to be working perfectly

Maybe you should refer on the instructions that the module needs libcurl to work. I tested this on my localhost, and got an error because it wasn't installed.

it does say it... sorry, should read things carefully before posting stupid things...

[ryan]

Soma, an alternative to libcurl is to just use copy(). At least, that's how ProcessWire does it with images that you add() from remote URLs. It's true that it requires PHP to have allow_url_fopen setting enabled. But then again so does file_get_contents(), which you are already using with a remote URL.

copy("http://remote file", "local file"); 

As for the work on it and future. I'm pretty sure at some point I would love to get any help (any time), I would love someone with more experience in all those advanced stuff could help. Especially if it should be a core implementation once, I think you would be the person #1 to help and take it over. But I also think there's so much about it that can or should be done my head already explodes.

I'll be happy to help, though I think you've already got as much "advanced stuff experience" as anyone here. I don't want to take over, just want to collaborate and work towards getting it in the core (if you are open to it), but you stay in charge of it. (I already got too much to keep track of). This will be one of the best and most important core additions to date. It literally eliminates the need to add future optional core modules. For instance, there would no longer need to be core-but-uninstalled modules like LanguageSupport, FieldtypeRepeater, etc., since they could be installed just as easily via the Modules Manager.

I am wondering how we should integrate this into the existing Modules page in the PW admin? I'm thinking this should probably replace the current Modules list in PW, but there would need to be a clear differentiation between what's installed and what's installable (maybe each as a tab on the modules screen). The "what's installable" screen should probably start out with a category selection, since I'm anticipating the installable modules to get into the hundreds and eventually thousands, so needs to be more scalable than a long linear list. Just initial thinking and ideas. You may have already put a lot more thought on this stuff than I have.

[Soma]

Wonderful, I should say that every time! The quality of modules here are so high, never seen it elsewhere on CMS land. Think it has to to with the simplicity & the flexibility of the core. This is without doubt beautiful work Soma ! You're on a killing Spree!

Thanks Martijn! Yes there's many high quality modules as also exists in other frameworks or CMS' and I also think it depends a lot on the system and community. Yeah the simplicity and flexibility in PW is astounding and almost cries out "EXTEND ME!". The fact that it is a framework that is so well designed and using that foundation, the CMS as such is built using it makes it very versatile and logik. But the best part is that it is built modular using different types of modules and mini systems (process, fieldtypes, inputfields, js modules ...) and in the exact same way you can extend it and build modules that work exactly the same! You can take almost any module out from the wire/modules/ and move it to the site/modules/ folder modify it to your needs and install it. To go even further it uses same API as you use (or can use) in templates, the transition is almost seamless. So you can also look at core code and modules as reference and example.

I'm not the very advanced php programmer and always got my problems with OOP, but after digging into PW after seeing how modules are done it took me no time to get into it and already built 15 modules in 1 year, AND IT WAS FUN! I'm the guy in love of least resistance and having fun . After using MODX 1-2 years, all I got was a half backed test module that had no purpose but was damn hard to setup, but also didn't attract me as much as PW did. Same goes for other systems that seem like having extra hard layer that's designed for modules, but too complicated to be fun, or like another system inside a system. So even if may not perfect in all regards PW is a great foundation that will show and have impact on other systems. MODX already is announcing a light weight distribution with 3.0, and I'm proud modern systems like PW played a role in it.

[Pete]

What he said

[Martijn Geerts]

What a uplifting piece of text you wrote over there Soma. TNX

[Soma]

Soma, an alternative to libcurl is to just use copy(). At least, that's how ProcessWire does it with images that you add() from remote URLs. It's true that it requires PHP to have allow_url_fopen setting enabled. But then again so does file_get_contents(), which you are already using with a remote URL.

copy("http://remote file", "local file"); 

Thanks a lot for pointing that out. I'm not sure which would be the best solution. Also towards a solution that is compatible with most hosting environments.

I'll be happy to help, though I think you've already got as much "advanced stuff experience" as anyone here. I don't want to take over, just want to collaborate and work towards getting it in the core (if you are open to it), but you stay in charge of it. (I already got too much to keep track of). This will be one of the best and most important core additions to date. It literally eliminates the need to add future optional core modules. For instance, there would no longer need to be core-but-uninstalled modules like LanguageSupport, FieldtypeRepeater, etc., since they could be installed just as easily via the Modules Manager.

Thanks, I'd be glad to help getting this done, just think I could need some help in finding best methods to implement it and you know a lot more than I. It would be a great addition and already is. Well most important is that it is a nice put-on-top feature that still allows managing modules if no internet access is present or if the webservice is down.

If anyone willing to help I would really appreciate it. Some more advanced users here? Originally this is meant to be proof of concept and makes a good module, but merging it with the already existing and coding all the "make it easy, nice and safe" will be some work.

I am wondering how we should integrate this into the existing Modules page in the PW admin? I'm thinking this should probably replace the current Modules list in PW, but there would need to be a clear differentiation between what's installed and what's installable (maybe each as a tab on the modules screen). The "what's installable" screen should probably start out with a category selection, since I'm anticipating the installable modules to get into the hundreds and eventually thousands, so needs to be more scalable than a long linear list. Just initial thinking and ideas. You may have already put a lot more thought on this stuff than I have.

I just updated the Modules Manager module to have category select filter! It also show now the summary as tooltip if you hover module name

I think if we take this route it would have to be well designed. I'm also wondering how it could be merged with the current with all the features needed. But have not made really come to conclusions.

• For sure some splitting and filtering would make sense using tabs or alike to separate core and third-party modules.
  
• Also some flags or categories to help filtering would be nice.
  
• What really is needed is still have the ability to keep it work without internet.
  
• Also deciding what we could do with modules like language packs, themes and such modules that are not meant to be installed.
  
• I'm a little fuzzy atm what all should be considered but it's a lot
  

[ryan]

Thanks a lot for pointing that out. I'm not sure which would be the best solution. Also towards a solution that is compatible with most hosting environments.

What would probably be best is to abstract it to a $this->copy($url, $dest); function that checks what's available. If CURL is available, it delegates to $this>copyCURL(). If not, it checks if ini_get(allow_fopen_url) and calls $this->copyPHP(). Or something like that.

ZipArchive is another thing that may not be in all installations (PW uses exec('unzip') instead, which also isn't universally available. If the file system isn't writable, using PHP to FTP to localhost is another option to perform installation.

You are right, lots of potential factors. But probably best to start simple and require ZipArchive and a writable modules dir, and then expand upon that later.

Well most important is that it is a nice put-on-top feature that still allows managing modules if no internet access is present or if the webservice is down.

Do you mean like uploading a ZIP file of a module?

I just updated the Modules Manager module to have category select filter! It also show now the summary as tooltip if you hover module name

Looking good! I just let my Modules Manager update itself, which is always fun.

Also deciding what we could do with modules like language packs, themes and such modules that are not meant to be installed.

Language packs are already installable from the admin, so don't think we need to do anything there. Admin themes would be a fun thing to have installable in the modules manager, longer term probably. Site profiles won't ever need to be installable through the Modules Manager, since they get installed before the site exists.

[Pete]

What would probably be best is to abstract it to a $this->copy($url, $dest); function that checks what's available. If CURL is available, it delegates to $this>copyCURL(). If not, it checks if ini_get(allow_fopen_url) and calls $this->copyPHP(). Or something like that.

ZipArchive is another thing that may not be in all installations (PW uses exec('unzip') instead, which also isn't universally available. If the file system isn't writable, using PHP to FTP to localhost is another option to perform installation.

You are right, lots of potential factors. But probably best to start simple and require ZipArchive and a writable modules dir, and then expand upon that later.

I was going to suggest something like this too - having it check for necessary functions and if it can't find either a suitable copy or unzip method then instead of the download option erroring, it simply pops up a modal informing the user why it can't install it automatically, steps to solve this (ask webhost to enable X, Y or Z) and the link to simply download the module as well as generic installation instructions (just covering all bases ).

I'm finally going to download this now I have five minutes, but I already know how awesome it is from the comments and screenshot above

[Nico Knoll]

Hey, looks great!

Some weeks ago I also worked on a module which upgrades Processwire. I did the download like this:

$git = file_get_contents('https://api.github.com/repos/ryancramerdesign/Processwire');
$decoded = json_decode($git, true);
$content = file_get_contents($decoded['html_url'].'/zipball/master/');

chdir($this->config->paths->AutoUpgrade);

$fp = fopen("upgrade.zip", "w");
fwrite($fp, $content);
fclose($fp);

Maybe it'll help you

/ Nico

[Soma]

Thanks Nico, this is basicly what I do. Though I got other github url to deal with (though it could be easily changed) but good to know github api.. I actualy also looked at your module when creating my module.

New update 0.0.8

I commited a new update yesterday.

• Added additonal check for "allow_url_fopen" and throw error if not enabled.
  
• I changed to use copy() instead of curl.
  
• I tested it locally and on a low budget hosting and a good one, now it works on all without problems (though locally I got permission problems though not relevant).
  
• I found that using copy or file_get_contents would work better, curl gave me "open_basdir" (low budget shared hosting) error on which would cause problems if not configured correctly on server. So this should be better now. Not sure about other restrictions and limits (size?).
  
• Also improved the remove function to recusively remove the extracted folder after content is copied, it now also deleted .* files like .htaccess.
  

If you already got it installed just click "refresh" then "update" the module. Oh and let me know if any issues arises.

Edit: Heh, just refreshed and new categories and modules pop up! Love it.

[Pete]

Cheers for the update Soma - changing from curl to copy() fixed it so that it will now download files on a default installation of XAMPP.

Before that it was erroring as I don't think CURL is turned on by default in XAMPP, so whilst it's easy to change the PHP.ini file it's also great that it now works out of the box

[ceberlin]

Hi,

isn't allow_url_fopen a big security risk, if enabled? My povider suggests not to use that.

Is there an alternative?

For me, it would be good enough, if I would (like Drupal) receive an email, if modules, which I use, have updates, and an emergency email, if the module update is fixing a security bug. The cron would check that for me. (Or do we have that already and I had just screwed up my setup?)

[Soma]

Hi ceberlin, thank for your post.

Do they have cURL enabled? We could then implement to use it if allow_url_fopen is disabled. I think in case this doesn't work out it could be a simple download link to download and intall it manually.

[ryan]

isn't allow_url_fopen a big security risk, if enabled? My povider suggests not to use that.

I'm guessing Media Temple? That's the only place I've seen allow_url_fopen disabled.

It is only a security risk with poorly written PHP code. Anything powerful is a security risk if it's coded blindly. Disabling allow_url_fopen makes a host feel more secure because they are placing extra limits on what you can do with your web hosting account. It means they don't have to have as many people looking after things (drives profit). It also means you have a disabled web account. Unless you are getting free or super-cheap hosting, I would not consider this type of intentionally-limited account unless there is an override for those that want it (like being able to specify your own php.ini to enable such things).