Best practices for bundling dependencies (third party modules) with site profiles

[teppo]

So, I've been working on a site profile and came across something I'd like to get some input on. Like probably many other site profiles out there, this one also depends on (and thus includes) third party modules. What I'm wondering is: what would be the best way to include them with the site profile?

I can think of three possible / feasible approaches:

1. Bundle the modules into the site profile as-is, i.e. include the full module directories.
   • Benefits: just install the site profile and the module is instantly available. No other dependencies and doesn't require any knowledge beyond "how to install a site profile".
   • Drawbacks: updating the module requires updating the files manually, or updating via Admin (which may not be permitted, and may not be a suitable option in case the whole site is stored in a version control system). Unless the site profile is manually updated from time to time, it may initially contain very old module versions when installed.
2. Add module repositories as Git submodules.
   • Benefits: modules are easy to install and update with Git, just run "git submodule init" followed by "git submodule update".
   • Drawbacks: Git and some basic understanding of its command-line use is required, and simply installing the site profile itself isn't enough to get it up and running.
3. Add modules as Composer dependencies.
   • Benefits: same as with the submodule approach, but perhaps a bit more flexible – possible to define minimum versions, minimum stability, etc. Also: this would be familiar approach for many PHP devs, while something like Git submodules seem to be less commonly used.
   • Drawbacks: obviously requires Composer and command-line access, plus a basic understanding of what Composer is and how it works. Module directories won't contain .git folder, so can't be updated with Git commands (not sure if this is a real drawback, though).

What would you prefer, or which approach do you currently use? Any other ideas?

I'm kind of leaning towards the Composer method, mainly because I'll likely have other Composer dependencies as well, so it would actually require fewer steps than including modules as Git submodules. Still the submodule approach does have certain benefits, like easier updates (sort of, at least) ?

--

Note: since there's no separate forum section for site profiles, and technically this may apply to literal modules as well, I'm posting this the Module/Plugin Development area. If any of the moderators disagrees, I'm happy to move the thread to another area instead ?

[horst]

This is not an easy question, and so also isn't the answer. 

Personally I would prefer Git submodules as it would be less effort for users who doesn't have either Git nor Composer understanding. Also Git is very tiny action to install and running, composer seems to take more effort. So, for users who have both capabilities, I also would go with option 3, the composer dependencies.

My conclusion is, it depends on your main target group.

[Robin S]

Not sure if you have in mind a site profile for generally sharing with the community, or something that is made available only to some specific users/clients.

If it's the former I would think option 1 would be necessary if you want the profile to be usable by a wide audience. I expect there are many here who are not running Git or Composer. And as you say you would update the profile from time to time (not really different in that regard from a module you have authored).

If it's the latter then another option could be @bernhard's Kickstart tool: 

You would provide users with Kickstart settings that point to a remote profile zip that does not include any third-party modules within it, and then add the third-party modules in the "recipe". That way a user who is installing the profile always gets the latest module versions.

I suppose you could use the Kickstart approach for a generally shared site profile too but it comes back to the audience thing again. If PW beginners should also be able to use the profile then it's probably best if the profile can be installed as per the standard procedure without them needing to learn any additional tools.

[teppo]

Thanks @horst and @Robin S for the input! Those are some valuable insights right there.

I do have to agree that #1 would be beneficial for the widest possible audience. As you pointed out, Robin, this is likely still the best approach for most site profiles out there, simply because it's the only solution that works right out of the box. Anything else would require either the use of somewhat developer-oriented tools (such as Git or Composer) or a totally different approach (ProcessWire Kickstart) – and as such would be a potential turn-off for some users.

This is just me thinking out loud, but if site profiles were able to provide a list of dependencies that ProcessWire could then satisfy (or tell the user to handle manually if it couldn't), that could be nice. Or perhaps ProcessWire should provide some sort of built-in support for "recipes"... ?

Anyway, in my case the site profile I'm working on is going to be rather developer-oriented, so if Composer is enough to scare someone away – well, that's probably for the best. Basically what Horst said above: it depends on the target group, and in this case my target group are developers, so going with a set of tools that are anyway considered essential in the business isn't necessarily a bad thing. I could also go with Kickstarter, but since I'm trying to reach for the "serious developer" territory here, I think using industry-wide best practices is actually going to be a benefit in itself ?

(And yes, I'm currently leaning towards Composer approach. Though please let me know if you have better ideas – I'm all ears!)

 

[LostKobrakai]

To me site profiles are startpoints. I see it as bundle of dummy/base content + the necessary modules for letting the site work in its installed form. For keeping the profile itself up to date for future installs I‘d probably use git submodules, but distribute it as plain zip with everything in it (iirc github does that for the zip download). 

When things in a site should be managed after the initial installation as well I think it should be done by a module (can be installed by the profile as well). The module can handle all the necessary changes to content on update and has ways to setup dependencies to other modules. 

Sadly we‘re currently in a weird middle state of having kinda integrated composer, but also really just using is as a parallel universe, which means you either use composers dependency resolution or processwire‘s, which to me means skipping conposer is currently the better way if broad adoption is the goal.

[teppo]

Thanks Benjamin!

What you're saying makes sense, and I've also come to terms with the fact that the contents of the site profile itself (mainly what's within the templates directory) won't be easily updated. They are, after all, intended to be modified, and as such "updating" them doesn't make a whole lot of sense. I do also agree that we're in a bit of a strange place right now: on one hand there are native (to ProcessWire) ways to handle things like dependencies, but on the other hand we've also got certain level of built-in support for Composer.

That's not necessarily a bad thing, though: just for an example, folks have been doing stuff like installing modules from within the Admin for quite some time now, while I still haven't enabled that on any of the sites I've built – and won't, unless it's for a site that someone else is going to manage. First of all I prefer not to give the site easy write access to executable code (security), and second of all I've had some trouble making that work with my version control and deployment practices. (There are other reasons too, but those have always been my "top two".)

The way I see it, the tools and practices I prefer are quite likely not what the "mainstream user" prefers – and I'm OK with that. Same goes for the dependency management: in my case I'm thinking that whatever solution is used should be reliable, easy (comparatively: running a few commands from command-line is easy), automatable, and compatible with different workflows (including things like CI/CD tools). The fact that ProcessWire caters for these tools just as it does for the built-in alternatives is a strength in my books.

Anyway, I guess this once again boils down to the rather well established point that there's more than one way to do it ?

--

Sidenote: to my best knowledge GitHub download feature actually doesn't include submodules, so the only way to have everything as an easily downloaded package is to bundle it all together. This is part of the reason why I don't (currently) think that submodules are worth the hassle.

[szabesz]

Sounds like this issue is sowewhat related to this one: https://processwire.com/talk/topic/21716-processwire-profiles-whats-missing/?do=findComment&comment=186721

Maybe @Jonathan Lahijani has already come up with his "solution"? Quote:

"...I don't know what term would accurately describe it, but its a ProcessWire module that's an opinionated, update-able starting point, oriented towards developers..."

[teppo]

27 minutes ago, szabesz said:

Sounds like this issue is sowewhat related to this one: https://processwire.com/talk/topic/21716-processwire-profiles-whats-missing/?do=findComment&comment=186721

Maybe @Jonathan Lahijani has already come up with his "solution"? Quote:

"...I don't know what term would accurately describe it, but its a ProcessWire module that's an opinionated, update-able starting point, oriented towards developers..."

At least to some degree related, no doubt!

The reason I'm asking about ways to bundle dependencies into site profiles is that I was originally a bit conflicted about how to move forward with a project I'm currently working on. Based on the description in that thread there are definitely certain similarities with my project and what Jonathan is working on, but I'm not yet quite sure if there's any real common ground (except, perhaps, the target group) ?

Either way it'd definitely be interesting to hear more about Jonathan's project – and perhaps share/borrow some ideas as well ?