Thoughts on module structure for a web app

[MarkE]

I am currently redeveloping a ProcessWire app I wrote a few years back. The app provides integrated website and admin system for a membership organisation and has worked very well. I have other potential users for a similar system, so I am making it more generic and portable, as well as enhancing maintainability and appearance. The system uses quite a large number of custom modules (mostly Process modules but some Lister Page Actions) plus templates and custom Page Classes. 
I am now writing a ‘wrapper’ module which will hold settings in its config and include all the init and ready scripts. 
My question is what is the best practice for structuring the directories? I am pretty clear that it would be a good idea to bundle all the custom process modules in the directory for the ‘wrapper’ module. I could also put all the Page Classes there, but is it better to leave them in a separate classes directory?

If anyone has done something similar, I would be interested to know how you structured it, and any related thoughts you might have.

Thanks  

 

[FireWire]

@MarkE I know this is a late answer, but I think this is a great question and something that can feel daunting. Hopefully it's still helpful for you and possibly others. I originally wrote my translation module back in 2020 and when it came time to build upon it, add new features, and respond to feedback from the community that first version felt like a straightjacket in terms of structure. I rewrote it last year and have been able to iterate on it much more easily and confidently.

This is really a decision based on how you view your project and what "feels good". When you get to writing modules as complex as the one you're describing, as always, the best way to reason about your approach is consistency and maintainability. This is more critical than trying to fit your code into someone else's structure. The most important way to think about organization is considering where you would first look to find a specific feature or behavior and if there are enough of these to warrant their own directory.

My translation module probably isn't as complex as yours is, but it may not matter because we're both writing software applications that must be understood and maintained. Anyone can view the repo I shared above, but here are some notes about my reasoning. The purpose of this isn't meant to be a set of instructions or a roadmap, but more of a way to help you think about what is best for your application.

[app] - This contains everything that supports what the module must do. If it has logic or supports logic, it goes here.
  [Caching] - Caching is a performance requirement and required abstraction and logic to tailor PW's caching feature for my use
    EngineLanguagesCache.php
    TranslationCache.php
  [Components] - Fieldsets used in one or more places where abstracting helped keep clean code in other files, like the main Module
    Traits - Shared behaviors used by Component files
    FluencyApiUsageTableFieldset.php
    FluencyStandaloneTranslatorFieldset.php
  [DataTransferObjects] - All DTOs that encapsulate data provided to and returned by methods in different classes
    Traits - Shared behaviors used by DTOs
    AllConfiguredLanguagesData.php
    ConfiguredLanguageData.php
    DEVDOC.md - A guide for me and others about how to use DTOs and their purpose as general data objects
	EngineApiUsageData.php
	....
  [Engines] - Self contained "submodules" that interact with 3rd party services, both current and future.
    [DeepL]
    [GoogleCloudTranslation]
    [Traits]
    DEVDOC.md
    FluencyEngine.php
    FluencyEngineConfig.php
    FluencyEngineInfo.php
  [Functions] - General functions imported to support specific sub-behaviors that serve a specific purpose in more than one-two places
    fluencyEngineConfigNames.php
    typeChecking.php
  [Services] - Features that are added to Fluency that extend beyond the primary function of translating page content
    FluencyProcessWireFileTranslator.php
  FluencyErrors.php
  FluencyLocalization.php
  FluencyMarkup.php
[assets] - Files that contain admin-facing UI files and compiled code, Gulp compiles and outputs to this directory
	[img]
    [scripts]
    [styles]
[src] - Source JS and SASS that gets compiled and output to the assets directory
	[scripts]
    [scss]
...(editor/dotfiles)
CHANGELOG.md - Semantic versioning is your friend, this is as important for you as it is for others
Fluency.info.php
Fluency.module.php
FluencyConfig.php
LICENSE
README.md - This goes into high detail of all features because it also organizes my understanding of everything Fluency does
...(JS module configs and Composer file)

I spent a lot of time on this decision. Probably overthought it, and since then there are things that I would do differently- however, how it exists now has been great to work with. My approach was entirely based on what this module does now, and what it may do in the future. The focal point and goals of this approach serve these purposes:

- Fluency.info.php - Not required to separate, but helped keep the main module file cleaner and purpose-driven
- Fluency.module.php - Aims to primarily contain methods needed to integrate with ProcessWire and any methods available via the global $fluency object
- FluencyConfig.php - ProcessWire needs this to be configurable, but the amount of logic it contains certainly benefits from its own file

Things I like having lived with it for over a year now.

The directory structure is great and supports future expansion since it organizes by purpose. The specific way I structured my subfolders may not make sense for most others, but the message here is the logic behind the approach. Like I said, it seemed overengineered at the time, but now I know exactly where everything is and where everything should be in the future.

This module has heavy UI integration and the way the JS files in src/ are broken down makes it clear what everything does. Because JS can import modules from other files, it really doesn't matter how many you have. The JS files for each Inputfield has on more than one occasion enabled me to add features or fixes that would have taken much longer without the granularity I put in place.

Initially this module only supported DeepL for translation, but because of how Engines is structured as individual units, it was relatively easy to come back and add Google Cloud Translation because of how the individual units of behavior were organized and supported.

What would I do differently having lived with it for over a year now?

I would change the file names in app/ and remove the 'Fluency' prefix. That's why we have namespaces. Name collisions are handled by PHP so a simple file naming approach would have been best. This is something that I could change though without impacting the end user.

FluencyErrors, FluencyMarkup, and FluencyLocalization might be better located in a 'Support' directory or something, but that would just be applying my approach to organization rather than needing to serve a hard purpose. It doesn't cause any confusion or problems as is.

Probably some little things here or there, but not nagging enough to mention or even remember.

Anyway.

I chose a directory heavy structure with nesting which was a personal choice. There are a lot of great modules out there with different structures. A mark of good organization is asking the question "if I've never looked at this codebase before, would it be something I could get up to speed quickly and contribute without difficulty?". This may seem like a disconnected and impersonal approach, but inevitably you will feel like you need to relearn your own codebase in the future if you've spent enough time away from it.

On 5/10/2024 at 3:33 PM, MarkE said:

I am now writing a ‘wrapper’ module which will hold settings in its config and include all the init and ready scripts. 

This is a great place to start.

On 5/10/2024 at 3:33 PM, MarkE said:

I am pretty clear that it would be a good idea to bundle all the custom process modules in the directory for the ‘wrapper’ module.

In a complex enough project, it may be worth considering separate modules that are installed as dependencies. This would let you group like-kind behavior where it may also be possible that some separate functionality could stand on its own. If you can organize things to separate concerns enough, it's trivial to access other installed modules from within modules using

$this->wire('modules')->get('YourOtherModule');

and access all of its features and functionality. Since you can configure your module to have required dependencies, you can be sure that the modules you need will always be available. Organizing by purpose and features is really helpful. If the behavior doesn't provide anything useful as a standalone, then consider keeping it within the "wrapper" module. It depends on how portable you need individual behaviors to be separate from others.

On 5/10/2024 at 3:33 PM, MarkE said:

I could also put all the Page Classes there, but is it better to leave them in a separate classes directory?

I'm not sure too much on the specifics of the implementation, but these are files that perform a specific type of work and would be candidates for their own directory. There's no rule for this, but it makes sense because it gives you a single place to look for these files.

Long story short, trust yourself now and think about future you who will work on this later. You'll thank yourself for it.

Hope this was relevant enough to be helpful even though it's a late-in-game response.