Jump to content

Recommended Posts

Part 1 of a 2 part Module & Service Reveal.

I'm currently working on a new module: ModuleReleaseNotes that was inspired by the work I originally did on making Ryan's ProcessWireUpgrades module "release" aware. In the end, I decided to ditch the approach I was originally taking and instead work on a module that hooked in to the UpgradeConfirmation dialog and the module edit page.

Aims

My aims for this module are as follows...

  1. Make discovery of a module's changes prior to an upgrade a trivial task.
  2. Make breaking changes very obvious.
  3. Make reading of a module's support documentation post-install a trivial task.
  4. Make module authors start to think about how they can improve the change discovery process for their modules.
  5. Make sure the display of information from the module support files/commit messages doesn't introduce a vulnerability.

Looking at these in turn...

Making discovery of a module's changes prior to upgrade a trivial task.

This is done by adding a "What's changed section" to the upgrade confirmation dialog.  This section takes a best-effort approach to showing what's changed between the installed version and the updated version that's available via the module repository.

At present, it is only able to talk to github-hosted repositories in order to ask them for the release notes, the changelog file (if present) and a list of commits between the git tag that matches the installed version and the tag matching the latest version.

It will display the Release Notes (if the author is using the feature), else it will display the commits between the tags (if tagging is used by the module author) else it will show the changelog file (if present) else it will show the latest N commits on the master branch (N, of course, being configurable to your liking.)

An example of the Github Release Notes pulled in for you, taken from Mike Rockett's TextformatterTypographer Module...

typographer-003.png.38fd13d185ec06585bb1e8064a7b07ec.png

An example of a tag-to-tag commit list from the same module...

typographer-004.png.7392c1cbddca205e517922112a64f08b.png

An example of a changelog - formatted to show just the changes (formatting styles will change)...

file-editor_004.png.fb2bedfb4e846be178ffe4bc803c51f0.png

Finally, an example of a fallback list of commits - sorry Adrian :)...

recent-commits_001.png.01e9a3e67853406fe7363c0e3639a464.png

 

Making breaking changes obvious.

This is currently done by searching for a set of configurable search strings. Later versions may be able to support breaking change detection via use of Semantic Versioning - but this may require some way of signalling the use of this versioning standard on a module-by-module basis.

For now, then, you can customise the default set of change markers. Here I have added my own alias to the list of breaking change markers and the changes section of the changelog is styled accordingly (these will be improved)...

file-editor_005.png.478eb6591d8ac5df120702676b9ffa65.png

 

Make reading of a module's support documentation, post-install, a trivial task.

This is done by making some of the support files (like the README, CHANGELOG and LICENSE files) readable from the module's information/settings screen. There is an option to control the initial open/closed state of this section...

smush_001.png.c6fe574ddf846b606e9d385ad362c9c8.png

Here is Tracy's README file from within the module settings page... 

tracy-003.thumb.png.febe8b6032626ac8f1338998c15fe673.png

 

Make module authors start to think about how they can improve the change discovery process for their modules.

There are notes in each of the sections displayed on the upgrade confirmation page that help authors use each of the features...

notes_001.png.a098fb1296fae2d1a811a335d01d2c09.png

 

Make sure display of external information doesn't introduce a vulnerability.

This is an ongoing concern, and is the thing that is most likely to delay or prevent this module's release lead to this module's withdrawl should a vulnerability be found. Currently, output is formatted either via Markdown + HTML Purifier (if it was originally a Markdown file) or via htmlspecialchars() if it has come from a plaintext file.

If you discover a vulnerability, please get in contact with me via the forum PM system.

 

Ongoing...

For now, I've concentrated on integration with GitHub, as most people use that platform to host their code. I know a few people are hosting their repositories with BitBucket (PWFoo comes to mind) and some with GitLab (Mike Rockett?) and I would eventually like to have adaptor implementations for these providers (and perhaps GitKraken) - but for now, GitHub rules and the other hosts are unsupported.

 

Links

Github: ModuleReleaseNotes

PW Module Repository: Here

  • Like 21
  • Thanks 4

Share this post


Link to post
Share on other sites

Sounds amazing!

3 hours ago, netcarver said:

It will display the Release Notes (if the author is using the feature), else it will display the commits between the tags (if tagging is used by the module author) else it will show the changelog file (if present) else it will show the latest N commits on the master branch (N, of course, being configurable to your liking.)

3 hours ago, netcarver said:

There are notes in each of the sections displayed on the upgrade confirmation page that help authors use each of the features...

It would be good to get some guidance/opinions on what is the optimal way to prepare release notes. I'd rather not have to duplicate the same information in release notes, tags, changelog and detailed commit notes, but don't really want to be "told off" by the module for not doing it either. :-)

  • Like 2

Share this post


Link to post
Share on other sites

Thanks for the encouraging feedback folks!

@Robin S There's an option to turn off the timed author-auto-shock-via-email(tm) feature which should stop it telling you off too much :)

Actually, the module gives you some options here - including doing nothing more than using git to commit your code - which we all do already. But I think I take your point; I guess the module could induce some of our users to nag us about keeping the upgrade process free of yellow notes.

Regarding guidance, I think it partially comes down to your own style working and issuing releases. For example, I commit lots of small changes - which makes my commit log pretty long and difficult to review. In my case, forcing myself to either keep a concise changelog file or to git tag my releases and then use the github release-notes feature, can be better for my module users. Even if I didn't want to start using Github's Release Notes or keeping a changelog, simply learning to add a git tag on each version increment is an improvement, as it allows the length of the commit list to be automatically limited to what is relevant.

For someone like Ryan, who tends to commit larger changes at lower frequencies, just using the git-tags feature to limit the list of commits to those between your installed version and the newest version is probably all that is needed.

 

  • Like 4

Share this post


Link to post
Share on other sites

I forgot to mention that the support file list is not limit to a single readme file. If your module has more than one, they can all be listed and read. This might prove useful for people looking for a self-hosted documenting solution. As an example, here is a screenshot of the readme files from the module itself...

mrn_001.jpeg.7a9c0fbad9e39fff4e4495d212668a95.jpeg

Which allows the documentation to be split.

mrn_002.thumb.jpeg.f5af0ba6b497ca0dd33d910038d645f7.jpeg

  • Like 1

Share this post


Link to post
Share on other sites
12 hours ago, netcarver said:

Make reading of a module's support documentation post-install a trivial task

There is the case of paid modules which tend to come with manual install option only. I'm wondering how this module could somehow support those modules as well. Easy access to the "Release Notes" would be also nice to have (post-install, in the admin). 

  • Like 1

Share this post


Link to post
Share on other sites

Netcarver is da man. I want it all and I want it now! Where is the download link.... can not wait ;)

  • Like 1

Share this post


Link to post
Share on other sites

Changelog styles now improved. Here's a changelog from the update screen - no breaking changes detected...

file-editor_010.png.e8949032c93f15a7f82c75dfea4ba8fa.png

... and here it is with breaking changes ...

file-editor_012.png.9af26bea5ede59d49978f90604b36d75.png

Raw markdown also get's labelled...

file-editor_011.png.4c3edd26e2402f13ef86ec707faf9642.png

 

  • Like 7

Share this post


Link to post
Share on other sites

Version 0.7.6 is now available with the first round of reported problems fixed. Please note, the changelog for the version of the module includes the term "Breaking Changes" so if you go through the module upgrade path for this module, you'll see breaking change highlighting in action. In this case, it's OK to proceed.

mrn_003.png.44c5ae0b9f44ff892e2c68577bbe3752.png

 

Update: version 0.8.0 is available. Adds support for code highlighting in support files thanks to Abdus' TextformatterPrism module (if it happens to be installed.)

mrn_004.png.dd560735ab102e6f49349ddb717f3632.png

  • Like 4

Share this post


Link to post
Share on other sites

This is now up to version 0.9.1 and I've moved the module's status from Alpha to Beta.

New features include showing git tag points in the commit logs...

tracy-004.png.5734987340661834b064b0a5c53cad5c.png

...better detection+handling of out-of-order versions in changelogs (plus some improved styles thanks to @matjazp) and the display of the remote repository-host's API read depletion condition.

tracy-005.png.c5e4fb1d1ad4954f00bc2a24e62b7650.png

  • Like 3

Share this post


Link to post
Share on other sites
On 11/16/2017 at 7:20 AM, szabesz said:

There is the case of paid modules which tend to come with manual install option only. I'm wondering how this module could somehow support those modules as well. Easy access to the "Release Notes" would be also nice to have (post-install, in the admin). 

Although the "What's Changed" section can't be currently displayed unless there is a remote git host to query, the module's support files are readable in the module's config page post-installation as they are all pulled form the local installation, not the remote host. 

  • Like 1

Share this post


Link to post
Share on other sites
1 hour ago, netcarver said:

better detection+handling of out-of-order versions in changelogs

I don't remember how it was before, but is this what you are expecting in this scenario? I actually think it probably is the right way to go, but just wanting to make sure you had seen this issue with AOS.

5a1444862be2e_ScreenShot2017-11-21at7_16_39AM.png.c4fb00823933449eedbfe74b07f4cb14.png

  • Like 1

Share this post


Link to post
Share on other sites

@adrian,

Thanks for posting this. It's an interesting case. Here's the markdown content of the changelog for that module...

# Changelog

### 1.6.71 (2017-11-14)
- prevNextLinks: keep page sort order


### 1.6.7.2 (2017-11-19)
- ctrl+s save fix by Robin S


### 1.6.7 (2017-11-13)
- new tweak 'prevNextLinks' to Misc, based on Macrura's 'PrevNextTabs' module (thanks!)


### 1.6.6 (2017-11-07)
- UikitTweaks: login page sticky...

You can see that I'm just honouring the changelog as it has been published. ModuleReleaseNotes simply marks up the installed version and the latest available version (which is 1.6.7.2) according to what the PW module repo reports as the latest available for the module being looked at, and what the modules code in PW reports as the installed version. I'm OK with marking up the changelog - but not with re-ordering.

BTW, if anyone has a more advanced version of a module installed locally than has been publicly published via the module's repo, you'll also see a situation like the above.

I guess I could change the label shown in this case from "Latest" to "Latest Published" or something like that.

Opinions welcome.

Share this post


Link to post
Share on other sites
7 minutes ago, netcarver said:

 

I guess I could change the label shown in this case from "Latest" to "Latest Published" or something like that.

Maybe the "Latest" tag isn't needed? Perhaps all that is relevant is the "Installed" tag. The issue with AOS is a mistaken version number 1.6.71 (I think @tpr intended 1.6.7.1). The more likely scenario is what you described where you might have a version that is newer, but not published yet - that can happen relatively often.

  • Like 1

Share this post


Link to post
Share on other sites

If possible, I would like to make it visually clear which version of the module the user will be getting once they "push the button". If I leave the label off, that won't happen.

  • Like 1

Share this post


Link to post
Share on other sites
1 minute ago, netcarver said:

If possible, I would like to make it visually clear which version of the module the user will be getting once they "push the button". If I leave the label off, that won't happen.

Not sure it's quite right, but what about "Upgrade to:" as the label? Or some other way to highlight it as what you'll be upgrading to, rather than it being the latest available?

  • Like 1

Share this post


Link to post
Share on other sites

Hmm, thing is, that is the latest available. In this case, it is an accurate description. The changelog could remove the ambiguity with an edit to list things in date order.  I suppose it would be possible to detect the dates for each entry and re-order chronologically - but that would mean enforcing a date standard + edge case code for dates as well, it's not a place I want to go.

Share this post


Link to post
Share on other sites
1 minute ago, netcarver said:

Hmm, thing is, that is the latest available.

Latest available on Github (and what you'll get if you upgrade), but not actually the latest, necessarily.

I am actually confused by the changelog in AOS - the dates for 1.6.71 a 1.6.7.2 are out of order - but again, this really is an unusual situation.

Anyway, I really wouldn't worry too much about any of this - it was really just meant as an example of things to look out for - I don't really think anything needs changing.

  • Like 1

Share this post


Link to post
Share on other sites

https://semver.org/

Quotes:

"...
Given a version number MAJOR.MINOR.PATCH, increment the:

  • MAJOR version when you make incompatible API changes,
  • MINOR version when you add functionality in a backwards-compatible manner, and
  • PATCH version when you make backwards-compatible bug fixes.

Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
...
How should I deal with revisions in the 0.y.z initial development phase?
The simplest thing to do is start your initial development release at 0.1.0 and then increment the minor version for each subsequent release.

How do I know when to release 1.0.0?
If your software is being used in production, it should probably already be 1.0.0. If you have a stable API on which users have come to depend, you should be 1.0.0. If you’re worrying a lot about backwards compatibility, you should probably already be 1.0.0.
..."

  • Like 2

Share this post


Link to post
Share on other sites

Version 0.10.0 is in the module repository.

This has a major behind-the-scenes refactoring to allow for remote repository adaptors (with result caching).

mrn_005.png.0ae0235c8a978c5be021f2789fc4060a.png

Github is supported fully, and I've made a start on the BitBucket adaptor...

formhelper_001.png.bf17dc97bd9ee8d9788e5f81589397c0.png

This version also fixes a couple of issues that have been reported by @adrian and @matjazp.

  • Like 5

Share this post


Link to post
Share on other sites

Just saw this when upgrading AOS. Doesn't seem right. I see that @tpr is using integer versions again, but 168 was also an integer, so not sure why it would be reporting like this.

image.thumb.png.72a0241fcb69a6ca84ab36e1f19cbd49.png

Share this post


Link to post
Share on other sites

Not sure if you can do anything about this. wireMailSMTP's changelog is not being parsed properly is it?

Also, you have a Tracy bd() call on line 702.

OT, but any reason you prefer: \TD::barDump() to bd() ?

 

image.thumb.png.200f04c061b06c06bb43bda466730e4a.png

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By d'Hinnisdaël
      Format Datetime fields as Carbon instances.
      You can find the latest release and the complete readme on Github.
      Installation
      composer require daun/datetime-carbon-format Usage
      All Datetime fields will now be formatted as Carbon instances instead of strings. Some examples:
      // $page->date is a Datetime field // Output format: j/n/Y echo $page->date; // 20/10/2020 echo $page->date->add('7 days'); // 27/10/2020 echo $page->date->format('l, F j'); // Monday, October 20 echo $page->date->year; // 2020 echo $page->date->diffForHumans(); // 28 minutes ago Frontend only
      The ProcessWire admin seems to expect datetime fields to be strings. This module will only return Carbon instances on frontend page views.
      Date output format
      When casting a Carbon instance to a string (usually when outputting the field in a template), the field's date output format will be respected.
      Links
      GitHub • Readme • Carbon docs
       
       
      PS. I remember reading about a Carbon module in a recent newsletter, but couldn't find it anywhere. Was that you, @bernhard?
    • By MoritzLost
      TrelloWire
      This is a module that allows you to automatically create Trello cards for ProcessWire pages and update them when the pages are updated. This allows you to setup connected workflows. Card properties and change handling behaviour can be customized through the extensive module configuration. Every action the module performs is hookable, so you can modify when and how cards are created as much as you need to. The module also contains an API-component that makes it easy to make requests to the Trello API and build your own connected ProcessWire-Trello workflows.
      Warning: This module requires ProcessWire 3.0.167 which is above the current stable master release at the moment.
      Features
      All the things the module can do for you without any custom code: Create a new card on Trello whenever a page is added or published (you can select applicable templates). Configure the target board, target list, name and description for new cards. Add default labels and checklists to the card. Update the card whenever the page is updated (optional). When the status of the card changes (published / unpublished, hidden / unhidden, trashed / restored or deleted), move the card to a different list or archive or delete it (configurable). You can extend this through hooks in many ways: Modifiy when and how cards are created. Modify the card properties (Target board & list, title, description, et c.) before they are sent to Trello. Create your own workflows by utilizing an API helper class with many convenient utility methods to access the Trello API directly. Feedback & Future Plans
      Let me know what you think! In particular:
      If you find any bugs report them here or on Github, I'll try to fix them. This module was born out of a use-case for a client project where we manage new form submissions through Trello. I'm not sure how many use-cases there are for this module. If you do use it, tell me about it! The Trello API is pretty extensive, I'll try to add some more helper methods to the TrelloWireApi class (let me know if you need anything in particular). I'll think about how the module can support different workflows that include Twig – talk to me if you have a use-case! Next steps could be a dashboard to manage pages that are connected to a Trello card, or a new section in the settings tab to manage the Trello connection. But it depends on whether there is any interest in this 🙂 Links
      Repository on Github Complete module documentation (getting started, configuration & API documentation) TrelloWire in the modules directory Module configuration

    • By David Karich
      ProcessWire InputfieldRepeaterMatrixDuplicate
      Thanks to the great ProModule "RepeaterMatrix" I have the possibility to create complex repeater items. With it I have created a quite powerful page builder. Many different content modules, with many more possible design options. The RepeaterMatrix module supports the cloning of items, but only within the same page. Now I often have the case that very design-intensive pages and items are created. If you want to use a content module on a different page (e.g. in the same design), you have to rebuild each item manually every time.
      This module extends the commercial ProModule "RepeaterMatrix" by the function to duplicate repeater items from one page to another page. The condition is that the target field is the same matrix field from which the item is duplicated. This module is currently understood as proof of concept. There are a few limitations that need to be considered. The intention of the module is that this functionality is integrated into the core of RepeaterMatrix and does not require an extra module.
      Check out the screencast
      What the module can do
      Duplicate multible repeater items from one page to another No matter how complex the item is Full support for file and image fields Multilingual support Support of Min and Max settings Live synchronization of clipboard between multiple browser tabs. Copy an item and simply switch the browser tab to the target page and you will immediately see the past button Support of multiple RepeaterMatrix fields on one page Configurable which roles and fields are excluded Configurable dialogs for copy and paste Duplicated items are automatically pasted to the end of the target field and set to hidden status so that changes are not directly published Automatic clipboard update when other items are picked Automatically removes old clipboard data if it is not pasted within 6 hours Delete clipboard itself by clicking the selected item again Benefit: unbelievably fast workflow and content replication What the module can't do
      Before an item can be duplicated in its current version, the source page must be saved. This means that if you make changes to an item and copy this, the old saved state will be duplicated Dynamic loading is currently not possible. Means no AJAX. When pasting, the target page is saved completely No support for nested repeater items. Currently only first level items can be duplicated. Means a repeater field in a repeater field cannot be duplicated. Workaround: simply duplicate the parent item Dynamic reloading and adding of repeater items cannot be registered. Several interfaces and events from the core are missing. The initialization occurs only once after the page load event Attention, please note!
      Nested repeaters cannot be supported technically. Therefore a check is made to prevent this. However, a nested repeater can only be detected if the field name ends for example with "_repeater1234". For example, if your MatrixRepeater field is named like this: "content_repeater" or "content_repeater123", this field is identified as nested and the module does not load. In version 2.0.1 the identification has been changed so that a field ending with the name repeater is only detected as nested if at least a two-digit number sequence follows. But to avoid this problem completely, make sure that your repeater matrix field does NOT end with the name "repeater".
      Changelog
       
      2.0.1
      Bug fix: Thanks to @ngrmm I could discover a bug which causes that the module cannot be loaded if the MatrixRepeater field ends with the name "repeater". The code was adjusted and information about the problem was provided 2.0.0
      Feature: Copy multiple items at once! The fundament for copying multiple items was created by @Autofahrn - THX! Feature: Optionally you can disable the copy and/or paste dialog Bug fix: A fix suggestion when additional and normal repeater fields are present was contributed by @joshua - THX! 1.0.4
      Bug fix: Various bug fixes and improvements in live synchronization Bug fix: Items are no longer inserted when the normal save button is clicked. Only when the past button is explicitly clicked Feature: Support of multiple repeater fields in one page Feature: Support of repeater Min/Max settings Feature: Configurable roles and fields Enhancement: Improved clipboard management Enhancement: Documentation improvement Enhancement: Corrected few typos #1 1.0.3
      Feature: Live synchronization Enhancement: Load the module only in the backend Enhancement: Documentation improvement 1.0.2
      Bug fix: Various bug fixes and improvements in JS functions Enhancement: Documentation improvement Enhancement: Corrected few typos 1.0.1
      Bug fix: Various bug fixes and improvements in the duplication process 1.0.0
      Initial release Support this module
      If this module is useful for you, I am very thankful for your small donation: Donate 5,- Euro (via PayPal – or an amount of your choice. Thank you!)
      Download this module (Version 2.0.1)
      > Github: https://github.com/FlipZoomMedia/InputfieldRepeaterMatrixDuplicate
      > PW module directory: https://modules.processwire.com/modules/inputfield-repeater-matrix-duplicate/
      > Old stable version (1.0.4): https://github.com/FlipZoomMedia/InputfieldRepeaterMatrixDuplicate/releases/tag/1.0.4
    • By picarica
      so i am trying to put CustomHooksForVariations.module, a custom module, i am placing it into site/modules direcotry yet my modules page in admin panel gives me errors
      so this is the screen show when i refresh modules, i dont know why the shole hook is written on top of the page :||

      and this next image is when i try to install it, i saw that it is not defiuned modules.php but it shouldnt need to be ?, any ways i dont want to edit site's core just to make one moulde work there has to be a way

    • By Robin S
      A new module that hasn't had a lot of testing yet. Please do your own testing before deploying on any production website.
      Custom Paths
      Allows any page to have a custom path/URL.
      Note: Custom Paths is incompatible with the core LanguageSupportPageNames module. I have no experience working with LanguageSupportPageNames or multi-language sites in general so I'm not in a position to work out if a fix is possible. If anyone with multi-language experience can contribute a fix it would be much appreciated!
      Screenshot

      Usage
      The module creates a field named custom_path on install. Add the custom_path field to the template of any page you want to set a custom path for. Whatever path is entered into this field determines the path and URL of the page ($page->path and $page->url). Page numbers and URL segments are supported if these are enabled for the template, and previous custom paths are managed by PagePathHistory if that module is installed.
      The custom_path field appears on the Settings tab in Page Edit by default but there is an option in the module configuration to disable this if you want to position the field among the other template fields.
      If the custom_path field is populated for a page it should be a path that is relative to the site root and that starts with a forward slash. The module prevents the same custom path being set for more than one page.
      The custom_path value takes precedence over any ProcessWire path. You can even override the Home page by setting a custom path of "/" for a page.
      It is highly recommended to set access controls on the custom_path field so that only privileged roles can edit it: superuser-only is recommended.
      It is up to the user to set and maintain suitable custom paths for any pages where the module is in use. Make sure your custom paths are compatible with ProcessWire's $config and .htaccess settings, and if you are basing the custom path on the names of parent pages you will probably want to have a strategy for updating custom paths if parent pages are renamed or moved.
      Example hooks to Pages::saveReady
      You might want to use a Pages::saveReady hook to automatically set the custom path for some pages. Below are a couple of examples.
      1. In this example the start of the custom path is fixed but the end of the path will update dynamically according to the name of the page:
      $pages->addHookAfter('saveReady', function(HookEvent $event) { $page = $event->arguments(0); if($page->template == 'my_template') { $page->custom_path = "/some-custom/path-segments/$page->name/"; } }); 2. The Custom Paths module adds a new Page::realPath method/property that can be used to get the "real" ProcessWire path to a page that might have a custom path set. In this example the custom path for news items is derived from the real ProcessWire path but a parent named "news-items" is removed:
      $pages->addHookAfter('saveReady', function(HookEvent $event) { $page = $event->arguments(0); if($page->template == 'news_item') { $page->custom_path = str_replace('/news-items/', '/', $page->realPath); } }); Caveats
      The custom paths will be used automatically for links created in CKEditor fields, but if you have the "link abstraction" option enabled for CKEditor fields (Details > Markup/HTML (Content Type) > HTML Options) then you will see notices from MarkupQA warning you that it is unable to resolve the links.
      Installation
      Install the Custom Paths module.
      Uninstallation
      The custom_path field is not automatically deleted when the module is uninstalled. You can delete it manually if the field is no longer needed.
       
      https://github.com/Toutouwai/CustomPaths
      https://modules.processwire.com/modules/custom-paths/
×
×
  • Create New...