Jump to content


  • Content Count

  • Joined

  • Last visited

  • Days Won


ryan last won the day on November 22

ryan had the most liked content!

Community Reputation

18,532 Excellent

About ryan

  • Rank
    Hero Member

Contact Methods

  • Website URL

Profile Information

  • Gender
  • Location
    Atlanta, GA

Recent Profile Visitors

The recent visitors block is disabled and is not being shown to other users.

  1. @Ivan Gretsky GitHub is a requirement for modules that are automated with regard to automatically pulling version and readme, etc. If someone needs a case where the module can't be managed at GitHub (like a commercial module or other specific case) then I would just ask them to contact me so I can add it manually. I feel it's safest for our users by having downloadable modules provided by GitHub. They have security measures in place that give me a lot more peace of mind than a link to a random ZIP file off on some site none of us know. Without GitHub as a middle man, I think there's a much higher probability of getting something questionable in a ZIP file, whether it's because of a hacked site or any number of other reasons. GitHub also provides great security options for protecting individual accounts, which I think further increases the safety for consumers of ProcessWire modules. The other side of it is that GitHub provides an API that we can read data from in order to ensure our module information is up-to-date and that users don't have to manage that information in more than one place. That API also includes a function to safely convert GitHub-flavored markdown to HTML, which is very useful for the modules directory. Presumably it would be possible to also implement this for other providers (GitLab was mentioned), so if there's significant demand for it then it would be worthwhile. But it took me a long time to implement the GitHub integration as it is, and at present I likely couldn't afford the time investment to implement another different API right now. Down the road I'm open to adding support for more Git providers. For now though I had to work within the bandwidth I could afford to devote to this project. @Mikie The module class name is used throughout, but it's true it doesn't literally say "class name:" anywhere. I think you are right it would help with clarity if it does that somewhere, so I'll plan to add it. As for version compatibility, I'm going through all of the PW 2.x modules and deleting any that aren't 3.x compatible and don't look like they will be. This will take me a little while, but basically the intention with the directory is that you can assume it's 3.x compatible. For more fine grained detail on version compatibility, it's going to read the module info array. It doesn't report that info at present, but this project isn't fully finished yet so it will be reporting more from the module info array in the coming weeks.
  2. There’s a new modules directory on the ProcessWire site now up and running. In this post we’ll cover a few details about what’s changed and what’s new. While last week's post introduced the modules directory, this post (part 2) focuses on newly added features for module authors (just pushed live today)— https://processwire.com/blog/posts/new-processwire-modules-directory/
  3. @adrian The modules directory does not distribute Pro modules or track their version numbers. Though I am planning to have it track version numbers for Pro modules, but that'll be added on once the rest of it is finished. Pro modules will continue to be distributed through the dedicated support boards since IP.Board manages the access for all of it. Maybe someday I can build more in that regard, but I don't have the bandwidth for that currently. The new modules directory makes no changes to the web services portion, which will continue to be provided by modules.processwire.com. I didn't want the changes to interrupt anything, as the web service gets quite a lot of traffic. @Ralf Thanks for the suggestions. I will look into it further. I have found that searching the body for modules introduces a few too much noise into the results, but including the module summary in the search might be a good middle ground. The keywords are of course a good idea too.
  4. This week I’ve been continuing to work on the new ProcessWire modules directory, this time focusing on the front-end editing aspects for module authors. This is the part that deviates the most from the existing modules directory (and makes revolutionary improvements upon it), so none of this is on the production server yet, and it’s going to take a little more time to develop. LoginRegisterPro and FormBuilder are saving a lot of time in the development. LoginRegisterPro handles the account management side, while all module editing forms are powered by FormBuilder. By next week I’m hoping I might be able to launch the new editing features so that module authors can begin to use the system, but we’ll see how it goes. This week there have also been several updates to the core dev branch, including both improvements and issue fixes. By next week I also expect to bump the version to 3.0.169. For more on the modules directory improvements, be sure to also see last week’s post.
  5. @adrian Thanks, pagination fixed. I meant to mention about non PW3 modules above, that I'm going to be removing them from the directory, or at least from the generated lists that it outputs. I don't think there's any need for the directory to maintain PW2 modules anymore, but I wanted to go through them one-by-one just in case there are any that are mislabeled as PW2, or maybe some are still actively developed and work with PW3. But the reason the new modules directory doesn't say anything about PW version is because it was going to be exclusive to PW3 compatible modules going forward.
  6. This week I’ve continued work on the new modules directory and today have launched an updated version of it on the site. This is just an initial version of it, as there’s still plenty to do. But the basics are up and running if you’d like to take a look at https://processwire.com/modules/ The new modules site is using ProcessWire’s multi-instance support to boot a copy of modules.processwire.com and to pull and manipulate data from it. In order to prevent content duplication, I’ve setup most of the pages at modules.processwire.com to redirect to their equivalent versions in processwire.com/modules/. There’s not much new here in terms of data that is being displayed, though it is also different in several ways. For starters, it finally has the look-and-feel of the main site. The modules homepage takes a different approach and lists 3 modules each in these different sections: Recently added modules Recent updated modules Popular modules Modules recently liked by users Modules in the “recently updated modules” section are those that have had recent commits at GitHub. The “popular modules” section randomly selects 3 modules that have a certain threshold of “likes” quantity and date of user activity, combined with being updated within the last 6 months. Or you can view them all (no random selection) at the dedicated popular modules page ... this section will grow as users interact with it. The “modules recently liked by users” section looks for modules that have had the “like” button clicked on recently, and puts them into this bucket. Like the “popular modules” section, this one will grow and increase in relevance once there is more user interaction with the new modules directory. There’s also a new feature on this modules site where you can maintain a “cart” (of sorts) that keeps track of which modules you have liked. In addition, the new modules site estimates how much usage each module has by analyzing request data from the ProcessWire modules web service. This is something the old modules site does not do. It uses this data solely for providing a unique sortable view of modules. Because this data hasn’t been tracked for long, it’ll increase in relevance over time. One area that I want to build out quite a bit more is the module “authors” section. Since the new modules site will be using authenticated login sessions with LoginRegisterPro, we’ll be able to maintain a lot more “profile” info for module authors (full name, bio, photo, website, anything else?) There are a few things still in development, so you won’t see them just yet: Edit module ability (it directs you to the old modules site for that) Add module ability (also directs you to the old modules site) Informational pages (how to install modules, etc.) More to come here, but that’s where it is now. Nothing all that exciting I know, but still an improvement hopefully. If you find anything that isn’t working, or have other feedback or suggestions, please let me know. Thanks for reading and have a great weekend!
  7. We got hit by hurricane/tropical storm Zeta overnight Wednesday and it took down the power lines in our neighborhood, and in many places in Atlanta. We aren’t supposed to get the electricity back till Monday, though I’m hoping they might surprise us and have it back sooner. But I’m basically offline other than my cell phone, charged in the car. I work off a desktop computer (iMac) so not able to do any kind of computer work until the electricity is back. As a result, I don’t have any updates for you this week. Though I was making great progress on the ProcessWire modules directory site up until the storm came through and we lost electricity. For now my job is to keep the flashlights working and find ways to keep what’s left of the food from going bad (since the fridge/freezer is also offline). The same thing happened with hurricane Irma, back in ProcessWire 3.0.75. It took out the power for several days. Though we weren’t in the middle of a covid hotspot back then. The deal is that Atlanta has a lot of big old trees, and all the power lines are above ground. So any time a significant storm rolls through, it knocks down a tree, the tree knocks down a power line, and the power goes out. Atlanta is pretty big, so multiply that by a few hundred trees and power lines (and a pandemic), and much of the city is out for awhile. Even a little storm sometimes knocks out the power for a few hours. That’s just one of the joys of where I live. I’m sorry about that, but since we’re supposed to be back online by Monday, I expect to have good updates for you next week. Thanks for reading and have a great weekend!
  8. ProcessWire 3.0.168 contains 16 commits relative to 3.0.167 and is focused largely on minor issue fixes and improvements, with 8 reported issues fixed and 8 improvements. This week the larger focus was on the ProcessWire modules site. I finally got some serious work done with that this week, and am building good momentum. The front-end of the modules site is being moved into the main ProcessWire site, though the back-end will remain an independent ProcessWire installation. The main ProcessWire site uses multi-instance support to boot the modules site whenever it needs to pull or update data from it. Here's a simplified example: $site = new ProcessWire("/htdocs/modules.processwire.com/"); $items = $site->pages->find("template=module, sort=-created, limit=10"); echo $items->each("<li><a href='/module/{name}/'>{title}</a></li>"); The nice thing is that I’m finding performance to be excellent here, about the same as if I weren’t booting multiple ProcessWire installations. I’m sure there’s some overhead if measured, but it sure isn’t felt. One thing I did learn is that when it comes to pagination, if you want your separately booted site to be aware of the current site’s pagination, you need to tell it the page number. Otherwise the bit of code above will always return the first 10 modules, regardless of pagination number. It seems obvious now, but it took me a minute to realize why. So if pagination is being supported, you'd add this before the $site->pages->find(...) in the example above: $site->input->setPageNum($input->pageNum); For front-end work like this, it's also a good idea to tell your booted site if you want output formatting enabled, so that page titles and such come out entity encoded, for example: $site->pages->setOutputFormatting(true); ...or if you prefer the shorter alias: $site->pages->of(true); One big difference with the new modules directory is on the management side for module authors. This part is powered by LoginRegisterPro so that now you have an account to manage all of your modules within. Further, you have the option of maintaining your module author public profile and protecting your account with PW’s two-factor authentication. That's just for starters. All of this is in the early stages of development, but if the development schedule remains as planned, I’ll be following up with more info over the coming weeks, in addition to the regular core and module updates. Have a great weekend!
  9. @alexmercenary Thanks, glad that you are liking this module (saw your post in the FormBuilder board too). Makes my day actually, thank you. This is a module that I thought was necessary to build before December, when SCA in Europe will apparently be a requirement. I wasn't able to find a way that the existing Stripe Inputfield could be updated to support 3D Secure, at least not in a way that would be reliable with the form workflow. Basically, with 3D Secure, the charge takes place at the time the user inputs the card and an independent popup verifies it. So if the card collection is part of another form (as the Stripe Inputfield is), and someone pays but never submits the form (or submits but never fixes validation errors), you end up with an orphan charge. Stripe's solution to this scenario is that you should issue a refund. I thought that was too much for people to keep track of (charges that are missing a form submission). So thought it was necessary to build this module and include it with FormBuilder, especially for any of those in Europe that might already be using the Stripe Inputfield.
  10. @rick If you needed to back out of the payment screen, I haven't yet found a way to make the browser back button play nicely here. But Stripe provides a back button for this purpose though (see the ProcessWire logo on the left where it has a left arrow and says "Back" when you hover), and this one does enable you to effectively cancel the transaction and return to the form.
  11. This week a second new module for processing Stripe payments has been added to FormBuilder. Unlike our other Stripe Inputfield for FormBuilder, this new one uses Stripe Checkout and supports 3D Secure (SCA) payments. We’ll take a closer look at it in this blog post, plus we’ve got a live demo of it here too— https://processwire.com/blog/posts/stripe-payment-processor-form-builder/
  12. Last week I told you how I was working on a getting a new Stripe payment method working with FormBuilder… and I’m still working on it. That wasn’t exactly the plan. Stripe isn’t quite as easy to work with as it used to be, or maybe Stripe thinks I’m not as easy to work with as before. Either way, I’m learning, and it’s been a good opportunity to expand FormBuilder with its own class for plugin “action” modules. Hopefully some of this work on the Stripe side can apply for other payment methods, or any type of action you’d want to take with a form submission. It’s probably going to take another week or so before this module is ready to release in the FormBuilder board, but it’s going to be built well and accompany a new version of FormBuilder too (that supports these plugin actions by way of PW modules). Having these actions as ProcessWire modules opens up new doors for FormBuilder, and I may even move some of the built-in actions (like saving to Google Sheets) into this type of module, which would be nice for hooks and maintainability. There’s not a lot to report on the core side this week. There are a few commits and improvements, but not yet enough where I’m ready to bump the version to 3.0.168. One small but useful improvement is that handling of selector queries for words with apostrophes has been improved. One of my clients noticed they were having trouble with their site search engine matching terms like “Alpe d’Huez” and “L’estello”, and so our page finding engine has been improved to narrow them down with the fulltext engine and then find the exact matches with regular expression queries. This improvement enhances most of the partial text matching operators. It also solves the issue of there being different kinds of apostrophes (ascii straight vs utf-8 curly), among other things. Overall the core is running running very smoothly on the dev branch, so I’m thinking we may try and do another merge to master before 3.0.170. Thanks for reading and have a great weekend!
  13. This week I’ve been working on updates for ProcessWire 3.0.168 as well as a new Stripe payments module for FormBuilder. In the core, the ProcessPageView module that handles all requests got a pretty significant refactoring. While it was largely focused on optimization, I also just like to go in every few years and rewrite stuff in high traffic modules like this to keep the module fresh, as well as keep my mind fresh on how it all works. I find there’s often room to do something better. The ProcessLogin module also got some upgrades this week that are primarily focused on maintaining the contents of a request URL in the admin between guest and admin user states. Now it can retain most query strings in a request URL after login. There’s still more to do in 3.0.168 so the version bump isn’t being made this week but may be ready next week. So far there are also a couple of GitHub issue resolutions and $sanitizer method improvements as well. On the FormBuilder side, we have the Stripe payments Inputfield, but since that module has been built, Stripe has released new APIs to handle SCA payments (soon required for EU transactions). With these types of payments, as I understand it, a multi-factor type process takes place where the user confirms the transaction with their credit provider. So it changes the payment flow quite a bit… enough that I’ve found the current Stripe payments module can’t really be adapted for it, at least not in a reliable way. That’s because the new API requires that the payment take place interactively before the form is submitted, since the user has to manually confirm it outside the form. So this will inevitably lead to cases where a payment has been charged but the final form isn’t submitted for one reason or another. Maybe it would work most of the time, but it just doesn’t seem like a reliable transaction flow to me. For this reason, I’m building a separate module for FormBuilder that provides a better alternative for the SCA transactions. With this module, the user submits the form, but FormBuilder saves it as a partial/pending form submission. Following form submission, the user goes through the SCA payment process with Stripe. When that completes, Stripe sends a webhook to FormBuilder that converts the form submission from pending to completed, at which point emails are sent, FormBuilder actions executed, etc. So it's Stripe that submits the final form, rather than the user. I’ve got a lot of work still to do here, but since a few people have contacted me looking for SCA support with Stripe payments, I just wanted to keep you up to date and let you know it’s in progress. The existing InputfieldFormBuilderStripe module will of course continue to be developed as well, as most regions outside the EU do not require SCA. Thanks for reading and have a great weekend!
  14. ProcessWire 3.0.167 is the newest version on the dev branch and contains the updates mentioned last week, as well as the following: Improvements and optimizations to several database fulltext index-based text-searching operators (such as *=, ~=, *+=, ~+=) so that they can better match words too short to index, as well as many stopwords. (code changes) New $input->queryStringClean() method is like the existing $input->queryString() method except that it enables you to get a cleaned up version according to rules you specify in an $options argument. I added this method primarily because it’s something I need in the core for some planned updates. But it’s in the public API because I thought it might be useful for some other cases as well. Kind of technical, but some minor improvements to the $sanitizer array methods were made so that they can now support associative arrays with a ‘keySanitizer’ option that lets you specify which sanitizer method to clean up array indexes/keys. The $sanitizer->validateFile() method was rewritten and improved plus new ‘dryrun’ and ‘getArray’ options were added. The core file and image fields have been updated with the ability to require (rather than just utilize) a FileValidatorModule for certain file upload extensions. This was motivated by SVG files being increasingly problematic (or at least my understanding of them) because they can contain the kind of bad stuff that regular markup can (scripts, loading external assets, etc.). But since SVG files are used by many just like the bitmap image formats (JPG, GIF, PNG, etc.) exploits can spread quite easily and unknowingly. I’m not aware of that ever occurring in a ProcessWire site, but it’s one of those things that strikes me as being an inevitable problem for any website (on any software) that has regular SVG uploads. ProcessWire doesn’t allow SVG file uploads by default, but you can manually add “SVG” to your allowed upload extensions for any file/image field… and from what I understand now, a lot of people are doing this—using SVG files not just during development, but as an image format that their clients upload too. The usual protections of having a trusted user admin environment don’t help much here. That’s because what matters is not whether the uploading user is trusted or not, but whether the source of the SVG is trusted. And that’s something we have no control over. It seems apparent the core could provide some extra help and guidance in this area. So ProcessWire now requires you install the FileValidatorSvgSanitizer module if you want to have SVG uploads, OR you can check a box in the file/image field settings to whitelist the extension (acknowledging you understand the risks). Now you could manually add HTML or JS as allowed upload extensions to a files field as well— and if you are rendering the resulting HTML markup or JS directly in the code of your website, then the same risks would be present as with SVG. But the risks seem obvious for doing this with HTML or JS files. Plus, why would someone do that with HTML or JS files unless for some very specific development purpose? I think it’s unlikely any of us are running websites where clients might be downloading HTML or JS files from other sources and uploading them into the website. And even further unlikely that we’d written the site’s code to include the contents of those files among the site's markup. Whereas, this is essentially what happens with SVG files, from what I understand. As I’ve come to learn this week, it’s even quite common to do this (maybe I’m late to the party). Given the above, this week I rebuilt the existing FileValidatorSvgSanitizer module from the ground up, so that it now uses a better/newer SVG sanitization library found by Adrian. I’d suggest installing the module if you are supporting SVG uploads in your site. It seems to do a nice job of cleaning the harmful stuff out of SVG files. I’m not aware of anything it can miss, but I’d still advise some caution with SVG file uploads even if you have the module installed. If you currently allow SVG file uploads but don’t really need them, just remove SVG as an upload option, as that’s still the safest bet. That’s essentially what the current dev branch version does: if you’ve added SVG as an allowed upload extension, it’ll disable it until you decide if you need it; and if you do, then install the sanitizer module or whitelist the extension. I know not everyone is going to like that, but it seems like it’s the right and safe thing to do; as well as a good strategy going forward for any other file formats with similar concerns. When our next master version arrives, it’ll be in the upgrade instructions as well. The core has been updated so that it can support the same means for any other file formats we come across in the future that might be problematic in similar ways. That’s everything I can remember that’s new to 3.0.167. Thanks for reading and have a great weekend!
  15. This week the core version remains at 3.0.166, but 3.0.167 should be ready by next week. The main reason I'm not bumping the version is just because a couple additional updates I want to get in 3.0.167 are started by not yet fully finished. Below is a look at what's been committed to the dev branch this week so far: Added custom Page class support for the Language class by implementing your own LanguagePage class that extends it (i.e. /site/classes/LanguagePage.php). Added a PR from MoritzLost with improvements to the CURL support in the WireHttp class. Added a new method to the Fieldtype interface named getMatchQuerySort() this method lets a Fieldtype module optionally manage the query when a $pages->find() requests a sort by a field/subfield handled by the Fieldtype. The first implementation of the getMatchQuerySort() method was added for FieldtypeOptions, which now lets you sort by option values or titles, despite those values and labels being in a separate table that the rest of ProcessWire doesn’t know about. Added a new getAfterLoginUrl() method to the Process module interface which lets Process modules optionally sanitize and validate request URLs to the module for a non-logged-in user. The resulting URL can be automatically redirected to once the user has logged-in. While the method has been added and implemented in several core Process modules, it is not yet used by the core after login—that will come next week in 3.0.167. Previously the only aspect of an admin URL that could survive login was an “id” integer in the query string. This week there were also several optimizations and improvements made to the PageFinder class and resolutions to 4 issue reports. Thanks for reading, have a great weekend!
  • Create New...