Jonathan Lahijani

Thanks for the reply. The purpose of my question is ultimately related to priming my ProCache cache. Right now, I have a whole bunch of optimizations in place, but when I deploy my site, I purposely clear all cached pages as they need to be rebuilt. If I have a script to primes 50 pages that each take 20 seconds to prime, one after another right after deploy, I'm wondering to what extent this makes the site inaccessible during the priming process. My question is what is it that exactly causes a site to lag for one person while another person (or my priming script) is loading intensive pages?

Let's say I have a page A that takes 20 seconds to load due to the number of database calls being generated. And then I have another page B that takes 2 seconds to load because it's less database intense. If person 1 goes to page A, the server will start the page rendering process and 20 seconds later the page will be served. If person 2 goes to page B right after after person 1 went to page A, will person 2 see the page B in 2 seconds or 22 seconds? Meaning, what factors go into play in terms of how Apache/PHP/MySQL responds multiple large requests, server resources (cores, memory), etc.? This is something I never really understood but I'm doing some big performance optimizations and it would be helpful to have an understanding what's going on deep down. Any resources would be appreciated.

The TCC site actually uses aMember Pro which is what I integrated with (the handshake). I didn't set up aMember Pro originally, but I believe TCC likes the system, although there has been some concern with the recent Russia/Ukraine situation as there are developers from both countries that are involved in the project (and could lead to potential support issues should they arise given the displacement of many people). Creating the handshake was straight-forward and didn't involve too much code. I could give you some tips as to how I did it should you go that route, but in short, when a member logs in via ProcessWire's admin form, a hook it hits a script on the main site (via a get request) that bootstraps aMember's system and runs a check on the credentials entered.

Over the course of 2021, I re-developed the Transferware Collectors Club Database of Patterns and Sources. It is a private, members only database that contains nearly 20,000 patterns of various types of transferware, which in layman's terms is basically art on plates, bowls, cups, etc. (Wikipedia) Many archaeologists and transferware enthusiasts use this database to learn more about the historical nature of their collections or archaeological finds. The system had a long history and there were two versions prior. I believe the first iteration was made with a PHP framework of some sort, then it was re-developed around 2015 in ProcessWire. One special aspect of the previous version was that it utilized Elastic Search. Advanced search capability of the system, both in terms of free-form text entry as well as filters were the crux of the system. In 2020, I was introduced to this project as the previous developer decided to retire. While doing some various maintenance type work, I realized that there was much that could be done to improve the system, from re-imagining the user interface to utilizing all the latest features in the newer versions of ProcessWire, as it had been locked to version 3.0.135 or something at least 3-4 years old. Also, Elastic Search had become deeply outdated and unreliable since the entire site had been moved to a new server. Finally, I saw various aspects of the data model that could be improved as well as reduce several dependencies, while also using new techniques like HTMX. I redeveloped the system over the course of 2021 and the end result has greatly impacted the capabilities of the system as well as its ability to be maintained with little complication years into the future. Search: First, it became apparent that the way in which Elastic Search was being used had little benefit, became unstable and was outdated in the time since it was implemented. I don't like relying on 3rd party solutions, especially for something so critical and I thought about how I could develop a great search experience with ProcessWire alone. To do this, I utilized the WireWordTools module to handle lemmatization and such. Then, I advocated for this feature to be developed into the core, which could allow the results from multiple search queries to be easily stacked one after another and not affect order or pagination. This was a key change allowed for "ranked" search results. Finally, I integrated the BigHugeThesaurus API so that entered search terms can have their synonyms automatically generated. Filters: The previous version of the system had filtering capability, but it was too cumbersome and in my opinion, just didn't feel right. A significant amount of thought went into how to improve it while untangling some issues with the data model (and just understanding what Transferware is in the first place). The end result has has been an extensive set of filters with a special user interface so that users can easily select what they need. Frontend: I take the "classic", server-rendered approach when it comes to web-apps, and during this time, HTMX started becoming a favorite among such developers. By using HTMX, searches can be conducted without having to hit the submit button every time. It gives it that polished ajax-y feel without having to write any JavaScript whatsoever and allowing the use of ProcessWire in the expected way as opposed to it acting as some API end-point. To be able to do this is somewhat of a game-changer and I feel like I'm not "missing out" on these features which in the past were a pain to have to invent as there really wasn't a clean, straight-forward way to do it. HTMX is a gem and it couples very nicely with ProcessWire. I'm also using Alpine.js lightly for toggling things here and there. My goal was to make this project jQuery-free. UIkit 3 is being used as the frontend CSS framework. User Authentication: As mentioned, the database must be accessed after a membership has been purchased. TCC already has a 3rd party membership system, so I developed a handshake that allows a user to log in with the ProcessWire admin form using their credentials from that 3rd party system. If it's the member's first time logging in, it will then automatically create a ProcessWire account for the member so they can bookmark patterns, save searches, etc. The previous version had a different type of handshake that ultimately only logged the users into one master "member" account, so users wouldn't be able to save any information since everyone technically shared the same account. New Server: I put the site on its own dedicated droplet to squeeze out even more performance and be independent of their main marketing site (based on Drupal). We had a couple instances of configuration changes causing some clashes, so this move isolated it from any potential breaking changes that may occur on the main site. A 2-core droplet is all that is needed for now, which is cost effective. Demo Video and Article: While the site can't be visited given it requires an account, I did create an in-depth search tutorial video which shows off quite a bit which I can post here since it's publicly accessible: https://player.vimeo.com/video/666912411?h=30a94132db Feel free to read more about the TCC DB in this interview with Dr. Loren Zeller, who I worked closely with the entire time and who provided invaluable feedback and guidance along the way: https://www.transferwarecollectorsclub.org/sites/default/files/pdf/special-interest-pdf/0225_d_001_analink.pdf

Thanks for your response. In my case it was definitely the check_access option. To figure this out, I had to diff the MySQL queries as superuser vs. non-superuser and see where the bottleneck was and it had to do with calls to the pages_access table on a particular query.

Ack, well it had to do with 'check_access' not being set to zero in my selector. I don't think I've ever had to deal with that before. https://processwire.com/docs/selectors/#access_control

I'm having an extremely weird issue that I can't pinpoint in my code. When a particular $pages->find is done as a superuser, the search is quick. However if the user doesn't have the superuser role, it's very slow (over 15 seconds). There's no code in my codebase that would cause this. Any ideas would be appreciated.

@OllieMackJames I've experimented with most of the popular page builders in WordPress, including Oxygen. The direction I'm taking is vastly different from all of them. Simply put, the page builder being based on RepeaterMatrix is what I'm trying to accomplish and for it to be usable with the developers CSS framework of choice (Uikit, Tailwind, Bootstrap, Codyframe, no framework, etc.). Anything beyond that is outside my skillset and the scope of what I'm truing to do. I have no interest in using React, Vue, etc. and going to the extreme like the WP page builders do. The page builder will simply be "good enough" to make marketing pages based on the capabilities of whatever CSS framework is installed.

I just picked up my Model Y today. I've been driving my previous car for ~15 years. Here's a photo of the first webpage I loaded in the browser 🙂

Just wanted to add my 2 cents and say that a database migration system (like Ruby on Rails... they've had it perfected since 2005 or so) takes ProcessWire from being a CMS/CMF to something more web application framework-like, at least from my point of view. That's a defining feature the way I see it and what I believe Bernard is shooting for (I haven't experimented with RM yet). Personally, I do everything by hand the way Ryan described (because I'm impatient and it's fast enough), combined with little 1-off scripts that modify big chunks of data as needed, but that approach will fall apart when there's multiple developers involved, syncing changes, or even re-implementing your own changes on a production site that were originally done on a development site. I do wonder if I would use a migrations feature if it were native to ProcessWire. Right now, I rarely even use Field/Template/Page Exports when making field/template/page changes from dev to production, but I definitely understand the use case (having worked with web applications frameworks extensively). While having a database migration system is the more 'proper' and 12-factor-y way to do complex development, I don't personally view ProcessWire as a web application framework like Laravel and Rails. There's something to be said about being able to throw ProcessWire around and experiment with things quickly. It has had a real impact on my productivity and solutions. Hand-writing every field or template added or changed would be tiring (although it would optional). Having it auto-recorded like CraftCMS would be interesting and there have been attempts to do that. Not sure where I'm going with this, but just some thoughts I felt like sharing.

Funny you say that because I wrote a similar list as well. Here it is with some added commentary. As you can see I solved some things but listed them anyway and other things I didn't list because I solved them. general when clicking something in an unfocused window, the first click will focus the window; another click must be made to actually do something in the window I didn't realize how much this annoyed me. I thought Front and Center fixes this but it doesn't. finder (and Finder replacements) can't move or even remove the finder icon in the dock I don't want Finder to be first; to counteract, I've added a bunch of spacers so it's visually separated from my core programs (vscode, chrome, forklift, iterm) there is no "cut" option in finder (must copy, then use the move/paste with command + alt + v -- requires 2 hands) can't have custom categories in finder (just favorites, locations, icloud and tags) can't have the main folders load from a custom location like you can in windows; must use symlink instead this messes up how Finder search works hitting 'enter' on a file or folder does rename instead of launching the file / entering the folder (naturally, 'enter' seems like the right key to hit to ENTER a folder, not rename it) this means i have to use whatever the keyboard shortcut macOS wants you to use, and I don't remember what it is, but I remember I must use 2 hands -- ridiculous! the files list does not refresh automatically if a new file was written somewhere else (!) I use ForkLift as a replacement to Finder, but nothing in the macOS ecosystem comes even close to XYplorer (I used that for 13 years and it's updated constantly; it's like the ProcessWire of file managers... a Swiss Army Knife) mouse mouse wheel uses acceleration which makes sense for touchpads and mice without an actual WHEEL; it's not linear like you would expect counteract with https://github.com/emreyolcu/discrete-scroll the fact that macOS uses acceleration based scrolling for notch-wheel mice is ridiculous. that doesn't make any sense. macOS mouse movement physics is weird (at least compared to Windows) using outdated steelseries exactmouse to counteract that program is from 2010 (!); there's nothing more recent! no middle-click + scroll use "AutoScroll" extension in Chrome; it doesn't have the same feel like in Windows chrome can't open a new tab inside a chrome app like you can in windows; must first focus a regular chrome window other software Transmit (the SFTP client) doesn't support folder bookmarks (really wish it did so I don't have to use FileZilla; I don't want to use ForkLift for SFTP)

M1 Mac Mini My first Mac. I love the hardware, but macOS is just not for me and it does some idiotic UX things that I can't stand, to the point where it's become a deal breaker. May go back to Windows and keep this as a second machine or just switch to Linux (Fedora KDE most likely); will probably stop using WSL2 and set up a dedicated Linux server for development (I don't like working with virtual machines). LAMP stack (installed and configured everything with Brew); everything ARM-based VSCode with various plugins ProcessWire with various modules (Profields being a must) + my super module (still developing it; lot's of breaking changes until the dust finally settles) I too set up a $config->env variable Laravel Mix for frontend asset management Hosting: DigitalOcean with Ubuntu Server (but considering Fedora Server in the future)

The upload limit is set by php.ini's 'post_max_size' setting. If you are using php-fpm, make sure to restart php itself so that the setting takes effect: sudo service php7.4-fpm restart

I believe I hit this issue in 2014 with Windows 7 and WAMP server. This thread helped me. I believe I commented in there too: https://stackoverflow.com/questions/10750964/wamp-is-running-very-slow

What version of PW are you running? What is your server setup? Be specific.