ryan

@Jeffro: Great to have you here! Thanks for joining the community, and thanks for writing about ProcessWire. @Joss: Just read your comment posted there and that is epic, you always have a way with words. I wish I had your gifts with words... composition... voiceovers... humor.... plants... and who knows what else you've mastered, but just glad you are here!

Great link Diogo. Thanks to you guys that replied to the article. It would really be quite an upset if a WordPress cheerleading site named WP Tavern came out with article saying they were switching their publishing platform to ProcessWire, wouldn't it?

Btw, if you are on WHM/cPanel, moving accounts (to another WHM/cPanel) setup is really easy. Pretty much a 1-click process. So if whatever change you make involves moving to a server where they can't migrate it for you, just use the migration tool built into WHM which has worked great in my experience. I used to host some clients on my own VPS and when they grew and moved to their own dedicated server, this migration tool made life easy.

If there is any evidence of it being a broader trend that extends beyond your experience, we'll look at it. But as far as I can tell it is an isolated issue. I'm just sorry you are the one to experience it. The growth has been slow and steady over a very long time. I'm not aware of any major new changes in growth there, other than acquiring the Amsterdam data center a couple years ago. I agree, that does seem like a long time. Without knowing the details of exactly what the problem was or how far it spread, I don't think I could analyze how long it should take to fix though. If you want to know why, definitely ask them about it. They are pretty straight up with these kinds of questions. We are on a dedicated server in the Reston data center. If they are able to share a dedicated server, it would be news to me. I like the way you are thinking though. But if I were you I'd make a more major change so that you can get off the bad luck train. Try out the LA or Amsterdam data center at least. Reston data center has been great to us, but not so good to you, so change up the location or even the host, otherwise you or your clients' minds will still be on outages and thus attracting them. I think your affected clients would also appreciate your effort in making those changes. But before you decide anything, just call them, tell them what you've run into, that you need a big change, and ask them what they suggest. They will also be the best ones to advise on your redundancy questions. I've found them to be very knowledgeable on this stuff.

Matthew, sorry to hear you are continuing to have outage issues. I always feel obligated to reply here because I stake my name and reputation on ServInt by recommending them here on the site, and that recommendation comes with a lot of long term experience with ServInt and numerous other hosts. I feel badly that you've not had a good experience. The only outages I've seen were 10 years ago, the latest Reston one, and that's it (though both went largely unnoticed by my clients). Beyond this site, I've got pretty much all of my clients at ServInt, in both Reston and LA data centers, all with their own dedicated servers and/or VPS accounts. I have experience with more than a dozen servers there over the last decade. I have also dealt with numerous other hosts over that time period. My experience has led me to trust ServInt very strongly relative to others, and I consider them the best in the business. I'm stating this not to contradict what you are saying or take anything away from the unfortunate experience you've had–it definitely gives me pause. Rather, I'm stating it to say that I think your experience may be unusual and unique. At least I hope that's the case. If it turns out to be a broader trend, we'll act on it, but I don't see evidence of that. Sometimes when things get started on a negative path, it's hard to get off of it (like the law of attraction), and you've got to throw a wrench into it by making a change. I would encourage you to call them and communicate your experience and ask them to make things right. Since your clients that use this server are in Europe, a switch to the Amsterdam data center seems to make sense. But just change something, whether as little as asking them to change the host node, or as major as switching to an entirely different data center (or even web host). I hope you will communicate your experience directly to them, because I don't think they will ever see this thread, which is unfair both to you and them. The site http://www.webhostingtalk.com/ is probably a better place for this discussion because I know they do sometimes read that and may have the opportunity to respond to your experience and make things right (something that can't be done here). Your best bet is just to call them though.

Both. The goal with ProcessWire is to be attractive web designers/developers with programming skills and those without programming skills. ProcessWire is primarily a CMF, somewhere in between a framework and a CMS, but with the ability to be completely one or the other as needed. The admin UI is an application on top of the API. No action in ProcessWire actually requires the admin UI. The admin UI is there to help you. If you don't like that kind of help, you are likely in the minority, but ProcessWire still has you covered. However, if you want to work purely in code you should also look beyond CMF/CMS and towards dedicated frameworks like Laravel. ProcessWire is fundamentally different from any of the CMSs you mentioned, and if you don't realize that yet then I'd encourage you to get to know the system better, and we're all here to help. While our system is fundamentally different, the web designer/developer side of our audiences are similar to those CMSs. Speaking outside of anything specific, we can't ignore what are likely our largest audiences. Perhaps in your system. Whatever limitations your opinion is based on are not present in ProcessWire. I put together a proof-of-concept here when the request came up a few messages above and I can assure you it works quite well. If there's enough demand for it, I'll finish and migrate it to the core. Though if it's something that less than 30% of people will actually use, then it doesn't belong in the core. This is why I'm more enthusiastic about supporting specific-audience features like this by extending API calls and hooks that help other module developers build these tools. Even when tools like this exist in a stable state, I see their value as being for migration and versioning, not for configuration. I don't believe any developer would choose to configure the rather comprehensive aspects of templates and fields from a text file. ProcessWire is more powerful than the systems you've mentioned. The options available to configure for any given field/template are not predefined like they might be in other systems or frameworks. The options vary broadly depending on what Fieldtype and Inputfield is in use (all of which are themselves plugin modules that define their own options). Options can also be affected by other plugin modules. Dependencies are also in place, meaning the presence and requirements of some configuration options can depend on the values already present in other options. Further, some Fieldtypes can ride on top of others (like some of the ProFields), maintaining their own configuration along with those of their host. All of these things lend themselves extremely well to a user interface, and greatly benefit the developer in many ways, especially with regard to instruction and documentation. All of these things can be configured from the API, and could certainly be configured from a text file. But I would consider myself very lazy and remiss in my responsibilities if I expected people to use text files (YAML or otherwise) as the primarily method of configuration for this. I sure wouldn't be a happy web developer having to use that. On the other hand, if you are dealing with a system that is limited to predefined options that are self documenting, then I'm sure a text file for configuration is just fine. But there's little point in comparing ProcessWire to such systems, because ProcessWire provides a level of power that goes far beyond that, and far beyond what I think anyone would find desirable to configure from text files. We use a user interface here for exactly what a user interface is meant for.

I'm not aware of anything broken with the caches in 2.4.8, though there was an issue in 2.4.6 that has been fixed. However, it looks like your MySQL version does not meet the minimum requirements. You mentioned you are using 5.0.8-dev, but the minimum required MySQL version is 5.0.15. If I recall, 5.0.8 is the version of MySQL that had some major issues with sorting. I'm not positive on that, or even sure if that MySQL version is the reason why you got an error, but it seems like the most likely possibility so far. If you are able to reproduce the error again, you might try enabling debug mode so that you could see if any other error messages get output. But chances are that there will be incompatibilities between PW and versions of MySQL prior to 5.0.15.

You are right–I'd forgotten we had that in the htaccess. Yes, I'd leave it for now. I just wanted to point them out because I think these will be beneficial for this module once 2.5 is stable. I agree, you don't need it for now. My default is always to double up on security, but thinking through it more it's probably not necessary here. I mention it as a possible future addition though just because the URLs hitting a website aren't always confidential. The token is only as private as the logs. For most of us, that's a non issue. For some it's a potential ddos entry point, but only if the token gets in the wrong hands. I think what you've got is just right for the majority, and if someone needed something more, like an IP limiter, then probably better to leave it to them to add in rather than making everyone else fuss with it. Sorry, I missed that wget was already there. There may be some benefits to having the cron job run the PHP File directly, but it would be more difficult for the user to setup (creating executable PHP shell scripts and such). Also, having initialization of the job URL accessible makes it easier for people to use external CRON services. As a result, I think sticking to the method you are using is better. Thanks for adding to the modules directory!

This looks fantastic djr! I've had a need for something exactly like this and will definitely look forward to using it. I took a quick look through the code and think it looks very well put together. I do have a few minor suggestions: Rather than backing up the DB to the root path (where it is temporarily web accessible) I'd recommend backing it up to a non web accessible directory, like /site/assets/cache/. Likewise for the tar/gz file. Beyond just looking for "runbackup" in the request URI, I recommend designating a page that it will only run on. For instance, if you wanted it to only run on the homepage: $shouldBackup = $page->path === '/' && (strpos($_SERVER['REQUEST_URI'], self::RUN_BACKUP_PATH) !== FALSE) && $this->wire('input')->get->token && $this->wire('input')->get->token === $this->token; This might be a good module to experiment with conditional autoloads. In your getModuleInfo, you can do this: 'autoload' => function() { return (strpos($_SERVER['REQUEST_URI'], self::RUN_BACKUP_PATH) !== FALSE); } In truth, conditional autoloads are more reliable in PW 2.5 (a few minor issues have been fixed) so this may be a v2 kind of thing as well. In PW 2.5, you can also isolate the entire getModuleInfo() to a separate ModuleName.info.php file. Beyond just the token, it might be worthwhile to have an IP address limiter since there's a good chance one's CRON job is always going to be coming from the same IP. Though not sure it's totally necessary. In your docs file, I would mention if possible what command you recommend for the CRON job, for instance: wget quiet no-cache -O - http://www.your-site.com/runbackup?token=abc123 > /dev/null Lastly, might be good to mention that it requires exec/system access for executing commands (many hosts have these disabled by default, but there's usually a way to enable them). Please add to the modules directory when ready! Thanks for putting this together!

Matthew, why were you offline for 4+ hours? This morning's outage was an emergency update planned for all servers, not just a small number. Is the outage you are talking about different from the one that was planned? I'm just curious if your outage is related to the emergency update or if it was something different entirely? In either case, it sounds like your host node has some bad karma or something. Maybe it's worth asking to be switched to a different node in the data center. Just make sure you are communicating directly with them through the portal (or by phone), and not Twitter or other social networks. They are extremely helpful and knowledgeable, but you have to use their secure channels to communicate because they have to maintain client confidentiality. Another thing to consider is that any kind of maintenance that would require taking down a server is something they usually do overnight or early AM, since that's the least busy time (though this kind of maintenance is rare). But in your case, your clients are in Europe (I think?). Whether for scheduling or ping times, maybe it would be beneficial to be in their Amsterdam data center rather than the Reston, VA data center?

Matthew, I got an email about this planned outage last week (planned 15 mins between 5 and 7 am), as did all my clients. Are you sure ServInt has your correct email address on file? The outages are just to reboot the servers (kernel patch only seen after server reboots), and I read the 15 minutes as more like a maximum. If your server is down it sounds like that's something different, you should submit an urgent ticket or call them. It's worth noting that they are fixing a vulnerability that affects most web hosts, not just ServInt, and appears they are fixing it before everyone else. Please let us know what you find out from them as to why your server isn't online.

I've updated this module to version 3 which adds the ability to use headlines as page breaks and adds support for several shortcodes.

You are not already screwed if your writable files aren't executable or don't represent files that can blow up the site. Obviously, the intention is that nobody else can write to the writable files, but the reality is when designing a system we have to plan for the worst. Your environment and my environment may be well secured, but I try to think about the bigger picture out there where there are a lot of insecure environments. Security is #1 when it comes to ProcessWire. What are secure permissions and what are not is dependent on the server and not something we can detect. A security best practice is to get everything important out of any writable files on the file system and not have any single writable files that could take down the whole system. Meaning, I think it's acceptable for cache files, image and other independently served assets to exist as writable files, but the files we're talking about here are definitely out. Another reason we can't detect this stuff is because there are plenty of hosts running phpsuexec and the like where apache runs as the user and everything is writable by default. This tends to be okay, so long as there isn't also a copy of WordPress somewhere on the account. Basically, what are the right permissions for a given site comes down to the server, environment and host, and not ProcessWire. If someone is able to write something to some file on your account, that does not imply they can access the DB. That "someone" is usually an automated script and not a real person, or it's one of your account neighbors on a shared webhost. One of the things that regularly falls in my lap (for better or worse) is to fix hacked WordPress sites. Rarely has the DB been touched, while most of the writable file system has to be quarantined. I disagree. There is a direct connection of field schema to the data represented in the DB. Template files are not coupled to fields like that because you are talking about your code, not the system. You have to make your changes independently regardless of where they are stored. Changing a field doesn't automatically change a template file. But changes to fields are an end-point in terms of how they reflect on the database. I've run out of time for today so have to stop. But I want to be clear that PW is not going to regress to storing this stuff on the file system. I recognize there is a benefit to some developers in having files they can version control or migrate independently. I personally would find the drawbacks to far outweigh the benefits (I have to support this project for a large audience), but understand why some find it desirable. Like I've mentioned, I would consider in the future mirroring the data to files that you can version control and selectively migrate in the admin. I believe that would bring the benefits of what's been discussed here, without the drawbacks.

At some point, I'll setup fields and templates to maintain mirror copies of files on the file system for those that want to version control them that way. They are in the database for very good reasons, but that doesn't mean that they can't be on the filesystem too. It's just that the database is the better default place for them to live and be maintained from for the vast majority. I don't consider writable files particularly trustworthy in the grand scheme, so the master source would always have to be the DB, but the mirror files could certainly be monitored, versioned and migrated at the developers discretion. I have no problem with that so long as we're not letting the file system lead. Something to add here that I think gets overlooked is that PW is not just a development tool. In the majority of cases, the development phase is just a very short blip in the overall timeline of the project developed with ProcessWire (whether a website, application or whatever it is). While we put a lot of focus on the development tool aspect of PW, the big design decisions are made for the entire lifecycle, not just the development phase. CMSs have to be written for large scale realities and diversity across hosting providers, varying amounts of security and expertise levels. While files here seem beneficial from one perspective, that does not translate to a net benefit in the larger context. In fact it translates to something rather undesirable in the larger context. Having important data in writable files on the file system is something you try to avoid with a CMS. While they are a necessary evil in some cases, if you can keep writable data in the DB, it's going to result in stronger security over a broad installation base and over a broad time period. These writable files are often the weakest link on shared hosting accounts. They can be perfectly secure, but there's little doubt the DB is safer. When it comes to data that needs to be editable, I consider the DB a much more trustworthy storage mechanism for important data across a large set of installations. I'm stating the diverse reality of our big picture context and not any individual's server. Some of us are running on servers where it would make no difference at all from a security aspect, but that's not something we can count on. Outside of the situations mentioned in this thread, I think most would not find it desirable to have the field and template data that disconnected from the content it represents. I can imagine some real headaches with schema getting disconnected from the data. When I backup my DB, I like to know that I've got everything necessary to put it back together without having to chase down multiple files with multiple tools, not to mention possibly files in different versions. I don't want to open the door to having having schema files that are not consistent with the data that they are supposed to represent. Data loss is always a possibility with schema changes and should always be accompanied by a confirmation. Automation by movement of schema in files (whether by git or SSH, FTP, etc.) is problematic for a lot of reasons. The issue described about one person's changes overwriting another's may be a potential real world case, but is a non-issue for most of us because we don't migrate those kinds of changes at the DB level, nor do I recommend doing that. I understand that there are challenges for a team of developers having multiple versions of schema, or developers that want to migrate schema changes to a live server in an automated fashion rather than re-playing those changes themselves. I actually think less automation is ultimately a safer solution here, even if not as convenient for some. Though I'm still very enthusiastic about continuous integration projects and doing anything I can to support them. But I do not support moving storage away from the DB as our primary storage for these things. I understand using the file system may seem like a smart choice in certain contexts (and I don't disagree on them), but in the larger context it's not a wise choice. I'll do my best to find ways to mirror the data to files for those that might want this or may benefit from it.

I'm not exactly sure what the issue was, as I wasn't able to reproduce it locally. But I did make several adjustments in the Modules class that I think might help (just now committed to dev). I'm curious to know if it resolves the issue you guys were experiencing there? I've also updated the DynamicRoles module to use info.json files rather than a getModuleInfo() method. I should have done that in the first place since DynamicRoles extends another module (ProcessPageType), so it's also possible the issue was related to that. I would suggest upgrading the core first, and then the Dynamic Roles module if you still see errors. Thanks and please let me know what you find.

I've pushed an update to this module that corrects the issue with roles missing from the user selection options. Also pushed a minor core update that should correct the issue with users not showing up in the preview modal. They weren't showing up because pages in the admin were excluded, and users are pages in the admin.

There's a direct link to our dev branch on the download page. A lot of us use the dev branch for non-live stuff, sites in development, etc., and sometimes production sites when they are not mission critical, or if we need a feature that's only available on dev (and can keep an close eye on the site). The dev branch represents the next major version of ProcessWire, which would be 2.5 in this case. Once momentum drops on adding/fixing things and no major problems in the queue, we'll release it as 2.5. We haven't nailed down a date on that, but I think we're getting close. But if you want to start using it now, and don't mind watching for and reporting issues, use the dev branch.

Dynamic Roles are a powerful access control tool for ProcessWire. They pick up where traditional roles leave off, and allow you to assign permissions at runtime based on any factor present with the user. Once a user receives one or more dynamic roles (at runtime), those dynamic roles then specify what pages the user can view, edit, or add children to. If traditional roles are a sledgehammer, Dynamic Roles are a scalpel, allowing nearly any finely tuned access control scenario. Traditional ProcessWire roles are limited to assignment of view/edit/add access on a per-template basis. Dynamic roles go outside those limitations and enable you to assign that access based on any factors present with a page (i.e. match any field values). Dynamic Roles assign new access, but do not revoke existing access provided by traditional roles. As a result, Dynamic Roles can be used together with traditional roles, and the two work beautifully well together. Though Dynamic Roles can also replace all situations where you would use traditional roles for access control assignments. If using Dynamic Roles to assign page-view access, you would typically want to use traditional roles to revoke view access from at least the "guest" role at the template level. Then use Dynamic Roles to assign view access to those pages in a more granular manner. This module directly affects the results of all page getting/finding operations by applying the access control directly to the database queries before pages are loaded. As a result, it is fast (regardless of scale), pagination friendly, and requires no further intervention by the developer other than configuring the dynamic roles as they see fit. Because it relies upon new features present only in ProcessWire 2.4.6+, it requires the current dev branch. Sponsored by Avoine Concept by Antti Peisa Code by Ryan Cramer PLEASE NOTE: This module is in pre-release state (like the PW dev branch it requires) and is not recommended for production use just yet. Though we do appreciate any testing and/or feedback that you are able to provide. While not required, this module benefits from ProFields Multiplier. If you have ProFields Multiplier installed before installing this module, it will make this module more powerful by making all of your access control selectors have the ability to use OR-group conditions. Depending on your access control needs, this enables you to accomplish more with fewer Dynamic Roles. How to install Make sure you are running ProcessWire 2.4.6 (dev branch) or newer. Download from GitHub (we will add this module to the Modules directory later). Place all files from this module in /site/modules/DynamicRoles/. In your admin, go to Modules > Check for new modules. Click "install" for the Dynamic Roles module (ProcessDynamicRoles). Click to Access > Dynamic Roles for the rest (see example and instructions below). Example and instructions Lets say you ran a Skyscrapers site and wanted a role enabling users with "portmanusa.com" in their email address to have edit access to skyscrapers designed by architect John Portman, with at least 40 floors, and built on-or-after 1970. Yes, this is an incredibly contrived example, but it is an example that also demonstrates the access control potential of this module. 1. In your admin, you would click to Access > Dynamic Roles. 2. Click "Add Dynamic Role". Enter a name for the dynamic role, like: "skyscraper-test-editor" and save. 3. Under "Who is in this dynamic role?" section, click "Add Field" and choose: Email => Contains Text => "portmanusa.com". This will match all users having "portmanusa.com" in their email address. 4. Under "permissions" check the boxes for: page-view and page-edit. 5. For this contrived example, we will assume the user already has view access to all skyscrapers, so we will leave the "What can they view?" section alone. 6. For the "What can they edit?" section: Click "Add Field" and choose: template => Equals => Skyscraper. Click "Add Field" and choose: architect => Equals => John Portman. Click "Add Field" and choose: floors => Greater Than Or Equal => 40. Click "Add Field" and choose: year => Greater Than Or Equal => 1970. 7. Click Save. Now users matching the conditions of your dynamic role will be able to edit the matching pages, but not any others (unless assigned by traditional roles).

Here's a live example of the Pagination Textformatter in action at CMS Critic: http://www.cmscritic.com/how-to-create-a-social-network/ I've also updated this module to version 2, which adds an API giving you further control over pagination.

If you want to 404 that situation, add this to your .htaccess file, somewhere after "RewriteEngine On": # ----------------------------------------------------------------------------------------------- # Send URLs with non name-format characters to 404 page # ----------------------------------------------------------------------------------------------- RewriteCond %{REQUEST_URI} "[^-_.a-zA-Z0-9/~]" RewriteCond %{REQUEST_FILENAME} !-f RewriteCond %{REQUEST_FILENAME} !-d RewriteRule ^(.*)$ index.php?it=/http404/ [L,QSA]

More info on the outage here: http://blog.servint.net/2014/06/29/learning-mistakes-growing-crisis/

Glad you are enjoying it! I recommend grabbing the dev branch of ProcessWire, lots of new goods. If you like Repeaters, you'll like PageTable even better.

Matthew, sorry to hear you were so affected by this outage. It sounds like this particular outage was one that couldn't have been anticipated by anyone. From what I gather reading on other sites and on twitter, it sounds like a piece of network hardware that failed but provided no failure indicators. If that's the case, that would have made it particularly difficult to track down and left little room to put all that redundancy to work. Perhaps this particular type of outage is a once in a lifetime thing, but the reality is that outages occur everywhere and no webhost is immune to them. Not to mention outages can occur anywhere when it comes to networks, with the webhosts like ServInt probably being the most solid part of that chain. I was fairly lucky here in that I didn't really notice the outage other than someone emailed me about it when I was cooking dinner. But all seemed to be back online 30 minutes later and didn't go out again as far as I know. I've got most of my clients hosted at that Reston, VA data center, but the time the outage occurred was one of the least traffic times for the sites I work on, so I never heard from anyone about it. In 11+ years, I've only experienced one other major outage at ServInt and that was several years ago. Someone apparently got sloppy with a back-hoe in a barnyard and apparently cut off all lines of communication to McLean, VA. If I recall that outage was quite a bit longer than this one, but it's been awhile. There is absolutely nothing you could have or should have done extra here. On the other hand, if your client is giving a presentation, they are probably the ones that should have a backup plan. Anyone experienced giving presentations knows that you have to keep everything you need with you. You can't ever count on something being accessible from the internet, though usually for other reasons (bad wireless signal, something broken at the conference center's internet, etc.) So when it comes to presentations, you can only count on what's on your computer. Having a local running copy of a site, or a presentation with screenshots are good plans. If they couldn't access the site, hopefully that's what they did. One thing to take comfort in is that if this particular outage had occurred at some other host, chances are they would still be down right now. My opinion is I don't think there's any value in looking elsewhere due to this particular incident. I already know ServInt has the best people in the business. This kind of stuff can happen to any of them, and ServInt now has some experience that the others don't. Outages are a fact of life in the business and nobody is immune, but ServInt's history is that they are less prone to outages than most, and better equipped to handle them when the inevitable strikes.

Teppo this looks fantastic, nice work! While I haven't yet been able to test it out here I will be soon, as I have a regular need for a tool like this. It's also one of those things that come up with clients a lot: "how do I keep track of when a link no longer works?". I've been using Google Webmaster tools for 404 discovery in the past, but it's often hard to separate the noise from the goods there, and it's not particularly client friendly either. Regarding the cron side of this, I immediately thought of IftRunner (which itself is triggered by cron) and how this might work great as a PageAction with IftRunner. PageActions can also be executed by ListerPro and presumably other tools in the future as well.

With regards to a media manager, this is a topic not everyone agrees on so when I state "something better" I'm stating my opinion, formed largely on the sites I've built over the years. I don't believe I've ever dealt with a site that would have been better served by a media manager than with PW's way of managing assets, especially as the scale increases. But not all sites have the same needs, and we like differences of opinion here so I aways want to encourage discussion and questions. Keep asking questions and decide for yourself. If you are coming from a background of having used a media manager on past sites, I'd definitely encourage you to post more about the things you are trying to solve so that we can direct you towards simple ways of solving them. There are lots of different ways to accomplish things in PW.