ryan

Matthew, why were you offline for 4+ hours? This morning's outage was an emergency update planned for all servers, not just a small number. Is the outage you are talking about different from the one that was planned? I'm just curious if your outage is related to the emergency update or if it was something different entirely? In either case, it sounds like your host node has some bad karma or something. Maybe it's worth asking to be switched to a different node in the data center. Just make sure you are communicating directly with them through the portal (or by phone), and not Twitter or other social networks. They are extremely helpful and knowledgeable, but you have to use their secure channels to communicate because they have to maintain client confidentiality. Another thing to consider is that any kind of maintenance that would require taking down a server is something they usually do overnight or early AM, since that's the least busy time (though this kind of maintenance is rare). But in your case, your clients are in Europe (I think?). Whether for scheduling or ping times, maybe it would be beneficial to be in their Amsterdam data center rather than the Reston, VA data center?

Matthew, I got an email about this planned outage last week (planned 15 mins between 5 and 7 am), as did all my clients. Are you sure ServInt has your correct email address on file? The outages are just to reboot the servers (kernel patch only seen after server reboots), and I read the 15 minutes as more like a maximum. If your server is down it sounds like that's something different, you should submit an urgent ticket or call them. It's worth noting that they are fixing a vulnerability that affects most web hosts, not just ServInt, and appears they are fixing it before everyone else. Please let us know what you find out from them as to why your server isn't online.

I've updated this module to version 3 which adds the ability to use headlines as page breaks and adds support for several shortcodes.

You are not already screwed if your writable files aren't executable or don't represent files that can blow up the site. Obviously, the intention is that nobody else can write to the writable files, but the reality is when designing a system we have to plan for the worst. Your environment and my environment may be well secured, but I try to think about the bigger picture out there where there are a lot of insecure environments. Security is #1 when it comes to ProcessWire. What are secure permissions and what are not is dependent on the server and not something we can detect. A security best practice is to get everything important out of any writable files on the file system and not have any single writable files that could take down the whole system. Meaning, I think it's acceptable for cache files, image and other independently served assets to exist as writable files, but the files we're talking about here are definitely out. Another reason we can't detect this stuff is because there are plenty of hosts running phpsuexec and the like where apache runs as the user and everything is writable by default. This tends to be okay, so long as there isn't also a copy of WordPress somewhere on the account. Basically, what are the right permissions for a given site comes down to the server, environment and host, and not ProcessWire. If someone is able to write something to some file on your account, that does not imply they can access the DB. That "someone" is usually an automated script and not a real person, or it's one of your account neighbors on a shared webhost. One of the things that regularly falls in my lap (for better or worse) is to fix hacked WordPress sites. Rarely has the DB been touched, while most of the writable file system has to be quarantined. I disagree. There is a direct connection of field schema to the data represented in the DB. Template files are not coupled to fields like that because you are talking about your code, not the system. You have to make your changes independently regardless of where they are stored. Changing a field doesn't automatically change a template file. But changes to fields are an end-point in terms of how they reflect on the database. I've run out of time for today so have to stop. But I want to be clear that PW is not going to regress to storing this stuff on the file system. I recognize there is a benefit to some developers in having files they can version control or migrate independently. I personally would find the drawbacks to far outweigh the benefits (I have to support this project for a large audience), but understand why some find it desirable. Like I've mentioned, I would consider in the future mirroring the data to files that you can version control and selectively migrate in the admin. I believe that would bring the benefits of what's been discussed here, without the drawbacks.

At some point, I'll setup fields and templates to maintain mirror copies of files on the file system for those that want to version control them that way. They are in the database for very good reasons, but that doesn't mean that they can't be on the filesystem too. It's just that the database is the better default place for them to live and be maintained from for the vast majority. I don't consider writable files particularly trustworthy in the grand scheme, so the master source would always have to be the DB, but the mirror files could certainly be monitored, versioned and migrated at the developers discretion. I have no problem with that so long as we're not letting the file system lead. Something to add here that I think gets overlooked is that PW is not just a development tool. In the majority of cases, the development phase is just a very short blip in the overall timeline of the project developed with ProcessWire (whether a website, application or whatever it is). While we put a lot of focus on the development tool aspect of PW, the big design decisions are made for the entire lifecycle, not just the development phase. CMSs have to be written for large scale realities and diversity across hosting providers, varying amounts of security and expertise levels. While files here seem beneficial from one perspective, that does not translate to a net benefit in the larger context. In fact it translates to something rather undesirable in the larger context. Having important data in writable files on the file system is something you try to avoid with a CMS. While they are a necessary evil in some cases, if you can keep writable data in the DB, it's going to result in stronger security over a broad installation base and over a broad time period. These writable files are often the weakest link on shared hosting accounts. They can be perfectly secure, but there's little doubt the DB is safer. When it comes to data that needs to be editable, I consider the DB a much more trustworthy storage mechanism for important data across a large set of installations. I'm stating the diverse reality of our big picture context and not any individual's server. Some of us are running on servers where it would make no difference at all from a security aspect, but that's not something we can count on. Outside of the situations mentioned in this thread, I think most would not find it desirable to have the field and template data that disconnected from the content it represents. I can imagine some real headaches with schema getting disconnected from the data. When I backup my DB, I like to know that I've got everything necessary to put it back together without having to chase down multiple files with multiple tools, not to mention possibly files in different versions. I don't want to open the door to having having schema files that are not consistent with the data that they are supposed to represent. Data loss is always a possibility with schema changes and should always be accompanied by a confirmation. Automation by movement of schema in files (whether by git or SSH, FTP, etc.) is problematic for a lot of reasons. The issue described about one person's changes overwriting another's may be a potential real world case, but is a non-issue for most of us because we don't migrate those kinds of changes at the DB level, nor do I recommend doing that. I understand that there are challenges for a team of developers having multiple versions of schema, or developers that want to migrate schema changes to a live server in an automated fashion rather than re-playing those changes themselves. I actually think less automation is ultimately a safer solution here, even if not as convenient for some. Though I'm still very enthusiastic about continuous integration projects and doing anything I can to support them. But I do not support moving storage away from the DB as our primary storage for these things. I understand using the file system may seem like a smart choice in certain contexts (and I don't disagree on them), but in the larger context it's not a wise choice. I'll do my best to find ways to mirror the data to files for those that might want this or may benefit from it.

I'm not exactly sure what the issue was, as I wasn't able to reproduce it locally. But I did make several adjustments in the Modules class that I think might help (just now committed to dev). I'm curious to know if it resolves the issue you guys were experiencing there? I've also updated the DynamicRoles module to use info.json files rather than a getModuleInfo() method. I should have done that in the first place since DynamicRoles extends another module (ProcessPageType), so it's also possible the issue was related to that. I would suggest upgrading the core first, and then the Dynamic Roles module if you still see errors. Thanks and please let me know what you find.

I've pushed an update to this module that corrects the issue with roles missing from the user selection options. Also pushed a minor core update that should correct the issue with users not showing up in the preview modal. They weren't showing up because pages in the admin were excluded, and users are pages in the admin.

There's a direct link to our dev branch on the download page. A lot of us use the dev branch for non-live stuff, sites in development, etc., and sometimes production sites when they are not mission critical, or if we need a feature that's only available on dev (and can keep an close eye on the site). The dev branch represents the next major version of ProcessWire, which would be 2.5 in this case. Once momentum drops on adding/fixing things and no major problems in the queue, we'll release it as 2.5. We haven't nailed down a date on that, but I think we're getting close. But if you want to start using it now, and don't mind watching for and reporting issues, use the dev branch.

Dynamic Roles are a powerful access control tool for ProcessWire. They pick up where traditional roles leave off, and allow you to assign permissions at runtime based on any factor present with the user. Once a user receives one or more dynamic roles (at runtime), those dynamic roles then specify what pages the user can view, edit, or add children to. If traditional roles are a sledgehammer, Dynamic Roles are a scalpel, allowing nearly any finely tuned access control scenario. Traditional ProcessWire roles are limited to assignment of view/edit/add access on a per-template basis. Dynamic roles go outside those limitations and enable you to assign that access based on any factors present with a page (i.e. match any field values). Dynamic Roles assign new access, but do not revoke existing access provided by traditional roles. As a result, Dynamic Roles can be used together with traditional roles, and the two work beautifully well together. Though Dynamic Roles can also replace all situations where you would use traditional roles for access control assignments. If using Dynamic Roles to assign page-view access, you would typically want to use traditional roles to revoke view access from at least the "guest" role at the template level. Then use Dynamic Roles to assign view access to those pages in a more granular manner. This module directly affects the results of all page getting/finding operations by applying the access control directly to the database queries before pages are loaded. As a result, it is fast (regardless of scale), pagination friendly, and requires no further intervention by the developer other than configuring the dynamic roles as they see fit. Because it relies upon new features present only in ProcessWire 2.4.6+, it requires the current dev branch. Sponsored by Avoine Concept by Antti Peisa Code by Ryan Cramer PLEASE NOTE: This module is in pre-release state (like the PW dev branch it requires) and is not recommended for production use just yet. Though we do appreciate any testing and/or feedback that you are able to provide. While not required, this module benefits from ProFields Multiplier. If you have ProFields Multiplier installed before installing this module, it will make this module more powerful by making all of your access control selectors have the ability to use OR-group conditions. Depending on your access control needs, this enables you to accomplish more with fewer Dynamic Roles. How to install Make sure you are running ProcessWire 2.4.6 (dev branch) or newer. Download from GitHub (we will add this module to the Modules directory later). Place all files from this module in /site/modules/DynamicRoles/. In your admin, go to Modules > Check for new modules. Click "install" for the Dynamic Roles module (ProcessDynamicRoles). Click to Access > Dynamic Roles for the rest (see example and instructions below). Example and instructions Lets say you ran a Skyscrapers site and wanted a role enabling users with "portmanusa.com" in their email address to have edit access to skyscrapers designed by architect John Portman, with at least 40 floors, and built on-or-after 1970. Yes, this is an incredibly contrived example, but it is an example that also demonstrates the access control potential of this module. 1. In your admin, you would click to Access > Dynamic Roles. 2. Click "Add Dynamic Role". Enter a name for the dynamic role, like: "skyscraper-test-editor" and save. 3. Under "Who is in this dynamic role?" section, click "Add Field" and choose: Email => Contains Text => "portmanusa.com". This will match all users having "portmanusa.com" in their email address. 4. Under "permissions" check the boxes for: page-view and page-edit. 5. For this contrived example, we will assume the user already has view access to all skyscrapers, so we will leave the "What can they view?" section alone. 6. For the "What can they edit?" section: Click "Add Field" and choose: template => Equals => Skyscraper. Click "Add Field" and choose: architect => Equals => John Portman. Click "Add Field" and choose: floors => Greater Than Or Equal => 40. Click "Add Field" and choose: year => Greater Than Or Equal => 1970. 7. Click Save. Now users matching the conditions of your dynamic role will be able to edit the matching pages, but not any others (unless assigned by traditional roles).

Here's a live example of the Pagination Textformatter in action at CMS Critic: http://www.cmscritic.com/how-to-create-a-social-network/ I've also updated this module to version 2, which adds an API giving you further control over pagination.

If you want to 404 that situation, add this to your .htaccess file, somewhere after "RewriteEngine On": # ----------------------------------------------------------------------------------------------- # Send URLs with non name-format characters to 404 page # ----------------------------------------------------------------------------------------------- RewriteCond %{REQUEST_URI} "[^-_.a-zA-Z0-9/~]" RewriteCond %{REQUEST_FILENAME} !-f RewriteCond %{REQUEST_FILENAME} !-d RewriteRule ^(.*)$ index.php?it=/http404/ [L,QSA]

More info on the outage here: http://blog.servint.net/2014/06/29/learning-mistakes-growing-crisis/

Glad you are enjoying it! I recommend grabbing the dev branch of ProcessWire, lots of new goods. If you like Repeaters, you'll like PageTable even better.

Matthew, sorry to hear you were so affected by this outage. It sounds like this particular outage was one that couldn't have been anticipated by anyone. From what I gather reading on other sites and on twitter, it sounds like a piece of network hardware that failed but provided no failure indicators. If that's the case, that would have made it particularly difficult to track down and left little room to put all that redundancy to work. Perhaps this particular type of outage is a once in a lifetime thing, but the reality is that outages occur everywhere and no webhost is immune to them. Not to mention outages can occur anywhere when it comes to networks, with the webhosts like ServInt probably being the most solid part of that chain. I was fairly lucky here in that I didn't really notice the outage other than someone emailed me about it when I was cooking dinner. But all seemed to be back online 30 minutes later and didn't go out again as far as I know. I've got most of my clients hosted at that Reston, VA data center, but the time the outage occurred was one of the least traffic times for the sites I work on, so I never heard from anyone about it. In 11+ years, I've only experienced one other major outage at ServInt and that was several years ago. Someone apparently got sloppy with a back-hoe in a barnyard and apparently cut off all lines of communication to McLean, VA. If I recall that outage was quite a bit longer than this one, but it's been awhile. There is absolutely nothing you could have or should have done extra here. On the other hand, if your client is giving a presentation, they are probably the ones that should have a backup plan. Anyone experienced giving presentations knows that you have to keep everything you need with you. You can't ever count on something being accessible from the internet, though usually for other reasons (bad wireless signal, something broken at the conference center's internet, etc.) So when it comes to presentations, you can only count on what's on your computer. Having a local running copy of a site, or a presentation with screenshots are good plans. If they couldn't access the site, hopefully that's what they did. One thing to take comfort in is that if this particular outage had occurred at some other host, chances are they would still be down right now. My opinion is I don't think there's any value in looking elsewhere due to this particular incident. I already know ServInt has the best people in the business. This kind of stuff can happen to any of them, and ServInt now has some experience that the others don't. Outages are a fact of life in the business and nobody is immune, but ServInt's history is that they are less prone to outages than most, and better equipped to handle them when the inevitable strikes.

Teppo this looks fantastic, nice work! While I haven't yet been able to test it out here I will be soon, as I have a regular need for a tool like this. It's also one of those things that come up with clients a lot: "how do I keep track of when a link no longer works?". I've been using Google Webmaster tools for 404 discovery in the past, but it's often hard to separate the noise from the goods there, and it's not particularly client friendly either. Regarding the cron side of this, I immediately thought of IftRunner (which itself is triggered by cron) and how this might work great as a PageAction with IftRunner. PageActions can also be executed by ListerPro and presumably other tools in the future as well.

With regards to a media manager, this is a topic not everyone agrees on so when I state "something better" I'm stating my opinion, formed largely on the sites I've built over the years. I don't believe I've ever dealt with a site that would have been better served by a media manager than with PW's way of managing assets, especially as the scale increases. But not all sites have the same needs, and we like differences of opinion here so I aways want to encourage discussion and questions. Keep asking questions and decide for yourself. If you are coming from a background of having used a media manager on past sites, I'd definitely encourage you to post more about the things you are trying to solve so that we can direct you towards simple ways of solving them. There are lots of different ways to accomplish things in PW.

The above is just the start of image manipulation potential in PW. There are so many things you can do from the API side with images. Just today we added some new cropping options to the dev branch (thanks to Horst, who is one of the best in the world when it comes to image related code). Btw, your site is great and I will definitely visit a lot in the future. We're in Atlanta, but have family moving to Orlando and plan on spending a lot of time at Disney World when we can (we have two young daughters that of course love everything Disney).

I agree, I think ProcessWire would be an excellent fit for your needs. With regard to centralized media manager, there's a reason we don't have one built in and that's because we've got something better. You may have to think differently about how you manage them, but once you get it I don't think you'd want to go back to an old style media manager. If you need any help understanding how a particular situation would be accomplished in PW I'm happy to give more info. This is a potential security hole. Allowing URLs to create images on the fly has high potential as a ddos (denial of service) hole that is an easy attack target. Someone can write a quick script to formulate and call millions of those URLs, consuming your server for hours or days till eventually filling up the hard drive (or your hosting quota). Basically, it's a security problem if non-predefined resize dimensions are coming from the client (user input, like URLs) rather than from the server. I substituted some other width/height values in there and can see it will take whatever I give it (no restraints). For example, this call uses 3 megabytes on disk, takes up several seconds of server time, and consumes 3 megabytes of your bandwidth: http://wdwfans.com/files/thumb/21638/3500/3500/fit Here's how you'd approach creating 500x500 images in PW: $image = $page->image->size(500, 500); echo "<img src='$image->url' />"; Our default behavior would be the same as your "fit" method described. I'm not sure there is a legitimate use for the "fill" method as it distorts the photo, something that I can't imagine is desirable on any site. But if you wanted to duplicate that, you'd turn cropping off: $image = $page->image->size(500, 500, array('cropping' => false));

you are right, I forgot to publish. (it's Friday, end of the day, excuses and such)

This Textformatter module for ProcessWire enables you to break up a single textarea field (using either TinyMCE or CKEditor) into multiple pages. You include all of the content in a single rich text field and separate each pagination with a line of hyphens (5+). When rendered on the front-end, the user will see pagination links at the bottom of the page enabling them to move forward and backward through the content/article. Also included is the option for title pagination. That means assigning a title/headline to each pagination and giving the user a list of those titles they can click on to move to each section of the article. Example of this module in action Documentation and customization options GitHub Page Module Page Download ZIP Install class: TextformatterPagination

Relying on cookies alone is just not enough. Everything you do to trying to maintain uniqueness with anonymous requests is a compromise. I think remembering the IP for some length of time (which is configurable, btw) is a necessary and worthwhile compromise for most.

FieldtypeLikes is a module I've been working on (it's what is used here on the PW sites, as well as on CMSCritic). I've just been trying to find the time to finish it up so that others can use it, and hopefully can here soon. Tyssen I was thinking I could get you to beta test since you've got a more immediate need? Here's a section from the documentation which may answer some of the questions above: Likes Fieldtype for ProcessWire This Fieldtype enables you to have a "like" button that users can click on to like a particular page and have that be remembered. The fieldtype itself stores a single integer representing the number of likes each page has in total. As a result, the field can be used for the purpose of sorting pages, i.e. "most liked pages." FieldtypeLikes also adds a $session->getLikedPages(); that returns a PageArray of pages that the current session has clicked Like on. The liked pages are remembered with a cookie for up to 30 days. This enables you to have a separate page (or perhaps a sidebar on every page) that for example shows the user bookmarks of pages they liked. Before deciding whether this Fieldtype is suitable for your particular application, be sure to read the section on preventing duplicate likes. Preventing duplicate likes FieldtypeLikes is not connected with the user system, and may be used anonymously via any page on your site. The benefit is that anyone visiting can "like" things and generate a list of likes (for their own review), without having to login or create an account. That makes it much more likely that users will participate in liking pages and making use of what this module provides. The drawback to this approach is that it may be difficult to prevent one user from trying to manipulate the quantity of likes, perhaps trying to boost the rank of a page they have some interest in. Beyond cookies (which we use), in order to limit the potential for duplication, FieldtypeLikes connects likes IP addresses and remembers them (server side) for a week. As a result, there is a limit of 1-like, per page, per IP address, per week. While not ideal, this is a necessary compromise in order to have some protection for the data. Note that a very determined person could still use proxy servers or other methods of obtaining unique IP addresses. So we have also implemented some additional methods of protection. But ultimately you should always remember that if someone is determined enough, its impossible to prevent them from finding some way to manipulate the quantities of likes. As a result, do not use likes data for making decisions on hiring/firing, awards, grants or anything to be taken too seriously. That being said, I do feel the solutions we have implemented here are stronger than other anonymous rating systems I have come across, thus far. But always remember that any anonymous voting tool is open to manipulation and the results should always be take in that context, whether from this tool, or any other you've ever used or seen.

upgrade.php is not a ProcessWire file. I'm guessing maybe it was a leftover from a previous WP or Joomla install? It sounds like it could be related to the exploit you experienced, but be careful not to assume it ends there. ProcessWire's core does not have very many input opportunities on the front-end of your site. Actually, the URL itself is really the only input ProcessWire deals with on the front-end, and that is validated by the htaccess before being sent to PW, and then thoroughly sanitized then validated again by the core. As a result, it's unlikely for ProcessWire itself to be exploited in the same ways that some other CMSs might be, simply because there are a lack of input opportunities to a guest visitor. What ProcessWire does instead is give you an API that lets you control all the aspects of when user input results in something output. If there were to be an exploit on a particular PW site, it would be much more likely to be the result of the code unique to that site, or a 3rd party module, rather than PW itself. If you were dealing with a site that had other software running, either presently or in the past (perhaps WP powering a blog alongside) then it'd be much more likely for that to be the source of the issue than PW. In fairness to WP, most exploits have to do with 3rd party WP plugins or themes and not WP itself. WP and Joomla are also much larger targets than PW, so they are usually broken into with automated scripts rather than actual people sitting at their computer. When you've got a site that you know has been broken into at the filesystem level (like yours might have been), it's unusual for it to be limited to just one file. There are usually backdoors built elsewhere. Even if the site is fixed for the moment, it's good to still think of everything as tainted until proven otherwise. I'm not necessarily a fan of restoring from a backup in this case, unless you know for certain that the backup itself does not contain the exploit. Sometimes a backdoor will be present for months before taken advantage of. If you had Joomla running on this server a long time ago, the exploit may have originated there and simply been hidden somewhere on the file system. The first thing you'd want to do is remove any other software installed on the server that doesn't need to be there–old copies of WP, Joomla, etc., or files leftover by them. If you aren't certain, then just move it to a non web accessible quarantine directory. For your ProcessWire site, you don't need anything in your web root directory except for: /wire/, /site/, /index.php and .htaccess. Remove your entire /wire/ directory and put in a new fresh copy, as well as your /index.php and /.htaccess file. Basically do the same thing you would do if performing an upgrade. In /site/modules/ you may have some 3rd party modules installed. Replace each of the directories in /site/modules/ with fresh copies. This is the same thing you'd do if upgrading those modules. That essentially leaves /site/ and everything in it to analyze. If the hack originated from an automated script targeting DrumlaPress, chances are it left your PW installation alone, but you never know–it might have gone after every single PHP and JS file it could find on the file system. You'll want to look for anything unusual in your /site/templates/*.php files and /site/config.php file. By unusual, I mean anything that you didn't put there. Start by looking at the beginning and ending of each file. Red flags are usually extra <script> statements, base64 function calls, exec function calls, extra JS attributes on markup elements that you didn't put there (like onclick), include or require statements with variable names in them or referencing files you don't recognize. Also consider that new directories may have been introduced anywhere. I would download a copy of your entire /site/ structure and analyze it locally, looking for any extra files or directories that you didn't put there. Also be on the lookout for extra .htaccess files, and give them a close look if you find any. Meaning, you'll need to make sure you are seeing hidden files (those preceded with a period). Compare your /site/ structure to a /site-default/ structure as included with a new copy of PW. Once you've cleaned your /site/ directory or at least verified that it's clean, make a good backup of your site so that you've got a known clean starting point (hopefully). Open a browser to your site with Chrome and go to View > Developer > Developer Tools. Click to the "Network" tab. Reload the page. Look for any offsite requests that you don't recognize. That may reveal something hidden that might still need to be cleaned, but hopefully not. Now click to the "Elements" tab. This shows the post-JS markup of your site. Look for any generated elements that you don't recognize, especially at the end or beginning. This again can reveal extras that have been added into your page by an exploit. Lastly (or maybe firstly?), take a look at your file permissions. If on a shared host, you want to be certain that your files aren't readable or writable to other users on the same server. Especially your /site/config.php file, and your /site/assets/ directory (and everything below). What permissions are ideal here depend on the web host and what type of PHP they are running, so it's best to inquire with them on how you can have files writable to your website that aren't writable by other accounts. Btw, I have never seen or heard of a compromised PW installation, regardless of what other compromised software was running on the server. I have seen plenty of compromised WP installations that had PW running alongside them. Thankfully, the PW installation has always been left alone, with the exploit limited to the WP installation. But it's best to go into these things assuming everything on the server is tainted, so always play it safe.

That SetEnv line is only used by the installer to determine if mod_rewrite might be inactive. It is totally fine to remove it. Great suggestion by Craig to wrap it in the IfModule block–I will add that.

Thanks guys, I have fixed that issue by adding a new getModule($key, $options) method to the Modules class that lets you retrieve a module with $options. In this case an option called 'noPermissionCheck'. The ServicePages module has been updated to use the new method (when available). Adrian, I wasn't seeing the "it" GET variable. That is a GET variable used internally by PW, but it gets removed before the request starts. So if you are seeing an "it" variable then you probably shouldn't be. At least I'm not seeing it here. I did go ahead and add that as something for ServicePages to ignore though, per your suggestion.