ethanbeyer

Thanks @Robin S, you are the realest. I am going to poke into the issue later this week because I'd also like to know what's going on with it - especially why its failure took over ProcessWire's normal routing for edits as well as error logging.

@Robin S Holy macaroni. That was it. I thought I had tried everything. Where...where do I paypal you some beer money? Seriously. DM me if that's possible. I owe you big time.

That's just what is rendered in an AJAX call to the homepage: _main.php <?php // include page structure if not an ajax request if(!$config->ajax): include("./_head.php"); ?> <div id="page"> <div class="content-container clearfix"> <div class="content current-content"> <?php endif; // endif ajax ?> <?php // if ajax then only return $content echo $content; ?> <div id="pageID" data-pageID="<?php echo $page->id; ?>" class="d-none"></div> <?php if(!$config->ajax): ?> </div> </div> </div> <?php include("./_foot.php"); endif; // endif ajax ?>

The more I look at this, the more it seems like it's an .htaccess issue. Some uploads work, some don't. I've just noticed that some Page Edit links resolve to the same 404 page, so this seems less like an image upload problem, and more like a routing problem. But as for the reason, I am completely stumped.

haha @adrian, I have updated TracyDebugger to the most recent! I am confused by your response because the screenshot I posted was of the Browser Tools Network tab - the "404 ... POST" you see is the JPG upload attempt. Here are more screenshots, with the request details panel open. I am not seeing any errors in the response: just the HTML for the site's homepage. When a JPG is uploaded: When a PNG is uploaded: (and this is the same page, with no reload in between attempting these two uploads) Request 1 and 3 are the JPG and PNG upload, respectively.

I didn't realize the `always_populate_raw_post_data` option was for 5.6. The server that's bricking uploads is on 7.0.29. Symptoms: Uploads show the progress bar, and then the spinner shows up. It never goes away. Checking the page's directory in `/site/assets/files/{id}` shows that the file has uploaded, and isn't corrupted. ProcessWire isn't recognizing this because the response from the AJAX is the content of the site's homepage. I have disabled modules to no effect. It seems that the uploader is specifically having trouble with JPGs. Debugging Info (from TracyDebugger thank you @adrian) : ProcessWire: 3.0.123 PHP: 7.0.29 Webserver: Apache MySQL: 5.6.41-84.1 allow_url_fopen: 1 max_execution_time: 600 (changeable) max_input_nesting_level: 64 max_input_time: 600 max_input_vars: 1000 memory_limit: 512M post_max_size: 30M upload_max_filesize: 30M xdebug: xdebug.max_nesting_level: mod_rewrite: 1 mod_security: GD: bundled (2.1.0 compatible) GIF: 1 JPG: 1 PNG: 1 EXIF Support: 1 FreeType: 1 Imagick Extension: 1 BatchChildEditor: 1.8.10 ConnectPageFields: 0.1.8 EncryptDecryptRc4: 1.0.0 HtmlBodyClasses: 1.0.0 ImportPagesCSV: 1.0.6 InputfieldSimpleMDE: 1.0.5 Migrations: 0.3.1 PageCleanEmptyDirs: 0.0.3 PageRenameOptions: 1.0.5 ProcessChildrenCsvExport: 1.7.0 ProcessDatabaseBackups: 0.0.4 ProcessDirectoryMemberTools: 1.1.1 ProcessMigrations: 0.1.3 ProcessWireUpgrade: 0.0.7 ProcessWireUpgradeCheck: 0.0.7 TemplateCacheOverview: 0.0.3 TracyDebugger: 4.10.19 VirtualParents: 0.0.1 WireMailSmtp: 0.3.0 Things I've tried: Upload a JPG. Spinner. Open that same JPG in Photoshop, save it as a PNG (tripling its filesize) and upload. Upload takes, PW recognizes it. Save page. Resave the PNG as a JPG, upload that. Uploader spins. Check directory in FTP, all the files are there. Changed the field to use server-side resizing, no effect. Changed the max megapixels from NULL to 12.0, no effect. Resize the original JPG to ~1000px, upload it, and it works. o_0 --- Not sure what to do now. It seems similar to @Batyr's problem, in terms of the POST returning a 404. This does not happen on my local box with the same codebase. Does anyone have any ideas? EDIT: The weirdest part is that this just started happening on this server. I don't know the exact date or time, as the client just informed me the image uploads were struggling.

With the previous question out of the way, I am also having the same issue. I'm still trying to debug this, but for some reason, the network response to https://example.com/admin/access/users/edit/?&id=1266&InputfieldFileAjax=1 is a 404 with the HTML of my homepage.

Is there something about the way that ProcessWire handles uploads that is different from other systems? It seems I am often using some obscure PHP setting workaround (`always_populate_raw_post_data=-1`) or changing something at the server level. I hate wordpress, but I've never needed to perform any server admin to get their uploads to work–vagrant box or otherwise–does anyone know the reason ProcessWire is finicky here?

Thanks again for the responses, @Wanze, @wbmnfktr, @Autofahrn, and @LostKobrakai. All of your answers (again) help me make sense of things I didn't know or understand as fully as I thought I did. I've thought for a long time that the .htaccess file that ProcessWire uses was way more robust (read: better) than other CMS' that don't hide most of the app outside of webroot, but I hadn't thought about necessity of most of it. I think all of my thoughts/worries have been sufficiently addressed.

Sorry to be the guy that comes back like, "wait a minute!"–but I'm still thinking about this. Originally, I was asking, "does ProcessWire need .htaccess, and if so, why?" All the answers given boil down to yes, PHP CMS' need an .htaccess file, and the reasons are speed, security, and delegating to the HTTP server what the HTTP server does best. I hear you all! Thank you again for those replies. Now I want to ask...is the ProcessWire .htaccess file doing too much? Here it is, with comments stripped: Options -Indexes Options +FollowSymLinks # Options +SymLinksifOwnerMatch ErrorDocument 404 /index.php <Files favicon.ico> ErrorDocument 404 "The requested file favicon.ico was not found. </Files> <Files robots.txt> ErrorDocument 404 "The requested file robots.txt was not found. </Files> <IfModule mod_headers.c> Header always append X-Frame-Options SAMEORIGIN Header set X-XSS-Protection "1; mode=block" # Header set X-Content-Type-Options "nosniff" </IfModule> <FilesMatch "\.(inc|info|info\.json|module|sh|sql)$|^\..*$|composer\.(json|lock)$"> <IfModule mod_authz_core.c> Require all denied </IfModule> <IfModule !mod_authz_core.c> Order allow,deny </IfModule> </FilesMatch> <IfModule mod_php5.c> php_flag magic_quotes_gpc off php_flag magic_quotes_sybase off php_flag register_globals off </IfModule> DirectoryIndex index.php index.html index.htm <IfModule mod_rewrite.c> RewriteEngine On AddDefaultCharset UTF-8 # RewriteCond %{HTTPS} off # RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301] # RewriteCond %{HTTP:X-Forwarded-Proto} =http # RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301] <IfModule mod_env.c> SetEnv HTTP_MOD_REWRITE On </IfModule> # RewriteBase / # RewriteBase /pw/ # RewriteBase /~user/ RewriteRule "(^|/)\.(?!well-known)" - [F] # RewriteCond %{HTTP_HOST} !^www\. [NC] # RewriteRule ^ http://www.%{HTTP_HOST}%{REQUEST_URI} [L,R=301] # RewriteCond %{REQUEST_URI} "[^-_.a-zA-Z0-9/~]" # RewriteCond %{REQUEST_FILENAME} !-f # RewriteCond %{REQUEST_FILENAME} !-d # RewriteRule ^(.*)$ index.php?it=/http404/ [L,QSA] RewriteCond %{REQUEST_URI} !(^|/)site-[^/]+/install/[^/]+\.(jpg|jpeg|png|gif)$ RewriteCond %{REQUEST_URI} (^|/)\.htaccess$ [NC,OR] RewriteCond %{REQUEST_URI} (^|/)(site|site-[^/]+)/assets/(cache|logs|backups|sessions|config|install|tmp)($|/.*$) [OR] RewriteCond %{REQUEST_URI} (^|/)(site|site-[^/]+)/install($|/.*$) [OR] RewriteCond %{REQUEST_URI} (^|/)(site|site-[^/]+)/assets.*/-.+/.* [OR] RewriteCond %{REQUEST_URI} (^|/)(wire|site|site-[^/]+)/(config|index\.config|config-dev|env)\.php$ [OR] RewriteCond %{REQUEST_URI} (^|/)(wire|site|site-[^/]+)/templates-admin($|/|/.*\.(php|html?|tpl|inc))$ [OR] RewriteCond %{REQUEST_URI} (^|/)(site|site-[^/]+)/templates($|/|/.*\.(php|html?|tpl|inc))$ [OR] RewriteCond %{REQUEST_URI} (^|/)(site|site-[^/]+)/assets($|/|/.*\.php)$ [OR] RewriteCond %{REQUEST_URI} (^|/)wire/(core|modules)/.*\.(php|inc|tpl|module|info\.json)$ [OR] RewriteCond %{REQUEST_URI} (^|/)(site|site-[^/]+)/modules/.*\.(php|inc|tpl|module|info\.json)$ [OR] RewriteCond %{REQUEST_URI} (^|/)(COPYRIGHT|INSTALL|README|htaccess)\.(txt|md|textile)$ [OR] RewriteCond %{REQUEST_URI} (^|/)site-default/ RewriteRule ^.*$ - [F,L] RewriteCond %{REQUEST_URI} "^/~?[-_.a-zA-Z0-9/]*$" # RewriteCond %{REQUEST_URI} "^/~?[-_./a-zA-Z0-9æåäßöüđжхцчшщюяàáâèéëêěìíïîõòóôøùúûůñçčćďĺľńňŕřšťýžабвгдеёзийклмнопрстуфыэęąśłżź]*$" RewriteCond %{REQUEST_FILENAME} !-f RewriteCond %{REQUEST_FILENAME} !-d RewriteCond %{REQUEST_FILENAME} !(favicon\.ico|robots\.txt) # RewriteCond %{REQUEST_FILENAME} !\.(jpg|jpeg|gif|png|ico)$ [NC] # RewriteCond %{REQUEST_FILENAME} !(^|/)site/assets/ RewriteRule ^(.*)$ index.php?it=$1 [L,QSA] # RewriteRule ^(.*)$ /index.php?it=$1 [L,QSA] </IfModule> That's 86 lines. For comparison, here's Laravel's: <IfModule mod_rewrite.c> <IfModule mod_negotiation.c> Options -MultiViews -Indexes </IfModule> RewriteEngine On # Handle Authorization Header RewriteCond %{HTTP:Authorization} . RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}] # Redirect Trailing Slashes If Not A Folder... RewriteCond %{REQUEST_FILENAME} !-d RewriteCond %{REQUEST_URI} (.+)/$ RewriteRule ^ %1 [L,R=301] # Handle Front Controller... RewriteCond %{REQUEST_FILENAME} !-d RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^ index.php [L] </IfModule> And Wordpress': <IfModule mod_rewrite.c> RewriteEngine On RewriteBase / RewriteRule ^index\.php$ - [L] RewriteCond %{REQUEST_FILENAME} !-f RewriteCond %{REQUEST_FILENAME} !-d RewriteRule . /index.php [L] </IfModule> I am wondering if the .htaccess has to do as much as it's doing right now, or is some of this stuff redundant to validation already in Core? I fully agree we need it, but I wonder if we could move some of the common configurations out of it, because at the end of the day, the app should be able to work out of the box on either Apache or NGINX with limited knowledge of either system and not need gobs of HTTP configuration. I don't personally know how to do spearhead this effort, but other systems have found a way to do so, and I think ProcessWire could garner more market share if the initial configuration appeared hella simple. The .htaccess (and by extension, conversion to NGINX) scares some people. I think it would be worthwhile to discuss the possibility and plausibility of moving some of those .htaccess configs to the core in some form or fashion.

Thank you all for your answers! It was very exploratory because this is a blind spot for me. I haven't known exactly how this worked, and after all of the explanations, I think I understand a little more fully. Thanks!

Yeah, I don't know the "better way" either, at least in terms of actual methodology - I suppose what I was wondering is if anyone had experience in this realm and could help the ProcessWire community at large detach ProcessWire from viewing Apache as a necessity out of the box. Does that make sense?

I've introduced ProcessWire (gleefully) to quite a few devs I know. One, who is quite a bit more experienced than I am, raised a question that I couldn't answer: Does ProcessWire rely on .htaccess *too much*? I realize that .htaccess handles a lot of baseline security and routing issues within the ProcessWire ecosystem. Inevitably, this leads to a large reliance on Apache, unless one is willing to try to convert those rules to NGINX configurations. This has led me to wonder, is there a better way (PHP based?) of handling routing and security than .htaccess?

Thank you all for your input! This is really helpful, and how I had started seeing the issue as well. What I meant by "attention" was the development maintenance, but you're right: updates don't equate to usefulness. I'm with you - it's what I have used most, also, and for me and the sites I'm building with ProcessWire, I think it makes sense to use one method instead of both. I particularly like using the namespaces in the WireCache as @Zeka mentioned as well - it makes expiring similar things, or preloading similar things, very easy.

Hello, I'm hoping for a little help understanding the different types of caching baked in to ProcessWire, as well as when one should use which. As I understand it, there are two methods for caching, with several different interfaces: 1. Database Caching Within the realm of Database Caching, there is the WireCache class, accessible through the $cache variable. This has various methods, but you can save just about anything to the caches table in the ProcessWire Database, and set a time for expiration, and retrieve it. Nice! It seems like this is the most feature-rich way to cache something that isn't a template, but I don't know how it works performance-wise against, say, MarkupCache. 2. File Caching Then there's File Caching - and the two interfaces that I know of that use this method are Template Caching and the MarkupCache plugin. Template Caching generates the full $page->render(), whereas MarkupCache takes whatever markup one wants to save and outputs that to a file that expires with the given time. Questions and Considerations If saving Markup, which is faster: Database or File Caching? Should we use MarkupCache at all, when WireCache seems to get more attention?

Sorry for the late reply - thank you! That had been confusing me a lot.

I've been looking for the answer for this everywhere - and I appreciate the answer. But *where* in the Core is the "textformatters" attribute instantiated?! I can't find it anywhere. I don't see it in the Textformatter class, or Field, Fieldtype or FieldtypeText. Any ideas?

@Jon this sounds a little strange. Reading most of your comment, I would think that this might have something to do with your Office365 having sending limits that you're going over. The TLS issue is what makes it weird, though - because if it was a limits problem, that would not be happening. Is there anything in the logs that gives what error is encountered, rather than saying it just stops sending?

At the risk of sounding very pedantic, this condition would result in the timestamp for any page with template cache enabled, which is not what I'm looking for. I only want the timestamp to appear on cached versions of the page - so if I am logged in, and am served a page that doesn't give cached pages to logged-in users, I don't want the timestamp (or maybe I would have a different comment that said this page was generated now and is not cached, etc). On the other hand, if I was not logged in, and looked at the source for the same page and was served the Cached version, I'd expect to see the timestamp. Essentially, I am looking to inject the comment ONLY at the time of the creation of the cached page, not on normal render - so this makes me wonder if there's a hook I could grab before page render to determine whether or not the page render was being performed in order to create a CacheFile. Does this make sense?

Well, I feel a bit silly. I didn't think that needed to be done because of the call to $sanitizer->selectorValue($input->get('s')); . Anyhow, you solved my problem - and I think you're right: String Selectors are probably an easier method to go with than Array Selectors until they've been documented better.

In theory, yes - but the largest problem with that approach is that the timestamp is always there. This is mostly for my OCD brain, but I essentially wanted a sanity check in the source that would quickly show that a page was or was not cached, and when.

As a follow-up...has anyone ever built a site using delayed output with template caching? This is a new frontier for me and I don't know how to set it up.

Not sure if this is possible - but I'd like to add something like this to cached pages via Template Cache: <!-- Page Cache Generated 2018-05-22 @ 12:45:44 UTC / Expires 2018-05-29 @ 12:45:44 --> I thought there might be a way to see if a cache file existed for the page in question, and if it did not, to include that comment in the render->cacheFile->save(). Does anyone know?

I don't think anything has changed with Repeaters in this realm yet.

I'm trying to build a Search Template. The way I had been doing it was by constructing a Selector one string at a time in an array, and then imploding that array with ", " in order to create a Selector that was passed to $pages->find($selectorStr). This is a bare-bones example of what I was doing: $search = $sanitizer->selectorValue($input->get('s')); $selector[] = 'template=member'; $selector[] = "field1|field2|field3%={$search}"; $selector[] = "(repeaterField.fieldX|repeaterField.fieldZ%={$search})"; $selectorStr = implode($selector, ", "); $results = $pages->find($selectorStr); This worked as expected, and I was able to utilize OR-groups within the Selector. There is a problem, though: for whatever reason, despite sanitizing the incoming value, if the user input a string like "test,foo", ProcessWire threw this error: Exception: Unknown Selector operator: '' -- was your selector value properly escaped? field='foo', value='', selector: 'include=all, children.count>0, title%=test,foo' (in /var/www/public/wire/core/Selectors.php line 378) So I moved on to Selector Arrays, and now I'm having trouble with OR-groups. I found out about Selector Arrays (not documented at all as far as I can see in the API Reference) through a ProcessWire blog post about 3.0.13. Here's the code (using the actual field names, unlike above): $selectorArray = []; // template $selectorArray[] = ['template', 'directory-member']; // basic fields $selectorArray[] = [ 'field' => [ 'dm_firstName', 'dm_middleName', 'dm_lastName', 'email', 'dm_ficmTitle', 'dm_addressStreet', 'dm_addressCity', 'dm_addressState', 'dm_addressPostalCode', 'dm_birthday', 'dm_phone1', 'dm_phone2' ], 'operator' => '%=', 'value' => $searchStr ]; //repeaters $selectorArray[] = [ 'field' => [ 'dm_contactREP.dm_contactMethod', 'dm_contactREP.dm_contactValue' ], 'operator' => '%=', 'value' => $searchStr, 'group' => 'repeaters' ]; // search the parent's name $selectorArray[] = [ 'group' => 'parentTitle', 'field' => 'parent.title', 'operator' => '%=', 'value' => $searchStr, ]; // sort $selectorArray[] = ['sort', 'dm_lastName']; $selectorArray[] = ['sort', 'dm_firstName']; The problem is that the Selector is rendered like this (clauses broken into Newlines): template=directory-member, dm_firstName|dm_middleName|dm_lastName|email|dm_ficmTitle|dm_addressStreet|dm_addressCity|dm_addressState|dm_addressPostalCode|dm_birthday|dm_phone1|dm_phone2%=test, repeaters@dm_contactREP.dm_contactMethod|dm_contactREP.dm_contactValue%=(test), parentTitle@parent.title%=(test), sort=dm_lastName, sort=dm_firstName, status<1024 For OR-groups, the whole clause should be wrapped in parentheses - not just the value I'm searching for! So, two questions: Why would a comma in the search value break things that are supposed to be sanitized, and two, how do I get the OR-groups to work as intended?