Pages throw 404 Error after Migration

[Neo]

Hey,

I am trying to migrate a couple of Processwire sites from their respective, small VPS to a larger VPS on Digitalocean that will host all sites in the future.

For that purposes, I installed a fresh droplet with Debian 10, Apache 2.4.38, PHP 7.3.4-2 and MariaDB 10.3.15.

I created separate root directories for every site and a respective Virtual Host configuration file for each domain, for instance:

sudo mkdir -p /var/www/example.com/public_html

sudo chown -R $USER:$USER /var/www/example.com/public_html

sudo cp /etc/apache2/sites-available/000-default.conf /etc/apache2/sites-available/example.com.conf

The content of the Virtual Host configurations looks like this:

<VirtualHost *:80>
        ServerName example.com
        ServerAlias www.example.com

        ServerAdmin admin@example.com
        DocumentRoot /var/www/example.com/public_html

        <Directory /var/www/example.com/public_html/>
                Options FollowSymLinks
                AllowOverride All
                Require all granted
        </Directory>

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

# This part seems to have been added by Let's Encrypt during SSL generation
RewriteEngine on
RewriteCond %{SERVER_NAME} =www.example.com [OR]
RewriteCond %{SERVER_NAME} =example.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

I then enabled the configuration files and restarted Apache:

sudo a2ensite example.com.conf

sudo systemctl restart apache2

Mod_rewrite has then been enabled via the following:

sudo a2enmod rewrite

 

I migrated the first site and database to the new server and the homepage loads fine via https (both non-www and www versions are available although it should redirect automatically to www which does not work).

However, any link you try to visit throws a 404 error:

Not Found

The requested URL /link/ was not found on this server.
Apache/2.4.38 (Debian) Server at example.com Port 443

In order to test if the Processwire .htaccess file is detected, I added some random characters at top, which should break the site and throw a 500 error. However, nothing happens. It still loads the homepage and 404 errors for the subpages. .htaccess syntax checks out ok.

What am I missing?

Thanks for your feedback.

 

[Edison]

Hi @Neo, I gave a quick view to your virtual host conf file and it seems ok. However I see you are using SSL and virtual host:80 automatically rewrites to https. I guess you just did not mention, but just to double check... do you have a second virtual host for https? I have a second virtual host:443, which, if I well recall was created by Letsencrypt with a name similar to example.com-le-ssl.conf, and that also needs to be enabled.

You may wish to have a look at apache error.log/access.log as this could help to identify what is the cause of the problem.

Last question, is PW installed in the root directory (likely ?) or in a subdirectory?

 

[Neo]

Thanks for your feedback, @Edison. You were, right. Let's Encrypt created another virtual host file for port :443 to which I had to add the directory rules and restart the server:

<Directory /var/www/example.com/public_html/>
	Options FollowSymLinks
	AllowOverride All
	Require all granted
</Directory>

So my links are working now. However, I am encountering another problem. Some pages are listed as blank.

When I enable the debug mode, I get the following errors:

Warning: file_put_contents(/var/www/example.com/public_html/site/assets/cache/FileCompiler/site/modules/AllInOneMinify/AllInOneMinify.module): failed to open stream: Permission denied in /var/www/example.com/public_html/wire/core/FileCompiler.php on line 242

Warning: file_put_contents(/var/www/example.com/public_html/site/assets/cache/FileCompiler/site/templates/_init.php): failed to open stream: Permission denied in /var/www/example.com/public_html/wire/core/FileCompiler.php on line 242

Warning: file_put_contents(/var/www/example.com/public_html/site/assets/cache/FileCompiler/site/templates/services.php): failed to open stream: Permission denied in /var/www/example.com/public_html/wire/core/FileCompiler.php on line 242

Warning: file_put_contents(/var/www/example.com/public_html/site/assets/cache/FileCompiler/site/templates/_func.php): failed to open stream: Permission denied in /var/www/example.com/public_html/wire/core/FileCompiler.php on line 242

Permissions for directories are set to 755 and for files to 644 in the Processwire installation.

Any idea what could be wrong?

 

[teppo]

1 hour ago, Neo said:

Warning: file_put_contents(/var/www/example.com/public_html/site/assets/cache/FileCompiler/site/modules/AllInOneMinify/AllInOneMinify.module): failed to open stream: Permission denied in /var/www/example.com/public_html/wire/core/FileCompiler.php on line 242

Warning: file_put_contents(/var/www/example.com/public_html/site/assets/cache/FileCompiler/site/templates/_init.php): failed to open stream: Permission denied in /var/www/example.com/public_html/wire/core/FileCompiler.php on line 242

Warning: file_put_contents(/var/www/example.com/public_html/site/assets/cache/FileCompiler/site/templates/services.php): failed to open stream: Permission denied in /var/www/example.com/public_html/wire/core/FileCompiler.php on line 242

Warning: file_put_contents(/var/www/example.com/public_html/site/assets/cache/FileCompiler/site/templates/_func.php): failed to open stream: Permission denied in /var/www/example.com/public_html/wire/core/FileCompiler.php on line 242

Permissions for directories are set to 755 and for files to 644 in the Processwire installation.

Any idea what could be wrong?

644 means that only owner is allowed to write to the files. I'd start by checking if you already have matching files (the ones mentioned in the warnings), and if so, which user is set as their owner (and if the permissions to said files is really 644). In case the owner is not the same as the Apache user, that's likely the problem here.

If you've migrated these files with the site, a safe bet would be to clear everything within the FileCompiler directory to get a "blank state" – but if permissions are too strict, that obviously won't help for long.

(Note: if you sometimes run ProcessWire via CLI using the bootstrap method as another user, that can also result in these compiled files being owned by the user you've executed that CLI command with, which can cause similar issues.)

[Edison]

Hi @Neo, I have the same permissions 755-644 in PW config. I think I encountered this problem when I initially set them. It was depending from Apache process's owner.

Please create a simple php file in the root including the following and run it from the browser:

<?php echo `whoami`;

If it returns "apache" or "www-data" it should not be possible to use restricted permissions. You need to get your web user name (your website name) to support them.

If you get the first output, it is possible to solve it using php-fpm and creating separated pools. Just le me know if you want to go for that, I can describe all the steps.

[teppo]

7 minutes ago, Edison said:

If it returns "apache" or "www-data" it should not be possible to use restricted permissions. You need to get your web user name (your website name) to support them.

If you get the first output, it is possible to solve it using php-fpm and creating separated pools. Just le me know if you want to go for that, I can describe all the steps.

It's quite possible that I've misunderstood what you're saying here, but if by "restricted permissions" you mean current settings (755/644), then I don't really see a problem here – as long as the Apache user (apache, www-data, or whatever it is called in this case) is the owner of the FileCompiler directory and compiled files are created by this user (via web requests), it should work just fine ?

[Edison]

Hi @teppo, yes you are right. I should have mentioned in my case the user owner is not the same as the Apache user. ?

[Neo]

As suggested, I created a test.php with the following content:

<?php echo exec('whoami');

Probably as expected, the result is the following:

www-data

When I installed Debian, I created a new sudo user for security reasons, let's call him "MyUser" in this case, and assigned the ownership of the Processwire installation to that user:

sudo chown -R $USER:$USER /var/www/example.com/public_html

This is a convenient setup, because it allows me to quickly edit files via SFTP and Filezilla without getting a permission error.

However, it seems to cause the permission issue now as all files / directories are assigned to owner "MyUser" and group "MyUser".

So what would be the suggested way to give Apache the required permissions, keep convenient writing access via Filezilla for my sudo user and maintain a secure installation?

[teppo]

The easiest approach might be setting the owner of the whole /site/assets/ directory (and everything in it) as the Apache user (www-data:www-data).

If you want to retain easy write access to those directories (without sudo), you can set your own user as the owner, the owning group as www-data, and change the permissions for that directory to 664 instead of 644 – though depending on your setup when www-data creates new files and directories, it will likely overwrite the owner for those files anyway. Usually this isn't a problem, though, since you shouldn't upload files to /site/assets/ manually, and even then it would only be limited to files that www-data is already managing ?

[Neo]

Following your advice, I adjusted file ownership and permissions accordingly @teppo :

sudo chown -R MyUser:www-data /var/www/example.com/public_html

find /var/www/example.com/public_html/site/assets/ -type f -exec chmod 664 {} \;

All pages are loading fine now... ?

So I assume that would be a safe configuration to use, considering that only my sudo user and the Apache group now have write access to the files contained in site/assets/, correct?

Edit:

When trying to edit and save pages in the backend now, I am encountering a couple of interesting notifications / errors:

Notice: A non well formed numeric value encountered in /var/www/example.com/public_html/wire/modules/Inputfield/InputfieldFile/InputfieldFile.module on line 124

Session: destinationPath is not writable

 

[teppo]

55 minutes ago, Neo said:

Following your advice, I adjusted file ownership and permissions accordingly @teppo :

sudo chown -R MyUser:www-data /var/www/example.com/public_html

find /var/www/example.com/public_html/site/assets/ -type f -exec chmod 664 {} \;

All pages are loading fine now... ?

So I assume that would be a safe configuration to use, considering that only my sudo user and the Apache group now have write access to the files contained in site/assets/, correct?

I wouldn't necessarily recommend making www-data the owning group of the entire directory, but that's just me. As long as the group doesn't have write access to all files, that doesn't really make matter a whole lot. Technically Apache only needs write access to /site/assets/, and (if you want to use the built-in modules installer in Admin) the /site/modules/ directory. (Latter is something I personally never allow, as I don't think it's worth the risk.)

Anyway, in my opinion the setup you've described above should be safe. When configuring the permissions of a system like this, there are always two important factors:

1. Are there other users in addition to you who have access to the system, i.e. is it a shared system? If so, giving "others" write/read/execute permissions can sometimes be problematic, as it could technically allow any logged-in system user to access those files – though in shared systems some sort of jail mechanism is usually implemented to prevent this. Also, I'm assuming that this isn't the case here, since you've mentioned setting up the server yourself.
2. The Apache user (www-data) is what most "third parties" will technically access your site with (through the web server). While ProcessWire is known to be exceptionally secure, it's still a good idea to limit the excess to which this user/group can access (and particularly write to) content on your server. For an example, I prefer not to give www-data write access to directories that contain executable code, such as /site/modules/ (as mentioned above).

In the case of ProcessWire www-data needs write access to /site/assets/files/, or at the very least specific directories below it. It also needs read access to most files on your site. So yeah – your settings make sense to me ?

55 minutes ago, Neo said:

When trying to edit and save pages in the backend now, I am encountering a couple of interesting notifications / errors:

Notice: A non well formed numeric value encountered in /var/www/example.com/public_html/wire/modules/Inputfield/InputfieldFile/InputfieldFile.module on line 124

Session: destinationPath is not writable

 

This is strange: it seems that a file or image field on your site is saying that the assets/files/ directory it's trying to write to is not writable. I'd start by checking which user/group is the owner of the /site/assets/files/[page ID]/ directory, and if it's writable for www-data. Same thing for the /site/assets/files/ directory. Alternatively: is it possible that sometimes the user actually isn't www-data?

Just guessing here, since I'm not familiar with your particular setup.

Edit: just realised that in your message above you mentioned adding chmod 664 to all files within /site/assets/. Please make sure that group has write access to directories as well – otherwise you'll no doubt run into problems.

Edited August 3, 2019 by teppo

[Neo]

Following your feedback, I was able to get it fixed with the following permissions @teppo :

# Reset ownership to my sudo user

sudo chown -R $USER:$USER /var/www/example.com/public_html

# Make www-data the owning group of /site/assets 

sudo chown -R $USER:www-data /var/www/example.com/public_html/site/assets/

# Give recursively 664 permission to all files within the /site/assets folder 

find /var/www/example.com/public_html/site/assets/ -type f -exec chmod 664 {} \;

# Give recursively 775 permission to all directories within the /site/assets folder

find /var/www/example.com/public_html/site/assets/ -type d -exec chmod 775 {} \;

User and group can now write files AND directories within /site/assets while the rest of the installation is only owned by my user.

I am the only user of this VPS.

I am not getting an error message in the backend any longer when saving and publishing a page change, which now works fine.

However, the following error persists on page save:

Notice: A non well formed numeric value encountered in /var/www/example.com/public_html/wire/modules/Inputfield/InputfieldFile/InputfieldFile.module on line 124

I also get this, when visiting the modules page in the backend:

Notice: A non well formed numeric value encountered in /var/www/example.com/public_html/wire/modules/Inputfield/InputfieldFile/InputfieldFile.module on line 124

Warning: count(): Parameter must be an array or an object that implements Countable in /var/www/example.com/public_html/wire/modules/Inputfield/InputfieldFile/InputfieldFile.module on line 439

Warning: count(): Parameter must be an array or an object that implements Countable in /var/www/example.com/public_html/wire/modules/Inputfield/InputfieldFile/InputfieldFile.module on line 148

That does not sound like a permission issue, but rather a problem or potential bug in the module that handles the inputfield for file uploads.

Isn't that a ProcessWire core module?

On the front-end I am not using any forms that would utilize a file upload field. So I assume this must relate to the standard image upload fields in the backend somewhere, where I did not make any modifications at all. The site is based on ProcessWire 3.0.32.

[teppo]

51 minutes ago, Neo said:

However, the following error persists on page save:

Notice: A non well formed numeric value encountered in /var/www/example.com/public_html/wire/modules/Inputfield/InputfieldFile/InputfieldFile.module on line 124

I also get this, when visiting the modules page in the backend:

Notice: A non well formed numeric value encountered in /var/www/example.com/public_html/wire/modules/Inputfield/InputfieldFile/InputfieldFile.module on line 124

Warning: count(): Parameter must be an array or an object that implements Countable in /var/www/example.com/public_html/wire/modules/Inputfield/InputfieldFile/InputfieldFile.module on line 439

Warning: count(): Parameter must be an array or an object that implements Countable in /var/www/example.com/public_html/wire/modules/Inputfield/InputfieldFile/InputfieldFile.module on line 148

That does not sound like a permission issue, but rather a problem or potential bug in the module that handles the inputfield for file uploads.

Isn't that a ProcessWire core module?

On the front-end I am not using any forms that would utilize a file upload field. So I assume this must relate to the standard image upload fields in the backend somewhere, where I did not make any modifications at all. The site is based on ProcessWire 3.0.32.

I'm not entirely sure about the "non well formed numeric value" problem, but your version of ProcessWire (3.0.32) actually predates our current GitHub repository, which means that it's ~3 years old – I wouldn't be too surprised if this was something that has since then been fixed. Of course it would require a bit of testing, but I'd suggest updating your site to a more recent version if possible.

Countable notices are also problems that have been fixed in the dev branch, see this issue: https://github.com/processwire/processwire-issues/issues/408. If it's the same issue, this is due to a change in PHP 7.2, where they changed the count() behaviour in a backwards-incompatible way.

Note that at least some of those countable-related fixes are currently only fixed in the dev branch; in the case of ProcessWire the dev branch tends to be quite stable, but if this is a production site you may still consider living with some of these warnings until these fixes are merged to master.

[Neo]

Thanks for your help and feedback @teppo .

The warnings do not interfere with the functionality of the site, but I will consider to update the ProcessWire version as soon as a find some time.