Memory exhausted

[driedstr]

Hey folks,

First time here so: thanks devs and community for creating such a great thing! ProcessWire was a treat to work with and I hope I get to use it on future projects.

However, I have a perplexing issue. I launched a site a few weeks ago and revisited it yesterday to find the following error:

Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 2441118600 bytes) in /srv/users/serverpilot/apps/artsdeir/public/wire/core/WireData.php on line 19

Warning: Cannot modify header information - headers already sent by (output started at /srv/users/serverpilot/apps/artsdeir/public/wire/core/WireData.php:19) in/srv/users/serverpilot/apps/artsdeir/public/wire/core/shutdown.php on line 119
Unable to complete this request due to an error.

Confusingly, the site is working fine in my dev environment. Live, I can access files in the public directory but anything that uses the pw api spits backs the same error. The database dumpfile is effectively identical to that from my dev environment and since launch I've neither changed any of the files nor updated pw (using v2.6).

Can anyone suggest why pw might be trying to grab over 2gb of memory? As far as I could tell with dropping memory_get_usage() statements into the source files, it's borking at line ~100 in /index.php at 

$config = new Config();

I did do some poking around with the ServerPilot Apache/Nginx config files shortly after launch, but when I left it, the site was working fine.

Any leads are greatly appreciated!

[kongondo]

Hi driedstr,

Welcome to PW and the forums.

Here are some typical causes of that PHP error: 

http://stackoverflow.com/questions/8028957/how-to-fix-headers-already-sent-error-in-php

Any of those apply to you?

[driedstr]

Hi driedstr,

Welcome to PW and the forums.

Here are some typical causes of that PHP error: 

http://stackoverflow.com/questions/8028957/how-to-fix-headers-already-sent-error-in-php

Any of those apply to you?

Hi Kongondo. I am assuming the memory allocation error is the bigger problem (and is causing the headers already sent error) -- is this correct, or am I chasing the wrong bug?

[kongondo]

I am guessing they are related...

Applying this to your case...

Here "line 100" refers to the script where the header() invocation failed.

The "output started at" note within the parenthesis is more significant. It denominates the source of previous output. In this example it's auth.php and line 52. That's where you had to look for premature output.

It means line 119 refers to the script where the header() invocation failed (i.e. in shutdown.php). However, the source is line 19 in WireData.php. In my PW version, that's the beginning of the Class WireData. So unless you touched that, then it is probably something calling WireData.php that's failing.. Hard to tell though. The last time I got this error was trying to delete some page via API and it didn't work and it got stuck in memory and had to restart Apache...I know this is probably not too helpful...

[kongondo]

Errors in ProcessWire logs? A server error, etc?

[driedstr]

I am guessing they are related...

Applying this to your case...

It means line 119 refers to the script where the header() invocation failed (i.e. in shutdown.php). However, the source is line 19 in WireData.php. In my PW version, that's the beginning of the Class WireData. So unless you touched that, then it is probably something calling WireData.php that's failing.. Hard to tell though. The last time I got this error was trying to delete some page via API and it didn't work and it got stuck in memory and had to restart Apache...I know this is probably not too helpful...

I'm pretty sure it's the $config = new Config(); statement in the index.php I mentioned in the first post (Config extends WireData). I get this from echoing the memory usage before the class definitions and at various points in index.php. The Config definition is loaded, WireData is loaded, and the next line -- $config->urls = new Paths($rootURL); -- is never reached.

I don't really have a deep knowledge of how the framework functions internally, but I'm guessing there's some sort of recursion happening somewhere that's somehow appending data in memory to itself.

There is no error.txt, only modules.txt, session.txt, system-updater.txt, none of which have anything exciting. Restarting Apache and nginx has no effect.

Thanks for helping me with this, kongondo.

[horst]

Hi @driedstr and welcome to the forums.

If you are sure it is this line when start building $config, it can be worth to have a look what you put into your site/config.php ??

Have you put customized stuff / settings into it?

[Webrocker]

Hi @driedstr

welcome to the forum

I'm with @horst on that - most likely there's something in your site/config that's working fine in your local env, but not on the server.

I had some memory alloc errors with processwire, but usually only with extensive api things working on several thousand page objects.

Maybe looking directly on the server with 'top' gives a clue what eats the memory?

cheers

tom

[cstevensjr]

I read this over and over.   What stood out, to me, was these two statements:

I did do some poking around with the ServerPilot Apache/Nginx config files shortly after launch, but when I left it, the site was working fine.

Restarting Apache and nginx has no effect.

I don't know what effect this has on the stated problem, however a little more information regarding that configuration (ServerPilot Apache/Nginx config files), I believe, is warranted.

In addition, what @kongondo stated in reply #2 is where the PW error logs would lead one to first try to eliminate/rule out.

[driedstr]

Hi @driedstr and welcome to the forums.

If you are sure it is this line when start building $config, it can be worth to have a look what you put into your site/config.php ??

Have you put customized stuff / settings into it?

Hello horst. I haven't changed config.php beyond the required settings. Here it is, comments & sensitive data removed:

<?php

/**
 * ProcessWire Configuration File
 * [...]
 */

if(!defined("PROCESSWIRE")) die();

/*** SITE CONFIG *************************************************************************/

/**
 * Enable debug mode?
 * [...]
 */
$config->debug = false;

/*** INSTALLER CONFIG ********************************************************************/

/**
 * Installer: Database Configuration
 * 
 */
$config->dbHost = /* [...] */;
$config->dbName = /* [...] */;
$config->dbUser = /* [...] */;
$config->dbPass = /* [...] */;
$config->dbPort = /* [...] */;

/**
 * Installer: User Authentication Salt 
 * [...]
 */
$config->userAuthSalt = /* [...] */; 

/**
 * Installer: File Permission Configuration
 * 
 */
$config->chmodDir = '0755'; // permission for directories created by ProcessWire
$config->chmodFile = '0644'; // permission for files created by ProcessWire 

/**
 * Installer: Time zone setting
 * 
 */
$config->timezone = 'America/Toronto';

/**
 * Installer: HTTP Hosts Whitelist
 * 
 */
$config->httpHosts = array('artsdeir.ca', 'www.artsdeir.ca');

Hi @driedstr

welcome to the forum

I'm with @horst on that - most likely there's something in your site/config that's working fine in your local env, but not on the server.

I had some memory alloc errors with processwire, but usually only with extensive api things working on several thousand page objects.

Maybe looking directly on the server with 'top' gives a clue what eats the memory?

cheers

tom

Hi Webrocker.

The site has maybe a few dozen page objects, so I don't think it's that. The heavy lifting that the site does is in some form processing. After checking the incoming form data, it's encrypted with libsodium and stashed in an sql table as a blob. A module I developed allows the admin to list the stored entries without decrypting them, then select one for download. The download process involves decrypting the blob data, generating a PDF with dompdf, then encrypting that with phpaes. The resulting .aes files are ~3kb, and temp files are being deleted properly as far as I can see.

A few items pop to the top of top when requesting the site. Nothing breaks 1% CPU or 2.1% MEM:

• httpd
• nginx
• sp-agent
• mysqld 

I read this over and over.   What stood out, to me, was these two statements:

I don't know what effect this has on the stated problem, however a little more information regarding that configuration (ServerPilot Apache/Nginx config files), I believe, is warranted.

In addition, what @kongondo stated in reply #2 is where the PW error logs would lead one to first try to eliminate/rule out.

Hi cstevensjr. Here are the serverpilot apache and nginx config files. I had tried a few things with nginx main.custom.conf, but reverted to default when I was unsuccessful and deleted the file. The goal was to try to set up an unsigned SSL certificate to use with CloudFlare's free service (without paying for the next tier of serverpilot), but I might just buy it from serverpilot. I haven't used nginx before, so that part of the server config is fairly opaque to me.

apache main.conf and main.custom.conf (identical):

AcceptPathInfo on

DirectoryIndex index.html index.htm index.php

<Directory ${DOCUMENT_ROOT}>
    AllowOverride All
    Require all granted

    RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_URI} !-f
    RewriteRule \.php$ - [R=404]
</Directory>

RewriteEngine On
RewriteCond %{HTTP:Authorization} .+
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]

<Files *.php>
    SetHandler proxy:${PHP_PROXY_URL}
</Files>

<Proxy ${PHP_PROXY_URL}>
    ProxySet timeout=3600 retry=0
</Proxy>

nginx main.conf:

location / {
    proxy_pass      $backend_protocol://$backend_host:$backend_port;
}

Thank you all for your help!

[BitPoet]

I'm not completely convinced that the error happens where you think it does, driedstr. The Config class has no constructor at all, and Paths' one simply sets its $_root member variable.

Have you tried downloading a copy of the online version and verifying with a directory diff tool like meld that the php files are really identical? Does a downloaded copy show the same error?

If that doesn't help, also check INCLUDE_PATHs. Somewhere an identically named file might get included and cause the loop. If in doubt, xdebug could help you get a full trace and spot the offending file.

[horst]

additionally you could/should set $config->debug = true; in the site/config.php. So, maybe it isn't of help in this case.

[driedstr]

Good morning from Toronto. Curious development: the site is now only trying to claim 1873625992 bytes. Not sure what caused it, the only things I've been able to try (via SSH from my phone due to limited internet access) that might have changed it are setting $config->debug = true; (again and still to no avail), and setting a few loops over a couple PageArrays to iterate instead over the php array returned from getArray().

I'm not completely convinced that the error happens where you think it does, driedstr. The Config class has no constructor at all, and Paths' one simply sets its $_root member variable.

Have you tried downloading a copy of the online version and verifying with a directory diff tool like meld that the php files are really identical? Does a downloaded copy show the same error?

If that doesn't help, also check INCLUDE_PATHs. Somewhere an identically named file might get included and cause the loop. If in doubt, xdebug could help you get a full trace and spot the offending file.

Just tried diffing site/templates, site/modules, and wire/core, no substantial differences beyond a url reference in a js file. PHP versions in dev and live are the same.

You're probably right about the error location. Disabling the header() call in ProcessWireShutdown() (wire/core/shutdown.php) then dumping error_get_last() spits out

Array
(
    [type] => 8
    [message] => Trying to get property of non-object
    [file] => /srv/users/serverpilot/apps/artsdeir/public/wire/core/Wire.php
    [line] => 112
)

Wire.php:112 is part of the following block:

	 /**
	 * Get the Fuel specified by $name or NULL if it doesn't exist
	 *
	 * @param string $name
	 * @return mixed|null
	 * @internal Fuel is an internal-only keyword.  
	 * 	Use $this->wire(name) or $this->wire()->name instead, unless static is required.
	 * @deprecated
	 *
	 */
	public static function getFuel($name = '') {
		if(empty($name)) return self::$fuel;
		return self::$fuel->$name;
	}  

I'm not really sure what the Fuel is or why I might be running out of it, but maybe it rings some bells for someone else?

I'll start scanning through files for potentially problematic include() statements in a few minutes.

additionally you could/should set $config->debug = true; in the site/config.php. So, maybe it isn't of help in this case.

Unfortunately there is no change in behaviour and no error logs (either in the live or dev environments).

[BitPoet]

I'm not really sure what the Fuel is or why I might be running out of it, but maybe it rings some bells for someone else?

 

That's probably a red herring. Initialization fails early on due to memory exhaustion, so it makes sense that Wire::$fuel is unset, as it's the property in which the Wire class stores all the magic variables you call up by invoking wire('users'), wire('pages') etc.

 

While you're digging through the includes, have an eye out for anything that re-assigns to a reserved ProcessWire variable like e.g. $config or $users.

[driedstr]

I just installed xdebug to get a trace and the site is working again -- I'm guessing as a result of restarting php (which I, er, failed to try when restarting apache and nginx  ). These are the most costly calls from the trace file (maybe they mean something, maybe not):

Showing the 5 most costly calls sorted by 'time-own'.
                                                     Inclusive        Own
function                                     #calls  time     memory  time     memory
--------------------------------------------------------------------------------------
ProcessWireClassLoader                           64  0.2980  3167496  0.2336  1645184
WireData->__get                                4611  0.3229    29144  0.0917     2240
WireData->get                                  3010  0.1937    11784  0.0850      152
PDOStatement->execute                            61  0.0706   297384  0.0706   297384
Modules->includeModule                           60  0.0928   895832  0.0689   464504

...

Showing the 5 most costly calls sorted by 'memory-own'.
                                                     Inclusive        Own
function                                     #calls  time     memory  time     memory
--------------------------------------------------------------------------------------
ProcessWireClassLoader                           64  0.2980  3167496  0.2336  1645184
array_merge                                     459  0.0089  1362320  0.0089  1362320
str_replace                                     311  0.0087  1106160  0.0087  1106160
require                                          71  0.8750  1990912  0.0279  1001896
Modules->includeModule                           60  0.0928   895832  0.0689   464504
 
...
 
Showing the 5 most costly calls sorted by 'memory-inclusive'.
                                                     Inclusive        Own
function                                     #calls  time     memory  time     memory
---------------------------------------------------------------------------------------
{main}                                            1  2.2128  5465832  0.0004    -6120
ProcessWire->__construct                          1  1.1367  4215712  0.0000     1008
ProcessWire->load                                 1  1.1354  4212632  0.0026     3400
ProcessWireClassLoader                           64  0.2980  3167496  0.2336  1645184
spl_autoload_call                                64  0.2997  3144592  0.0017   -22904
Wire->runHooks                                  215  1.5904  2633744  0.0033    14136

 

That's probably a red herring. Initialization fails early on due to memory exhaustion, so it makes sense that Wire::$fuel is unset, as it's the property in which the Wire class stores all the magic variables you call up by invoking wire('users'), wire('pages') etc.

While you're digging through the includes, have an eye out for anything that re-assigns to a reserved ProcessWire variable like e.g. $config or $users.

 

Thanks for the warning . Changed all includes to include_once for good measure. No overwriting of any API variables taking place in my templates, but dompdf uses the variables $page and $user -- might this be a problem? dompdf is used within my module and only from the admin side...

Happy to have the site working again, but worried that the problem might not be fully resolved. Thanks again to everyone for your insight, I'll keep poking at and monitoring this thing, hopefully the problem reveals itself...

[horst]

In witch scope does dompdf use them? (class internal? global?)

Also if your module does its work on the admin side, it can be that you use autoload with it? If yes, it is called very early.

You should thouroughly check if the $user and $page from dompdf and pw does not interfere.

[driedstr]

In witch scope does dompdf use them? (class internal? global?)

Also if your module does its work on the admin side, it can be that you use autoload with it? If yes, it is called very early.

You should thouroughly check if the $user and $page from dompdf and pw does not interfere.

Yes, the module is admin-side only, but it isn't autoload. $page and $user are both used class-internally and not globally. An instance of the dompdf class is only created when a PDF is generated from the backend so it really shouldn't interfere with the site loading. After watching the whole PDF rendering process with xdebug, it appears that the variables don't interfere -- $page and $user are the same before and after the PDF is generated.