[SOLVED?] $pages->uncacheAll() doesn't seem to be working.

[elabx]

Hi! I am iterating through a very big CSV that check for existing pages on a PW installation . This all works fine on local development and if the records already exists in PW, I basically skip to the next iteration. This was initially causing memory issues until i added a $pages->uncacheAll() method, which cleared the result of the $page->get() from memory. Note, this is running through a CLI script.

foreach($csv as $i => $record){

echo round(memory_get_usage(true) / 1048576.2) . PHP_EOL;
$selector = "template=property";
$existing = $pages->get($selector);

    if ($existing->id) {
   
        $pages->uncacheAll();
        continue;
    }
}

Unfortunately on the AWS Lightsail VPS I'm using, it doesn't seem to matter and memory usage just keeps growing.

Am I missing some setting on PHP configuration maybe?

I already checked the memory for the CSV iteration only (using PHP League CSV library) and it keeps memory in a fixed value (pretty much how it happens locally while iterating and using uncacheAll() to clear memory).

Adding gc_collect_cycles() doesn't seem to do anything either.

[Robin S]

If you don't need to do anything with the existing page then you can avoid loading it:

if($pages->count($selector)) continue;

 

[adrian]

9 minutes ago, Robin S said:

If you don't need to do anything with the existing page then you can avoid loading it:

if($pages->count($selector)) continue;

 

I haven't used it yet, but would the new has() method also work in that case?
https://processwire.com/blog/posts/pw-3.0.153/

[elabx]

Just tested both and I still see the memory go up and up :S

Weird thing is the original setup just works okay on local even tested on another server previously and worked fine, it was all since I set it up in the AWS Ligthsail VPS that it failed, that's why my initial thought was some mystery php.ini setting. Even stripped down the script to the bare minimum to discard something else I was not seeing. Anyway, will keep on looking. 

[rick]

3 hours ago, elabx said:

Just tested both and I still see the memory go up and up :S

Weird thing is the original setup just works okay on local even tested on another server previously and worked fine, it was all since I set it up in the AWS Ligthsail VPS that it failed, that's why my initial thought was some mystery php.ini setting. Even stripped down the script to the bare minimum to discard something else I was not seeing. Anyway, will keep on looking. 

I am not familiar with AWS, but I am going to be using it with an upcoming project. My hesitation is based on other users having resource usage climb to higher price points based on AWS metrics. Meaning that AWS escalates usage parameters to move users into higher price brackets. It is only hearsay, so I don't have personal experience.

@elabx, Do you think this is the case with your setup?

[elabx]

The site will mostly be cached with ProCache so I'm hoping CPU usage will only be heavily used when making the first import run. I'm planning to do about 3-6 million pages. Not entirely sure of the spike costs cause I'm not relly handling hosting payments. 

UPADTE: I don't know what the heck happened but now the script  is performing ok with no memory spikes :S