Debugging Load Delay - Where to start

[opalepatrick]

Trying to work out what is causing an approx. 20 second when loading a page of thumbs for a product catalogue. I have debug = true without any issues. Also, I have checked the server error logs and nothing either. I have several processwire installations on this server all operating fine. Some with very similar setups. I have checked the selector with selector test module and it seems OK. Looking on chrome inspect, the problem is right at the start where the delay occurs. Once that passes, everything loads almost instantly. Also, this occurs on chrome & firefox. Oh, and I should say that no other page is affected that I am aware of.

So, as I am finding this a bit frustrating, I was wondering if anyone has any tips as how to debug this? Just in case I have put the code for the page below.

edit: One other thing the product_cats field is a page fieldtype.

<?php
require_once("inc/header.inc"); 
$type = $page->title;

$selector = "product_cats.title=" . $type;

$products = $pages->find("$selector, limit=12");
//print_r($products);
$pagination = $products->renderPager(array(
    'nextItemLabel' => "Next",
    'previousItemLabel' => "Prev",
    'listMarkup' => "<ul class='MarkupPagerNav pagination'>{out}</ul>",
    'currentItemClass' => "current",
    'firstItemClass' => "arrow",
    'lastItemClass' => "arrow"
));
?>
<div class="row firstrow lastrow">
	<div class="pagination-centered">
		<?=$pagination?>
	</div>
	<?php
	foreach($products AS $product) {
		$pageimage = $product->images->first();
	 	?>
	    <div class="small-6 medium-4 large-4 columns" itemscope itemtype="http://schema.org/Product">
			<a href="<?= $product->url ?>"><img src="<?=$pageimage->pimLoad('myPrefix')->canvas(350, 350, array(255,255,255,0))->pimSave()->url ?>" itemprop="image"></a>
			<div class="prod_panel panel">
				<h5 itemprop="name"><?= $product->id ?> <?= $product->title ?></h5>
			</div>
		</div>
		<?php
	}
	?>
	<div class="clearer"> </div>
	<div class="pagination-centered">
		<?=$pagination?>
	</div>
</div>
<?php
require_once("inc/footer.inc");

[Joss]

Just for interest, have you looked at the file size of the thumbs?

PIM maybe creating a higher quality thumb than you need.

[opalepatrick]

Thanks Joss, they are good at about 5k each. Also, most of them are loading from cache

[Soma]

Either the pim function for the image

Or what are you searching with?

$selector = "product_cats.title=" . $type;

$products = $pages->find("$selector, limit=12");

Is that a repeater? How many repeaters are there?

[Joss]

Other than that, try stripping out things like canvas and calls to schema.org and generally simplify it down and see what that achieves.

I sometimes kick out my headers and footers as well so I am not adding JS and CSS to the problem - doesn't matter what it looks like, just how fast it loads at that point.

Then start adding bits back ...

[opalepatrick]

Thanks Joss. I did strip the pim stuff and other bits, bit by bit, but not the headers. Should do that. That is a multiple page fieldtype Soma (not a repeater afaik). I did wonder if that was causing an issue.

edit: Stripped headers footers and pim and still got the delay. So, has to be the selector? I should have a loop?

[opalepatrick]

So, just following on from above. I have used that exact same script previously without a problem.

It appears to be specific to this setup. Basically when I run it, the cpu shoots up to 100% and stays there and stops every other site on my vps from doing anything.  It was working and I cannot for the life of me think what has happened that could have messed things up. I did run a few queries on the database directly without an issue, and on the server using top, the cpu problem is on the apache service (not mysqld).

Anyway... rambling. Bit stuck. Any comments gratefully received.

[Joss]

er... not knowing exactly what your set up here is, but it sounds like you are basically grabbing a huge amount of information and then trying to deal with it.

But I cant see where that is happening.

Now, your database query (find) it is looking for what exactly? The fact that you have used this script before sound like the problem may be in the quantity you are dealing with in this case, or something like that. In other words, not so much what you are returning but what you are searching.

If the CPU usage is rocketing, then it sounds like you are getting stuck in a loop somewhere - searching on itself or something.

Can you refine the search in anyway to make it more specific?

[Soma]

So if you remove all that code, it is gone?

You can also use 

$runtimer = Debug::timer();

... code

$runtimer = Debug::timer($runtimer);
echo "Time: $runtimer";

Edit: corrected syntax...

[opalepatrick]

Well, been investigating around in a few more circles

I can't execute Soma's code as the page isn't even loading now. It hangs completely using 100% of cpu. (Actually, I went to make a nice cup of horlicks and it had loaded after 5 minutes or so)

The max results on this page would be 10 or so products max. So not even scratching it.

I replaced wire btw, not that it did any good . Removed unwanted modules as well.

[Joss]

What on earth are you calling that is doing that?

just for an experiment, what happens to the page if you remove the query and its associated code completely? Just in case it is NOT the product query that is causing the problem

[opalepatrick]

Hi Joss. Minor breakthrough!

As you can imagine I have a lot for different categories using this script. I have been testing using just one category. I had stripped everything back to core and instead of looking for 'side chairs', I got bored and looked for sofas and bingo! instant result. So then I restored everything and checked every category without a problem save for that single category 'side chairs' - Really bizarre.

So, now I am looking at the difference between that category and every other. Strange one as all I am doing is matching the category with the category field in products. Same as for all the others. 

[Joss]

Well, at least that means that your script structure is probably fine.

Thinking back to the maintenance man in my old studio who used to shout to us to check the plug was in the wall before phoning him saying "its broke again," start going through the obvious.

I am assuming that the categories are a group of pages under a set parent using the same parent. And you are using the title field to search by.

To start with, the problem with using the title field is if you rename the category you might have an issue.

So, just for interest, search by id instead. See if that is okay.

[opalepatrick]

Thanks for all your patience Joss. I am using the scenario you referred to with the cats under one parent and the actual products just listed under a different parent.

I tried using the id, and it is a tad faster, although it doesn't resolve the issue. Now I am getting more to thinking it may be something to do with the page weight. So I am going to load up some of the other categories to match the number on the problem one and see what happens. The images are definitely not large, but it may be pointing to another issue re my server. So, I will return!

It's true about the handyman though

[Joss]

The quickest way to check the page weight is to stop rendering the images.

One thing I had with PIM the other day (the first time I had used it) was that in its default configuration, the images it was producing had a significantly higher file size than the originals, despite being smaller pixel wise. This was because it was saving at a much higher quality than I had saved the originals.

Then again, you only have 12 images here.

I am not sure if you are also overworking your code a bit (this is the point when Soma might tell you that I am talking rubbish - he knows WAY more than I do)

Just digging out a similar bit of code from an old blog of mine:

function getPostsbyCategory() {
	$thisCategory = wire("page");

	$posts = wire("pages")->find("post_category_select=$thisCategory");

	$out =" ";
	foreach ($posts as $post) {
		$text = implode(' ',array_slice(str_word_count($post->summary,1),0,$maxpostlength));
		$out .="<div class='posts clearfix'>";
		$out .="<a href='{$thisCategory->url}{$post->name}'><h4>{$post->title}</h4></a>";
		$out .="$text...<a href='{$thisCategory->url}{$post->name}'>read more</a>";
		$out .="</div>";
	}
	
echo $out;
}

Ignore me truncating the text in the middle.

I don't appeared to have explicitly referenced the title field, but I cant remember why. I have left out any limits or pagination here.

[opalepatrick]

The previous advice you gave about checking plugs was on the money, in the sense that it paid to go back to the beginning and step it through.

It was caused by the images. Two things compounding. One, I had for some reason uploaded very large images, something I normally never do, but if I remember, I was checking pim and forgot to go back and optimise afterwards. Also, that was compounded by me not using pim properly and stating the quality. I also have a feeling (may be wrong) that the image was being resized every time as opposed to the cache being used. But  can't be sure about that. It may have just been the sheer load on the page.

I have reverted to previous practice and optimised the images first. Quick with imagemagick and everything appears to be sorted. So with your help, Joss, and Soma's I have managed to swirl back up and out of the rabbit hole I got myself into. Thanks once again. By the way, I reverted to just using size() as well.

[Joss]

Yeah, that damned mains plug in the wall - gets you every time...

Good luck with the rest of the site!