Performance issues with only 2000 pages

[gebeer]

Helo,

I have a site running on PW 2.7.2 stable that is solely used for data logging. No frontend code at all.

Every log entry is saved as a page.

The template for the log pages has 5 fields:

The channels field is a Pro Tables field with 4 columns:

The data stored for the logs looks like this:

Apart from that there are only about 10 more pages and about 12 templates sharing 12 fields amongst them.

The logs are coming in from another PW site via REST API and are stored every minute. So no big overhead there.

ATM there is only one log coming in every minute. But for the future there will be 100 or even 1000 logs per minute.

The logs will be stored in the DB for 1 month, then stored away to csv and deleted from the DB.

I am experiencing quite slow page loading already for the login screen and in the backend once there are more than 1500 log pages. Also loading the listing for the logs with Lister Pro takes quite some time (about 15 seconds for loading of approx 2000 logs and paginating them).

The site is hosted on a godaddy 2GB VPS at the moment which hosts only one more site with no heavy traffic. What I can see from the server side there is not much average CPU load but quite some memory usage.

I wonder how performance will be once there are 10000s of log pages in the DB (will know in about 2 weeks).

Only 1 log per minute already amounts to over 40.000 log pages a month.

Do you think PW can scale well with about 400.000 - 1 million pages (then on a better server, of course) ?

[gebeer]

Update on performance:

I copied the install to my local vagrant development box where it is working smoothly. No long page loads and Lister Pro is fast even with 5000 logs.

So the performance issues are related to the godaddy server - who would have thought that

[Pete]

"I said go daddy, stop lazing about on the sofa"

[ryan]

I recently installed a client site at GoDaddy (I tried to talk them out of it, but had to work with what they had). It was one of those $5/month sites, not a VPS. I had to change the RewriteBase in the .htaccess file to "/", but otherwise found that it worked just fine. Granted it's not speedy, but everything runs acceptably for a low traffic site. With ProCache installed it's actually quite fast. 

Based on the things you've said, I would guess that there's a bottleneck somewhere, and that the GoDaddy server is just making that bottleneck more obvious. Speed should not be significantly affected by quantity of pages in the site, so it points to something, somewhere in the site that is loading a large amount of pages (most likely a hook or module). I would enable debug mode, and then view the Debug info in the bottom of the admin screen on one of the slow loading pages (though not Lister, since it ajax-loads its pages separately).  Click on the "Pages" section to see what's being loaded. I'm guessing you might have something in there loading all 2000 pages. This might help to point to the source of the bottleneck. 

[gebeer]

@ryan

Thank you for taking the time to comment on this.

There are no custom hooks running on that site.

Installed modules:

-ProFields: Table

-ListerPro

-Database Backups

-Upgrades

As performance is fine on my vagrant dev box, I think the problem is related to the godaddy VPS. It was very slow 2 days ago. Now it is much better again. I saw that more than 1.5 of the 2 GB RAM of the VPS are in use and have to investigate where this originates from.

The second PW install that is running on the same VPS has some quite complex code running and I am about to profile that site and then go on from there.

For the long run my client will eventually move away from godaddy as we are expecting quite some traffic on the project.