Jump to content

Delete 100 000+ pages


Roberts R
 Share

Recommended Posts

Last night I run into issue. I was trying to empty my trash that have 140 000+ pages init. All pages have repeater fields. When I pressed "Empty" on trash menu I wasn't able to use back-end for 2h+ (I kept refreshing page when It time-outed). After PW process was done  30k pages were deleted. I'm still left with 110k pages to remove.

Is there a way to safely remove all pages but with increased speed? Thanks

Link to comment
Share on other sites

You can use wireshell as well. This is really handy. Just give the following command and grab a coffee ;)

wireshell page:emptytrash

I'm not really sure if it will delete everything in one run, but I deleted 60k pages the other day without problems. Really fast too and the main advantage is that you are not tied to a PHP session.

  • Like 8
Link to comment
Share on other sites

  • 10 months later...

I have a site that I oftentimes sync the live database to my dev database (I made a bash script to automate the process).  I generally like to "clean" the database once it has been copied over to my dev machine, which involves running a script that deletes 15,000 pages (orders in my case), among other things.  Doing this using $page->delete() in a script (which I'm running through the command line for added performance), takes about 30 minutes which is painful.

I thought it through further and I came up with the following relatively simple script that can achieve the same result in a few seconds!  It achieves this speed by running MySQL delete queries directly, bypassing PW's API.

Here it is (modify accordingly):

<?php namespace ProcessWire; ?>
<?php
include(dirname(__FILE__).'/index.php'); // bootstrap pw

$order_ids = $pages->findIDs('parent=/orders/,template=order');
if(!count($order_ids)) exit;
$order_ids = implode(",", $order_ids);

$t = $templates->get("order");
foreach($t->fields as $f) {
  if( $f->type!="FieldtypeFieldsetOpen" && $f->type!="FieldtypeFieldsetClose" ) {
    $table = "field_".$f->name;
    $database->query("DELETE FROM $table WHERE pages_id IN ($order_ids)");
  }
}
$database->query("DELETE FROM pages WHERE id IN ($order_ids)");

 

  • Like 7
  • Thanks 1
Link to comment
Share on other sites

On 10/11/2017 at 7:33 AM, Jonathan Lahijani said:

It achieves this speed by running MySQL delete queries directly, bypassing PW's API.

Kind of pointing out the obvious here, but that could be asking for trouble. A number of modules (including core ones, such as Repeaters) perform cleanup etc. with hooks. Unless you're really careful, you could easily leave a ton of orphaned data behind with this method.

  • Like 6
  • Thanks 1
Link to comment
Share on other sites

In the past, I've had to delete multiple tens of thousands of records (pages) and did it by pushing the ids of the pages to be deleted into a WireQueue and then, at regular intervals (think cron-driven), pull out small batches of them to delete. To completely empty the queue takes a long time, but the admin interface remains responsive and you simply need to check the number of items remaining in the queue to know how far through the process you are.

  • Like 1
  • Thanks 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...