Tool for managing redirects

[apeisa]

Is there any tool for managing 301 redirects? These can be configured on code or server level, but it is often needed to offer possibility to clients to add new redirects (or at least see what redirects there is). This is very important when making redesign and if url schema changes.

It is easy to create this at template level, so any page could be redirect to another page, but this is good for short urls (ie. www.domain.com/campaign/ => www.domain.com/products/offers/super-offer/) and should be 302 redirect. In larger usage it pollutes page tree.

So what I would like to see is somekind of tool for adding/editing/removing permanent redirects. Is there something like that? If I would build that as a module, what would be best route?

[ryan]

Is there any tool for managing 301 redirects? These can be configured on code or server level, but it is often needed to offer possibility to clients to add new redirects (or at least see what redirects there is). This is very important when making redesign and if url schema changes.

When replacing a site, I usually handle some of the 301 redirects in the .htaccess file:

RewriteRule ^about.html$ /about/ [R=permanent,L]

But a lot of these sites have thousands of URLs to replace. Very often, they are URLs with variables in them, like:

www.site.com/products.cfm?product_id=8573

To handle that, I'll add a field in ProcessWire called old_product_id, and add it to the new "product" template. When importing the products into the new site, I make sure that old_product_id field gets populated with it's ID from the old site. If you are dealing with thousands of URLs, you want to make sure to automate this.

Next, I'd either create a page in ProcessWire named "product.cfm" (along with a template), or create an actual file on the server called product.cfm. I'm more likely to create the file, since I don't really like pages from the old site in the new site tree. So I'd name that file /product.cfm and update the .htaccess file to run cfm files as php:

AddType application/x-httpd-php .cfm

Here's what the contents of product.cfm might look like:

<?php 

// load ProcessWire's API 
require("./index.php");

// if no 'id' var present, send them to the new products page
if(!isset($_GET['id'])) $wire->session->redirect("/products/");

// get the requested product id
$id = abs((int) $_GET['id']); 

// find the new page with that product id  
$page = $wire->pages->get("old_product_id=$id"); 

// if it was found, redirect them to it. 
if($page->id) $wire->session->redirect($page->url);
    // otherwise send them to the new products index
    else $wire->session->redirect("/products/"); 

This is just one approach... but one I had to implement recently, so figured I'd paste it in as an example.

It is easy to create this at template level, so any page could be redirect to another page, but this is good for short urls (ie. www.domain.com/campaign/ => www.domain.com/products/offers/super-offer/) and should be 302 redirect. In larger usage it pollutes page tree.

The template-level redirects are great for future redirects that the client might setup. And it's easy to add a field called redirect_href and then setup a template that deals with it, i.e.

/site/templates/redirect.php:

<?php
if($page->redirect_href) $session->redirect($page->redirect_href);  

Then your clients can select "redirect" as the template when they want to create a redirect. Though, they still might pollute the site tree over time with lots of extra and unnecessary redirect pages... I've seen it happen, and it's a problem.

If you want a 302 redirect, specify 'false' as the second argument to the redirect function, i.e. $session->redirect($url, false). So you might add a checkbox field to the page's template called 'permanent', and handle it like this:

$session->redirect($page->redirect_href, $page->permanent == 1); 

So what I would like to see is somekind of tool for adding/editing/removing permanent redirects. Is there something like that? If I would build that as a module, what would be best route?

There was in PW1 but not yet in PW2. I think it would be a worthwhile module to have, especially for the post-launch redirects. If you are interested in building a module, I think it would be a great. I'll be happy to add any of the necessary hooks into the core or assist with anything you'd like me to. As for best route, I'm thinking I would add a hook before 'ProcessPageView::execute' that checks the value of $_GET['it']. That value is what PW's htaccess file uses to send PW the requested page URL. You could also look at any of the $_SERVER[] variables, to see if any of them suit your needs better, but I'm guessing $_GET['it'] is what you need. If your module finds a matching URL, it would redirect to it. Otherwise, it would just return, letting ProcessPageView continue on about it's way.

A better solution might be to have your module hook into some function that only gets called when a page is not found. That way it wouldn't have to compare redirects for every request. I don't think I have a good function for you to hook into for that yet, but I will be happy to add one.

One thing I should mention is that ProcessWire only sees URLs with the path/file containing ASCII these characters: -_.a-zA-Z0-9/

That's applicable to the path/file only, not the query string. Those characters accounts for most URLs you are likely to run across, but ProcessWire won't ever see URLs with commas or other more rare characters that are technically allowed in URLs. This is because the htaccess file filters out any requests for those requests and never sends them to ProcessWire (since they don't follow PW's name format). I mention this just because any redirect module built into PW also wouldn't see URLs that don't follow that format.

Thanks,

Ryan

[apeisa]

Thanks for great answer. I have upcoming site with about 60 urls that need 301 redirect and I actually don't like doing this on .htaccess file since those old urls are also clean ones. So there is nothing like products.cfm?id=3232 that I could map. These are all custom ones so it might add lot's of confusion later on if these are "hidden" .htaccess file.

A better solution might be to have your module hook into some function that only gets called when a page is not found. That way it wouldn't have to compare redirects for every request. I don't think I have a good function for you to hook into for that yet, but I will be happy to add one.

I think this would be cleanest solution here. If there is a page, no redirect can be done (unless done at template level like described above). What I was thinking about would be new admin page: Redirects and also new permission "ProcessRedirects". Small custom table for permissions, which only holds two fields: pw_path and redirect_href, which take values like: /summer/ & http://www.domain.com/products/summer-special/

I'll start diving into this right away. I hope to get something released soon enough. Ryan, I probably need some help with permissions, but otherwise I have pretty good clue how this works (well, we'll see ).

[apeisa]

I started working on this, and I have a good start already. As last time when developing a module I find pw codebase very fun and easy to work with. Usually when looking for other coders work I don't have clue what is happening, but this stuff actually makes sense for me

Here is what I have so far:

<?php

class ProcessRedirects extends Process {

public static function getModuleInfo() {
	return array(
		'title' => 'Redirects',
		'summary' => 'Manage url redirects',
		'href' => 'http://processwire.com/talk/index.php/topic,167.0.html',
		'version' => 100,
		'permanent' => false,
		'autoload' => true,
		'singular' => true,
	);
}

public function init() {

}

public function ___execute() {
	$out = "Hello world! Here we will have CRUD for redirects.";
	return $out;
}

public function ___install() {
	parent::___install();

	$p = new Page();
	$p->template = $this->templates->get("admin");
	$p->parent = $this->pages->get($this->config->adminRootPageID);
	$p->title = 'Redirects';
	$p->name = 'redirects';
	$p->process = $this;
	$p->save();

	// I want different permission to view and to add/edit/remove redirects
	$this->installPermission('Edit');
}

public function ___uninstall() {
	$p = $this->pages->get('template=admin, name=redirects');
	$p->delete();
}
}

What classes should I use for creating simple crud for this? Table will be super simple with only two datafields (pw_path & redirect_href), so this would be easy to create in custom way, but I would like to this using pw classes as much as possible.

[ryan]

Looks like you've got a good start here!

To create the table, I would do it in your install function. The only part of PW that creates tables are the fieldtypes, and those aren't useful here, so you'll want to issue the create table yourself. Something like this:

<?php

public function ___install() {
    parent::___install();
    $p = new Page();
    $p->template = $this->templates->get("admin");
    $p->parent = $this->pages->get($this->config->adminRootPageID);
    $p->title = 'Redirects';
    $p->name = 'redirects';
    $p->process = $this;
    $p->save();

    // hold off on the permissions if you can because this system is being redesigned. 
    // $this->installPermission('Edit');

    $sql = <<< _END

    CREATE TABLE {$this->className} (
        id int unsigned NOT NULL auto_increment, 
        redirect_from varchar(512) NOT NULL DEFAULT '',
        redirect_to varchar(512) NOT NULL DEFAULT '',
        UNIQUE KEY(redirect_from)    
    ); 

_END; 

    $this->db->query($sql); 
}

public function ___uninstall() {
    $p = $this->pages->get('template=admin, name=redirects');
    $p->delete();
    $this->db->query("DROP TABLE {$this->className}"); 
}

For your CRUD, I would make your default function (execute) the one that lists the redirects, and then create functions to edit, save and delete. Here's an example of how you might do some of it as a starting point (written in the browser, so not tested!):

<?php

/**
* List Redirects - Called when the URL is: ./
*
*/
public function ___execute() {

    $this->setFuel('processHeadline', 'List Redirects'); 

    // example here uses a PW module to output this table, but that's not required. 

    $table = $this->modules->get("MarkupAdminDataTable"); 
    $table->headerRow(array('Redirect From', 'Redirect To', 'Delete')); 

    $result = $this->db->query("SELECT * FROM {$this->className} ORDER BY created"); 

    while($row = $result->fetch_assoc()) {
         // output in table rows with edit link and delete checkbox?
         $table->row(array(
             $row['redirect_from'] => "edit/?id=$row[id]",
             $row['redirect_to'] => "edit/?id=$row[id]", 
             "<input type='checkbox' name='delete[]' value='$row[id]' />"
             )); 
    }

    $table->action(array('Add Redirect' => 'edit/?id=0')); 
    
    return "<form action='./delete/' method='post'>" . $table->render() . "</form>";    
}

/**
* Edit/Add Redirect - Called when the URL is: ./edit/
*
*/
public function ___executeEdit() {

    $this->fuel->breadcrumbs->add(new Breadcrumb('./', 'Redirects'));     

    $id = (int) $this->input->get->id; 

    if($id > 0) {
        // edit existing record
        $result = $this->db->query("SELECT id, redirect_from, redirect_to FROM {$this->className} WHERE id=$id"); 
        list($id, $from, $to) = $result->fetch_array(); 
        $this->setFuel('processHeadline', "Edit Redirect"); 
        
    } else {
        // add new record 
        $id = 0;
        $from = '';
        $to = '';
        $this->setFuel('processHeadline', "Add Redirect"); 
    }

    $form = $this->modules->get("InputfieldForm");
    $form->method = 'post'; 
    $form->action = './save/';

    $field = $this->modules->get("InputfieldHidden");
    $field->name = 'id';
    $field->value = $id; 
    $form->add($field); 

    // maybe use InputfieldURL for the 'redirect_from' and 'redirect_to' inputs
   
    return $form->render(); 
}

/**
* Save Redirect - Called when the URL is ./save/
*
*/
public function ___executeSave() {

    // perform save of existing record if id>0, or add new if id=0 

    $this->message("Saved Redirect"); 
    $this->session->redirect("./"); // back to list
    
}

/**
* Delete Redirect - Called when URL is ./delete/
*
*/
public function ___executeDelete() {

    if(!is_array($input->post->delete) || empty($input->post->delete)) {
        $this->message("Nothing to delete"); 
        $this->session->redirect("./"); // back to list
    }

    foreach($input->post->delete as $id) {
        $id = (int) $id; 
        $this->db->query("DELETE FROM {$this->className} WHERE id=$id"); 
    }
   
    $this->message("Deleted " . count($input->post->delete) . " redirect(s)"); 
    $this->session->redirect("./"); // back to list
}

I will work on setting up an appropriate hook in the ProcessPageView module.

thanks,

Ryan

[apeisa]

Thank you Ryan! As always, this was super helpful. Just filling the blank spots and fixing few typos and that's it

We have now CRUD ready. There is few questions at the source, but it is pretty much working now. I also moved it under setup, so Redirects won't pollute the main menu at the admin. Few little issues and after that only grabbing the new hook to make actual redirects.

<?php

class ProcessRedirects extends Process {

public static function getModuleInfo() {
	return array(
		'title' => 'Redirects',
		'summary' => 'Manage url redirects',
		'href' => 'http://processwire.com/talk/index.php/topic,167.0.html',
		'version' => 100,
		'permanent' => false,
		'autoload' => true,
		'singular' => true,
	);
}

public function init() {

}

public function ___execute() {
	$this->setFuel('processHeadline', 'Redirects');

	$table = $this->modules->get("MarkupAdminDataTable");
	$table->headerRow(array('Redirect From', 'Redirect To', 'Delete'));

	$result = $this->db->query("SELECT * FROM {$this->className}");

	while($row = $result->fetch_assoc()) {

		 // output in table rows with edit link and delete checkbox?
		 $table->row(array(
			 $row['redirect_from'] => "edit/?id=$row[id]",
			 $row['redirect_to'] => "edit/?id=$row[id]",

			 //This actually doesn't work, row 40 at MarkupAdminDataTable class encodes this
			 //Should I build custom table or should we extend AdminDataTable?
			 "<input type='checkbox' name='delete[]' value='$row[id]' />"
			 ));
	}

	$button = $this->modules->get("InputfieldButton");
	$button->type = 'submit';
	$button->value = 'Remove selected redirects';

	$table->action(array('Add Redirect' => 'edit/?id=0'));

	// Is there clean way to add button to right side?
	return "<form action='./delete/' method='post'>" .$table->render() . "<span style='float: right'>".$button->render() ."</span>"  . "</form>";
}

/**
 * Edit/Add Redirect - Called when the URL is: ./edit/
 *
 */
public function ___executeEdit() {

	$this->fuel->breadcrumbs->add(new Breadcrumb('../', 'Redirects'));

	$id = (int) $this->input->get->id;

	if($id > 0) {
		// edit existing record
		$result = $this->db->query("SELECT id, redirect_from, redirect_to FROM {$this->className} WHERE id=$id");
		list($id, $from, $to) = $result->fetch_array();
		$this->setFuel('processHeadline', "Edit Redirect");

	} else {
		// add new record
		$id = 0;
		$from = '';
		$to = '';
		$this->setFuel('processHeadline', "Add Redirect");
	}

	$form = $this->modules->get("InputfieldForm");
	$form->method = 'post';
	$form->action = '../save/';

	$field = $this->modules->get("InputfieldHidden");
	$field->name = 'id';
	$field->value = $id;
	$form->add($field);

	$field = $this->modules->get("InputfieldURL");
	$field->label = 'Redirect from';
	$field->description = 'Enter relative url with slashes, like: /summer/';
	$field->name = 'redirect_from';
	$field->value = $from;
	$form->add($field);

	$field = $this->modules->get("InputfieldURL");
	$field->label = 'Redirect to';
	$field->description = 'Enter a valid URL, i.e. www.otherdomain.com/dir/ or relative url like /season/summer/';
	$field->name = 'redirect_to';
	$field->value = $to;
	$form->add($field);

	$field = $this->modules->get("InputfieldButton");
	$field->type = 'submit';
	if($id > 0 ) {
		$field->value = 'Update Redirect';
	} else {
		$field->value = 'Add New Redirect';
	}

	$form->add($field);

	return $form->render();
}

/**
 * Save Redirect - Called when the URL is ./save/
 *
 */
public function ___executeSave() {

	$from = $this->sanitizer->url($this->input->post->redirect_from);
	$to = $this->sanitizer->url($this->input->post->redirect_to);
	$id = (int) $this->input->get->id;

	// perform save of existing record if id>0, or add new if id=0
	if ($id > 0) {
		$sql = "UPDATE {$this->className} SET redirect_from = '$from', redirect_to = '$to' WHERE id = $id";
	} else {
		$sql = "INSERT INTO {$this->className} SET redirect_from = '$from', redirect_to = '$to';";
	}

   $this->db->query($sql);

   $this->message("Saved redirect from \"$from\" to \"$to\"");
   $this->session->redirect("../"); // back to list

}

public function ___executeDelete() {

	if(!is_array($this->input->post->delete) || empty($this->input->post->delete)) {
		$this->message("Nothing to delete");
		$this->session->redirect("../"); // back to list
	}

	foreach($this->input->post->delete as $id) {
		$id = (int) $id;
		$this->db->query("DELETE FROM {$this->className} WHERE id=$id");
		$count++;
	}

	$this->message("Deleted " . $count . " redirect(s)");
	$this->session->redirect("../"); // back to list
}

public function ___install() {
	parent::___install();

	$p = new Page();
	$p->template = $this->templates->get("admin");
	$p->parent = $this->pages->get("template=admin, name=setup");
	$p->title = 'Redirects';
	$p->name = 'redirects';
	$p->process = $this;
	$p->save();

	$sql = <<< _END

	CREATE TABLE {$this->className} (
		id int unsigned NOT NULL auto_increment,
		redirect_from varchar(255) NOT NULL DEFAULT '',
		redirect_to varchar(255) NOT NULL DEFAULT '',
		PRIMARY KEY(id),
		UNIQUE KEY(redirect_from)
	) ENGINE = MYISAM;

_END;

    $this->db->query($sql);

}

public function ___uninstall() {
	$p = $this->pages->get('template=admin, name=redirects');
	$p->delete();
	$this->db->query("DROP TABLE {$this->className}");
}
}

[ryan]

This actually doesn't work, row 40 at MarkupAdminDataTable class encodes this

Should I build custom table or should we extend AdminDataTable?

Sorry I forgot something. Add this right after you get the $table from $modules:

$table->setEncodeEntities(false); 

Is there clean way to add button to right side?

I recommend having ProcessRedirects live in it's own directory, like: /site/modules/ProcessRedirects/ProcessRedirects.module

Then create another file in there called ProcessRedirects.css

Use that CSS file to place the button element where you want. In your module file, you can give the button an id attribute so that it's easier to target in the css file:

$button = $this->modules->get("InputfieldButton");
$button->id = 'submit_delete'; // assign an ID attribute
$button->type = 'submit';
$button->value = 'Remove selected redirects';

Note that id attribute may actually be assigned to a containing element of the button (not positive). If it is, then you'd want to target in your CSS as "#submit_delete button" rather than just "#submit_delete".

In your executeSave function, I recommend escaping the $from and $to vars before insertion into the DB. The reason is that there are some chars allowed in URLs that would need to be escaped for the database (at last I think there are). To do that, you'd just use PHP's mysqli escape_string function:

$from = $this->db->escape_string($from);
$to = $this->db->escape_string($to); 

I'm working on adding the hook and should have that this morning!

Edit: One last thing is to add is "$count = 0;" at the top of your executeDelete function. Otherwise PHP will throw notice errors about an uninitialized variable when ProcessWire is in debug mode.

Thanks,

Ryan

[ryan]

I'm not sure how I missed this before, but turns out we already have a hook that looks like it'll work. It's ProcessPageView::pageNotFound

Here's one approach that could be used to handle the hook and redirect. This is coded in the browser/forum, so not tested and probably has parse errors, etc., but hopefully the general idea works.

<?php

public function init() {
    $this->addHook('ProcessPageView::pageNotFound', $this, 'checkRedirects'); 
}

public function checkRedirects($event) {

    // determine the URL that wasn't found 
    $url = $_SERVER['REQUEST_URI']; 

    // if installed in a subdirectory, make $url relative to the directory ProcessWire is installed in
    if($this->config->urls->root != '/') {
        // you might need to double check that below results in a path with a slash still at beginning
        $url = substr($url, strlen($this->config->urls->root)-1);
    }

    // we'll check the DB against trailing slash version and non-trailing slash version
    // and escape them for the DB select while we are at it
    $url = $this->db->escape_string(rtrim($url, '/')); 
    $url2 = $this->db->escape_string($url . '/');

    // now see if it's in the DB
    $sql = "SELECT redirect_to FROM {$this->className} WHERE redirect_from='$url' OR redirect_from='$url2'";
    $result = $this->db->query($sql); 

    // if there is a match, then redirect to it
    if($result->num_rows) {
        list($redirect_to) = $result->fetch_array(); 
        $this->session->redirect($redirect_to); 
    }
}

[apeisa]

I think we are ready here: http://processwire.com/talk/index.php/topic,171.0.html