404 search module

[cmscritic]

I have an idea for a module that I think might be useful. When I was on WP, we had one we could install that would auto search for whatever the user was trying to find whenever it encountered a 404 error page and present a list of matching options. As I have a few 404's showing up on my MOZ.com report, I wondered if this would be something other members would find useful as well. I thought I'd throw it out there if someone thinks it might be useful. I'd be happy to donate a few bucks to the development of it if there's someone willing to take it on.

[SiNNuT]

Do you mean some kind of "did you mean" list of suggestions a user sees when they get the 404 page (in the frontend), based on the url they tried to access?

[adrian]

Maybe just replace slashes in the url with + and send those terms to the search page? Sounds pretty simple to achieve. Not tested ...

$this->addHook('ProcessPageView::pageNotFound', $this, 'search');

public function search($event) {

    $request = parse_url($_SERVER['REQUEST_URI']);
    $path = $request["path"];
    $result = trim(str_replace(basename($_SERVER['SCRIPT_NAME']), '', $path), '/');
    $result = explode('/', $result);
    $term = implode('+', $result);
    $session->redirect("/search/?q=$term");
}

Might be more things to consider. Maybe no real need to explode/implode. Maybe just a str_replace('/','+', $result), but you get the idea.

[adrian]

Ok, how's this work for you?

http://modules.processwire.com/modules/process404-search/

Edited August 12, 2014 by adrian
Old attached version of module removed to avoid confusion

[kongondo]

Gotta love this community...just like that...here's a module! Thanks Adrian!

[Soma]

So there will never be a 404 for those links? Seo?

[kongondo]

@Adrian.

Thanks, works a treat! Just tested on the blog profile...and got a nice 404! This is  because the URL in session->redirect is "almost hard corded". In the blog profile search is under /tools/search  . Assuming people would not always store their search page directly under root, maybe, in the instructions, ask user to change path in the module (tricky for some + would be overwritten with updates), or make module configurable. Maybe latter is better since some people might not even call the search page search?....Thanks again!  

[adrian]

I knew the issue of wanting to configure the path to the search page was going to come up

This was supposed to be a quick response while procrastinating from real work!

Give me a few minutes!

[cmscritic]

Do you mean some kind of "did you mean" list of suggestions a user sees when they get the 404 page (in the frontend), based on the url they tried to access?

Yes, that was my idea.

So there will never be a 404 for those links? Seo?

It's more about attempting to retain your visitors by giving them something else to read. It should still report as a 404 but give a list of options such as "Were you looking for?" and a list of posts.

[kongondo]

And now it's stopped working? Don't know what I did  ....I'm getting server 404s

[cmscritic]

Ok, how's this work for you?

Holy smokes, I wasn't expecitng a module that fast!

Gotta love this community...just like that...here's a module! Thanks Adrian!

I know, this is one of the reasons I recommend PW so often.

It appears Adrian is on a mission. Let me know when you'd like me to test and I'm happy to do so.

[adrian]

Good point Soma and now I better understand what you are thinking @cmscritic. Agreed - this shouldn't redirect to the search page. It should load the 404 page, but populate it with content from the search results. Will have to rethink for sure!

[adrian]

Ok, here is something I'm not exactly proud of, but I really don't have anymore time right now, so I thought I'd post anyway in case it inspires someone else to come up with something better. This is the key line that shows you how things are working:

echo $this->pages->get($this->searchPage)->render(array('q' => $term));

This way, there is no redirect, so the http headers still return a 404 status code, so I think this solves the SEO issue.

There is a new config option that lets you choose your search page, so that should take care of the issue kongondo brought up.

With this new version,you need to add this line to the top of your search.php file

if(isset($options['q'])) $input->get->q = $options['q'];

Or this if you want the title on the search page replaced:

if(isset($options['q'])){
    $input->get->q = $options['q'];
    $page->title = $pages->get(27)->title; //27 is the ID of the 404 page that comes with PW
}

Anyone have any brighter ideas?

PS Due to the use of render($options), you need PW 2.4 or a recent 2.3 dev version.

Edited August 12, 2014 by adrian
Old attached version of module removed to avoid confusion

[adrian]

Just had a thought for a slightly better version.

I replaced this:

echo $this->pages->get($this->searchPage)->render(array('q' => $term));

with:

echo $this->page->render($this->pages->get($this->searchPage)->template->filename, array('q' => $term)); 

which seems a little cleaner.

This change also means that now you only have to add just the one line to your search.php file:

if(isset($options['q'])) $input->get->q = $options['q'];

No need to change the page title anymore because we are actually now still rendering the 404 page with it's existing fields, just using the template file of the search page.

However, I also added the 404 page's body to my search.php file:

if(isset($options['q'])){
    $input->get->q = $options['q'];
    $out .= $pages->get(27)->body;
}

With this approach you can easily then edit the body field of the "404 Page Not Found" page in the PW admin to read something like this:

<h3>The page you were looking for is not found.</h3>
<p>We have tried to find the content you were looking for, but if we didn't get it right, please use our search engine or navigation above to find the page.</p>

Any thoughts on this new version, or the approach in general?

Any problems from anyone who has tested it?

[adrian]

Alternatively you could use this line in the module to send the 404 Page Not Found body to the search template

echo $this->page->render($this->pages->get($this->searchPage)->template->filename, array('q' => $term, 'out' => $this->page->body));

 and have it output in the search template using:

if(isset($options['q'])){    
    $input->get->q = $options['q'];
    $out .= $options['out'];
} 

Lots of options really

[cmscritic]

Wicked Adrian! I'll test it out and get back to you tomorrow. Thanks so much for the help with this, love this community.

[adrian]

Any thoughts on whether this should be released as a module in the directory? I am thinking that I will start using it on all sites now. I can't see any negatives with the way it is working now. Obviously I'd prefer a one click install, but can't see a way around having to add that one snippet to the search.php file.

Anyone have any strong thoughts one way or the other?

[cstevensjr]

I would like for you to release it and thanks for the great work

[adrian]

Ok, here it is:

Modules Directory: http://modules.processwire.com/modules/process404-search/

Github: https://github.com/adrianbj/Process404Search

Includes a small tweak to also convert dashes and underscores to "+" which should improve search results.

[OllieMackJames]

@cmscritic, great idea!

@adrian, man you really rock, thanks lots for making this, going to install it on all my PW sites, already installed on one and works flawless! THANKS again.

[OrganizedFellow]

Not working as expected.

I navigate to a page that does not exist, or enter a search term that does not exist. The resulting page is nothing different than before.

I added this to the top of my search.php page:

if(isset($options['q'])) $input->get->q = $options['q'];

Maybe I was expecting something different?

[adrian]

It should work just fine. Did you remember to select your search page in the module config settings?

 

Is your search.php file the standard one that came with PW?

 

Are you using the version posted to the modules directory (http://modules.processwire.com/modules/process404-search/) / Github (https://github.com/adrianbj/Process404Search) or one of the older ones posted as attachments to posts in this thread?

 

Also, what version of PW are you using? It needs to support the passing of arguments via render(), so you need 2.4

[OrganizedFellow]

@Adrian

I have properly selected the search.php page in the configuration setting page.

The search.php page is the default that came with ProcessWire 2.4.0.

<?php

if(isset($options['q'])) $input->get->q = $options['q'];

$subheader = "Please enter a search";

if($q = $sanitizer->selectorValue($input->get->q)) {

	// Send our sanitized query 'q' variable to the whitelist where it will be
	// picked up and echoed in the search box by (in _main.php)
	$input->whitelist('q', $q); 

	// limit=the max number of items to show per pagination
	// to use this, enable pagination on your search template (in template settings)
	$limit = 10; 

	// first try a fulltext match of keywords anywhere in title/body
	$matches = $pages->find("title|body~=$q, limit=$limit"); 

	// if the above doesn't find anything (perhaps due to MySQL minimum word 
	// length or stopwords), switch to non-indexed phrase match
	if(!count($matches)) $matches = $pages->find("title|body%=$q, limit=$limit"); 

	$total = $matches->getTotal();
	$start = $matches->getStart()+1;
	$end = $matches->getStart() + count($matches); 
	
	if($total) {

		foreach($matches as $item) {
			// if item has no summary, substitute something else
			if(!$item->summary) $item->summary = "Found in: $item->path"; 
		}

		$subheader = "Matches $start to $end of $total:"; 
		$body .= renderBodyNav($matches); 

	} else {
		// $subheader = "Sorry, no results were found.";
		$subheader = "The page you were looking for is not found.";
	}
}

$body = "<h2 class='subheader'>$subheader</h2>$body<p>We have tried to find the content you were looking for, but if we didn't get it right, please use our search engine or navigation above to find the page.</p>";

Am using module version 0.1.1 via ModulesManager.

[adrian]

Looks like the problem is that you are overriding $body in the last line of code completely instead of concatenating - change it to a ".=" and it should work.

Although I don't actually think you want that there because it would be after the results anyways. If you look at the module Readme:

https://github.com/adrianbj/Process404Search

Note that I recommend to put text like that in the body field of your 404 page and then use this instead at the top of your search.php file:

if(isset($options['q'])){
   $input->get->q = $options['q'];
   $out .= $pages->get(27)->body;
}

Note that in your case you want to replace $out with $body

Edited August 11, 2014 by adrian

[Pete]

Adrian - you may want to remove all the downloads of versions from the previous page and instead direct users to the modules page.

I was about to download the first one without reading any further