module Jumplinks

[Mike Rockett]

@guy - I have submitted a fix to the dev branch. Please let me know if it works for you now.

[guy]

Awesome pal, that's sorted it.

[Mike Rockett]

Have just submitted an update to the dev branch containing two new features:

1. 404 Monitor: keep a log of '404 Not Found' hits (containing request URI, referrer, and user agent) so that you can later create jumplinks for them. You can clear the monitor at any time. This feature is turned off by default, and can be enabled in the module settings.
2. Per @ceberlin's request, the date and time of the last hit on a jumplink is now recorded. The date can be seen by hovering over its hit counter, or by opening the jumplink editor. Furthermore, if a jumplink hasn't been hit in the last 30 days, that jumplink will be highlighted in the table, and a message saying that it is safe to delete will appear when editing it.

Have tested on all systems available to me, and no issues have come up. That said, I'll bump to 1.3.0 tomorrow, after updating the documentation.

[Mike Rockett]

Version Bump to 1.3.0 - Important Upgrade

Release Notes:

1. Debug Mode now logs each wildcard in a jumplink.
2. Important fixes for zero/null timestamps.
3. You can now make slashes optional with /? or [/] instead of just the latter.
4. Classes are now only imported when needed.
5. Each jumplink now remembers when it was last hit and notifies you if it has gone stale, as discussed above.
6. 404 Monitor, as discussed above. Turned off by default.
7. Schema version bumped to v4 to accommodate timestamp fixes and the new 404 monitor.

If you come across any bugs with this release, please open an issue over at GitHub.

Docs will be updated shortly.

[Mike Rockett]

Version Bump to 1.3.1
 

Hotfix: Drop 404 Monitor table on uninstall

[Macrura]

@Mike - i'm wondering if there is any chance that this module may be conflicting with page path history?

I have page path history enabled, and also page path history manager; I see the URLs that PPH should be redirecting to, but they no longer work;

I haven't uninstalled Jumplinks to test this because i'm fearful of losing my existing Jumplinks (much to my horror, when i uninstalled the Redirects Module for testing, it deleted all of my redirects; fortunately this was after i already imported them to JL)...

[Mike Rockett]

@Mike - i'm wondering if there is any chance that this module may be conflicting with page path history?

I have page path history enabled, and also page path history manager; I see the URLs that PPH should be redirecting to, but they no longer work;

Going to have to give this a spin later. I don't see why there should be a conflict though. Sure, Jumplinks may have a higher priority, but if there's no jumplink for the request, it should hand over to PagePathHistory, or anything else that grabs a lower priority. Unless I'm missing something...

In the meantime, I think you should just export the 3 processjumplinks tables to sql files, and uninstall the module. Then check if it is indeed the case. You can then install Jumplinks again, and import the tables.

Edit: In a quick test, I could not reproduce this. However, I did not install the PPH Manager - not sure if there could be conflict there. In a rush, however, so will look at this again later.

Edited May 4, 2015 by Mike Rockett

[Mike Rockett]

Definitely not able to reproduce this... I'm sure something else is interfering. Let me know if the problem doesn't persist after uninstalling Jumplinks.

[Macrura]

hey thanks a ton for looking at that, i will continue testing later today and this week and see what's going on.

[Mike Rockett]

Just as a follow-up to the shortlinks tutorial posted yesterday: Jumplinks can also come in handy if you want to redirect ID-based URIs.

This is a result of the Destination Selectors feature I added a while back, thanks to the suggestion by Macrura.

The Process:

Let's say you have a Journal located in the root of your page tree, named journal. Each journal entry is assigned to the journal-entry template If you want to redirect ID-based URIs to their named equivalents, create a new jumplink like so:

Source: journal/{id}

Destination: [[template=journal-entry,id={id}]]

So, when requesting /journal/1078, for example, you'll be redirected accordingly to /journal/hello-world/.

Of course, you could take this further, but this is just a basic idea.

Might be handy for someone using Jumplinks.

[horst]

Hi Mike,

I have installed jumplinks on my new homepage and enabled the 404 logger. When I want to just edit a small piece in a page, I could see life a scanning attack for vulnarebilities:

These attacks can test several hundred links. So, after such a scan, the 404 Monitor (limited to the last 100 hits) is pretty useless.

What about (optional) alternative storage of the links? Simple logfile or maybe better a sqlite db file? This way you don't need to spam the MySQL-DB with those spam entries but the useful links also don't get dropped. So, with lots of entries, the 404-Monitor table needs to have pagination then and maybe a function to select / delete entries. If you interested in this, I can provide a ready to use wrapper class for sqlite,  (create, insert, update, read, delete).

-----------

Edit:

Also useful solution could be to exclude the 404-Monitor as separate module and give it some more functions like detecting IPs and when seeing those attacks, temporary block the IP for a short time. But this has nothing to do with jumplinks. That's why I said make 404-Monitor a external / submodule, that can be used for / from different other modules.

Edited May 24, 2015 by horst

[Mike Rockett]

Interestingly, I bumped into this issue yesterday on a live site - thanks for bringing it up.

Initially, I just implemented it as-is due to the fact that it can be useful in most cases. That said, these kinds of attacks are becoming quite regular these days, and so I think it's a very good idea.

Thank you, I'd very much appreciate a wrapper class, which, in my opinion, is much better than using a log file. When I have some time - hopefully soon - I'll upgrade the monitor.

[Mike Rockett]

Edit:

Also useful solution could be to exclude the 404-Monitor as separate module and give it some more functions like detecting IPs and when seeing those attacks, temporary block the IP for a short time. But this has nothing to do with jumplinks. That's why I said make 404-Monitor a external / submodule, that can be used for / from different other modules.

I'm just wondering if I should maybe integrate the monitor with the new 404 logger by kixe. What do you think?

Scratch that. Unfortunately, the logger doesn't capture as much information as I'd like to show. An SQLite-base solution will be better.

Edit: Perhaps a separate module, included in the package, will be best. Let me think on this.

Edited May 24, 2015 by Mike Rockett

[horst]

This is nothing that urges.

In the past (not a PW site!) I simply logged all 404 to 2 files, one with only the urls. Then, from time to time, I have manually loaded it into my editor which has a function to make all entries (lines) unique (drop all copies). From the resulting lines I created entries for the htaccess like:

Redirect gone /admin/
Redirect gone /db/
Redirect gone /dbadmin/
Redirect gone /ecommerce/
Redirect gone /sqlmanager/
Redirect gone /typo3/
Redirect gone /webstats/
Redirect gone /wordpress/
Redirect gone /connector.php
Redirect gone /index2.php
Redirect gone /wp-login.php

This way my 404 log becomes more and more clean and the attackers hopefully gets a feeling like "Damn, to late again!"

A 404 monitor module that determine a scan attack optionally can send 410 headers for the next 5 or 10 minutes to the originating IP. No more editing the htaccess for this.

I will send you the sqlite wrapper. I have looked at it, but currently it depends on a subclass of mine in regards to filelocking. I will change this to result in a single class before sending.

Edited May 24, 2015 by horst

[Mike Rockett]

After some thought:

I'm a little undecided as to how much functionality I would want in this monitor. Currently, Jumplinks serves its purpose, and adding to much to it might not be the greatest idea.

That said, I would be prepared to release a separate 404 monitor module that includes this functionality. Firstly, Jumplinks would be able to tie into this module if it exists. Secondly, I'd be able to add certain security features to it without bloating Jumplinks.

Preferably, I'd want to implement this module in such a way that it modifies the .htaccess file in the following way: when browsing the log, a user with the relevant permissions would be able to blacklist a certain URI for a fixed/indefinite period of time. They'd also be able to block an IP (based on the findings in the log) for a certain/indefinite amount of time. When these 'blocks' are defined/modified, they'd be added to the .htaccess file, and the module would periodically check (lazy-cron-like, say twice a day) to see if the file is up to date. An example would look like this:

#< BEGIN ModuleName <last-check: <time>>

Redirect 410 /wp-admin # always
Redirect 410 /etc # <time>-<time>
Redirect 410 /etc.php <time>-<time>

Require all granted
Require not ip xxx.xxx.xxx.xxx # <time>-<time>
Require not ip xxx.xxx.xxx.xxy # always

#> END ModuleName

For me, this is a performance thing - i.e. booting up the whole PW system just to block/limit access is unnecessary.

With the above in mind, this may land up becoming an .htaccess security-driven module, similar to those made for WP (the bain of my existence); however, it would only include essentials. (It would more than likely not be called '404Monitor'.)

As said, the added benefit would be that the module can provide a 'safe' list of 404s from which Jumplinks can derive it's data.

Thoughts?

[Frank Vèssia]

I had a website with a lot of subdomains, now I want to redirect all subdomains to internal pages, is there a way to do this with jumplinks?

I would like also to suggest a couple of small "improvements" as result of my experience with this fantastic module.

1. possibility to filter 404 errors per type, like images, pages, css, etc.. I have a lot of 404 errors and having such option could be very helpful.

2. check for already created jumplink. When moving a lot of pages sometimes I created duplicated links without noticing...

[Mike Rockett]

I had a website with a lot of subdomains, now I want to redirect all subdomains to internal pages, is there a way to do this with jumplinks?

Unfortunately not - it isn't something that happens often, and so it isn't a feature. It might be best for you to use .htaccess for that.

1. possibility to filter 404 errors per type, like images, pages, css, etc.. I have a lot of 404 errors and having such option could be very helpful.

2. check for already created jumplink. When moving a lot of pages sometimes I created duplicated links without noticing...

1. I will be rebuilding the 404 Monitor as a separate module, and may include that.
2. That's also on the to do list. Originally, I skipped the validation as I found it was not a common occurrence. But, I do believe that some kind of validation should be put in place.

I do have quite a lot of work at the moment, and so cannot commit to actually getting these things done.

Thanks for your suggestions!

[Mike Rockett]

Just a heads-up: As mentioned, lot's on the plate these days. But I've made a decision regarding the module going forward. When the next one or two PW minor stables come out, I'm going to start making Jumplinks 2.6-only. Lots of goodies I want to use, and as stated earlier in this thread, Jumplinks is mostly for new sites resulting from migrations. Of course, Jumplinks won't upgrade for anyone using it on 2.5.3. Also, and because of this decision, I'll be switching over to some kind of autoload solution so I can split up logic etc. It's essentially going to be a rewrite. Nonetheless, it will function exactly the same, sans the 404 Monitor, which I'll try to release at the same time.

Speaking of the 404 monitor, I've already drawn out some blueprints and flow charts - ideas have come to mind, and I think it's going to be a really helpful module. As said, it will mesh well with Jumplinks.

Edit: Okay, I see that 2.6.1 is already on stable (of course, it should be). I'll need some time before I start this up.

Edited May 30, 2015 by Mike Rockett

[ceberlin]

Hi Mike, the 404 log starts showing lots of request from Google starting like so:

index.php.pwpj/…

I think the "pwpj" extension is from this module. I have no clue how this fake extension(?) made its way into Google, what the reason is for it, and how to handle it when creating a jumplink.

Can you update the documentation to give a litte insight?

And by the way, a feature request: Is it possible to set a "410 gone" also?

I have a bunch of links to non-existing images that I do not plan to redirect.

[Mike Rockett]

Hi Mike, the 404 log starts showing lots of request from Google starting like so:

index.php.pwpj/…

I think the "pwpj" extension is from this module. I have no clue how this fake extension(?) made its way into Google, what the reason is for it, and how to handle it when creating a jumplink.

Can you update the documentation to give a litte insight?

It's a method I'm currently using to allow users to create jumplinks that start with index.php. Not sure why Google is indexing those, especially considering they are 404s. If you have a jumplink that checks for index.php?cat=32, for example, but you make a request for anything but the 32 and leave the index.php in the URI, Jumplinks will redirect index.php to index.php.pwpj so that it can be tested.

I will more than likely be using a better method when I rewrite the module.

And by the way, a feature request: Is it possible to set a "410 gone" also?

I have a bunch of links to non-existing images that I do not plan to redirect.

I could indeed do that, but I think it would be better for that to be handled with .htaccess, like so:

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} \.(jpg|png|gif)$
RewriteRule ^ - [G,L]

1. Check the request doesn't match an existing file
2. Check that an image is being requested
3. Send 410 Gone if both conditions match

[Ivan Gretsky]

Good day, Mike (and all the other redirect maniacs)!

I  got a site that responds differently to different subdomains and got Jumplinks installed. I am writing a .htaccess redirect from www of a subdomain to no-www.

RewriteCond %{HTTP_HOST} ^www\.subdomain.domain\.ru$ [NC]
RewriteRule ^(.*)$ http://subdomain.domain.ru/$1 [R=301,L]

If I run www.subdomain.domain.ru it gets redirected to subdomain.domain.ru. But if I go to something like www.subdomain.domain.ru/something I get an error:

You don't have permission to access /index.php.pwpj on this server.

I turned the module off and realised that it is not the module that is the reason, as I still get redirected to subdomain.domain.ru/index.php?it=something/ not to subdomain.domain.ru/something.

I tried putting my code before and after PROCESSWIRE HTACCESS DIRECTIVES block without a difference. Could you please give me a hint here?

[Mike Rockett]

@Ivan - Hmm, seems strange. Could be a 301 cache issue? Generally better to work with 302 until you're sure it's working correctly.

If that's not the case, I'll clone your setup here and see what happens.

[OllieMackJames]

Mike, thanks for the module, I just migrated a modx site to pw and am using this full on - running into trouble though.

When I input a jumplink it does a 302 and that is NOT good for google, I need them to be 301.

So the question is how can I make sure all the redirects are 301 and there are no 302

302 means google gets confused and that means my traffic hurts.

Thanks for the help!

Baie dankie

[mr-fan]

have you set a start or end time for the jumplink?

[OllieMackJames]

have you set a start or end time for the jumplink?

jumplinks.jpg

NO start and NO end date set anywhere, tnx!