Recommended Posts

Awesome pal, that's sorted it.

  • Like 1

Share this post


Link to post
Share on other sites

Have just submitted an update to the dev branch containing two new features:

  1. 404 Monitor: keep a log of '404 Not Found' hits (containing request URI, referrer, and user agent) so that you can later create jumplinks for them. You can clear the monitor at any time. This feature is turned off by default, and can be enabled in the module settings.
  2. Per @ceberlin's request, the date and time of the last hit on a jumplink is now recorded. The date can be seen by hovering over its hit counter, or by opening the jumplink editor. Furthermore, if a jumplink hasn't been hit in the last 30 days, that jumplink will be highlighted in the table, and a message saying that it is safe to delete will appear when editing it.

Have tested on all systems available to me, and no issues have come up. That said, I'll bump to 1.3.0 tomorrow, after updating the documentation.

  • Like 3

Share this post


Link to post
Share on other sites

Version Bump to 1.3.0 - Important Upgrade

Release Notes:

  1. Debug Mode now logs each wildcard in a jumplink.
  2. Important fixes for zero/null timestamps.
  3. You can now make slashes optional with /? or [/] instead of just the latter.
  4. Classes are now only imported when needed.
  5. Each jumplink now remembers when it was last hit and notifies you if it has gone stale, as discussed above.
  6. 404 Monitor, as discussed above. Turned off by default.
  7. Schema version bumped to v4 to accommodate timestamp fixes and the new 404 monitor.

If you come across any bugs with this release, please open an issue over at GitHub.

Docs will be updated shortly.

  • Like 3

Share this post


Link to post
Share on other sites

@Mike - i'm wondering if there is any chance that this module may be conflicting with page path history?

I have page path history enabled, and also page path history manager; I see the URLs that PPH should be redirecting to, but they no longer work;

I haven't uninstalled Jumplinks to test this because i'm fearful of losing my existing Jumplinks (much to my horror, when i uninstalled the Redirects Module for testing, it deleted all of my redirects; fortunately this was after i already imported them to JL)...

Share this post


Link to post
Share on other sites

@Mike - i'm wondering if there is any chance that this module may be conflicting with page path history?

I have page path history enabled, and also page path history manager; I see the URLs that PPH should be redirecting to, but they no longer work;

Going to have to give this a spin later. I don't see why there should be a conflict though. Sure, Jumplinks may have a higher priority, but if there's no jumplink for the request, it should hand over to PagePathHistory, or anything else that grabs a lower priority. Unless I'm missing something...

In the meantime, I think you should just export the 3 processjumplinks tables to sql files, and uninstall the module. Then check if it is indeed the case. You can then install Jumplinks again, and import the tables.

Edit: In a quick test, I could not reproduce this. However, I did not install the PPH Manager - not sure if there could be conflict there. In a rush, however, so will look at this again later.

Edited by Mike Rockett

Share this post


Link to post
Share on other sites

Definitely not able to reproduce this... I'm sure something else is interfering. Let me know if the problem doesn't persist after uninstalling Jumplinks.

Share this post


Link to post
Share on other sites

hey thanks a ton for looking at that, i will continue testing later today and this week and see what's going on.

  • Like 1

Share this post


Link to post
Share on other sites

Just as a follow-up to the shortlinks tutorial posted yesterday: Jumplinks can also come in handy if you want to redirect ID-based URIs.

This is a result of the Destination Selectors feature I added a while back, thanks to the suggestion by Macrura.

The Process:

Let's say you have a Journal located in the root of your page tree, named journal. Each journal entry is assigned to the journal-entry template If you want to redirect ID-based URIs to their named equivalents, create a new jumplink like so:

Source: journal/{id}

Destination: [[template=journal-entry,id={id}]]

So, when requesting /journal/1078, for example, you'll be redirected accordingly to /journal/hello-world/.

Of course, you could take this further, but this is just a basic idea.

Might be handy for someone using Jumplinks. :-)

  • Like 2

Share this post


Link to post
Share on other sites

Hi Mike,

I have installed jumplinks on my new homepage and enabled the 404 logger. When I want to just edit a small piece in a page, I could see life a scanning attack for vulnarebilities:

post-1041-0-28321700-1432454395_thumb.jp

These attacks can test several hundred links. So, after such a scan, the 404 Monitor (limited to the last 100 hits) is pretty useless.

What about (optional) alternative storage of the links? Simple logfile or maybe better a sqlite db file? This way you don't need to spam the MySQL-DB with those spam entries but the useful links also don't get dropped. So, with lots of entries, the 404-Monitor table needs to have pagination then and maybe a function to select / delete entries. If you interested in this, I can provide a ready to use wrapper class for sqlite,  (create, insert, update, read, delete).

-----------

Edit:

Also useful solution could be to exclude the 404-Monitor as separate module and give it some more functions like detecting IPs and when seeing those attacks, temporary block the IP for a short time. But this has nothing to do with jumplinks. That's why I said make 404-Monitor a external / submodule, that can be used for / from different other modules.

Edited by horst

Share this post


Link to post
Share on other sites

Interestingly, I bumped into this issue yesterday on a live site - thanks for bringing it up. :)

Initially, I just implemented it as-is due to the fact that it can be useful in most cases. That said, these kinds of attacks are becoming quite regular these days, and so I think it's a very good idea.

Thank you, I'd very much appreciate a wrapper class, which, in my opinion, is much better than using a log file. :) When I have some time - hopefully soon - I'll upgrade the monitor.

Share this post


Link to post
Share on other sites

Edit:

Also useful solution could be to exclude the 404-Monitor as separate module and give it some more functions like detecting IPs and when seeing those attacks, temporary block the IP for a short time. But this has nothing to do with jumplinks. That's why I said make 404-Monitor a external / submodule, that can be used for / from different other modules.

I'm just wondering if I should maybe integrate the monitor with the new 404 logger by kixe. What do you think?

Scratch that. Unfortunately, the logger doesn't capture as much information as I'd like to show. An SQLite-base solution will be better.

Edit: Perhaps a separate module, included in the package, will be best. Let me think on this.

Edited by Mike Rockett
  • Like 1

Share this post


Link to post
Share on other sites

This is nothing that urges. :)

In the past (not a PW site!) I simply logged all 404 to 2 files, one with only the urls. Then, from time to time, I have manually loaded it into my editor which has a function to make all entries (lines) unique (drop all copies). From the resulting lines I created entries for the htaccess like:

Redirect gone /admin/
Redirect gone /db/
Redirect gone /dbadmin/
Redirect gone /ecommerce/
Redirect gone /sqlmanager/
Redirect gone /typo3/
Redirect gone /webstats/
Redirect gone /wordpress/
Redirect gone /connector.php
Redirect gone /index2.php
Redirect gone /wp-login.php

This way my 404 log becomes more and more clean and the attackers hopefully gets a feeling like "Damn, to late again!" :)

A 404 monitor module that determine a scan attack optionally can send 410 headers for the next 5 or 10 minutes to the originating IP. No more editing the htaccess for this. :)

I will send you the sqlite wrapper. I have looked at it, but currently it depends on a subclass of mine in regards to filelocking. I will change this to result in a single class before sending.

Edited by horst
  • Like 1

Share this post


Link to post
Share on other sites

After some thought:

I'm a little undecided as to how much functionality I would want in this monitor. Currently, Jumplinks serves its purpose, and adding to much to it might not be the greatest idea.

That said, I would be prepared to release a separate 404 monitor module that includes this functionality. Firstly, Jumplinks would be able to tie into this module if it exists. Secondly, I'd be able to add certain security features to it without bloating Jumplinks.

Preferably, I'd want to implement this module in such a way that it modifies the .htaccess file in the following way: when browsing the log, a user with the relevant permissions would be able to blacklist a certain URI for a fixed/indefinite period of time. They'd also be able to block an IP (based on the findings in the log) for a certain/indefinite amount of time. When these 'blocks' are defined/modified, they'd be added to the .htaccess file, and the module would periodically check (lazy-cron-like, say twice a day) to see if the file is up to date. An example would look like this:

#< BEGIN ModuleName <last-check: <time>>

Redirect 410 /wp-admin # always
Redirect 410 /etc # <time>-<time>
Redirect 410 /etc.php <time>-<time>

Require all granted
Require not ip xxx.xxx.xxx.xxx # <time>-<time>
Require not ip xxx.xxx.xxx.xxy # always

#> END ModuleName

For me, this is a performance thing - i.e. booting up the whole PW system just to block/limit access is unnecessary.

With the above in mind, this may land up becoming an .htaccess security-driven module, similar to those made for WP (the bain of my existence); however, it would only include essentials. (It would more than likely not be called '404Monitor'.)

As said, the added benefit would be that the module can provide a 'safe' list of 404s from which Jumplinks can derive it's data.

Thoughts?

  • Like 2

Share this post


Link to post
Share on other sites

I had a website with a lot of subdomains, now I want to redirect all subdomains to internal pages, is there a way to do this with jumplinks?

I would like also to suggest a couple of small "improvements" as result of my experience with this fantastic module.

1. possibility to filter 404 errors per type, like images, pages, css, etc.. I have a lot of 404 errors and having such option could be very helpful.

2. check for already created jumplink. When moving a lot of pages sometimes I created duplicated links without noticing...

Share this post


Link to post
Share on other sites

I had a website with a lot of subdomains, now I want to redirect all subdomains to internal pages, is there a way to do this with jumplinks?

Unfortunately not - it isn't something that happens often, and so it isn't a feature. It might be best for you to use .htaccess for that.

1. possibility to filter 404 errors per type, like images, pages, css, etc.. I have a lot of 404 errors and having such option could be very helpful.

2. check for already created jumplink. When moving a lot of pages sometimes I created duplicated links without noticing...

  1. I will be rebuilding the 404 Monitor as a separate module, and may include that.
  2. That's also on the to do list. Originally, I skipped the validation as I found it was not a common occurrence. But, I do believe that some kind of validation should be put in place.

I do have quite a lot of work at the moment, and so cannot commit to actually getting these things done.

Thanks for your suggestions!

Share this post


Link to post
Share on other sites

Just a heads-up: As mentioned, lot's on the plate these days. But I've made a decision regarding the module going forward. When the next one or two PW minor stables come out, I'm going to start making Jumplinks 2.6-only. Lots of goodies I want to use, and as stated earlier in this thread, Jumplinks is mostly for new sites resulting from migrations. Of course, Jumplinks won't upgrade for anyone using it on 2.5.3. Also, and because of this decision, I'll be switching over to some kind of autoload solution so I can split up logic etc. It's essentially going to be a rewrite. Nonetheless, it will function exactly the same, sans the 404 Monitor, which I'll try to release at the same time.

Speaking of the 404 monitor, I've already drawn out some blueprints and flow charts - ideas have come to mind, and I think it's going to be a really helpful module. As said, it will mesh well with Jumplinks.

:-)

Edit: Okay, I see that 2.6.1 is already on stable (of course, it should be). I'll need some time before I start this up.

Edited by Mike Rockett
  • Like 5

Share this post


Link to post
Share on other sites

Hi Mike, the 404 log starts showing lots of request from Google starting like so:

index.php.pwpj/…

I think the "pwpj" extension is from this module. I have no clue how this fake extension(?) made its way into Google, what the reason is for it, and how to handle it when creating a jumplink.

Can you update the documentation to give a litte insight?

And by the way, a feature request: Is it possible to set a "410 gone" also?

I have a bunch of links to non-existing images that I do not plan to redirect.

Share this post


Link to post
Share on other sites

Hi Mike, the 404 log starts showing lots of request from Google starting like so:

index.php.pwpj/…

I think the "pwpj" extension is from this module. I have no clue how this fake extension(?) made its way into Google, what the reason is for it, and how to handle it when creating a jumplink.

Can you update the documentation to give a litte insight?

It's a method I'm currently using to allow users to create jumplinks that start with index.php. Not sure why Google is indexing those, especially considering they are 404s. If you have a jumplink that checks for index.php?cat=32, for example, but you make a request for anything but the 32 and leave the index.php in the URI, Jumplinks will redirect index.php to index.php.pwpj so that it can be tested.

I will more than likely be using a better method when I rewrite the module.

And by the way, a feature request: Is it possible to set a "410 gone" also?

I have a bunch of links to non-existing images that I do not plan to redirect.

I could indeed do that, but I think it would be better for that to be handled with .htaccess, like so:

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} \.(jpg|png|gif)$
RewriteRule ^ - [G,L]
  1. Check the request doesn't match an existing file
  2. Check that an image is being requested
  3. Send 410 Gone if both conditions match
  • Like 1

Share this post


Link to post
Share on other sites

Good day, Mike (and all the other redirect maniacs)!

I  got a site that responds differently to different subdomains and got Jumplinks installed. I am writing a .htaccess redirect from www of a subdomain to no-www.

RewriteCond %{HTTP_HOST} ^www\.subdomain.domain\.ru$ [NC]
RewriteRule ^(.*)$ http://subdomain.domain.ru/$1 [R=301,L]

If I run www.subdomain.domain.ru it gets redirected to subdomain.domain.ru. But if I go to something like www.subdomain.domain.ru/something I get an error:

You don't have permission to access /index.php.pwpj on this server.

I turned the module off and realised that it is not the module that is the reason, as I still get redirected to subdomain.domain.ru/index.php?it=something/ not to subdomain.domain.ru/something.

I tried putting my code before and after PROCESSWIRE HTACCESS DIRECTIVES block without a difference. Could you please give me a hint here?

Share this post


Link to post
Share on other sites

@Ivan - Hmm, seems strange. Could be a 301 cache issue? Generally better to work with 302 until you're sure it's working correctly.

If that's not the case, I'll clone your setup here and see what happens.

Share this post


Link to post
Share on other sites

Mike, thanks for the module, I just migrated a modx site to pw and am using this full on - running into trouble though.

When I input a jumplink it does a 302 and that is NOT good for google, I need them to be 301.

So the question is how can I make sure all the redirects are 301 and there are no 302

302 means google gets confused and that means my traffic hurts.

Thanks for the help!

Baie dankie

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Rudy
      Hi,
      I am writing a custom module that requires storing array of settings in the module config/settings.
      Is there a built-in fieldtype that allows me to store settings in an array (sort of like Repeater)?
      I tried using InputfieldAceExtended and InputfieldTextarea to store JSON array. Both times, my data was stored in the module settings (in the database) but upon reload, that information was not retrieved.
      Thanks
      Rudy
       
    • By Anssi
      A simple module to enable easy navigation between the public and the admin side of the site. After installation a green bar will appear to the upper side of the screen, containing a few navigation elements and displaying the PW version number.
      Heavily inspired by @apeisa's great AdminBar (Thanks!). I needed a bit simpler tool for my projects and as a result, this was made. Available on GitHub .

    • By Sebi
      I've created a small module which lets you define a timestamp after which a page should be accessible. In addition you can define a timestamp when the release should end and the page should not be accessable any more.
      Github: https://github.com/Sebiworld/PageAccessReleasetime
      Usage
      PageAccessReleasetime can be installed like every other module in ProcessWire. Check the following guide for detailed information: How-To Install or Uninstall Modules
      After that, you will find checkboxes for activating the releasetime-fields at the settings-tab of each page. You don't need to add the fields to your templates manually.
      Check e.g. the checkbox "Activate Releasetime from?" and fill in a date in the future. The page will not be accessable for your users until the given date is reached.
      If you have $config->pagefileSecure = true, the module will protect files of unreleased pages as well.
      How it works
      This module hooks into Page::viewable to prevent users to access unreleased pages:
      public function hookPageViewable($event) { $page = $event->object; $viewable = $event->return; if($viewable){ // If the page would be viewable, additionally check Releasetime and User-Permission $viewable = $this->canUserSee($page); } $event->return = $viewable; } To prevent access to the files of unreleased pages, we hook into Page::isPublic and ProcessPageView::sendFile.
      public function hookPageIsPublic($e) { $page = $e->object; if($e->return && $this->isReleaseTimeSet($page)) { $e->return = false; } } The site/assets/files/ directory of pages, which isPublic() returns false, will get a '-' as prefix. This indicates ProcessWire (with activated $config->pagefileSecure) to check the file's permissions via PHP before delivering it to the client.
      The check wether a not-public file should be accessable happens in ProcessPageView::sendFile. We throw an 404 Exception if the current user must not see the file.
      public function hookProcessPageViewSendFile($e) { $page = $e->arguments[0]; if(!$this->canUserSee($page)) { throw new Wire404Exception('File not found'); } } Additionally we hook into ProcessPageEdit::buildForm to add the PageAccessReleasetime fields to each page and move them to the settings tab.
      Limitations
      In the current version, releasetime-protected pages will appear in wire('pages')->find() queries. If you want to display a list of pages, where pages could be releasetime-protected, you should double-check with $page->viewable() wether the page can be accessed. $page->viewable() returns false, if the page is not released yet.
      If you have an idea how unreleased pages can be filtered out of ProcessWire selector queries, feel free to write an issue, comment or make a pull request!
    • By David Karich
      Thanks to the great Pro module "RepeaterMatrix" I have the possibility to create complex repeater items. With it I have created a quite powerful page builder. Many different content modules, with many more possible design options. The RepeaterMatrix module supports the cloning of items, but only within the same page. Now I often have the case that very design-intensive pages and items are created. If you want to use this module on a different page (e.g. in the same design), you have to rebuild each item manually every time.
      With this proof of concept I have created a module which adds the feature to copy a repeater item to the clipboard so that you can paste this item to another page with the same repeater field. The module has been developed very rudimentarily so far. It is currently not possible to copy nested items. There is also no check of Min/Max. You can also only copy items that have the same field on different pages. And surely you can solve all this more elegantly with AJAX. But personally I lack the deeper understanding of the repeaters. Also missing on the Javascript side are event triggers for the repeaters, which would make it easier. Like e.g. RepeaterItemInitReady or similar.
      it would be great if @ryan would implement this functionality in the core of RepeaterMatrix. I think he has better ways to implement this. Or what do you think, Ryan?
      Everybody is welcome to work on this module and improve it, if it should not be integrated into the matrix core. Therefore I put it for testing and as download on GitHub: https://github.com/FlipZoomMedia/InputfieldRepeaterMatrixDublicate
      You can best see the functionality in the screencast: 
       
    • By anderson
      Hi,
      Please take a look at this:
      https://templatemag.com/demo/Good/
      The upper nav bar, including dropdowns like "pages" and "portfolios", what do you call this whole thing? At first I guess it's called "dropdown nav bar", but seems not.
      AND of course, what's the simplest way/module to achieve this in PW?
      Thanks in advance.