Jump to content

Recommended Posts

Have just submitted an update to the dev branch containing two new features:

  1. 404 Monitor: keep a log of '404 Not Found' hits (containing request URI, referrer, and user agent) so that you can later create jumplinks for them. You can clear the monitor at any time. This feature is turned off by default, and can be enabled in the module settings.
  2. Per @ceberlin's request, the date and time of the last hit on a jumplink is now recorded. The date can be seen by hovering over its hit counter, or by opening the jumplink editor. Furthermore, if a jumplink hasn't been hit in the last 30 days, that jumplink will be highlighted in the table, and a message saying that it is safe to delete will appear when editing it.

Have tested on all systems available to me, and no issues have come up. That said, I'll bump to 1.3.0 tomorrow, after updating the documentation.

  • Like 3

Share this post


Link to post
Share on other sites

Version Bump to 1.3.0 - Important Upgrade

Release Notes:

  1. Debug Mode now logs each wildcard in a jumplink.
  2. Important fixes for zero/null timestamps.
  3. You can now make slashes optional with /? or [/] instead of just the latter.
  4. Classes are now only imported when needed.
  5. Each jumplink now remembers when it was last hit and notifies you if it has gone stale, as discussed above.
  6. 404 Monitor, as discussed above. Turned off by default.
  7. Schema version bumped to v4 to accommodate timestamp fixes and the new 404 monitor.

If you come across any bugs with this release, please open an issue over at GitHub.

Docs will be updated shortly.

  • Like 3

Share this post


Link to post
Share on other sites

@Mike - i'm wondering if there is any chance that this module may be conflicting with page path history?

I have page path history enabled, and also page path history manager; I see the URLs that PPH should be redirecting to, but they no longer work;

I haven't uninstalled Jumplinks to test this because i'm fearful of losing my existing Jumplinks (much to my horror, when i uninstalled the Redirects Module for testing, it deleted all of my redirects; fortunately this was after i already imported them to JL)...

Share this post


Link to post
Share on other sites

@Mike - i'm wondering if there is any chance that this module may be conflicting with page path history?

I have page path history enabled, and also page path history manager; I see the URLs that PPH should be redirecting to, but they no longer work;

Going to have to give this a spin later. I don't see why there should be a conflict though. Sure, Jumplinks may have a higher priority, but if there's no jumplink for the request, it should hand over to PagePathHistory, or anything else that grabs a lower priority. Unless I'm missing something...

In the meantime, I think you should just export the 3 processjumplinks tables to sql files, and uninstall the module. Then check if it is indeed the case. You can then install Jumplinks again, and import the tables.

Edit: In a quick test, I could not reproduce this. However, I did not install the PPH Manager - not sure if there could be conflict there. In a rush, however, so will look at this again later.

Edited by Mike Rockett

Share this post


Link to post
Share on other sites

Definitely not able to reproduce this... I'm sure something else is interfering. Let me know if the problem doesn't persist after uninstalling Jumplinks.

Share this post


Link to post
Share on other sites

hey thanks a ton for looking at that, i will continue testing later today and this week and see what's going on.

  • Like 1

Share this post


Link to post
Share on other sites

Just as a follow-up to the shortlinks tutorial posted yesterday: Jumplinks can also come in handy if you want to redirect ID-based URIs.

This is a result of the Destination Selectors feature I added a while back, thanks to the suggestion by Macrura.

The Process:

Let's say you have a Journal located in the root of your page tree, named journal. Each journal entry is assigned to the journal-entry template If you want to redirect ID-based URIs to their named equivalents, create a new jumplink like so:

Source: journal/{id}

Destination: [[template=journal-entry,id={id}]]

So, when requesting /journal/1078, for example, you'll be redirected accordingly to /journal/hello-world/.

Of course, you could take this further, but this is just a basic idea.

Might be handy for someone using Jumplinks. :-)

  • Like 2

Share this post


Link to post
Share on other sites

Hi Mike,

I have installed jumplinks on my new homepage and enabled the 404 logger. When I want to just edit a small piece in a page, I could see life a scanning attack for vulnarebilities:

post-1041-0-28321700-1432454395_thumb.jp

These attacks can test several hundred links. So, after such a scan, the 404 Monitor (limited to the last 100 hits) is pretty useless.

What about (optional) alternative storage of the links? Simple logfile or maybe better a sqlite db file? This way you don't need to spam the MySQL-DB with those spam entries but the useful links also don't get dropped. So, with lots of entries, the 404-Monitor table needs to have pagination then and maybe a function to select / delete entries. If you interested in this, I can provide a ready to use wrapper class for sqlite,  (create, insert, update, read, delete).

-----------

Edit:

Also useful solution could be to exclude the 404-Monitor as separate module and give it some more functions like detecting IPs and when seeing those attacks, temporary block the IP for a short time. But this has nothing to do with jumplinks. That's why I said make 404-Monitor a external / submodule, that can be used for / from different other modules.

Edited by horst

Share this post


Link to post
Share on other sites

Interestingly, I bumped into this issue yesterday on a live site - thanks for bringing it up. :)

Initially, I just implemented it as-is due to the fact that it can be useful in most cases. That said, these kinds of attacks are becoming quite regular these days, and so I think it's a very good idea.

Thank you, I'd very much appreciate a wrapper class, which, in my opinion, is much better than using a log file. :) When I have some time - hopefully soon - I'll upgrade the monitor.

Share this post


Link to post
Share on other sites

Edit:

Also useful solution could be to exclude the 404-Monitor as separate module and give it some more functions like detecting IPs and when seeing those attacks, temporary block the IP for a short time. But this has nothing to do with jumplinks. That's why I said make 404-Monitor a external / submodule, that can be used for / from different other modules.

I'm just wondering if I should maybe integrate the monitor with the new 404 logger by kixe. What do you think?

Scratch that. Unfortunately, the logger doesn't capture as much information as I'd like to show. An SQLite-base solution will be better.

Edit: Perhaps a separate module, included in the package, will be best. Let me think on this.

Edited by Mike Rockett
  • Like 1

Share this post


Link to post
Share on other sites

This is nothing that urges. :)

In the past (not a PW site!) I simply logged all 404 to 2 files, one with only the urls. Then, from time to time, I have manually loaded it into my editor which has a function to make all entries (lines) unique (drop all copies). From the resulting lines I created entries for the htaccess like:

Redirect gone /admin/
Redirect gone /db/
Redirect gone /dbadmin/
Redirect gone /ecommerce/
Redirect gone /sqlmanager/
Redirect gone /typo3/
Redirect gone /webstats/
Redirect gone /wordpress/
Redirect gone /connector.php
Redirect gone /index2.php
Redirect gone /wp-login.php

This way my 404 log becomes more and more clean and the attackers hopefully gets a feeling like "Damn, to late again!" :)

A 404 monitor module that determine a scan attack optionally can send 410 headers for the next 5 or 10 minutes to the originating IP. No more editing the htaccess for this. :)

I will send you the sqlite wrapper. I have looked at it, but currently it depends on a subclass of mine in regards to filelocking. I will change this to result in a single class before sending.

Edited by horst
  • Like 1

Share this post


Link to post
Share on other sites

After some thought:

I'm a little undecided as to how much functionality I would want in this monitor. Currently, Jumplinks serves its purpose, and adding to much to it might not be the greatest idea.

That said, I would be prepared to release a separate 404 monitor module that includes this functionality. Firstly, Jumplinks would be able to tie into this module if it exists. Secondly, I'd be able to add certain security features to it without bloating Jumplinks.

Preferably, I'd want to implement this module in such a way that it modifies the .htaccess file in the following way: when browsing the log, a user with the relevant permissions would be able to blacklist a certain URI for a fixed/indefinite period of time. They'd also be able to block an IP (based on the findings in the log) for a certain/indefinite amount of time. When these 'blocks' are defined/modified, they'd be added to the .htaccess file, and the module would periodically check (lazy-cron-like, say twice a day) to see if the file is up to date. An example would look like this:

#< BEGIN ModuleName <last-check: <time>>

Redirect 410 /wp-admin # always
Redirect 410 /etc # <time>-<time>
Redirect 410 /etc.php <time>-<time>

Require all granted
Require not ip xxx.xxx.xxx.xxx # <time>-<time>
Require not ip xxx.xxx.xxx.xxy # always

#> END ModuleName

For me, this is a performance thing - i.e. booting up the whole PW system just to block/limit access is unnecessary.

With the above in mind, this may land up becoming an .htaccess security-driven module, similar to those made for WP (the bain of my existence); however, it would only include essentials. (It would more than likely not be called '404Monitor'.)

As said, the added benefit would be that the module can provide a 'safe' list of 404s from which Jumplinks can derive it's data.

Thoughts?

  • Like 2

Share this post


Link to post
Share on other sites

I had a website with a lot of subdomains, now I want to redirect all subdomains to internal pages, is there a way to do this with jumplinks?

I would like also to suggest a couple of small "improvements" as result of my experience with this fantastic module.

1. possibility to filter 404 errors per type, like images, pages, css, etc.. I have a lot of 404 errors and having such option could be very helpful.

2. check for already created jumplink. When moving a lot of pages sometimes I created duplicated links without noticing...

Share this post


Link to post
Share on other sites

I had a website with a lot of subdomains, now I want to redirect all subdomains to internal pages, is there a way to do this with jumplinks?

Unfortunately not - it isn't something that happens often, and so it isn't a feature. It might be best for you to use .htaccess for that.

1. possibility to filter 404 errors per type, like images, pages, css, etc.. I have a lot of 404 errors and having such option could be very helpful.

2. check for already created jumplink. When moving a lot of pages sometimes I created duplicated links without noticing...

  1. I will be rebuilding the 404 Monitor as a separate module, and may include that.
  2. That's also on the to do list. Originally, I skipped the validation as I found it was not a common occurrence. But, I do believe that some kind of validation should be put in place.

I do have quite a lot of work at the moment, and so cannot commit to actually getting these things done.

Thanks for your suggestions!

Share this post


Link to post
Share on other sites

Just a heads-up: As mentioned, lot's on the plate these days. But I've made a decision regarding the module going forward. When the next one or two PW minor stables come out, I'm going to start making Jumplinks 2.6-only. Lots of goodies I want to use, and as stated earlier in this thread, Jumplinks is mostly for new sites resulting from migrations. Of course, Jumplinks won't upgrade for anyone using it on 2.5.3. Also, and because of this decision, I'll be switching over to some kind of autoload solution so I can split up logic etc. It's essentially going to be a rewrite. Nonetheless, it will function exactly the same, sans the 404 Monitor, which I'll try to release at the same time.

Speaking of the 404 monitor, I've already drawn out some blueprints and flow charts - ideas have come to mind, and I think it's going to be a really helpful module. As said, it will mesh well with Jumplinks.

:-)

Edit: Okay, I see that 2.6.1 is already on stable (of course, it should be). I'll need some time before I start this up.

Edited by Mike Rockett
  • Like 5

Share this post


Link to post
Share on other sites

Hi Mike, the 404 log starts showing lots of request from Google starting like so:

index.php.pwpj/…

I think the "pwpj" extension is from this module. I have no clue how this fake extension(?) made its way into Google, what the reason is for it, and how to handle it when creating a jumplink.

Can you update the documentation to give a litte insight?

And by the way, a feature request: Is it possible to set a "410 gone" also?

I have a bunch of links to non-existing images that I do not plan to redirect.

Share this post


Link to post
Share on other sites

Hi Mike, the 404 log starts showing lots of request from Google starting like so:

index.php.pwpj/…

I think the "pwpj" extension is from this module. I have no clue how this fake extension(?) made its way into Google, what the reason is for it, and how to handle it when creating a jumplink.

Can you update the documentation to give a litte insight?

It's a method I'm currently using to allow users to create jumplinks that start with index.php. Not sure why Google is indexing those, especially considering they are 404s. If you have a jumplink that checks for index.php?cat=32, for example, but you make a request for anything but the 32 and leave the index.php in the URI, Jumplinks will redirect index.php to index.php.pwpj so that it can be tested.

I will more than likely be using a better method when I rewrite the module.

And by the way, a feature request: Is it possible to set a "410 gone" also?

I have a bunch of links to non-existing images that I do not plan to redirect.

I could indeed do that, but I think it would be better for that to be handled with .htaccess, like so:

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} \.(jpg|png|gif)$
RewriteRule ^ - [G,L]
  1. Check the request doesn't match an existing file
  2. Check that an image is being requested
  3. Send 410 Gone if both conditions match
  • Like 1

Share this post


Link to post
Share on other sites

Good day, Mike (and all the other redirect maniacs)!

I  got a site that responds differently to different subdomains and got Jumplinks installed. I am writing a .htaccess redirect from www of a subdomain to no-www.

RewriteCond %{HTTP_HOST} ^www\.subdomain.domain\.ru$ [NC]
RewriteRule ^(.*)$ http://subdomain.domain.ru/$1 [R=301,L]

If I run www.subdomain.domain.ru it gets redirected to subdomain.domain.ru. But if I go to something like www.subdomain.domain.ru/something I get an error:

You don't have permission to access /index.php.pwpj on this server.

I turned the module off and realised that it is not the module that is the reason, as I still get redirected to subdomain.domain.ru/index.php?it=something/ not to subdomain.domain.ru/something.

I tried putting my code before and after PROCESSWIRE HTACCESS DIRECTIVES block without a difference. Could you please give me a hint here?

Share this post


Link to post
Share on other sites

@Ivan - Hmm, seems strange. Could be a 301 cache issue? Generally better to work with 302 until you're sure it's working correctly.

If that's not the case, I'll clone your setup here and see what happens.

Share this post


Link to post
Share on other sites

Mike, thanks for the module, I just migrated a modx site to pw and am using this full on - running into trouble though.

When I input a jumplink it does a 302 and that is NOT good for google, I need them to be 301.

So the question is how can I make sure all the redirects are 301 and there are no 302

302 means google gets confused and that means my traffic hurts.

Thanks for the help!

Baie dankie

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By d'Hinnisdaël
      Happy new year, everybody 🥬
      I've been sitting on this Dashboard module I made for a client and finally came around to cleaning it up and releasing it to the wider public. This is how it looks.
      ProcessWire Dashboard

      If anyone is interested in trying this out, please go ahead! I'd love to get some feedback on it. If this proves useful and survives some real-world testing, I'll add this to the module directory.
      Download
      You can find the latest release on Github.
      Documentation
      Check out the documentation to get started. This is where you'll find information about included panel types and configuration options.
      Custom Panels
      My goal was to make it really simple to create custom panels. The easiest way to do that is to use the panel type template and have it render a file in your templates folder. This might be enough for 80% of all use cases. For anything more complex (FormBuilder submissions? Comments? Live chat?), you can add new panel types by creating modules that extend the DashboardPanel base class. Check out the documentation on custom panels or take a look at the HelloWorld panel to get started. I'm happy to merge any user-created modules into the main repo if they might be useful to more than a few people.
       Disclaimer
      This is a pre-release version. Please treat it as such — don't install it on production sites. Just making sure 🍇
      Roadmap
      These are the things I'm looking to implement myself at some point. The wishlist is a lot longer, but those are the 80/20 items that I probably won't regret spending time on.
      Improve documentation & add examples ⚙️ Panel types Google Analytics ⚙️ Add new page  🔥 Drafts 🔥 At a glance / Page counter 404s  Layout options Render multiple tabs per panel panel groups with heading and spacing between ✅ panel wrappers as grid item (e.g. stacked notices) ✅ Admin themes support AdminThemeReno and AdminThemeDefault ✅ Shortcuts panel add a table layout with icon, title & summary ✅ Chart panel add default styles for common chart types ✅ load chart data from JS file (currently passed as PHP array) Collection panel support image columns ✅ add buttons: view all & add new ✅
    • By Gadgetto
      Status update links (inside this thread) for SnipWire development will be always posted here:
      2020-01-14 --> new date range picker, discount editor, order notifiactions, order statuses, and more ... 2019-11-15 --> orders filter, order details, download + resend invoices, refunds 2019-10-18 --> list filters, REST API improvements, new docs platform, and more ... 2019-08-08 --> dashboard interface, currency selector, managing Orders, Customers and Products, Added a WireTabs, refinded caching behavior 2019-06-15 --> taxes provider, shop templates update, multiCURL implementation, and more ... 2019-06-02 --> FieldtypeSnipWireTaxSelector 2019-05-25 --> SnipWire will be free and open source If you are interested, you can test the current state of development:
      https://github.com/gadgetto/SnipWire
      Please note that the software is not yet intended for use in a production system (alpha version).
      If you like, you can also submit feature requests and suggestions for improvement. I also accept pull requests.
      ---- INITIAL POST FROM 2019-05-25 ----
      I wanted to let you know that I am currently working on a new ProcessWire module that fully integrates the Snipcart Shopping Cart System into ProcessWire. (this is a customer project, so I had to postpone the development of my other module GroupMailer).
      The new module SnipWire offers full integration of the Snipcart Shopping Cart System into ProcessWire.
      Here are some highlights:
      simple setup with (optional) pre-installed templates, product fields, sample products (quasi a complete shop system to get started immediately) store dashboard with all data from the snipcart system (no change to the snipcart dashboard itself required) Integrated REST API for controlling and querying snipcart data webhooks to trigger events from Snipcart (new order, new customer, etc.) multi currency support self-defined/configurable tax rates etc. Development is already well advanced and I plan to release the module in the next 2-3 months.
      I'm not sure yet if this will be a "Pro" module or if it will be made available for free.
      I would be grateful for suggestions and hints!
      Please have a look at the screenshots to get an idea what I'm talking about (open spoiler):
      (Please note: these screenshots are from an early development state of SnipWire. To see actual screens please have a look at later posts below!)
       
    • By Robin S
      This module is inspired by and similar to the Template Stubs module. The author of that module has not been active in the PW community for several years now and parts of the code for that module didn't make sense to me, so I decided to create my own module. Auto Template Stubs has only been tested with PhpStorm because that is the IDE that I use.
      Auto Template Stubs
      Automatically creates stub files for templates when fields or fieldgroups are saved.
      Stub files are useful if you are using an IDE (e.g. PhpStorm) that provides code assistance - the stub files let the IDE know what fields exist in each template and what data type each field returns. Depending on your IDE's features you get benefits such as code completion for field names as you type, type inference, inspection, documentation, etc.
      Installation
      Install the Auto Template Stubs module.
      Configuration
      You can change the class name prefix setting in the module config if you like. It's good to use a class name prefix because it reduces the chance that the class name will clash with an existing class name.
      The directory path used to store the stub files is configurable.
      There is a checkbox to manually trigger the regeneration of all stub files if needed.
      Usage
      Add a line near the top of each of your template files to tell your IDE what stub class name to associate with the $page variable within the template file. For example, with the default class name prefix you would add the following line at the top of the home.php template file:
      /** @var tpl_home $page */ Now enjoy code completion, etc, in your IDE.

      Adding data types for non-core Fieldtype modules
      The module includes the data types returned by all the core Fieldtype modules. If you want to add data types returned by one or more non-core Fieldtype modules then you can hook the AutoTemplateStubs::getReturnTypes() method. For example, in /site/ready.php:
      // Add data types for some non-core Fieldtype modules $wire->addHookAfter('AutoTemplateStubs::getReturnTypes', function(HookEvent $event) { $extra_types = [ 'FieldtypeDecimal' => 'string', 'FieldtypeLeafletMapMarker' => 'LeafletMapMarker', 'FieldtypeRepeaterMatrix' => 'RepeaterMatrixPageArray', 'FieldtypeTable' => 'TableRows', ]; $event->return = $event->return + $extra_types; }); Credits
      Inspired by and much credit to the Template Stubs module by mindplay.dk.
       
      https://github.com/Toutouwai/AutoTemplateStubs
      https://modules.processwire.com/modules/auto-template-stubs/
    • By Robin S
      Add Image URLs
      Allows images/files to be added to Image/File fields by pasting URLs.

      Usage
      Install the Add Image URLs module.
      A "Paste URLs" button will be added to all image and file fields. Use the button to show a textarea where URLs may be pasted, one per line. Images/files are added when the page is saved.
       
      https://github.com/Toutouwai/AddImageUrls
      https://modules.processwire.com/modules/add-image-urls/
    • By gebeer
      Hello all,
      sharing my new module FieldtypeImageReference. It provides a configurable input field for choosing any type of image from selectable sources. Sources can be: 
      a predefined folder in site/templates/ and/or a  page (and optionally its children) and/or the page being edited and/or any page on the site CAUTION: this module is under development and not quite yet in a production-ready state. So please test it carefully.
      UPDATE: the new version v2.0.0 introduces a breaking change due to renaming the module. If you have an older version already installed, you need to uninstall it and install the latest master version.
      Module and full description can be found on github https://github.com/gebeer/FieldtypeImageReference
      Install from URL: https://github.com/gebeer/FieldtypeImageReference/archive/master.zip
      Read on for features and use cases.
      Features
      Images can be loaded from a folder inside site/templates/ or site/assets Images in that folder can be uploaded and deleted from within the inputfield Images can be loaded from other pages defined in the field settings Images can be organized into categories. Child pages of the main 'image source page' serve as categories mages can be loaded from any page on the site From the API side, images can be manipulated like native ProcessWire images (resizing, cropping etc.), even the images from a folder Image thumbnails are loaded into inputfield by ajax on demand Source images on other pages can be edited from within this field. Markup of SVG images can be rendered inline with `echo $image->svgcontent` Image names are fully searchable through the API $pages->find('fieldname.filename=xyz.png'); $pages->find('fieldname.filename%=xy.png'); Accidental image deletion is prevented. When you want to delete an image from one of the pages that hold your site-wide images, the module searches all pages that use that image. If any page contains a reference to the image you are trying to delete, deletion will be prevented. You will get an error message with links to help you edit those pages and remove references there before you can finally delete the image. This field type can be used with marcrura's Settings Factory module to store images on settings pages, which was not possible with other image field types When to use ?
      If you want to let editors choose an image from a set of images that is being used site-wide. Ideal for images that are being re-used across the site (e.g. icons, but not limited to that).
      Other than the native ProcessWire images field, the images here are not stored per page. Only references to images that live on other pages or inside a folder are stored. This has several advantages:
      one central place to organize images when images change, you only have to update them in one place. All references will be updated, too. (Provided the name of the image that has changed stays the same) Installation and setup instructions can be found on github.
      Here's how the input field looks like in the page editor:

      If you like to give it a try, I'm happy to hear your comments or suggestions for improvement. Install from URL: https://github.com/gebeer/FieldtypeImageReference/archive/master.zip
      Eventually this will go in the module directory, too. But it needs some more testing before I submit it. So I'd really appreciate your assistance.
      Thanks to all who contributed their feedback and suggestions which made this module what it is now.
       
×
×
  • Create New...