module Jumplinks

[Mike Rockett]

Progress Update:

I'd like to drop debug mode in favor of a testing tab that I'm quite sure has been brought up before. The idea is to not have to throw your entire site in to JL debug mode just to test one thing in production. I've added this tab now, seems to work well.

[adrian]

Hi @Mike Rockett - just had a: PDOException: SQLSTATE[22001]: String data, right truncated: 1406 Data too long for column 'referrer' at row 1 #22001 in /var/www/html/site/assets/cache/FileCompiler/site/modules/ProcessJumplinks/ProcessJumplinks.module.php:534

because someone was trying to hack in with this link:

/plus/download.php?open=1&arrs1[]=99&arrs1[]=102&arrs1[]=103&arrs1[]=95&arrs1[]=100&arrs1[]=98&arrs1[]=112&arrs1[]=114&arrs1[]=101&arrs1[]=102&arrs1[]=105&arrs1[]=120&arrs2[]=109&arrs2[]=121&arrs2[]=116&arrs2[]=97&arrs2[]=103&arrs2[]=96&arrs2[]=32&arrs2[]=40&arrs2[]=97&arrs2[]=105&arrs2[]=100&arrs2[]=44&arrs2[]=101&arrs2[]=120&arrs2[]=112&arrs2[]=98&arrs2[]=111&arrs2[]=100&arrs2[]=121&arrs2[]=44&arrs2[]=110&arrs2[]=111&arrs2[]=114&arrs2[]=109&arrs2[]=98&arrs2[]=111&arrs2[]=100&arrs2[]=121&arrs2[]=41&arrs2[]=32&arrs2[]=86&arrs2[]=65&arrs2[]=76&arrs2[]=85&arrs2[]=69&arrs2[]=83&arrs2[]=40&arrs2[]=57&arrs2[]=48&arrs2[]=49&arrs2[]=51&arrs2[]=44&arrs2[]=64&arrs2[]=96&arrs2[]=92&arrs2[]=39&arrs2[]=96&arrs2[]=44&arrs2[]=39&arrs2[]=123&arrs2[]=100&arrs2[]=101&arrs2[]=100&arrs2[]=101&arrs2[]=58&arrs2[]=112&arrs2[]=104&arrs2[]=112&arrs2[]=125&arrs2[]=102&arrs2[]=105&arrs2[]=108&arrs2[]=101&arrs2[]=95&arrs2[]=112&arrs2[]=117&arrs2[]=116&arrs2[]=95&arrs2[]=99&arrs2[]=111&arrs2[]=110&arrs2[]=116&arrs2[]=101&arrs2[]=110&arrs2[]=116&arrs2[]=115&arrs2[]=40&arrs2[]=39&arrs2[]=39&arrs2[]=57&arrs2[]=48&arrs2[]=115&arrs2[]=101&arrs2[]=99&arrs2[]=46&arrs2[]=112&arrs2[]=104&arrs2[]=112&arrs2[]=39&arrs2[]=39&arrs2[]=44&arrs2[]=39&arrs2[]=39&arrs2[]=60&arrs2[]=63&arrs2[]=112&arrs2[]=104&arrs2[]=112&arrs2[]=32&arrs2[]=101&arrs2[]=118&arrs2[]=97&arrs2[]=108&arrs2[]=40&arrs2[]=36&arrs2[]=95&arrs2[]=80&arrs2[]=79&arrs2[]=83&arrs2[]=84&arrs2[]=91&arrs2[]=103&arrs2[]=117&arrs2[]=105&arrs2[]=103&arrs2[]=101&arrs2[]=93&arrs2[]=41&arrs2[]=59&arrs2[]=63&arrs2[]=62&arrs2[]=39&arrs2[]=39&arrs2[]=41&arrs2[]=59&arrs2[]=123&arrs2[]=47&arrs2[]=100&arrs2[]=101&arrs2[]=100&arrs2[]=101&arrs2[]=58&arrs2[]=112&arrs2[]=104&arrs2[]=112&arrs2[]=125&arrs2[]=39&arrs2[]=41&arrs2[]=32&arrs2[]=35&arrs2[]=32&arrs2[]=64&arrs2[]=96&arrs2[]=92&arrs2[]=39&arrs2[]=96

I'm not sure the best approach to preventing this exception - not sure if you should change the field type for the referrer column to support larger entries, or if you should just truncate the referrer if it's longer than allowed.

[teppo]

3 hours ago, adrian said:

I'm not sure the best approach to preventing this exception - not sure if you should change the field type for the referrer column to support larger entries, or if you should just truncate the referrer if it's longer than allowed.

Collecting super long referrer values seems pointless (they're likely spam anyway), so truncating seems like a good approach. And the same applies to UA string as well ?

--

Edit: just checked a few log files, and I've got similar errors there... and some of these:

Exception: SQLSTATE[22001]: String data, right truncated: 1406 Data too long for column 'user_agent' at row 1 (in /site/modules/ProcessJumplinks/ProcessJumplinks.module.php line 534)

 

[gebeer]

On 1/28/2016 at 8:40 PM, ceberlin said:

• If I use the 404 monitor to fix a link the link should be removed from the 404 monitor so I do not fix it twice

@Mike Rockett has this been addressed yet? I have a customer complaining about this. Would be really nice if those 404 entries got removed. Any plans on implementing this? And thousand thanks for this great module!

[Mike Rockett]

@gebeer – Thanks ? This hasn't been addressed in v1, as the plan was to carry the behaviour into v2 (sorry it's taken so long…) by way of the separate module. I picked up the project again recently and was giving it some thought. I might just include the 404 logger in JL v2, but am still 50/50 on it.

@teppo @adrian – You know, I should have truncated that alongside truncation of the request_uri. Going to see if I can squeeze this in over the weekend while I work on v2’s frontend.

[gebeer]

On 9/11/2020 at 6:14 AM, Mike Rockett said:

Thanks ? This hasn't been addressed in v1, as the plan was to carry the behaviour into v2 (sorry it's taken so long…) by way of the separate module. I picked up the project again recently and was giving it some thought. I might just include the 404 logger in JL v2, but am still 50/50 on it.

Thanks for clarifying. Which hook would you use to trigger deletion of logger entries, ___executeEntity or ___executeCommit?

[Mike Rockett]

@gebeer It'll likely be executeCommit as executeEntity returns the view. However, the 'routes' will be split out in v2 (namely, executeStore, executeUpdate, executeDisable, executeEnable, the first two of which will probably be used to trigger a cleanup).

[gebeer]

If you are using v1 of this module and you want to auto-remove entries from the 404 monitor when you create a jumplink from an entry there, you can use this simple hook in site/templates/admin.php

$this->addHookBefore('ProcessJumplinks::executeCommit', function(Hookevent $event){
    $input = $this->input->post;
    $sourcePath = $input->sourcePath;

    if($sourcePath) {
        $statement = "DELETE FROM process_jumplinks_nf WHERE request_uri = :sourcePath";
        try {
            $this->database->prepare($statement)->execute([
                'sourcePath' => $sourcePath
            ]);
            $this->message("Eintrag '$sourcePath' wurde aus 404 Monitor Liste entfernt");
        } catch (\Throwable $th) {
            $this->error("Fehler beim Entfernen des Eintrags '$sourcePath' aus der 404- Monitor Liste: " . $th->getMessage());
        }
    }
});

and adapt the messages to your liking :-)

[teppo]

On 5/15/2020 at 7:33 PM, Mike Rockett said:

@a-ok Jumplinks 1 doesn't have built in support for automatically dealing with query strings. This is planned for Jumplinks 2, but I simply haven't had the time to complete it. For the time being, I'd be inclined to use the {all} wildcard on the end.

Source: chance-to-dance[/]{all}
Destination: somewhere/{all} or {!all} to skip wildcard cleaning

Hey @Mike Rockett, and sorry to bother you with this again, but just wanted to run something by you.

I've tried the solution above, but at least in my case it doesn't seem to be quite that simple. For some background, in this case the source path is "tilaukset" (orders in Finnish), and it should always link to a specific target URL. This works fine as-is, but when it's shared on Facebook, following the link from there results in 404 due to added query string.

Going step by step:

1. If I append [/] to the end, it works with or without slash at the end — so far so good.
2. If I also append {all} or {!all}, the redirect only seems to work if there's content there, i.e. plain "tilaukset" or "tilaukset/" no longer work.

I'm not sure if this is intentional or not, but I guess the question is if there's a way to specify that "I don't care what comes after this, it could contain nothing or it could contain content, either way ignore it"? ?

I'm using <tilaukset|tilaukset?{all}> for now, but that seems pretty complicated when what I really want is (in terms of regex) "tilaukset.*" (or alternatively "tilaukset/?.*"). This also keeps the query string intact; I'd prefer it to be dropped, but that's not by any means a major issue.

Am I missing something?

[Jason Huck]

FWIW, we needed to preserve query string parameters (mostly UTM params for campaign tracking) with a lot of permutations, and after trying a few different things, we ended up adding this bit to init.php. It seems to be working well for our particular use case.

	// Hook the 404 routine to catch valid URLs with UTM params.
	$this->addHookBefore('ProcessPageView::pageNotFound', function($event){	
		// Get the originally requested URL and its query string.
		// $_SERVER['QUERY_STRING'] is not reliable in this environment.
		$source_url = parse_url($_SERVER['REQUEST_URI']);
		$source_path = $source_url['path'];
		$source_querystring = $source_url['query'];	
		
		// Do nothing if there is no query string. Jumplinks can manage that.
		if(!isset($source_querystring)) return;

		// See if we have a jumplinks entry for the source path WITHOUT the query string.
		$db = wire('database');
		$source_string = trim($source_path, '/');
		$sql = "SELECT source, destination FROM process_jumplinks WHERE source = '".$source_string."/?'";
		$result = $db->query($sql);
		$match = $result->fetch(\PDO::FETCH_ASSOC);
		
		// If any matches are found, try redirecting to it, appending the original query string.
		$target = $match['destination'];
			
		if($target != ''){
			// is this a path or a page ID?
			if((strpos($target, 'page:') !== false)){
				$id = (int) ltrim($target, 'page:');
				$page = wire('pages')->get($id);
				$target_url = $page->url.'?'.$source_querystring;
			}else{
				$target_url = '/'.str_replace('{!all}','',$target);
				$target_url .= '?'.$source_querystring;
			}

			// wire('log')->save('querystrings', 'Source: '.$source_path.', Target: '.$target_url);		
			wire('session')->redirect($target_url);
		}
	});

Maybe others will find it helpful.

 

[Mike Rockett]

@teppo @Jason Huck – In JL1, this unfortunately requires two jumplinks – one for including the query string, and the other for excluding it. I know, it really sucks, but JL2 solves this problem by using optional parameters and a little bit of internal trickery. I do still need to add in a boolean flag on jumplinks to indicate whether or not they should even care about the query string. I would prefer to not implement this flag, but sometimes it's useful when you don't care about it so much that you'd rather not even make mention of it in the source.

[bbeer]

Hi Folks
I wonder if I could strip with Jumplinks from certain URL's like
https://www.site.com/de/produkte/kuehlgeraete-fuer-schaltschraenke/kg_4304/43042001/#prodDescription
just the last part #prodDescription. Could also be a general thing, that whenever this #prodDescription appears, it would be removed.
The client uses those links into a tab on an other System, which means there would several hundert links to be changed.

Your help is much appreciated!

[d'Hinnisdaël]

 

2 minutes ago, bbeer said:

I wonder if I could strip with Jumplinks from certain URL's like
https://www.site.com/de/produkte/kuehlgeraete-fuer-schaltschraenke/kg_4304/43042001/#prodDescription
just the last part #prodDescription.

I'm pretty sure those anchor hashes never reach the server and are evaluated by the browser only.

[bbeer]

sorry you're right they got to the browser as %23prodDescription. So we would need to strip %23prodDescription this.

[d'Hinnisdaël]

1 minute ago, bbeer said:

sorry you're right they got to the browser as %23prodDescription. So we would need to strip %23prodDescription this.

I think what I meant to say was that the hash (#prodDescription) is never sent to the server. ProcessWore doesn't know about it. Any redirects that are dependent on the presence or absence of a hash need to be done in JavaScript on the client.

[bbeer]

when you call the page with those # it opens the correct page. The Problem is, that they copied all those links with the # and use them in the CRM. So when they generate Quotes the link to this pages form a generated PDF Quote. Therefor it would be helpful if we could remove this particular part %23prodDescription of the url, then the Link would work perfectly. Else you just get an 404.

[Mike Rockett]

@bbeer – if you are getting URLs with the encoded %23, then I think that htaccess would be your best bet until JL2 is ready (working on it as I write this). It must be that the CRM is encoding the hash when storing the URL (bad move, CRM), which the browser then sends to the server, which sees it as part of the actual URI. The browser will never send it if it is not encoded. 

If htaccess is too messy, then you could perhaps append the source with {all}, which would catch everything after the last 'segment' – the destination could then just ignore this. Not the best solution I know, but seems like it could work for the time being…

[paul@johnsonville.org]

I just installed the ProcessJumplinks module. I am trying to have one of my existing pages redirect to a separate url.

I would like to have visitors to https://www.johnsonville.org/recipes/ be redirected to https://www.johnsonville.org/docs/!recipes/doku.php

Here's what I've set as a jumplink:

Source: recipes/
Destination: https://johnsonville.org/docs/!recipes/doku.php

I'm not seeing any redirection... is this something that can be accomplished with ProcessJumplinks?

[adrian]

Jumplinks is only designed to kick in when a URL 404s.

Your best bet is you place the redirect in the template file that is used to display the /recipes/ page, eg:

$session->redirect('/docs/!recipes/doku.php');

 

[paul@johnsonville.org]

Thanks for the clarification!

[Didjee]

Line 785 of ProcessJumplinks.module results in unexpected behaviour:

$this->session->redirect($convertedWildcards, !($starts || $ends));

As soon as the start time of a jumplink is set (after the first hit), the redirect wil be 302 (temporary) even if the end date is empty/not set.
I think only redirects with an end date should be temporary (302), otherwise permanent (301).

[Mike Rockett]

New Maintainer Needed

Jumplinks is in need of a new maintainer, as I’m simply unable to commit to continued development. As it is, Jumplinks v2 has been in the pipeline for around four years now. Whilst v1 is stable (after so long, excluding a few minor issues here and there), there are features that people would like in v2 that are not in v1, and so the idea is that a capable maintainer takes over and breathes some life back into the project as I'm unable to due to work commitments.

Current state of v2: backend is mostly functional, and the frontend was being re-written in Svelte (though whomever takes over is free to change this) after having been partially done in jQuery (I no longer use it for anything). As discussed previously, the backend side of things is powered by some Illuminate (Laravel) components, making it easier to interact with the DB and perform tasks on different models. Again, whomever takes over the project is free to change this, or even start from scratch if necessary.

If you are reading this message and are interested in taking over the development of Jumplinks, please pop me a private message and we can take it from there.

Thanks all! ?

[Peter Knight]

Hi @Mike Rockett

Are JumpLinks to external sites generating 301 redirects?
I need to redirect most of the content of a site to an eternal site and unsure if I should use JumpLinks or a HT access 301 redirect.
Many thanks
 

[Mike Rockett]

Hey @Peter Knight

All redirects are 301s, unless they are timed.

[Peter Knight]

Hi Mike.

I was having issues with a site recently where the database seemed to baloon in size (98 MB) and I tracked it down to the Jumplinks Module.

I think the culprit is the 404 monitor which only displays the latest 100 but might possibly keep a record of everything?

After logging into the Jumplinks admin area and clearing the 404 records, the table is back down to a few kb.

Actually, my entire database is down from 103 MB to 5 MB

Is there a way to prevent this instead of manually clearing the 404 recording? Assuming this is the culprit...

Thanks