module Jumplinks

[Mike Rockett]

@Jason Huck - did you manage to see what the issue may be?

[Jason Huck]

4 minutes ago, Mike Rockett said:

@Jason Huck - did you manage to see what the issue may be?

Yes, thanks for following up. As you pointed out, Jumplinks wasn't even being invoked. I had the basic-page template set to allow URL segments, and the custom logic for that was conflicting with the normal 404 handling (both automatic and via throwing a Wire404Exception manually). The fix was as simple as moving the 404 page to its own template.

[Mike Rockett]

@Jason Huck - Magic, glad you got it sorted.

[Jason Huck]

On the topic of pagination for the main jumplinks page, in a current project, I've migrated over 6,000 URLs from a legacy site. The structure of the URLs changed significantly in multiple ways. I had to write a custom export script for the old system, as well as a custom import script for PW, to get everything to line up correctly. There wasn't any practical way to take advantage of wildcards or mapping collections that would have resulted in a reduction of the total number of entries — it's one-to-one.

Overall, it works well enough. There are two places where the number of entries seems to impact performance. For a legacy URL that has a jumplink, performance is fine. For anything that results in an actual 404, it can take a very long time (about 14 seconds on dev) to return the 404 page.

The other thing that takes a really long time to load of course is the main jumplinks page — so yeah, some pagination (and a keyword filter, actually) would be nice...

[Mike Rockett]

1 hour ago, Jason Huck said:

For anything that results in an actual 404, it can take a very long time (about 14 seconds on dev) to return the 404 page.

That seems strange - it really shouldn't take so long at all... If it is indeed Jumplinks that is causing that delay, v2 of the module will more than likely solve it.

Actually, I take that back - the handler would slow things down. That said, JL2 will speed things up.

1 hour ago, Jason Huck said:

The other thing that takes a really long time to load of course is the main jumplinks page

This was never benchmarked in v1, so I believe you on this one. Perhaps some pagination is in order then... Or perhaps I should switch over to an AJAX-driven table ith filtering...?

[ceberlin]

Can't wait for V2 then :-)))

[Jason Huck]

25 minutes ago, Mike Rockett said:

That seems strange - it really shouldn't take so long at all... If it is indeed Jumplinks that is causing that delay, v2 of the module will more than likely solve it.

Actually, I take that back - the handler would slow things down. That said, JL2 will speed things up.

This was never benchmarked in v1, so I believe you on this one. Perhaps some pagination is in order then... Or perhaps I should switch over to an AJAX-driven table ith filtering...?

It's the same 14-second delay in both cases (and only those cases), so I'm assuming it's related to jumplinks in some way, but admittedly I haven't tried to verify that.

Is tablesorter a standard widget in the PW admin UI? You might consider switching to DataTables ( https://www.datatables.net/ ) which would give you pagination, AJAX loading, keyword filtering, etc. in the default configuration...

[Mike Rockett]

13 minutes ago, Jason Huck said:

It's the same 14-second delay in both cases (and only those cases), so I'm assuming it's related to jumplinks in some way, but admittedly I haven't tried to verify that.

Is tablesorter a standard widget in the PW admin UI? You might consider switching to DataTables ( https://www.datatables.net/ ) which would give you pagination, AJAX loading, keyword filtering, etc. in the default configuration...

Datatables would indeed be my first option. There are others, and I'd ideally like to use one that has the required features and looks good in the admin UI without having to modify much (I would like to also try and retain the design I have above).

I've done some quick, prelim benchmarking on 6000 jumplinks (generated by faker) in v2:

Request → 404 (no hit): Server Processing = ~900ms; Receiving = 0.1ms
Loading Jumplinks table: Server Processing = ~2900ms; Receiving = ~9900ms (memory limit increased to 512MB as it exceeded the default 128MB)

This was done on my laptop dev machine - a 4th-gen Core i7 running PHP 7.0.9.

So it seems all the pre-processing going on in JL2 really doesn't use much at all, and FastRoute makes things much faster by combining all the jumplinks into a single regex for matching and dispatching to the post-processing handler. 900ms (half of that, actually, due to the loading of PW) to scan through 6000 jumplinks is really awesome!

All-in-all, it looks like the only issue is the front-end, which is expected for larger datasets. Given that you say there is no logical structure that could minimise your jumplinks down drastically enough to improve performance, I'm going to go ahead and migrate to Datatables (or something else that may be more suitable; I have seen a few, but can't remember what they are called).

[Mike Rockett]

@Jason Huck and everyone else working with large datasets:

I'm playing around with DataTables, and these are the initial benchmark results for 6000 jumplinks.

• Jumplinks Manager containing only page markup:
  • Server Processing = ~430ms
  • Receiving = ~120ms
• JSON AJAX request for DataTable:
  • Server Processing = ~6320ms
  • Receiving = ~110ms (reduced as there is no bulky table markup)
  • Peak Memory = ~52MB (much better, also reduced to lack of table markup)

Essentially, the loading time has been cut in half. However, this will be further improved the links for the table are being processed server-side via Eloquent accessors. I'm going to move that to client-side processing to cut down the load time even more. There is the option of server-side processing on a per-page basis, but I think that's overkill.

I'm pretty new to datables, to be honest, and so not sure how to do a whole bunch of things. But I'm sure it'll be easy enough (readthedocs, hehe).

[Macrura]

in my datatables testing, my results were that the fastest render were if you have the data in a js object on the page; this way there is no 2nd request for the ajax; and no parsing of the table markup, since it is all pure JS.

[Mike Rockett]

15 minutes ago, Macrura said:

in my datatables testing, my results were that the fastest render were if you have the data in a js object on the page; this way there is no 2nd request for the ajax; and no parsing of the table markup, since it is all pure JS.

The way I see it, doing it that way won't trim much off as most of the time is dedicated to processing the JSON object anyway. The difference is negligible, and would delay the loading of the admin interface markup.

[Macrura]

yeah, tis true; i guess my results apply more for front end, and assumes you are caching the page..

[Mike Rockett]

1 minute ago, Macrura said:

yeah, tis true; i guess my results apply more for front end, and assumes you are caching the page..

Indeed. I think the big thing for me is to let the table show that something is being loaded as well - a feedback mechanism is always good.

[Zeka]

Hi @Mike Rockett. Thanks for this module.

After installation I get this error: 

ProcessJumplinks - SQLSTATE[42S21]: Column already exists: 1060 Duplicate column name 'last_hit'

There were some posts relative to this error earlier, where was suggestion to reinstall module, but i didn't work for my in my case..

It looks like it doesn't work at all for me.

I created simple jumplink, turned on 404 Monitor, went to url which was defined as source for jumplink.

I got notification about 404 error in notification bar, but there was no redirect and 404 monitor is empty.

Do you have any suggestions how to deal with it?Or do I something wrong? 

PW 3.0.35

[Mike Rockett]

@Zeka - Thanks for reporting this. Unfortunately, I haven't allocated some time to see if Jumplinks is compatible with the new PW3 master - it was tested on earlier builds. That said, I still don't understand why this could be happening. The only thing that springs to mind is a possible conflict with another module. I'm not sure which module it is, but I think there is a module that allows you to disable (not uninstall) other modules. Are you perhaps using any of those? If not, mind sharing the source and destination of the jumplink to see what the problem could be?

I'm busy with a client project at the moment, but will try test the module on the latest PW3 master in a few hours.

[Zeka]

@Mike Rockett No, I don't use such module. Could you point me to it? 

I was thinking that it could be conflict with redirect module, so I uninstalled it, but nothing has changed.  

Project on localhost.

Source

about-us.html

Destination

/company/about-us/

 

[Mike Rockett]

@Zeka - I can't remember if it was Admin On Steroids or Module Toolkit... But I doubt you are using either of those - otherwise you would have known if it was disabled or not.

Please turn on debug mode in Jumplinks' config and request about-us.html again - you'll be shown a debug-log which you can then paste into this thread so I can review it.

Thanks

[tpr]

1 hour ago, Mike Rockett said:

@Zeka - I can't remember if it was Admin On Steroids or Module Toolkit... 

Most probably Tracy Debugger to disable modules.

[Mike Rockett]

12 minutes ago, tpr said:

Most probably Tracy Debugger to disable modules.

That's the one! Haha, totally didn't even know which one it was! 

[Zeka]

@Mike Rockett

I have disabled TracyDebbuger, AdminOnSteroids, turned on debug in Jumplinks configuration ... and I don't see debug message on frontend. 

But I have great amout or errors in module log 

[Mike Rockett]

@Zeka - I see - let me quickly test this out on 3.0.35 and see what happens.

[Mike Rockett]

@Zeka - I'm afraid I'm not getting any errors on this side; everything is running smoothly as expected.

I assume that something went wrong during a schema upgrade (which happens when you install the module the first time, or when it is upgraded), and this prevented the schema version tracker from being updated.

The safest bet would be to delete the last_hit column from the process_jumplinks table using whichever database admin tool you have on your server (phpMyAdmin, I'd assume). Then simply refresh the admin interface or request about-us.html again and the schema should update.

Alternatively, I'd be happy to have a look at your database, if you'd prefer.

Edited October 10, 2016 by Mike Rockett

[Zeka]

@Mike Rockett

Thanks for your suggetions and effort.

I deleted  last_hit  column, after that I got error

Failed to init module: ProcessJumplinks - SQLSTATE[42S01]: Base table or view already exists: 1050 Table 'process_jumplinks_nf' already exists

So I deleted this table too and after that, finally Jumplinks started to work.

Thanks!!!

[Mike Rockett]

@Zeka - Great! Glad it worked. I just can't figure out why the schema tracker didn't update before. You don't remember any strange errors popping up (other than the ones you've already seen) at any point, do you? I'd assume this is an edge-case, meaning that it doesn't happen very often, but I would love to be able to find out why it happened.

[Jason Huck]

On 11/14/2015 at 0:04 AM, Mike Rockett said:

In the discussion, I mentioned there would either be a checkbox in the entity editor that allows you to make query strings optional, or that we could use square parenthises to mark them as optional, like this: bioh[?gclid={token:any}].

An option to automatically pass along (but otherwise not match on) query strings would be ideal. I'm seeing a lot of automatically appended tracking codes breaking rules. For example:

?platform=hootsuite

For now, I assume i can do this:

path/to/page/[?platform=hootsuite]

...or even:

path/to/page/[?{token:any}]

...to zap any query string, but poor HootSuite won't get credit for the referral. Tough point to cover when training non-technical administrators to use the module, though.

 

A configuration option to automatically assume trailing slashes are optional would also be very handy, for the same reason. That way admins wouldn't have to remember to append ? to the end of every source URL.

 

Another issue is that source URLs are URL encoded when stored, even if the actual source wasn't. So for example, if the source URL contains an unencoded comma and/or ampersand:

path/to/this,that&theother/

...it gets stored like this:

path/to/this%2cthat%26theother/

...which, while corrected, is not the actual source URL, so it doesn't match.