image URL on pages (problem loading)

[Joe]

Hello everyone!

I have been testing ProcessWire to use as my new CMS for clients. I quite like it. Now I´ve run into a curious problem I couldn´t find any explanation for searching this forum, so I decided to join and see if anyone knows.

So here my question:

I upload an image on a page and put it in the page body via the editor. When I view the page the image takes an exceptionally long time for loading. I finally found the reason for this:

The URL for the image on the page is:

(I had to remove "http://" in front of all these URLs here so they stay readable and are not posted as an actual link, so you have to imagine the "http://" in front of each of these URLs)

1.2.3.13/bmi/1.2.3.11/bmi/this_domain.com/folder_for_this_installation/site/assets/files/1006/x.jpg

whereby "this_domain.com" represents the domain the installation is running under and "folder_for_this_installation" is the directory it is installed in. Everything else seems to be running fine and all the pages are visible under

this_domain.com/folder_for_this_installation/

(this is not the actual URL, just for illustration purposes).

When I call up the URL for the image it does load, both with:
1.2.3.13/bmi/1.2.3.11/bmi/this_domain.com/folder_for_this_installation/site/assets/files/1006/x.jpg
and with:
this_domain.com/folder_for_this_installation/site/assets/files/1006/.x.jpg
the difference however being that in the second instance it loads much faster, the way it should be.

To make sure the problem was not caused by any modifications of mine I used a brand new "out of the box" installation.

Does anyone know what may be causing this? Thank you!

Joe

[teppo]

Hello Joe -- welcome to the forum!

Can't really say right away what could cause this, but that sounds like image resizing gone wrong. Are you using master or dev branch of ProcessWire and is there anything strange either at the server error log or PW's own error log at /site/assets/logs/errors.txt?

There have been some changes to image handling in dev branch lately, but I haven't really been keeping track of those, so if you're using dev you could always try installing new PW from master branch to see if the issue exists there too. Also, are there any other files in that directory, apart from x.jpg and cabin1.x.jpg (are these actual filenames by the way?)

It would also be helpful if you posted some basic details about your server.. is it running Linux, OS X or Windows, is the web server Apache, ngingx or something else etc.

[Joe]

Hello teppo,

thank you for your quick reply.

I just saw I made a mistake when posting: I´m calling the image "x.jpg" here to make things clearer. It is the only image, "cabin1.x.jpg" doesn´t exist. (corrected in the post now)

No, there are no other images, except "x.300x0.jpg" and "x.0x100.jpg" - I resized the image when putting it on the page.

It is a Linux-Apache server, if it will be of any help I can post the output of phpinfo. And actually it is the second Apache server this is happening on, I tried it on another server first.

It is the latest stable version 2.3.0 just downloaded about a week ago directly from http://processwire.com/ , and as I said, for testing purposes I installed a fresh copy without making any changes to it.

[Joe]

Hi, teppo,

I just checked: /site/assets/logs/errors.txt does not exist, /site/assets/logs/ is empty.

I´m on shared hosting and do not believe the server logs are accessible  to me.

[kongondo]

Hi Joe,

Welcome to PW and the forums.

Did you test PW on a local install? (WAMP, XAMPP, etc). Did it work then? 

Seems maybe some of your files were not transferred to the server? You can manually create the errors.txt file and see what PW writes to it.

Do not post your phpinfo publicly. You can PM it to Teppo to have a look.

PHP version? MySQL version?

Did you try with other images? Same results?

[Joe]

Hello kongondo,

yes, I first tested locally on a WAMP-Server, did not notice the problem there.

Well, the instance of ProcessWire I used for testing this now I installed by uploading the ProcessWire-master.zip to the server and unzipping it there.

Ok, I will put an errors.txt file there.

Greetings

[Joe]

Hello kongondo,

I manually created the errors.txt in /site/assets/logs/ . I then created a new page and repeated the image upload and insertion. Same problem - image URL is given on the page as:

1.2.3.13/bmi/1.2.3.11/bmi/this_domain.com/folder_for_this_installation/site/assets/files/1006/x.jpg
instead of:
this_domain.com/folder_for_this_installation/site/assets/files/1006/.x.jpg

(leaving "http://" in front out here because then the URL becomes unreadable here in the post)

I must point out that both URLs work, but the one I get on the page with "1.2.3.13/bmi/1.2.3.11/bmi" in it takes much longer to load. But since both work there is not really an error I suppose and therefore probably nothing written to errors.txt I assume.

[kongondo]

OK, but have you actually checked if anything is written in errors.txt?

Could you confirm whether it is one image causing issues or you have tested with different images?

[Joe]

Hello kongondo,

yes, just checked again, there is nothing written in errors.txt (after repeating the test).

No, it happens with each and every image.

And also it happened the same way on another apache server I was testing on first.

[kongondo]

Strange....well it seems definitely a server configuration issue. 

Confirm your PHP and MySQL versions.

I'd also suggest testing with some low size image (just trying to rule out stuff here) of different types - jpg, png, gif...

Btw, is that TinyMCE you are using or CKEditor?

[Joe]

PHP Version 5.3.27

mysql Client API version 5.1.70

It is TinyMCE, because for tracking down the problem I use an "out of the box" installation, as downloaded without any changes of adjustments whatsoever. But the same thing happened with CKEditor, which I was using before.

It happened with all images. The .jpg I used for this last test is about 30KB. Now I just tested with a .gif and a .png as well, both under 5KB, same thing.

And like I said, this happened consistently with each and every image, on several test installs on two different servers. The images were always loading very slowly - I only found out that this is related to the image URL today.

[Joe]

So here I found a possible hint at what might be going on, even though I can´t make sense of it yet:

Like I described at the beginning, the problem is that "1.2.3.13/bmi/1.2.3.11/bmi/" gets inserted into the image URLs of the images on the pages, only the image URLs though, not the page URL. They still load, but much slower. It also happened on another server, got these instead:
"1.2.3.10/bmi/1.2.3.12/bmi/"

Now I found that all these IPs point to:

IP Information for 1.2.3.12
IP Location:    Australia Australia South Brisbane Apnic Debogon Project
ASN:    Australia AS15169 GOOGLE - Google Inc. (registered Mar 30, 2000)
IP Address:    1.2.3.12 [Whois] [Reverse-Ip] [Ping] [DNS Lookup] [Traceroute]
Whois Server    whois.apnic.net

(Info from http://whois.domaintools.com/1.2.3.12 )

On some forum I found the following:

A 'bogon' is an unallocated IP address. There are still ranges of IP addresses which are reserved but not yet in use. These are filtered or ignored by many ISPs as they will only show up if something is misconfigured or something malicious is going on. But IP addresses are running out. The 'debogonizing' (ugh) project is designed to get these addresses assigned to companies and out of the ISP filters.

1.2.3.11 would have been a bogon, but isn't now (although your ISP might still think it is). A google search implies it's something to do with mobile phone proxy servers. If you remove '1.2.3.11/bmi/' from the URL, you should see the picture as it was originally intended.

But why and how would that get into the image URL? I also read some vague hints about trojan infections or the like. However, where would that be? On my computer locally? On the servers??

Anyway, I´m going offline to run a comprehensive antivirus security check locally to see...

[Joe]

So I finally found out how the "1.2.3.13/bmi/" gets into the image URLs! It´s not a virus or the like, rather the Mobile Broadband provider injects this into the URL and you get the image via a proxy (a more compressed version of it, it is being reported). So I suppose for the CMS, when the new content just has been added the proxy doesn´t have it in the cache of course and therefore fetches it and compresses it, that´s why it takes so long. Whereas if you visit a more high volume site the images already are in the cache and so it´s not noticeable.

I checked: a large percentage of the image-URLs of sites I visit have the "1.2.3.13/bmi/" in them, I just hadn´t noticed because they seemed to load normally. (I have only been using my current Mobile Broadband provider for a few weeks now).

This seems to be fairly common with mobile broadband. They do it in order to save bandwidth (serving more compressed images rather than the original). I saw some hint that it is accomplished via an inserted javascript. Well, whichever way it´s done, I certainly don´t like the fact that my pages are changed on the way to the browser!

The solution is probably to insert a a header that prevents caching - I still have to work out whether to do this by setting it on the server or inside ProcessWire.

But anyway, it is not actually really a ProcessWire problem as such. Thank you teppo and kongondo for your advice. When I have resolved the issue I will post it here.

[apeisa]

The solution is probably to insert a a header that prevents caching - I still have to work out whether to do this by setting it on the server or inside ProcessWire.

Glad you figured out. I don't think the solution is good though: that would break the caching for all - and make site loading generally pretty sluggish. I would just live with it (or change mobile broadband).

[kongondo]

Joe,

Nice to know it is resolved...

[horst]

... and you get the image via a proxy (a more compressed version of it, ...

If you don't want that in some particular situations, you can use HTTP-Header like Cache-Control "no-transform" for images!

http://mobiforge.com/developing/story/setting-http-headers-advise-transcoding-proxies

[Joe]

apeisa, on 10 Sept 2013 - 20:55, said:

Glad you figured out. I don't think the solution is good though: that would break the caching for all - and make site loading generally pretty sluggish. I would just live with it (or change mobile broadband).

Hello, apeisa:

It is not really just a matter of me living with it - The images really load incredibly slowly, even small ones. When I visit other sites though, images load normally. And I had the same feedback from a prospective client trying out my test version for a PW-site, so this would affect visitors to the site.

As far as changing mobile broadband: That would only solve the problem for me, not for other users visiting the site via mobile broadband providers that set this up in a similar fashion, and I gather this is done quite a bit by mobile broadband service providers.

[Joe]

If you don't want that in some particular situations, you can use HTTP-Header like Cache-Control "no-transform" for images!

http://mobiforge.com/developing/story/setting-http-headers-advise-transcoding-proxies

Thank you, Horst

Actually I am struggling to accomplish this right now. I put the following lines into the .htaccess file of the directory ProcessWire resides in:

<FilesMatch "\.(jpg|jpeg|png|gif)$">

Header set Cache-Control "no-transform, must-revalidate, private"

</FilesMatch>

with no effect whatsoever.

Header set Cache-Control "no-cache"   instead did not work either.

[horst]

hhm, I'm not sure with apache stuff, but I think its some kind of wrong syntax.

If you are more pragmatic then theoretic (like me), you may try:

Header append Cache-Control "no-transform"
Header append Cache-Control "private"

and look if/what headers are sent.

[Joe]

horst, thanx, you mean I talk too much ?

well:

<FilesMatch "\.(jpg|jpeg|png|gif)$">
Header append Cache-Control "no-transform"
Header append Cache-Control "private"
</FilesMatch>

...seems there is no effect. I'm also no apache expert....

[horst]

horst, thanx, you mean I talk too much ?

no - I've thought the theoretic way is to search via google or at apache: http://httpd.apache.org/docs/current/mod/mod_headers.html under examples 6)

[horst]

Oh, sorry I have missread your post. I've thought using header set with only one value has worked.

I think you first have to check if  mod_headers  is available.

[interrobang]

I never had this problem myself, but HTML5 Mobile Boilerplates .htaccess has some rules for this:

# Prevent some of the mobile network providers from modifying the content of
# your site: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.5.

<IfModule mod_headers.c>
   Header set Cache-Control "no-transform"
</IfModule>

# Prevent mobile transcoding

<FilesMatch "\.(php|cgi|pl)$">
   <IfModule mod_headers.c>
       Header append Cache-Control "no-transform"
       Header append Vary "User-Agent, Accept"
   </IfModule>
</FilesMatch>

[Joe]

I never had this problem myself, but HTML5 Mobile Boilerplates .htaccess has some rules for this:

# Prevent some of the mobile network providers from modifying the content of
# your site: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.5.

<IfModule mod_headers.c>
   Header set Cache-Control "no-transform"
</IfModule>

# Prevent mobile transcoding

<FilesMatch "\.(php|cgi|pl)$">
   <IfModule mod_headers.c>
       Header append Cache-Control "no-transform"
       Header append Vary "User-Agent, Accept"
   </IfModule>
</FilesMatch>

Thank you interrobang!

I put this into the .htaccess file and bang: The images load lightening fast!

I still don´t quite understand the mechanism, not sure if the second part below "# Prevent mobile transcoding" is needed here and how it affects things. >> Looking again, I suppose I do understand it. - The php-output is kept from being re-written and thus having the proxy address inserted into the image URLs.

Also, the images seem to still get delivered via a proxy, as their URLs still have "1.2.3.9/bmi/" in them.  >> Turns out this only happened at first, or at least so it appears. Now the "1.2.3.9/bmi/" is gone!

So thank you again everybody for your advice and in particular to interrobang for the solution that worked!