Jump to content

Module: AIOM+ (All In One Minify) for CSS, LESS, JS and HTML


David Karich

Recommended Posts

Hi David, I'll try out that change you mentioned, and I appreciate the whitelist feature - thanks!

I'm trying to imagine a scenario where including a javascript/css/less file outside of the templates directory poses a security risk. How could that be exploited?

Hi Jonathan , 

yes, in this function paths are filtered to prevent directory traversal attacks. AIOM+ loads only allowed files that are located in the template folder. I think about it, in one of the upcoming versions, introduce a whitelist for asset folder.

For a workaround change the following line (749) in AllInOneMinify.module

$_path  = str_ireplace(array('../', './', '%2e%2e%2f', '..%2F'), '', (wire('config')->paths->templates.$_file));

to 

$_path  = wire('config')->paths->templates.$_file; 

I have not tested it and I can not recommend it too.

Link to comment
Share on other sites

Hi David, I'll try out that change you mentioned, and I appreciate the whitelist feature - thanks!

I'm trying to imagine a scenario where including a javascript/css/less file outside of the templates directory poses a security risk. How could that be exploited?

The easiest way is by infected third-party applications. Also on Bower or Composer, its possible, that compromised packages can be delivered. I have currently identified no scenario targeted at Process Wire, rather it is one of my personal programming rules. Just do not make it possible. Asset files belong in the template folder. This approach I have always in development, regardless of platform or environment. I have some clients who work with sensitive data, and the safety requirements are very high. Therefore, this is my natural data schizophrenia. :D

  • Like 3
Link to comment
Share on other sites

Would it be possible to provide an alternate less compiler for those with node available on the server? 

I'm not really familiar with Node.js and I have no test environment. You're welcome to implement an alternative and send a pull request on GitHub. I think it requires only an option in the backend more and instead of PHP-based LESS parser an exec-command-line function call.

The question is rather, the detour via PHP is really faster? Because you can embed the LESS files directly into the template and can render there via LESS.js or over the Node.js package. The Node.js package has a minimization option on board.

  • Like 1
Link to comment
Share on other sites

I'm not really familiar with Node.js and I have no test environment. You're welcome to implement an alternative and send a pull request on GitHub. I think it requires only an option in the backend more and instead of PHP-based LESS parser an exec-command-line function call.

The question is rather, the detour via PHP is really faster? Because you can embed the LESS files directly into the template and can render there via LESS.js or over the Node.js package. The Node.js package has a minimization option on board.

Compiling all of bootstrap with the php implementation takes 3+ seconds which is an annoyance when developing themes, with the node.js implementation it's half a second. I agree that using less in the template being a great option. I think I'll set things up that way. Thanks David

Link to comment
Share on other sites

  • 5 weeks later...
  • 4 weeks later...

Hi David,

Thanks again for a great module. Just wondering if you did any experimenting with other js minification libraries? I am no expert on this, but noticed that I can get significantly smaller files (in one case 400kb down to 300kb) by minifying using http://jscompress.com/ I know that runs on nodejs, but there are other php alternatives out there, although maybe none of them are any better.

Maybe you've already been through this process and decided JSMin is the best option, but just thought I'd ask :)

Link to comment
Share on other sites

Hi David,

Thanks again for a great module. Just wondering if you did any experimenting with other js minification libraries? I am no expert on this, but noticed that I can get significantly smaller files (in one case 400kb down to 300kb) by minifying using http://jscompress.com/ I know that runs on nodejs, but there are other php alternatives out there, although maybe none of them are any better.

Maybe you've already been through this process and decided JSMin is the best option, but just thought I'd ask :)

Hi adrian,

for sure. JSMin is a little bit outdated and generates not the best compression result, but it is the only stable php port without any conditions to third party libraries. Some other minifications libraries destroyed the compressed js code and JSMin is the only one, with which I never had such a problem. Anyway, i'm thinking about, to integret the google closure compiler as an alternative third party service.

Link to comment
Share on other sites

Thanks for the explanation. I am sure you've seen it, but this looks promising:

https://github.com/tedivm/JShrink

Anyway, thanks again for this module - it's a real time saver.

On an unrelated note - I did notice something yesterday when using the loadOn option. It worked fine for CSS, but it didn't seem to work properly with JS. Before you go delving too deep, I should do some more testing and see what I can find - might have been an issue at my end. I'll try again soon and let you know how it goes.

Link to comment
Share on other sites

Thanks for the explanation. I am sure you've seen it, but this looks promising:

https://github.com/tedivm/JShrink

Anyway, thanks again for this module - it's a real time saver.

On an unrelated note - I did notice something yesterday when using the loadOn option. It worked fine for CSS, but it didn't seem to work properly with JS. Before you go delving too deep, I should do some more testing and see what I can find - might have been an issue at my end. I'll try again soon and let you know how it goes.

Oh, no. This library has not yet come under my eyes. I'll be looking at this a bit more detail when I have more time again. Thanks for the tip.

Yes, check again and let me know, because it is the identical function that is called. If there should be a bug, CSS should not work as well. If you were able to find something, then just open a ticket on Github. :)

Link to comment
Share on other sites

Author of JShrink here (I found you guys in my web logs). I thought I'd pop in and say hello, and talk about JShrink a bit. 

I originally made it as a drop in replacement for JSMin, because JSMin is technically not open source (it has the "do no evil" license, so projects that take open source seriously, such as Debian, won't bundle projects with it). Since then I've made improvements on speed, added additional features like support for conditional comments and license preservation, and have really  put work into the test suite itself. Ryan Grove, who originally ported JSMin to PHP, has officially dropped the JSMin project and is referring people to JShrink on the Github page for it.

Yesterday, after several years of being in the "beta" phase, I launched the JShrink v1.0 to make the API official and stable. 

On a tangental note, I also maintain a caching library called Stash that may be useful to you.

  • Like 5
Link to comment
Share on other sites

AIOM minifies my CSS file without any problems, but for some reason I have not been able to minify any javascript files. 

While outputing javascript the old way works:

// _init.php
$jsPath = $config->urls->templates . 'styles/js/';

// _init.php
$page->jsFiles = array(
    'jquery.min.js',
    'bootstrap.min.js',
    'jquery.fancybox.pack.js',
    'jquery.isotope.min.js',
    'jquery.sticky.js',
    'retina.js',
    'scripts.js',
);

// _done.php
foreach ($page->jsFiles as $file) {
    $fullPath = $jsPath . $file;
    echo "<script src='{$fullPath}'></script>\n";

}

// Output
// <script src='/processwire/site/templates/styles/js/jquery.min.js'></script>
// <script src='/processwire/site/templates/styles/js/bootstrap.min.js'></script>
// <script src='/processwire/site/templates/styles/js/jquery.fancybox.pack.js'></script>
// <script src='/processwire/site/templates/styles/js/jquery.isotope.min.js'></script>
// <script src='/processwire/site/templates/styles/js/jquery.sticky.js'></script>
// <script src='/processwire/site/templates/styles/js/retina.js'></script>
// <script src='/processwire/site/templates/styles/js/scripts.js'></script>
 
Using AIOM as shown below always resulted in a file with the content of
/** Generated: Thursday, 8th of May 2014, 11:36:39 PM // Powered by AIOM+ (All In One Minify) by www.flipzoom.de **/:
 
// _done.php
$jsFiles = array();

foreach ($page->jsFiles as $js) {
  $fullPath  = $jsPath . $js;
  $jsFiles[] = $fullPath;
}

$minified = AIOM::JS($jsFiles);

echo "<script src='{$minified}'></script>";

// Output 
// <script src='/processwire/site/assets/aiom/js_d41d8cd98f00b204e9800998ecf8427e_dev.js?no-cache=1399617391'></script>

I also tried to output each javascript file one by one using AIOM, but this did not work, either: 

  foreach ($page->jsFiles as $file) {
    $fullPath = $jsPath . $file;
    $minJS = AIOM::JS($fullPath);
    echo "<script src='{$minJS}'></script>\n";
  }

Output:
// <script src='/processwire/site/assets/aiom/js_d41d8cd98f00b204e9800998ecf8427e_dev.js?no-cache=1399617391'></script>
// <script src='/processwire/site/assets/aiom/js_d41d8cd98f00b204e9800998ecf8427e_dev.js?no-cache=1399617391'></script>
// <script src='/processwire/site/assets/aiom/js_d41d8cd98f00b204e9800998ecf8427e_dev.js?no-cache=1399617391'></script>
// <script src='/processwire/site/assets/aiom/js_d41d8cd98f00b204e9800998ecf8427e_dev.js?no-cache=1399617391'></script>
// <script src='/processwire/site/assets/aiom/js_d41d8cd98f00b204e9800998ecf8427e_dev.js?no-cache=1399617391'></script>
// <script src='/processwire/site/assets/aiom/js_d41d8cd98f00b204e9800998ecf8427e_dev.js?no-cache=1399617391'></script>
// <script src='/processwire/site/assets/aiom/js_d41d8cd98f00b204e9800998ecf8427e_dev.js?no-cache=1399617391'></script>

Interestingly, each time the same file is output.

I installed AIOM version 3.1.3 with Processwire 2.4

 
I am probably missing the obvious, but I haven't been able to spot it. 
 
Cheers, 
 
Stefan
Link to comment
Share on other sites

EDIT: Ah, you have it in the first line:

// _init.php
$jsPath = $config->urls->templates . 'styles/js/';

try it with changed this to:

// _init.php
$jsPath = $config->paths->templates . 'styles/js/';

-------------

I don't know if it needs the fullpath or only a url, but you are definietly provide the url and not the fullpath:

$fullPath = $jsPath . $file;   // = /proseeswire/site/assets/...

I would do a quick try with:

$_SERVER['DOCUMENT_ROOT'] . $jsPath . $file;

If this succeeds, I would change the "$_SERVER['DOCUMENT_ROOT'] . $jsPath" to some PW $config-var (e.g. $config->paths ...)

Edited by horst
Link to comment
Share on other sites

@horst

Thanks for your reply. 

Unfortunately, replacing $config->urls with $config->paths lead to the same result and at the same time got me 'Page not found' errors for all script files added the normal way:

// _init.php
$jsPath = $config->urls->templates . 'styles/js/';

// _init.php
$page->jsFiles = array(
    'jquery.min.js',
    'bootstrap.min.js',
    'jquery.fancybox.pack.js',
    'jquery.isotope.min.js',
    'jquery.sticky.js',
    'retina.js',
    'scripts.js',
);

// _done.php
foreach ($page->jsFiles as $file) {
    $fullPath = $jsPath . $file;
    echo "<script src='{$fullPath}'></script>\n";
}

// Output:
// <script src='/home/sovonex/Programs/rubystack-2.0.0-12/apps/processwire/htdocs/site/templates/styles/js/jquery.min.js'></script>
// <script src='/home/sovonex/Programs/rubystack-2.0.0-12/apps/processwire/htdocs/site/templates/styles/js/bootstrap.min.js'></script>
// <script src='/home/sovonex/Programs/rubystack-2.0.0-12/apps/processwire/htdocs/site/templates/styles/js/jquery.fancybox.pack.js'>//

// </script>
// <script src='/home/sovonex/Programs/rubystack-2.0.0-12/apps/processwire/htdocs/site/templates/styles/js/jquery.isotope.min.js'></script>
// <script src='/home/sovonex/Programs/rubystack-2.0.0-12/apps/processwire/htdocs/site/templates/styles/js/jquery.sticky.js'></script>
// <script src='/home/sovonex/Programs/rubystack-2.0.0-12/apps/processwire/htdocs/site/templates/styles/js/retina.js'></script>
// <script src='/home/sovonex/Programs/rubystack-2.0.0-12/apps/processwire/htdocs/site/templates/styles/js/codeslider.js'></script>
// <script src='/home/sovonex/Programs/rubystack-2.0.0-12/apps/processwire/htdocs/site/templates/styles/js/scripts.js'></script>
 
I installed Processwire via the Bitnami stack, that's where the long path names are coming from. 
 
To be fair, I am not quite sure, when to use $config->paths and when to use $config->urls most of the time.
 
As for CSS and javascript files, I chose $config->urls, because I saw it done this way on the skyscraper installation:
 
Cheers, 
 
Stefan
 
 
 
Link to comment
Share on other sites

@Stefan: unfortunately I don't know the AIOM well. It was just a thought.

I would think one mostly need the $config->urls->something if one have to provide a link to the clients browser

and need the $config->paths->something mostly to tell a php-script / function where to grap the file(s) in filesystem for preprocessing something.

Link to comment
Share on other sites

The instructions for this module say to use the path relative to your templates directory, eg:

styles/mystyles.css or scripts/myscript.js

There is a setting in the module config:

"Allow Directory Traversal: Enable the directory traversal option to make it possible to add files from outside of the template folders. (../)"
 
which you can use if you need to access css and js files above the templates directory, but otherwise they should all be in subfolders of "templates" and linked to with a path relative to templates.
 
Make sense?
 
EDIT: not relevant to this module, but path vs url - depends on what is being done with the file being linked to. If it's a css or js file, typically you are going to want the url option. The path option is the full server path to the file which is useful for php operations on files, but no good for front-end display/calling of files. 
  • Like 4
Link to comment
Share on other sites

@adrian

Using the relative path from the template folder did the trick and so the following code is working:

 
// _init.php
// Relative path from the templates folder to the javascript folder
$relativeJSPath = 'styles/js/';

// _init.php
$page->jsFiles = array(
    'jquery.min.js',
    'bootstrap.min.js',
    'jquery.fancybox.pack.js',
    'jquery.isotope.min.js',
    'jquery.sticky.js',
    'retina.js',
    'scripts.js',
);

$jsFiles = array();
foreach ($page->jsFiles as $js) {
  $fullPath  = $relativeJSPath . $js;
  $jsFiles[] = $fullPath;
}

$minified = AIOM::JS($jsFiles);

echo "<script src='{$minified}'></script>";
 
However, now the code in scripts.js, where I initialize all plugins, does not work anymore. Only when I copy its content inside script tags below the javascript file produced by AIOM does it work again. This is regardless of whether the javascript files have been minified or just been merged into a single file (developer mode). 
 
Does anyone have any idea what might have happened here?
 
Cheers, 
 
Stefan
Link to comment
Share on other sites

However, now the code in scripts.js, where I initialize all plugins, does not work anymore. Only when I copy its content inside script tags below the javascript file produced by AIOM does it work again. This is regardless of whether the javascript files have been minified or just been merged into a single file (developer mode). 

 
Does anyone have any idea what might have happened here?
 
Cheers, 
 
Stefan

The generated code can be incorrect if your code is not written cleanly. A missing semicolon can already generate faulty code. Please inspect your script, for example, with jslint.com.

  • Like 1
Link to comment
Share on other sites

  • 2 weeks later...

Hi David,

sorry for my late response. I missed the notification email and thought there was no new reply.

You suggestion was spot on! The problem was in fact a syntax error in my Javascript code. Once I fixed that, AIOM worked again.

Today I have one other question:

When using AIOM, the paths to background images, icons, etc. in my CSS files somehow get changed and cannot be found anymore.

Here is an example

CSS code

.some-class {
    background: url(images/sprite.png);
}

This path works

// _init.php
$cssPath = $config->urls->templates . 'styles/stylesheets/';

// _done.php
<link href="<?= $cssPath . 'main.css' ?>" rel="stylesheet">

Path: background: url(h

                                      ttp://localhost:8080/processwire/site/templates/styles/stylesheets/images/sprite.png.); 

This path does not work

<link href="<?= AIOM::CSS('styles/stylesheets/main.css') ?>" rel="stylesheet">

Path: background: url(re/htdocs/site/templates/styles/stylesheets/images/sprite.png);

Do you have any any idea what is happening here?

Cheers,

Stefan

Edited by bytesource
Link to comment
Share on other sites

  • 2 weeks later...

CSS code

.some-class {
    background: url(images/sprite.png);
}

This path works

// _init.php
$cssPath = $config->urls->templates . 'styles/stylesheets/';

// _done.php
<link href="<?= $cssPath . 'main.css' ?>" rel="stylesheet">

Path: background: url(http://localhost:8080/processwire/site/templates/styles/stylesheets/images/sprite.png.); 

How can the path be different here at once if you have only "images/sprite.png" in CSS? There is no URL rewriting or parsing with direct linked CSS.

I wanted to bring up this question again as I haven't found a solution yet.

Does anyone have any idea of why the paths inside a CSS file are not translated correctly?

Stefan, I'm sorry. I can not reproduce your problem. For me it works with your folder structure. 
Link to comment
Share on other sites

How can the path be different here at once if you have only "images/sprite.png" in CSS?

That is exactly what I don't understand, either. 

Maybe I should add that the file main.css is the result of using the SASS compass plugin stitching together several SASS (SCSS) files. But then again, without AIOM everything works fine, so I don't think using a CSS preprocessor is what causes the problem here. 

I guess I just have to keep looking...

Cheers, 

Stefan

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...