Jump to content
David Karich

Module: AIOM+ (All In One Minify) for CSS, LESS, JS and HTML

Recommended Posts

Hi David, I'll try out that change you mentioned, and I appreciate the whitelist feature - thanks!

I'm trying to imagine a scenario where including a javascript/css/less file outside of the templates directory poses a security risk. How could that be exploited?

Hi Jonathan , 

yes, in this function paths are filtered to prevent directory traversal attacks. AIOM+ loads only allowed files that are located in the template folder. I think about it, in one of the upcoming versions, introduce a whitelist for asset folder.

For a workaround change the following line (749) in AllInOneMinify.module

$_path  = str_ireplace(array('../', './', '%2e%2e%2f', '..%2F'), '', (wire('config')->paths->templates.$_file));

to 

$_path  = wire('config')->paths->templates.$_file; 

I have not tested it and I can not recommend it too.

Share this post


Link to post
Share on other sites

Hi David, I'll try out that change you mentioned, and I appreciate the whitelist feature - thanks!

I'm trying to imagine a scenario where including a javascript/css/less file outside of the templates directory poses a security risk. How could that be exploited?

The easiest way is by infected third-party applications. Also on Bower or Composer, its possible, that compromised packages can be delivered. I have currently identified no scenario targeted at Process Wire, rather it is one of my personal programming rules. Just do not make it possible. Asset files belong in the template folder. This approach I have always in development, regardless of platform or environment. I have some clients who work with sensitive data, and the safety requirements are very high. Therefore, this is my natural data schizophrenia. :D

  • Like 3

Share this post


Link to post
Share on other sites

I have the new version of AIOM+ uploaded (3.1.2). From this version it is now possible to disable the directory traversal filter. Also, the LESS parser was updated. Now compatible with the official LESS version 1.7.

post-1973-0-14216100-1394147982_thumb.pn

  • Like 4

Share this post


Link to post
Share on other sites

Would it be possible to provide an alternate less compiler for those with node available on the server? 

I'm not really familiar with Node.js and I have no test environment. You're welcome to implement an alternative and send a pull request on GitHub. I think it requires only an option in the backend more and instead of PHP-based LESS parser an exec-command-line function call.

The question is rather, the detour via PHP is really faster? Because you can embed the LESS files directly into the template and can render there via LESS.js or over the Node.js package. The Node.js package has a minimization option on board.

  • Like 1

Share this post


Link to post
Share on other sites

I'm not really familiar with Node.js and I have no test environment. You're welcome to implement an alternative and send a pull request on GitHub. I think it requires only an option in the backend more and instead of PHP-based LESS parser an exec-command-line function call.

The question is rather, the detour via PHP is really faster? Because you can embed the LESS files directly into the template and can render there via LESS.js or over the Node.js package. The Node.js package has a minimization option on board.

Compiling all of bootstrap with the php implementation takes 3+ seconds which is an annoyance when developing themes, with the node.js implementation it's half a second. I agree that using less in the template being a great option. I think I'll set things up that way. Thanks David

Share this post


Link to post
Share on other sites
Hey guys, 

i have released a new version of AIOM+. The update to version 3.1.3 includes a few improvements in the LESS parser and CSS minification. Also, I have set the status from AIOM+ to stable.

  • Like 4

Share this post


Link to post
Share on other sites

Hi David,

Thanks again for a great module. Just wondering if you did any experimenting with other js minification libraries? I am no expert on this, but noticed that I can get significantly smaller files (in one case 400kb down to 300kb) by minifying using http://jscompress.com/ I know that runs on nodejs, but there are other php alternatives out there, although maybe none of them are any better.

Maybe you've already been through this process and decided JSMin is the best option, but just thought I'd ask :)

Share this post


Link to post
Share on other sites

Hi David,

Thanks again for a great module. Just wondering if you did any experimenting with other js minification libraries? I am no expert on this, but noticed that I can get significantly smaller files (in one case 400kb down to 300kb) by minifying using http://jscompress.com/ I know that runs on nodejs, but there are other php alternatives out there, although maybe none of them are any better.

Maybe you've already been through this process and decided JSMin is the best option, but just thought I'd ask :)

Hi adrian,

for sure. JSMin is a little bit outdated and generates not the best compression result, but it is the only stable php port without any conditions to third party libraries. Some other minifications libraries destroyed the compressed js code and JSMin is the only one, with which I never had such a problem. Anyway, i'm thinking about, to integret the google closure compiler as an alternative third party service.

Share this post


Link to post
Share on other sites

Thanks for the explanation. I am sure you've seen it, but this looks promising:

https://github.com/tedivm/JShrink

Anyway, thanks again for this module - it's a real time saver.

On an unrelated note - I did notice something yesterday when using the loadOn option. It worked fine for CSS, but it didn't seem to work properly with JS. Before you go delving too deep, I should do some more testing and see what I can find - might have been an issue at my end. I'll try again soon and let you know how it goes.

Share this post


Link to post
Share on other sites

Thanks for the explanation. I am sure you've seen it, but this looks promising:

https://github.com/tedivm/JShrink

Anyway, thanks again for this module - it's a real time saver.

On an unrelated note - I did notice something yesterday when using the loadOn option. It worked fine for CSS, but it didn't seem to work properly with JS. Before you go delving too deep, I should do some more testing and see what I can find - might have been an issue at my end. I'll try again soon and let you know how it goes.

Oh, no. This library has not yet come under my eyes. I'll be looking at this a bit more detail when I have more time again. Thanks for the tip.

Yes, check again and let me know, because it is the identical function that is called. If there should be a bug, CSS should not work as well. If you were able to find something, then just open a ticket on Github. :)

Share this post


Link to post
Share on other sites

Author of JShrink here (I found you guys in my web logs). I thought I'd pop in and say hello, and talk about JShrink a bit. 

I originally made it as a drop in replacement for JSMin, because JSMin is technically not open source (it has the "do no evil" license, so projects that take open source seriously, such as Debian, won't bundle projects with it). Since then I've made improvements on speed, added additional features like support for conditional comments and license preservation, and have really  put work into the test suite itself. Ryan Grove, who originally ported JSMin to PHP, has officially dropped the JSMin project and is referring people to JShrink on the Github page for it.

Yesterday, after several years of being in the "beta" phase, I launched the JShrink v1.0 to make the API official and stable. 

On a tangental note, I also maintain a caching library called Stash that may be useful to you.

  • Like 5

Share this post


Link to post
Share on other sites

AIOM minifies my CSS file without any problems, but for some reason I have not been able to minify any javascript files. 

While outputing javascript the old way works:

// _init.php
$jsPath = $config->urls->templates . 'styles/js/';

// _init.php
$page->jsFiles = array(
    'jquery.min.js',
    'bootstrap.min.js',
    'jquery.fancybox.pack.js',
    'jquery.isotope.min.js',
    'jquery.sticky.js',
    'retina.js',
    'scripts.js',
);

// _done.php
foreach ($page->jsFiles as $file) {
    $fullPath = $jsPath . $file;
    echo "<script src='{$fullPath}'></script>\n";

}

// Output
// <script src='/processwire/site/templates/styles/js/jquery.min.js'></script>
// <script src='/processwire/site/templates/styles/js/bootstrap.min.js'></script>
// <script src='/processwire/site/templates/styles/js/jquery.fancybox.pack.js'></script>
// <script src='/processwire/site/templates/styles/js/jquery.isotope.min.js'></script>
// <script src='/processwire/site/templates/styles/js/jquery.sticky.js'></script>
// <script src='/processwire/site/templates/styles/js/retina.js'></script>
// <script src='/processwire/site/templates/styles/js/scripts.js'></script>
 
Using AIOM as shown below always resulted in a file with the content of
/** Generated: Thursday, 8th of May 2014, 11:36:39 PM // Powered by AIOM+ (All In One Minify) by www.flipzoom.de **/:
 
// _done.php
$jsFiles = array();

foreach ($page->jsFiles as $js) {
  $fullPath  = $jsPath . $js;
  $jsFiles[] = $fullPath;
}

$minified = AIOM::JS($jsFiles);

echo "<script src='{$minified}'></script>";

// Output 
// <script src='/processwire/site/assets/aiom/js_d41d8cd98f00b204e9800998ecf8427e_dev.js?no-cache=1399617391'></script>

I also tried to output each javascript file one by one using AIOM, but this did not work, either: 

  foreach ($page->jsFiles as $file) {
    $fullPath = $jsPath . $file;
    $minJS = AIOM::JS($fullPath);
    echo "<script src='{$minJS}'></script>\n";
  }

Output:
// <script src='/processwire/site/assets/aiom/js_d41d8cd98f00b204e9800998ecf8427e_dev.js?no-cache=1399617391'></script>
// <script src='/processwire/site/assets/aiom/js_d41d8cd98f00b204e9800998ecf8427e_dev.js?no-cache=1399617391'></script>
// <script src='/processwire/site/assets/aiom/js_d41d8cd98f00b204e9800998ecf8427e_dev.js?no-cache=1399617391'></script>
// <script src='/processwire/site/assets/aiom/js_d41d8cd98f00b204e9800998ecf8427e_dev.js?no-cache=1399617391'></script>
// <script src='/processwire/site/assets/aiom/js_d41d8cd98f00b204e9800998ecf8427e_dev.js?no-cache=1399617391'></script>
// <script src='/processwire/site/assets/aiom/js_d41d8cd98f00b204e9800998ecf8427e_dev.js?no-cache=1399617391'></script>
// <script src='/processwire/site/assets/aiom/js_d41d8cd98f00b204e9800998ecf8427e_dev.js?no-cache=1399617391'></script>

Interestingly, each time the same file is output.

I installed AIOM version 3.1.3 with Processwire 2.4

 
I am probably missing the obvious, but I haven't been able to spot it. 
 
Cheers, 
 
Stefan

Share this post


Link to post
Share on other sites

EDIT: Ah, you have it in the first line:

// _init.php
$jsPath = $config->urls->templates . 'styles/js/';

try it with changed this to:

// _init.php
$jsPath = $config->paths->templates . 'styles/js/';

-------------

I don't know if it needs the fullpath or only a url, but you are definietly provide the url and not the fullpath:

$fullPath = $jsPath . $file;   // = /proseeswire/site/assets/...

I would do a quick try with:

$_SERVER['DOCUMENT_ROOT'] . $jsPath . $file;

If this succeeds, I would change the "$_SERVER['DOCUMENT_ROOT'] . $jsPath" to some PW $config-var (e.g. $config->paths ...)

Edited by horst

Share this post


Link to post
Share on other sites

@horst

Thanks for your reply. 

Unfortunately, replacing $config->urls with $config->paths lead to the same result and at the same time got me 'Page not found' errors for all script files added the normal way:

// _init.php
$jsPath = $config->urls->templates . 'styles/js/';

// _init.php
$page->jsFiles = array(
    'jquery.min.js',
    'bootstrap.min.js',
    'jquery.fancybox.pack.js',
    'jquery.isotope.min.js',
    'jquery.sticky.js',
    'retina.js',
    'scripts.js',
);

// _done.php
foreach ($page->jsFiles as $file) {
    $fullPath = $jsPath . $file;
    echo "<script src='{$fullPath}'></script>\n";
}

// Output:
// <script src='/home/sovonex/Programs/rubystack-2.0.0-12/apps/processwire/htdocs/site/templates/styles/js/jquery.min.js'></script>
// <script src='/home/sovonex/Programs/rubystack-2.0.0-12/apps/processwire/htdocs/site/templates/styles/js/bootstrap.min.js'></script>
// <script src='/home/sovonex/Programs/rubystack-2.0.0-12/apps/processwire/htdocs/site/templates/styles/js/jquery.fancybox.pack.js'>//

// </script>
// <script src='/home/sovonex/Programs/rubystack-2.0.0-12/apps/processwire/htdocs/site/templates/styles/js/jquery.isotope.min.js'></script>
// <script src='/home/sovonex/Programs/rubystack-2.0.0-12/apps/processwire/htdocs/site/templates/styles/js/jquery.sticky.js'></script>
// <script src='/home/sovonex/Programs/rubystack-2.0.0-12/apps/processwire/htdocs/site/templates/styles/js/retina.js'></script>
// <script src='/home/sovonex/Programs/rubystack-2.0.0-12/apps/processwire/htdocs/site/templates/styles/js/codeslider.js'></script>
// <script src='/home/sovonex/Programs/rubystack-2.0.0-12/apps/processwire/htdocs/site/templates/styles/js/scripts.js'></script>
 
I installed Processwire via the Bitnami stack, that's where the long path names are coming from. 
 
To be fair, I am not quite sure, when to use $config->paths and when to use $config->urls most of the time.
 
As for CSS and javascript files, I chose $config->urls, because I saw it done this way on the skyscraper installation:
 
Cheers, 
 
Stefan
 
 
 

Share this post


Link to post
Share on other sites

@Stefan: unfortunately I don't know the AIOM well. It was just a thought.

I would think one mostly need the $config->urls->something if one have to provide a link to the clients browser

and need the $config->paths->something mostly to tell a php-script / function where to grap the file(s) in filesystem for preprocessing something.

Share this post


Link to post
Share on other sites

The instructions for this module say to use the path relative to your templates directory, eg:

styles/mystyles.css or scripts/myscript.js

There is a setting in the module config:

"Allow Directory Traversal: Enable the directory traversal option to make it possible to add files from outside of the template folders. (../)"
 
which you can use if you need to access css and js files above the templates directory, but otherwise they should all be in subfolders of "templates" and linked to with a path relative to templates.
 
Make sense?
 
EDIT: not relevant to this module, but path vs url - depends on what is being done with the file being linked to. If it's a css or js file, typically you are going to want the url option. The path option is the full server path to the file which is useful for php operations on files, but no good for front-end display/calling of files. 
  • Like 4

Share this post


Link to post
Share on other sites

@adrian

Using the relative path from the template folder did the trick and so the following code is working:

 
// _init.php
// Relative path from the templates folder to the javascript folder
$relativeJSPath = 'styles/js/';

// _init.php
$page->jsFiles = array(
    'jquery.min.js',
    'bootstrap.min.js',
    'jquery.fancybox.pack.js',
    'jquery.isotope.min.js',
    'jquery.sticky.js',
    'retina.js',
    'scripts.js',
);

$jsFiles = array();
foreach ($page->jsFiles as $js) {
  $fullPath  = $relativeJSPath . $js;
  $jsFiles[] = $fullPath;
}

$minified = AIOM::JS($jsFiles);

echo "<script src='{$minified}'></script>";
 
However, now the code in scripts.js, where I initialize all plugins, does not work anymore. Only when I copy its content inside script tags below the javascript file produced by AIOM does it work again. This is regardless of whether the javascript files have been minified or just been merged into a single file (developer mode). 
 
Does anyone have any idea what might have happened here?
 
Cheers, 
 
Stefan

Share this post


Link to post
Share on other sites

However, now the code in scripts.js, where I initialize all plugins, does not work anymore. Only when I copy its content inside script tags below the javascript file produced by AIOM does it work again. This is regardless of whether the javascript files have been minified or just been merged into a single file (developer mode). 

 
Does anyone have any idea what might have happened here?
 
Cheers, 
 
Stefan

The generated code can be incorrect if your code is not written cleanly. A missing semicolon can already generate faulty code. Please inspect your script, for example, with jslint.com.

  • Like 1

Share this post


Link to post
Share on other sites

Hi David,

sorry for my late response. I missed the notification email and thought there was no new reply.

You suggestion was spot on! The problem was in fact a syntax error in my Javascript code. Once I fixed that, AIOM worked again.

Today I have one other question:

When using AIOM, the paths to background images, icons, etc. in my CSS files somehow get changed and cannot be found anymore.

Here is an example

CSS code

.some-class {
    background: url(images/sprite.png);
}

This path works

// _init.php
$cssPath = $config->urls->templates . 'styles/stylesheets/';

// _done.php
<link href="<?= $cssPath . 'main.css' ?>" rel="stylesheet">

Path: background: url(h

                                      ttp://localhost:8080/processwire/site/templates/styles/stylesheets/images/sprite.png.); 

This path does not work

<link href="<?= AIOM::CSS('styles/stylesheets/main.css') ?>" rel="stylesheet">

Path: background: url(re/htdocs/site/templates/styles/stylesheets/images/sprite.png);

Do you have any any idea what is happening here?

Cheers,

Stefan

Edited by bytesource

Share this post


Link to post
Share on other sites

Sorry, I copied the wrong code above, which probably made my question completely incomprehensible. 

I edited my previous comment and hope that now everything is correct. 

Share this post


Link to post
Share on other sites

I wanted to bring up this question again as I haven't found a solution yet.

Does anyone have any idea of why the paths inside a CSS file are not translated correctly?

Share this post


Link to post
Share on other sites

CSS code

.some-class {
    background: url(images/sprite.png);
}

This path works

// _init.php
$cssPath = $config->urls->templates . 'styles/stylesheets/';

// _done.php
<link href="<?= $cssPath . 'main.css' ?>" rel="stylesheet">

Path: background: url(http://localhost:8080/processwire/site/templates/styles/stylesheets/images/sprite.png.); 

How can the path be different here at once if you have only "images/sprite.png" in CSS? There is no URL rewriting or parsing with direct linked CSS.

I wanted to bring up this question again as I haven't found a solution yet.

Does anyone have any idea of why the paths inside a CSS file are not translated correctly?

Stefan, I'm sorry. I can not reproduce your problem. For me it works with your folder structure. 

Share this post


Link to post
Share on other sites
How can the path be different here at once if you have only "images/sprite.png" in CSS?

That is exactly what I don't understand, either. 

Maybe I should add that the file main.css is the result of using the SASS compass plugin stitching together several SASS (SCSS) files. But then again, without AIOM everything works fine, so I don't think using a CSS preprocessor is what causes the problem here. 

I guess I just have to keep looking...

Cheers, 

Stefan

Share this post


Link to post
Share on other sites

Hello, is there any way for AIOM to accept a FilenameArray object for minifying or do I need to convert the object to an array manually first?

  • Like 2

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By MoritzLost
      This is a new module that provides a simple solution to clearing all your cache layers at once, and an extensible interface to perform various cache-related actions.
      The simple motivation behind this module was that I was tired of manually clearing caches in several places after deploying a change on a live site. The basic purpose of this module is a simple Clear all caches link in the Setup menu which clears out all caches, no matter where they hide. You can customize what exactly the module does through it's configuration menu:
      Expire or delete all cache entries in the database, or selectively clear caches by namespace ($cache API) Clear the the template render cache. Clear out specific folders inside your site's cache directory (/site/assets/cache) Refresh version strings for static assets to bust client-side browser caches (this requires some setup, see the full documentation for details). This is the basic function of the module. However, you can also add different cache management action through the API and execute them through the module's interface. For this advanced usage, the module provides:
      An interface to see all available cache actions and execute them. A system log and logging output on the module page to see verify what the module is doing. A CacheControlTools class with utility functions to clear out different caches. An API to add cache actions, execute them programmatically and even modify the default action. Permission management, allowing you granular control over which user roles can execute which actions. The complete documentation can be found in the module's README.
      Beta release
      Note that I consider this a Beta release. Since the module is relatively aggressive in deleting some caches, I would advise you to install in on a test environment before using it on a live site.
      Let me know if you're getting any errors, have trouble using the module or if you have suggestions for improvement!
      In particular, can someone let me know if this module causes any problems with the ProCache module? I don't own or use it, so I can't check. As far as I can tell, ProCache uses a folder inside the cache directory to cache static pages, so my module should be able to clear the ProCache site cache as well, I'd appreciate it if someone can test that for me.
      Future plans
      If there is some interest in this, I plan to expand this to a more general cache management solution. I particular, I would like to add additional cache actions. Some ideas that came to mind:
      Warming up the template render cache for publicly accessible pages. Removing all active user sessions. Let me know if you have more suggestions!
      Links
      https://github.com/MoritzLost/ProcessCacheControl ProcessCacheControl in the Module directory

    • By joshua
      This module is (yet another) way for implementing a cookie management solution.
      Of course there are several other possibilities:
      - https://processwire.com/talk/topic/22920-klaro-cookie-consent-manager/
      - https://github.com/webmanufaktur/CookieManagementBanner
      - https://github.com/johannesdachsel/cookiemonster
      - https://www.oiljs.org/
      - ... and so on ...
      In this module you can configure which kind of cookie categories you want to manage:

      You can also enable the support for respecting the Do-Not-Track (DNT) header to don't annoy users, who already decided for all their browsing experience.
      Currently there are four possible cookie groups:
      - Necessary (always enabled)
      - Statistics
      - Marketing
      - External Media
      All groups can be renamed, so feel free to use other cookie group names. I just haven't found a way to implement a "repeater like" field as configurable module field ...
      When you want to load specific scripts ( like Google Analytics, Google Maps, ...) only after the user's content to this specific category of cookies, just use the following script syntax:
      <script type="optin" data-type="text/javascript" data-category="statistics" data-src="/path/to/your/statistic/script.js"></script> <script type="optin" data-type="text/javascript" data-category="marketing" data-src="/path/to/your/mareketing/script.js"></script> <script type="optin" data-type="text/javascript" data-category="external_media" data-src="/path/to/your/external-media/script.js"></script> <script type="optin" data-type="text/javascript" data-category="marketing">console.log("Inline scripts are also working!");</script> The type has to be "optin" to get recognized by PrivacyWire, the data-attributes are giving hints, how the script shall be loaded, if the data-category is within the cookie consents of the user. These scripts are loaded asynchronously after the user made the decision.
      If you want to give the users the possibility to change their consent, you can use the following Textformatter:
      [[privacywire-choose-cookies]] It's planned to add also other Textformatters to opt-out of specific cookie groups or delete the whole consent cookie.
      You can also add a custom link to output the banner again with a link / button with following class:
      <a href="#" class="privacywire-show-options">Show Cookie Options</a> <button class="privacywire-show-options">Show Cookie Options</button> This module is still in development, but we already use it on several production websites.
      You find it here: https://github.com/blaueQuelle/privacywire/tree/master
      Download: https://github.com/blaueQuelle/privacywire/archive/master.zip
      I would love to hear your feedback 🙂
      Edit: Updated URLs to master tree of git repo
       
    • By David Karich
      Admin Page Tree Multiple Sorting
      ClassName: ProcessPageListMultipleSorting
      Extend the ordinary sort of children of a template in the admin page tree with multiple properties. For each template, you can define your own rule. Write each template (template-name) in a row, followed by a colon and then the additional field names for sorting.
      Example: All children of the template "blog" to be sorted in descending order according to the date of creation, then descending by modification date, and then by title. Type:
      blog: -created, -modified, title  Installation
      Copy the files for this module to /site/modules/ProcessPageListMultipleSorting/ In admin: Modules > Check for new modules. Install Module "Admin Page Tree Multible Sorting". Alternative in ProcessWire 2.4+
      Login to ProcessWire backend and go to Modules Click tab "New" and enter Module Class Name: "ProcessPageListMultipleSorting" Click "Download and Install"   Compatibility   I have currently tested the module only under PW 2.6+, but think that it works on older versions too. Maybe someone can give a feedback.     Download   PW-Repo: http://modules.processwire.com/modules/process-page-list-multiple-sorting/ GitHub: https://github.com/FlipZoomMedia/Processwire-ProcessPageListMultipleSorting     I hope someone can use the module. Have fun and best regards, David
    • By dimitrios
      Hello,
      this module can publish content of a Processwire page on a Facebook page, triggered by saving the Processwire page.
      To set it up, configure the module with a Facebook app ID, secret and a Page ID. Following is additional configuration on Facebook for developers:
      Minimum Required Facebook App configuration:
      on Settings -> Basics, provide the App Domains, provide the Site URL, on Settings -> Advanced, set the API version (has been tested up to v3.3), add Product: Facebook Login, on Facebook Login -> Settings, set Client OAuth Login: Yes, set Web OAuth Login: Yes, set Enforce HTTPS: Yes, add "https://www.example.com/processwire/page/" to field Valid OAuth Redirect URIs. This module is configurable as follows:
      Templates: posts can take place only for pages with the defined templates. On/Off switch: specify a checkbox field that will not allow the post if checked. Specify a message and/or an image for the post.
      Usage
      edit the desired PW page and save; it will post right after the initial Facebook log in and permission granting. After that, an access token is kept.
       
      Download
      PW module directory: http://modules.processwire.com/modules/auto-fb-post/ Github: https://github.com/kastrind/AutoFbPost   Note: Facebook SDK for PHP is utilized.


×
×
  • Create New...