Module: AIOM+ (All In One Minify) for CSS, LESS, JS and HTML

[ndolph]

Calling AIOM/chmod experts...

Just today a whole website of mine using AIOM ceased to function.

Precicely 1 month after installing AIOM on the site, I was getting a 500 error. Looking at the PW log file I saw countless repetitions of...

2014-10-18 15:59:18	guest	http://***.com/?/	Error: 	Exception: The permissions (chmod) to delete old cache files could not be changed. (in /srv/***/www/***.com/public_html/site/modules/AllInOneMinify/AllInOneMinify.module line 646)

So, I guess one or more of my cached files had hit a 30 day expiry date.

I did a bit of investigating, and it seems that before deleting cache files, AIOM like to chmod 777 the file. If it can't chmod the file, it throws this error and stops the site loading. No logging into PW admin, nothing.

It seems similar to this problem here:

https://github.com/ryancramerdesign/ProcessWire/issues/648

And the exact line of code in question is highlighted by Ryan there. https://github.com/FlipZoomMedia/ProcessWire-AIOM-All-In-One-Minify/blob/AIOM%2B/AllInOneMinify.module#L641... It's line 646.

Is there any real need to change the permissions to 777 before deleting?

The problem in my case is that www-data doesn't own these files. I copied this installation over from another server, and the owner (chown) is actuall "admin". But I have group rw permission to www-data, so the files can be deleted. I think that should be okay, right?

So, what I did was comment line 646 and added the following three lines

        // ------------------------------------------------------------------------
        // Remove all files that are older than the maximum lifetime.
        // ------------------------------------------------------------------------
        foreach ($_cacheFiles as $_cacheFile) {
            if (((filemtime(wire('config')->paths->assets.self::$cacheDir.DIRECTORY_SEPARATOR.$_cacheFile->getFilename()) + self::$cacheMaxLifetime) < time() OR $force_all === true) AND is_file($_cacheFile)) {
                $_file  = wire('config')->paths->assets.self::$cacheDir.DIRECTORY_SEPARATOR.$_cacheFile->getFilename();
                if(chmod($_file, 0777) !== false) {
                    if(unlink($_file) === false) {
                        throw new WireException('The old cache files could not be deleted.');
                    }
                } else {
                    // throw new WireException('The permissions (chmod) to delete old cache files could not be changed.');
                    if(unlink($_file) === false) {
                        throw new WireException('The old cache files could not be deleted.');
                    }                    
                }
            }
        }

And now everything seems to work again.

Stuff like this sort of scares me. The last thing I want is for sites to completely and arbitrarily stop working!

I literally just had this error, and had to apply the fix mentioned. This was never fixed?

[David Karich]

I literally just had this error, and had to apply the fix mentioned. This was never fixed?

Please update AIOM to the latest version 3.2.3. This problem has been fixed with the chmod and uses ProcessWire-own functions for it. This should not happen anymore.

[PWaddict]

Hello,

I'm using an HTTP/2 server so the recommended way is to have separate CSS/JS without merging them but when I try for example to minimize 5 files, at least 1 file is getting duplicated. This is issue doesn't happen when I merge them.

Can you please fix it?

Thanks

[David Karich]

Hello,

I'm using an HTTP/2 server so the recommended way is to have separate CSS/JS without merging them but when I try for example to minimize 5 files, at least 1 file is getting duplicated. This is issue doesn't happen when I merge them.

Can you please fix it?

Thanks

Unfortunately I have no server with this configuration available to reproduce the problem. Are you sure that you do not have modules like Google Page Speed / SPDY activated? These do the minimization and compression automatically.

[blynx]

EDIT: Yay, everything is fine ... turns out, as you can see I forgot the ; at the end of the lines ... stylus habit;

Hi,

I was just trying getting some .less files compiled but something doesn't work with @import.

I have an index.less file which shall import several sub-stylessheets - and that index.less is the only one given to the ::CSS() function:

@import "variables.less"
@import "layout/index.less"
@import "boxes/index.less"

but that just gives me:

#_____LESS_____ERROR_____REPORT_____ {content:"ParseError: Unexpected input in anonymous-file-0.less on line 7, column 1
05| @sth: #ffffff;
06| 
07| @import "/dev/site/templates/styles.less/variables"  // global variables
08| @import "/dev/site/templates/styles.less/layout/index.less"
09| @import "/dev/site/templates/styles.less/boxes/index.less"
10| "}

Am I doing something wrong or is this feature missing in AIOM?

[kixe]

I would like to implement your module in my template engine. Unfortunately it doesn't allow paths outside of the template folder. Would be a big enhancement to accept and find the following. I couldn't get it work.

$styles = array();
// relative to the root paths like
$styles[] = '/site/modules/whatever/style.css';
$styles[] = '/wire/templates-admin/styles/font-awesome/css/font-awesome.min.css'; // I use this very often

// absolute paths
$styles[] = 'http://example.org/stylesheet.css';

$urls = AIOM::CSS($styles);
$output = "<link rel=\"stylesheet\" href=\"$urls\">\n";

Herewith I put this on the wishlist!   Maybe somebody else would like this too

The Option 'Allow directory traversal' doesn't really exist. It expects each stylesheet path relative to the templates folder if I understand the following codelines correctly.

        // ------------------------------------------------------------------------
        // Filter Directory Traversal (default: yes)
        // ------------------------------------------------------------------------
        $_path  = (self::$directoryTraversal !== true) ? str_ireplace(array('../', './', '%2e%2e%2f', '..%2F'), '', wire('config')->paths->templates.$_file) : wire('config')->paths->templates.$_file;

Maybe i missed something and you could help. Anyway a great module you made. Thanks for this.
 

Edited March 29, 2016 by kixe

[Can]

Just tried it. Just enable directory traversal in AIOM's module config and then use this path

$styles[] = '../../wire/templates-admin/styles/font-awesome/css/font-awesome.min.css';

[kixe]

I forked the module on github which allows the following paths for example:
styles/style.css (relative to template - default)
/site/folder/style.css (relative to root - NEW)
https://ajax.googleapis.com/ajax/libs/jquery/2.1.4/jquery.min.js (absolute path to external source via http request -NEW)

https://github.com/kixe/ProcessWire-AIOM-All-In-One-Minify
 

[Can]

Sounds good thanks for sharing

[szabesz]

I forked the module on github which allows the following paths for example:

May I ask why it is a good idea to fork it, instead of including these features in the original module? Am I missing something perhaps? I'm not a GitHub guru, so I ask: doesn't it mean that these are two completely separate projects from now on?

[Can]

May I ask why it is a good idea to fork it, instead of including these features in the original module? Am I missing something perhaps? I'm not a GitHub guru, so I ask: doesn't it mean that these are two completely separate projects from now on?

I would say because kixe is not the author so he can't just implement those features in the official release, but if needed before the author can implement or pull a request he can just fork to make his own version 

[PWaddict]

Unfortunately I have no server with this configuration available to reproduce the problem. Are you sure that you do not have modules like Google Page Speed / SPDY activated? These do the minimization and compression automatically.

It has nothing to do with the HTTP2 server. Just some css / js files are getting the same filename.

[PWaddict]

Unfortunately I have no server with this configuration available to reproduce the problem. Are you sure that you do not have modules like Google Page Speed / SPDY activated? These do the minimization and compression automatically.

Should I replace the _getCacheName function with the one from here?

[kixe]

May I ask why it is a good idea to fork it, instead of including these features in the original module? Am I missing something perhaps? I'm not a GitHub guru, so I ask: doesn't it mean that these are two completely separate projects from now on?

From the PW modules directory you will get always the original version. If the Author like this feature, he has the freedom to include it. I had to make these changes, to get it working properly with my template function library and I want to share this enhancement. Use it if you like it.

[tpr]

May I ask why it is a good idea to fork it, instead of including these features in the original module?

To sum up: this is the standard (github) way of contributing to other's repo.

[szabesz]

From the PW modules directory you will get always the original version. If the Author like this feature, he has the freedom to include it. I had to make these changes, to get it working properly with my template function library and I want to share this enhancement. Use it if you like it.

Thank you for the answer! Is it also standard not to reflect this in README.md and just copy it over? I mean, how can one tell apart the various forked versions? Just by looking at the "main pages" of the projects, they seem identical to me. Sorry if it happens to be an off-topic GitHub question.

[tpr]

That would be a good practice, but maybe even better having a separate CHANGELOG.md file to log changes. You can also check commit messages to see what has changed, provided that the author included one.

[teppo]

Thank you for the answer! Is it also standard not to reflect this in README.md and just copy it over? I mean, how can one tell apart the various forked versions? Just by looking at the "main pages" of the projects, they seem identical to me. Sorry if it happens to be an off-topic GitHub question.

Check the name of the repository at GitHub. In this case there's this line right below it:

forked from FlipZoomMedia/ProcessWire-AIOM-All-In-One-Minify

That's a pretty good indication that this particular repository might be a fork

In the GitHub workflow if you fork another repository you can send a pull request containing all your changes to the original repository / author. If you've changed the README in the fork, those changes will also be included in the PR. That's one reason not to include indication about this being a fork or anything like that.

Of course you can get over this limitation by creating a separate branch for the changes intended for the PR, but that's extra work and most developers are just plain lazy

[szabesz]

Thanks for educating me, teppo! I have never used GitHub (apart from downloading ZIP files and simple browsing) nor noticed the "forked from" line. Sooner or later I need to find the time to familiarize myself with GitHub, I suppose. And again, sorry for the off-topic question.

[kixe]

@szabesz
Here you can find a list of the 250+ forks of ProcessWire itself.

https://github.com/ryancramerdesign/ProcessWire/network/members

[szabesz]

@kixe: Thanks! This can be quite useful sometimes. I have never noticed this list before.

[Robin S]

A feature request: Sass support (specifically SCSS). Thanks!

[Can]

How about adding media query grouping like http://stackoverflow.com/a/18191985/3484824?

I've test implemented it at line 621 or maybe more accurately right before the "Write the minimized file to the file system" comment

// search media query in CSS
preg_match_all('#@media(.*?)\{(.+?}[ \n])\}#si',$_css,$match,PREG_SET_ORDER);//$MatchingString now hold all strings matching $pattern.

$media = array();

// group same media query
foreach ($match as $val)
{
    if (!isset($media[$val[1]])) $media[$val[1]] = '';

    $media[$val[1]] .= $val[2];
}

// delete media query of cache
$_css = preg_replace('#@media(.*?)\{(.+?}[ \n])\}#si', '', $_css);

// add groups of media query at the end of CSS
$final = $_css . "\n";
foreach ($media as $id => $val)
{
    $final .= "\n" . '@media' . $id . '{' . $val . '}' . "\n";
}
$_css = $final;

// ------------------------------------------------------------------------
// Write the minimized file to the file system.
// ------------------------------------------------------------------------

One optimization could be sort ordering of grouped media queries like min-width40em before min-width50em and so on..

OR

# Recursively descend and match all `{ .. }` blocks.
preg_match_all('#(?<media>(?:@media[^{]+)?){(??:[^{}]+)|(?R))*}#s', $_css, $captures);
# Loop through all matches by key.
foreach (array_keys($captures[0]) as $key) {
  # If the value does not contain a @media query, drop it from the array. Or
  # even if we found a @media query, make sure it was not within a string or
  # a comment.
  if (strpos($captures[0][$key], '@media') === FALSE || empty ($captures['media'][$key])) {
    unset ($captures[0][$key]);
  }
}
# Erase all @media queries from the input CSS.
foreach ($captures[0] as $query) {
  $_css = preg_replace('#\s*' . preg_quote($query, '#') . '#u', '', $_css);
}
# Utility function to create an array key from a media query.
function sanitize_media_query_for_key($query) {
  $key = trim($query);
  $key = preg_replace('#[^\w\d]+#u', '-', $key);
  $key = trim($key, '-');
  return $key;
}
# For each @media query, group CSS code together.
$groups = array();
foreach (array_keys($captures[0]) as $key) {
  $media = $captures['media'][$key];
  # The code for this block starts off with the @media query in it...
  $code  = $captures[0][$key];
  # ...which then gets stripped off...
  $code  = preg_replace('#' . preg_quote($media, '#') . '#u', '', $code);
  # ...and any leading or trailing curly bracket also gets removed.
  $code  = preg_replace('#^{|}$#u', '', $code);
  # If this is the first time the @media query is encountered, a new group is
  # created.
  $group_key = sanitize_media_query_for_key($media);
  array_key_exists($group_key, $groups) OR ($groups[$group_key] = array($media, '{'));
  $groups[$group_key][] = $code;
}
$result = NULL;
# Output is the input stripped of any @media queries...
$result = $result . rtrim($_css);
# ...followed by those @media queries grouped together.
foreach (array_keys($groups) as $key) {
  $result = $result . PHP_EOL . PHP_EOL . implode(NULL, $groups[$key]) . '}';
}
$_css = $result;

from https://gist.github.com/StanAngeloff/3164569

but still sorting the result could be interesting(?) Couldn't figure it out myself so far..but it's late..

Edited June 6, 2016 by Can

[bernhard]

i've just created a pull request that makes it possible to modify less variables. it also adds a check if the js/css file exists and throws a note either in console if tracy is installed or in the processwire log.

that's how you can modify your less variables:

<?php
$config->styles->append('less/theme.less');
$lessVars = array(
    'tm-primary-bg' => '#568AEA',
    'background-image' => 'url("' . $page->backgroundimage->url . '")',
);
?>
<link rel="stylesheet" href="<?php echo AIOM::CSS($config->styles->unique(), $lessVars); ?>">

https://github.com/FlipZoomMedia/ProcessWire-AIOM-All-In-One-Minify/pull/57

[Stefanowitsch]

I've installed the module and it works fine.

But:

When I set Development Mode to "yes" the files get combined. Is there a way to disable this? I cannot track the root css files this way...