nik

I wasn't sure about superuser only. Because if you want to test not Logged in or with another user...

I will add the config true test.

This is what I was thinking too. And the database queries are not logged if debug=false as well.

I see you've decided not to output anything in the admin.. I guess that's because most of the data already is visible there? And while the module adds no hooks in the admin, it does make ChromePhp available for use - so nevermind, just thinking aloud here .

Sorry guys to get it done before you. We need some board to check in if somebody starts on a module so people can check and see if already something is in work. Although this took me 1 evening lol.

With small modules like this one there shouldn't be a problem. I think I even managed to learn something from the few hours I spent with this one. About PW itself and that I tend to overcomplicate things (had a couple of unnecessary config vars and a little abstraction layer to support other plugins as well later on). Sometimes (well, most of the time) it would be better to get something released and then keep on making it better - if there really is any need after all.

I'm trying to keep this in mind: "If you’re not embarrassed when you ship your first version you waited too long." (see ma.tt/2010/11/one-point-oh/ for the whole essay). A rule not to be followed by people implementing safety measures to nuclear power plants though.

But if I was to start building something bigger, it would be nice to know if someone else is already working on the same thing. That would of course be useful to us mere mortals with somewhat smaller modules too. Not all of us are able to get these kind of useful things from scratch to release during one evening like you are Soma .

Good job Soma!

I like the way you're thinking - I had a module with just slightly less functionality coming up, this week even . But now I don't need to find time to finish it as you've already done things bigger and better.

One thing I'd like to see is a little if-statement to disable this when $this->config->debug is false. That would prevent internal structures being displayed at production stage if (when!) debug-mode has been properly disabled.

It seems that "$=ie/die" issue has to do with several things. Buggish behaviour, I'd say. I'll try to explain some of the things that are happening with operators ^=, $= and *= inside the core and database as well.

MySQL fulltext searches are able to match either whole words or word beginnings - so no way to match a word ending with a fulltext search only. PW tries to anchor search term to the beginning/end of the field value by adding a RLIKE with some regex magic. While this works nicely in the beginning on the value, it quite often fails when matching end part. Actually, when it doesn't fail, is only when

• searching for complete words (the very meaning in the first place I suppose) OR
  
• the search term is a predefined stopword.
  

Yes, this does sound a bit backwards, but that's how it's implemented at the moment. Stopwords ("ie" being one!) are filtered out by PW, thus leaving only that RLIKE - which matches on its own (this is why $=ie matches "Élodie").

Additionally PW forces the search term to exists (by adding a '+' operator to the beginning) and tries to find partial matches (by adding a '*' operator to the end). This is problematic because the wildcard operator only matches word beginnings and that will never happen for a word ending shorter than the whole word (this is why $=die does not match "Élodie").

And as MySQL doesn't include short words in the fulltext index at all, there's no way those searches will ever match even a whole word if it's short enough (using "^=Le" doesn't match "Le p'tit").

Well, I guess here's enough explanation for Ryan to get a hold of this when he has time. Hope I got all of the above right... But as I said before, when using fulltext searches, it's all about whole words (mostly at least). And words long enough. And not in the stopword list.

New password security updates

• ProcessWire now uses Blowfish hashing for passwords when you are on PHP 5.3 or newer. If your database were to ever be compromised, this provides better protection from someone attempting to reverse engineer passwords from the hashes. Beyond blowfish hashing, we still use double salting as before (with one salt on the file system, and other unique to each user in the database). For existing accounts, the blowfish hash doesn't actually take effect until you change your password, so it'll be a gradual transition for many of us. Though the admin does give you a reminder every now and then. However, once you go blowfish, you can't go back, so don't develop a site in PHP 5.3 and then launch it to a PHP 5.2 server, unless you don't mind changing your password(s).
  

We've encountered a problem with Blowfish-hashed passwords when using PW 2.3-to-be (dev-branch). Setting or changing a password breaks it so that the user is unable to login - not even right after otherwise successful installation. I've done a fair amount of reading, digging around and testing various things trying to understand this - and yet I'm not quite sure whether it's a PW bug that needs to be fixed (and how) or a PHP bug in certain versions that some of just have to get around in a way or another.

My apologies for the way too long post - just wanted to give all the information to whoever is able to grab this thing and check the facts for real.

Let's get to the details. What I found the problem to be is that PHP crypt() requires for the Blowfish salt to be at least 22 digits from the given set, but only when PHP version is around 5.3.2. As 5.3.0 introduced internal Blowfish implementation in the first place and 5.3.7 fixed a security bug in it, I'm assuming this could be the version range affected.

PW 2.3 creates salt for Blowfish by using only 21 digits + a '$', which seems to be just fine in most cases. However, according to http://php.net/manual/en/function.crypt.php, there should be one more digit from the given alphabet:

CRYPT_BLOWFISH - Blowfish hashing with a salt as follows: "$2a$", "$2x$" or "$2y$", a two digit cost parameter, "$", and 22 digits from the alphabet "./0-9A-Za-z". Using characters outside of this range in the salt will cause crypt() to return a zero-length string.

Then again, the answer at http://stackoverflow.com/questions/4683350/blowfish-salt-length-for-the-crypt-function does state that salt should be 21 chars + the trailing $ as currently implemented in PW 2.3. Dollar sign seems to be commonly used separator/terminator for salt strings in *NIX at least, but that PHP.net article doesn't say it should be used. I'm nowhere close to an expert on this, but according to my tests the $-terminator is not required.

Here's some more information on the matter to digest (feel free to dive in, but don't blame me if your brain melts during the process): http://stackoverflow.com/questions/2225720/why-does-crypt-blowfish-generate-the-same-hash-with-two-different-salts

Here's a little script I used to check for PHP version differences:

#!/usr/bin/php
<?php

$salt20 = '12345678901234567890';
$pw = 'test';

// see if Blowfish hashing is possible
var_dump(CRYPT_BLOWFISH);

echo "Hash with salt20: " . crypt($pw, '$2a$11$'    . $salt20) . "\n";
echo "Hash with salt21: " . crypt($pw, '$2a$11$x'   . $salt20) . "\n";
echo "Hash with salt22: " . crypt($pw, '$2a$11$xy'  . $salt20) . "\n";
echo "Hash with salt23: " . crypt($pw, '$2a$11$xyz' . $salt20) . "\n";

And here are the results for the few PHP versions I was able to test this on:

PHP 5.2.17 (cli) (built: Jul 4 2012 17:10:25)

int(0)

Hash with salt20: $2vU67iv49YBo

Hash with salt21: $2vU67iv49YBo

Hash with salt22: $2vU67iv49YBo

Hash with salt23: $2vU67iv49YBo

PHP 5.3.2-1ubuntu4.18 with Suhosin-Patch (cli) (built: Sep 12 2012 19:33:42)

int(1)

Hash with salt20: *0

Hash with salt21: *0

Hash with salt22: $2a$11$xy1234567890123456789uUGWvzprkfSZicG0PhBtSK5UzC3n7g/i

Hash with salt23: $2a$11$xyz123456789012345678uWEkwqDg/kw42gJyByIvxheqqUx8E4UO

PHP 5.3.14 (cli) (built: Jul 4 2012 17:24:17)

PHP 5.3.15 (cli) (built: Jul 31 2012 18:42:11)

PHP 5.4.4 (cli) (built: Jul 4 2012 17:28:56)

int(1)

Hash with salt20: $2a$11$12345678901234567890$.sTHQ7O4gHgO7alM/cri4fhJvojMP3v2

Hash with salt21: $2a$11$x12345678901234567890.2CYfZ3UUxkaqHY9XWQ.UEKooJv0TT7S

Hash with salt22: $2a$11$xy1234567890123456789uUGWvzprkfSZicG0PhBtSK5UzC3n7g/i

Hash with salt23: $2a$11$xyz123456789012345678uWEkwqDg/kw42gJyByIvxheqqUx8E4UO

PHP 5.2 had no Blowfish support.

PHP 5.3.14+ (at least) pads under 22 digit salt with extra $'s or takes the first 22 digits from a longer salt string.

PHP 5.3.2 does the same for longer strings but fails to return any valid hash for salt with less than 22 digits (this being the very problem we encountered).

Here are the changes I made to get things working for us. Generate 22 digits instead of 21 and separate 7 ($2a$11$) + 22 (random salt) = 29 digits of salt instead of 28 (29th would be the trailing $) from the resulting hash.

$ git diff wire/core/Password.php
diff --git a/wire/core/Password.php b/wire/core/Password.php
index 5ae8668..96a019d 100644
--- a/wire/core/Password.php
+++ b/wire/core/Password.php
@@ -103,7 +103,7 @@ class Password extends Wire {

               // if it's a blowfish hash, separate the salt from the hash
               if($this->isBlowfish($hash)) {
-                       $this->data['salt'] = substr($hash, 0, 28);
+                       $this->data['salt'] = substr($hash, 0, 29);
                       $this->data['hash'] = substr($hash, 29);
               } else {
                       $this->data['hash'] = $hash;
@@ -134,7 +134,7 @@ class Password extends Wire {
               $len = strlen($chars)-1;

               // generate a 21 character random blowfish salt
-               for($n = 0; $n < 21; $n++) $salt .= $chars[mt_rand(0, $len)]; 
+               for($n = 0; $n < 22; $n++) $salt .= $chars[mt_rand(0, $len)]; 
               $salt .= '$'; // plus trailing $

               return $salt;

The trailing $ doesn't seem to be required but does no harm either. The nasty thing here is that anyone having let the current version write Blowfish hashes to the database would be left with broken passwords if the salt generation was changed like this now.

We decided to fall back to old hashes until this has been solved properly (- make supportsBlowfish() return always false to have it this way). Better way for us would be updating PHP to a more recent version, but we're still using Ubuntu 10.04 LTS which only has PHP 5.3.2 by default. So we'd have to either upgrade the OS or tweak a little to get the current one running with PHP 5.3.10 for example.

Another associated thing with Blowfish crypted and salted passwords if that if one sets up a site using current PW 2.3 with PHP 5.3.0-5.3.6 installed, all the passwords will be corrupted when updating PHP to version 5.3.7 or above. See http://php.net/security/crypt_blowfish.php for details. The passwords can be fixed by replacing $2a$ with $2x$ for the old hashes, but there's no bullet-proof way for PW to correct this on its own. And there shouldn't be as those old hashes are a security risk after all. Newly created passwords would work perfectly though, so instructing users to request a new password might be one way to get over this scenario.

Phew. Over and out.

You can use the same selectors with get() than you'd use with find(). So for example getting a page called "footer" under page /blocks/ would be something like:

$pages->get("parent=/blocks/, name=footer")

When using get(), if the selector matches more than one page, only the first match will be returned.

You've got an extra semicolon after the foreach. Thus the loop is empty and echo gets executed only after the last iteration.

So you were doing the right thing basically, just take the semicolon away and you're good.

Good to hear someone finds this useful . There are still some things waiting for a coding session in my todo-list, so more updates are to come - someday.

Thanks Ryan, although that's quite an overstatement what you're saying. But I'll take it.

It's always nice to get feedback, especially positive one coming from the Guru Himself.

@digitex: You may have a logical problem, if I've understood you right. Let's say you've got rental periods like this:

A1: 2013-01-05 ... 2013-01-12, cottage A, booked: 0
B1: 2013-01-05 ... 2013-01-12, cottage B, booked: 0
A2: 2013-01-12 ... 2013-01-19, cottage A, booked: 1
B2: 2013-01-12 ... 2013-01-19, cottage B, booked: 0

Now if you're targeting the repeater items with a selector like this (left some details out) "date_from>=$fromvalue, date_to<=$tovalue, booked=0" to find out which cottages are available from 2013-01-05 to 2013-01-19, you'd get items A1, B1 and B2. Then, using the method Ryan described in post #14 of this thread, it's easy to find those items correspond to cottages A and B. But this does not mean both of those cottages are available for the whole period, just like you're saying.

So, if the rental period (say 14 days) spans over more than one rental term (7 days), it's necessary to repeat the procedure for each of the terms individually and only then you're able to see which cottages appear in the results for all the terms. Combining all of the above, I'm suggesting something like this (haven't tested so no guarantees):

// rental terms of the desired period in a handy structure
$termsInPeriod = array(
 array(
   'start' => '2013-01-05',
   'end' => '2013-01-12'
 ),
 array(
   'start' => '2013-01-12',
   'end' => '2013-01-19'
 )
);

$cottages = array();
foreach($termsInPeriod as $term) {
 // find matches for this term
 $matches = $pages->find("template=repeater_rental_period, date_from>={$term['start']}, date_to<={$term['end']}, booked=0, include=all");

 // no matches this round --> no matches on the whole, no matter what may have been found before
 if(!count($matches)) {
   $cottages = array();
   break;
 }

 // array to hold found cottages for this term
 $termCottages = array();

 foreach($matches as $item) {
   $property = $pages->get((int) substr($item->parent->name, 9));
   if(!$property->viewable()) continue; // skip if property is unpublished or something

   // now you have the $property and the matching repeater $item
   $termCottages[] = $item->id;
 }

 // some results from previous rounds?
 // - yes, then find out common matches
 // - nope, first round --> use the results as is
 if(count($cottages)) $cottages = array_intersect($cottages, $termCottages);
 else $cottages = $termCottages;
}

In the end $cottages should hold all the cottages available for the given period. I'm using $item->id because of the array_intersect(). I'll leave it to you to form a structure like $termsInPeriod from your input - and probably this is fuel for your own thinking at most anyway.

@bcartier: The ImportPagesCSV-module can't do this as is. But I tried making a tiny addition to make it support FieldtypePage (those used to make page references) and it worked amazingly well. The only change needed was to add 'FieldtypePage' to $fieldtypes array (just before init() function if you take a look at the module file), like this:

       protected $fieldtypes = array(
               'FieldtypePageTitle',
               'FieldtypeText',
               'FieldtypeTextarea',
               'FieldtypeInteger',
               'FieldtypeFloat',
               'FieldtypeEmail',
               'FieldtypeURL',
               'FieldtypeCheckbox',
               'FieldtypeFile',
               'FieldtypePage', // add this line
               );

After that addition it's possible to choose a Page field when connecting the fields from the CSV to the ones in the chosen template. I had pre-populated categories at the target site and used their id's in the CSV file to reference those categories. Multiple categories worked like a charm as well, just use a pipe in between id's (123|456|789).

Moreover, if you've got only one category per entry to reference, then you don't even need the id's of the categories - you can use paths as well. Here's a little example:

cat.csv:

title
one
two
three
four

entries.csv:

title,categories
a,/cats/four/
b,/cats/three/
c,/cats/one/
d,/cats/two/

Import cat.csv using a template for categories with (at least) title field, under a page at /cats/. Then import entries.csv using a template for entries, having a title field and a page field. This should leave you with entries that are connected to categories. I hope this gets you somewhere.

@ryan: Looks like page references could be supported very easily. I just used this successfully to import ~3500 pages with category references from an old site to a new PW one. But maybe there's still something else to be done before they're fully supported?

Antti, yes, it's certainly possible. You can hook whenever and wherever you like. It's just a matter of being certain your hook has been registered before the event you're aiming for takes place. So you only need an autoload module to hook something you don't have full control yourself, and don't want to or are not able to require some initialization being called before using the hook.

Here goes. And this one I tested a little so I know it works, for me at least .

function myCustomAuthentication($event) {
 $user = $event->arguments[0];
 $pass = $event->arguments[1];

 // TODO: do whatever check is needed to authenticate $user
 // $pass has whatever you like, a token of some kind probably

 // must set replace-flag to prevent the original Session::authenticate() being called
 $event->replace = true;

 // return value is boolean
 // true: successfully authenticated
 // false: authentication failed
 $event->return = true;
}

// ...aquire a user name, somewhere, somehow...

// hook *before* Session::authenticate() to override it
// second argument is null because we're using a plain function and not a method inside an object
$session->addHookBefore('authenticate', null, 'myCustomAuthentication');

// log in the user, passing whatever needed by myCustomAuthentication() as a password - if anything
$user = $session->login("some-username", "some-token-with-a-meaning-in-this-very-context");

I'll actually be using this piece of code myself as well, this week I hope.

I'm having trouble figuring out why would a repeater be needed here? A plain file field allows you to upload multiple files with descriptions - do you need something more? Maybe I just didn't get it.

If you've got field "files", you can iterate files in it like this:

foreach($page->files as $file) {
 echo $file->url;
 echo $file->description;
}

So almost what you already had there.

Then, check Setup -> Fields -> "files" -> Details -> Valid file extensions to see which types of files are accepted by your field - that setting might explain why certain files are rejected.

I hope this helps.

Change that line with the foreach to

foreach($results as $child) {

That should do it. The first line could be inside the if block too, but should work there as well.

@interrobang: that's probably the main reason (possibly get() and trash() aren't even supposed to be run there). Once again I dived too deep.

That line (Pages.php:779) says:

if(!$this->isDeleteable($page) || $page->template->noTrash) throw new WireException("This page may not be placed in the trash");

And isDeletable() if defined like this:

public function isDeleteable(Page $page) {

$deleteable = true;
if(!$page->id || $page->status & Page::statusSystemID || $page->status & Page::statusSystem) $deleteable = false;
else if($page instanceof NullPage) $deleteable = false;

return $deleteable;
}

So, based on the above there are a few possibilities:

• $transaction is a NullPage object and/or hasn't got id defined: very likely, as this would be the case if no page was found when calling get() with that selector
  
• $transaction has status flag statusSystemID or statusSystem: very unlikely, but you'd probably know if such a flag had been set
  
• transaction template has noTrash property: possible, if you've checked it from Templates -> transaction -> System -> Disable Trash Option (System tab is shown only if you've got advanced=true in your config)
  

I hope this helps.

Edit: To sum it up, try wrapping that trash() call like this:

if($transaction->id) {
$transaction->trash();
}

Edit 2: Sorry for confusing with all that stuff... Just noticed that trash() is being called with a NullPage object as an argument, so get() has definitely found no page. [ #0 [internal function]: Pages->___trash(Object(NullPage)) ]

@soma: At least I used your solution from that very thread as I happened to need something very similar the next week you had shared it. Not that it would necessarily make you feel any better, but just wanted to tell that you were acknowledged back then .

Anyway, clever usage of what ProcessWire makes possible, from the both of you.

You definitely deserve a lecture for debugging PW dev branch there - concentrate man!

Just noticed the logos in the footer aren't working. (just a heads up).

Not sure I understand? Or at least, I don't see them not working. What browser/version?

Href-attribute seems to be broken, see the screenshot.

I'm using Chrome Version 23.0.1271.64 on a Mac.

Edit: ...and same with Safari. Though why would the href be browser dependent anyways.

I think there's a setting for this in site/config.php which defaults to false. Can't remember the name though (on a mobile now).

All the countries with <25% new visits have (or have had) at least one (very) active (>100 posts) forum member -

• Finland - Antti, teppo
  

You can count me in - I'm still under 100 posts myself but I've been reading almost every single post for a half a year now. I'd like to see the figures for Finland without visits from the three of us.

OK, then it just might explain what has happened. Did you try clearing Modules.* files from cache like WillyC suggested before? Even if you did a successful database import after an unsuccessful one, there could be broken metadata cached there.

Gazley, any chance you tried to import data to a pre-populated database with no "drop table ..." statements in your dump? Then some create/insert statements from the beginning of the dump would have been run successfully, until an error due to duplicate key somewhere would have stopped the import leaving the database somewhat broken.

Well, as you managed to get over that missing field already, I'm probably not on the right track with this. Just thinking aloud here.

Just made version 1.1 of this module available (GitHub).

Changes (additions actually) in latest version:

• Explore properties and data of matching pages in a tree view
  
  • Language aware: multi-language and language-alternate fields supported
    
  • Repeater fields and values
    
  • Images and their variations on disk
    
  • More data is loaded on-demand as the tree is traversed deeper
    

  [*]Quick links to edit/view pages, edit templates and run new selectors (select pages with the same template or children of a page)

  [*]Page statuses visualized like in default admin theme

I'll update the first post in this thread and include some screenshots there as well.

Good job @yesjoar!

Tested it a little and found a couple of tiny bugs:

• Variable $browser can be uninitialized at this point leading to PHP notice "Undefined variable" (fetched a page with curl). You may want to assign some default value like you've done with the rest of the variables.
  
• There's a typo here, no $u_agent defined anywhere else (anymore, I guess).
  
• You're using a question mark ("?") for browser version if no version can be found, but that's not a valid character in CSS class name - unless escaped, but that's just confusing. Dashes (-), underscores (_), letters (a-zA-Z) and numbers (0-9) are ok (yes, it's more complicated than that to be exact).
  

Otherwise seems to be doing what it says though.

Hi Brian,

While that could work as is, you might also face memory issues by looping through those 2000 pages at once. This other recent thread may be helpful to you as well: