Relationship between sleepValue, wakeupValue and sanitizeValue methods

[Rob]

I've just written a little fieldtype that fetches some data from a 3rd party DB and then returns a populated InputfieldSelect object.

What I can't work out is that I want the current selected value to be pre-selected in the page when it is re-edited.

Also, I want to automatically pull the entire related record to stick i nthe page data for templating, whereas the only bit that needs to be stored is the ID of the related record.  At the moment I seem to have intermittent behaviour that sometimes saves the ID int othe field table and sometimes doesn't.

Is it correct that the place I want to do the extra work (grab entire record based on saved ID) is in wakeupValue()?  And in this instance, when the field is onl ysaving the ID from the selelct box as usual, can I leave sleepValue() alone?  That is to say, not override it?

[ryan]

Is it correct that the place I want to do the extra work (grab entire record based on saved ID) is in wakeupValue()?

That's correct. wakeupValue is where you take whatever value is stored in the DB, and expand that to be how you want the value represented at runtime. So in your case, it sounds like you are doing it right if you are taking that ID and then building a an object from it with other data.

And in this instance, when the field is only saving the ID from the selelct box as usual, can I leave sleepValue() alone?  That is to say, not override it?

sleepValue does the opposite of wakeupValue. It will be given the value you created with wakeupValue, and your sleepValue needs to take that and reduce it back to its representation in the DB. In your case, it sounds like that is just the ID number.

[Rob]

Thanks Ryan.

I've just figured out that sanitizeValue() has to return the correct value that will make sense in the context of the rendered field in the page edit AFTER the wakeupValue() method has been executed.  I think!!  Correct me if I'm wrong.

So in my case, because wakeupValue() is loading an entire record and turning into an associative aray, for the InputfieldSelect to correctly pre-populate the value when it loads a page for editing, my sanitizeValue() appeas to need to return the correct array element from the wakeupValue() result.

I hope all this is useful to other developers!!  I'm enjoying the learning process and hopefully this can all feed back into the documentation in the long run.  It's a good CMS and even with the few issues I've run into so far the module/api/hook system is far superior to many other CMS'.  It just takes a bit of experimentation it seems.

Are you planning any proper tutorial materials?  I think a step-by-step of some common tasks or even more advanced module development would be useful and encourage more people to get on board.

[Rob]

Nope, I spoke too soon, I'm back to being thoroughly confused!

[ryan]

In your case, you can probably just have sanitizeValue return the value it's given (or don't override that method at all). The sanitizeValue method is just meant to sanitize a value set to a Page. The simplest example would be for an integer field. Lets say that you did this in the API:

$page->some_integer_field = 123; 

When you do that, what happens behind the scenes is that $page does this:

<?php
$field = wire('fields')->get('some_integer_field'); 
$fieldtype = $field->type; 
$value = $fieldtype->sanitizeValue(123); // or whatever value you set
$this->set('some_integer_field', $value); 

So your sanitizeValue method gets called every time a value is set to the page for any known field. In the case of FieldtypeInteger, it would just need to ensure that the value is in fact an integer. So FieldtypeInteger's sanitizeValue() implementation would look like this:

<?php
$value = (int) $value; // typecast to an integer
return $value; 

That sanitizeValue method just needs to sanitize and convert it to whatever the expected format is. Implementation of this method is not crucial, so this is something you can come back to.

This is different from sleepValue and wakeupValue because those two are for converting data to and from the format it should be in the database. For something like an integer (like above) sleepValue and wakeupValue wouldn't need to do anything at all, since an integer will be represented the same way in the DB as in the $page. But more complex things that take values from the DB and convert it to/from an object are more the type of thing where sleepValue/wakeupValue come into play.

I hope all this is useful to other developers!!  I'm enjoying the learning process and hopefully this can all feed back into the documentation in the long run.  It's a good CMS and even with the few issues I've run into so far the module/api/hook system is far superior to many other CMS'.  It just takes a bit of experimentation it seems.

I'm very happy to have you experimenting with it. This is also helpful to me as I move towards making more official documentation and tutorials for creating this stuff.

Are you planning any proper tutorial materials?  I think a step-by-step of some common tasks or even more advanced module development would be useful and encourage more people to get on board.

Definitely!

[Rob]

Ryan,

What isn't clear to me is when sanitizeValue() is called.

If i put print_r($value) as the first line of my sanitizeValue() method then it dumps out an array because this is what is coming from my wakeupValue() method.  This implies that sanitizeValue() is being passed the value from my wakeupValue() method.

Also, I seem to need to return the value that will logically pre-populate my selectbox field (the id value) otherwise it does not pre-populate.  THis doesn't make sense to me at all.  Is this a secondary purpose of the value returned from this method?!

Basically, if I don't have wakeupValue() or sleepValue() methods and my sanitizeValue() method just has a single line that returns $value, then my selectbox works properly in terms of saving a value, loading a value (pre-populate the field) but I then as soon as I try to add any wake or sleep functions to get my entoire record to be loaded into the page data, it all falls apart.

I basically have trouble, at this moment, believing that it's only purpose is to sanitise data goign into a page object, becasue it clearly has impact on how my selectbox is rendered.

If I am loading an entire record with my wakeupValue() method (whcih I blieve is the correct place for that functionality) it then seems that I need to do some sort of extra work in sanitizeValue() to make the selectbox work properly which is conteray to what you're saying about them being separate and not related.  The two are definitely related in some way.

Either that, or I am misunderstanding the whole process of designing a selectbox Fieldtype.

It's very frustrating that I can save a simple integer reference to a DB record, but not then be able to grab that data easily.  In theory I could grab it in the template using the simple ID passed through, but this seems a bit clunky to me.

[ryan]

If i put print_r($value) as the first line of my sanitizeValue() method then it dumps out an array because this is what is coming from my wakeupValue() method.  This implies that sanitizeValue() is being passed the value from my wakeupValue() method.

That's correct. Anything set to a $page passes through sanitizeValue, regardless of where it comes from. But I suggest that you ignore the sanitizeValue method for now, because there is no reason to implement it at this stage (unless you've extended another Fieldtype that does something with it that you don't want).

Also, I seem to need to return the value that will logically pre-populate my selectbox field (the id value) otherwise it does not pre-populate.  This doesn't make sense to me at all.  Is this a secondary purpose of the value returned from this method?!

I'm not sure that I understand exactly what you are saying. But your sanitizeValue shouldn't attempt to modify the value it's given unless it's of the wrong type. So in your case, I would suggest leaving sanitizeValue out of the equation for now and don't bother implementing it until you have everything else working.

Basically, if I don't have wakeupValue() or sleepValue() methods and my sanitizeValue() method just has a single line that returns $value, then my selectbox works properly in terms of saving a value, loading a value (pre-populate the field) but I then as soon as I try to add any wake or sleep functions to get my entoire record to be loaded into the page data, it all falls apart.

If things are working until you try to implement them, try doing this: leave the existing implementation in place, but put in sleepValue/wakeupValue methods just to observe what's going back and forth. That should help to clear up ambiguity about why it might be breaking when you try to provide an implementation for them.

<?php

public function ___wakeupValue(Page $page, Field $field, $value) {
    $value = parent::___wakeupValue($page, $field, $value); 
    echo "<pre>wakeupValue: ";
    var_dump($value); 
    echo "<pre>";
    return $value; 
}

public function ___sleepValue(Page $page, Field $field, $value) {
    $value = parent::___sleepValue($page, $field, $value); 
    echo "<pre>sleepValue: ";
    var_dump($value); 
    echo "</pre>";
    return $value;
}

That way you can see exactly what data is getting sent back and forth without modifying it. Once you know exactly what's getting sent around, you should be able to expand upon it. If you are dealing with a multi field table, then an array is going to be used as the native storage format, and it will have an index called 'data' that contains the primary value. But it's best to observe exactly what's there rather than me tell you since I don't know the full context of your situation.

It's very frustrating that I can save a simple integer reference to a DB record, but not then be able to grab that data easily.  In theory I could grab it in the template using the simple ID passed through, but this seems a bit clunky to me.

You should be able to do what you are wanting to do. Lets say that you want to have your 'id' integer that's stored in the database convert to an instance of MyClass during runtime. Here is pseudocode for what you would want to do:

<?php

// convert an instanceof MyClass to an integer id
function sleepValue($value) {
    return $value->id; 
}

// convert an integer id to a MyClass
function wakeupValue($value) {
    return new MyClass($value); 
}

// make sure $value is a MyClass, but don't bother with this for now... 
function sanitizeValue($value) {
    if($value instanceof MyClass) return $value;
        else throw new WireException("value was not a MyClass"); 
}

[ryan]

Small correction: sanitizeValue is an abstract method in the Fieldtype class, so you have to implement it. But just make your implementation nothing but this for now:

public function sanitizeValue(Page $page, Field $field, $value) {
     return $value; 
}

[Rob]

Thanks Ryan.

It seems that I had to create a custom Inputfield that extended InputfieldSelect in order to add the custom behaviour that preselects the saved value in the selectbox.

Next problem - When I now print_r() the content of $value in the sleepValue() function of the Fieldtype, it is giving me the record is it currently is in the DB (as it would be since being loaded and passed through wakeupValue()) but it doesn't seem to have updated with the value from the post data.  So essentially I can't overwrite the value in the DB.

So my next thought is this - presumably I need to do something extra in my overridden custom Inputfield class to pass the posted value back to the Fieldtype object?  Does that sound about right?

Apologies for th constant stream of questions, I hope you don't mind!!  I hope it will be a useful reference for others, and once I have it clear I may post some tutorials myself.

[Rob]

I don't know what the hell I've done now, but now my sleepValue() method isn't being called at all!!

[ryan]

So my next thought is this - presumably I need to do something extra in my overridden custom Inputfield class to pass the posted value back to the Fieldtype object?  Does that sound about right?

What is the context? Is this something that's being edited in a page like any other, or are you using this somewhere else? If in PageEdit, then that process handles all the communication of values between the Inputfield and the $page. If you are trying to use Inputfields somewhere else, then you may need to pull the value from it and set it to your $page or wherever else you want it set.

So my next thought is this - presumably I need to do something extra in my overridden custom Inputfield class to pass the posted value back to the Fieldtype object?  Does that sound about right?

No, Inputfields don't know anything about Fieldtypes. They are designed to function independently. They don't set values to anywhere other than themselves. This enables them to be useful for your own forms elsewhere, regardless of whether they are used by some Fieldtype or not. If used in PageEdit, the setting of values to Inputfields and pages is handled internally by PW, so you shouldn't have to do anything there.

PW may set a 'value' attribute to your Inputfield twice. First before it processes the form (the existing value), and during processing the form (the new value). That enables your Inputfield to tell if the value has changed. If you start working with more complex data types in your Inputfield, you may need to tell PW when the value has changed. You can do that just by calling $this->trackChange('value') in your Inputfield. This probably isn't necessary in your case, but if you run up against a wall, try adding that in your Inputfield's processInput() function, just in case. 

Post some code! I think it'll be a lot easier to help if I can see what you are doing. It's a lot easier for me to understand code by seeing it rather than reading about it. If you don't want to post here, feel free to PM it to me too.

[Rob]

Before I get to posting code Ryan, any thoughts on why sleepValue() wouldn't be getting called at all?

If I put any sort of debugging code, dumping values out to the screen, call exit(), anything, it does nothing, it just carries on and claims to have saved the filed but nothing has been updated in the DB.  It really seems like that method just suddenly stopped getting called.

I'm banging my head against the table here, it shouldn't have taken me two days to get this custom field code written!

[ryan]

When saving a page, sleepValue won't get called if PW doesn't think the field actually changed. When you save a page, PW only saves the fields that it detects changed. So what I mentioned about $this->trackChange('value') in your Inputfield may be more applicable to your case than I thought. But I don't know the context well enough here to say for sure. Still don't know if we are even talking about pages saved from PageEdit.

[Rob]

When saving a page, sleepValue won't get called if PW doesn't think the field actually changed. When you save a page, PW only saves the fields that it detects changed. So what I mentioned about $this->trackChange('value') in your Inputfield may be more applicable to your case than I thought. But I don't know the context well enough here to say for sure. Still don't know if we are even talking about pages saved from PageEdit.

Right I think I've finally cracked it!!

I had to write a custom processInput() in my custom Inputfield because, for some reason I couldn't figure out, it wasn't correctly populating the underlying fieldtype object with the posted value so I had to do it myself.

<?php

class FieldtypeFXProductList extends Fieldtype {

    public static function getModuleInfo() {
	return array(
		'title' => 'Product List',
		'version' => 001,
		'summary' => 
			'Provides a dropdown list of products'
		);
}
    
    public function getInputfield(Page $page, Field $field) {
        $inputfield = $this->modules->get('InputfieldFXProductSelect');
        
        try {
            $db = new Database(...);
            $db->select_db(...);
            $result = $db->query("SELECT product_id AS id, name FROM product ORDER BY name ASC");
            while($row = $result->fetch_object()) {
                $inputfield->addOption($row->id, $row->name);
            }
        } catch(Exception $e) {
            echo $e->getMessage();
        }
      
        return $inputfield;
    }
    
    public function ___wakeupValue(Page $page, Field $field, $value) {
        $db = new Database(...);
        $db->select_db(...);
        $result = $db->query("SELECT * FROM product WHERE product_id = $value LIMIT 0,1");
        $data = NULL;
        if($result) $data = $result->fetch_assoc();
        return $data;
    }
    
    public function ___sleepValue(Page $page, Field $field, $value) {
        return $value['product_id'];
    }
    
    public function sanitizeValue(Page $page, Field $field, $value) {
        return $value;
}
    
}
?>

<?php

class InputfieldFXProductSelect extends InputfieldSelect {
    
    public static function getModuleInfo() {
	return array(
		'title' => 'Product Select',
		'version' => 001,
		'summary' => 'Selection of a single product from a select pulldown',
		'permanent' => true, 
		);
}
    
    public function isOptionSelected($value) {

	if($this instanceof InputfieldHasArrayValue) {
		return in_array($value, $this->value['product_id']); 
	}
        if(!isset($this->value['product_id'])) return FALSE;
	return "$value" == (string) $this->value['product_id']; 
}
    
    public function ___processInput(WireInputData $input) {
        if(isset($input[$this->name])) {
            if($input[$this->name] != $this->value['product_id']) {
                $value = $this->value;
                $value['product_id'] = $input[$this->name];
                $this->value = $value;
                $this->trackChange('value');
            }
        }
        return $this;
    }
    
}
?>

[ryan]

Rob, glad that you got it working! Thanks for posting the code.

Also, I was working on a Fieldtype and Inputfield to provide a good example of how it all works when creating a Fieldtype that needs to hold multiple pieces of data. It's called FieldtypeMapMarker and it holds a mappable address, latitude and longitude. Once you enter an address and save, it geocodes the address to latitude and longitude and shows you the location on a map. Not sure if this is still as helpful to you since it sounds like you already figured it out, but wanted to post the link to it anyway just in case:

https://github.com/ryancramerdesign/FieldtypeMapMarker

[apeisa]

Ryan, are you trying to hide new modules from us?

I don't want to hijack this conversation (you should post release topic!), quickly tested this Map Marker and it works fantastic. Only one minor thing: it converted lat and lng to integers (or integer floats: 60.000000 / 25.000000).

[ryan]

Not trying to hide anything. Just figured I should test it a little more before announcing it in the modules forum. But I was thinking I'd post something about it tomorrow. As it is now, I thought it was probably the best example I had of a Fieldtype/Inputfield that has multiple columns of data, and how to create one. Since Rob had these needs in his Fieldtype, I thought it would be useful to post here. I didn't originally plan to make it a Fieldtype, but realized it would translate to a good example or starting point for a tutorial.

I can't seem to duplicate the integer floats issue. Just testing here again geocoding various addresses and am always getting the full exact lat/lng. Is there a specific address you are geocoding that does that, or is it any address? I'm wondering if it's getting cut by a float precision setting in PHP or MySQL. I might have to keep these numbers as purely strings from the PHP side to keep out any uninvited float precision settings. But before I do that, I want to find out more and see if I can duplicate here. If not, then I want to at least understand where they might be getting converted. Let me know if there's anything I need to duplicate, and I'll do more research here. Thanks for letting me know.

[apeisa]

All addresses. This "8300 Riding Ridge Place, McLean, VA 22102" turns to: 38 / -77 (and six zeros).

Precision setting is 14 in php, not sure what/where I should look from MySQL?

[ryan]

Unfortunately I don't know where it could be getting rounded to an integer. There are 3 places where it could be: PHP, MySQL or JS. I'm guessing it's not JS though. Clearly there's some server side precision setting that's breaking the floats. Rather than worry over that, I'm just going to use strings instead. I just pushed an update that changes these to be strings in MySQL rather than FLOAT(10,6). If this doesn't fix it, then I'm really wrong about what the problem is. You'll have to delete your current MapMarker field and create it again since the schema has changed. Thanks for our help in testing this.

[apeisa]

Now it works - but on admin UI it leaves lat and lng empty for some reason.

This is the notice it gives after saving:

Geocode OK ROOFTOP: '8300 Riding Ridge Place, McLean, VA 22102'

But both Latitude and Longitude remains empty on UI. When I look into the db it has correct values.

[ryan]

Interesting. Still can't duplicate here, so I'm wondering if this has something to do with PHP or JS localization. Would a number like 33.7766779 be represented any differently than that where you are? like 33,7766779 (comma rather dot). The module will set the number to blank if is_numeric() fails, so I'm wondering if is_numeric() is failing due to differences in localization.

[apeisa]

Ah, that is it. It saves it to database with comma. I have this on my config.php:

setlocale(LC_ALL, array('fi_FI.UTF-8','fi_FI@euro','fi_FI','finnish'));

If I remove it, then it works fine. I had no idea that it would affect floats also (makes sense though). I am setting locale for everything, but I think it would be enough for just date and times.

[apeisa]

I changed my setlocale to this:

setlocale(LC_TIME, 'fi_FI.UTF-8');

And everything works fine.

EDIT: I did hijacked this topic, sorry for that Rob. Ryan - thanks for your help and great module. I really like this (implemented already for our site-profile) and will be great example for future modules.

[ryan]

Great, that's it then. That also explains why the floats were getting rounded to integers. Now maybe I should go back to floats in the DB, not sure. But I've just committed an update that replaces any commas with periods when it sets the lat/lng values, so hopefully that will fix it regardless of the localization settings. Thanks again for your help testing. The whole family is now up and I'm getting told to turn off the computer. Glad we were able to fix this one.

[apeisa]

Now maybe I should go back to floats in the DB, not sure.

Probably yes. That way we could do more efficient searches based on location. Find all places within 5km from this point etc.