Jump to content
gyo

Filter system for extensible data modification

Recommended Posts

Hello everyone,

After playing a week with ProcessWire I'm really loving it. As a first project I wanted to create a Profile module based on the existing FrontendUserProfile trying to make it suit my needs, and I found the need for a filter system like the one WordPress has.

I couldn't find anything similar or maybe I didn't completely grasp the Hooks concept, so I've created a simple ProcessWire module to handle that:

Filter

https://github.com/gyopiazza/pw.filter

The module is autoloaded, and once installed you can use the 2 functions provided to filter data. This feature is very useful in certain situations where you want to be able to change some values in a simple way.

function my_function1($value)
{
	return 'filtered by my_function1';
}

function my_function2($value)
{
	return 'filtered by my_function2';
}

// Parameters: filter name, function name or array(class, method), priority
// Higher priority means later execution, default is 10
addFilter('some_filter_name', 'my_function1', 10);
addFilter('some_filter_name', 'my_function2', 10);

$result = 'default value';
$result = applyFilter('some_filter_name', $result);

// $result = 'filtered by my_function2'

Hope you like it, cheers! :)

Giordano

  • Like 2

Share this post


Link to post
Share on other sites

Hello everyone,

After playing a week with ProcessWire I'm really loving it. As a first project I wanted to create a Profile module based on the existing FrontendUserProfile trying to make it suit my needs, and I found the need for a filter system like the one WordPress has.

I couldn't find anything similar or maybe I didn't completely grasp the Hooks concept, so I've created a simple ProcessWire module to handle that:

Hello and welcome!

The Hooks are meant to extend (or even replace) the functionality of ProcessWire and it's modules. They are a general concept and not something that provide features on their own. Instead you attach to them, to implement the feature you are needing.

In your case, one example for their usage could be hooking into Fieldtype::formatValue and automatically filtering some specific fields when they are output in a template. I don't know how you are using the filters, so this might not be very useful. Here's a quick example anyway

// Filter.module

public function init()
{
  $this->addHookAfter("FieldType::formatValue", function(HookEvent $e) {
    $page = $e->arguments(0);
    $field = $e->arguments(1);
    $value = $e->arguments(2);

    // Execute for my_field only
    if($field->name != "my_field")
      return;

    $e->return = applyFilter("my_field", $value);
  });
}

// Demonstration in CLI

require 'index.php';

addFilter('my_field', function($value) { return strtoupper($value); });
addFilter('my_field', function($value) { return md5($value); });

// If you are inside a template this isn't needed
$page = $pages->get('/some-example-page');
$page->setOutputFormatting(true); 

echo $page->my_field; // Would print a MD5-hash of the uppercase version of the original value

It would be quite trivial to extend this so that you could do

$field = $fields->get('my_field');
$field->addFilter(function($value) { ... });
$field->addFilter(function($value) { ... });
  • Like 2

Share this post


Link to post
Share on other sites

Hi sforsman, actually that's the very reason I made the plugin for: fields. :)

It's useful to format the fields values, because you don't always need the same exact value coming from the database.

For instance the Select fields: since the keys get stored, you could retrieve their corresponding labels by using a filter.

Will FieldType::formatValue be attached to any field? That gave me an idea...

public function init()
{
    $this->addHookAfter("FieldType::formatValue", function(HookEvent $e)
    {
        $page = $e->arguments(0);
        $field = $e->arguments(1);
        $value = $e->arguments(2);
        
        // Apply filter for any field
        $value = applyFilter('formatValue', $value);
        // Apply filter for a specific field type
        $value = applyFilter('formatValue_'.$field->type, $value); // Example: formatValue_FieldtypeText
        // Apply filter for a specific field name
        $value = applyFilter('formatValue_'.$field->name, $value); // Example: formatValue_myfield
        
        // Finally return the value
        $e->return = $value;
    });
}

I'm just thinking if that's flexible enough. Surely you can remove the hook at runtime if it's not needed or there could be 2 more methods in the plugin to remove/get the filters. So – to disable the filters temporarily – you could: store the current filters, remove them, do whatever needs to go "unfiltered", and add them back if necessary.

If you think the Filter plugin is useful I'll definitely expand it, and thanks for the feedback!

Share this post


Link to post
Share on other sites

I am on mobile atm but when I get home I will give you the code to implement a flexible way to temporarily disable/enable the execution of the filters. I will also explain formatValue more, but yes, it is there always.

Share this post


Link to post
Share on other sites

Will FieldType::formatValue be attached to any field?

Yes it will be - it's a method provided by the class all Fieldtypes are extending from. This method will be called when you echo the field inside a template, or to be more precise, when output formatting has been turned on.

There is also a hook for FieldType::wakeupValue, which is called when the value for the field is retrieved from the database and attached to the Page object. In other words, if you want the filters to run regardless of the state of output formatting, that's where you want to hook. Or you could even hook both.

I'm just thinking if that's flexible enough

Since your module is an autoloading singular module, there's a very easy way to provide a flexible way to disable/enable the filters temporarily - globally or per field. Here's an example

// Filter.module

protected static $disabled = false;
protected static $disabled_fields;

// Turn the whole filtering system off
public static function setFiltersOff($disable = true)
{
  self::disabled = $disable;
}

// Turn filters off for a specific field
public static function setFieldFiltersOff($field, $disable = true)
{
  // If you pass a string instead of a Field-object, it has to be the name of the field
  if(is_string($field))
    $field = wire('fields')->get($field);

  if(!($field instanceof Field))
    throw new Exception("Invalid field");

  // Initialize
  if(!isset(self::disabled_fields))
    self::disabled_fields = new SplObjectStorage();

  if($disable)
    self::disabled_fields->attach($field);
  else
    self::disabled_fields->detach($field);
}

// Modify the init() to support these features
public function init()
{
  $class = __CLASS__;

  $this->addHookAfter("FieldType::formatValue", function(HookEvent $e) use ($class) {
    if($class::disabled)
      return;

    $page = $e->arguments(0);
    $field = $e->arguments(1);
    $value = $e->arguments(2);

    if(isset($class::disabled_fields) and $class::disabled_fields->contains($field))
       return;

    // The rest is business as usual...

  });

  // Just a bonus hook for another shortcut
  $this->addHook("Field::setFiltersOff", function(HookEvent $e) use ($class) {
    $field = $e->object;
    $disable = ($e->arguments(0) !== null) ? $e->arguments(0) : true;
    $class::setFieldFiltersOff($field, $disable);
  });
}

// Now you can use these like this

// Disable all filters
wire('modules')->get('Filter')->setFiltersOff();

// Turn them back on
wire('modules')->get('Filter')->setFiltersOff(false);

// Disable a specific field
wire('fields')->get('my_field')->setFiltersOff();

// Turn it back on 
wire('fields')->get('my_field')->setFiltersOff(false);

Edit: Added examples

You could obviously implement this in other ways too.

Edited by sforsman
  • Like 1

Share this post


Link to post
Share on other sites

I'm not sure I really understand it enough, is it just to format values? But isn't that what TextFormatter modules are in PW. There can be multiple and sorted, attached to a per field basis. If there's any it will run the value through the Textformatters. 

Fieldtype::formatValue is used to sent through a value of a field after wakeup, but only when outputformatting is on ($page->of(true), you can also disable it or get the unformatted value). Some fields use that to format a value like Datetime or Page. It can also be hooked via modules or in templates. 

To get a unformatted value you can use $page->getUnformatted("fieldname");

A hook in a module example

$this->addHookAfter("Fieldtype::formatValue", $this, function(HookEvent $event){
     $value = $event->return;
     $value = $event->arguments("value");
     $page = $event->arguments("page");
     $field = $event->arguments("field");
     $event->return = "new value";
});

About your module, I still don't understand the real world use case for this or and if it's really best way to go for it. Anyway I think it's a little problematic (?) to have the helper functions outside the module in the global namespace (no namespace yet in PW) and if it's good to name a module  just "Filter" ?

Edit: I'm maybe also confused by the word "filter" in context to PW, cause there's a $wirearray->filter(selector); but it's not about the same thing?

  • Like 1

Share this post


Link to post
Share on other sites
Edit: I'm maybe also confused by the word "filter" in context to PW, cause there's a $wirearray->filter(selector); but it's not about the same thing?

I agree, but the term and the concept is originating from his WP -background - so each to their own, I guess :)

Share this post


Link to post
Share on other sites

I'm not sure I really understand it enough, is it just to format values? But isn't that what TextFormatter modules are in PW. 

As you call it, they are TEXTformatters. The given example with the select field, which has a label and a key, is more than text. I could also image using this, if you use more than one date format throughout your layout and you would like the ability to quickly customize each of the formats. 

Share this post


Link to post
Share on other sites

@Soma

The idea is not to just format values but to modify (filter) a variable, whatever that might be. I understand the confusion, because the hook in the init method we were discussing actually restricts the plugin usage to fields. Just brainstorming. :)

The real difference between using the Filter module and standard hooks is that the first was meant to run "on demand" (only when the function applyFilter() is executed), while the second is attached to any possible field that is output, even if you don't need it. About the naming convention, yeah, the word "filter" is some heritage from the WP concepts, but for the modules names, whenever is possible and if they don't collide, I personally prefer them short and single-worded. And yeah2 those helper functions might pollute the global namespace, they're probably not necessary as one could just call wire('Filter')->add(...);

Anyways, I believe that to use hooks would be fine, just a little more verbose but they would just work:

$this->addHookAfter("Fieldtype::formatValue", $this, function(HookEvent $event){
    $field = $event->arguments("field");

    if ($field->name = 'myfield') {
        $value = $event->return;
        $value = $event->arguments("value");
        $page = $event->arguments("page");
        $event->return = "new value";
    } 
});

With Filter it would be:

wire('Filter')->add('formatValue_myfield', function($value){
    return 'new value';
});

But it only makes sense if somewhere else that value is filtered (like the init method of the Filter module, or inside template files etc...):

$value = wire('Filter')->apply('formatValue_myfield', $value);

I think I can go with the standard hooks even if it takes more code because I prefer to use core features, my only concern is: can we add many hooks without excessive overhead, for example due to the huge size of the $event object?

Thanks for the feedback!

Share this post


Link to post
Share on other sites

I think I can go with the standard hooks even if it takes more code because I prefer to use core features, my only concern is: can we add many hooks without excessive overhead, for example due to the huge size of the $event object?

Hey gyo! I'm a bit busy at the moment so I'll just quickly comment on this part only. You have absolutely nothing to worry about on the size of the $event-object - it is a tiny little object that will basically just hold references to other objects (that already exist in the memory anyway). This does not increase the size of the overall memory usage (in a way that really matters, at least).

It sure is true that every single function call is theoretically "overhead", but we are talking about a very, very small overhead (in the context of hooks). The only thing you need to make sure of (to reduce the overhead), is that if you hook into something that's used a lot and your hook isn't always applicable, you need to return from the hook asap (i.e. check the conditions for execution in the first possible moment).

If it makes you feel better, there's about a hundred hooks added by the PW "core" modules themselves ;)

On a side note, I think it's very cool that ProcessWire uses it's own core architecture to provide many of the "core" services we are used to (e.g. the admin-site).

PS. I have put the word "core" inside quotes because the actual PW core is pretty lightweight. Much of the "core" that ships with the installation is actually just normal modules that hook in to the system.

Edited by sforsman
  • Like 1

Share this post


Link to post
Share on other sites

Hey sforsman! :)

Of course it's a nice thing that PW core uses it's own architecture to provide functionalities, and the whole "modular" idea it's really a big selling point. The modular admin is just awesome, it's like anything is possible. Which I love.

I'll have to dive more into hooks and get better used to it, because that's where the extensibility power really lies.

In any case thanks everybody for the precious help  ^-^

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Paul Greinke
      Hi there. I wrote a custom module for one of my projects. In fact I maybe want to use my module in other projects too. In order to be variable and customizable  I need to implement some custom hooks into my module. So I can afterwards hook into the my functions in order to modify them to match the needs of the new project.
      I tried simply defining functions with the '__' prefix. But that did not work. I'm imagining something like the following:
      <?php class MyClass { public function ___someFunction() { // Do something } } // ready.php $this->addHookBefore('MyClass::someFunction', function($event) { // some customization }); Is there a way to accomplish that? 
    • By adrian
      Hi everyone,
      Here's a new module that I have been meaning to build for a long time.
      http://modules.processwire.com/modules/process-admin-actions/
      https://github.com/adrianbj/ProcessAdminActions
       
      What does it do?
      Do you have a bunch of admin snippets laying around, or do you recreate from them from scratch every time you need them, or do you try to find where you saw them in the forums, or on the ProcessWire Recipes site?
      Admin Actions lets you quickly create actions in the admin that you can use over and over and even make available to your site editors (permissions for each action are assigned to roles separately so you have full control over who has access to which actions).
       
      Included Actions
      It comes bundled with several actions and I will be adding more over time (and hopefully I'll get some PRs from you guys too). You can browse and sort and if you have @tpr's Admin on Steroid's datatables filter feature, you can even filter based on the content of all columns. 

      The headliner action included with the module is: PageTable To RepeaterMatrix which fully converts an existing (and populated) PageTable field to either a Repeater or RepeaterMatrix field. This is a huge timesaver if you have an existing site that makes heavy use of PageTable fields and you would like to give the clients the improved interface of RepeaterMatrix.
      Copy Content To Other Field
      This action copies the content from one field to another field on all pages that use the selected template.
      Copy Field Content To Other Page
      Copies the content from a field on one page to the same field on another page.
      Copy Repeater Items To Other Page
      Add the items from a Repeater field on one page to the same field on another page.
      Copy Table Field Rows To Other Page
      Add the rows from a Table field on one page to the same field on another page.
      Create Users Batcher
      Allows you to batch create users. This module requires the Email New User module and it should be configured to generate a password automatically.
      Delete Unused Fields
      Deletes fields that are not used by any templates.
      Delete Unused Templates
      Deletes templates that are not used by any pages.
      Email Batcher
      Lets you email multiple addresses at once.
      Field Set Or Search And Replace
      Set field values, or search and replace text in field values from a filtered selection of pages and fields.
      FTP Files to Page
      Add files/images from a folder to a selected page.
      Page Active Languages Batcher
      Lets you enable or disable active status of multiple languages on multiple pages at once.
      Page Manipulator
      Uses an InputfieldSelector to query pages and then allows batch actions on the matched pages.
      Page Table To Repeater Matrix
      Fully converts an existing (and populated) PageTable field to either a Repeater or RepeaterMatrix field.
      Template Fields Batcher
      Lets you add or remove multiple fields from multiple templates at once.
      Template Roles Batcher
      Lets you add or remove access permissions, for multiple roles and multiple templates at once.
      User Roles Permissions Batcher
      Lets you add or remove permissions for multiple roles, or roles for multiple users at once.
       
      Creating a New Action
      If you create a new action that you think others would find useful, please add it to the actions subfolder of this module and submit a PR. If you think it is only useful for you, place it in /site/templates/AdminActions/ so that it doesn't get lost on module updates.
      A new action file can be as simple as this:
      class UnpublishAboutPage extends ProcessAdminActions { protected function executeAction() { $p = $this->pages->get('/about/'); $p->addStatus(Page::statusUnpublished); $p->save(); return true; } } Each action:
      class must extend "ProcessAdminActions" and the filename must match the class name and end in ".action.php" like: UnpublishAboutPage.action.php the action method must be: executeAction() As you can see there are only a few lines needed to wrap the actual API call, so it's really worth the small extra effort to make an action.
      Obviously that example action is not very useful. Here is another more useful one that is included with the module. It includes $description, $notes, and $author variables which are used in the module table selector interface. It also makes use of the defineOptions() method which builds the input fields used to gather the required options before running the action.
      class DeleteUnusedFields extends ProcessAdminActions { protected $description = 'Deletes fields that are not used by any templates.'; protected $notes = 'Shows a list of unused fields with checkboxes to select those to delete.'; protected $author = 'Adrian Jones'; protected $authorLinks = array( 'pwforum' => '985-adrian', 'pwdirectory' => 'adrian-jones', 'github' => 'adrianbj', ); protected function defineOptions() { $fieldOptions = array(); foreach($this->fields as $field) { if ($field->flags & Field::flagSystem || $field->flags & Field::flagPermanent) continue; if(count($field->getFieldgroups()) === 0) $fieldOptions[$field->id] = $field->label ? $field->label . ' (' . $field->name . ')' : $field->name; } return array( array( 'name' => 'fields', 'label' => 'Fields', 'description' => 'Select the fields you want to delete', 'notes' => 'Note that all fields listed are not used by any templates and should therefore be safe to delete', 'type' => 'checkboxes', 'options' => $fieldOptions, 'required' => true ) ); } protected function executeAction($options) { $count = 0; foreach($options['fields'] as $field) { $f = $this->fields->get($field); $this->fields->delete($f); $count++; } $this->successMessage = $count . ' field' . _n('', 's', $count) . ' ' . _n('was', 'were', $count) . ' successfully deleted'; return true; } }  
      This defineOptions() method builds input fields that look like this:

      Finally we use $options array in the executeAction() method to get the values entered into those options fields to run the API script to remove the checked fields.
      There is one additional method that I didn't outline called: checkRequirements() - you can see it in action in the PageTableToRepeaterMatrix action. You can use this to prevent the action from running if certain requirements are not met.
      At the end of the executeAction() method you can populate $this->successMessage, or $this->failureMessage which will be returned after the action has finished.
       
      Populating options via URL parameters
      You can also populate the option parameters via URL parameters. You should split multiple values with a “|” character.
      You can either just pre-populate options:
      http://mysite.dev/processwire/setup/admin-actions/options?action=TemplateFieldsBatcher&templates=29|56&fields=219&addOrRemove=add
      or you can execute immediately:
      http://mysite.dev/processwire/setup/admin-actions/execute?action=TemplateFieldsBatcher&templates=29|56&fields=219&addOrRemove=add

      Note the “options” vs “execute” as the last path before the parameters.
       
      Automatic Backup / Restore
      Before any action is executed, a full database backup is automatically made. You have a few options to run a restore if needed:
      Follow the Restore link that is presented after an action completes Use the "Restore" submenu: Setup > Admin Actions > Restore Move the restoredb.php file from the /site/assets/cache/AdminActions/ folder to the root of your site and load in the browser Manually restore using the AdminActionsBackup.sql file in the /site/assets/cache/AdminActions/ folder I think all these features make it very easy to create custom admin data manipulation methods that can be shared with others and executed using a simple interface without needing to build a full Process Module custom interface from scratch. I also hope it will reduce the barriers for new ProcessWire users to create custom admin functionality.
      Please let me know what you think, especially if you have ideas for improving the interface, or the way actions are defined.
       
       
    • By joshua
      ---
      Module Directory: https://modules.processwire.com/modules/privacy-wire/
      Github: https://github.com/blaueQuelle/privacywire/
      Packagist:https://packagist.org/packages/blauequelle/privacywire
      Module Class Name: PrivacyWire
      Changelog: https://github.com/blaueQuelle/privacywire/blob/master/Changelog.md
      ---
      This module is (yet another) way for implementing a cookie management solution.
      Of course there are several other possibilities:
      - https://processwire.com/talk/topic/22920-klaro-cookie-consent-manager/
      - https://github.com/webmanufaktur/CookieManagementBanner
      - https://github.com/johannesdachsel/cookiemonster
      - https://www.oiljs.org/
      - ... and so on ...
      In this module you can configure which kind of cookie categories you want to manage:

      You can also enable the support for respecting the Do-Not-Track (DNT) header to don't annoy users, who already decided for all their browsing experience.
      Currently there are four possible cookie groups:
      - Necessary (always enabled)
      - Functional
      - Statistics
      - Marketing
      - External Media
      All groups can be renamed, so feel free to use other cookie group names. I just haven't found a way to implement a "repeater like" field as configurable module field ...
      When you want to load specific scripts ( like Google Analytics, Google Maps, ...) only after the user's content to this specific category of cookies, just use the following script syntax:
      <script type="text/plain" data-type="text/javascript" data-category="statistics" data-src="/path/to/your/statistic/script.js"></script> <script type="text/plain" data-type="text/javascript" data-category="marketing" data-src="/path/to/your/mareketing/script.js"></script> <script type="text/plain" data-type="text/javascript" data-category="external_media" data-src="/path/to/your/external-media/script.js"></script> <script type="text/plain" data-type="text/javascript" data-category="marketing">console.log("Inline scripts are also working!");</script> The data-attributes (data-type and data-category) are required to get recognized by PrivacyWire. the data-attributes are giving hints, how the script shall be loaded, if the data-category is within the cookie consents of the user. These scripts are loaded asynchronously after the user made the decision.
      If you want to give the users the possibility to change their consent, you can use the following Textformatter:
      [[privacywire-choose-cookies]] It's planned to add also other Textformatters to opt-out of specific cookie groups or delete the whole consent cookie.
      You can also add a custom link to output the banner again with a link / button with following class:
      <a href="#" class="privacywire-show-options">Show Cookie Options</a> <button class="privacywire-show-options">Show Cookie Options</button>  
      I would love to hear your feedback 🙂
      CHANGELOG
      You can find the always up-to-date changelog file here.
    • By joshua
      As we often use Matomo (former known as Piwik) instead of Google Analytics we wanted to embed Matomo not only in the template code but also via the ProcessWire backend.
      That's why I developed a tiny module for the implementation.
      The module provides the possibility to connect to an existing Matomo installation with the classical site tracking and also via the Matomo Tag Manager.
      If you have also PrivacyWire installed, you can tell MatomoWire to only load the script, if the user has accepted cookies via PrivacyWire.
      To offer an Opt-Out solution you can choose between the simple Opt-Out iFrame, delivered by your Matomo installation, or a button to choose cookies via PrivacyWire.
      You'll find the module both in the module directory and via github:
      ProcessWire Module Directory MatomoWire @ GitHub MatomoWire @ Packagist ->installable via composer require blauequelle/matomowire I'm looking forward to hear your feedback!


    • By Robin S
      If your module has a lot of config fields you might want to divide them into groups inside a tabbed interface. Here is a demonstration module showing how this can be done.
      https://github.com/Toutouwai/ModuleConfigTabs

      Thanks to @kixe for providing my starting point in this forum topic.
×
×
  • Create New...