WireArray filterData selector

Recommended Posts

Found myself needing to filter repeater items based upon start and end date fields.  These fields aren't required, so repeater items with no dates should be included and those with dates (or a just one date -- start or end) should be evaluated for inclusion.

I slowly figured out that I was dealing with in-memory filtering (of a RepeaterPageArray) and not a database-driven operation.  So, that eliminated the possibility of using or-groups in my selector.  I still thought I could use the 'filter' and 'not' functions to get the job done.  Turns out the way selectors are handled for in-memory filtering is quite different from database-driven selectors.

Here is what I want(ed) to do to filter the dates:

// Start date not specified or in the past
// Not when end date is specified and has passed
$repeater->not('end_time>0, end_time<'.$now);

The first one worked exactly as expected.  The second it where I ran into problems.  The 'filter' function (which calls the 'filterData' function) takes a $selector argument.  From everything I read about selectors, I assumed that the entire selector would have to match.  In other words, each comma represented an 'and' scenario.  Turns out that each separate comma-separated-selector in the $selector as a whole gets evaluated independently.  In the case of the 'filterData' function, the elements are removed if any one of those selectors matches.  Essentially, we have an 'or' scenario instead of an 'and' scenario.

In my case above, there was no way for me to filter for a non-empty date AND one that occurs before a given date...at least that I could think of.

So, my main question is if the filter (and related) function should operate on the entire selector passed to the function instead of its individual parts in some sequence.  That is what I would have assumed a selector to do.

In my project, altering the segment of the WireArray::filterData function solved my problem for now.  ...at least till I forget about it and overwrite it when I upgrade next.  Here is the code I changed within the function

// now filter the data according to the selectors that remain
foreach($this->data as $key => $item) {
	$filter_item = true;
	foreach($selectors as $selector) {
		if(is_array($selector->field)) {
			$value = array();
			foreach($selector->field as $field) $value[] = (string) $this->getItemPropertyValue($item, $field);
		} else {
			$value = (string) $this->getItemPropertyValue($item, $selector->field);
		if($not === $selector->matches($value))
		$filter_item = false;
	if($filter_item && isset($this->data[$key])) {
		$this->trackRemove($this->data[$key], $key);

I would love to hear what you all think.  If I have misunderstood something, then I would welcome a different solution.

  • Like 1

Share this post

Link to post
Share on other sites

Hi and welcome @dscONE!

Yes, I do think the behaviour you have discovered is unexpected. Just to reiterate in a simplified way what I think you are saying....

The expected behaviour is that for any selector, the combined results of $pagearray->find($some_selector) and $pagearray->not($some_selector) should equal the entire PageArray. But this is not the case, as can be demonstrated with a PageArray containing three pages titled "red car", "blue car", "green truck"...

$matches = $pagearray->find("title~=car"); // "red car", "blue car"
$not_matches = $pagearray->not("title~=car"); // "green truck"
// So far, so good

$matches = $pagearray->find("title~=car, !title~=red"); // "blue car"
$not_matches = $pagearray->not("title~=car, !title~=red"); // expected "red car", "green truck" but got unexpected empty PageArray

Would you please report this issue over at the PW issues GitHub repo? https://github.com/processwire/processwire-issues/issues



  • Like 1

Share this post

Link to post
Share on other sites

Thank you, Robin S, for the response.  I must have missed your first one, so I apologize for not reporting the issue myself.  But, thanks for the assist.

It seems to me that you have represented my problem accurately with your example.  The problem arises with multiple selectors and because it looks like the code does the actual filtering (destructive) as soon as it finds a match as it loops through the array of selector items parsed from the selector as a whole.

Thanks again.  I appreciate your response.

  • Like 1

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By ivineets
      Whenever I try to create a repeater field on my website, I get following error:
      General error: 3161 Storage engine MyISAM is disabled (Table creation is disallowed).
      If I still continue, it creates the field but doesn't allow storing any kind of data. How do I resolve this?  Is there any alternative to repeatable content that I can use?
      Thanks for your time.
    • By ethanbeyer
      I recently posted in this topic, but I decided to start my own thread because while I believe my issue is related to the one in that thread, they are not exactly the same:
      I have created a custom User Template in the method outlined in the docs. I am creating a directory, so it made sense that every page in the directory was a Directory Member, so they could log in and edit their own information while also keeping the entire directory protected behind a login wall.
      So the new user type is created: "directory-member".
      I then created two new roles: "member" and "directory-admin":
      The "member" only has the ability to View directory-member pages, and "profile-edit", which allows them to manage their own information. The "directory-admin" has the ability to edit any directory-member pages, and administer users. Some Directory Members are both, but all have at least the "member" role.
      The first hint that something was wrong was when I was testing a "member" user and I could not add a new item to a repeater on that profile. The url for the profile edit (this will be important shortly) is site.dev/admin/profile. The repeater is set up to load new items through AJAX. If this option is turned off, the rest of this issue is no longer completely valid. But as I have found what I believe to be a pretty large issue in the Processwire codebase, I thought it worth bringing up.
      See, every page (even a user) has a $page->editUrl() method, and it returns a URL like this: site.dev/admin/access/users/edit/?id=2096. That's all good and fine for users that have page-edit permissions, but if they don't, that link will resolve to the admin's equivalent of a 404.
      So the way that Processwire currently gets around this is by creating a specific editing area for a user to interact with only their profile: /admin/profile. And that works pretty nicely, except for the fact that nowhere is editUrl() ever made aware of the difference. editUrl() is not hookable, and whether or not a page is editable is based on the PagePermissions module.
      On top of that, there are several core modules that hardcode a search-and-replace (see InputfieldRepeater.module:627) where the editing screen is for Users. This doesn't allow for a huge degree of flexibility that is offered in other places throughout Processwire. If line 627 of InputfieldRepeater is changed from this:
      $editorUrl = str_replace('/access/users/edit/', '/page/edit/', $editorUrl); to this:
      $editorUrl = str_replace('/access/users/edit/', '/profile/', $editorUrl); ...the AJAX repeaters work. It's maddening!
      As is brought up in the thread I attached above, a lot of the features of page editing are missing within /admin/profile/, and it just makes for an altogether strange editing experience. A user who has "page-edit" permissions for templates other than directory-member, but does have "profile-edit" permissions, will see their user in the Page List, but cannot edit their Page unless they hover over the wrench and click the "Profile" link. It just seems - off.
      I think what this hinges on for me is that the editUrl() of the user should be "/admin/profile/" if that user is logged in (and their page should be editable from the Page List), or the "/admin/access/users/edit/" url; regardless of the URL, both links should resolve to the Page Edit screen, as the Profile Edit screen seems to be a unnecessarily neutered version of Page Edit.
    • By cst989
      I have a file I want to access with ajax:
      The purpose of this file is to iterate through a repeater, and get the image from each entry.
      The number of images in the image field is set to 1, and just for good measure, to return a single image.
      And my code:
      // site/ajax/processImage.ajax.php?group=1206 require_once($_SERVER['DOCUMENT_ROOT'].'/index.php'); $resourceGroup = (int) $_REQUEST['group']; // get the Repeater field $resources = $pages->get($resourceGroup)->resources; foreach ($resources as $resource) { echo $resource->title; // Works as expected echo $resource->image->url; // /site/assets/files/1259/ echo $resource->image; // filename.jpg echo $resource->image->description; // nothing } See comments above for what is output, why isn't URL giving me a full URL, and no description is available?
      If I try to access $image->size() I get the following fatal error:
      Error: Uncaught exception 'ProcessWire\WireException' with message 'Method Pageimages::size does not exist or is not callable in this context' in F:\sites\<sitename>\wire\core\Wire.php:519
    • By Jest
      Hi all,
      today I've run into some strange error when using selectors. The following selector works fine:
      $pages->find('template=TemplateName, (many|of|different|fields*=Searching), (year=Searching), (year>=1941, year<=1960), author*=unknown, publisher*="two, maybe three words"|unknown|"another two, maybe three words", area*=All'); Whereas this one throws an error:
      $pages->find('template=TemplateName, (many|of|different|fields*=Searching), (year=Searching), (year>=1941, year<=1960), author*=unknown, publisher*="two, maybe three words"|unknown, area*=All'); The only difference is in publisher field having only two values. The error is Exception: Unknown Selector operator: '' -- was your selector value properly escaped?
       Additional info:
      ProcessWire 3.0.62 All values are sanitized using $sanitizer->selectorValue(string) publisher value usually has commas or parenthesis publisher="one, two"|"three, four" works publisher="one, two"|three|"four, five" works publisher="one, two"|five doesn't work publisher=five works The only conclusion I can make is the selector fails when the last value of publisher is a single word. Any thoughts?
      All help is greatly appreciated!
    • By activestate
      I'm having problems with seatching for a value in repeater.
      One of pages have attached repeater called 'insight_repeater'.
      This repeater has h1_tag (text field) and i'm trying to search for repeater items that contains 'lor' phrase.

      And right now i'm trying to do simple search, looking for phrase 'lor'. I've tried many approaches, but nothins seems to work, so i want to ask what i'm doing wrong here:

      <?php $query = $page->get("insight_repeater")->find('h1_tag~=Lor'); ?>

      Thanks for help.