Weekly update – 2 August 2024

[ryan]

This week I received some client specifications for project that indicated it was going to need a lot of fields, and fields within fields. There are a lot of tools for this kind of stuff in ProFields, but none that I thought would make this particular project "easy" per se. The closest fit was the Combo field, but with 40-50 or so subfields/columns within each of the "groups", it was still going to be kind of a pain to put together. Not to mention, just a lot of fields and subfields to manage. To add more challenge to it, there was another group of 30 or so fields that needed to be repeatable over any number of times (in a repeater field).

I wasn't looking forward to building out all these fields. That scale of fields/subfields is at the point where I don't really want to build and manage it interactively. Instead, I want to just edit it in a code editor or IDE, if such a thing is possible. With any large set of fields, there's often a lot of redundant fields where maybe only the name changes, so I wanted to be able to just copy and paste my way through these dozens of fields. That would save me a lot of time, relative to creating and configuring them interactively in the admin. 

Rather than spending a bunch of time trying to answer the specifications with existing field types, I ended up building a new Fieldtype and Inputfield to handle the task. Right now it's called the "Custom" field (for lack of a better term). But the term kind of fits because it's a field that has no structure on its own and instead is defined completely by a custom JSON file that you place on your file system. (It will also accept a PHP file that returns a PHP array or InputfieldWrapper). 

Below is an example of defining a custom field named "contact" with JSON. The array keys are the field names. The "type" can be the name of any Inputfield module (minus the "Inputfield" prefix). And the rest of the properties are whatever settings are supported by the chosen Inputfield module, or Inputfield properties in general. 

/site/templates/custom-fields/contact.json

{
  "first_name": {
    "type": "text",
    "label": "First name",
    "required": true,
    "columnWidth": 50
  },
  "last_name": {
    "type": "text",
    "label": "Last name",
    "required": true,
    "columnWidth": 50
  },
  "email": {
    "type": "email",
    "label": "Email",
    "placeholder": "person@company.com",
    "required": true
  },
  "colors": {
    "type": "checkboxes",
    "label": "What are your favorite colors?",
    "options": {
      "r": "Red",
      "g": "Green",
      "b": "Blue"
    }
  },
  "address": {
    "type": "fieldset",
    "label": "Address",
    "children": {
      "address_street": {
        "type": "text",
        "label": "Street"
      },
      "address_city": {
        "type": "text",
        "label": "City",
        "columnWidth": 50
      },
      "address_state": {
        "type": "text",
        "label": "State/province",
        "columnWidth": 25
      },
      "address_zip": {
        "type": "text",
        "label": "Zip/post code",
        "columnWidth": 25
      }
    }
  }
}

The result in the page editor looks like this:

The actual value from the API is a WireData object populated with the names mentioned in the JSON definition above. If my Custom field is named "contact", I can output the email name like this:

echo $page->contact->email;

Or if I want to output everything in a loop:

foreach($page->contact as $key => $value) {
  echo "<li>$key: $value</li>";
}

After defining a couple large fields with JSON, I decided that using PHP would sometimes be preferable because I could inject some logic into it, such as loading a list of 200+ selectable countries from another file, or putting reusable groups of fieldsets in a variable to reduce duplication. The other benefits of a PHP array were that I could make the field labels __('translatable'); and PHP isn't quite as strict as JSON about extra commas. So the module will accept either JSON or PHP array. Here's the equivalent PHP to the above JSON:

/site/templates/custom-fields/contact.php

return [
  'first_name' => [
    'type' => 'text',
    'label' => 'First name',
    'required' => true,
    'columnWidth' => 50
  ],
  'last_name' => [
    'type' => 'text',
    'label' => 'Last name',
    'required' => true,
    'columnWidth' => 50
  ],
  'email' => [
    'type' => 'email',
    'label' => 'Email address',
    'placeholder' => 'person@company.com'
  ],
  'colors' => [
    'type' => 'checkboxes',
    'label' => 'Colors',
    'description' => 'Select your favorite colors',
    'options' => [
      'r' => 'Red', 
      'g' => 'Green',
      'b' => 'Blue'
    ],
  ],
  'address' => [
    'type' => 'fieldset',
    'label' => 'Mailing address',
    'children' => [
      'address_street' => [
        'type' => 'text', 
        'label' => 'Street'
      ],
      'address_city' => [
        'type' => 'text',
        'label' => 'City',
        'columnWidth' => 50
      ],
      'address_state' => [
        'type' => 'text',
        'label' => 'State/province',
        'columnWidth' => 25
      ],
      'address_zip' => [
        'type' => 'text',
        'label' => 'Zip/post code',
        'columnWidth' => 25
      ]
    ]
  ]
];

The downside of configuring fields this way is that you kind of have to know the names of each Inputfield's configuration properties to take full advantage of all its features. Interactively, they are all shown to you, which makes things easier, and we don't have that here. There is yet another alternative though. If you define the fields like you would in a module configuration, your IDE (like PhpStorm) will be able to provide type hinting specific to each Inputfield type. I think I still prefer to use JSON or a PHP array, and consult the docs on the settings; but just to demonstrate, you can also have your custom field file return a PHP InputfieldWrapper like this:

$form = new InputfieldWrapper();

$f = $form->InputfieldText;
$f->name = 'first_name';
$f->label = 'First name';
$f->required = true;
$f->columnWidth = 50;
$form->add($f);

$f = $form->InputfieldText;
$f->name = 'last_name';
$f->label = 'Last name';
$f->required = true;
$f->columnWidth = 50;
$form->add($f);

$f = $form->InputfieldEmail;
$f->name = 'email';
$f->label = 'Email address';
$f->required = true;
$f->placeholder = 'person@company.com';
$form->add($f);

// ... and so on

Now that this new Fieldtype/Inputfield is developed, I was able to answer the clients specs for lots of fields in a manner of minutes. Here's just some of it (thumbnails): 

There's more I'd like to do with this Fieldtype/Inputfield combination before releasing it, but since I've found it quite handy (and actually fun) to build fields this way, I wanted to let you know I was working on it and hope to have it ready to share soon, hopefully this month. Thanks for reading and have a great weekend! 
 

[wbmnfktr]

So... this is Functional Fields Ultimate Advanced Pro Extended.

Wow!

But to fully understand this: It won't create any real field in the backend a client could change/configure.
Correct?

 

Have a great weekend @ryan!

[Kiwi Chris]

This approach to defining fields looks not unlike what Bernhard has done with Rock Migrations, although admittedly, in his case they're not subfields.

I haven't checked to see if the Combo fieldtype is compatible with Rock Migrations, but if it is then that might be another way of achieving this, with the bonus that Rock Migrations lets you have it both ways; you can define your fields in code or via the ProcessWire admin UI.

If Combo isn't currently compatible with Rock Migrations, it would be worth ensuring that it is.

I think there's this overlap because I'm not sure if @bernhard uses ProFields, and I'm not sure that @ryan uses Rock Migrations as you each have different workflows, but in this case it might be worth checking out each other's efforts.

A couple of options I can think of:

• Talk to Bernhard about including Rock Migrations in the core. Pros - it's always there. Cons. It's always there, and for simpler sites where people are happy to use admin UI may be unnecessary.
• Build an empty site profile just with Rock Migrations installed. That way when someone needs to start a new project with this functionality they can get it straight away when they set up a new instance of ProcessWire. Pros - There when you need it, not when you don't. Cons - if you've already set up your site profile and realise you need it can't go back and reinstall profile (but still really easy to install the module from the directory).

 

[ryan]

@wbmnfktr

Quote

But to fully understand this: It won't create any real field in the backend a client could change/configure.

Each "custom" field is just one "real" field in the admin, with any number of subfields within it. The subfields are not themselves ProcessWire fields. Instead, the subfields are similar to subfields/columns in Combo fields. If using the JSON configuration, you can technically edit it in the admin (Setup > Fields) if you want to, rather than maintaining a file. But I figure most probably won't be editing JSON from their browser. You can however configure other settings related to the custom field in Setup > Fields > your_custom_field, such as usual ones (label, description, icon, etc.), plus others like entity encoding when output formatting is on, whether the subfields should have an outer wrapping fieldset (same as the hideWrap setting in Combo), and more. 

@Kiwi Chris I am not that familiar with Rock Migrations, but I'm thinking we might be talking about different things as this is a Fieldtype and Inputfield, not a module related to migrations. It doesn't create any new fields or anything like that, but it lets you define the structure of subfields within a "custom" field.

[BrendonKoz]

Seemingly quite similar to @ukyo's solution via Mystique. I have a local SQLite database that is a mirror of an external vendor's data for our event calendar (I use an API to sync their external with our local so that I can more easily integrate items into our website rather than constantly referring to their API). By parsing available data in the SQLite database (and making it available via a local REST URL) to determine options in the field, I can then use the chosen values as a filter to then query/display matching records in the SQLite database. This particular field was explicitly allowed as a RepeaterMatrix Fieldtype, and then added, in my case.

My Mystique template:

/site/templates/configs/Mystique.calendar.php

<?php

namespace ProcessWire;

/**
 * Resource: Dynamic Calendar settings field
 */
return function ($page = null, $field = null, $value = null) {
	// Get the data to feed possible selection values
	$audiences  = json_decode(@file_get_contents(input()->httpHostUrl() . '/calendar/audiences.php?activeMerge=1'), true);
	$categories = json_decode(@file_get_contents(input()->httpHostUrl() . '/calendar/categories.php?activeMerge=1'), true);

    $fields = [
		'event_type' => [
			'label' => __('Event Type'),
			'type' => 'InputfieldCheckboxes',
			'options' => [
				'events' => 'Library Events',
				'meetings' => 'Public Meetings'
			],
			'required' => true,
			'columnWidth' => 33
		],
		'limit_days' => [
			'label' => __('Upcoming Days to Check'),
			'type' => Mystique::INTEGER,
			'min' => 1,
			'max' => 60,
			'defaultValue' => 1,
			'initValue' => 1,
			'inputType' => 'number',
			'required' => true,
			'columnWidth' => 34
		],
		'limit_events' => [
			'label' => __('Maximum Events to Display'),
			'type' => Mystique::INTEGER,
			'min' => 0,
			'defaultValue' => 0,
			'initValue' => 0,
			'inputType' => 'number',
			'required' => true,
			'columnWidth' => 33,
			'notes' => __('Per event type. Leave at 0 for no limit.')
		],
		'not_found_msg' => [
			'label' => __('Message if No Matches Found'),
			'description' => __('This will be displayed if no events match the given criteria. Leave blank for no message.'),
			'type' => Mystique::TEXT,
			'columnWidth' => 100,
			'notes' => __('HTML cannot be used here. Markdown is allowed.')
		],
		'private' => [
			'label' => __('Include Private (Calendar) Library Events'),
			'type' => Mystique::CHECKBOX,
			'value' => false,
			'showIf' => [
				'event_type' => '=events'
			],
			'columnWidth' => 33,
			'notes' => __('This is not recommended.')
		],
		'output_style' => [
			'label' => __('Output Styling'),
			'description' => __('Choose the presentation style for any events that will be displayed.'),
			'type' => 'InputfieldRadios',
			'options' => [
				'list' => 'A Simple Bulleted List',
				'grid' => 'A Grid-Like Display'
			],
			'showIf' => [
				'event_type' => '=events'
			],
			'columnWidth' => 33
		],
		'show_images' => [
			'label' => __('Display associated event images?'),
			'type' => Mystique::CHECKBOX,
			'showIf' => [
				'output_style' => '=grid'
			],
			'columnWidth' => 34,
			'notes' => __('Images are always taken directly from the calendar and cannot be properly optimized.')
		],
        'fieldset' => [
            'label' => __('Filters for Library Events'),
            'type' => Mystique::FIELDSET,
			'collapsed' => true,
			'showIf' => [
				'event_type' => '=events'
			],
            'children' => [
				'audiences' => [
					'label' => __('Audiences'),
					'description' => __('Only show:'),
					'type' => 'InputfieldCheckboxes',
					'options' => $audiences,
					'defaultValue' => '',
					'columnWidth' => 20,
					'notes' => __('Leave blank to show all.')
				],
				'categories' => [
					'label' => __('Categories'),
					'description' => __('Only show:'),
					'type' => 'InputfieldCheckboxes',
					'options' => $categories,
					'defaultValue' => '',
					'optionColumns' => 3,
					'columnWidth' => 80,
					'notes' => __('Leave blank to show all.')
				]
			]
		]
    ];

    return [
        'name' => 'calendar_events',
        'title' => 'Calendar of Events',
        'fields' => $fields
    ];

};

Which then renders:

I could've looped the field generation instead of the values generation, similarly to what Ryan did above, but that wasn't my use case here.

Regardless of solution, it's a very handy way to generate fields, options, and values.

[wbmnfktr]

33 minutes ago, ryan said:

Each "custom" field is just one "real" field in the admin, with any number of subfields within it. The subfields are not themselves ProcessWire fields. Instead, the subfields are similar to subfields/columns in Combo fields.

Ok, now I get it and that's what I was thinking about how it would work.

@Kiwi Chris is right. There is some overlap here. At least it looks quite similar, yet RockMigration goes a different route and creates real fields, templates, and relationships between them aka adds to fields to templates.

@BrendonKoz , I never thought about using Mystique that way and create such a big set of fields with it.

 

Learned new things today. Again.

[Kiwi Chris]

@ryan What I was thinking looking at your syntax, is that it looks a lot like Rock Migrations syntax. If Combo pretty much does what you need but is too slow defining through the UI with a lot of subfields, if Rock Migrations fully supports Combo, then it would be very easy to define combo fields in code like you've done.

Although Rock Migrations is primarily about migrations, it takes a definition in code of the state of fields and templates that looks a lot like what you've illustrated, that's easy to move between dev and live environments, so it's absolutely possible to use it to define fields, and you can be selective about it. 

If there are fields or templates you want to maintain the traditional way via the UI, then don't include them in the migration definition files.

It's not all or nothing. 

On the other hand, because it works with fieldtypes, if you start out doing things via code, then decide you want to make some tweaks via the admin UI, that's possible, as it displays all the code for a field definition, that you can copy and paste back into the migration file, so it can work both ways.

I'm not sure if it needs some tweaks for your use case scenario, but I think if Combo does what you need but is too cumbersome to define via the UI with a lot of subfields, then as long as it works with Rock Migrations, then it would probably be an existing way to achieve something very similar.

As always with ProcessWire there is often more than one way to achieve the same outcome. ?

[ryan]

@BrendonKoz Looks like Mystique defines fields (subfields?) in a very similar way, in that it is also using the Inputfields settings/API (as an array) to define them. Though looks like the use of the "type" property might be something different, since it is mapping to a constant in the module rather than the name of an Inputfield. Maybe those constants still map to Inputfield names. Mystique looks good, perhaps I could have just used Mystique to solve the client's request! Though admittedly I have a lot of fun building Fieldtypes and Inputfields, and I'm guessing there will be just as many differences as similarities, but glad to see there are more options. 

Quote

What I was thinking looking at your syntax, is that it looks a lot like Rock Migrations syntax. 

@Kiwi Chris The syntax I'm using is just the Inputfields API. Looks like the Mystique field above also uses the same API as well. Nice that there is some consistency between these different use cases.

Quote

If Combo pretty much does what you need but is too slow defining through the UI with a lot of subfields, if Rock Migrations fully supports Combo, then it would be very easy to define combo fields in code like you've done.

That's good to know. Though in this case it was also that I needed a hierarchy of fieldsets which Combo doesn't support. Plus wanted something a little more lightweight, because there were so many fields and the client just needs the ability to store them and not so much to query them. 

Quote

If there are fields or templates you want to maintain the traditional way via the UI, then don't include them in the migration definition files.

I don't know if Combo is compatible with the migrations module or not, but always good to know about the options. I usually do prefer to manage stuff in the admin, but this is a case where it just seemed like it was not going to be fun.

 

[Jonathan Lahijani]

As others have mentioned, this is quite similar to the Mystique field which I'm a fan of and has the feel of how custom fields work in WordPress (more "on the fly").  I use Mystique for my matrix-based page builder block options which is a great fit.  Will consider switching to this since it will be first-party and more integrated.

Does it store the saved input data in json as well?

I'm guessing this doesn't work with repeaters or images, correct?

[Jonathan Lahijani]

Another question:  In Mystique, you are able to detect the page requesting the field, and as a result, modify the fields that actually get displayed.

This is very useful because it allows you to have one Mystique field but have it "morph" depending on the context.

On my sites, I typically only have 1 Mystique field called 'options'.  I then assign that to various templates that need it (even specific matrix-types like for my page builder).  Then when the field is loaded when editing a page or a matrix-type, I can "build" the field based on what template requested it because the $page variable is available.  My 'gallery' matrix-type/page builder block will have fields that are gallery configuration related, my 'video' block will have video configuration fields, but they are all still technically the single Mystique 'options' field.

@ryan So in your example of /site/templates/custom-fields/contact.php, is it able to determine what page requested it?

[MrSnoozles]

This is super useful, as is Mystique. Will it support searching and more complex fields, like repeaters?

12 hours ago, Jonathan Lahijani said:

In Mystique, you are able to detect the page requesting the field, and as a result, modify the fields that actually get displayed.

Agree, this is very useful.

[poljpocket]

On 8/2/2024 at 10:06 PM, ryan said:

Rather than spending a bunch of time trying to answer the specifications with existing field types, I ended up building a new Fieldtype and Inputfield to handle the task. Right now it's called the "Custom" field (for lack of a better term). But the term kind of fits because it's a field that has no structure on its own and instead is defined completely by a custom JSON file that you place on your file system. (It will also accept a PHP file that returns a PHP array or InputfieldWrapper). 

Very interesting and reading through it, I was also instantly reminded of Mystique. And also, why I am almost never using fields like it. That is because of it's almost complete lack of support for selectors.

Looking at your example, I am immediately thinking of problems like this one (people which have an email set, e.g. to filter contacts for a mail send operation):

<?php

$emailContacts = $pages->find('template=person,contact.email!=');

Does your implementation support this? Mystique doesn't and you are stuck with loading everything and then filtering manually once the JSON is decoded.

Edited August 3 by poljpocket
typo

[Jan Romero]

Obviously I haven’t used this myself I feel like there is some confusion in this thread regarding the storage model? My understanding is that the data in the database looks the same as any other field, ie. each field has its own table with subfields being individual columns. AFAIK that’s how Profields Combo works, which this has been compared to. That would alleviate @poljpocket’s concern about selectors. One would simply use the usual dot syntax.

As with all dynamic db schemas the issue then becomes migrating data inbetween schema changes. „Stock“ ProcessWire helps you here by only allowing safe changes through the UI and/or showing warnings if data loss may occur. Such safeguards are presumably absent here, but I guess you’re supposed to know what you’re doing when dealing with code anyway. 

Edited August 3 by Jan Romero

[wbmnfktr]

One thing that just came into my mind... I need support for ImportPagesCSV.

No matter if Combo, Mystique, or the upcoming new fieldtype.

I import thousands of pages each month via CSV upload using ImportPagesCSV - fortunately older projects and real fields so for now no problems.

(Yes, never tried to import into Combo fields, which I probably should.)

[ryan]

@Jonathan Lahijani

Quote

Does it store the saved input data in json as well?

Sort of, but it's not just that. I'll go into the details once I've got the storage part fully built out. 

Quote

I'm guessing this doesn't work with repeaters or images, correct?

It supports any Inputfield type that can be used independently of a Fieldtype. Meaning, the same types that could be used in a module configuration or a FormBuilder form, etc. It can be used IN repeater fields (tested and working), and likely in file/image custom fields as well (though not yet tested). But you wouldn't be able to use repeaters or files/images as subfields within the custom field. Files/images will likely be possible in a later version though, as what's possible in Combo will also be possible here.

Quote

So in your example of /site/templates/custom-fields/contact.php, is it able to determine what page requested it?

Yes, if you use the .php definition option (rather than the JSON definition option) then there are $page and $field variables in scope to your .php definition file. 

Quote

Will it support searching and more complex fields, like repeaters?

@MrSnoozles Searching yes. Can be used in repeaters (yes), but can't have repeaters used as subfield/column types within it. 

Quote

$emailContacts = $pages->find('template=person,contact.email!=');
Does your implementation support this? Mystique doesn't and you are stuck with loading everything and then filtering manually once the JSON is decoded.

@poljpocket Yes. While I've not fully developed the selectors part of it yet, the plan is that it will support selectors with subfields for searching from $pages->find(), etc. 

@wbmnfktr The API side of this should work like most other Fieldtypes with subfields, so I think it should be relatively simple for importing and may not need any special considerations there, but I've not tried using it with a module like ImportPagesCSV yet. 

 

[ryan]

Since it looks like there is a some crossover with Mystique, and it also looks like that module is active and supported, I'll release this module in ProFields instead. That way it's not competing with Mystique, which looks to already be a great module. I'll focus on making Custom fields have some features that maybe aren't available in Mystique, like ability to query from pages.find and perhaps supporting some more types, etc. Plus, the new module fits right in with the purpose of ProFields and a good alternative to Combo where each have a little bit different benefits to solve similar needs.

[elabx]

Would if be possible to base the array configuration based on a page tree branch? For example, to allow end users to define multiple taxonomies (like Skyscrapers City, Height, etc). This way the page that contains the custom field has "dynamic inputfield page reference fields" based on the tree branch. 

[Ivan Gretsky]

On 8/4/2024 at 5:59 PM, ryan said:

It supports any Inputfield type that can be used independently of a Fieldtype. Meaning, the same types that could be used in a module configuration or a FormBuilder form, etc. It can be used IN repeater fields (tested and working), and likely in file/image custom fields as well (though not yet tested). But you wouldn't be able to use repeaters or files/images as subfields within the custom field. Files/images will likely be possible in a later version though, as what's possible in Combo will also be possible here.

It might be a great idea to extend this fieldtype with another 2 inputfields:

1. One to be able to store images / files in a another field, selected in inputfields' settings. And to strore some kind of reference to that field. Not quite sure how to get it via API though. This way we can still have images manipulated from the custom fieldtype.
2. Another one is json-based repeater. Something like Textareas or Multiplier. It will cover most of the repeatable needs for me at least. Will Textareas or Multiplier work in custom fieldtype by the way? the did not in Mystique.