mindplay.dk

Names should be reasonably safe - it's a marginal case where two developers both decide to add a template named "blogpost", and it won't work either way, since you can't have two templates with the same name in the first place, so whether or not a GUID or some other new identifier enables you to identify the referenced template, you're still stuck with a problem that cannot (and should not) be solved.

The system makes assertions for every possible type of migration, ahead of attempting to apply them - for example, "add template" will abort the migration process if a template with the same name already exists, etc... so it's already reasonably hardened against poorly coordinated changes.

It's not intended (or expected) or designed to deal with poorly coordinated changes - it's not a "version control system", it can't "diff and merge" database changes or anything of that sort. What should work though, is if two developers independently add non-conflicting fields and templates, after coordinating and planning their work properly, and you'd be able to merge all the changes two ways. That's really all I was expecting or hoping for. It will permit other changes, like re-ordering of fields in a template, but the last applied order "wins", there is no (worthwhile) way to merge those changes.

> I'm guessing you'd convert a template_id to a template name, and a parent_id to a page path

Exactly right

> You are right that without knowing what the properties mean ahead of time, you'd probably need some interface that externalizes those values to something considered more portable

Probably the simplest (perhaps the only?) solution is to add hooks, so that field types can replace foreign keys with names or paths, and vice-versa.

It makes me wish though, that field values containing e.g. template/page/field IDs were identifiable as such... this could be done in at least three different ways:

1. Foreign key values could be somehow identifiable as foreign key values themselves - e.g. using some kind of value-type or value-object, or a URI scheme of some kind, like for example "pw://template/123" rather than simply "123".
2. Foreign key values could be made globally unique, e.g. using GUIDs, so that when you encounter a GUID, you can go back to the ProcessWire API and ask for the resource with that GUID, whether it be a Template, Field, Page or some other PW type.
3. Adding some kind of "schema" to Page models - e.g. allowing you to define what you're storing; so you would still be able to store an integer of course, but there would be some way to define what datatype you're storing, e.g. Template ID or Field ID, etc.

I think every approach would have it's own advantages and drawbacks. A URI scheme would be very developer-friendly, since URIs would be easily readable in var_dump() etc... GUIDs would have advantages in terms of data portability in general, since any resource created on any system would be unique identifiable - this would help in the case of migration and import/export of any type of resource in general... Adding an actual schema to Page models might be the most complicated solution (for developers) and I'm not sure if it has any advantages over the other two.

Thoughts?

I've been working hard on this module this week, and so far, so good - it's now recording and can repeat most changes to Fields, Templates and Fieldgroups.

One remaining issue is foreign keys... some field types store Template IDs, Page IDs or other types of ID within the Field data.

For the known field types such as PageReference, I can handle those cases explicitly, but for third-party types, there is no saying what they're storing, where, how or when.

The only real solution I can think of is some kind of hook that permits other modules to support this module explicitly, by implementing methods that clean up foreign keys and replace them with names, and vice-versa... but that's not really elegant and the module won't work generally with every third-party field type out of the box.

Any thoughts?

It's very little code, just take a look - you'll get it

Being able to return another event effectively makes it possible to write "recursive" events, as demonstrated in the example - with the added advantage that this happens in a do-while loop, which means it effectively works like recursion, but isn't... which means (potentially big) memory savings too, since the call-stack won't get any deeper from doing recursion - the call-stack will unwind after each "recursion", freeing up memory as it goes.

(That also potentially means it could be a lot harder to debug, and obviously there is much more work to do here - like an event history and call-stack for debugging purposes, but with the upshot of being able to turn it off in a production environment...)

I've written probably three dozen event sink prototypes over the past 6-7 years, and I think I may have finally found what I'm looking for.

Taking it one step further, check this out:

http://www.tehplayground.com/#dpQwxg8wF

The submit() method now (optionally) accepts a closure that produces the actual event - this allows for even more "laziness", since it can check if anybody is listening before the event is even created. (Because this is done by constructing the actual event object inside the event sink, this will only work for event objects with empty constructors, but I have the opinion that event constructors should always be empty, so I'm happy with that, and the performance advantage here could be huge.)

I also made it possible for listeners to return an event object, and the returned event will be broadcast immediately.

And above all, it's still simple

I just thought of a way to implement a statically-typed event sink in PHP - check it out:

http://www.tehplayground.com/#hh9WpisbY

This has a couple of interesting properties - primarily the fact that you can register listeners without passing a class-name or method-name as a string, but also the type-hints would get you IDE support, e.g. auto-complete and safe refactoring in PhpStorm.

I'm using a "hack" to delay auto-loading as well - it turns out, by parsing the type-hint with a regular expression applied to the string-representation of a ReflectionParameter (of the registered closure) you can avoid auto-loading until the registered closure actually gets called.

In other words, you can register listeners without auto-loading the class that implements the message it's listening for - that is, $sink->register(function(Foo $foo) {}) will not cause class Foo to auto-load, which means you could register all your global listeners of every module on start-up without the overhead of loading any additional classes.

I don't think this really fits with the existing concept or architecture in ProcessWire, but it seems like an extremely powerful idea, and I figured, if somebody is going to find this idea useful, this might be the right place to post it.

Anyhow, there it is

Just tagged version 1.0.2 which adds an "any" template, documenting all available Fields - this may be useful if you're writing code that works with any type of Page.

This discussion came up in the office again today with a designer, and I came across this thread while googling the subject.

I've been obsessing over this topic for a decade - I've hated WYSIWYG editors since they emerged, and I still hate them, for the same reasons.

All the technical issues aside, conceptually, giving clients complete control over HTML in my opinion is a bad idea to begin with. But also, I have the opinion that bare HTML is simply the wrong medium for content, because it provides no structure and no control; no means of enforcing any standards on content.

What I would really like to see, is something like Hallo or Aloha integrated with a template-based content editor - and I mean templates in the sense of structural HTML templates of some kind. This content editor would let you pick from templates pre-created by designers/programmers for the site. It would have semi-rich content areas here and there, meaning paragraphs, h1-h6, strong/em, a-tags, ul/ol/li - but not things like inline images or tables. Instead you would have nested "micro templates" for things like "image with caption", "image gallery", and other much richer types of content than you can feasibly do with just a WYSIWYG.

This would provide much better control than just an HTML editor - and it is possible now, with contenteditable being widely supported, and great in-place editors like Hallo and Aloha now available. Bigger or smaller sections in deeply nested "micro templates" could provide the right level of in-place editing for actual content, and modern user-friendly UI for everything else - without trying to bake everything into user-editable HTML markup ("mce_"-attributes in your HTML markup anyone?) and without losing the ability to actually update these templates on the server-side, reflecting changes on existing content.

I long and yearn for the day when I can actually present a client with a content editor that I myself could stand to use... ;-)

I spent a couple of hours hacking/tweaking Ryan's module today as well, did some searching and found this thread.

The access control model isn't working so well for my needs. I also need to grant certain users view/edit permissions from a branch down.

How come access control is tied to Templates?

I mean, it makes sense in some instances - and it's easy to extend the security model and build custom authorization rules on the front-end, but on the back-end it's not as easy as I had hoped for.

I think part of the issue is that RBAC frameworks normally have two access control components - rights and permissions. Rights are the binary grants you give to users or groups - meaning you either have a given right or you don't. Rights don't traditionally have any parameters - therefore rights usually are more granular in their descriptions, e.g. "can edit all posts" and "can edit posts created by me" are two distinct rights. The rights are independent from permissions, which are defined in code - permissions are checks you can actually perform, and they usually have access to a "context" which might consist of current user, current page, current template, today's date, and of course the current user's rights - anything you might want to use for security checks. An example of a permission would be "can edit post", which might check for several rights, e.g. "can edit all posts", and "can edit posts created by me" which it would check by getting the current Page object from the "context".

I get why it's implemented the way it is - for one, you want to be able to filter pages based on rights, at query-time. With a traditional RBAC model, that is not possible. But so many other things aren't possible with a fixed security model - it grows increasingly complex with increasing requirements, and it gets more rigid the more you extend it. Traditional RBAC can be extended indefinitely and never really increases in complexity because all of it's part function independently.

I think a more traditional RBAC model could be really flexible and incredibly scalable if implemented using hooks in ProcessWire - modules could easily add new permission methods and new security context properties with hooks.

I could see this making ProcessWire much more attractive as a platform for multi-tenant applications, where free-form access control is really crucial.

Sorry, it is closed-source for now. It will probably be open-source eventually, but I have to figure out how to dual-license it...

@Hari the idea of recording SQL statements and using that as a means of implementing migrations was already discussed earlier - it isn't viable. If you examine the queries that get executed, you will realize quickly why this approach doesn't really work. For one, template-names and field-names are not used in queries - the primary keys are. For another, some modules cause INSERT/DELETE/UPDATE statements that make changes to meta-data - these queries would be impossible to distinguish from queries that make changes to content.

I haven't made any progress on the module these past couple of months because I've been too busy, but the module is actually very far along. I haven't done any ProcessWire work recently, so I haven't had an occasion to get my head back in this project...

I still want to finish it, and it's actually not too far from usable.

Neat! I have used this UI pattern many times in the past - it'll be awesome to have this ready to just drop into a PW site

To your knowledge, does this UI pattern officially have a name? I usually refer to these as "stock lists" or "swap lists" when talking to clients...

@mvolke the module I'm working on synchronizes structure only - Templates, Fields and Fieldgroups. It stores captured model operations in JSON files.

I currently have no plans to synchronize content (Pages) though it may prove to be necessary at some point - because Pages are used for so many things, certain Pages may need to synchronized, for example options for drop-downs.

I'm having Ryan take a look at the module now, and his initial reaction was positive - I think we can make this work, and I think it'll work well.

This is only possible if you define the structure in code or in config files and not in the database and via a backend UI.

You would think that, but if you look at the ProcessWire codebase, the entire meta-model, with all possible operations, is encapsulated and supports hooks - so it is actually perfectly feasible to implement this in ProcessWire.

As proof of concept, I already have all Field operations captured and repeatable. Because this is implemented at the lowest API level, it is actually independent of controllers and user-interface - that is, if you were to build your own admin modules that (for some reason) make changes to any part of the meta-model, those changes would be correctly captured and would be repeatable, independently of any admin UI.

There is still substantial work to do on this module, but I would say it's about half-done at this point, and there are no major roadblocks to completion - the fundamental idea is proven and works, so it's a matter of building it out completely.

An "anything goes" type Inputfield where you override the render() and/or processInput() with hooks would likely only be useful for situations where you want to substitute the way the input is performed at the UI level... but that's the purpose of Inputfields in the first place, so the question would be: why not just make an Inputfield? 

The reason most UI/form/validation frameworks allow for some kind of composition, is that inheritance is hierarchical, and validation concerns aren't always hierarchical.

You can anticipate basic validations such "required" and build that into the base-class of all input-types, but you can't anticipate every possible cross-cutting validation concern.

To give a concrete example, class A might validate that the input looks like an e-mail address. Class B might extend A and also validate that the e-mail address is at a particular domain. Class C might extend A and validate that the e-mail address does not contain the word "noreply".

Now if class D needs to validate all of the above, suddenly you have an interesting problem - you can't extend both B and C, so now you have to refactor and most likely use static methods or code duplication to solve this issue.

Plus you end up with crazy class names like InputfieldEmail, InputfieldEmailAtDomain, InputfieldEmailAtDomainNotNoReply, etc.

For validation concerns, composition just seems like a more natural choice than aggregation, and it is the general approach taken to validation by most frameworks.

@ryan init() is too late for the load() hook, as Fields at least (not sure about Templates) have already loaded at that time. That's why I'm forced to attach my hooks during __construct() rather than init().

Like any regular module, I had my hooks defined in init() initially, but it doesn't work - I just retested, and it still doesn't work. The load() hook never fires.

I tried changing to non-static hooks in init() as well - no difference.

I also tried non-static hooks in __construct() and that doesn't work - the hooks don't fire at all...

I haven't worked on the project since we last discussed it, and it worked fine up until I switched to the master release where those hooks were missing.

Note that I'm attaching my handlers statically:

    public function __construct()
    {
        $this->addHookAfter('Fields::load', $this, 'hookFieldLoaded');
        $this->addHookAfter('Fields::saved', $this, 'hookFieldSaved');
        $this->addHookAfter('Fields::added', $this, 'hookFieldAdded');

        $this->addHookAfter('Templates::load', $this, 'hookTemplateLoaded');
        $this->addHookAfter('Templates::saved', $this, 'hookTemplateSaved');
        $this->addHookAfter('Templates::added', $this, 'hookTemplateAdded');
    }

Ryan, I just pulled the latest dev-branch... Templates::saved() and Fields::saved() etc. are now firing again, but they're now firing twice every time I save. What gives?

Edit: I attempted to debug_backtrace() to see where the call is coming from, but that just results in "Error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 123473921 bytes)" - something causes infinite recursion internally in debug_backtrace() so tracing this seems impossible...

Thank you Soma, this is educational

For a computed value that is a product strictly of values within the same context, physically materializing the computed value works fine - for computed values that are a product of any value outside of that context, that approach doesn't really work.

@Ryan: do you have any plans to open the query language to computed attributes via modules somehow?

To use a different case/example, consider a Currency field - you would need an external table of exchange rates in order to compute prices in the user's currency, say, when searching for products by price range. Because the exchange rates can change, storing the converted prices is a no-go.

In this case, you would need an expression like 'template=product, price.usd < 100', where the 'usd' property is the price-value dynamically converted to US dollars.

You can support $page->price->usd at run-time by exposing the price-value as an object, computing the value in PHP.

But is there any way to define and perform the SQL equivalent of that computation at query-time?

This makes a good "textbook" example of a Fieldtype/Inputfield pair that is easy to understand - maybe this should be referenced in the Wiki?

Question: is it possible to extend this with support for a computed (as opposed to stored) value, such as volume = width * height * depth ... adding the computed value would be easy enough, I suppose - but getting the same name to work in an SQL query via the PW query language, is that possible?

I'm not opposed to writing code. I'm actually not a fan of everything needing a graphical user interface, and validation is a good example of one of those features where you can expand indefinitely and never actually meet everybody's needs. It's also a good example of functionality that tends to be less generic and more application-specific.

How about a simpler API extension, like a type of module (or module-interface) that can validate a Page instance?

The UI would consist only of a list of checkboxes (one for each validator module) that you can select when editing a Template - no graphical configuration for validators, just a simple on/off switch that indicates whether it applies.

The interface would be something along the lines of validate(Page $page, ValidationErrorList $errors) and possibly a boolean argument indicating whether the Page was edited via the admin UI or via a custom (public facing) form.

This much simpler approach would enable you to build out not only individual Field validations, but also co-dependent validations, for example: option A is valid only when checkbox B is checked, etc...

Thoughts?

You most likely didn't configure the allowed template for your Page-field? If so, it should work.

Check the "Input" tab in the Field editor, under "Template of Selectable Page(s)".

Take some of that with a grain of salt, please - I just jotted the whole thing down on a diagram, and some of what I said doesn't quite make sense.

Mainly, I think I had to put it on a diagram to realize that URL, e-mail, color, and other string-based types can be correctly modeled as storage-types, if you wish - validating correctly formed URLs or e-mail addresses at the storage-level (Fieldtype) for example, is perfectly sound.

I will try to finish this diagram to better illustrate what I'm trying to say.

That all sounds great, Ryan

The only thing I have to disagree with, is the idea that fields, which are concerned with storage, should be concerned about the type of data contained in a string. A string is a string - as I think you will realize when you start thinking/designing for validations. I believe you will find it's actually eaiser to deal with strings when they are consistently represented, managed and stored by the same type.

Your HTML5 example is interesting - yes, URL and e-mail inputs are represented by distinct input-types. That is because HTML forms deal with validation, which is an input concern, not a storage concern - the fact that the value-attribute is always a string demonstrates that fact, because the value-attribute deals with storage. You also see evidence of this when you post the form - a text-input, URL, e-mail or textarea all get posted as the same data-type, a string; nothing indicates what the data-type on the form was, because that's an input concern, not a storage concern.

You can see this also in .NET, where data-types like e-mail and URL are defined by annotating a field with it's data-type, rather than by extending the String class into Email and URL classes - it was designed that way for the same reason: because you need the String type to express a storage concern, while the data-type is used for input-concerns such as validation.

Between storage and input concerns, some concerns may often seem to overlap - but it's important to distinguish "hard" storage concerns from "soft" input concerns. To use the simplest possible practical example, consider the "required" validation, an input concern, also known as "nullable", a storage concern - an Inputfield, dealing with input, might produce a nice human-readable error-message, while the Fieldtype, dealing with storage, really ought to throw an exception if the "nullable" constraint fails, since this has nothing to do with input. When dealing with storage, the consumer is the software - you should assume there is no user, because in some cases there really won't be.

Having to sanitize away invalid values in Fieldtypes is a symptom of this - if you handle input concerns at the storage level, you don't have much of a choice, short of throwing an exception, which isn't very user-friendly.

Another interesting overlapping concern is "uniqueness" or "distinct values" - this almost has to be implemented at the storage-level, and if I were to programmatically violate that constraint, I would expect an exception to be thrown. Rather than duplicating the uniqueness property at the input-level, an input-type should be able to go back to it's underlying storage-type via a standard interface and obtain a list of validations that are required to satisfy the "hard" constraints imposed by the storage-type, in addition to the "soft" constraints imposed by the input-type.

I realize changing any of this would break backwards compatibility, since you already have numerous storage-types that deal with input, so maybe you should consider pushing off the validation architecture to a major release, and consider cleaning up this portion of the API?

Templates in TinyMCE or CKEditor get baked into the mark-up... so they're static templates, not dynamic templates.

The particularly hard part here, is that whatever the format of the templates, they have to be supported on both the client and the server side - on the client to avoid a round-trip to the server every time you make a change, and on the server so that the templates can be pre-rendered for SEO purposes.

There is a chance the rendering the could be done client-side though - the editor would then send the data as well as the rendered content for the server to store. The server would no longer be responsible for rendering the templates at all, so you'd lose some control that way. You also would have no way to "mass update" rendered pages after making a correction to a template.

There is almost definitely no quick fix and no shortcuts to something like this. A really reliable solution has to have both client-side and server-side support for whatever you're using for templating...

If you have phone numbers written down, you usually have a name written down next to each number, right? This is metadata that puts the numbers in context. So yeah, data-types like phone numbers and e-mail addresses are necessary, but they usually have metadata associated with them to give them meaning. They're also both examples of data you can associate with a human relation. Images? not so much.

Anyhow, we're edging on that topic again, so I'll just say - I still think the real problem is WYSIWYG, which is too limited and simple. For every other thing the user would enter into a user interface, we have some kind of interface that provides guidance, meaningful constraints, structure, advice, context, process, or flow - except for the most important type of input, the one for the actual content... for that, we're perfectly happy leaving the user at his own devices, working with the raw media (HTML) without any real constraints or guidance, short of picking an image from a list - censoring certain dangerous HTML tags, or otherwise crippling the user-interface to limit their use of the medium. It's just no good.

Solutions involving codes or tags are usually chosen because they're easier to implement, not because they're "better" - after all, you wanted a WYSIWYG in the first place, and codes or tags are not "what you get". Somebody went through hell and high water to create a WYSIWYG, to enable the end users to work with the raw code without seeing the code - and then we just go and invent more code, and put it back in the WYSIWYG. To me that completely defeats the purpose of using a WYSIWYG in the first place.

Anyhow, I've said all this before, but I don't think working with the raw medium is the answer. Yes, content management should be visual - but somebody (me? if only I had more time) needs to invent a proper abstraction for content. Something that accommodates images, as well as other media-types, as well as any other solution-specific data-types, without having to climb Mt. Everest to get there.

On thing that comes to mind is these Bootstrap drag-and-drop design tools that are cropping up every week now, have you guys seen those? They provide a structured, guided, visual way to build entire web-pages out of components, which is not precisely what I'm looking for - but something along those lines. Something that allows you to get creative without editing raw HTML or entering codes or tags. Something that guides you through the creation of rich, structured content, without getting in your way - and where you can easily drop in another component template at the user's request.

I will keep dreaming of this in obstinance until I or somebody else creates it