Weekly update – 22 July 2022

[ryan]

In addition to site development work and ProcessWire core development, this week I upgraded my OS X version (from Mojave to Monterey), my PhpStorm version (from 2018 to 2022), and my Apache/PHP/MySQL environment versions. I tend to wait a bit too long between upgrades on my computer and so I ended up with a lot of things to resolve, and a still few remaining. While waiting for upgrades to install, I randomly came across one of the first sites I ever designed/developed out of college, working for Grafik in the year 2000: https://americanhistory.si.edu/subs/ ... I can't believe it's still there. It looks really dated now (it's 22 years old), but reminded me of how much things have changed in web design/development. While I'd been developing sites for a few years before this, it was just a hobby, and it wasn't until this time that it was my job.

Back in 2000 there weren't a lot of people that knew how to create websites and it always felt like you were breaking new ground. Internet Explorer was king (and nobody liked it then either), Google was just a small startup, but AltaVista, InfoSeek and Yahoo were big. 

Sites were developed in a way that would make you cringe now. I don't think we used CSS hardly at all, but we did use tables for everything layout related. There was no such thing as "mobile" (the iPhone didn't come till 7 years later). There was no such thing as "responsive" layout and accessibility was pretty much an unknown. Most of the time we used images for a lot more than was appropriate (headlines and much more) because HTML and HTML fonts were so limited. It all seems so primitive now but what's the same is that it was fun then and it's fun now, actually it's more fun now. I don't have any point here, just that it's funny to look back at how much as changed.

Last week I mentioned that we're likely to upgrade CKEditor 4 to CKEditor 5 sometime in the next year. There were several comments about that and so just wanted to talk a little more about it. 

First off, I really like CKEditor 4, it's been a great fit for ProcessWire. If the company behind it was going to continue building and supporting version 4 after 2023 then we'd likely stick with it. But the CKEditor 4 end-of-life is sometime in 2023 and I don't think it's an option to stick with it (in the core) after the developer of it is no longer updating or supporting it... at least not long term.

While CKEditor 5 is a different animal than CKEditor 4, it's also still the closest upgrade path from CKEditor 4, and I'm hopeful it will be able to serve as a near drop-in replacement from the perspective of users (only). My hope is that by the time we've completed the upgrade to CKE 5, our clients won't necessarily know the difference or have to re-learn anything, unless they want to take advantage of something new exclusive to CKE 5.

From my perspective as a developer integrating CKEditor 5 into ProcessWire, the development side is not a drop in replacement for CKE 4 (not even close), as all supporting code will have to be redeveloped. By supporting code, I mean things like the code in the InputfieldCKEditor.module file, the code for our custom CKE plugins (pwimage and pwlink), as well as anything else development related that is referring to CKEditor. There's no doubt it'll be a lot of work. But the rich text editor is one of the most important input types in the ProcessWire admin, so it's fine, it's worth putting a lot of time into. 

As for CKEditor 5 being bloated relative to CKEditor 4, I very much doubt that's the case. It was a complete rewrite, the folks at CKEditor know what they are doing, and it's safe to assume it's even more optimized and streamlined than CKE 4. In terms of size, the download for CKE 4 and CKE 5 are both 1.7 megabytes. As I understand it, they started with a new codebase for CKEditor 5 in part to start fresh and avoid legacy bloat. So I see this upgrade as being the opposite of bloat. 

So what happens with CKEditor 4 in ProcessWire when it likely is replaced with CKEditor 5? So long as CKE 5 can be a near drop in replacement for our users, and for the markup it generates, then the CKE 4 module will move out of the core and into an optional module you can install from the modules directory, when/if someone wants it. On the other hand, if the transition is not completely clean between versions then we may end up supporting both in the core for a short period of time. Though I'm hopeful this won't be necessary.

There are some other interesting looking editors out there that have been mentioned, and it'd be nice to have more input options available. I see something like Imperavi's Article as a good option for some but not a replacement for our current rich text editor. At least I know my clients would not be happy to have that big of a change occur from a PW version upgrade. Likewise, something like the Easy Markdown Editor is a great option for some, and I'd like to be able to install a module for that and use it in some cases. But folks used to using CKEditor 4 in their work would not be happy to have that big of a change either. 

CKEditor 4 works really well for what it does, and I think the goal has to be that clients using CKEditor 4 now should be able to continue doing what they are doing with as few changes to their workflow as possible. I'm hopeful we'll be able to get there with CKEditor 5, while also gaining some benefits in the process. Where other input options make a lot of sense is when building a new site where there aren't already users depending on one input method or another. And it may be that at some point (sooner or later) it will make sense for ProcessWire to have another textarea input option that's different enough from CKE to make it worthwhile. But in my opinion, that would be a potential additional option, not a replacement, as CKE is pretty well established and expected in PW. 

 

[wbmnfktr]

36 minutes ago, ryan said:

I don't have any point here, just that it's funny to look back at how much as changed.

That's actually a very good point. And a very nice addition to the usual updates, content and discussions here as well.

[pwired]

From my perspective as a developer integrating CKEditor 5 into ProcessWire ...

Seems to be the only perspective ...

[Kiwi Chris]

1 hour ago, ryan said:

Back in 2000 there weren't a lot of people that knew how to create websites and it always felt like you were breaking new ground.

There weren't very many CMS/CMF platforms either. I started out with SSIs to start reducing redundancy and duplication, then ended up starting my own CMS around 2000, but I picked a loser in terms of development language, and I realised trying to maintain a CMS for a modest number of clients on my own wasn't really efficient use of time.

Someone introduced me to ProcessWire in 2015, and since then nearly all my websites have been converted to it.

Web development and database development have progressed together for me, so ProcessWire's separation of content and presentation, and strong access control are critical for me.

I evaluated my other CMS platforms before I settled on ProcessWire and I simply didn't like them as they either had a steep learning curve or were too output focused or both. I wouldn't use Word to try to build a relational database, nor would I use WordPress.

I think if there is one major thing still on my wish list, it would be the ability to have a UI using ProcessWire inputfields to create data entry forms for arbitrary SQL tables.

A common scenario for me is converting offline databases (often Microsoft Access) to cloud based alternatives. Usually it's possible to restructure everything within ProcessWire's pages model, but sometimes I just want to be able to dump the data across and build a UI quickly with no need for data to exist within a page model. I've been playing around a bit looking at building module(s).

I'd love the goodness of Lister/(Pro), but instead of picking a template, pick an arbitrary SQL table and specify fields from it, set access control etc in the same way Lister Pro woks, and then have I guess what I'd call an SQLTemplate where you'd pick an arbitrary SQL table, select fields from it, and for each one specify an inputfield, within the contraints of the SQL field type.

Of course none of this data would be accessible within ProcessWire's page structure out of the box, however there's already $database->query() to execute an arbitrary SQL query and return a result set, so using arbitrary SQL data within templates is already easy, so the only bit missing is a UI to manage arbitrary SQL tables as easily as ProcessWire pages in admin, but it's already halfway there with inputfields that don't care where their data comes from.

[MarkE]

On 7/22/2022 at 11:28 PM, Kiwi Chris said:

so the only bit missing is a UI to manage arbitrary SQL tables as easily as ProcessWire pages in admin, but it's already halfway there with inputfields that don't care where their data comes from.

I'm wondering if this could be achieved with a module...?

[ryan]

@Kiwi Chris @MarkE I think that probably could be achieved with a Fieldtype+Inputfield module pair. It sounds kind of like what FieldtypeCombo does except that you define the table structure in Combo, rather than Combo reading an existing table structure. No doubt it would be simpler than Combo since it would just read the table for its settings. Something like translating MySQL VARCHAR column to an InputfieldText, and a TEXT/MEDIUMTEXT column to a Textarea or CKE, and perhaps ENUM/SET columns to InputfieldRadios/InputfieldCheckboxes... that sort of thing. Could be fun to build! 

[MarkE]

14 minutes ago, ryan said:

Something like translating MySQL VARCHAR column to an InputfieldText, and a TEXT/MEDIUMTEXT column to a Textarea or CKE, and perhaps ENUM/SET columns to InputfieldRadios/InputfieldCheckboxes... that sort of thing. Could be fun to build! 

Good plan. I was also thinking that you could use config inputfields to define the mapping (as types allow) and to specify a custom form layout. Not sure if I will get around to trying it*, but maybe @Kiwi Chris might like to. My attempt at documenting these types of modules might help.

*I am pondering rewriting ProcessDbMigrate to operate with its own database (a bit like TextformatterHannaCode) but, even if I do, it is unlikely to happen in the near future.

[Kiwi Chris]

7 hours ago, ryan said:

It sounds kind of like what FieldtypeCombo does except that you define the table structure in Combo, rather than Combo reading an existing table structure. No doubt it would be simpler than Combo since it would just read the table for its settings. Something like translating MySQL VARCHAR column to an InputfieldText, and a TEXT/MEDIUMTEXT column to a Textarea or CKE, and perhaps ENUM/SET columns to InputfieldRadios/InputfieldCheckboxes... that sort of thing. Could be fun to build! 

@ryanFieldtypeCombo comes pretty close, but like you say, you define the table structure with the fieldtype. Also as a fieldtype it needs to be associated with a template, that links into the ProcessWire page tree, which isn't always desirable with SQL tables from an existing database.

I'm thinking there would probably need to be two configurable Process modules. An ProcessSQLLister and ProcessSQLForm module. Names are just suggestions ProcessSQLTemplate could be another option however as it's not an output template, it's an admin UI for editing arbitrary SQL data.

ProcessSQLLister would be similar to Lister Pro, except for selecting a source, you'd select an SQL table, then choose what fields from the table to display in the lister. Dealing with lookup fields which are similar to page reference fields, but can refer to an arbitrary database table would probably be the biggest challenge, but the basic lister should be pretty straightforward. Edit functionality would depend on a related ProcessSQLForm being defined for the specified table.

ProcessSQLForm would involve picking an SQL table, then selecting fields from the table to include on the form in the order desired, and mapping and configuring a sane InputField type depending on the SQL field type. @kixe has already made FieldtypeSelectExtOption which can handle lookups from arbitrary SQL tables, so that InputField should be able to handle relational reference fields, and core InputFields should be able to map to other SQL fields types.

For both, access control at the table level would be desirable. While access control at the field level would be technically possible, I don't think it's necessary at least initially, especially if it adds complication since the fields are SQL fields rather than ProcessWire fields.

I can try to mock something up, but if someone else beats me to it, then feel free to go ahead.

I think ProcessWire has all the nuts and bolts to do this, it's just a case of putting them together.

[bernhard]

8 hours ago, Kiwi Chris said:

I think ProcessWire has all the nuts and bolts to do this, it's just a case of putting them together.

This statement is true for almost any use case ? 

Back to topic: I like your idea! It would be handy for my workflows as well. I'm developing almost anything that I do in a modular reusable way. That's possible and almost as quick as doing it the regular way thanks to RockMigrations. But storing data for non-public data (pages that are not viewable on the frontend) is one of those things that are a little tedious to build. You always have to create a parent template and a child template, then link both together, etc...

Though I'm not deep enough in the topic to know how exactly that could work. For example I've never used the ProfieldsTable module which sounds like it's doing something similar? At least for my case I'd also need the possibility to create custom tables, not only read them.

Or is that exactly what the table field does and you are talking about only reading existing tables? Couldn't we build somthing that supports both needs? Just brainstorming/asking... ? 

[Kiwi Chris]

17 minutes ago, bernhard said:

Though I'm not deep enough in the topic to know how exactly that could work. For example I've never used the ProfieldsTable module which sounds like it's doing something similar? At least for my case I'd also need the possibility to create custom tables, not only read them.

Or is that exactly what the table field does and you are talking about only reading existing tables? Couldn't we build somthing that supports both needs? Just brainstorming/asking... ? 

The ProFields Combo field and Table fields create SQL tables behind the scenes, but they're still fieldtypes, so are expected to be added to a ProcessWire template and are linked to the page tree. Sometimes that's not necessary or desirable.

With regards to creating custom tables, I'm quite happy writing SQL statements for this, or using the generated CREATE TABLE statements with a database export from mySQL, or for that matter even other dialects of SQL, with a bit of editing if necessary. It's not hard to write a CREATE TABLE statement initially, then an ALTER TABLE statement if subsequent changes are needed.

I think this would play nice with your RockMigrations module to generate SQL schema changes in code, and I don't want to turn ProcessWire into mySQLAdmin or Adminer which is installed with Tracy Debugger, but it would be nice to have an end user friendly UI to view and edit custom SQL tables in the ProcessWire admin.

You have made me thing of something I hadn't considered, about how to migrate SQL listers and editing forms, if the InputField settings or fields to display in a lister change, but if they're accessible via the API, it should be possible to manage migrations in a similar way to ProcessWire native fields and templates.

[gornycreative]

On 7/22/2022 at 2:55 PM, ryan said:

Sites were developed in a way that would make you cringe now.

I just remember endless toil trying to get standard appearances across browsers - still an issue, but nothing like the nightmare of trying to get Opera, Netscape, IE and Safari to be friends. And so much flash animation. So much.

And then XHTML was going to be a thing, and UML was going to make everything play nice.

And then those fun proprietary tools like coldfusion - I thought CFML was pretty cool. And the not fun tools. Pretty much every yahoo store RTML project was a shotgun to the face.

Things have gotten so much more enjoyable, thanks to preprocessors and more uniform standards, etc. Really amazing how for we've come.

I built my own CMS in ASP, then realized being part of a CMS community was important - Textpattern did everything I wanted out of my own system. Then clients asked for WordPress. Now its just got to be interoperable, fast, flexible and Processwire serves that purpose very well. At the same time, there's so much that can be done without developers now - it's really been interesting.