Weekly update – February 28, 2020

[ryan]

I’ve been working on some major upgrades to our InputfieldDatetime core module and they are just about ready, but because there’s a lot of new code involved, I just want to test things out more thoroughly before pushing to the dev branch. I’ll have more details on this for you next week, but here’s a summary of what’s new in our Date/Time Inputfield coming in 3.0.152: 

If you’ve used date inputs in ProcessWire before (whether in the admin, FormBuilder or elsewhere), you know that they consist of a text field with a configurable date format and optional jQuery UI powered date picker.  On recent projects, I’ve wanted more options here. First off, I’ve wanted support for the HTML5 “date” and “time” input types, because on some browsers (mobile-based in particular) the browser implementation is quite good. On mobile clients (testing in Android Chrome at least), I find it’s actually pretty amazing, certainly preferable to the jQuery UI date picker. On desktop clients, I think the jQuery UI date picker might still be preferable, but the browser implementation is still quite good (though varies widely from browser to browser). So I’ve upgraded InputfieldDatetime to support both HTML5 “date” and “time” inputs as configurable options, as well as the ability to use them both together. 

Another thing I’ve often wished for when it comes to date inputs is the ability to have the Month, Day and Year isolated into separate selects. This is something that would work in any environment, and not require any particular feature support from the browser, nor would it require jQuery UI (like if you are wanting lightweight date fields in FormBuilder). This type of date selection seems to be the simplest, easiest and most portable way to go, and it’s something I’ve wanted for quite a long time. So support for this has also been added to InputfieldDatetime this week as well! Of course the order of month, day and year is completely configurable. 

Some other useful additions include support for HTML5 step values (date or time), minimum and maximum dates (and times), support for seconds in time selections, automatic localized month names in selects (via strftime), and the ability to ask for just month and year (when day is not applicable), or month and day (when year is not applicable). These updates—along with others—will appear in ProcessWire 3.0.152 on the dev branch within the next week. 
 

[Zeka]

@ryan There is an issue with jQuery Timepicker Addon.

https://github.com/processwire/processwire-issues/issues/1088

We have 1.6.1 in the core, but the latest version is 1.6.3, so maybe it's good time to update.

Thanks.

[bernhard]

Hey @ryan this sounds great ? 

Could you please have a look at https://github.com/BernhardBaumrock/RockDatetime and see if that might be something that find its way into the core? I think some solid date and time handling API would be a great addition to PW! RockDatetime was developed because I need it for a new module that I'm working on for dealing with date ranges. See this recent post from today about the latest additions: 

 

I'm especially talking about this post: https://processwire.com/talk/topic/23097-previewdiscussion-rockdaterange-fieldtype-inputfield-to-easily-pick-daterange-or-timerange/?do=findComment&comment=198191 It's neither well tested nore complete. But feel free to copy/paste/improve as much as you want ? I think having an API for dates and times would be a very logical addition to the PW API awesomeness ? I'm talking about things like this:

// calculating FROM/TO for RockDaterange
case 'hour':
  $from = new RockDatetime("{$obj->date} {$obj->time}:00:00");
  if($isTo) $to = $from;
  else $to = $from->copy()->move("+1 hour");
  break;

// check if FROM starts at 00:00 on that day
if($this->from->equals($this->from->firstOfDay()) ...

// show modified date when different from created day
$updated = new RockDatetime($page->getUnformatted("modified"));
$date = "Zuletzt aktualisiert am ".$updated->format("%d.%m.%Y");
if($updated->sameDay($page->getUnformatted("created"))) $date = "";

// formatting
$d = new RockDatetime("25.02.2020 14:00");
echo $d->format(); // 25.02.2020 14:00
echo $d->format("%A, %d.%m.%y (%H:%M Uhr)"); // Dienstag, 25.02.20 (14:00 Uhr)
echo $d->format(['time' => "%H|%M|%S"]); // 25.02.2020 14|30|00

 

 

On a side note I want to ask you if you please could consider to have a look at this request: https://github.com/processwire/processwire-requests/issues/326

I'm struggling on almost all my module development with proper file path and url handling and I'm doing the implementations over and over again because I don't want to add a module dependency for that. This little thing should really be in the core! We've had some problems with windows/unix file path handling recently and it shows again how important it is to have the right tools at our hands without implementing those things on our own: 

PS: As I've started to build my first "real" Fieldtype module with custom DB schema I want to say thank you for that awesome piece of software that you've developed here! Unfortunately there are no docs about that and the Events Fieldtype was at least for me not of much help, because it's hard to know where to start and what to add where and at what point of time to make everything work together. But once you get how it works it is really brilliant! I hope I can write a blog post about that some day if there's interest in that. So far I'd just want to express my appreciation ? 

 

[d'Hinnisdaël]

On 2/29/2020 at 12:33 AM, bernhard said:

Could you please have a look at https://github.com/BernhardBaumrock/RockDatetime and see if that might be something that find its way into the core? I think some solid date and time handling API would be a great addition to PW! RockDatetime was developed because I need it for a new module that I'm working on for dealing with date ranges. See this recent post from today about the latest additions: 

How is RockDatetime different from wrapping date fields in Carbon instances? Not trying to be dismissive here, just curious how this would help to establish a sane default for dealing with datetimes (which ProcessWire is currently lacking, unfortunately).

[bernhard]

12 hours ago, d'Hinnisdaël said:

How is RockDatetime different from wrapping date fields in Carbon instances? Not trying to be dismissive here, just curious how this would help to establish a sane default for dealing with datetimes (which ProcessWire is currently lacking, unfortunately).

Wow! Well, I just didn't know anything about https://carbon.nesbot.com/ ? Looks like I have to refactor my module once more ? Thx for bringing that to my attention!

Difference so far: RockDatetime 19.97KB, Carbon 2.42MB ?

Edit2: Just found this: https://try-carbon.herokuapp.com/

Edit: I've put together a quick helper module that loads Carbon into ProcessWire so everybody interested can give it a quick shot eg in the Tracy Console: https://github.com/BernhardBaumrock/RockCarbon

[ryan]

The focus of the InputfieldDatetime updates that this topic is about is purely about the Inputfield element and wouldn't affect anything about front-end API of date fields. Basically just the means by which users input dates into the system. When it comes to working with dates and times at the API side on the front-end, that's a different topic, but I guess I've always seen this as something that PHP does really well already, particularly the DateTime class can be very helpful. PW's always trying to work alongside PHP and with what PHP provides, as much as possible. Maybe some people encounter areas of working with dates and times that PHP doesn't cover well, but admittedly I haven't yet in projects here. Though there are a few specific cases that are covered in WireDateTime ($datetime API var) mostly for date/time functions that were needed by the core, but can also occasionally be useful on the front-end. But I see PHP's date/time classes and functions as being the tools a PW powered site would typically use when working with dates/times. Not only is there tons of documentation and examples out there, but these are tools that many PHP developers are already going to know before they know PW. I like it when people can take stuff they already know when keep using it in PW. I'm definitely open to expanding what PW's $datetime provides if there's a really common need that PHP doesn't cover well. Though I'm glad to see there are also quality modules like the ones mentioned above, as that seems like a great way to accommodate these needs too. 

[bernhard]

On 2/29/2020 at 12:33 AM, bernhard said:

On a side note I want to ask you if you please could consider to have a look at this request: https://github.com/processwire/processwire-requests/issues/326

I'm struggling on almost all my module development with proper file path and url handling and I'm doing the implementations over and over again because I don't want to add a module dependency for that. This little thing should really be in the core! We've had some problems with windows/unix file path handling recently and it shows again how important it is to have the right tools at our hands without implementing those things on our own: 

Hi Ryan, thx for the comments on the Datetime topic. Do you also have some words on $config->urls($path) and $config->paths($url) ?

[MrSnoozles]

And again, this has nothing to do with the Inputfield that is currently being worked on but what I was wishing for a couple times lately was some kind of "strtotime" support for page selectors when used on date fields. This would for example allow for filtering for users that signed up in the last 6 months, and that list is updated automatically, without any custom code involved.

$pages->find('template=user,created>"-6 months"');

 

I know, this is not related to the Inputfield, but I wanted to bring it up in case it's an easy addition while working on dates.

[Robin S]

2 hours ago, MrSnoozles said:

what I was wishing for a couple times lately was some kind of "strtotime" support for page selectors when used on date fields.

Have you tried it? There is already support for strtotime strings for datetime fields in PageFinder selectors, and your example query works just as you wrote it. ?

For in-memory (WireArray) selectors you have to use a timestamp in the selector.

[ryan]

Quote

Hi Ryan, thx for the comments on the Datetime topic. Do you also have some words on $config->urls($path) and $config->paths($url) ?

@bernhard While that need hasn't often come up here, it sounds like it might be a common need elsewhere (?) per what you said, so I think it sounds worthwhile. Though I'm thinking maybe off the $files API variable as more obvious $files->diskPathToUrl() and $files->urlToDiskPath() type functions, but it would also be simple enough to have the $config methods respond to them too, so long as there's a at least one slash present in the value.

Back to date/time API methods, were there any in particular that you thought were commonly needed but missing from PHP's built in set of tools? The two I found over time were the ability to produce relative date/time strings like "1 hour ago", and a common function that could accept different types of format strings (like those from date and strftime) so that's why PW's $datetime has those.

[bernhard]

Hi @ryan thank you very much - that will be a tiny but very helpful addition! ?

You might be right about it belonging to files instead of config - I really don't care about that. One thing that might be helpful would be a second parameter wheter it should return a trailing slash or not:

$less = __DIR__ . "/my/great/file.less";
if(is_file($less)) echo $files->url($less); // /site/templates/my/great/file.less

$dir = __DIR__ . "/blocks"; // see comment 2
$blocks = ['file1.php', 'file2.php', 'file3.php'];
foreach($blocks as $block) {
  $file = $files->path($dir, true).$block; // see comment 1
  if(!is_file($file)) continue;
  echo "<link rel='stylesheet' href='{$files->url($file)}'>";
}

// comment 1
// see how that is better compared to something like this:
$file = "$dir/$block"; // this could fail when $dir is dynamic (eg user input from module config)
$file = rtrim($dir, "/")."/".$block; // better, but tedious
// and finally, what if somebody used DIRECTORY_SEPARATOR --> it could fail on windows because of the backslash

Comment 2: This could also be a relative path, eg $dir = "blocks" (being the blocks folder of the current module). Checking for is_file() can then become tedious as you might have to add $config->paths->whatsoever. $files->path($dir, true) would make sure to return the path with a trailing slash. Or maybe it would be better if the trailing slash was returned by default?

I'd really vote for $files->url(...) and $files->path(...) instead of diskPathToUrl/urlToDiskPath; I know one should not be too short in variable/method names, but we already have the equivalents $config->paths and $config->urls that do the same, so I think this would be perfectly reasonable. ? 

And yes, I think you are right about it being a file tool, therefore we'd better use $files->url(...) instead of $config->url(...) ?

Really looking forward to that, thank you again!

----------

Regarding my datetime request: Please disregard this one. First, I didn't know about the WireDateTime at all (Sorry!) and second, the reason why I didn't like PHP's features was that it doesn't provide a nice and easy to understand API. But that's solved when using carbon and that can perfectly be built into a 3rd party module (not bloating the core).

[bernhard]

Just wanted to share another use case for $files->url() and $files->path()

// old version
	function mirrorFilesfromLiveServer(HookEvent $event)
    {
        $config = $this->wire('config');
        $file = $event->return;

        if ($event->method == 'url') {
            // convert url to disk path
            $file = $config->paths->root . substr($file, strlen($config->urls->root));
        }

		if(!is_file($file)) return;

This example is taken from this recent post by @gebeer and it would be a lot better to read and maintain with the new methods:

// new version
	function mirrorFilesfromLiveServer(HookEvent $event)
    {
		$file = $this->wire('files')->path($event->return);
		if(!is_file($file) return;

In the old version he gets the filename from the $event. He then has to check for the $event->method, because in one case the file is a PATH and in the other it is a URL. That's why he has to modify $file if the method is "url"...

And then he does what many of us do in such situations: Use string operations. They are not only ugly but also error prone: Think of different directory separators, missing or doubled slashes (eg /foo/bar//myfile.js or /foo/barmyfile.js). This was all not necessary if we had $files->path() and $files->url() ? 

See the second example. In one single $files->path() call all the complicated code from example 1 is obsolete. $files->path() returns the path of the file, no matter if it was provided as url or path. So simple, so beautiful.

[thetuningspoon]

These updates sound great. Thanks, Ryan!

On a related subject, I recently (and painfully) discovered that the DateTime field stores dates in the database as a string (MySQL DateTime) in whatever timezone PW is currently configured to. So if you change your PW time zone, the date string you get back from the DB is now interpreted to be in that new time zone rather than the one it was originally entered as. In other words, the unix timestamp you get out of it is no longer the same as the one you put into it.

It seems to me that the "correct" way to handle this would be for the unix timestamp to always be converted to and stored as a UTC string in the MySQL DateTime field, and then converted back to PW's current timezone at run time. This would be an extremely simple change to PW's DateTime field (using gmdate() instead of date() when storing and using php's DateTime class with UTC timezone specified when getting the value back).

Then when someone changes their timezone in PW, the absolute values of the dates would stay the same. Only the time zone (how they are represented on the front end) would change. So different users could have different time zone settings and view the PW back end in their own time zone, etc.

The problem, of course, is that this would break existing installations, so this would have to be added as a new module or as an alternative version included in the core. Ideally when you converted an existing datetime field to the new version it could automatically update your database values from the current PW timezone to UTC.

[LostKobrakai]

51 minutes ago, thetuningspoon said:

Then when someone changes their timezone in PW, the absolute values of the dates would stay the same. Only the time zone (how they are represented on the front end) would change. So different users could have different time zone settings and view the PW back end in their own time zone, etc.

The problem, of course, is that this would break existing installations, so this would have to be added as a new module or as an alternative version included in the core. Ideally when you converted an existing datetime field to the new version it could automatically update your database values from the current PW timezone to UTC.

There's one more problem. While the UTC time stays the same in the absolute term the wall time shown to the user might change. Say today someone enters a date in the future at 11 o'clock their timezone. At midnight an OS updates happen and a new timezone db is installed. The next day the time might be 10 o'clock that day. For dates - especially in the future - you'll need to be aware what matters to your users. Walltime or absolute time (UTC). Because the difference (the utc and std offset) are subject to change. 

If you want to compare datetimes in the db though all datetimes all need to be in the same timezone (or really UTC), so if that's a need it gets complex quickly because you need to store timezone and offset at time of writing to the db as well, besides the UTC timestamp. This is the only way to later come back to the intended wall time and being able to detect changes in timezone definition, which might result in a different walltime.

[thetuningspoon]

Are you saying that there is something wrong with my proposed solution, or are you saying that there is something else wrong with the existing field type? I think either your post went over my head, or you may be misunderstanding my proposed solution.

I think the solution is not actually that complicated. ProcessWire natively works with timestamps at runtime, which are always UTC-based. If I save something as a certain timestamp and then my PW/server/php time changes for any reason, I should be able to expect that the timestamp I get back from the field remains the same as the one I put into it. This is how it would work if the field stored a UTC string instead of a local string. Currently, what I get back is effectively a corrupted/meaningless value. I cannot change my $config->timezone once I've initially set it.

What timezone the user sees or inputs a date in on the site's front end is a separate issue and is up to the programmer to determine and make clear to the user and convert to/from as necessary. But the programmer should be able to trust the timestamp they are working with when they save and retrieve it from the database.

45 minutes ago, LostKobrakai said:

At midnight an OS updates happen and a new timezone db is installed.

What does this mean?

[LostKobrakai]

So the key information here is that the offsets of timezones are not fixed for eternity. There are databases you can download to your OS or e.g. from groups like IANA, which hold information about which offset a certain timezone was exposed to at which period of time. And those databases are updated about a handful of times a year (but not each time for actual new rulings). A good example for actual changes is the EU, which is currently starting the process of migrating away from switching to/from DST. Generally you're mostly fine for past datetimes, but the farther you're in the future the more likely it becomes the offset you expect the timezone to have at the time could change until that time.

If you convert 12.12.2030 10:00 for Europe/Berlin to 12.12.2030 9:00 UTC with the current offset of +1 in the winter and we go forward 10 years it's not clear if you get back 12.12.2030 10:00 for Europe/Berlin. The EU's efforts to remove DST might have been fruitful and Europe/Berlin is now at +2 offset all year (all year DST). So you take the db value of 12.12.2030 9:00 UTC, you add the offset for Europe/Berlin, which then is +2 and you get 12.12.2030 11:00 for Europe/Berlin. The UTC datetime was preserved, but the "wall time" wasn't – wall time being the time I see on the clock here in my office. You might be an hour late to some important appointment.

You can ignore all the above if you never convert between timezones, which as far as I can see, is what processwire is doing. 12.12.2030 10:00 for Europe/Berlin will still be 12.12.2030 10:00 for Europe/Berlin when read from the db in 10 years. Essentially the wall time is preserved. This comes at the expense of absolute distance of time between two datetimes being subject to change and not being able to have absolute ordering for dates of different timezones. The latter is not a problem if the whole system just uses one timezone.

This is why I said it's important to know what you care for. The absolute time passing by until a datetime or the actual time on the wall on a certain day. If you want UTC timestamps in your db, but not give up preserving the correct walltime you'll need to store more information in the db besides the utc timestamp to be able to react to changes like I described.

[thetuningspoon]

@LostKobrakai Ahhh... I understand now. Thank you for explaining that.

Having said that, I'm having a hard time thinking of many cases where I wouldn't want the local time/wall time to change when the definition of the timezone has changed. Otherwise, the value is no longer correct according to the new definition. I'm sure that there are cases where you would want the old wall time preserved (even in your example it is not clear to me whether you would want the wall time or the absolute time preserved for your appointment), but it seems like they would be be few and far between compared to the cases where you would want the values to be able to update dynamically and to be able to easily convert between time zones.

My current solution to this problem is to set $config->timezone to UTC at the start of a project that needs to work with multiple time zones and leave it there, converting values to local time as needed on the front end. Perhaps this is what @ryan always had in mind for such situations, but it does require some foresight. Otherwise you do end up with a real mess.

Can we all just agree that time zones should be abolished? ?

[LostKobrakai]

1 minute ago, thetuningspoon said:

Having said that, I'm having a hard time thinking of many cases where I wouldn't want the local time/wall time to change when the definition of the timezone has changed.

Basically any calendar application. If I have an appointment in a year at 10 o'clock, then I want to be reminded for it being at 10 o'clock. People usually don't care about the absolute time in such cases. Any change in wall time would be considered incorrect.

On the other hand you're correct. If e.g. a job is supposed to run a fixed amount of time then a change in timezone definition shouldn't make it run longer. 

[bernhard]

On 3/3/2020 at 11:47 AM, ryan said:

@bernhard While that need hasn't often come up here, it sounds like it might be a common need elsewhere (?) per what you said, so I think it sounds worthwhile. Though I'm thinking maybe off the $files API variable as more obvious $files->diskPathToUrl() and $files->urlToDiskPath() type functions, but it would also be simple enough to have the $config methods respond to them too, so long as there's a at least one slash present in the value.

Hi @ryan another tedious task came up today where this could be helpful! This is what I'd wish we had in the API:

$files->path("mytheme/foo/bar.php", $config->paths->templates);

This could make sure to return the full disc path to the given file relative to the templates folder. In the example it looks as if that could easily done like this:

$path = $config->paths->templates . "mytheme/foo/bar.php";

But unfortunately things are not always that easy! What if the file was already a path? What about trimming slashes? What about normalization? ...

$file = "/var/www/site/templates/mytheme/foo/bar.php";
$path = $config->paths->templates . $file; // fails!

Would be great to get these little helpers soon ?