Preview/Discussion: RockDaterange. Fieldtype + Inputfield to easily pick daterange or timerange

[bernhard]

I'm currently building a Fieldtype/Inputfield for selecting date and time ranges (eg for events). There was quite some interest in this thread, so I thought I start a dedicated discussion...

 

Background:

I guess everybody of us knows the problem: You want to present events on your website and you start with a new template and a title and body field... For events you'll also need a date, so you add a datetime field called "eventdate". Maybe the event does not take place on a specific day but on multiple days, so you need a second field... Ok, you rename the first field to "date_from" and add a second field called "date_to". So far, so good.

Then you want to list your events on the frontend. Simple thanks to the pw API, you might think. But you realize that it's not THAT simple... Which events take place on a specific day? What would the selector be? Yeah, it's not that complicated... it would be something like:

$from = strtotime("2020-01-01");
$to = strtotime("2020-02-01");
$events = $pages->find("template=event, date_from<$to, date_to>$from");

Why? See this example, where the first vertical line represents the $to variable and the second is $from:

The start of the event must be left of $to and the end must be right of $from ? Ok, not that complicated... but wait... what if the date range of the event (or whatever) was not from 2020-01-18 to 2020-02-25 but from 18 TO 25 (backwards)? The selector would be wrong in that case. And did you realize the wrong operator in the selector? We used date_to>$from, which would mean that an event starting on 2020-01-01 would NOT be found! The correct selector would be >=$from. That's just an example of how many little problems can arise in those szenarios and you quickly realize that the more you get into it, the more complicated it gets...

Next, you might want to have full day events. What to do? Adding a checkbox for that could be a solution, but at the latest now the problems really begin: If the checkbox is checked, the user should not input times, but only dates! That's not possible with the internal datetime field - or at least you would have to do quite some javascript coding. So you add 2 other fields: time_from and time_to. You configure your date fields to only hold the date portion of the timestamp and show the time inputfields only if the "fullday" checkbox is not checked.

We now have 5 fields to handle a seemingly simple task of storing an event date. That's not only taking up a lot of space in the page editor, you'll also have to refactor all your selectors that you might already have had in place until now!

 

Idea

So the idea of this module is to make all that tedious task of adding fields, thinking about the correct selectors etc. obsolete and have one single field that takes care of it and makes it easy to query for events in a given timeframe.

The GUI is Google-Calendar inspired (I'm acutally right now liking the detail that the second time input comes in front of the date input). I went ahead and just adopted that:

 

Next steps

I'm now starting to build the FINDING part of the module and I'm not sure what is the best way yet. Options I'm thinking of are:

// timestamps
$from = strtotime("2020-01-01");
$to = strtotime("2020-02-01")+1; // last second of january

// option 1
$pages->find("template=event, eventdate.isInRange=$from|$to");
$pages->find("template=event, eventdate.isOnDay=$from");
$pages->find("template=event, eventdate.isInMonth=$from");
$pages->find("template=event, eventdate.isInYear=$from");

// option 2
$finder = $modules->get("RockDaterangeFinder");
$finder->findInRange("eventdate", $from, $to, "template=event");
$finder->findOnDay("eventdate", $from, "template=event");
...

I think option 1 is cleaner and easier to use and develop, so I'll continue with this option ? 

Future

As @gebeer already asked here I'm of course already thinking of how this could be extended to support recurring events (date ranges) in the future... I'm not sure how to do that yet, but I think it could make a lot of sense to build this feature into this module.

I'm not sure if/how/when I can realease this module. I'm building it now for one project and want to see how it works first. Nevertheless I wanted to share the status with you to get some feedback and maybe also get your experiences in working with dates and times or maybe working with recurring events (or the abandoned recurme field). For recurring events the finding process would be a lot more complicated though, so there it might be better to use an approach similar to option 2 in the example above.

 

[gebeer]

That looks great so far, will be a really useful fieldtype!

55 minutes ago, bernhard said:

I think option 1 is cleaner and easier to use and develop, so I'll continue with this option

I also prefer option 1. Much cleaner than having to involve yet another module.

55 minutes ago, bernhard said:

I'm of course already thinking of how this could be extended to support recurring events (date ranges) in the future... I'm not sure how to do that yet, but I think it could make a lot of sense to build this feature into this module

This would be so awesome. Especially since the Recurme module seems to have passed away silently. Maybe you can get some inspiration from there on how to handle recurring stuff. I wouldn't include the whole render a calendar stuff. People who need it can always use some existing framework for that.

55 minutes ago, bernhard said:

I'm not sure if/how/when I can realease this module. I'm building it now for one project and want to see how it works first.

Would be great if you could share the code pre-release on GH so people who are interested could collaborate.

55 minutes ago, bernhard said:

I wanted to share the status with you to get some feedback and maybe also get your experiences in working with dates and times or maybe working with recurring events

I did some date manipulation in the past because I'm involved with quite a few sites that needed it. What I found most challenging is the timezone calculations. Also did some recurring stuff as far as I remember. What I found very helpful at the time for those tasks are the PHP DateTime, DatePeriod and Dateinterval classes. Haven't delved into it for some 3 years but remember that it actually was fun. I guess today I would use the recurr library.

EDIT: As for the recurring events, I'd probably make this optional in the field settings and only present the inputs if needed.

[bernhard]

I've absolutely no experience in handling different timezones, so I'd be happy to get a quickstart or a good read for that topic ? 

[gebeer]

13 minutes ago, bernhard said:

I've absolutely no experience in handling different timezones, so I'd be happy to get a quickstart or a good read for that topic

This is really only necessary if you are dealing with user input of users from different time zones.
You'd also have to know the timezone of the logged in user. For projects where I needed that, I had a field in the user template to save the timezone and did calculations based on that. It is hard to foresee how other people implement it and therefore in the fieldtype you would not know how to get the user timezone. Maybe in a hookable method that by default returns the local server timezone. Then people could hook into it and provide the user timezone only if it differs from the server timezone.
The common base for calculation between different timezones is the unix timestamp. So the resulting timestamp for 01.01.2020 20:00 will be different, depending on what the timezone is. The DateTime class has the  setTimezone method which helps for conversion of DateTime objects between different timezones. Here's a quick read on the basics.

[szabesz]

@bernhard Thanks for this discussion, it is great that you raised this issue this way.

2 hours ago, gebeer said:

I guess today I would use the recurr library.

I've found another one: https://github.com/rlanvin/php-rrule this one does not seem too have to many issues but it is hard to tell...

I think supporting recurrence out of the box might be a good idea as adding it later might generate unnecessary refactoring work. I am just guessing though... As for time zones, most projects can do without it, so if I were Bernhard I would not deal with it.

[bernhard]

I think this already looks quite promising ? 

$selectors = [
    "template=event",
    "template=event, range.inRange=2020-01-01;2021-01-01",
    "template=event, range.inRange=2020-01-01;2020-02-01",
    "template=event, range.inRange=2020-02-01;2020-03-01",
    "template=event, range.inRange=2020-04-01;2020-05-01",
];
foreach($selectors as $selector) {
    $out = '';
    $nl = '';
    foreach($pages->find($selector) as $p) {
        $out .= $nl.$p->getFormatted('range');
        $nl = "\n";
    }
    d($out, $selector);
}

This now also works with backwards ranges (eg 19.3.2020 to 10.2.2020) because I store a separate START and END timestamp in the database:

I've also implemented onDay, inMonth, inYear selectors:

$selectors = [
    "template=event",
    "template=event, range.onDay=2020-04-15",
    "template=event, range.inMonth=2020-03",
    "template=event, range.inYear=2021",
];

This also works using timestamps:

$stamp = strtotime("2020-04-15");
$selectors = [
    "template=event",
    "template=event, range.onDay=$stamp",
    "template=event, range.inMonth=$stamp",
    "template=event, range.inYear=$stamp",
];

Even sorting works out of the box - PW is once more impressive ? 

[bernhard]

There might come many posts in the near future, sorry for that. I'll use this thread to let you follow and to have some kind of notes for writing docs when I release the module... It's also easier to quote and link to a specific post than to a section of one long post.

The field now has 2 settings for hiding the checkboxes for toggline the fullday and has-end setting. This means you can use this field as a replacement for the datetime field and use the custom selector options shown above (range.onDay/inMonth/inYear).

[bernhard]

Little improvements to the input:

1) Checkboxes do now show an icon. I think that's a lot more intuitive!

2) You can input dates and times using the "comma" keyboard key (both regular comma and the numblock comma). This is convenient for german keyboards that have a comma instead of a dot right beside the 0 digit):

I also did quite some refactoring and changed the db schema so that the data column holds the timestamp of the first second of the timerange. This makes it possible to choose the range field as sort option in the template settings.

[bernhard]

Other handy selectors added ?

Any other ideas that could be necessary or helpful? It's really easy to add others ? 

      // returns all pages with a date in a specified year
      case 'inYear':
        // split value by semicolon
        $str = $this->getFromToStrings($value, 'year');
        $query->where("{$table}.data<='{$str->to}'");
        $query->where("{$table}.end>='{$str->from}'");
        break;
        
      // returns all pages that start before a given date
      case 'startsBefore':
        // split value by semicolon
        $str = $this->getDatestring($value);
        $query->where("{$table}.data<'$str'");
        break;

Current options:

inRange
onDay
inMonth
inYear
startsBefore
endsBefore
startsAfter
endsAfter

 

[teppo]

2 hours ago, bernhard said:

Little improvements to the input:

1) Checkboxes do now show an icon. I think that's a lot more intuitive!

If I understood this correctly (instead of text, there's now just an icon?) I would actually recommend against this — simply because icons will never be as obvious as text labels. From an usability point of view text labels are much better ?

From an accessibility point of view, on the other hand, this is just fine — as long as your checkboxes still have proper labels. If the icons are images an alt text is quite enough, but if they're something else (<i> with FA classes etc.) you should hide them from screen readers with aria-hidden="true" and then introduce a separate screen reader only text version.

[dotnetic]

Hey Bernhard, labels above or in front of the fields would be a great UX enhancement and make it obvious what field is what.

[bernhard]

25 minutes ago, Jens Martsch - dotnetic said:

Hey Bernhard, labels above or in front of the fields would be a great UX enhancement and make it obvious what field is what.

Could you please be more specific on this?

 

1 hour ago, teppo said:

If I understood this correctly (instead of text, there's now just an icon?) I would actually recommend against this — simply because icons will never be as obvious as text labels. From an usability point of view text labels are much better ?

Yeah, it's icons with hover text (title attribute) - but the screenshot program closes the tooltip before it takes the screenshot...

Actually it doesn't matter too much if it's icons or text - what is really different now is that I changed the name of the first checkbox from "fullday" to "hasTime". This is now consistent with the second checkbox named "hasEnd" and it is more intuitive for the user I think. Check the checkbox = show time inputs, uncheck = don't show time inputs. Before it was the other way round: Checkbox checked meant fullday, meaning that the time input was hidden. The second checkbox worked the other way round: Checked = show inputs, unchecked = hide inputs.

I'm happy to get suggestions how to call those two checkboxes...

1. Show time input?
2. Show end input?

[teppo]

23 minutes ago, bernhard said:

Yeah, it's icons with hover text (title attribute) - but the screenshot program closes the tooltip before it takes the screenshot...

Title text is not a best practice when it comes to usability; visible text would be best. Not just because things shouldn't be hidden (unless there's a good reason for it, which I really don't see here), but also because it'll be a problem from accessibility point of view, for touch screen users, etc. Overall there are very few cases where the title attribute should be used ?

23 minutes ago, bernhard said:

Actually it doesn't matter too much if it's icons or text - what is really different now is that I changed the name of the first checkbox from "fullday" to "hasTime". This is now consistent with the second checkbox named "hasEnd" and it is more intuitive for the user I think. Check the checkbox = show time inputs, uncheck = don't show time inputs. Before it was the other way round: Checkbox checked meant fullday, meaning that the time input was hidden. The second checkbox worked the other way round: Checked = show inputs, unchecked = hide inputs.

This makes sense ?

23 minutes ago, bernhard said:

I'm happy to get suggestions how to call those two checkboxes...

1. Show time input?
2. Show end input?

I think your suggestions are already pretty good! "Show time input" could be just "show time" perhaps? And "show end input" could also be something along the lines of "date range", though not sure about that one.

This brings to mind one question: what do you think about making the labels configurable? For an example if we're talking about events, a label such as "all day event" would probably make most sense (and Outlook Calendar uses this as well, so it's somewhat likely to be familiar to users), but events are not the only use case, so the default label probably shouldn't mention "event" specifically ?

[adrian]

13 hours ago, szabesz said:

I've found another one: https://github.com/rlanvin/php-rrule this one does not seem too have to many issues but it is hard to tell...

I have found this library quite useful in the past.

If you are going to support recurring dates, please take a look at the inputfield interface for recurme (it works quite nicely), but please make sure if you implement momentjs for any part of that process be sure to use the timezone version of it. I also think the ability to output recurring dates using RRULE (https://icalendar.org/iCalendar-RFC-5545/3-8-5-3-recurrence-rule.html) please be sure not to follow the broken format used by Recurme. I would recommend a full read of the recurme support thread as it raises lots of issues which should help you to avoid them in this module ?

[gebeer]

5 hours ago, teppo said:

Overall there are very few cases where the title attribute should be used

Thanks for that link! An agency I'm sometimes working for wants to have title attributes on all links, no matter what. They say it's for SEO. Sorry for getting offtopic.

6 hours ago, bernhard said:

• Show time input?
• Show end input?

I'd call them something like 'Input time' and 'Input end' because that is actually what you can do after clicking on them and is also like an instruction of what to do or what can be done.

 

10 minutes ago, adrian said:

I would recommend a full read of the recurme support thread

That sure is a long read but well worth it.

[GhostRider]

Hi Bernhard,

Being able to enter and display Events (or the like) is such a valuable tool for many websites.

I am thrilled you have started this project.

I cannot over emphasize @adrian's comment on all fronts.

A few things that are very helpful in displaying recurring events is the ability to only display the event once.

Ie. if you have a recurring event from March 10-15th. You may not want the event to display 6 times if you are viewing the Event list page on March 10th.

The recurme module does do quite a few things right, unfortunately it wasn't quite there. Two features standout:

1. The user interface to create recurring dates which also has the ability to remove select dates. Ie. A range from March 10-15, but you can then exclude any date in the range. This avoids a second (essentially duplicate) event needing to be created.

2. Displaying recurring events can be quite a challenge, especially when there is a long list of dates. Recurme used a calendar display for this, which could really display a lot of recurring dates in a small space. Rather than displaying a long list dates in text format.

I will send you a pm with some live examples we have built to date and how we display events. Perhaps this will help highlight some of the challenges you may be facing or how people may use your module.

This will be such a great addition to the Processwire ecosystem!

 

[bernhard]

7 hours ago, gebeer said:

14 hours ago, bernhard said:

• Show time input?
• Show end input?

I'd call them something like 'Input time' and 'Input end' because that is actually what you can do after clicking on them and is also like an instruction of what to do or what can be done.

That sounds good, thx ? 

3 hours ago, GhostRider said:

A few things that are very helpful in displaying recurring events is the ability to only display the event once.

Ie. if you have a recurring event from March 10-15th. You may not want the event to display 6 times if you are viewing the Event list page on March 10th.

Yeah. There are some complicated things involved when displaying events... What if the user visited the List on March 14th? And how to do proper limit and pagination queries? Eg find exactly 12 events but show recurring only once... Not sure how one would solve that properly ?

 

Thx everybody for the feedback. I'll read the recurme thread that I followed only loosely and finish my current project with the basic implementation of RockDaterange (without RRULE). We'll see how that goes and what might be added in the future...

[teppo]

Just my five cents: this project seems like something that would be very useful without any support for recurring. I'd love to get my hands on this feature, and I'm pretty sure I'll never even need that latter part. In fact more commonly I've needed reoccurring events (hope that makes sense; basically I mean events with multiple, manually specified dates rather than a set of rules to govern when and how often they should recur).

Recurring rules can get extremely complicated: "this event occurs every Friday between January and August except specific days x, y, and z — and then it also occurs on this particular Wednesday and that Tuesday there, but on those days there needs to be this additional note on the content". Done that a few times, but I try my best to steer away from those implementations. It's very rarely worth the hassle. In my experience lot of that mess can be circumvented by allowing multiple dates (or date ranges) for one event  ?

[bernhard]

Hi @teppo

that's a very good point and a reminder, that adding rrule support might be overkill for this module. Maybe it would make more sense to have this in a separate module!

Just implemented a little funktion that can list items before and after a given date:

The nice thing is that you can specify multiple templates. The idea is to show events in the footer and there it might make sense to show also events of the near past ? Past events are sorted by end date, future events are sorted by start date. Quite nice and quite easy to query:

Spoiler

$wire->addHook("Pages::findBeforeAndAfter", function($event) {
  $defaults = [
    'template' => null,
    'before' => 3,
    'after' => 3,
    'time' => time(),
  ];
  $options = (object)array_merge($defaults, $event->arguments(0) ?? []);
  $all = new PageArray();

  // get items before reference date
  // sorted by end date
  $selector = "";
  if($options->template) $selector .= "template=".$options->template;
  $selector .= ",range.endsBefore=".$options->time;
  $selector .= ",sort=-range.end";
  $selector .= ",limit=".$options->before;
  $before = $this->pages->find($selector)->reverse();
  $all->add($before);

  // get items after reference date
  // now sorted by start date
  $selector = "id!=$before";
  if($options->template) $selector .= ",template=".$options->template;
  $selector .= ",range.endsAfter=".$options->time;
  $selector .= ",sort=range";
  $selector .= ",limit=".$options->after;
  $after = $this->pages->find($selector);
  $all->add($after);

  $event->return = (object)[
    'before' => $before,
    'after' => $after,
    'all' => $all,
  ];
});

 

Not sure where to put this though. Right now it's a hook in ready.php - but there it is not reusable. I thought of adding this to every RockDaterange object, but it feels wrong. Maybe adding a helper module for such features would make sense? Actually the wording of this method should be findBeforeAndAfter instead of GET... I'll change that!

[bernhard]

11 hours ago, bernhard said:

Not sure where to put this though. Right now it's a hook in ready.php - but there it is not reusable. I thought of adding this to every RockDaterange object, but it feels wrong. Maybe adding a helper module for such features would make sense? Actually the wording of this method should be findBeforeAndAfter instead of GET... I'll change that!

I've added a hooks folder where all daterange hooks can be put into:

All hooks can be disabled in the module config:

Not sure if that makes sense at all, but it was fun to build ? 

[LostKobrakai]

On 2/11/2020 at 10:50 AM, bernhard said:

I've absolutely no experience in handling different timezones, so I'd be happy to get a quickstart or a good read for that topic ? 

The biggest hurdle of timezones is actually in handling "the future". Timezone rules are in constant fluctuation (be it error corrections or actual changes the IANA timezone database changes a few times a year: https://www.iana.org/time-zones). One prominent example is the EU right now, where DST was ruled to be eliminated in the near future giving each country the option to choose if they want to stay on DST or non-DST offset. Therefore to store a point in time (in the future) you at best store not just a datetime/timestamp, but a timestamp as well as the source timezone and used offset. This allows you to detect if the offset of a timezone did change between the time is was stored to the db until the time of retrieval. Otherwise changes in timezone definition might lead to incorrect results (someone being at a place at 11:00 when the time was supposed to be 10:00 wall time). This blog has a few posts on the topic, even if they're not in php: http://www.creativedeletion.com/2015/03/19/persisting_future_datetimes.html

Edit: You can ignore changes in timezone definition by storing datetimes in their source timezone, but this won't let you easily compare multiple values in the db.

On 2/11/2020 at 9:54 AM, bernhard said:

Nevertheless I wanted to share the status with you to get some feedback and maybe also get your experiences in working with dates and times

I've some more interesting things on the topic to share. If you're working with intervals I highly suggest using Allen's Interval Algebra as well as watch this talk by Eric Evans: 

 

[spoetnik]

I once build e recurringEvents modules myself. 
https://github.com/spoetnik/ProcessRecurringEvents

 

maybe it could inspire you?

 

recurrence are ‘fictional’ and calculated, and only made to real Processwire pages when needed. 

[bernhard]

Thx @spoetnik

1 hour ago, spoetnik said:

recurrence are ‘fictional’ and calculated, and only made to real Processwire pages when needed. 

Could you give me some more details here please? I don't really understand. It looks like you create the page clones on page save when the recurring checkbox is saved? Im confused by your "fictional" explanation in that context ?

[bernhard]

Just wanted to share a learning from today when working with "recurring" events on my project. As we do not have real recurring events yet I did create recurring events manually (using trancy console once more). I realized 2 things:

1. Recurring events likely need to be real pages under the same parent as the master page (eg I have all my events living at my.site/events/my-event). I need a details page for every event, so if that event took place every week 4 times in a row, I'd also need my.site/events/my-event-2 ...3 and ...4
   Another solution would be to use url segments, eg my.site/events/my-event/2 ...3 and ...4
    
2. What about page data?
   First I thought of hiding all data fields for slave-events and pulling that data from the master event. That would work for displaying things, but it would not work for finding events...
   
   eg. $pages->find("template=event, location=fooplace"); would then only return the master event, but not the unpopulated slave pages ?
   
   Maybe we'd need some SQL join magic to get an efficient list of all recurring events populated with master data...

I think it will take some time to develop something solid...

[bernhard]

I had to do several page imports via Tracy Console today, so I had the challenge of populating the Daterange field easily via the API... I added support for basic daterange parsing from a given string:

RockDaterange("22.02.2020")
hasTime => false
hasEnd => false
fromH => "22.02.2020 00:00" (16)
toH => "22.02.2020 23:59" (16)

RockDaterange("22.02.2020 17:00 - 19:00")
hasTime => true
hasEnd => true
fromH => "22.02.2020 17:00" (16)
toH => "22.02.2020 19:00" (16)

RockDaterange("22.02.2020 17:00 - 20-3-1 19:00")
hasTime => true
hasEnd => true
fromH => "22.02.2020 17:00" (16)
toH => "01.03.2020 19:00" (16)

And it get's even better! This makes it possible to use $page->setAndSave() easily using string dateranges:

$page->setAndSave('range', "2020-02-22"); // single full-day event
$page->setAndSave('range', "2020-02-22 18:00"); // single day event with time
$page->setAndSave('range', "2020-02-22 18:00 - 22:00"); // single day event with time range
$page->setAndSave('range', "2020-02-22 - 2020-02-24"); // 3-day event (full-day)
$page->setAndSave('range', "2020-02-22 06:00 - 2020-02-24 23:00"); // 3-day event with times

You wonder how complicated that was to implement?! Once the parsing part was done it was nothing more than adding this one line to the sleepValue method of the fieldtype. How genius is ProcessWire?? ?

public function sleepValue($page, $field, $value) {
    if(is_string($value)) $value = new RockDaterange($value);
    ...

 

---

This is how I did the recurring events so far ? Got an excel from the client with page id of the master event and date + time of the recurring events. Using VSCode and multicursor I transferred this into a script to create pages:

The save() call at the end of each line would not be necessary but triggers a hook that renames the page and adds the daterange to the URL. This prevents ugly urls like this

/event-x
/event-x-1
/event-x-1-1
/event-x-1-1-1

And creates URLs like this instead:

/event-x-01.01.2020
/event-x-01.02.2020
/event-x-01.03.2020
/event-x-01.04.2020

There's a lot one has to think of when dealing with events ? I wonder if a "add daterange to URL" feature would make sense if a daterange field is present on a template... Would have to be optional of course.