Module Release: SchedulePages

[ceberlin]

Hi Jasper, I was ready today to trying the new version.

After installing, when going to edit the module's prefs, I get a blank page and this:

Error: Using $this when not in object context (line 202 of /.../site/modules/SchedulePages/SchedulePages.module) 

[formmailer]

Hi!

I made a mistake when making the module translatable. Sorry about that.

I just pushed a new version to Github that should work.

/Jasper

[ceberlin]

Hi, this could be another error with translations (seen in github code):

 ( $this->_("Your....

If I understand that correctly from reading the api manual, in modules the writing is differently  ( __("Your....

but I am not a modules programmer myself, so I can't tell for sure.

By the way: I had another mass-unsubscription today. Unfortunately I forgot to activate the module's new log-feature and have no details for you. Next time you get more info from me)

EDIT:

Funny that pages got unpublished that had no content in any of the 2 schedule fields.

There might be something with the searches;

$unpublished = wire("pages")->find("status=unpublished"); 

Does this actually find anything, if the user isn't superuser? Or is ";include:all" needed here?

$published = wire("pages")->find("status<" . Page::statusUnpublished . ", publish_until>0");

Why isn't the opposite of above code is used (status!=unpublished) but this fancy :: double colon scope operator instead (and what works better)?

Does this reliably exclude pages that have no content in the scheduled-fields? (In my case, pages with publish_until=0 were unpublished.)

Does it play a role that "setOutputFormatting(false)" is executed in the script and might influence time calculations somehow?

EDIT2:

I am on module 1.1.1 and PW dev.

[formmailer]

Hi!

You are right about the translation errors, I thought I got all of them now, but might have missed one.

The reason why I used <" . Page::statusUnpublished is because the status can be something else as well. This includes:

Page::statusOn (1)

Page::statusLocked (4)

Page::statusSystemID (8)

Page::statusSystem (16)

Page::statusHidden (1024)

But not

Page::statusUnpublished (2048)

Page::statusTrash (8192)

In my test setup all the searches seems to work fine.

SetOutputFormatting(false) is needed to get the date in unix timestamp format, so it can be easily compared with the other values.

It would be interesting what the logs say if it happens again.

//Jasper

[ceberlin]

OK I could catch a log now. Fortunately I had deactivated save() this time. so no damage happened.

Example:

2013-08-23 00:06:28    guest    http://www.mydomain.com/en/contact/contact-in/    Unpublished /en/help/ with publish_from value of 1970-01-01 01:00:00 and publish_until value of 1970-01-01 01:00:00. Current timestamp is 2013-08-23 00:06:27.

I can email you personally the full log with the correct links that I did not want to expose here.

Interesting is that the first url is always the same and that specific page one hold the "clock" script that is still under my suspicion to be the conflicting script.

So, renaming that one variable did not help.

I can give you full access to everything you need to investigate, but not over this forum. I write you a PM.

EDIT:

1970-01-01 01:00:00  is a value that is stored in the database for the "empty" publish_until field (instead of 0 or NULL).

So the problem might be a time string conversion problem. - I use a non-default setup of the publish_until time field:

{"dateOutputFormat":"Y-m-d H:i:s","dateInputFormat":"l, j F Y","datepicker":1,"_dateOutputFormat":"Y-m-d","_timeOutputFormat":"H:i:s","size":25,"description":"At this date this page will go offline.","tags":"all-Page-Controls","columnWidth":50,"_dateInputFormat":"l, j F Y","dateOutputFormat1231":"Y-m-d H:i:s","dateOutputFormat1234":"Y-m-d H:i:s","collapsed":2}

[Soma]

I also recognized that daytime fields in pw once set and saved then cleaned again it saves the field with the 1970 date. This is like 0 timestamp. It is ok when using in api but when directly queried from db you get the date so you have to take care of those. This is clearly a pitfall and should be fixed. I think I even reported this long time ago on github but not sure anymore.

[ceberlin]

Just a quick note:

I still believe that the offending part that causes all this might be the usage of date_default_timezone_set in PHP at another script somewhere else which seems to have unintended side effects on the module's script. That would explain why the script is only hurting me and almost no one else. (I de-activated that other script and will do some more logging now.)

[formmailer]

Hi, 

Since the previous logs showed that the unpublished pages have a publish_until date set to 1970-01-01 01:00:00 which actually is a Unix timestamp 0.

Could you please try something for me?

Please change line 180 from:

 if($p->publish_until <= $currenttime) {
 

into

if($p->publish_until <= $currenttime AND $p->publish_until > 1) {
 

This would prevent the module from unpublishing pages with the 0 (1970-01-01 01:00:00) value set in the publish until field and should resolve the issue (although it isn't a pretty solution. )

Let me know if it works. If it does, I'll push a new version to Github.

/Jasper 

[ceberlin]

Hi, this looks like a clever idea. It's pretty enough ;-)

I put this online and continue with loggings. 

[Soma]

Any news if the issue is solved with this?

[Soma]

I installed this modules as I have a need to a scheduler, thanks for all the work put into it.

I have a problem that the scheduler doesn't run then visitors visit the site. It only runs when I am logged in. This doesn't make a lot of sense, and I'm suprised that nobody run into this? Any ideas?

Also it would be nice when installing if it would check if fields already exists and only try to create them if not present. I already had them created beforehand with the same names (lucky) without knowing I would install this module. 

Also when deinstalling the module I'm not sure It should try to remove the fields at all ,even if the fields are not used on any template anymore. Maybe the dev wants to keep them.

Thanks

[Soma]

RE: Ahhh, I think I found what the problem is. It's that I have template cache enabled for most pages. Any ideas how to make it work with cache enabled? If that makes sense at all

Edit: Need to analyse this a litte if this is a problem at all. If at least some pages are not cached, or cache gets newly created for any of the pages, it should trigger tha lazy cron. So I can see this is not as much if a problem as I first thought.

[ceberlin]

Any news if the issue is solved with this?

Hi, I had the logging ON (debugging mode ON) and the actual actions OFF (commented those lines out) so I could monitor for 2 months how the module behaves.

I checked the log in messages.txt on both the staging and the live server and found no mass-unpublishing any more.

Looks like the issue I had was solved indeed (I am using the dev version of PW, and PHP 5.3).

[formmailer]

@Soma, I believe that you are right about the lazycron and cache. But if this is an issue (eg. with a very long cache time and (nearly) all pages cached, it should be quite easy to edit the module so that it can run using a regular cron job on your server.

@Ceberlin That's good to hear.

//Jasper

[Soma]

I've testing the scheduler and I had this morning a strange behavior for a page I was testing yesterday where only the publish from is filled with a datetime past the current time. The scheduler is set to 2min. I had the page unpublished and tried to see it would get published and this was what I got in the log for this page:

It was getting published and then unpublished at the same time... 

2013-12-04 10:30:27    guest    http://removed/    Published /de/medien/medienmitteilung/2006/2006_12_22_mm.php/ with publish_from value of 03.12.2013 16:04 and publish_until value of . Current timestamp is 2013-12-04 10:30:27. (SchedulePages)

2013-12-04 10:30:27    guest    http://removed/    Unpublished /de/medien/medienmitteilung/2006/2006_12_22_mm.php/ with publish_from value of 03.12.2013 16:04 and publish_until value of . Current timestamp is 2013-12-04 10:30:27. (SchedulePages)

 

2013-12-04 11:01:58    guest    http://removed/    Published /de/medien/medienmitteilung/2006/2006_12_22_mm.php/ with publish_from value of 03.12.2013 16:04 and publish_until value of . Current timestamp is 2013-12-04 11:01:58. (SchedulePages)

2013-12-04 11:01:58    guest    http://removed/    Unpublished /de/medien/medienmitteilung/2006/2006_12_22_mm.php/ with publish_from value of 03.12.2013 16:04 and publish_until value of . Current timestamp is 2013-12-04 11:01:58. (SchedulePages)

 

Really strange. And I was trying to find out but couldn't. I set the published_until and tested again but seems like it is gone away. I blanked the publish_until again and still couldn't get the behavior as before.

 

I tested with other pages but no luck. 

 

BTW I added before all testing, the suggested text fix you mentioned some posts earlier

 

#180 if($p->publish_until <= $currenttime AND $p->publish_until > 1)

 

I'm not sure what was causing it, and hope it's not something that will give surprises later

[teppo]

@Soma: I had planned to debug this a bit further before posting, but since you've already mentioned it: I've been seeing this same issue for couple of days now. Log entries are very similar to yours.

So far I'm guessing that this is somehow related to ProcessWire issue #284, which has since been fixed. At least in my case, where pages just got suddenly unpublished and I'm running dev version just before this fix, this seems to be the culprit.

Taking a look at field_publish_until for one particular page for which it should be empty, it looks like this:

+----------+---------------------+
| pages_id | data                |
+----------+---------------------+
|     9224 | 1970-01-01 02:00:00 |
+----------+---------------------+

Since SchedulePages first finds pages where publish_until>0 and then checks if publish_from <= current time, this particular page getting unpublished sounds logical

This might only be applicable to dev branch, but I haven't tested this a site running older release yet. Soma, what version of PW are you running? If you're using dev, could you update it and see if the issue persists?

Edit: Soma, I just realized that you mentioned the issue suddenly going away. Any chance that this was because you updated your PW at some point, applying aforementioned fix?

Edited December 4, 2013 by teppo

[Soma]

I have very latest dev from yesterday which fixes cache delete problem. But the problem was today, BUT there was something happening on date field after updating PW on first load... could be that this page I tested still got an issue with it. As since changing values it doesn't happen anymore.

[Soma]

BTW I thought I openend an issue with date fields but maybe haven't. But surely mentioned it in some threads that the date fields get saved with a 1970 timestamp 0 which I ran into when checking dates and empty dates. Hopefully it is now fixed.

[Soma]

What I have since the update to latest dev is this on the very bottom of the admin "pages tree" screen...  as mentioned before I thought it was only once but suddenly appeared again:

Warning: date() expects parameter 2 to be long, string given in /home/.../wire/modules/Fieldtype/FieldtypeDatetime.module on line 370

Which appears random it seems and I can't reproduce why, but wasn't there before the update.

[ryan]

I think both of these issues are fixed on dev, but please let me know if you are still experiencing either. 

[Craig]

Just thought I'd add some more info to this that might be relevant.

I'm currently working on a site that is a work in progress (using the PW dev branch), and I would encounter an initial page load time that was unnecessarily long (I would get a "max execution time exceeded" message) every now and then. To cut a long story short, it was the SchedulePages LazyCron issue concerning the date field having the '1970' value in the DB.

The site in question currently has over 10,000 (imported) news posts that can potentially use SchedulePages. The line below seemed to be returning all of these pages, causing some internal PW page-related functions to time out due to the volume of pages involved.

$published = wire("pages")->find("status<" . Page::statusUnpublished . ", publish_until>0");

My solution was just to run this query to remove the existing rows, so those pages with the incorrect values would not be picked up by the "publish_until>0" selector used in the SchedulePages module.

DELETE FROM field_publish_until WHERE data = '1970-01-01 01:00:00';

After that, using the bleeding-edge dev branch of PW, I tested the publish from/until fields on some posts and the DB rows get added and deleted as they should do.

[David Karich]

Since PW Version 2.4 i cant use this module anymore. I think its not really compatible. Every minute, all my pages are automatically unpublished. Unfortunately I have no time to search for the problem. Disable the module helps for now.

[teppo]

I've been using this module with 2.4 for couple of sites without issues.

David, your problem sounds a lot like issues reported earlier, i.e. "empty" date fields containing 1970's timestamp. I'd suggest trying what Craig explained above:

DELETE FROM field_publish_until WHERE data = '1970-01-01 01:00:00';

[ceberlin]

If I delete those empty fields (which I successfully did and had no incidents after that)... is it guaranteed that they do not come back?

I want to double check: does PW 2.4 now delete those fields if not used, so that error has gone forever? I think so...

Just for my curiosity: Would it help if the module ignore  pages from unpublishing which are older than, say, 1975 (very old but younger than the infamous 1970)?

[HueyGeek]

Hi formmailer, others,

I got the following error: "Error: Undefined class constant 'logOnly' (line 168 of /xxxxxxxxxxxx/site/modules/SchedulePages/SchedulePages.module) 

" in PW 2.3, with debugging turned on.

Thanks in advanced