Jump to content
kixe

Cronjob Database Backup

Recommended Posts

Hello kixe,

yes you are right, it is Ryans ProcessDatabaseBackups, which is responsible for the backups. So your module does the job quite well - the module creates a new backup after the time period I have choosen. It seems all ok unless I click on the backup to open it, then I get the error message (also at backups which were made with the latest dev version) - I will post this problem in the general section.

Best regards

Share this post


Link to post
Share on other sites

Hello Juergen,

please try to make a backup directly from ProcessDatabaseBackups. And restore from this backup. Did you get the same error?

Thanks regards

Share this post


Link to post
Share on other sites

Because it is maybe interesting for others I will answer a question I got in the personal messenger:

Does the module have the capability to run from a standard cron job?

Yes. The Backup is triggered by the hook function cronBackup() expecting a HookEvent as single argument, which is in fact not needed. If you want to trigger the backup from anywhere, just create a HookEvent object and call the cronBackup() function. Done.
 

$event = new HookEvent;
$modules->get('CronjobDatabaseBackup')->cronBackup($event);
  • Like 3

Share this post


Link to post
Share on other sites

I have spent the entire day working with websites with this module.  I am very impressed and thankful that @Kixe worked on this wonderful module.  Additional thanks to @LostKobrakai who mentioned this module in another thread, which got my interest and finally resulted in me testing it out.

I still don't know how I missed understanding the great usefulness of this module initially.

  • Like 3

Share this post


Link to post
Share on other sites

I have a question regarding the condition:

Runs only if current user has db-backup permission.

Is there some way around this? I don't want to give any other roles this permission besides superuser (and the only superuser is me) because I don't want to give site editors the ability to restore backups. But I also don't want to have to be regularly browsing the website myself just to ensure the backup cron job runs.

You mention running the backup from a standard cron job above - how does the db-backup permission come into play here?

  • Like 1

Share this post


Link to post
Share on other sites

@Robin S
Thanks for pointing this out. This note is obsolete. I missed to remove it from earlier versions. The cronjob is triggered by every user.
 

  • Like 4

Share this post


Link to post
Share on other sites

Hi @kixe,

first just want to say thanks for this useful module (if I don't have already)!

And I want to share a usage that propably wasn't intended with your module, but I find it useful for small sites where content changes slightly in a weekly manner or less. I use a hook into Session::logout in the ready.php file that invokes a DB-Backup. Currently the simplyfied code looks like:

/* Autobackups */
$wire->addHookBefore("Session::logout", function(HookEvent $event) {
    if(!wire('user')->hasPermission('db-backup')) return;
    if(wire('modules')->isInstalled('CronjobDatabaseBackup')) {
        // execute a cronBackup
        ignore_user_abort(true);
        set_time_limit(120);
        $cdb = wire('modules')->get('CronjobDatabaseBackup');
        $e = new HookEvent();
        $cdb->cronBackup($e);
    }
    return;
});

This is a bit hacky. What do you think about to embedd a public method to do this more transparent. Also it would be good if the description ($fileinfo) could be set dynamically with this method or as separate method or option. Currently I have hacked this into the module to be able to set it like: "logout horst (CronjobDatabaseBackup)".

Anyway, also without that, it is a big, big helper! :)

 

  • Like 8

Share this post


Link to post
Share on other sites

@horst
Thanks for your feedback and suggestions (I like them all). I pushed an update to github (Version 113) I made this very quickly. It seems to work, but I hadn't time to try out all scenarios. Could you test a bit? Thanks.

Version 1.1.3
NEW: user logout triggers cronjob  (db-backup permission required) instead of cycle interval.
NEW: extension of filename options (user properties)
NEW: custom fileinfo (via module settings). Same features like filename (hookable)

  • Like 3

Share this post


Link to post
Share on other sites

i got a very misterious behaviour today. after some time of debugging i made it down to the following:

i have this script in my _init.php

    function myHook() {
        wire('log')->save('lazy',"backup!");
    }
    // add a hook to your function:
    $pages->addHook('LazyCron::every30Seconds', null, 'myHook');
    wire('log')->save('lazy', 'hook added');
  • install lazycron
  • refresh the site
  • hook added and backup in the logs

BUT

  • install cronjobdatabasebackup
  • refresh the site
  • only "hook added" in the logs!

no matter what i do the cron seems not to be executed.

deleting the file /site/assets/cache/LazyCron.cache solved that problem. can you confirm that @kixe ? or did i do anything wrong? first time for me with both lazycron and your backup module :)

Share this post


Link to post
Share on other sites

hi kixe, some more questions :)

i want to have daily backups of my database, maximum 10 files.

2016-08-05 17_00_10-Modules • ProcessWire • mentalestaerke.at.png

when is the backup trigger triggered? is it triggered by EVERY pageload (also guest users) or only by logged in users? i want the backup be crated also on guest visits. do i have to grant the guest role "db-backup" permission?

next question for understanding: what happens when this role does not have this permission? is it correct, that the cronjob itself will be triggered once a day, but if it is triggered by "guest" it will NOT create a backup and if it is triggered by "superuser" it WILL create one?

i think some more explaining words in the field descriptions would be a great help! :)

thank you for sharing your module!

Share this post


Link to post
Share on other sites
On 6/08/2016 at 3:04 AM, bernhard said:

when is the backup trigger triggered? is it triggered by EVERY pageload (also guest users) or only by logged in users? i want the backup be crated also on guest visits. do i have to grant the guest role "db-backup" permission?

I believe the lazy cron can be triggered on every pageload by every user, including guest, and you don't need to add the db-backup permission to guest for this to happen. I asked about this too and @kixe responded...

On 27/05/2016 at 5:06 PM, kixe said:

Thanks for pointing this out. This note is obsolete. I missed to remove it from earlier versions. The cronjob is triggered by every user.

 

On 6/08/2016 at 3:04 AM, bernhard said:

i think some more explaining words in the field descriptions would be a great help! :)

Agreed. Also, the module description on GitHub and the modules directory seems to contradict the above so perhaps that needs to be corrected?

Quote

This cronjob (hooks on LazyCron) is executed by a call to module: ProcessPageView.
Runs only if current user has db-backup permission. Permission will be added with module installation. Assign the permission to the roles which should trigger the process.

 

  • Like 2

Share this post


Link to post
Share on other sites


@Robin S
Thanks for he help. I am a bit too busy to answer in time.
 

18 hours ago, Robin S said:

Agreed. Also, the module description on GitHub and the modules directory seems to contradict the above so perhaps that needs to be corrected?

Agreed. If I'll find some time, I will make an update. Apart from this I hope module is doing its job ...

Share this post


Link to post
Share on other sites

@since 1.1.5 added CTA Button to ProcessDatabaseBackups page (edit backups) 2016/11/28

for anyone wondering as well ;)

  • Like 3

Share this post


Link to post
Share on other sites


@bernhard

1 hour ago, bernhard said:

for anyone wondering as well ;)

CronjobDatabaseBackup and ProcessDatabaseBackups are siblings. I put them a little closer together.

  • Like 2

Share this post


Link to post
Share on other sites

Hi @kixe,

Have you seen my PR? It would be very convenient to be able to allow the job to run on certain hostnames. This idea is 'stolen' from the ProCache module.

Share this post


Link to post
Share on other sites

Feature request: the ability to have backups sent via email as well as saved on the server.

  • Like 1

Share this post


Link to post
Share on other sites

Thanks for the great Module! How about providing a default name if no name is present? 

#-%Y-m-d-H-i-s%

Or a better description with an example, I had to go into code before I realised that I had to surround "all" of the date format with %. before I tried to surround every parameter (like %Y%-%m% etc)...

Share this post


Link to post
Share on other sites

it will create a default name (database name). or are you talking about a configurable default name?

Share this post


Link to post
Share on other sites

Yes, it uses # as default name resulting in dbname-1.sql, dbname-2.sql etc – so one can not easily tell how old the date is... Since PW itself already uses a name with date appended when backing up to the same folder on upgrades, why not use the same?

The format PW uses:

dbname-Version-YYYY-MM-DD_HH-ii-ss.sql

processwire-3.0.86-2018-01-09_13-36-54.sql

  • Like 1

Share this post


Link to post
Share on other sites

@noelboss
ProcessDatabaseBackups (by Ryan) uses the same default format as CronjobDatabaseBackup (e.g. dbname-1.sql)
whereas the core class WireDatabaseBackup uses the default format: dbname_YYYY-MM-DD_HH-ii-ss.sql

2 hours ago, noelboss said:

dbname-Version-YYYY-MM-DD_HH-ii-ss.sql

I don' t know where this format is in use.

Since CronjobDatabaseBackup is built very close to ProcessDatabaseBackups I will stay compliant and don't want to change the current default format for the filename.
Anyway you have always the option to modify the format for your needs in module settings, even in the format you prefer.

  • Thanks 1

Share this post


Link to post
Share on other sites

Okay, sorry, didn't know - then its probably the update module that does format the files this way. Never the less, I still think having the date in the filename would be the best default in the majority of cases, since if you want to restore a backup, you‘d probably want to know the date anyway. 

But no problem for me, I figured it out, its just that I tought maybe that would help some to get started faster. Still great Module without this change.

  • Like 2

Share this post


Link to post
Share on other sites

Hi Kixe

Great module, thanks! Saved my site this weekend 😄

I have two suggestions as I reviewed my settings. I noticed someting that would probably be easy to implement and at least for me usefull:

– No maximum limit of backups (max): why would that option be limited to 100? If I want to store 6hours backups for 3 months, 100 is not enough.

- More flexible remove option (deadline): Provide an integer 1-99 plus a intervall: Hours, days, weeks, months years.

This way, one can tailor backups and removal very flexible. The current limitations seem very arbitrary for me.  On last thought: If it is possible, it would be nice if one can specify a time at which the backup would start – I know, this would just be an indication but still, one could plan the backups for say during night.

Share this post


Link to post
Share on other sites

Would it make sense to add an option to exclude tables from the backup, instead of choosing the tables to include?

This would be especially useful with regards to the sessions table. I have a project with a moderate amount of visitors, but 1million+ rows in the sessions table. The backup module fails since the process needs too much memory. If I exclude the sessions table, the export shrinks from 150MB to 30MB, which the server can easily handle.

Now, I can already select all tables and leave the table sessions unchecked. However, as soon as I add a field, template, etc., all future backups will be missing that table completely. In that scenario, an exclude option would be the better option.

I might just whip up a PR when I find the time, but was wondering if anybody else feels this is necessary or a good idea.

  • Like 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By eelkenet
      Hi! I've created a small Inputfield module called InputfieldFloatRange which allows you to use an HTML5 <input type="range" ../> slider as an InputField. I needed something like this for a project where the client needs to be able to tweak this value more based on 'a feeling' than just entering a boring old number. Maybe more people can use this so I'm hereby releasing it into the wild.  
       
      What is it?
      The missing range slider Inputfield for Processwire. 
      What does it do?
      This module extends InputfieldFloat and allows you to use HTML5 range sliders for number fields in your templates.
      It includes a visible and editable value field, to override/tweak the value if required.  
      Features
      Min/max values Precision (number of decimals) Steps (Read more) Manual override of the selected value (will still adhere to the rules above) Usage
      Clone / zip repo Install FieldtypeFloatRange, this automatically installs the Inputfield Create new field of type `Float (range)` or convert an existing `Float`, `Integer` or `Text` field. To render the field's value simply echo `$page->field` Demo
      A field with Min=0, Max=1, Step=0.2, Precision=2

      Field with settings Min=0, Max=200, Step=0.25, Precision=2

       
      Todo
      Make the display-field's size configurable (will use the Input Size field setting)  Hopefully become redundant If it's usable for others I'll add it to the Modules list  
      Changelog
      v002
      - Fix issue where setting the step value to an empty value created problem with validation
      - Make the display-field optional 
      v001
      - Initial release
       
      Thanks!
       
       
    • By Robin S
      Another little admin helper module...
      Template Field Widths
      Adds a "Field widths" field to Edit Template that allows you to quickly set the widths of inputfields in the template.

      Why?
      When setting up a new template or trying out different field layouts I find it a bit slow and tedious to have to open each field individually in a modal just to set the width. This module speeds up the process.
      Installation
      Install the Template Field Widths module.
      Config options
      You can set the default presentation of the "Field widths" field to collapsed or open. Field widths entered into the Template Field Widths inputfield are only applied if the Edit Template form is submitted with the Template Field Widths inputfield in an opened state. "Collapsed" is the recommended setting if you think you might also use core inputs for setting field widths in a template context. You can choose Name or Label as the primary identifier shown for the field. The unchosen alternative will become the title attribute shown on hover. You can choose to show the original field width next to the template context field width.  
      https://github.com/Toutouwai/TemplateFieldWidths
      https://modules.processwire.com/modules/template-field-widths/
    • By adrian
      Tracy Debugger for ProcessWire
      The ultimate “swiss army knife” debugging and development tool for the ProcessWire CMF/CMS

       
      Integrates and extends Nette's Tracy debugging tool and adds 35+ custom tools designed for effective ProcessWire debugging and lightning fast development
      The most comprehensive set of instructions and examples is available at: https://adrianbj.github.io/TracyDebugger
      Modules Directory: http://modules.processwire.com/modules/tracy-debugger/
      Github: https://github.com/adrianbj/TracyDebugger
      A big thanks to @tpr for introducing me to Tracy and for the idea for this module and for significant feedback, testing, and feature suggestions.
    • By adrian
      This module allows you to automatically rename file (including image) uploads according to a configurable format
      This module lets you define as many rules as you need to determine how uploaded files will be named and you can have different rules for different pages, templates, fields, and file extensions, or one rule for all uploads. Renaming works for files uploaded via the admin interface and also via the API, including images added from remote URLs.   Github: https://github.com/adrianbj/CustomUploadNames
      Modules Directory: http://modules.processwire.com/modules/process-custom-upload-names/
      Renaming Rules
      The module config allows you to set an unlimited number of Rename Rules. You can define rules to specific fields, templates, pages, and file extensions. If a rule option is left blank, the rule with be applied to all fields/templates/pages/extensions. Leave Filename Format blank to prevent renaming for a specific field/template/page combo, overriding a more general rule. Rules are processed in order, so put more specific rules before more general ones. You can drag to change the order of rules as needed. The following variables can be used in the filename format: $page, $template, $field, and $file. For some of these (eg. $field->description), if they haven't been filled out and saved prior to uploading the image, renaming won't occur on upload, but will happen on page save (could be an issue if image has already been inserted into RTE/HTML field before page save). Some examples: $page->title mysite-{$template->name}-images $field->label $file->description {$page->name}-{$file->filesize}-kb prefix-[Y-m-d_H-i-s]-suffix (anything inside square brackets is is considered to be a PHP date format for the current date/time) randstring[n] (where n is the number of characters you want in the string) ### (custom number mask, eg. 001 if more than one image with same name on a page. This is an enhanced version of the automatic addition of numbers if required) If 'Rename on Save' is checked files will be renamed again each time a page is saved (admin or front-end via API). WARNING: this setting will break any direct links to the old filename, which is particularly relevant for images inserted into RTE/HTML fields. The Filename Format can be defined using plain text and PW $page variable, for example: mysite-{$page->path} You can preserve the uploaded filename for certain rules. This will allow you to set a general renaming rule for your entire site, but then add a rule for a specific page/template/field that does not rename the uploaded file. Just simply build the rule, but leave the Filename Format field empty. You can specify an optional character limit (to nearest whole word) for the length of the filename - useful if you are using $page->path, $path->name etc and have very long page names - eg. news articles, publication titles etc. NOTE - if you are using ProcessWire's webp features, be sure to use the useSrcExt because if you have jpg and png files on the same page and your rename rules result in the same name, you need to maintain the src extension so they are kept as separate files.
      $config->webpOptions = array(     'useSrcExt' => false, // Use source file extension in webp filename? (file.jpg.webp rather than file.webp) ); Acknowledgments
      The module config settings make use of code from Pete's EmailToPage module and the renaming function is based on this code from Ryan: http://processwire.com/talk/topic/3299-ability-to-define-convention-for-image-and-file-upload-names/?p=32623 (also see this post for his thoughts on file renaming and why it is the lazy way out - worth a read before deciding to use this module). 
       
       
      NOTE:
      This should not be needed on most sites, but I work with lots of sites that host PDFs and photos/vectors that are available for download and I have always renamed the files on upload because clients will often upload files with horrible meaningless filenames like:
      Final ReportV6 web version for John Feb 23.PDF

×
×
  • Create New...