Jump to content
kixe

Cronjob Database Backup

Recommended Posts

Hello kixe,

yes you are right, it is Ryans ProcessDatabaseBackups, which is responsible for the backups. So your module does the job quite well - the module creates a new backup after the time period I have choosen. It seems all ok unless I click on the backup to open it, then I get the error message (also at backups which were made with the latest dev version) - I will post this problem in the general section.

Best regards

Share this post


Link to post
Share on other sites

Hello Juergen,

please try to make a backup directly from ProcessDatabaseBackups. And restore from this backup. Did you get the same error?

Thanks regards

Share this post


Link to post
Share on other sites

Independent if the backup is created manually or via cron job - same result.

Best regards

  • Like 1

Share this post


Link to post
Share on other sites

Because it is maybe interesting for others I will answer a question I got in the personal messenger:

Does the module have the capability to run from a standard cron job?

Yes. The Backup is triggered by the hook function cronBackup() expecting a HookEvent as single argument, which is in fact not needed. If you want to trigger the backup from anywhere, just create a HookEvent object and call the cronBackup() function. Done.
 

$event = new HookEvent;
$modules->get('CronjobDatabaseBackup')->cronBackup($event);
  • Like 3

Share this post


Link to post
Share on other sites

I have spent the entire day working with websites with this module.  I am very impressed and thankful that @Kixe worked on this wonderful module.  Additional thanks to @LostKobrakai who mentioned this module in another thread, which got my interest and finally resulted in me testing it out.

I still don't know how I missed understanding the great usefulness of this module initially.

  • Like 3

Share this post


Link to post
Share on other sites

I have a question regarding the condition:

Runs only if current user has db-backup permission.

Is there some way around this? I don't want to give any other roles this permission besides superuser (and the only superuser is me) because I don't want to give site editors the ability to restore backups. But I also don't want to have to be regularly browsing the website myself just to ensure the backup cron job runs.

You mention running the backup from a standard cron job above - how does the db-backup permission come into play here?

  • Like 1

Share this post


Link to post
Share on other sites

@Robin S
Thanks for pointing this out. This note is obsolete. I missed to remove it from earlier versions. The cronjob is triggered by every user.
 

  • Like 4

Share this post


Link to post
Share on other sites

Hi @kixe,

first just want to say thanks for this useful module (if I don't have already)!

And I want to share a usage that propably wasn't intended with your module, but I find it useful for small sites where content changes slightly in a weekly manner or less. I use a hook into Session::logout in the ready.php file that invokes a DB-Backup. Currently the simplyfied code looks like:

/* Autobackups */
$wire->addHookBefore("Session::logout", function(HookEvent $event) {
    if(!wire('user')->hasPermission('db-backup')) return;
    if(wire('modules')->isInstalled('CronjobDatabaseBackup')) {
        // execute a cronBackup
        ignore_user_abort(true);
        set_time_limit(120);
        $cdb = wire('modules')->get('CronjobDatabaseBackup');
        $e = new HookEvent();
        $cdb->cronBackup($e);
    }
    return;
});

This is a bit hacky. What do you think about to embedd a public method to do this more transparent. Also it would be good if the description ($fileinfo) could be set dynamically with this method or as separate method or option. Currently I have hacked this into the module to be able to set it like: "logout horst (CronjobDatabaseBackup)".

Anyway, also without that, it is a big, big helper! :)

 

  • Like 8

Share this post


Link to post
Share on other sites

@horst
Thanks for your feedback and suggestions (I like them all). I pushed an update to github (Version 113) I made this very quickly. It seems to work, but I hadn't time to try out all scenarios. Could you test a bit? Thanks.

Version 1.1.3
NEW: user logout triggers cronjob  (db-backup permission required) instead of cycle interval.
NEW: extension of filename options (user properties)
NEW: custom fileinfo (via module settings). Same features like filename (hookable)

  • Like 3

Share this post


Link to post
Share on other sites

i got a very misterious behaviour today. after some time of debugging i made it down to the following:

i have this script in my _init.php

    function myHook() {
        wire('log')->save('lazy',"backup!");
    }
    // add a hook to your function:
    $pages->addHook('LazyCron::every30Seconds', null, 'myHook');
    wire('log')->save('lazy', 'hook added');
  • install lazycron
  • refresh the site
  • hook added and backup in the logs

BUT

  • install cronjobdatabasebackup
  • refresh the site
  • only "hook added" in the logs!

no matter what i do the cron seems not to be executed.

deleting the file /site/assets/cache/LazyCron.cache solved that problem. can you confirm that @kixe ? or did i do anything wrong? first time for me with both lazycron and your backup module :)

Share this post


Link to post
Share on other sites

hi kixe, some more questions :)

i want to have daily backups of my database, maximum 10 files.

2016-08-05 17_00_10-Modules • ProcessWire • mentalestaerke.at.png

when is the backup trigger triggered? is it triggered by EVERY pageload (also guest users) or only by logged in users? i want the backup be crated also on guest visits. do i have to grant the guest role "db-backup" permission?

next question for understanding: what happens when this role does not have this permission? is it correct, that the cronjob itself will be triggered once a day, but if it is triggered by "guest" it will NOT create a backup and if it is triggered by "superuser" it WILL create one?

i think some more explaining words in the field descriptions would be a great help! :)

thank you for sharing your module!

Share this post


Link to post
Share on other sites
On 6/08/2016 at 3:04 AM, bernhard said:

when is the backup trigger triggered? is it triggered by EVERY pageload (also guest users) or only by logged in users? i want the backup be crated also on guest visits. do i have to grant the guest role "db-backup" permission?

I believe the lazy cron can be triggered on every pageload by every user, including guest, and you don't need to add the db-backup permission to guest for this to happen. I asked about this too and @kixe responded...

On 27/05/2016 at 5:06 PM, kixe said:

Thanks for pointing this out. This note is obsolete. I missed to remove it from earlier versions. The cronjob is triggered by every user.

 

On 6/08/2016 at 3:04 AM, bernhard said:

i think some more explaining words in the field descriptions would be a great help! :)

Agreed. Also, the module description on GitHub and the modules directory seems to contradict the above so perhaps that needs to be corrected?

Quote

This cronjob (hooks on LazyCron) is executed by a call to module: ProcessPageView.
Runs only if current user has db-backup permission. Permission will be added with module installation. Assign the permission to the roles which should trigger the process.

 

  • Like 2

Share this post


Link to post
Share on other sites


@Robin S
Thanks for he help. I am a bit too busy to answer in time.
 

18 hours ago, Robin S said:

Agreed. Also, the module description on GitHub and the modules directory seems to contradict the above so perhaps that needs to be corrected?

Agreed. If I'll find some time, I will make an update. Apart from this I hope module is doing its job ...

Share this post


Link to post
Share on other sites

@since 1.1.5 added CTA Button to ProcessDatabaseBackups page (edit backups) 2016/11/28

for anyone wondering as well ;)

  • Like 3

Share this post


Link to post
Share on other sites


@bernhard

1 hour ago, bernhard said:

for anyone wondering as well ;)

CronjobDatabaseBackup and ProcessDatabaseBackups are siblings. I put them a little closer together.

  • Like 2

Share this post


Link to post
Share on other sites

Hi @kixe,

Have you seen my PR? It would be very convenient to be able to allow the job to run on certain hostnames. This idea is 'stolen' from the ProCache module.

Share this post


Link to post
Share on other sites

Feature request: the ability to have backups sent via email as well as saved on the server.

  • Like 1

Share this post


Link to post
Share on other sites

Thanks for the great Module! How about providing a default name if no name is present? 

#-%Y-m-d-H-i-s%

Or a better description with an example, I had to go into code before I realised that I had to surround "all" of the date format with %. before I tried to surround every parameter (like %Y%-%m% etc)...

Share this post


Link to post
Share on other sites

it will create a default name (database name). or are you talking about a configurable default name?

Share this post


Link to post
Share on other sites

Yes, it uses # as default name resulting in dbname-1.sql, dbname-2.sql etc – so one can not easily tell how old the date is... Since PW itself already uses a name with date appended when backing up to the same folder on upgrades, why not use the same?

The format PW uses:

dbname-Version-YYYY-MM-DD_HH-ii-ss.sql

processwire-3.0.86-2018-01-09_13-36-54.sql

  • Like 1

Share this post


Link to post
Share on other sites

@noelboss
ProcessDatabaseBackups (by Ryan) uses the same default format as CronjobDatabaseBackup (e.g. dbname-1.sql)
whereas the core class WireDatabaseBackup uses the default format: dbname_YYYY-MM-DD_HH-ii-ss.sql

2 hours ago, noelboss said:

dbname-Version-YYYY-MM-DD_HH-ii-ss.sql

I don' t know where this format is in use.

Since CronjobDatabaseBackup is built very close to ProcessDatabaseBackups I will stay compliant and don't want to change the current default format for the filename.
Anyway you have always the option to modify the format for your needs in module settings, even in the format you prefer.

  • Thanks 1

Share this post


Link to post
Share on other sites

Okay, sorry, didn't know - then its probably the update module that does format the files this way. Never the less, I still think having the date in the filename would be the best default in the majority of cases, since if you want to restore a backup, you‘d probably want to know the date anyway. 

But no problem for me, I figured it out, its just that I tought maybe that would help some to get started faster. Still great Module without this change.

  • Like 2

Share this post


Link to post
Share on other sites

Hi Kixe

Great module, thanks! Saved my site this weekend 😄

I have two suggestions as I reviewed my settings. I noticed someting that would probably be easy to implement and at least for me usefull:

– No maximum limit of backups (max): why would that option be limited to 100? If I want to store 6hours backups for 3 months, 100 is not enough.

- More flexible remove option (deadline): Provide an integer 1-99 plus a intervall: Hours, days, weeks, months years.

This way, one can tailor backups and removal very flexible. The current limitations seem very arbitrary for me.  On last thought: If it is possible, it would be nice if one can specify a time at which the backup would start – I know, this would just be an indication but still, one could plan the backups for say during night.

Share this post


Link to post
Share on other sites

Would it make sense to add an option to exclude tables from the backup, instead of choosing the tables to include?

This would be especially useful with regards to the sessions table. I have a project with a moderate amount of visitors, but 1million+ rows in the sessions table. The backup module fails since the process needs too much memory. If I exclude the sessions table, the export shrinks from 150MB to 30MB, which the server can easily handle.

Now, I can already select all tables and leave the table sessions unchecked. However, as soon as I add a field, template, etc., all future backups will be missing that table completely. In that scenario, an exclude option would be the better option.

I might just whip up a PR when I find the time, but was wondering if anybody else feels this is necessary or a good idea.

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By d'Hinnisdaël
      Happy new year, everybody 🥬
      I've been sitting on this Dashboard module I made for a client and finally came around to cleaning it up and releasing it to the wider public. This is how it looks.
      ProcessWire Dashboard

      If anyone is interested in trying this out, please go ahead! I'd love to get some feedback on it. If this proves useful and survives some real-world testing, I'll add this to the module directory.
      Download
      You can find the latest release on Github.
      Documentation
      Check out the documentation to get started. This is where you'll find information about included panel types and configuration options.
      Custom Panels
      My goal was to make it really simple to create custom panels. The easiest way to do that is to use the panel type template and have it render a file in your templates folder. This might be enough for 80% of all use cases. For anything more complex (FormBuilder submissions? Comments? Live chat?), you can add new panel types by creating modules that extend the DashboardPanel base class. Check out the documentation on custom panels or take a look at the HelloWorld panel to get started. I'm happy to merge any user-created modules into the main repo if they might be useful to more than a few people.
       Disclaimer
      This is a pre-release version. Please treat it as such — don't install it on production sites. Just making sure 🍇
      Roadmap
      These are the things I'm looking to implement myself at some point. The wishlist is a lot longer, but those are the 80/20 items that I probably won't regret spending time on.
      Improve documentation & add examples ⚙️ Panel types Google Analytics ⚙️ Add new page  🔥 Drafts 🔥 At a glance / Page counter 404s  Layout options Render multiple tabs per panel panel groups with heading and spacing between ✅ panel wrappers as grid item (e.g. stacked notices) ✅ Admin themes support AdminThemeReno and AdminThemeDefault ✅ Shortcuts panel add a table layout with icon, title & summary ✅ Chart panel add default styles for common chart types ✅ load chart data from JS file (currently passed as PHP array) Collection panel support image columns ✅ add buttons: view all & add new ✅
    • By Pip
      Hi everyone!
      I'm trying out the Login/Register module for my site. Noted that the module assigns the newly registered user to login-register role. 
      Once you modify the login-register role's permissions, particularly adding page-edit, the new member role will be set to guest. 
      Thing is I'd like to grant my new users the power to create their own pages. Any advice? 
      Thanks. 
    • By Gadgetto
      SnipWire - Snipcart integration for ProcessWire
      Snipcart is a powerful 3rd party, developer-first HTML/JavaScript shopping cart platform. SnipWire is the missing link between Snipcart and the content management framework ProcessWire.
      With SnipWire, you can quickly turn any ProcessWire site into a Snipcart online shop. The SnipWire plugin helps you to get your store up and running in no time. Detailed knowledge of the Snipcart system is not required.
      SnipWire is free and open source licensed under Mozilla Public License 2.0! A lot of work and effort has gone into development. It would be nice if you could donate an amount to support further development:

      Status update links (inside this thread) for SnipWire development
      2020-07-03 -- SnipWire 0.8.7 (beta) released! Fixes some small bugs and adds an indicator for TEST mode 2020-04-06 -- SnipWire 0.8.6 (beta) released! Adds support for Snipcart subscriptions and also fixes some problems 2020-03-21 -- SnipWire 0.8.5 (beta) released! Improves SnipWires webhooks interface and provides some other fixes and additions 2020-03-03 -- SnipWire 0.8.4 (beta) released! Improves compatibility for Windows based Systems. 2020-03-01 -- SnipWire 0.8.3 (beta) released! The installation and uninstallation process has been heavily revised. 2020-02-08 -- SnipWire 0.8.2 (beta) released! Added a feature to change the cart and catalogue currency by GET, POST or SESSION param 2020-02-03 -- SnipWire 0.8.1 (beta) released! All custom classes moved into their own namespaces. 2020-02-01 -- SnipWire is now available via ProcessWire's module directory! 2020-01-30 -- SnipWire 0.8.0 (beta) first public release! (module just submitted to the PW modules directory) 2020-01-28 -- added Custom Order Fields feature (first SnipWire release version is near!) 2020-01-21 -- Snipcart v3 - when will the new cart system be implemented? 2020-01-19 -- integrated taxes provider finished (+ very flexible shipping taxes handling) 2020-01-14 -- new date range picker, discount editor, order notifiactions, order statuses, and more ... 2019-11-15 -- orders filter, order details, download + resend invoices, refunds 2019-10-18 -- list filters, REST API improvements, new docs platform, and more ... 2019-08-08 -- dashboard interface, currency selector, managing Orders, Customers and Products, Added a WireTabs, refinded caching behavior 2019-06-15 -- taxes provider, shop templates update, multiCURL implementation, and more ... 2019-06-02 -- FieldtypeSnipWireTaxSelector 2019-05-25 -- SnipWire will be free and open source Plugin Key Features
      Fast and simple store setup Full integration of the Snipcart dashboard into the ProcessWire backend (no need to leave the ProcessWire admin area) Browse and manage orders, customers, discounts, abandoned carts, and more Multi currency support Custom order and cart fields Process refunds and send customer notifications from within the ProcessWire backend Process Abandoned Carts + sending messages to customers from within the ProcessWire backend Complete Snipcart webhooks integration (all events are hookable via ProcessWire hooks) Integrated taxes provider (which is more flexible then Snipcart own provider) Useful Links
      SnipWire in PW modules directory SnipWire Docs (please note that the documentation is a work in progress) SnipWire @GitHub (feature requests and suggestions for improvement are welcome - I also accept pull requests) Snipcart Website  

       
      ---- INITIAL POST FROM 2019-05-25 ----
       
    • By Sten
      Hello
      Till now I hacked something with the twig template but it works no more with new PW versions so I look forward to create a module. I am working on a site in multiple languages : French, English, Italian, German, Spanish, Portuguese, Hebrew, Russian. The new posts are entered in any language with a field for language. Till now, I got twig files to get the translations with constants defined for each part of the pages.
      So I'd like to create a module to include theses files added according to the url /fr/en/...
      Have you some observations to do before I begin about the direction to take ?
      Thank you
    • By ukyo
      Mystique Module for ProcessWire CMS/CMF
      Github repo : https://github.com/trk/Mystique
      Mystique module allow you to create dynamic fields and store dynamic fields data on database by using a config file.
      Requirements
      ProcessWire 3.0 or newer PHP 7.0 or newer FieldtypeMystique InputfieldMystique Installation
      Install the module from the modules directory:
      Via Composer:
      composer require trk/mystique Via git clone:
      cd your-processwire-project-folder/ cd site/modules/ git clone https://github.com/trk/Mystique.git Module in live reaction with your Mystique config file
      This mean if you remove a field from your config file, field will be removed from edit screen. As you see on youtube video.
      Using Mystique with your module or use different configs path, autoload need to be true for modules
      Default configs path is site/templates/configs/, and your config file name need to start with Mystique. and need to end with .php extension.
      Adding custom path not supporting anymore !
      // Add your custom path inside your module class`init` function, didn't tested outside public function init() { $path = __DIR__ . DIRECTORY_SEPARATOR . 'configs' . DIRECTORY_SEPARATOR; Mystique::add($path); } Mystique module will search site/modules/**/configs/Mystique.*.php and site/templates/Mystique.*.php paths for Mystique config files.
      All config files need to return a PHP ARRAY like examples.
      Usage almost same with ProcessWire Inputfield Api, only difference is set and showIf usage like on example.
      <?php namespace ProcessWire; /** * Resource : testing-mystique */ return [ 'title' => __('Testing Mystique'), 'fields' => [ 'text_field' => [ 'label' => __('You can use short named types'), 'description' => __('In file showIf working like example'), 'notes' => __('Also you can use $input->set() method'), 'type' => 'text', 'showIf' => [ 'another_text' => "=''" ], 'set' => [ 'showCount' => InputfieldText::showCountChars, 'maxlength' => 255 ], 'attr' => [ 'attr-foo' => 'bar', 'attr-bar' => 'foo' ] ], 'another_text' => [ 'label' => __('Another text field (default type is text)') ] ] ]; Example:
      site/templates/configs/Mystique.seo-fields.php <?php namespace ProcessWire; /** * Resource : seo-fields */ return [ 'title' => __('Seo fields'), 'fields' => [ 'window_title' => [ 'label' => __('Window title'), 'type' => Mystique::TEXT, // or InputfieldText 'useLanguages' => true, 'attr' => [ 'placeholder' => __('Enter a window title') ] ], 'navigation_title' => [ 'label' => __('Navigation title'), 'type' => Mystique::TEXT, // or InputfieldText 'useLanguages' => true, 'showIf' => [ 'window_title' => "!=''" ], 'attr' => [ 'placeholder' => __('Enter a navigation title') ] ], 'description' => [ 'label' => __('Description for search engines'), 'type' => Mystique::TEXTAREA, 'useLanguages' => true ], 'page_tpye' => [ 'label' => __('Type'), 'type' => Mystique::SELECT, 'options' => [ 'basic' => __('Basic page'), 'gallery' => __('Gallery'), 'blog' => __('Blog') ] ], 'show_on_nav' => [ 'label' => __('Display this page on navigation'), 'type' => Mystique::CHECKBOX ] ] ]; Searching data on Mystique field is limited. Because, Mystique saving data to database in json format. When you make search for Mystique field, operator not important. Operator will be changed with %= operator.
      Search example
      $navigationPages = pages()->find('my_mystique_field.show_on_nav=1'); $navigationPages = pages()->find('my_mystique_field.page_tpye=gallery');
×
×
  • Create New...