Jump to content
kixe

Cronjob Database Backup

Recommended Posts

I've always used the all tables option but on my recent mathparser module I've used 3 fields for the same functionality:

1) radio active by default yes/no

2) if no, fields to include

3) if yes, fields to exclude

  • Like 1

Share this post


Link to post
Share on other sites
On 5/27/2016 at 7:06 AM, kixe said:

@Robin S
Thanks for pointing this out. This note is obsolete. I missed to remove it from earlier versions. The cronjob is triggered by every user.
 

Hey Kixe!
Thank you for this great module! I use it quite often.
For most projects I need the backup to get triggered when the client logout.
I don't want to give the client the backup permission, to prevent them to mess things up.
From your comment above it seems the permission is not needed.
But when I logout without the backup permission, the backup is not working.
It would be nice if the backup gets triggered by any user on logout, or at least an option for this.

  • Like 2

Share this post


Link to post
Share on other sites

Hi @kixe,

additionally to what @jploch said above, and when you are already at it ;-), it would be very helpful to have the option for auto-backup on login!

The real life usage of auto backup on logout is not consistent because some users simply leave the backend without logout process. So I would prefer to switch to auto backup on login process. :-)
(Login process can not be skipped like Logout process!)

 

  • Like 5

Share this post


Link to post
Share on other sites

@jploch @horst

Tonight a had a look in the module and I made an update to version 1.1.6

  • I removed any general permission from module. The Backup is triggered now by any user (if a time interval is set).
  • I added trigger option when a user logs in (thanks for the hint @horst).
  • I changed the name for the required permission (trigger on login or logout) from 'db-backup' to 'trigger-db-backup'. Also to prevent permission conflicts with ProcessDatabaseBackupsusing this permission.

 

  • Like 2

Share this post


Link to post
Share on other sites
13 hours ago, kixe said:

@jploch @horst

Tonight a had a look in the module and I made an update to version 1.1.6

Nice! Thanks for the quick update!
After I updated to the new version, the permission 'trigger-db-backup' wasn't available.
I created it manually and now it works. Maybe the permission wasn't created  because I updated, rather than installing a new copy? 
Anyway great update, triggering the backup on login is a nice option! 

  • Like 2

Share this post


Link to post
Share on other sites
4 hours ago, jploch said:

After I updated to the new version, the permission 'trigger-db-backup' wasn't available.
I created it manually and now it works. Maybe the permission wasn't created  because I updated, rather than installing a new copy? 

Yes, the permission will be created on install only.

  • Like 1

Share this post


Link to post
Share on other sites

Hey @kixe after upgrading from 116 to 119 I get this error:

mDyG23p.png

My workaround was uninstalling and installing it again and that worked.

Share this post


Link to post
Share on other sites

@bernhard

Thanks for pointing on this. I fixed this bug. Please update to 1.2.0

  • Like 2

Share this post


Link to post
Share on other sites

on version 1.2.0 for some reason my javascript files that are compiled with procache have the path to the js file in the beginning of the compiled file which is breaking all js on the site... This isn't happening with the earlier versions of the module....

/home/user/public_html/site/modules/CronjobDatabaseBackup/CronjobDatabaseBackup.js?v=120-1572797219;

Share this post


Link to post
Share on other sites

bump problem - still breaks all javascript on websites when using latest version, module is unusable now... sites going down...

Share this post


Link to post
Share on other sites

@kixe

So the issue here is that the module has a JS file named the same as the module - this is a problem because this module is autoload, and as such the module's JS file is polluting the front end output for any of us who use $config->scripts, meaning that our scripts for use on the front end inadvertently includes the module's js file, which is not only unnecessary, but also is breaking my frontend api.

a much better way of loading the JS file is to rename it and then just load the JS on the module's setting page, you can do that by checking the input and loading your js.

For now i just disabled the js file, i don't even know if it is needed, or being used, and this has fixed the problem.

  • Like 1

Share this post


Link to post
Share on other sites

@Macrura My two cents on this: I feel like it's the other way around — $config as a data store is (for the most part) internal to ProcessWire's backend. The $config->scripts and $config->styles arrays are wired up in a way that makes sense for the admin panel to work properly. Using them for the frontend is surely convenient (I've done that myself on a lot of sites) but probably not meant to be used that way and prone to breaking. I've personally stopped using the internal file arrays for that reason and created namespaced versions for the frontend.

  • Like 3

Share this post


Link to post
Share on other sites

@Macrura
Thanks for reporting. I fixed the issue. The JS is required only for the module settings UI. It is now only loaded there.

  • Like 1

Share this post


Link to post
Share on other sites
8 hours ago, d'Hinnisdaël said:

My two cents on this: I feel like it's the other way around — $config as a data store is (for the most part) internal to ProcessWire's backend. The $config->scripts and $config->styles arrays are wired up in a way that makes sense for the admin panel to work properly. Using them for the frontend is surely convenient (I've done that myself on a lot of sites) but probably not meant to be used that way and prone to breaking. I've personally stopped using the internal file arrays for that reason and created namespaced versions for the frontend.

In one sense that could be an interpretation, but another interpretation is that it can be used on the front end, and i know of some other devs that do use it on live sites so i think it's still risky to autoload scripts or styles on a module without just loading the assets where you need them..

Quote

Array used by ProcessWire admin to keep track of what javascript files its template should load. It will be blank otherwise. Feel free to use it for the same purpose in your own sites.

I don't currently know any modules where the $config->styles/scripts load on the front end inadvertently, but i'll keep a lookout. I have a module that loads into those for the purposes of use on the front end (Soundmanager).

The idea is to create an ecosystem where markup generating modules can have a way to load dependencies on the front end - in Soundmanager2, you need to load the various files based on your settings. By checking to see if there is anything in $config-styles/scripts, i can have modules that can load dependencies using the core $config class.

Maybe there is a better way, but $config is a useful class and i'm using it all over the place for front end development..

Share this post


Link to post
Share on other sites
1 hour ago, Macrura said:

i know of some other devs that do use it on live sites so i think it's still risky to autoload scripts or styles on a module without just loading the assets where you need them

The thing is, it's a risk that's carried by the site developer rather than the module developer. If an autoload module adds scripts and styles to $config->scripts and $config->styles it's your site that's at risk of breaking, not the module. Personally I'd never want to take that risk.

The simple solution is to create your own custom FilenameArray in $config that is reserved for front-end usage. This old post from @Soma explains how:

 

  • Like 2

Share this post


Link to post
Share on other sites

@Robin S@Macrura@d'Hinnisdaël

Interesting discussion. I suggest continuing this outside of this thread as it is not directly module-relevant. For my part, I did a useful update because the script doesn't have to be autoloaded everywhere.

Share this post


Link to post
Share on other sites

@kixe, I feel that the settings "Maximum number of backups" and "Remove backups older than" should only apply to backups created by CronjobDatabaseBackup. If the user has made manual backups, or backups have been made by a different module, then CronjobDatabaseBackup should never delete those backups. What do you think?

  • Like 3

Share this post


Link to post
Share on other sites
4 hours ago, Robin S said:

@kixe, I feel that the settings "Maximum number of backups" and "Remove backups older than" should only apply to backups created by CronjobDatabaseBackup. If the user has made manual backups, or backups have been made by a different module, then CronjobDatabaseBackup should never delete those backups. What do you think?

+1

I have automatic backups every hour during development and do manual backups when reaching milestones and they get deleted from time to time so I need to manually restore them. In my case it would be enough to exclude special filenames from deleting.

  • Like 1

Share this post


Link to post
Share on other sites
13 hours ago, Robin S said:

@kixe, I feel that the settings "Maximum number of backups" and "Remove backups older than" should only apply to backups created by CronjobDatabaseBackup. If the user has made manual backups, or backups have been made by a different module, then CronjobDatabaseBackup should never delete those backups. What do you think?

+2

All Backups from CronjobDatabaseBackup must be identified by comment signs, not by filename format. Filename formats may change for some reason.

Share this post


Link to post
Share on other sites
On 4/18/2020 at 4:37 AM, Robin S said:

@kixe, I feel that the settings "Maximum number of backups" and "Remove backups older than" should only apply to backups created by CronjobDatabaseBackup. If the user has made manual backups, or backups have been made by a different module, then CronjobDatabaseBackup should never delete those backups. What do you think?

This should have been already implemented with the option to set a custom storage path. Unfortunately I recognized a bug while looking in this. This bug is fixed now and I made some additional changements. Please update to v1.2.3.

The automated cleanup effects only the custom path and not site/assets/backups/database/ used by ProcessDatabaseBackups. Currently you cannot use ProcessDatabaseBackups to edit backups stored under a custom path.

I added functionality to protect files by 'stopword'. Files having this in its names or descriptions will be protected from beeing deleted by the cronjob. Just set the 'stopword' in the module config.

@bernhard @horst @Robin S
I hope the fix helps and feeds your needs (?)

 

  • Like 1
  • Thanks 2

Share this post


Link to post
Share on other sites

@kixe what do you think of this scenario: CronjobDatabaseBackup writes a Identifier Comment Word into all its files, and if that identifier is missing, just ignore it in all routines.?

So, stopword also sounds good, but needs manual action everytime. The other way round, if I create manual dumps and want to delegate them to CDBs cleaning routine, I can include the identifier word. This has less harm when forgot or mistake on including a word.

 

  • Like 2

Share this post


Link to post
Share on other sites

I actually ended up writing my own backup script because I wanted the backup to run by cron job at a fixed time rather than have the unpredictability inherent in LazyCron.

But in terms of this module my vote would go for having CronjobDatabaseBackup identify the backups that it creates either by filename prefix or something distinctive in the Description as per @horst's suggestion. Then the backups can be safely stored in /site/assets/backups/database/ where they can be viewed and restored via ProcessDatabaseBackups and there is no requirement to use a "file protection string" to protect backups not created by CronjobDatabaseBackup.

Share this post


Link to post
Share on other sites
On 4/21/2020 at 2:32 AM, Robin S said:

I actually ended up writing my own backup script because I wanted the backup to run by cron job at a fixed time rather than have the unpredictability inherent in LazyCron.

I use this 5 lines of code, placed in an access controlled folder under root, to run a cronjob at fixed time. Just set 'cycle' in module settings to 'none' to prevent database backups triggered by LazyCron as well.

<?php namespace ProcessWire;
$root = dirname(__DIR__);
include_once("$root/index.php");
$cdb = $modules->CronjobDatabaseBackup;
$cdb->cronBackup();

 

On 4/20/2020 at 5:04 PM, horst said:

The other way round, if I create manual dumps and want to delegate them to CDBs cleaning routine, I can include the identifier word

Thanks for the input. I thought about it. In the end, I'll leave it as it is, and the inclusion of manually created dumps in the cleaning routine remains the standard. When I manually create a dump, I usually move or download it immediately. I rarely (never?) leave it on the live server and I want the files removed automatically. The default name created by ProcessDatabaseBackups is the database name. Just use this as the 'stopword' and use a different name format in the setup of CronjobDatabaseBackup and voila it works exactly as you want it.  Furthermore there is also the option to save the files in a user-defined directory. Manually created dumps are then never affected by the CDB's cleaning routine.

  • Like 3

Share this post


Link to post
Share on other sites
On 4/18/2016 at 5:19 PM, kixe said:

Yes. The Backup is triggered by the hook function cronBackup() expecting a HookEvent as single argument, which is in fact not needed. If you want to trigger the backup from anywhere, just create a HookEvent object and call the cronBackup() function. Done.
 


$event = new HookEvent;
$modules->get('CronjobDatabaseBackup')->cronBackup($event);

Thanks for the module, it is installed on all PW sites I've ever made.
For the code snippet I still couldn't figure out how to run a database backup from an ordinary cron job.
Will I have to create a php file with the hook and call that from the cron r can I call a module file directly from cron?
Or is it possible to integrate it with the PWCron module?

Update: I got it working in a PHP file this way based on @kixe's answer:

<?php
namespace ProcessWire;

include_once("./index.php");

$event = new HookEvent();
$cron_backup = $modules->CronjobDatabaseBackup;
$cron_backup->cronBackup($event);

 

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By d'Hinnisdaël
      Happy new year, everybody 🥬
      I've been sitting on this Dashboard module I made for a client and finally came around to cleaning it up and releasing it to the wider public. This is how it looks.
      ProcessWire Dashboard

      If anyone is interested in trying this out, please go ahead! I'd love to get some feedback on it. If this proves useful and survives some real-world testing, I'll add this to the module directory.
      Download
      You can find the latest release on Github.
      Documentation
      Check out the documentation to get started. This is where you'll find information about included panel types and configuration options.
      Custom Panels
      My goal was to make it really simple to create custom panels. The easiest way to do that is to use the panel type template and have it render a file in your templates folder. This might be enough for 80% of all use cases. For anything more complex (FormBuilder submissions? Comments? Live chat?), you can add new panel types by creating modules that extend the DashboardPanel base class. Check out the documentation on custom panels or take a look at the HelloWorld panel to get started. I'm happy to merge any user-created modules into the main repo if they might be useful to more than a few people.
       Disclaimer
      This is a pre-release version. Please treat it as such — don't install it on production sites. Just making sure 🍇
      Roadmap
      These are the things I'm looking to implement myself at some point. The wishlist is a lot longer, but those are the 80/20 items that I probably won't regret spending time on.
      Improve documentation & add examples ⚙️ Panel types Google Analytics ⚙️ Add new page  🔥 Drafts 🔥 At a glance / Page counter 404s  Layout options Render multiple tabs per panel panel groups with heading and spacing between ✅ panel wrappers as grid item (e.g. stacked notices) ✅ Admin themes support AdminThemeReno and AdminThemeDefault ✅ Shortcuts panel add a table layout with icon, title & summary ✅ Chart panel add default styles for common chart types ✅ load chart data from JS file (currently passed as PHP array) Collection panel support image columns ✅ add buttons: view all & add new ✅
    • By Pip
      Hi everyone!
      I'm trying out the Login/Register module for my site. Noted that the module assigns the newly registered user to login-register role. 
      Once you modify the login-register role's permissions, particularly adding page-edit, the new member role will be set to guest. 
      Thing is I'd like to grant my new users the power to create their own pages. Any advice? 
      Thanks. 
    • By Gadgetto
      SnipWire - Snipcart integration for ProcessWire
      Snipcart is a powerful 3rd party, developer-first HTML/JavaScript shopping cart platform. SnipWire is the missing link between Snipcart and the content management framework ProcessWire.
      With SnipWire, you can quickly turn any ProcessWire site into a Snipcart online shop. The SnipWire plugin helps you to get your store up and running in no time. Detailed knowledge of the Snipcart system is not required.
      SnipWire is free and open source licensed under Mozilla Public License 2.0! A lot of work and effort has gone into development. It would be nice if you could donate an amount to support further development:

      Status update links (inside this thread) for SnipWire development
      2020-07-03 -- SnipWire 0.8.7 (beta) released! Fixes some small bugs and adds an indicator for TEST mode 2020-04-06 -- SnipWire 0.8.6 (beta) released! Adds support for Snipcart subscriptions and also fixes some problems 2020-03-21 -- SnipWire 0.8.5 (beta) released! Improves SnipWires webhooks interface and provides some other fixes and additions 2020-03-03 -- SnipWire 0.8.4 (beta) released! Improves compatibility for Windows based Systems. 2020-03-01 -- SnipWire 0.8.3 (beta) released! The installation and uninstallation process has been heavily revised. 2020-02-08 -- SnipWire 0.8.2 (beta) released! Added a feature to change the cart and catalogue currency by GET, POST or SESSION param 2020-02-03 -- SnipWire 0.8.1 (beta) released! All custom classes moved into their own namespaces. 2020-02-01 -- SnipWire is now available via ProcessWire's module directory! 2020-01-30 -- SnipWire 0.8.0 (beta) first public release! (module just submitted to the PW modules directory) 2020-01-28 -- added Custom Order Fields feature (first SnipWire release version is near!) 2020-01-21 -- Snipcart v3 - when will the new cart system be implemented? 2020-01-19 -- integrated taxes provider finished (+ very flexible shipping taxes handling) 2020-01-14 -- new date range picker, discount editor, order notifiactions, order statuses, and more ... 2019-11-15 -- orders filter, order details, download + resend invoices, refunds 2019-10-18 -- list filters, REST API improvements, new docs platform, and more ... 2019-08-08 -- dashboard interface, currency selector, managing Orders, Customers and Products, Added a WireTabs, refinded caching behavior 2019-06-15 -- taxes provider, shop templates update, multiCURL implementation, and more ... 2019-06-02 -- FieldtypeSnipWireTaxSelector 2019-05-25 -- SnipWire will be free and open source Plugin Key Features
      Fast and simple store setup Full integration of the Snipcart dashboard into the ProcessWire backend (no need to leave the ProcessWire admin area) Browse and manage orders, customers, discounts, abandoned carts, and more Multi currency support Custom order and cart fields Process refunds and send customer notifications from within the ProcessWire backend Process Abandoned Carts + sending messages to customers from within the ProcessWire backend Complete Snipcart webhooks integration (all events are hookable via ProcessWire hooks) Integrated taxes provider (which is more flexible then Snipcart own provider) Useful Links
      SnipWire in PW modules directory SnipWire Docs (please note that the documentation is a work in progress) SnipWire @GitHub (feature requests and suggestions for improvement are welcome - I also accept pull requests) Snipcart Website  

       
      ---- INITIAL POST FROM 2019-05-25 ----
       
    • By Sten
      Hello
      Till now I hacked something with the twig template but it works no more with new PW versions so I look forward to create a module. I am working on a site in multiple languages : French, English, Italian, German, Spanish, Portuguese, Hebrew, Russian. The new posts are entered in any language with a field for language. Till now, I got twig files to get the translations with constants defined for each part of the pages.
      So I'd like to create a module to include theses files added according to the url /fr/en/...
      Have you some observations to do before I begin about the direction to take ?
      Thank you
    • By ukyo
      Mystique Module for ProcessWire CMS/CMF
      Github repo : https://github.com/trk/Mystique
      Mystique module allow you to create dynamic fields and store dynamic fields data on database by using a config file.
      Requirements
      ProcessWire 3.0 or newer PHP 7.0 or newer FieldtypeMystique InputfieldMystique Installation
      Install the module from the modules directory:
      Via Composer:
      composer require trk/mystique Via git clone:
      cd your-processwire-project-folder/ cd site/modules/ git clone https://github.com/trk/Mystique.git Module in live reaction with your Mystique config file
      This mean if you remove a field from your config file, field will be removed from edit screen. As you see on youtube video.
      Using Mystique with your module or use different configs path, autoload need to be true for modules
      Default configs path is site/templates/configs/, and your config file name need to start with Mystique. and need to end with .php extension.
      Adding custom path not supporting anymore !
      // Add your custom path inside your module class`init` function, didn't tested outside public function init() { $path = __DIR__ . DIRECTORY_SEPARATOR . 'configs' . DIRECTORY_SEPARATOR; Mystique::add($path); } Mystique module will search site/modules/**/configs/Mystique.*.php and site/templates/Mystique.*.php paths for Mystique config files.
      All config files need to return a PHP ARRAY like examples.
      Usage almost same with ProcessWire Inputfield Api, only difference is set and showIf usage like on example.
      <?php namespace ProcessWire; /** * Resource : testing-mystique */ return [ 'title' => __('Testing Mystique'), 'fields' => [ 'text_field' => [ 'label' => __('You can use short named types'), 'description' => __('In file showIf working like example'), 'notes' => __('Also you can use $input->set() method'), 'type' => 'text', 'showIf' => [ 'another_text' => "=''" ], 'set' => [ 'showCount' => InputfieldText::showCountChars, 'maxlength' => 255 ], 'attr' => [ 'attr-foo' => 'bar', 'attr-bar' => 'foo' ] ], 'another_text' => [ 'label' => __('Another text field (default type is text)') ] ] ]; Example:
      site/templates/configs/Mystique.seo-fields.php <?php namespace ProcessWire; /** * Resource : seo-fields */ return [ 'title' => __('Seo fields'), 'fields' => [ 'window_title' => [ 'label' => __('Window title'), 'type' => Mystique::TEXT, // or InputfieldText 'useLanguages' => true, 'attr' => [ 'placeholder' => __('Enter a window title') ] ], 'navigation_title' => [ 'label' => __('Navigation title'), 'type' => Mystique::TEXT, // or InputfieldText 'useLanguages' => true, 'showIf' => [ 'window_title' => "!=''" ], 'attr' => [ 'placeholder' => __('Enter a navigation title') ] ], 'description' => [ 'label' => __('Description for search engines'), 'type' => Mystique::TEXTAREA, 'useLanguages' => true ], 'page_tpye' => [ 'label' => __('Type'), 'type' => Mystique::SELECT, 'options' => [ 'basic' => __('Basic page'), 'gallery' => __('Gallery'), 'blog' => __('Blog') ] ], 'show_on_nav' => [ 'label' => __('Display this page on navigation'), 'type' => Mystique::CHECKBOX ] ] ]; Searching data on Mystique field is limited. Because, Mystique saving data to database in json format. When you make search for Mystique field, operator not important. Operator will be changed with %= operator.
      Search example
      $navigationPages = pages()->find('my_mystique_field.show_on_nav=1'); $navigationPages = pages()->find('my_mystique_field.page_tpye=gallery');
×
×
  • Create New...