ProcessDbMigrate

[MarkE]

6 minutes ago, Flashmaster82 said:

Also when uninstall the module templates are still in the system and can´t be removed because its system templates.

Have you got a clean backup?

[Flashmaster82]

Not of my testsite no. I tried now to install to a clean testsite and it installed the module and the migrations link is under setup but on click

Snapsicles… Compile Error: Class declarations may not be nested (line 9 of E:\Wamp\www\Test2\wire\core\WireDebugInfo.php)

FYI I´m using the master version 3.0.229 on localhost/Wamp.

Edited September 24 by Flashmaster82

[MarkE]

45 minutes ago, Flashmaster82 said:

Compile Error: Class declarations may not be nested (line 9 of E:\Wamp\www\Test2\wire\core\WireDebugInfo.php)

That's odd. This is where it is really helpful to install TracyDebugger! It really is 100% recommended when trying to debug. 'Compile error' would normally indicate a problem with just the file in question (WireDebugInfo.php). Are you sure it hasn't been corrupted? What does your IDE show for that file?

[Flashmaster82]

I did a new installation and now it installed correctly. Maybe there where corrupt files in the database or something.

[Flashmaster82]

Migration Actions
Help (link)

Fatal error: Uncaught Error: Call to a member function bd() on null in E:\Wamp\www\Test1\site\modules\ProcessDbMigrate\DbMigrateHelp.php on line 3
( ! ) Error: Call to a member function bd() on null in E:\Wamp\www\Test1\site\modules\ProcessDbMigrate\DbMigrateHelp.php on line 3.

 

PagesEditor: Error saving field "Migration item" — Can’t save page (id=1065): /admin/repeaters/for-field-121/for-page-1062/1727215638-3146-1/: Call $page->of(false); before getting/setting values that will be modified and saved. [Page::statusCorrupted] fields: dbMigrateType, dbMigrateAction

 

On Migration Actions i cant see "install migration". And also the DB backup, is that a separate module or should that be installed with this module? Its a bit confusing. I dont know how to migrate from one database to another, is there an upload or something?

Edited September 24 by Flashmaster82

[MarkE]

25 minutes ago, Flashmaster82 said:

Fatal error: Uncaught Error: Call to a member function bd() on null in E:\Wamp\www\Test1\site\modules\ProcessDbMigrate\DbMigrateHelp.php on line 3
( ! ) Error: Call to a member function bd() on null in E:\Wamp\www\Test1\site\modules\ProcessDbMigrate\DbMigrateHelp.php on line 3.

v2.0.23 now available should fix that. Just upgrade from the module config (backup first just in case..).

26 minutes ago, Flashmaster82 said:

On Migration Actions i cant see "install migration". And also the DB backup, is that a separate module or should that be installed with this module? Its a bit confusing. I dont know how to migrate from one database to another, is there an upload or something?

At the moment the only 'migration' you will have is the bootstrap to install the module. Please read the help file to understand how to use the module.

[Flashmaster82]

Warning: Attempt to read property "id" on null in E:\Wamp\www\Test1\wire\core\PagesEditor.php on line 317

PagesEditor: Error saving field "Migration item" — Can’t save page (id=1070): /admin/repeaters/for-field-102/for-page-1067/1727216184-5518-1/: Call $page->of(false); before getting/setting values that will be modified and saved. [Page::statusCorrupted] fields: dbMigrateType, dbMigrateAction
Bootstrap not fully installed - Attempting re-install.

[MarkE]

1 hour ago, Flashmaster82 said:

Warning: Attempt to read property "id" on null in E:\Wamp\www\Test1\wire\core\PagesEditor.php on line 317

PagesEditor: Error saving field "Migration item" — Can’t save page (id=1070): /admin/repeaters/for-field-102/for-page-1067/1727216184-5518-1/: Call $page->of(false); before getting/setting values that will be modified and saved. [Page::statusCorrupted] fields: dbMigrateType, dbMigrateAction

What were you trying to do?

[Flashmaster82]

The first error was when i click on help under Migration Action. And the second error was when i created a new migration and hit save.

[MarkE]

22 hours ago, Flashmaster82 said:

The first error was when i click on help under Migration Action. And the second error was when i created a new migration and hit save.

Hmm. I can't replicate either of those. I'm not sure exactly what you are trying to do and would really need more details and a backtrace (from Tracy) to see what is going on. 

While I am happy to try and debug what I can remotely, I wonder if you actually need ProcessDbMigrate at this time. Your comment

On 9/24/2024 at 10:39 PM, Flashmaster82 said:

And also the DB backup, is that a separate module or should that be installed with this module? Its a bit confusing. I dont know how to migrate from one database to another, is there an upload or something?

seems to indicate that it is early days for you with ProcessWire. You don't need ProcessDbMigrate to copy a whole database from one environment to another. The ProcessDatabaseBackups module does this for you (or you can just use phpMyAdmin, but the module is easier). You only need something like ProcessDbMigrate to selectively update parts of a database in one environment from elements in a database in another environment. Typically this is the case with web apps rather than simple websites. The full description of what it does is available here.

[rastographics]

@MarkE Love this module, thank you for sharing it. My brain works better with the way this module works than other migration modules I've tried.

I have it setup and working perfectly in 1 installation.

Using it for a second site, and keep running into an issue after installation.

Immediately after installing in Dev site on my local machine, the page response time goes from <100ms to 1,000-1,500ms, and I think the reason is that on every single page load, the message is at the top:

Bootstrap not fully installed. Attempting to reinstall...

And also a list of 70 or so pages that have been modified or changed. This happens on every page load, front-end, back-end, doesn't matter.

I think because the bootstrap migration is not installing, it is attempting on every page load, which slows down the site. I did not run into this before on my other site.

How can I troubleshoot to figure out what is keeping the bootstrap from completing?

[MarkE]

40 minutes ago, rastographics said:

Immediately after installing in Dev site on my local machine, the page response time goes from <100ms to 1,000-1,500ms, and I think the reason is that on every single page load, the message is at the top:

Bootstrap not fully installed. Attempting to reinstall...

Firstly (after taking a backup), try updating to the latest version 2.0.24 which fixes a few bugs and should be a bit faster.

Second, try and ascertain the cause of the problem, by using the preview on the bootstrap page to identify the differences. You might be able to fix them manually in the database. Otherwise, let me know what it says.

Lastly, as a temporary fix, try unchecking 'Auto-install bootstrap' on the module config page and seeing whether it works OK.

[rastographics]

Thank you, i have been using the latest version  2.0.24. I tried several times also completly uninstalling and reinstalling.

 

I previewed the changes for the bootstrap migration and this is the output:

 

 

Differences between current and install data
Key	current	install
pages->new->/admin/dbmigrations/
> title	(No Value)	DB Migrations
pages->new->/admin/dbcomparisons/
> title	(No Value)	DB Comparisons
Differences in migration definition that do not affect data.json files (but do affect migration.json):	
pages->changed->/admin/dbmigrations/bootstrap/
> title	(No Value)	bootstrap
> dbMigrateLogChanges	(No Value)	2
> dbMigrateSummary	(No Value)	This is the initial migration to install all the required fields, templates and pages for the module to work. It is bootstrapped, on first running the ProcessDbMigrate module, from the json files.
> dbMigrateAdditionalDetails	(No Value)	
If you want to amend this bootstrap, for example to implement changes to the layout of the DbMigration template, then you may need to first make it 'exportable' in the source (development) environment by removing the meta('installable') from this page.

> dbMigrateItem	(No Value)	[
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 1,
            "dbMigrateAction": 1,
            "dbMigrateName": "dbMigrateType",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 1,
            "dbMigrateAction": 1,
            "dbMigrateName": "dbMigrateAction",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 1,
            "dbMigrateAction": 1,
            "dbMigrateName": "dbMigrateName",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 1,
            "dbMigrateAction": 1,
            "dbMigrateName": "dbMigrateOldName",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 2,
            "dbMigrateAction": 1,
            "dbMigrateName": "repeater_dbMigrateItem",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 1,
            "dbMigrateAction": 1,
            "dbMigrateName": "dbMigrateItem",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 2,
            "dbMigrateAction": 1,
            "dbMigrateName": "repeater_dbMigrateComparisonItem",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 1,
            "dbMigrateAction": 1,
            "dbMigrateName": "dbMigrateComparisonItem",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 1,
            "dbMigrateAction": 1,
            "dbMigrateName": "dbMigrateSummary",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 1,
            "dbMigrateAction": 1,
            "dbMigrateName": "dbMigrateAdditionalDetails",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 1,
            "dbMigrateAction": 1,
            "dbMigrateName": "dbMigrateRestrictFields",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 2,
            "dbMigrateAction": 1,
            "dbMigrateName": "repeater_dbMigrateSnippets",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 1,
            "dbMigrateAction": 1,
            "dbMigrateName": "dbMigrateSnippets",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 1,
            "dbMigrateAction": 1,
            "dbMigrateName": "dbMigrateRuntimeControl",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 1,
            "dbMigrateAction": 1,
            "dbMigrateName": "dbMigrateRuntimeAction",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 1,
            "dbMigrateAction": 1,
            "dbMigrateName": "dbMigrateRuntimeReady",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 1,
            "dbMigrateAction": 1,
            "dbMigrateName": "dbMigrateLogChanges",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 1,
            "dbMigrateAction": 1,
            "dbMigrateName": "dbMigrateTrackingScope",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 1,
            "dbMigrateAction": 1,
            "dbMigrateName": "dbMigrateFieldTracking",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 1,
            "dbMigrateAction": 1,
            "dbMigrateName": "dbMigrateTemplateTracking",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 1,
            "dbMigrateAction": 1,
            "dbMigrateName": "dbMigratePageTracking",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 1,
            "dbMigrateAction": 1,
            "dbMigrateName": "dbMigrateTrackingScope_END",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 2,
            "dbMigrateAction": 1,
            "dbMigrateName": "DbMigration",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 2,
            "dbMigrateAction": 1,
            "dbMigrateName": "DbComparison",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 2,
            "dbMigrateAction": 1,
            "dbMigrateName": "DbMigrations",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 3,
            "dbMigrateAction": 1,
            "dbMigrateName": "/admin/dbmigrations/",
            "dbMigrateOldName": ""
        }
    },
    {
        "template": "repeater_dbMigrateItem",
        "data": {
            "dbMigrateType": 3,
            "dbMigrateAction": 1,
            "dbMigrateName": "/admin/dbcomparisons/",
            "dbMigrateOldName": ""
        }
    }
]

[rastographics]

This is the result if I un-check auto-install bootrap and try to install it manually:

 

[rastographics]

Update: I did a fresh install of Processwire (3.0.242) and I get the same problem with failure to install bootstrap migration. 

What I've noticed with their problem is that that the dbMigrate fields and templates are present. But the bootstrap migration will never indicate that it is fully installed.

If I lock the bootstrap migration and try to add new migration on local machine, I can, but when I push the new migration to production, it says "already installed" but nothing changes, because no migration items appear under the migration for some reason. I think the reason is related to the fact that the bootstrap is not getting installed completely but I don't know enough about the raw database structure and how this module works under the hood to know what is going on.

The other site that I have dbMigrate working perfectly, is using 2.0.23. 
Maybe a bug introduced from 2.0.23 to 2.0.24?
 

Edited Monday at 02:18 PM by rastographics

[rastographics]

UPDATE 2: I was able to get the bootstrap migration to install without problems, after deleting the assets/cache/dbmigrations folder (I didn't know about that before) and also using version 2.0.23 of the module.

[MarkE]

9 hours ago, rastographics said:

Maybe a bug introduced from 2.0.23 to 2.0.24?

Maybe. I didn't do much but, given the complexity of the module, it is quite easy to accidentally break something when fixing something else. I suggest you stick with 2.0.23 for now while I investigate.

[MarkE]

Hi @rastographics. I think the problem is in the upgrade method itself and is unrelated to the changes I made in v2.0.24.

1. It may be that there are spurious files in the system which confuse the upgrade. Can you take a look in modules/ProcessDbMigrate/bootstrap/ and templates/DbMigrate/migrations/bootstrap/ ? They should each have two folders - /new/ and /old/ - each of which has two files - data.json and migration.json and nothing else. Sometimes Windows (😒) adds extra files with the suffix 'Zone.Identifier': I am going to do a new version which ignores those.
2. Can you do a comparison beteween the respective files? i.e. Compare modules/ProcessDbMigrate/bootstrap/new/data.json with templates/DbMigrate/migrations/bootstrap/new/data.json etc. Are there any differences? I have managed to replicate your problem where there are differences between the admin path names - it would be helpful to know if that is the cause of your differences too; I am going to work on a fix for that.

[MarkE]

For @rastographics and anyone else interested. I believe I have fixed the upgrade issues (or at least I have fixed them satisfactorily for 3 of my sites). Version 2.0.25 is now in the modules library. As well as fixing upgrade() there are also a small number of other minor fixes (including those made in v2.0.24 which did not upgrade properly). Before upgrading, I suggest you copy your existing working module (to, say, .ProcessDbMigrate.versionnumber) in case the fixes do not work in your set-up.

Please let me know of any issues (or, indeed, success 😀)