[SOLVED] Rockmigrations creates duplicates of repeater fields in templates table (template_id corrupts database)

[thei]

i  started to use Config Migrations in an existing processwire application. In directory site/RockMigrations/fields i created all the files defining the fields. Up to now i created only two templates (i.e. 2 of 15 but this shouldn't matter)  in the appropriate templates folder - so far. yesterday my database crashed after clicking the admin modules page - a PDO Error 1062 / duplicates... so i did some research in the database. i found the templates table contains something like duplicated repeater field rows - the field 'contact' has some duplicates with same name followed by numbers 'contact8' 'contact9' etc. I could repair the database by deleting these strange rows.   When i click on module refresh, some new repeater field rows will be created. The crash seems to be a rare case. Such rows can exist and the application still works in some way but some content of repeater fields is missing on pages.

The problem occurs with repeater fields only. Even when i removed the Config Migrations files for these repeater fields these repeater fields have "doubles" in the template fields.

When i switch of RockMigrations this behavior disappears.   

RockMigrations 6.5.0  / PW 3.0.227

Edited 5 hours ago by thei
solved

[bernhard]

Hey @thei thx for the report. Could you please provide steps to reproduce this issue so that I can investigate?

Or even better do you have time to find a fix for it?

[thei]

Thanx for your quick reaction.

Hm,   "steps"

i installed 6.5.0 (previously a had an old version (5.2..?) installed) i copied the code from the RockMigrations code field to set up the files using fieldname as filename in RockMigrations/fields. e.g. 'label.php' below

<?php
return [
   'collapsed' => 0,
   'columnWidth' => 100,
   'flags' => 0,
   'icon' => 'bookmark-o',
   'inputfieldClass' => '',
   'label' => 'Label',
   'maxlength' => 2048,
   'minlength' => 0,
   'pattern' => '',
   'placeholder' => '',
   'required' => '',
   'requiredAttr' => '',
   'requiredIf' => '',
   'showCount' => 0,
   'showIf' => '',
   'size' => 0,
   'stripTags' => '',
   'tags' => 'text',
   'textformatters' => [
       0 => 'TextformatterEntities',
   ],
   'themeBlank' => '',
   'themeBorder' => '',
   'themeColor' => '',
   'themeInputSize' => '',
   'themeInputWidth' => '',
   'themeOffset' => '',
   'type' => 'FieldtypeText',
];

i remember i did some changes via GUI and did not update the appropriate field file.

At the moment i have no idea how/where to start with debugging RockMigrations

 

 

[bernhard]

Thx. Please share exactly which field names you used and which db entries it created. That might save time when trying to duplicate it.

[thei]

<?php
/* repeater field contact */
return [
    'accordionMode' => 0,
    'allowContexts' => [ 0 => 'repeaterTitle', ],
    'collapsed' => 0,
    'columnWidth' => 100,
    'familyFriendly' => 0,
    'fields' => [
        'desc' => [ 'rows' => 1, ],
        'role' => [ 'columnWidth' => 25, ],
        'email' => [ 'columnWidth' => 25, ],
        'phone' => [ 'columnWidth' => 25,],
        'mobile' => ['columnWidth' => 25, ],
    ],
    'flags' => 0,
    'icon' => 'user-md',
    'label' => 'Contact',
    'lazyParents' => '',
    'loudControls' => 0,
    'parent_id' => 0,
    'rememberOpen' => 0,
    'repeaterAddLabel' => '',
    'repeaterCollapse' => 3,
    'repeaterDepth' => '',
    'repeaterLoading' => 1,
    'repeaterMaxItems' => '',
    'repeaterMinItems' => '',
    'repeaterTitle' => '',
    'required' => '',
    'requiredIf' => '',
    'showIf' => '',
    'tags' => 'address organisation',
    'template_id' => 0,
    'themeBorder' => '',
    'themeColor' => '',
    'themeOffset' => '',
    'type' => 'FieldtypeRepeater',
];

<?php
/* field 'desc' */
return [
    'collapsed' => 0,
    'columnWidth' => 100,
    'contentType' => 0,
    'flags' => 0,
    'htmlOptions' => '',
    'icon' => 'sticky-note-o',
    'inputfieldClass' => 'InputfieldTextarea',
    'label' => 'Description',
    'maxlength' => 0,'minlength' => 0, 'placeholder' => '', 'required' => '', 'requiredAttr' => '',
    'requiredIf' => '', 'rows' => 5, 'showCount' => 0, 'showIf' => '', 'stripTags' => '', 'tags' => 'text',
    'textformatters' => [ 0 => 'TextformatterEntities',  ],
    'themeBlank' => '',
    'themeBorder' => '',
    'themeColor' => '',
    'themeInputSize' => '',
    'themeOffset' => '',
    'type' => 'FieldtypeTextarea',
];

<?php
/* field 'role' */
return [
    'collapsed' => 0,
    'columnWidth' => 100,
    'flags' => 0, 'icon' => 'black-tie','inputfieldClass' => '','label' => 'Role',
    'maxlength' => 2048,  'minlength' => 0,
    'pattern' => '',  'placeholder' => '', 'required' => '',
    'requiredAttr' => '', 'requiredIf' => '',
    'showCount' => 0,  'showIf' => '',
    'size' => 0,  'stripTags' => '',  'tags' => 'organisation',
    'textformatters' => [  0 => 'TextformatterEntities',   ],  
    'themeBlank' => '',  'themeBorder' => '',  'themeColor' => '', 'themeInputSize' => '', 'themeInputWidth' => '','themeOffset' => '',
    'type' => 'FieldtypeText',
];

 

<?php
/* email */
return [
    'collapsed' => 0, 'columnWidth' => 100,
    'flags' => 9,   'icon' => 'at',
    'inputfieldClass' => '',  'label' => 'E-Mail Address',
    'maxlength' => 255, 'minlength' => 0,
    'pattern' => '', 'placeholder' => '', 'required' => '',
    'requiredAttr' => '',  'requiredIf' => '',
    'showCount' => 0, 'showIf' => '',
    'size' => 0,  'stripTags' => 1, 'tags' => 'communication',
    'textformatters' => [  0 => 'TextformatterEntities', ],  
    'themeBlank' => '', 'themeBorder' => '', 'themeColor' => '',  'themeInputSize' => '',  'themeInputWidth' => '', 'themeOffset' => '',
    'type' => 'FieldtypeText',
];
~          

 

<?php
/* phone */
return [
    'collapsed' => 0, 'columnWidth' => 100, 'flags' => 0, 
    'icon' => 'phone', 'inputfieldClass' => '', 'label' => 'Phone', 'maxlength' => 2048, 'minlength' => 0,
    'pattern' => '', 'placeholder' => '', 'required' => '', 'requiredAttr' => '', 'requiredIf' => '',
    'showCount' => 0, 'showIf' => '', 'size' => 0, 'stripTags' => '', 'tags' => 'communication',
    'textformatters' => [  0 => 'TextformatterEntities', ],  
    'themeBlank' => '', 'themeBorder' => '', 'themeColor' => '', 'themeInputSize' => '', 'themeInputWidth' => '', 'themeOffset' => '',
    'type' => 'FieldtypeText',
];

 

<?php
/* mobile */
return [
    'collapsed' => 0, 'columnWidth' => 100, 'flags' => 0,  'icon' => 'mobile',
    'inputfieldClass' => '', 'label' => 'Mobile', 'maxlength' => 2048, 'minlength' => 0,
    'pattern' => '', 'placeholder' => '', 'required' => '', 'requiredAttr' => '', 'requiredIf' => '',
    'showCount' => 0, 'showIf' => '', 'size' => 0,
    'stripTags' => '', 'tags' => 'communication',
    'textformatters' => [  0 => 'TextformatterEntities',  ],  
    'themeBlank' => '', 'themeBorder' => '', 'themeColor' => '', 'themeInputSize' => '', 'themeInputWidth' => '', 'themeOffset' => '',
    'type' => 'FieldtypeText',
];
~  

 

when i put the files shown above in RockMigrations/fields and click module refresh...

| 51 | repeater_contact  | {"noChildren":1,"noParents":1,"slashUrls":1,"pageClass":"RepeaterPage","noGlobal":1,"compile":3,"modified":1736510539,"_lazy":1,"_rockmigrations_log":"--- 2025-01-10 13:02:19 ---\n"} |
| 77 | repeater_contact7 | {"noChildren":1,"noParents":1,"slashUrls":1,"pageClass":"RepeaterPage","noGlobal":1,"compile":3,"modified":1736933902,"_lazy":1,"_rockmigrations_log":"--- 2025-01-15 10:38:22 ---\n"} |
| 79 | repeater_contact8 | {"noChildren":1,"noParents":1,"slashUrls":1,"noGlobal":1,"compile":3,"modified":1736933910,"_rockmigrations_log":"--- 2025-01-15 10:38:30 ---\n"}                                      |
| 80 | repeater_contact9 | {"noChildren":1,"noParents":1,"slashUrls":1,"pageClass":"RepeaterPage","noGlobal":1,"compile":3,"modified":1736933916}                                                                 |

 

 

[thei]

at the moment i cannot reproduce the effect, when moving away 'contact.php' and the problem does not disappear....  Cache? i deleted the cache

[bernhard]

19 minutes ago, thei said:

at the moment i cannot reproduce the effect, when moving away 'contact.php' and the problem does not disappear....  Cache? i deleted the cache

Sorry I don't understand.

[thei]

above i wrote "Even when i removed the Config Migrations files for these repeater fields these repeater fields have "doubles" in the template fields"  i.e. i couldn't reproduce this effect (new repeater-field rows when repeater-field Config-Migrations file is moved away.

But: a repeater field Config - Migrations file produces to an improper database when refreshing modules is clicked

[thei]

 

imported production database, installed RM 6.5 and copied RockMigrations folder with fields and repeater fields files. So this is the first effect:  PDO Error 1050

[thei]

The SQL error mentioned above is not related to RM.

but here some testing that show the strange behavior when updating / creating repeater fields - may be this helps

A) Situation: repeater field 'contact' exists, all caches cleared

MariaDB [PW]> select id,name,data from templates where id = 51;

| id | name             | data
| 51 | repeater_contact | {"noChildren":1, ... ,"modified":1727544508} |


1) running Config Migrations v6.7 with a ...fields/contact.php:
| id | name              | data                                                                                                         |
| 51 | repeater_contact  | {"noChildren":1, ... ,"modified":1737381992,"_lazy":1,"_rockmigrations_log":"--- 2025-01-20 15:06:31 ---\n"} |
| 68 | repeater_contact1 | {"noChildren":1, ... ,"modified":1737382002}                                                                 |

2) remove contact.php and refresh: no changes

3) delete from templates where id=68;
   and then refresh: a new contact repeater field without Config Migrations having a contact.php

| 51 | repeater_contact  | {"noChildren":1, ... ,"modified":1737381992,"_lazy":1,"_rockmigrations_log":"--- 2025-01-20 15:06:31 ---\n"} |
| 69 | repeater_contact2 | {"noChildren":1, ... ,"modified":1737382565}                                                                 |

4) delete from templates where id=69;
   disable Config Migrations in Module Settings
   and then refresh: a new contact repeater field without Config Migrations active:

| 51 | repeater_contact  | {"noChildren":1, ... ,"modified":1737381992,"_lazy":1,"_rockmigrations_log":"--- 2025-01-20 15:06:31 ---\n"} |
| 70 | repeater_contact3 | {"noChildren":1, ... ,"modified":1737382877}                                                                 |
================================================================
B) Situation: repeater field 'contact' DOES NOT exists, all caches cleared


1) running Config Migrations v6.7 with a ...fields/contact.php:
| 68 | repeater_contact  | {"noChildren":1, ... ,"modified":1737383770,"_rockmigrations_log":"--- 2025-01-20 15:36:10 ---\n"} |
| 69 | repeater_contact1 | {"noChildren":1, ... ,"modified":1737383771,"_rockmigrations_log":"--- 2025-01-20 15:36:11 ---\n"} |
| 70 | repeater_contact2 | {"noChildren":1, ... ,"modified":1737383771}                                                       |


Rock-Migrations Log entries:
Trigger RockMigrations::migrationsDone
--- config migration hook: afterData (0 files) ---
/site/RockMigrations/fields/contact.php
--- second run: migrate data ---
--- config migration hook: beforeData (0 files) ---
--- config migration hook: afterAssets (0 files) ---
Tag:
Name: contact
--- first run: create assets ---
--- config migration hook: beforeAssets (0 files) ---
--- create PHP constant traits ---
### Running Config Migrations ###
### Migrate items with priority #1000 ###
Running migrations from watchfiles ...
Detected change in /site/RockMigrations/fields/contact.php
-------------------------------------

 

[bernhard]

@thei you have lots of unnecessary properties in your field definitions. For example "template_id = 0" and "parent_id = 0"

This messes things up. Please remove them. I have added a note to hopefully avoid such issues in the future:

This is all you need for a repeater:

return [
  'label' => 'TEST',
  'type' => 'FieldtypeRepeater',
  'fields' => [
    'title',
    'mytext',
  ],
  'repeaterTitle' => '#n: {title}',
  'familyFriendly' => 1,
  'repeaterDepth' => 0,
  'tags' => 'test',
  'repeaterAddLabel' => 'Add New Item',
];

Please mark this topic [solved] thx!

[thei]

Thanks you very much for your help!

i did some tests and found out: the only bad guy is 'template_id' (or 'template_ids'). Now i eliminated only this parameter and ran the whole bunch (including the "unnecessary parameters"). AND it works fine.

I copied the code from the RockMirations Code field. i expected this generated code from RockMigration would be the correct code for setting up the fields.

The generated code includes template_id => 0 and  sometimes template_id => ''   both cause a corrupted database.

 

Edited 5 hours ago by thei
formatting

[thei]

HOW can i set this topic to "solved" ?

... there is no "solved" button. Hm, may be inserting in the topic title ....    😄

Edited 7 hours ago by thei
late thoughts

[bernhard]

Thx. I've also added a check that should remove 'template_id' from the shown code of repeater fields.