MarkE

Solved @bernhard! Table pages_paths was missing all ids < 1017. Rebuilt as follows: $pagePaths = $modules->get("PagePaths"); $int = $pagePaths->rebuild();

Me too. But this, which is even odder, may give some clues: Maybe a database corruption - I'll try Adminer.

Nope. As you see, the parent is correct. I also copied the path directly from the settings tab of the page.

Thanks @bernhard. So I tried it in a different installation and it worked fine, so I'm left puzzling about what might be odd in the first case ?

Now released - see here:

Sure. Part 1 is here: https://1drv.ms/v/s!AmS0-Sk4lDz9h5tDqJwj50qq6-KEhg and part 2 here: https://1drv.ms/v/s!AmS0-Sk4lDz9h5tS2DJsT94F92fJmw If you want to play with it over the holidays ?, the best thing to do is to create a clean PW installation - ddev is great for doing this quickly - and I would recommend add ProcessDatabaseBackups and TracyDebugger. Start by following similar steps to the video. Note that, in the video, I have 'simulated' the migration installation by taking a backup before adding new fields etc. Take another backup after creating the migration, then restore the original one, changing the database name in the module config. And don't forget to RTM: https://metatunes.github.io/DbMigrate/help.html

Demo part 2, showing the installation of the migration was going to be here, until I realised that the file constraint is for the thread as a whole! You can see it here: ProcessDbMigrate video - target - 1703029659248.mp4 See post 4 for HD video.

Season's Greetings ProcessWirers! I hope you enjoy the gift of this module, but use with care... TLDR: This module captures changes made in the development environment so that they can be easily migrated to the live environment without needing to specify the changes or write any code. The demo below gives a brief overview. Want to read? Read on. One of the (few) problems with ProcessWire, in my opinion, is the lack of any native way of handling migrations. Given that PW is such a powerful tool capable of sophisticated and complex web-based applications, this is less than ideal. There is a solution, however, in RockMigrations which accomplishes a lot in a controllable way, provided you are happy to specify your database set-up in code rather than via the UI (albeit that the latest versions allow you to grab much of the required code from the UI). If that suits your need, great. Around the same time as the first versions of RockMigrations, I started developing my own UI-based migrations module, which I have been using with reasonable success for some time. I halted development of the module for a while as RockMigrations developed and I considered switching to that route. However, I decided that my module suited me better and that a real improvement could be made if it was effectively automated so that I no longer needed to specify a migration. So that is exactly what it does: after configuring the module, you add a new migration page with ‘log changes’ enabled (which includes determining what types of objects are relevant for the migration) and work on your development system. Once you have made the desired changes (and tested them!) in the development environment, you go back to the migration page where it has magically captured the objects which have changed and listed them in dependency order. You then ‘export’ the changes, which creates json files to be uploaded to the live environment (via Git or FTP etc.), where they are then ‘installed’ to re-create the changes in the live system. The demo below illustrates this briefly. This first demo shows the creation of a migration. The installation demo will be in the next post, because of size constraints. See post 4 for HD video. Video-source small.mp4 There is a very extensive manual which covers all the features of the module, not just this ‘automatic’ method. Available on github at https://github.com/MetaTunes/ProcessDbMigrate and in the modules library here. PLEASE NOTE that this is still in 'alpha'. Do not use in production without fully testing and backing up at every stage. It is quite complex so, although I have tried hard to eliminate bugs, there will inevitably be some left!

I'm probably being dim, but can anyone explain why $pages->get('/processwire/access/permissions/') returns a null page? See Tracy console below:

Have you tried $page->save() ?

The specific problem above was fixed in v3.0.203. There were other problems - see https://github.com/MetaTunes/CustomDependSelects Does that help? (If it does, maybe I should add it to the modules library?)

I agree that would be nice. I’m still hoping to release my module before Christmas but you know what it’s like - tracking down bugs. For example I’ve been adding Page Table Field compatibility as an alternative to RepeaterMatrix, which throws a few issues especially if the table is hosted under a “foreign” parent. I’m just about there now….. To your specific point @bernhard, it had already occurred to me that there might be a way of combining the best of both modules - e.g. GUI in the development environment with deployment via code; or maybe creating a RockMigrations migration from my json files. However, I don’t even want to think about this until I have released my module and had feedback from other users.

With my system, the times are filled in automatically from the default for the property or system, but can be changed if, for example, early access is granted. Since the system also generates email correspondence with the guests, it needs to tell them the arrival and departure times. Sure, it could be done with separate fields, as it is now. Having thought about it a bit more, I can see that there might be some complications arising from including times in the range and you would certainly want it to be hidden in some circumstances. I have another app with event booking, where date and time are essential. Using the current interface, similar to the one in my example, is a bit fiddly.

You'll have to write some code for that @ShadowByte. This is my code for my blog tags (in Latte) which I'm sure you can adapt for your purposes (my blogs are in a repeater matix field here): {var $blogsPage = $page->motif_display_pageref} {var $blogs = $blogsPage->find("template=MotifDisplay, parent=$blogsPage")} {var $tags = []} {foreach $blogs as $blog} {foreach $blog->motif_layout_components as $component} {if $component->type == 'blog-post'} {var $blogTags = $component->motif_blog_tag->explode('title')} {* {bd($blogTags, 'blogtags')}*} {foreach $blogTags as $blogTag} {if !array_key_exists($blogTag, $tags)} {do $tags[$blogTag]['name'] = $blogTag} {do $tags[$blogTag]['count'] = 0} {/if} {do $tags[$blogTag]['count'] += 1} {/foreach} {/if} {/foreach} {/foreach} {* Sort the tags in descending order of frequency so that the most used are at the top *} {var $tagCount = []} {foreach $tags as $key => $tag } {do $tagCount[$key] = $tag['count']} {/foreach} {do array_multisort($tagCount, SORT_DESC, $tags)} {*<section>*} <ul n:foreach="$tags as $tagKey => $arr"> <li> <a href="{$blogsPage->url . 'tag/' . $tagKey}">{$tagKey}</a> </li> </ul> {*</section>*}

That's not how RockMigrations works. At least not at the moment. And that has several good reasons. I'm not saying that such an approach is bad. But it is very complicated to implement, it comes with a lot of (maybe unsolvable) problems and even if everything worked it has a lot of limitations by design. I didn't want that for RockMigrations, so I built it differently. Quite correct, except "even if everything worked it has a lot of limitations by design". I'm not sure what those might be. ProcessDbMigrate is working pretty well now and I will let people know when I am happy with the testing (and documentation!). Yes, it was very complicated to implement but it is now coming together nicely. I think RockMigrations is great for people who want to work that way, but I chose a different path and will happily accept criticism (and, even better, constructive suggestions) if it doesn't work as intended. That is a risk, but I don't find it a problem with my module. It deliberately omits some unnecessary properties. Yeah - that's nice. I wonder.... ProcessDbMigrate uses multiple migrations and you can define one to export the database to another project. I am using it to install another module in new projects and plan to use it to create a template project from a live one. I'll admit that it is early days yet and that it is probably not as flexible as RockMigrations, but I'm not convinced it is a 'huge drawback'. With ProcessDbMigrate, you use a Page class module in the same way (without all the field definitions etc.), create a very simple 'export' migration of the required fields and templates manually (just specify the names in repeaters), sync to your target and hit 'import'. ?Agreed, but I don't blame @bernhardfor that. I would welcome his comments when my module is released. True in my case too, as the module interprets them and presents differences etc., but they have been useful in debugging ? ProcessDbMigrate uses names and paths not ids. If a name is changed it tracks that, so that it updates the target correctly. So ids do not need to be the same. You cannot change fields etc. in the target if they are the subject of a live migration (unless you disable that feature), but you can after it has been fully installed and 'locked'. Of course if someone updates the live then someone else creates a new migration from an old database version that over-writes it, that is a problem! So far I have not had a problem with any type of change that you can make in the UI (including parent id and template id references - these are converted to names and then converted back on installation) Sorry if the above sounds a bit defensive, but I just wanted to set the record straight. Of course, you can disagree once the module is released and you have used it! Just to emphasise a couple of points: RockMigrations is a great free module and if it's what you want then use it. ProcessDbMigrate does what I want and I thought it would be helpful to share it (but only when I am happy that it will not cause any difficulties). I have no desire ever to sell it or to position it as some sort of 'competitor', but I would be grateful if others do not knock it without just cause.

I am assuming you selected 'text tags' as the inputfield type on the field 'input' tab and also selected a parent and template for the tags. Further down that tab, you need to select 'Allow new pages to be created from field' - please note the requirements listed there.

Use a Page Reference field for your tags. Then you can do pretty much what you will.

One thing I should mention (and would appreciate some feedback on) is that my migrations module (and I guess any that uses json files) is declarative. Even if you create a migration automatically while making your database changes, what happens is the module logs what objects have changed and exports the final state. It does not operate as a 'macro' logging each change separately. The disadvantage of this is that problems might arise where there is a 'dependency cycle' - e.g. a new field (say a page ref type) depends on a new template which includes the field. With my module, if this happens while you are logging changes it will (should ?) warn you. If you create the migration manually and then 'sort on save' to get the items in dependency order, it will give you an error message (see video at end). I think there are 3 ways of dealing with this: Do more than one migration, so that all required objects are present in the database. Include an item twice in the migration - firstly without the dependency, then with the dependency after the other object has been added. Install the migration twice and hope it will sort itself out (i.e. the misssing objects will be there on the second installation). My module encourages (1) but permits (3) - as you will see. It does not attempt (2) - this might be feasible by detecting the cycle, and breaking it for the first item, but a bit complex to implement. I'd be interested in any thoughts people have on this issue (and indeed on how RockMigrations deals with it - @bernhard?). Meanwhile, to amuse you (??) here is a video of a big complex migration I did as part of the testing process. As you will see, there are lots of cycles, but it installed with just two clicks of the 'install' button ?. cycle errors.mp4

Will do. Please note that the code on GitHub may be behind the latest version and may well contain bugs. I will post on the Forum when I am happy with it. I haven't tried this. It might work if you have not changed any pages at all. I would be afraid of breaking relationships and, personally, I sometimes use pages to hold settings that are not user-updatable and which I might need to migrate, so that would not be a solution for me. Fair point. Particularly re any special field types introduced in modules. ProcessDbMigrate does cater for most of them (I am just updating it for FieldtypePageTable) including some which store attributes as objects, but would need updating if there are major changes or additions to the core (my guess, though, is that these are very unlikely to be breaking changes). It is clear to me from previous discusssions that @ryanis not convinced of the need for this in the core and, indeed, does not feel the need for it personally. I consider that a shame as the lack of this functionality is in my view the only major missing feature in ProcessWire. The topic has been debated here numerous times with no clear conclusion. As a consequence, those of us who feel the need have had to turn to modules - either our own or others'. RockMigrations has been released for a while and is well-supported. Both that and ProcessDbMigrate are open-source so, unless @ryanchanges his mind, it will be up to the community to contribute to module development or, indeed, take it over should the original author no longer maintain it. That will depend on how important the community thinks it is. I think I reported some issues without any action being taken. Also, I recall that the export functions were left as work in progress some time ago. But I may mis-remember...

I have a json based migrations module that I released some time ago (ProcessDbMigrate). It needed further work, so I have not publicised it further after the initial proof of concept. In the meantime I have improved it enormously and tested it quite extensively with multiple field types including RepeaterMatrix. It is almost ready for re-release. It is a completely different concept to RockMigrations, which is an excellent and well-established module. My module will automatically track database changes as you make them in the back end UI and then export json for installation in the target. It also provides for roll-back, database comparisons and much more. Hopefully out before Christmas. PS I found during the course of development that there are quite a few flaws in the native export functions and had to re-write quite a bit.

Actually, I realised that even for the cottage booking system, although the booking form is only dates, not times, I use date and time in the back end because the time is the access/leave time (see below) - the default times are auto-filled from settings

That’s brilliant @ryan. I wish I’d had that a few years back when I was developing my holiday cottage booking system. Could have done the booking form for it in a jiffy! Shame it is only days though, not times, as I can’t use it for events in an app I am currently redeveloping.

@da² What I meant was being able to add them interactively in the back end

There also some (paid for) page builder modules that will do this (and more). I have my own page builder module which I might release for free, but which uses repeater matrix. So it looks like a choice between paying some money or writing some code.

I thought YOOtheme pro was a Wordpress/Joomla thing