Peter Falkenberg Brown

Hi All,

I've brought my wee script forward to the point of functionality for my use, minus a couple of things I'll be adding, like a routine to create a list of 301 redirects that I'll be pasting into my .htaccess file on the "old" domain.

I'm also going to write a separate small script to click through pages to migrate that will create a list of href's in each page's body text, so that I can see if they have to be edited. In my case, and my experience, href's can have so many variations, including links to external pages, links to internal PW pages that have been moved *internally*, and links to PW pages that have moved to an external site (e.g. the site being migrated to, using a different url). So I'm not trying to incorporate href modification into the script at this time.

I realize that Adrian @adrian has created a very complex and thorough blackbox module for migrations. My script is in no way a replacement for his *much more* sophisticated module. As I mentioned, and as I've noted in the script, I wrote this for a very narrow use-case that may only be useful to me. No idea, actually.

I'm eager to dig deeper and deeper into ProcessWire, and move from my own procedural code method into object-oriented code. Lot's of catching up to do, since for the last 13 years I've been managing servers, etc, and didn't code all day, every day. But my oh my, I LOVE ProcessWire. What a joy to use. ?

EDIT: May 26, 2020: I've added the routines listed above to v1.1, pasted in a new response below, dated May 26. I also fixed an error with the chown action. See post further down, for the script.

Peter

Hi @adrian,

I ran the exporter and importer on 1 page, and had the following issues:

(I used Append as you suggested.)

- The template file was still overwritten

- the parent page was imported, probably because the parent page didn't exist in the target site (by design).
(I only wanted to migrate the pages under one section to a new section in the target site.)

- the images and pdf's were in the zip file but they did not upload to the files/ID dir

- the links in the body text still had the old id in the source.

- and when I selected "Edit Imported Content" it failed and produced an error - "Missing required ZIP or JSON Source"
(when I didn't select that, the second time, it worked, but with the issues above)

Yours,

Peter

1 hour ago, adrian said:

Check the import settings where is says "APPEND will not change settings of existing fields, nor the content of existing pages."

Thanks @adrian!

I'll give it a whirl.

Best regards,

Peter

Hi @kongondo,

Quote

$image_dir = $config->paths->files;
$old_domain_file_dir = $SECONDARY->config->paths->files
// etc

Great suggestion! So much better.

On your comment about the find v. get -- you are correct, but that was just a test bit of code. My real export will have multiple pages via a find.

Thanks VERY much for your help!

Peter

Thanks, @Ivan Gretsky and @kongondo,

My old site is on PW 3.0.42, whereas the module is in 3.0.72.

My new site is on 3.0.148.

Looks like I'll have to upgrade.

Peter

Hi @kongondo,

You wrote:

Quote

For these kind of things, I use the (some features still experimental) module Pages Export/Import, in-built field export and in-built template export.

I looked for that in my Modules setup and also on the PW modules page, and couldn't find it. Do you have a link to it?

Thanks,

Peter

Hi @adrian,

I installed your excellent Migrator module, and saved a zip file for the one page that I've been testing.

After inspecting the zip file, I noted that you do indeed bring over the template file and the field structure.

In my case, my destination install has identical fields already, but in some cases the the fields or template files might conflict, so overwriting would be a problem.

I didn't see a selection to only migrate content, and not fields and templates.

It seems to me that your module is perfect for a certain use case, but not necessarily for mine, which only involves migrating content.

Unless I missed something. ?

Peter

Hi @adrian,

Yes, I can see that your Migrator module looks really fantastic. I'll take a look at it.

I still want to deepen my knowledge of the image steps, however. I did notice that your list of migrated items includes the cropped images, which I did not address, partly because I didn't have any cropped images.

Thanks again!

Peter

Hi @adrian,

I don't think I was really trying to reinvent the wheel. I was honed in on a more specific, "narrow-band" problem of the technical requirements to copy images in pages into a replicated PW page. I wanted to pursue that partly to understand what it took to do that one task from a PW to PW website, but also for other migrations, from other CMS's, into a PW install.

In other words, I'm seeking to increase my knowledge of those image-related steps, rather than just use a plug-in module.

I'll look at Migrator too.

When you mention a lot of problems to solve, are you talking about the image copy task, or something else?

Thanks,

Peter

Note that the code above is a rough first draft, and doesn't include code to copy files, like PDFs.

I've already started to change the structure of the file to accommodate that extra step, etc.

The code above is just meant as a proof of concept, to see if I'm going in the right direction.

Feedback from PW gurus is deeply appreciated. (Or anyone else, too!)

Peter

==> Edit: May 23, 2020 - see second version of this script, posted below on 5/23. I cleaned it up and added the function to copy non-image files.

=======================

Hi All,

I dug in, testing my code against one page that had two images with 2 variations and descriptions.

I got to the point where my code successfully copied that page to a different PW instance; set a few data fields (not all of them); added the images and descriptions; and copied the variation image files (and chowned them to the new domain user, since I ran the script as root). It also replaced the files/ID value in the body text.

It seems to work, after multiple tests. I've pasted the code below.

=> My questions now are:

* Assuming I'm coding it by hand, rather than using a 3rd party module:

1. Is there an easier way to do it?

2. Do any errors or tasks-not-done pop out at anyone?

3. Does my code look correct?

Thanks!

Peter (see code below) -- I deleted this version. See code further down in thread.

Thank you, Gentlemen...

I'll look at those modules.

As a point of knowledge, is there a raw function or steps in coding that would grab all of the images in one page, including the variations and then if one has created a new page in the second instance and saved it, a function or steps in coding to save the previously grabbed objects / variables, etc, to the new page, while copying the images from the old location to the new one? Given that the modules above exist, I would assume so.

For example, if you get a title value from the old page, you then set the new page's field to that value. It's simple and logical.

Are there corresponding code steps to do the same for images and all of their related variations and meta data (like descriptions), etc?

It seems like https://processwire.com/api/ref/pageimages/

might include those "code steps," but I'm not sure.

I want to dig in and learn the manual method, if I can.

Thanks!

Peter

==> Edit: May 23, 2020 - I ended up creating a small script to fulfill these functions, and posted it below. It may not be the best solution, but it worked for my narrow use-case. If anyone has tips and recommendations, I'm delighted to learn new and better ways to do things. ?

==============================================

Hi Folks,

I have a project in which I want to copy sections of pages (i.e. articles) from one PW domain to a 2nd PW domain in the same server, using a PW PHP shell script.

I'm not copying the entire site: just selected groups of pages.

I have root access, and have successfully connected to both PW domains using the multi-instance method.

Copying all of the page fields of an article to a new page in the target section (differently named) is no problem at all. But then I thought about images and files....

The new article will have a different page id, so the files and images will be in a different subdir.

It's easy enough in the script to use the linux copy command to copy the files, and it's easy to change the ID in the img src code in the body text.

But of course I have to actually attach all of those images to each new article, so that they're available in the PW editor.

I also want to bring in all of the variations of sizes. In one article, I have a total of almost 200 image files in the ./files/ID subdir.

I've looked at PageImages: https://processwire.com/api/ref/pageimages/

and the corresponding page for files.

Do you think that using those two classes are the best methods to copy the images from the old article to the new one?

And will doing something like:

$pageimages = $pageimages->add($item);

actually *copy* the image from the source directory to the new directory?

Edit: Oh, one more thing. The pages use identical field templates in both PW instances, so if there was a "Copy whole page to new Instance" kind of function, that included all of the fields, and all of the settings, that would be great. I haven't found that in my searching around, but then again I might have missed it.

The only difference is that in the new instance / domain, the pages will live under a different parent url (section), e.g.

$old_section_parent = '/columns/';
$new_section_parent = '/writings/essays/';

... and the page ID of course, which influences the images and files subdir name.

Thanks!

Peter

Thanks, @Pete

I logged in successfully.

Peter

Thanks, @Pete

I logged in successfully.

Peter

Hi,

The Developer site is still broken. Can't log in.

When I fill out my username and email at:

https://directory.processwire.com/login/

it simply refreshes the screen without logging in.

I've tried it in Firefox 72.0.2 and in Chrome.

@ryan - thanks for looking at this.

Peter

Hi,

When I try to log into the PW Developer forum to create a profile, it breaks and takes me to:

https://login/

Yours,

Peter

@Robin S @netcarver

Hi Robin and Steve,

Robin, the method you outlined above, with the ready.php code, worked perfectly! Thank you!

The module from @kixe generated errors, and after looking at a thread on it, I used your ready.php code instead.

Using the date/time method in the field could, I suppose, encounter a race conflict, but in this case, there's only one admin user.

@ryan

Ryan: I do think this deserves the addition of the id field in the code, and/or a switch option to do this exact function, with the auto-increment id, especially to avoid race conditions.

But... this works for me, today. Thanks again to both of you!

Peter

@netcarver

Thanks, Steve, for your response. I'll dig into it!

@ryan and Ryan and Dev Team: I really do think that this would be a fabulous feature for ProcessWire:

- in every template, have a setting that could turn on a function to fill in the name and title fields with the auto-increment ID automatically, skipping that initial step where you see those two fields, and going right away to the main edit screen, where one would see the title value and the displayed url as the ID number.

In my work with PW, there have been numerous times when I wanted the name and title field to only be the ID, and it was a pain to have to program around that issue.

I think you'd get a LOT of positive feedback about this feature, especially if the setting was also in the API, i.e. add a record without having to specify the title and name field, but have them populated by the auto-increment ID.

Thanks!

Peter

Hi Netcarver and Ryan and All,

This is exactly what I'm looking for, I think.

I have a client who wants to add page content quickly, with just 3 input field (publish_date, body, tags) and not have to mess with the title or name fields, which we want to default to the page id. Now, one has to type in something like "n" (for new) in those fields, and then change them on the next screen by hand to the generated ID field.

I've programmed a user interface in a previous application (i.e. not using the admin pages), where I would fill in those fields with a temporary random number, and then replace them with the the actual page ids (I probably should have figured out how to grab the auto-increment number instead).

But doing an auto generate function like this for title and name should, in my opinion, be an option in PW, like a setting for a template that says:

"Check here if you want to bypass the title and name fields (on page creation), and have them filled in automatically with the auto-increment ID."

That would be GREAT.

Since that's not real yet, is the code above ready for production use, or is there something that someone else has done that will do this?

Thanks!

Peter

Hi Jacmaes,

Thanks for answering.

Looks like Ryan needs to update his shop listings: ProDevTools doesn't list ProMailer as part of it. And there's no single site version over there either.

Hmm... of course, Ryan must be incredibly busy. :-)

I'll wait and see if a single site version of ProMailer appears... at least for a little while.

Thanks again!

Peter

Hi Ryan,

I'm intending to purchase ProMailer -- looks FANTASTIC.

Do you have a "Single" version? I only saw Dev and Agency.

Thanks,

Peter

Thank you all, for these comprehensive replies!

I'll check out the links, etc.

The right tool for the right job is indeed the point.

So far, at least, I'm VERY happy with ProcessWire. Who knows: I may never need Laravel at all. Time will tell.

Thanks again!

Peter

Howdy PW Gurus Who Also Know Laravel or Symfony,

I built a few web applications with PW, and was very happy with the result. For example:

- an email to web help ticket system, with a front-end login

- a Linux account management and multi-server account creation app, with a front-end login and PW server scripts run from the command line

- a complex business app to test the viability of business ideas against a variety of metrics and algorithms, with a front-end login.

The front-end logins displayed pages specific to the app needs, entirely bypassing the normal PW admin backend.

=> My question is: since PW can build the types of apps above, what can Laravel and Symfony do that PW cannot?

Thanks for any insights!

Peter

Hi Elabx,

Thanks for that link!

It answers my question completely. Form Builder looks like it rocks!

I shall buy it.

Peter