Share your PW dev setups / tools

[benbyf]

Hi Everyone!

I feel I'm falling behind and becoming a "dinosaur" with how I work on the web, so in the hope to make myself feel better and hopefully learn from others Please sahre how you usually work on projects and tools you might use.

Think: IDEs, servers and hosting, command line tools, PW moduels that help with development etc...

[benbyf]

I usually use:

• IDE: visual studio code
• Server: hosting with DigitalOcean droplets administered with Serverpilot
• Staging: I edit over FTPS (I know!) on the dev server, then either copy or git to a live server.
• PW Modules: always use AIOM, ProCache, RockMigrate (looking into), ManageFiles, ProcessRedirects

[horst]

Hey @benbyf, I believe I'm not of much help. ?

Me seems to be just another dinosaurs setup. ?

IDE: Nusphere PhpEd

PW Modules always: Admin DEV Mode Colors | ALIF - Admin Links In Frontend
 | Cronjob Database Backup | Process Database Backups
 | Login Notifier | User Activity
 | ProCache

I develop on local https hosts (via Laragon on Windows);
Where possible, I use Staging on subdomains of the (clients) LIVE server, as this mostly is the exact same environment setup like on LIVE;
Sync & migration between all installations with Beyond Compare and the PW Database Backup Module, and/or the remote-files-on-demand hook from Ryan;
Only very rare I directly work on STAGE via PhpED (SSH, SFTP), and afterwards pull it down to local;

That's it. ?

 

[wbmnfktr]

Yet another functional setup without bells and whistles.

IDE/Editor
My "IDE" is a simple text-editor called neovim but due to a few plugins it has all I need. Yet I don't want to call it an IDE as it starts within half a second and runs from the terminal. My neovim has support for all kinds of LSPs, snippets, Git, code formatting, and whatever I really need. Tons of stuff.

Tools
Neovim - hyperextensible Vim-based text editor
https://neovim.io/

As it says already it's a hyperextensible Vim-based text editor. I use it in a terminal and therefore it's pretty fast. Neovim has initialised a project already while VSCode starts. The absolute benefit for me is that I just need to download my configuration file(s) to a new setup, start neovim once or twice and everything is ready to go. No hassle. But the learning curve is ... weird.

Tmux - tmux is a terminal multiplexer
https://github.com/tmux/tmux/wiki

Allows me to use just one terminal window to do 90% all the things I need to do through the day, while developing something with ProcessWire or frontend-related things.

PrePros - Your Friendly Web Development Companion
https://prepros.io/

In case I need a tool that does all the SCSS, Sass, less, JS things I prefer the easy way here. A benefit is that I can save the config file for each project right in the project, put it into Git and whenever needed I reinstall Prepros and go from there. Another hassle-free tool.

ScreamingFrog - The industry leading website crawler for Windows, macOS and Ubuntu.
https://www.screamingfrog.co.uk/seo-spider/

The easiest way to find issues on a website and while looking for errors it's stress-testing the hosting environment. In case you do SEO for a project you should give it a try. It's probably ProcessWire under the installable SEO-Tools.

Missing NPM, Composer, yarn, npx and things like that? 
I'm too old for that. Whenever needed I download a module or use one of my site profiles right away.
And for most other things Prepros does its job.

Hosting/Server
webgo - german hosting company.
https://www.webgo.de/

Hosting Company of the Year in Germany since 2017 and probably the only hosting company I recommened to clients and even friends.

But almost any other hosting company that has git and Let's encrypt support. I curate a list for myself to know where to go but webgo would be probably my choice.

PW Modules

• FormBuilder
• ProCache
• ProFields (RepeaterMatrix, AutoLinks, VerifiedUrl)
• ProMailer
• AutocompleteModuleClassName
• Duplicator
• HannaCode
• ImportPagesCSV
• Jumplinks
• MenuBuilder
• PagefieldPairs
• PageHitCounter
• PricacyWire
• ProcessChangelog
• ProcessDatabaseBackups
• ProcessWireUpgrade
• RockHitCounter (PageHitCounter Addon by @bernhard)
• Snippets
• WireMailSmtp

I guess these are most of the time self-explanatory. There are a lot more modules I have used, used to use and maybe will use some day. But all of these are in my default tool-belt when needed right now.

Workflow
I run Debian Linux on my main-machine where I do all my work and therefore have a local Apache2/MariaDB/PHP server setup running. Here starts absolute every project in its very own VirtualHost and Git repository.

I prefer to use a similar setup like @horst and do the actual development on my local machine and then deliver it to a testing/stage/qa system where clients can take a look at the progress. The difference here is that my dev/testing/stage setups all run on my hosting accounts. I like to have a bit more control at this point.

Everything gets transferred via Git. From local to testing/stage/qa and later on even the live environment. Changes will be made only on and within the local dev setup. So it's super easy to see whenever a client changes files or something was changed on the remote server.

I guess that's all.

[teppo]

Talking about dinosaurs...

(To be fair I'm mostly just a "late adopter". I don't mind trying out new stuff, but rarely jump to new tools until they've been battle tested first.)

• IDE: VSCode, with a bunch of handy extensions (PHP Intelephense, Tailwind CSS IntelliSense, SCSS IntelliSense, phpcs, EditorConfig for VS Code, Git Blame, GitLens, Remote, etc.) Emacs when it makes sense to work on the live server, or I just need to make a few quick changes for a local site — though Remote extension for VScode works really well over SSH, so been using that more lately ?
• Dev environment: VirtualBox or Docker, depending on the project.
  • Some of my own old projects, mostly the ones that I rarely need to touch, I tinker with on the live server ?
  • Staging... is not really a part of my usual process. Changes go from local dev environment to live server ?‍♂️
  • Syncing between environments: combination of (bash) shell scripts and rsync ?
• Hosting: AWS for client stuff, Contabo for personal projects.
  • Currently considering switching away from Contabo. It's super cheap, but sluggish. Could be my own making, but I would happily spend a few more bucks a month if it helps.
• Modules: that's a long list, but development oriented ones are Tracy, Wireframe, and lately ProfilerPro.
  • Other modules I use on pretty much all sites include AdminBar, MarkupMenu, SearchEngine, ProCache, ProFields (particularly Repeater Matrix), FormBuilder, SchedulePages, Redirects, Process Changelog, Process Login History, WireMailMailgun, PrivacyWire ... and a handful of others, many of which were built by @Robin S ?
• Front-end: Tailwind CSS + build process powered by Gulp. Vue for some JS work, but mostly just plain old JS/ES. With all the goodies baked into the language nowadays, jQuery is obsolete. (Literally have had one case where I wanted to use it in the past two or three years: needed to serialize form data.)

What else?

Composer for managing PHP libraries and ProcessWire modules, NPM for JS, Git for version control (surprise). Think that's just about it ?

 

[rick]

Kind of embarrassed to mention my setup as it is all old school style stuff.

• local and remote LAMP stack
• SSH + RSYNC for bidirectional comms
• No composer, etc.
• Live server is with Ethernet Servers Ltd. Great VPSs. Great support too
• And the only module installed is Tracy
• VSCodium (VSCode without M$ nose buttin' in)
• Front end is plain html/css/js.

 

[wbmnfktr]

10 minutes ago, rick said:

Kind of embarrassed to mention my setup as it is all old school style stuff.

Some would and will call this... craftsmanship.

I like it. Easy to maintain. Nothing that interferes without knowledge.

I like the touch of open source and free software due to Codium. 

[bernhard]

13 hours ago, wbmnfktr said:

Some would and will call this... craftsmanship.

Reminds me of this:

Quote

Real Programmers
Don't Use Frameworks
They write web applications through the command-line straight to the server. Hats off to them. For the rest of us, Nette is a great help to our work.

nette.org ? 

Back to topic: My most important tools:

• VSCode + Intelephense + PWSnippets
  
  
   
• TracyDebugger (!!)
• RockMigrations
• Formerly Laragon now DDEV

[Craig]

I use Sublime Text as my editor. I've tried others, but haven't found anything close to Sublime in terms of usability and performance.

Most PW projects use the following modules:

• ProFields
• FormBuilder
• Wireframe
• ProcessCacheControl
• AdminTemplateColumns

Any additional PHP dependencies are added via Composer.

I use Laravel Mix to handle front-end compilation / bundling of SCSS and JS. It's a nice "wrapper" around Webpack that simplifies this process a lot. (View one of my webpack.mix.js files that runs PurgeCSS for production builds)

All code in Git repositories hosted on GitHub, and use Sublime Merge as a Git GUI.

Hosting-wise, most PW sites are on a Krystal reseller account or a client's hosting (if you use referral code 'CRAIG' we share £20 equally). For new sites, I do set up staging/development versions as a subdomain so it can be previewed.

For deployment, I use DeployHQ (referral link / normal link) (also by Krystal) - which handles the transfer of changed files from Git over to the hosting via SSH - either manually or on Git push. It also does the front-end build step and Composer package installation as well so I don't have to do that myself.

During development, I disable the template cache with this in a ready.php hook.

// (I also set `$config->env = 'development';` elsewhere)
if ($this->config->env === 'development') {
	$this->addHookBefore('Page::render', function($event) {
		$args = $event->arguments(1);
		$args['allowCache'] = false;
		$event->setArgument(1, $args);
	});
}

I recently posted about how I do separate configs for different environments.

[Jonathan Lahijani]

• M1 Mac Mini
  My first Mac.  I love the hardware, but macOS is just not for me and it does some idiotic UX things that I can't stand, to the point where it's become a deal breaker.  May go back to Windows and keep this as a second machine or just switch to Linux (Fedora KDE most likely); will probably stop using WSL2 and set up a dedicated Linux server for development (I don't like working with virtual machines).
• LAMP stack (installed and configured everything with Brew); everything ARM-based
• VSCode with various plugins
• ProcessWire with various modules (Profields being a must) + my super module (still developing it; lot's of breaking changes until the dust finally settles)
  I too set up a $config->env variable
• Laravel Mix for frontend asset management
• Hosting: DigitalOcean with Ubuntu Server (but considering Fedora Server in the future)

[adrian]

15 hours ago, Jonathan Lahijani said:

but macOS is just not for me and it does some idiotic UX things that I can't stand, to the point where it's become a deal breaker

I had the same problem when I switched to MacOS 10 years ago. I found ways to change almost all the issues I was having - there are lots of utilities out there that can help. The others I learned to live with and now I do think that overall the experience is better. 

[szabesz]

For each operating system one needs to find the right utilities to bend it to one's needs, at least doing it is a must for those who care...

If you want to give your Mac another try, then browsing lists like these can help:

• https://github.com/jaywcjlove/awesome-mac
• https://github.com/serhii-londar/open-source-mac-os-apps

I am somewhat biased of course, as I have never worked on anything else than a Mac. The most powerful tools I use to solve almost all of my UX issues are these:

• https://www.keyboardmaestro.com/main/
• https://manytricks.com/butler/
• https://www.stclairsoft.com/DefaultFolderX/
• https://www.trankynam.com/atext/
• https://rectangleapp.com/
• https://www.thnkdev.com/QuickRes/

They have some overlapping features as well and in such a case I always use the app that seems to be able to better solve the UX issue in question.

What I would no longer like to work without is Keyboard Maestro. For example, among other things I also use Keyboard Maestro to create shortcuts for features which would otherwise be impossible as it can even be used to "simulate mouse clicks" if a GUI element is in a fixed position relative to (for example) the main application window because Keyboard Maestro "can click" on it for me, and I can just assign a keyboard shortcut to that macro.

[adrian]

@Jonathan Lahijani - I'd actually really love to see your list of top 5 or 10 deal breaker issues because I bet many were similar to mine - I wrote out a list as a bit of a venting process, but also as a way to figure out what I could fix or just learn to live with. It helped me through the process where now I don't think I will ever go back to windows.

[Jonathan Lahijani]

2 hours ago, adrian said:

@Jonathan Lahijani - I'd actually really love to see your list of top 5 or 10 deal breaker issues because I bet many were similar to mine - I wrote out a list as a bit of a venting process, but also as a way to figure out what I could fix or just learn to live with. It helped me through the process where now I don't think I will ever go back to windows.

Funny you say that because I wrote a similar list as well.  Here it is with some added commentary.  As you can see I solved some things but listed them anyway and other things I didn't list because I solved them.

• general
  • when clicking something in an unfocused window, the first click will focus the window; another click must be made to actually do something in the window
    • I didn't realize how much this annoyed me.  I thought Front and Center fixes this but it doesn't.
• finder (and Finder replacements)
  • can't move or even remove the finder icon in the dock
    • I don't want Finder to be first; to counteract, I've added a bunch of spacers so it's visually separated from my core programs (vscode, chrome, forklift, iterm)
  • there is no "cut" option in finder (must copy, then use the move/paste with command + alt + v -- requires 2 hands)
  • can't have custom categories in finder (just favorites, locations, icloud and tags)
  • can't have the main folders load from a custom location like you can in windows; must use symlink instead
    • this messes up how Finder search works
  • hitting 'enter' on a file or folder does rename instead of launching the file / entering the folder
    • (naturally, 'enter' seems like the right key to hit to ENTER a folder, not rename it)
    • this means i have to use whatever the keyboard shortcut macOS wants you to use, and I don't remember what it is, but I remember I must use 2 hands -- ridiculous!
  • the files list does not refresh automatically if a new file was written somewhere else (!)
  • I use ForkLift as a replacement to Finder, but nothing in the macOS ecosystem comes even close to XYplorer (I used that for 13 years and it's updated constantly; it's like the ProcessWire of file managers... a Swiss Army Knife)
• mouse
  • mouse wheel uses acceleration which makes sense for touchpads and mice without an actual WHEEL; it's not linear like you would expect
    • counteract with https://github.com/emreyolcu/discrete-scroll
    • the fact that macOS uses acceleration based scrolling for notch-wheel mice is ridiculous.  that doesn't make any sense.
  • macOS mouse movement physics is weird (at least compared to Windows)
    • using outdated steelseries exactmouse to counteract
    • that program is from 2010 (!); there's nothing more recent!
  • no middle-click + scroll
    • use "AutoScroll" extension in Chrome; it doesn't have the same feel like in Windows
• chrome
  • can't open a new tab inside a chrome app like you can in windows; must first focus a regular chrome window
• other software
  • Transmit (the SFTP client) doesn't support folder bookmarks (really wish it did so I don't have to use FileZilla; I don't want to use ForkLift for SFTP)

[adrian]

@Jonathan Lahijani - I love it - your comments have a very similar tone of ridiculousness that my list has :)

Firstly, I am glad you found Forklift - I use it exclusively and I actually love it for FTP because I can open files directly from a remote server into my code editor and saving the file will send it straight back to the server. Obviously the dual pane is essential, especially given the stupidness of the cut / paste situation. The other crucial thing it has for me is being able to order folders first, and then files - that is truly the dumbest thing in MacOS. 

The Enter to rename drove me crazy for a while too, as did not being able to delete a file by hitting the delete key - don't even get me started on the lack of an actual delete (vs backspace) key on laptop keyboards :), but I am mostly over that now.

These days I only use the trackpad (because damn they are nice on macs - pity the magic mouse is garbage), so the acceleration stuff doesn't bother me, but I can imagine.

Oh, and yes to the window focus stuff - definitely annoying.

Another utility you should try is BetterTouchTool - it has lots of mouse / trackpad settings, along with window features like snapping, but most importantly bringing proper window maximizing (rather than full screen) with the plus icon.

Regarding having Finder first on the dock - I almost never use the dock because I have Alfred, so app launching / switching is with the keyboard. You might also like to try Hyperswitch for a more Windows like dock.

Anyway, despite all the bad UX stuff, I generally do find it much more stable and it's amazing to have not reformatted / reinstalled the OS once in 10 years. I also love the unix basis to the OS - I know there is now WSL, but it wasn't around when I was using Windows. 

[gebeer]

Developing on Linux (currently Arch/KDE Plasma) for the last 16 years. Would never go back to proprietary alternatives. Why pay for something that should really be free for all?

Devtools:

• Editor: VSCodium with PHP Intelephense, GitLense, PHP Debug (xdebug support), Prettier (Code Formatter), Todo Tree and @bernhards PWSnippets.
• Local dev env: after having used vagrant for a long time, about 4 years ago I switched to https://laradock.io/ for local docker environment. Ensures portable environments across multiple machines
• PW modules used on almost every project: TracyDebugger, WireMailSmtp, ProFields (mainly for Repeater Matrix), TablePro
• Asset building pipeline: npm scripts / gulp / webpack. Will have a look into Laravel Mix. Might save time although I actually like to fiddle with all the configs.
• Deployment: for older and ongoing projects mostly SFTP. For new projects git with git hooks. This is so much cleaner. Not using any service but creating own git hooks on the server. git must be available on the production server. Staging servers used rarely. Mostly deploy from local to production.
• Hosting: I do not offer hosting services. This is up to the client. Personally I use https://uberspace.de/en/ which is a command line configured shared hosting provider from DE with a pay what you want pricing model

[ngrmm]

What ever lived before dinosaurs, I'm one of those ?

I use Nova as Code Editor which has a FTP-client builtin. Most of the time I'm working on the live-server.
ProCache takes care of my scss-files
Modules: PrivacyWire, Jumplinks, ProtectedMode, lot of the ProModules, …
ProfileSiteExporter for moving Sites
I also do not offer hosting

Even with a limited setup as mine, there is a lot possible with ProcessWire. What a great CMS and community.

btw: Happy new year to you all!
 

 

[benbyf]

Happy new years too!

Thank you so much for sharing your setups, this has been an excellent thread, and its nice to see some people in a similar boat to me (and those perfering as little setup as possible)... Love that quote about "Real Programmers, Don't Use Frameworks.." I feel like I subscribe to this but without the expertise to actually pull it off ?‍♂️

I might look again at docker / vagrant / Virtualbox again and see if I can find a good local workflow on my M1 Mac (for referrence). 

I've now got so many links to check out, will report back if I hit any revelations. Please continue to share your setups as maybe useful for other peeps coming fresh too PW too.

[MoritzLost]

Real programmers use butterflies

To everyone using git for automated deployments: How are you handling field/schema changes in between environments? And what about module updates/new modules, module configuration, custom module data (like Hanna Code shortcodes)? And everything else that's in the database but affects the CMS, like selecting a specific page as allowed parent in a Page Reference field?

• A database dump is of course an option, but once a site is live you can't just import the dump in production since it will overwrite the live contents.
• Are you writing migrations using one of the migration modules by hand? For every single field, field setting etc? Assuming you're able to handle all the edge cases (mentioned above and otherwise) with custom migration code, isn't that hugely time-consuming? Particularly when it comes to creating complicated field setups like Repeater Matrix fields with multiple types, where some settings are stored separately from the field settings and not all properties are documented or type-hinted …
• Something else?

Anyone got a setup that allows them to use git and deploy from dev to staging/production without manual steps? If so, how are you handling the above issues?

[bernhard]

2 hours ago, MoritzLost said:

Anyone got a setup that allows them to use git and deploy from dev to staging/production without manual steps? If so, how are you handling the above issues?

Yes, RockMigrations

2 hours ago, MoritzLost said:

Are you writing migrations using one of the migration modules by hand?

Yes

2 hours ago, MoritzLost said:

For every single field, field setting etc?

Yes

2 hours ago, MoritzLost said:

Assuming you're able to handle all the edge cases (mentioned above and otherwise) with custom migration code, isn't that hugely time-consuming?

It depends. Imagine you already have a setup using RockMigrations and your class uses code like this:

<?php
$rm->migrate([
  'fields' => [
    'field1' => [
      'type' => 'text',
      'label' => 'Field ONE',
    ],
  ],
  'templates' => [
    'my_template' => [
      'fields' => [
        'title',
        'field1',
      ],
    ],
  ],
]);

And you wanted to add 2 new fields and show those fields in a 3-column layout in your page editor.

Using RockMigrations:

• open the file that holds your migrate code in your IDE (eg /site/classes/MyTemplate.php)
• add some lines of code (it's copy&paste most of the time and really easy once you are used to doing so):

  <?php
  $rm->migrate([
    'fields' => [
      'field1' => [
        'type' => 'text',
        'label' => 'Field ONE',
        'columnWidth' => 33,
      ],
      'field2' => [
        'type' => 'text',
        'label' => 'Field TWO',
        'columnWidth' => 33,
      ],
      'field3' => [
        'type' => 'text',
        'label' => 'Field THREE',
        'columnWidth' => 33,
      ],
    ],
    'templates' => [
      'my_template' => [
        'fields' => [
          'title',
          'field1',
          'field2',
          'field3',
        ],
      ],
    ],
  ]);

• run the migration
• add code that uses the new fields
• test your results, commit and push to staging/production

Not using migrations, option 1:

• login on the live system
• add new field2
• set column width, save
• add new field3
• set column width, save
• go to setup > fields > edit > field1
• set column width, save
• add code that uses the fields on live system
• hope that everything works and none of your site visitors sees anything from what you are doing...
• if anything goes wrong: restore the backup you took before creating your fields
• what? you forgot to create a backup? ok... you took care of that and lazycron is doing backups every day!
• write an email to your client that explains that you had to restore a database backup from last night, which means that his website lost XX comments, or YY orders, or ZZ new entries ...

That means that option 1 CAN be quicker, but can quickly get a lot more time-consuming than properly migrating your fields.

Not using migrations, option 2:

• take your site offline to make sure that you don't lose any live data (you already mentioned that)
• make a db dump on the live system
• import data from the live system to your dev setup
• go to setup > fields > edit > field1
• set column width, save
• go to setup > fields > edit > field2
• set column width, save
• go to setup > fields > edit > field3
• set column width, save
• create a db dump of your local system
• upload your local db dump to your live system
• take your site online again

Option 2 is a little more secure, but imagine you are working on that project as a team... You'll quickly get a huge overhead in not using migrations! You need to tell all your colleagues what you did somewhere. Imagine one of your colleagues works on a different feature at the same time and pushes changes to live overwriting your changes... You'll be in trouble!

RockMigrations was built for all those things you mentioned. And we have not even been talking about reusing things you do... Using RockMigrations you can literally copy&paste setups from one project to another. That can save you a huge amount of time.

All of what I mentioned depends a lot on how you work, what projects you work on, what clients you have etc... The fact that you are using automated deployments already and asking those questions sounds like you should definitely give RockMigrations a try though ? 

[horst]

ATM I simply use the manual export <> import  function from PW for fields and template settings.

Every time I also have to alter field and or template settings, I write down the timestamp on a pad of paper when I start to do so and then collect a list (on the left side) of all fields I touches in the next time until deployment, (and on the right side I collect all template names).*

When I'm ready for deploying, I may or may not shut down the live system for a short amount of time, depending on the amount of changes, then collect from the DEVs export selector all needed fields, copy the export json and drop it into the targets fields import field and proceed. Next step is the same with template changes.

There is no need to shutdown a live site for long time. Mostly when I had large deployments, it was closed for one hour, but most of the time I do and have done those db migrations without any maintenance mode, because most of my sites uses ProCache. If it really would matter, I can prime the cache by crawling the site once before starting to deploy.

It is not comfortable and also not the best I can think of, but it's not nearly as uncomfortable as described by @bernhard?

Using these functions is not nearly as error prone as manually writing EVERYTHING into one schema or another for every field and template change. 
Noting everything by hand is not error tolerant for typos, I might forget to note a value, a change to a template, etc.

All this is avoided by simply copy-&-paste ready generated JSON directly from PW. No typos, nothing forgotten, nothing added.
Not perfect but for me much faster, easier and safer than writing everything down again by hand.
 

Spoiler

 

PS: * In case you are not sure if you have written everything down on paper, you can generously include additional fields or templates in the export. In this case, it is better to have more (or even all) than too few. All unchanged settings will be recognized by PW during import and will not be processed further. ? 

 

[szabesz]

Probably my method is in the dinosaur category, but it still works without spending too much time on migrating a database:

If I do not mix up production with local then it is fool proof. It takes less than 15 secs from start to finish to clone a small database including all the keystrokes I have to perform at the beginning. Sure, it is the other way round regarding migrating, but for a one man show it is not a big deal. When modifying settings in production, I sometimes have to do the cloning more than once but since the whole process is so fast, for me it is a non-issue.

"I am still waiting" for an official automated migrating solution included in the ProcessWire core :P

[horst]

5 minutes ago, szabesz said:

"I am still waiting" for an official automated migrating solution included in the ProcessWire core ?

+1 ??

[elabx]

5 hours ago, MoritzLost said:

Real programmers use butterflies

To everyone using git for automated deployments: How are you handling field/schema changes in between environments? And what about module updates/new modules, module configuration, custom module data (like Hanna Code shortcodes)? And everything else that's in the database but affects the CMS, like selecting a specific page as allowed parent in a Page Reference field?

• A database dump is of course an option, but once a site is live you can't just import the dump in production since it will overwrite the live contents.
• Are you writing migrations using one of the migration modules by hand? For every single field, field setting etc? Assuming you're able to handle all the edge cases (mentioned above and otherwise) with custom migration code, isn't that hugely time-consuming? Particularly when it comes to creating complicated field setups like Repeater Matrix fields with multiple types, where some settings are stored separately from the field settings and not all properties are documented or type-hinted …
• Something else?

Anyone got a setup that allows them to use git and deploy from dev to staging/production without manual steps? If so, how are you handling the above issues?

Hey @Moritz! I use RockMigrations to handle most of the API related stuff along the initial Migrations module, which has a CLI interface letting me run pending migrations per site and keeps the track of it. The sites all share the same features and templates, nothing can be added as a "site specific feature" that involves a "custom pw stuff" in the database, it religiously needs to be in sync. 

I use bitbucket pipelines for deployments, and it's a very simple script involving rsync and running the Migrations CLI.

I do write everything that involves a "new feature" regarding fields on a page, repeater matrix, etc. I'd say once you got it figured out, most of the time i'm copy pasting between migrations and adapting. I'd love to get in sync with some sort of automated testing (end to end with cypress?), but  right now it just involves manually testing, and deploying to a few sites to test, fix otherwise and deploy again. I guess this is more of a management issue?? Maybe I am not understanding your full perspective here.

No db dumps becase as you mention, it will mess up content. 

HannahCodes I migrate using the text version export of the hannahcode haha

Module updates I guess just upgrading the files?? Haven't got into a deeper issue where something breaks while doing these. I think, up to know, I have just installed new modules.

I keep processwire install outside of this equation, I am not sure if this was a wise decision, but it is like that lol

One thing that is super important that I haven't though very well is rollbacks. Right now I basically would just run another migration on top, chekout/cherrypick from earlier branch or sth. Though haven't had enough time on this setup to mess up like this and need a rollback or where just doing a migration on top wouldn't work. 

[MoritzLost]

Thanks everyone for the insights! Nice to see all the different approaches, and that I'm not the only one struggling to find a 'perfect' solution …

@bernhard Thanks for this. Look real close to a version-controllable schema config. Can the array syntax for fields/templates handle field overrides in a template context as well? What about settings that are environment-specific, like specific parent pages in page reference fields?

How would you approach an existing site that doesn't use migrations – add all the existing fields to one big migration file? Or include a database dump as the base state and perform all further changes in the form of migrations?

You're right, doing everything manually is a pain. But handling everything in code is a pain as well. The problem for me is that too many areas of ProcessWire are not built with version control in mind. Migrations solve field/template configuration. But then you need to keep track of installed modules, if you want to install modules with Composer you need custom config so they're installed in the right folder. Of course you can't put the entire modules folder in gitignore, since site-specific modules go into the same folder. And what about ProcessWire itself, I don't want the entire core in my version control. So again, custom download script. The list goes on. You don't want to keep track of uploaded files, so the asset folder goes into gitignore. But I do want translations in my version control, turns out those are in random directories in the assets folder … the list goes on. There are just so many small issues if you want a 'complete' version controlled ProcessWire site, which to me means (1) the site can be rebuilt completely from the repository alone and (2) there's no external code or dependencies. And if you solve all of these with scripts, those become hard to manage and even harder to enforce in a team … not sure where I'm going with this to be honest ?

I'm a bit of an all-or-nothing person, if I can't have 'clean' version control I'd rather not have version control at all. Probably unwise. To be honest, for new projects I'm currently settled on option 3: to not use ProcessWire at all, unless the project won't require any significant updates after launch. Even all the problems mentioned above can be solved one way or another, it's just not worth it compared to other systems where I get full compatibility with version control by default. I don't think this will change as long as ProcessWire doesn't support version control natively.

@horst Yeah, that's close to what I'm doing for updates as well. As long as you're keeping track of any field/template changes diligently, it's not too much work. Though it still bugs me doing the same thing twice. And of course it doesn't solve the problem of working on an update with multiple people at the same time.

@szabesz We do that for significant updates to older projects where we don't have development versions any more. Though it means the client can't edit anything while updates are being done, and it doesn't work for sites with user-generated content or user accounts where you can't just freeze the live database for a larger amount of time …

@elabx So all your sites are identical in feature set? That definitely doesn't apply to our projects ? The point regarding bitbucket pipelines is interesting. Though I'm hesitant to use automated deployments if I have too write too much custom logic for it or mix partially automated deployments with manual steps. Too error-prone and difficult to enforce in a team, in particular if you want to work on site updates with multiple people …