Jump to content

thetuningspoon

Members
  • Posts

    684
  • Joined

  • Last visited

  • Days Won

    6

Posts posted by thetuningspoon

  1. @bernhard Thanks for the clarifications. I think we are on the same page now.

    Regarding the order agnostic application of migrations, I had assumed that it was not order agnostic because of something you said to me in the RockMigrations thread about defining dependencies after creating fields/templates. does this also work when you have migrations spread across multiple files?

    I guess I am also a bit confused about the difference between using the migrate() function and using the createField() and createTemplate() functions. Is it just a matter of preference or are there differences in how these are processed (i.e. do you need to put everything in a single migrate() in order for it to be order agnostic)? I didn't see anything about the difference between these functions in your documentation and it took me a while to figure out that I should use the migrate function when copying and pasting the data from the field/template back end editor.

     

    • Like 3
  2. On 11/1/2022 at 4:57 AM, bernhard said:

    You did an 100% exact description of RockMigrations. Really, that's exactly how it works.

    Sorry, I didn't see this comment before my previous post. I know that RockMigrations does not remove fields and templates unless you explicitly request it to. Does RockMigrations support the additional features I mentioned?

    • Automatic update of config files when changes are made in the admin (I know it offers the copy & paste feature, which is a great start and preferable when using multiple migration files)
    • Order-agnostic migration
    • Field/template cleanup function for those not defined in migration files
    • Option to only apply changes manually via cli or button press in admin
    • Like 1
  3. @bernhard I know you've put a lot of work into your module and I can see why my post pissed you off (to be honest, after I posted it I thought about removing it because I was worried this might happen).

    I did "test drive" RockMigrations last week. Unfortunately, my test drive started with me trying to create and migrate a ProField for a project I'm working on, and the result was that my new field's settings were wiped out each time the migration ran. Then I read a post you wrote that made it sound like you didn't plan on supporting ProFields and didn't understand why anybody would have need for them since you don't use them yourself. All of our projects use ProFields, so it's a showstopper for us if these aren't going to be supported. And I don't feel like I should really have to justify that need and I'm not interested in arguing about it.

    That's why the system that I have in mind would use the existing JSON export/import mechanism from the core, which already supports migrating ProFields, and presumably will continue to support any new 1st party field types that are built. It would just bolt onto it and make the export/import process more automated. And as I described in my last post, it would be agnostic about the order of the field and template definitions so the user wouldn't have to worry about dependencies with order of import.

    So I'm not really sure what to say. There's no reason why we can't have more than one approach to this. You're saying that you're interested in feedback, but then it feels like you're mocking our ideas. If you think our ideas are stupid or too limiting or whatever, then at least let us have fun with them on our own :-)

    EDIT: I just looked over the processwire-requests thread on Github again and I see that some other people who have gotten deeper into this already have reported issues with the core JSON export/import module that would need resolving before certain fields could work. Other than the FieldtypeOptions I haven't had many issues lately in my experience. I also saw that you expressed interest in supporting the Combo field in that thread, so I apologize if I've misinterpreted. If RockMigrations can support my fields and desired workflow with just a bit of additional development (from either you, me, or someone else), then I would certainly prefer that to having to write a new module from scratch! 

    Also, I have to say that your car analogy had me in stitches after reading it fully from start to finish. :D

     

    On 11/1/2022 at 7:37 AM, MoritzLost said:

    @thetuningspoon I completely disagree, because that is farther removed from the goal of having a declarative dec config and closer to the territory of migrations. You don't want to give the system a list of instructions on how to build the correct state, you want to have a declarative list of configuration values that describes the correct state. Getting there happens under the hood. Similar to the difference between declarative or functional programming and imperative programming - you only describe what to do, not how to do it. The system can compare the list of fields in the config and in the database, add and remove fields as needed, and update config values. Combined with version control, this allows you to go back seamlessly to any previous state, revert changes to the declarative config etc, which is something that migrations struggle with, as mentioned.

    I agree in theory, but I would personally rather err on the side of caution. Neglecting to delete a field or template doesn't effect the operation of the system, but deleting one unintentionally (I'm picturing something like a merge conflict being resolved incorrectly) would require restoring from a backup. I see the list of deleted templates/fields as still declarative in the sense that it doesn't matter what order they're in or when they're added. It's just saying "these fields and templates should never exist".

    Anyway, I can see wanting it to work either way. So this could simply be a matter of a setting in the module.

    • Like 7
  4. On 2/3/2022 at 11:32 AM, MoritzLost said:

    @horst I just meant that existing fields that already exist in the database and still exists in the config aren't wiped and recreated when applying the config (since that would wipe the data as well). The config always includes all fields (as well as entry types, settings, etc) that exist on the site. So if I remove a field in dev, its removed from the config. If I apply that config to other environments, the field is removed from those as well.

    I've been thinking about this and I think any declarative configuration system should be extremely conservative about removing any data from the database. One way of handling this would be to have a separate location in the config for listing out the names of templates and fields that you explicitly want to remove. If an install contains a field or template not in the configuration or in the removal list, it would simply ignore it.

    Another approach would be to ignore any field or template not in the config, and provide a separate "cleanup" function that could be triggered with a button in the admin or a cli command. This would go through and remove any fields or templates that are not listed in the config.

    While the config files could be updated automatically as changes are made in the admin interface, I think that any outside changes to the config files should require a manual sync and not be applied automatically. A notice in the admin could alert the superuser that the config file contains unapplied changes and then they could click a button or use a cli command to apply the changes.  

    The algorithm for adding new fields and templates should first check for the existence of each field and template by name. If one is missing, it should proceed to create the field or template and with its name and type (but no other settings). After that it can go back and apply the settings for everything, at which point dependencies between fields and templates should resolve without issue (family settings, page fields that reference templates in their settings, etc.).

    I've been tempted to try working on a module for this for a while now and was close to having a project that required it at work that would allow me to invest the time into it. That project is currently stalled but may still come about at some point at which point I could try working on this if someone else hasn't already gotten to it.

    • Like 3
  5. Github: https://github.com/thetuningspoon/AdminPreSaveValidation

     

    pre-save-validation-demo.gif.5c7cba6600f45644057fd7022b904e6b.gif

     

    This module prevents admin editors from saving changes to a page that has one or more invalid fields. It uses an ajax call to check for errors and javascript to populate the page edit form with any error messages that were returned. This way the user can correct the issues and resubmit the form without invalid data getting saved to the database or the editor losing changes.

    I've wanted a way to implement this common workflow in ProcessWire for some time now and this relatively simple ajax-based approach makes it possible without getting too much in the way of how ProcessWire normally works. For example, all of the post-save actions (Save + Exit, Save + View, etc.) still work as usual. The only downside is that a successful save will take a bit longer as it involves two sequential http requests (the initial ajax request that checks for errors and the normal page submit after no errors are returned).

    This module has also been tested successfully with repeaters and other nested inputs.

    By hooking after Inputfield::processInput, you can add your own custom validations to inputfields and this module will pick up on and display them as well.

    /*
     * Example of hook that adds an error to an inputfield. This will add an error to every input.
     */
    $this->wire()->addHookAfter("Inputfield::processInput", function(HookEvent $event) {
       $event->object->error('Invalid input!');
    });

    I hope others will find this module useful!

    • Like 13
  6. @bernhard Great video! The thing that I was confused about with RockMigrations (and the video doesn't address directly but I think I'm finally understanding anyway) is that it is about declarative template/field configurations that can be modified at any time rather than a series of instructions for updating PW. (When I think of the word "migrations", it makes me think more of files or snippets of code that apply a certain sequential set of changes for a specific feature.)

    So I my question now is, can I just have a single migrate.php file that has ALL of the fields and templates defined in it? Or maybe a separate file for fields and one for templates? What's the advantage of tying the migration with the page class like you did in the video vs. doing it that way?  (I can see how if you built a feature you wanted to use on multiple sites (like a shopping cart) that having a separate migrate function would be better for that).

    My other question is what is triggering the migration to run in your video? Does the default migrate.php file just run on every page load, or just in the page editor? I would probably only want it to run via CLI.

    Thanks!

  7. Very cool. I developed something like this for Solution Innovators, along with a front end for our customers to view their invoices and pay off their total balance. I like how you modified the page editor here.

    I used separate pages for the invoice items and payments instead of repeaters because I found repeaters difficult to work with when summing and reporting data, and also mapping them to models required some hacking (or did at the time - https://github.com/processwire/processwire-requests/issues/239). How are you handling this? (or is that beyond the feature set of this project?)

    We also built our whole time tracking software on ProcessWire :)

    • Like 2
  8. We had an incident where we tried to upgrade one of our sites to 3.0.200 and needed to roll back (we had only intended to upgrade the staging site but some configuration confusion led to the live database also being affected!) The site was previously on 3.0.165. When we tried to roll back the files, we were unable to because it resulted in numerous errors about missing module classes being triggered inside of other modules. Here are some examples:

    Uncaught Error: Class 'ProcessWire\InputfieldText' not found in /var/www/live/wire/modules/Inputfield/InputfieldEmail.module:16
    Uncaught Error: Class 'ProcessWire\InputfieldCheckbox' not found in /var/www/live/site/modules/InputfieldSlideToggle/InputfieldSlideToggle.module:3
    Uncaught TypeError: Argument 1 passed to ProcessWire\Pagefile::__construct() must be an instance of ProcessWire\Pagefiles, null given, called in /var/www/live/site/templates/model/TransmissionProcessor.php on line 598 and defined in /var/www/live/wire/core/Pagefile.php:131
    Exception: Method Languages::hasPageNames does not exist or is not callable in this context (in wire/core/Wire.php line 564)
    Fatal Error: Uncaught Error: Class 'Fieldtype' not found in site/modules/FieldtypeDecimal/FieldtypeDecimal.module:13

    This persisted even after clearing compiled files and triggering a module refresh. Can anyone think of what database changes might have been triggered by the upgrade from 3.0.165 to 3.0.200 that would result in these sorts of errors? I have occasionally "downgraded" ProcessWire before without having this kind of issue, or at least it wouldn't persist after refreshing modules and clearing compiled files.

  9. Sorry for the delay. At this point we're really looking for someone who is in the South Windsor, CT area or who could relocate for this full-time position. We currently work in the office 2 days a week and work from home the other three. But my boss (the owner of the company) says he is also interested in making connections as there is the possibility of contract work on an individual project basis or perhaps full time remote work at a later date.

    • Like 3
  10. The company I work for in central Connecticut, Solution Innovators, is looking to hire a programmer with ProcessWire experience to join our small but growing team. We've been working with PW since 2012 and have numerous marketing sites and web applications built on it. Strong PHP skills are a must, with experience building object oriented systems and organizing and documenting code in a clean, maintainable and reusable fashion (like the ProcessWire core). Skills in Vue.js, javascript, jQuery, html, and css are a plus. DM me for more information!

    • Like 5
  11. We've built an intranet for one of clients in ProcessWire before and it worked out nicely. Be sure to test all your roles and permissions carefully by creating test users for each. You might need to do some hooks or use some modules to get the level of control you need for your users.

    • Like 1
  12. Have you tried adding the namespace to the autoloader inside site/init.php instead of site/ready.php?

    wire('classLoader')->addNamespace('ProcessWire', wire('config')->paths->templates . "traits/");

    If you have usePageClasses set to true in the config.php and are using the /site/classes/ folder then I would think this wouldn't even be necessary since that should already be included in the places for the autoloader to look.

    I haven't come across the need for traits yet, although I keep forgetting that they exist in php now. When I think of composition I usually think of moving certain functionality out into a separate class that would then be instantiated somewhere inside the other class. You can create classes that don't extend Page (perhaps that that extend Wire or WireData if you want easy API access) and then use those in your Page derived class.

    Your example is probably intentionally oversimplified, but sometimes I find that sometimes a bit of duplicated functionality is really just incidental and doesn't actually justify creating a new layer of inheritance or a separate class at all, especially as you might later discover that the two functions actually serve slightly different purposes and at that point are tied together and difficult to pull apart.

    You could also use a PW hook to add this function to both classes, although it wouldn't be as elegant.

    • Like 2
  13. Thanks @bernhard, that's helpful. I'm still not sure how the typical workflow would look, though. Would you create a new file for each set of changes that you make to your site? Or do you just keep updating that one array that defines all of the templates and fields for the whole site (like a declarative configuration file)?

     

  14. I actually find that ProcessWire plays pretty well with Git, certainly in comparison to WordPress. The main thing is to avoid installing modules via the admin UI (just download the module and put it in your repo). And of course exclude /assets/ from the repo (PW conveniently keeps all of this together).

    It would be nice to be able to exclude the wire folder and have Composer handle that instead, but I don't find it that big of a deal to keep the wire folder in Git. In the very rare case that a direct core mod is necessary for a project I'm working on, it's good to be able to make that change and commit it to the repo.

    The ability to maintain a master configuration file containing all of the meta data for the templates and fields in a project and to be able to put this into version control would be a real game changer, though. This is a real pain point we're running into on a lot of our projects, especially now that we have a couple systems that have multiple deployments and multiple developers working on them.

    Since we already have JSON export for templates and most field types (the options field export/import is still not fully functional), it doesn't seem like this is too far off. Maybe PW could cache a copy of the JSON configuration file and then if it gets updated externally, show a message in the admin that there are pending field/template changes to be applied, allowing the user to apply these changes with a single click. If it makes it easier/cleaner, there could be 2 config files, one for fields and one for templates (and maybe another for modules?)  I'm curious how other systems handle this under the hood.

    Edit: Here's how Craft CMS does it: https://craftcms.com/docs/3.x/project-config.html#propagating-changes  This sounds a lot like what I'm thinking. The command line option for applying changes is also a great idea since some changes could take a while to apply on large projects.

    • Like 3
  15. 3 minutes ago, bernhard said:

    @thetuningspoon doesn't that mean that you put all your database secrets into git?

    You don't have to. For example, you could put the username and passwords into the config-env.php instead, and still have the rest of the advantages of sharing the config file. But if you manage your own git server securely then it shouldn't be a problem. I'm interested in hearing your thoughts if you think otherwise, though.

    In your example, are you not including the config file in git? I'm confused as to why you would be including both the dev and prod settings in a single config file if you aren't.

  16. What you've done with the first example looks super powerful. Nice work. The second approach is what we use. It's always a battle between giving the client more control vs. making it easier for them to manage (and making sure they don't mess up the site's aesthetic!). So we tend to give them more clearly defined components and then add options to them as they need them, or build out new components if they want something totally new. With the example of the text w/ image block, you could create a single block but add a radio button for which side the image goes on.

    • Like 3
  17. Just wanted to share the new approach we've started using for our configs.

    Instead of multiple config files, we are using a single shared /site/config.php plus a custom /site/config-env.php file that is unique for every environment the site is deployed (or developed) in. The config-env.php file simply contains one $env variable with a string that defines which environment we're working in (e.g. "production" "staging" "dev") and we include this file within the usual /site/config.php file. The config-env.php file is excluded from git (gitignore) and from our deployment scripts, so it won't be overwritten or touched after it's set up.

    Using a file to determine the environment rather than the domain or server variables means that it will work regardless of where or how it's invoked (cli vs browser). Here's the basic format:

    Example /site/config-env.php

    <?php
    
    $env = 'production'; // A string identifying the current environment

     

    Example /site/config.php

    <?php
    
    if(!defined("PROCESSWIRE")) die();
    setlocale(LC_ALL,'en_US.UTF-8');
    ini_set('session.gc_probability', 1);
    ini_set("log_errors_max_len",0);
    
    /*** SITE CONFIG *************************************************************************/
    
    /**
     * Set up all your defaults and settings that are shared by all environements below
     */
    
    $config->debug = false;
    
    $config->adminEmail = 'example@example.com';
    $config->advanced = true;
    $config->templateCompile = false;
    $config->uploadTmpDir = dirname(__FILE__) . '/assets/uploads/';
    
    $config->chmodDir = '0775';
    $config->chmodFile = '0664';
    
    $config->timezone = 'America/New_York';
    
    $config->defaultAdminTheme = 'AdminThemeUikit';
    
    $config->installed = 1527694349;
    
    
    
    /**
     * Environment-specific configs
     */
    
    if(!file_exists(__DIR__ . '/config-env.php')) {
    	echo 'No /site/config-env.php file could be located. Please create a /site/config-env.php file with a single $env variable containing a string identifying the current environment. This file should NOT be committed to Git (add it to the /.gitignore file) and must be excluded from deployment.';
    	exit;
    }
    require(__DIR__ . '/config-env.php');
    if(!isset($env)) {
    	echo 'No $env variable found in /site/config-env.php';
    	exit;
    }
    
    switch($env) {
    
    	/**
    	 * Production
    	 */
    	case 'production':
    
    		$config->dbHost = '...';
    		$config->dbName = '...';
    		$config->dbUser = '...';
    		$config->dbPass = '...';
    		$config->dbPort = '3306';
    		$config->dbEngine = 'InnoDB';
    
    		$config->httpHosts = array('...');
    
    		break;
    
    	/**
    	 * Staging
    	 */
    	case 'staging':
    
    		$config->dbHost = '...';
    		$config->dbName = '...';
    		$config->dbUser = '...';
    		$config->dbPass = '...';
    		$config->dbPort = '3306';
    		$config->dbEngine = 'InnoDB';
    
    		$config->httpHosts = array('...');
    
    		break;
    
    	/**
    	 * Developer One's Dev environment
    	 */
    	case 'dev1':
        
        	$config->debug = true;
    
    		$config->dbHost = '...';
    		$config->dbName = '...';
    		$config->dbUser = '...';
    		$config->dbPass = '...';
    		$config->dbPort = '3306';
    		$config->dbEngine = 'InnoDB';
    
    		$config->httpHosts = array('...');
    
    		break;
    
    	/**
    	 * Developer Two's Dev environment
    	 */
    	case 'dev2':
        
        	$config->debug = true;
    
    		$config->dbHost = '...';
    		$config->dbName = '...';
    		$config->dbUser = '...';
    		$config->dbPass = '...';
    		$config->dbPort = '3306';
    		$config->dbEngine = 'InnoDB';
    
    		$config->httpHosts = array('...');
    
    		break;
    
    	default:
    		echo "No configuration found in /site/config.php for the environment string \"$env\"";
    		exit;
    }

    If you wanted, you could make the $env variable $config->env instead, which would allow you to access it from anywhere else on the site using wire('config')->env

    • Like 1
  18. On 5/2/2019 at 6:26 AM, dotnetic said:

    Hi @bernhard

    Here is my solution which uses the server name instead of a directory structure. So you are safe if you are changing paths or the OS where paths are different.

    $base_url = $_SERVER['SERVER_NAME'];
    $config->base_url = $base_url;
    switch ($base_url) {
      case "p-jentschura.localhost":
      case "p-jentschura.acemanpc":
      case "p-info.localhost":
      case "p-info.acemanpc":
        $config->dbHost = 'localhost';
        $config->dbName = 'dbname';
        $config->dbUser = 'root';
        $config->dbPass = 'dbpass';
        $config->dbPort = '3306';
        $config->debug = false;
        $config->httpHosts = array(
          'p-jentschura.localhost',
          'p-jentschura.acemanpc',
          '192.168.178.16',
          '192.168.178.23',
          'localhost.p-jentschura'
        );
        break;
      default:
        // LIVE CONFIG
        $config->dbHost = '127.0.0.1';
        $config->dbName = 'dbname';
        $config->dbUser = 'dbuser';
        $config->dbPass = 'dbpass';
        $config->dbPort = '3306';
        $config->debug = false;
        $config->httpHosts = array(
          'p-jentschura.com',
          'www.p-jentschura.com',
          'p-jentschura.de',
          'www.p-jentschura.de',
          'p-jentschura.nl',
          'www.p-jentschura.nl',
          'p-jentschura.shop'
        );
    }

     

    Watch out with this. If you're running a cli script or cron job, the SERVER_NAME will not be set and you may end up running it on your live database instead of your development!

    • Like 1
  19. On 12/3/2020 at 12:00 PM, horst said:

    What I do not understand is why the original code has worked for 15+ years and now (within a sudden) it should not do anymore. As far as I could not find any changes in the EXIF specs. ??

    This has been an intermittent issue for me with PW as long as I can remember.

    • Like 1
×
×
  • Create New...