Jump to content

MoritzLost

Members
  • Posts

    351
  • Joined

  • Last visited

  • Days Won

    18

Everything posted by MoritzLost

  1. @bernhard You're right, a simple setup like that is unlikely to break. That's also close to the setup my tutorials recommend and that I'm using for all ProcessWire projects. What I had in mind are smaller problems / incompatibilities that crop up between PW and Twig from time to time. We recently did a round of updates to 3.0.200 and ended up with some exceptions because of the way we were accessing fields that may or may not exist on some pages. The way Twig tries to access object properties was causing some unusual errors. If you don't know both ProcessWire and Twig very well, this can be really hard to debug. If you're using an actively maintained plugin that comes with some integrated features, the developer will probably notice those problems as they come up and either fix them or provide guidance on how to avoid them. What I meant above was that with a custom setup, you can end up with a bug that you don't know how to fix and have nobody to turn to for help. But maybe those points apply regardless of how you include Twig, manually or through a plugin … in any case, I wasn't thinking about your upcoming module there, I'm sure it'll be a useful time-saver!
  2. @wbmnfktr Fair enough! For now I've added a disclaimer with a link to this post to the Twig setup tutorial (just below the introduction) so people looking for reasons to use Twig can understand my reasoning. Maybe in the future I'll rework this article a bit and put it all on processwire.dev. Probably not a good idea to just copy it over as is, Google will think I'm a spambot πŸ˜… That tutorial already exists on processwire.dev though, see the links in my initial post (part one – part two). Those two tutorials go through the complete Twig setup and a basic structure for Twig templates that should fit most use-cases and integrates nicely into the existing PHP templates, as well as some pointers on extending it with custom functionality. Or is there something that you feel is missing there? πŸ™‚ Well, in the end my setup is just one way to integrate Twig into ProcessWire, and there are a couple of different options (like Bernhards upcoming module). And there are definitely some edge-cases that arise from Twig not being "officially" integrated into ProcessWire – like the translation system that can't detect translatable strings in Twig. There are workarounds for those, but you often have to invest a bit more time to get things working. I think Twig is a great benefit and wouldn't want to miss it in any of my projects. But it comes with some strings attached, and the setup might break in unexpected ways with every new ProcessWire update. So a community-provided module might be the 'safer' option if there's an active maintainer behind it who will keep everything up to date and working with new PW versions, and do all the Twig setup and config 'under the hood'.
  3. @wbmnfktr I thought about putting this post on processwire.dev, but it somehow feels like it doesn't belong there. Not sure why – maybe it doesn't provide as much a benefit as (I feel) the other tutorials there do. I've had this topic on my mind because I've been repeatedly reading that Twig doesn't do anything that you can't do in plain PHP. I obviously disagree and needed to get this out of my head and onto a page so I can link to it when it comes up πŸ˜… But if you don't need convincing to use Twig, the post doesn't provide anything the other two tutorials don't already cover. Maybe I'll integrate this post in processwire.dev at some point …
  4. I've written before about how to use Twig with ProcessWire (see my tutorials on integrating Twig into ProcessWire and extending Twig with custom functionality for ProcessWire). But those posts don't really talk about why I like to use Twig instead of plain PHP templates. For me, this comes down to one killer feature that I'm going to talk about below. But first, let's look at some of the more commonly mentioned advantages of Twig and why I don't actually think they're all that important in the context of ProcessWire: The syntax is nicer. While I personally agree with this, it's entirely subjective (and familiarity is comforting while trying something new is scary). Autoescaping provides security by default. This is true to a degree, but most ProcessWire projects (at least for me) aren't the type of expansive community-driven sites with lots of user-generated content where this would be most relevant. Most of my ProcessWire projects so far have featured a few trusted editors managing content, where you don't really need autoescaping for every template to make sure nobody slips in some malicious code. Twig forces separation of concerns between logic and presentation. Again, this is true, but not relevant to most ProcessWire projects. Most of my ProcessWire projects (and, judging by the showcase, most ProcessWire projects period) are mostly classic brochure sites without a lot of interactivity or app-like behaviour. Those projects are 99% presentation with only some small snippets of logic in between, so separating the two isn't really an issue. With that out of the way, let's talk about the killer feature that makes Twig essential to my work: Inheritance and block-based overwrites. To explain why this is important, I'll start out with a basic template for a header component in PHP and see how it can handle additional content being added to it. Then I'll write the same component in Twig for comparison. If you need a general guide on template inheritance in Twig, read this first: https://twig.symfony.com/doc/3.x/tags/extends.html The reusable header template Here's our basic reusable header template written in PHP: <header class="header"> <h1 class="header__headline"><?= $page->title ?></h1> <?php if ($page->subline): ?> <p class="header__subline"><?= $page->subline ?></p> <?php endif; ?> <?php if ($page->image) echo wireRenderFile('inc/responsive-image.php', ['image' => $page->image]) ?> </header> Sidenote, I'll use wireRenderFile to keep the examples brief, you could also write the image tag inline here. The header component may be included in a page template like this: <?= wireRenderFile('inc/header.php') ?> You have two options for where to do this. Option one is to include this template in every template that needs it (templates/home.php, templates/project.php, templates/news.php). Option two is to use the appendTemplateFile setting to keep the basic page layout in a shared template file that's always included at the end of the request (_main.php). Option one allows you to pass the template different variables depending on context, but it also means you've already started with the code duplication. Option two is probably the more common approach, but with this option you can only pass it one set of variables – those variables might be overwritten by the page template, but this will also lead to some problems as you'll see shortly. Let's introduce our first change request, one particular page needs to display a video instead of an image. No problem, we can just check if the page has a video field and display it conditionally: if ($page->video) { wireRenderFile('inc/video.php', ['video' => $page->video]); } elseif ($page->image) { wireRenderFile('inc/responsive-image.php', ['image' => $page->image]); } This still works fine. But crucially, the logic for the video header is now part of the header template, not part of the template for the page with video headers. This means that every time I want to edit the header template, this little piece of conditional logic is something I have to deal with. But that's fine, multiple pages might need a video header, so having this switch in the header template is acceptable. But then another change request come in: In the page template for some kind of project page, instead of the image, we want to display a list of project data coming from a separate project_data field. Again, we can adjust the template: if ($page->project_data) { wireRenderFile('inc/project-data.php', ['data' => $page->project_data]); } elseif ($page->video) { wireRenderFile('inc/video.php', ['video' => $page->video]); } elseif ($page->image) { wireRenderFile('inc/responsive-image.php', ['image' => $page->image]); } But now some display logic that's specific to one template is part of the global header template, not part of the project.php. This trend will continue: every custom feature required for the header of any page template will inflate the header.php file, and every adjustment requires reading all of it and making sure my change doesn't break any of the other features. This is unsustainable and inherently unscalable. Another example, what if a specific page has both the video and the image fields, but I want to display the image instead of the video? Currently, this is not possible. Now I have to build in some kind of switch: $preferImage = $preferImage ?? false; if ($page->project_data) { wireRenderFile('inc/project-data.php', ['data' => $page->project_data]); } elseif ($page->video && !$preferImage) { wireRenderFile('inc/video.php', ['video' => $page->video]); } elseif ($page->image) { wireRenderFile('inc/responsive-image.php', ['image' => $page->image]); } Again, this solution doesn't scale. Did you notice the subtle bug in there? The noise to signal ratio is becoming worse with every feature. Now you're probably thinking that you would approach those change requests in a different way. Let's look at some of the possible solutions to those problems. Lots of variables You can solve this to a degree by using lots of variables to control what you're template is doing. If we're using a shared _main.php template file that includes the inc/header.php template, the project-specific template (e.g. templates/project.php) is loaded first. So those templates can set some variables that change the content and behaviour of the header component. For example, say you want to do keep the template for the project data block in your project.php so it's easy to find. Let's go back to the original header template and introduce an optional variable that can be used to replace the image with something else: <?= $headerImageContent ?? wireRenderFile('inc/responsive-image.php', ['image' => $page->image]); Now you can set the $headerImageContent variable in your project.php and it will replace the image. But what if I want both the normal image (without duplicating code) AND some custom content? No problem, add even more variables: <?= $headerImageBeforeContent ?? '' ?> <?= $headerImageContent ?? wireRenderFile('inc/responsive-image.php', ['image' => $page->image]); <?= $headerImageAfterContent ?? '' ?> Now repeat that for every part of the header template which might need to be adjusted for some of the page templates (hint: it's all of them). You end up with a template that uses tons of variables, the signal to noise ratio becomes abhorrent. Throw in the fact that those variables are all unscoped, so there's no way to tell where they are being set or overwritten, and variable names have be very specific to avoid collisions. All of this might make sense to you the day you've written it, but what about your colleague that hasn't touched this project yet? What about yourself in six months? Make templates more granular Another solution is to make the templates more granular. I've started this trend above by using wireRenderFile to put little isolated template parts into their own dedicated template – for example, to display a single responsive image or an HTML5 video player. In the same grain, you can split up the header.php into multiple smaller template to mix and match and include include those you want to in each specific context. But this has downsides as well: You end up with a fractal nightmare, a deluge of templates with increasing granularity and decreasing utility, just to be able to include those smaller templates separately from each other. Cohesion and readability is reduced, and there's no way from directory structure alone to tell which templates go together in what ways. Splitting an existing template into two smaller templates is not backwards compatible – you have to make an adjustment in every place the original template was included. Or you keep the original template but change it to just include the two new templates. I said fractal nightmare already, didn't I? Duplicate code You can, of course, just keep separate header templates for each page type. But then you're duplicating the common parts of those templates all over again, and changing those means you have to touch a lot of separate files – definitely not DRY. Most real-life solutions will include a mix of the three approaches. I tried to be fair and write the templates in the leanest and cleanest way possible, but things still got out of hand quickly. Now let's look at the same component written in Twig: Resuable components in Twig Here's the basic header template but written in Twig: {# components/header.twig #} <header class="header"> <h1 class="header__headline">{{ page.title }}</h1> {% if page.subline %} <p class="header__subline">{{ page.subline }}</p> {% endif %} {% block header_image %} {% if page.image %} {{ include('components/responsive-image', { image: page.image, }) }} {% endif %} {% endblock %} </header> One important difference is the block tag defining the header_image block. So far we don't need that, but it will become important in a second. For the page templates, it's common to have a base template that all other templates inherit from: {# html.twig #} <!doctype html> <html lang="en" dir="ltr"> <head> <title>{%- block title -%}{%- endblock -%}</title> {% block seo %} {{ include('components/seo', with_context = false) }} {% endblock %} </head> <body> {% block header %} {{ include('components/header') }} {% endblock %} {% block content %}{% endblock %} {% block footer %} {{ include('components/footer') }} {% endblock %} </body> The base template defines some blocks and includes some default components (seo, header, footer). Now the template for the project page just inherits this: {# project.twig #} {% extends 'html' %} With the PHP template, things got difficult once we wanted to overwrite part of the header template with some content specific to one page template. This is where the header_image block comes in handy: {# project.twig #} {% extends 'html' %} {% block header %} {% embed "components/header" %} {% block header_image %} {# project data template … #} {% endblock %} {% endembed %} {% endblock %} Now the project.twig extends the base html.twig template and overwrites the header block. Then it includes the components/header template and overwrites only the header_image block while keeping the rest. This approach has some major advantages over the plain PHP template: All the code for the project template is in one place – to see what's special about this particular page in comparison to the base template, I just have to look at one template. I didn't have to repeat any of the header template code, so I can still change the header in a central place. The components/header template stays small and manageable, it doesn't know or care what other templates extend it and which parts get overwritten where. As a sidenote, some people may not like the embed syntax. Another approach would be to once again create a custom header template for the project template. But this time, we don't need to repeat any code because we can use inheritance: {# components/project-header.twig #} {% extends "components/header" %} {% block header_image %} {# project data template … #} {% endblock %} I prefer the embed approach because it keeps all the related code together. But both approaches allow for full flexibility with no code duplication. Now what if you want to change other parts of the components/header.twig template in an extending template? In this case, you can always add more blocks: {# components/header.twig #} {% block header_headline %} <h1 class="header__headline">{{ page.title }}</h1> {% endblock %} Adding blocks doesn't change anything about the base template, so it's 100% backwards-compatible. You can always add more blocks without ever having to worry about breaking any existing templates or introducing bugs. Another challenge for the PHP template was to add some additional content to a part of the header template while still keeping the default content. Let's say we want to display a publication date above the headline in a news template, but keep the headline as is. No problem: {# project.twig #} {% block header_headline %} <time>{{ entry.published_date }}<time> {{ parent() }} {% endblock %} The parent() function returns the content of the block in the base template, so you can extend a block without overwriting it completely. Conclusion You can solve all the challenges I posed here in PHP. Most solutions will include a combination of the three approaches mentioned above (making templates more granular, using lots of variables and duplicating code). And a well thought-out mix of those approaches can work reasonably well. The problem is that while those solutions improve reusability and scalability, they usually require lots of boilerplate code and unscoped variables. This reduces the readability and makes the system harder to modify, while making it easier for bugs to creep in. Again, there are solutions for those problems that introduce other problems until the solutions cancel each other out in trade-offs. To me, Twig is a great alternative that requires fewer trade-offs. It allows you to achieve complete freedom and flexibility in your templates all while keeping your templates DRY and keeping code that belongs together in a single file. On top of that, Twig uses a nice, readable syntax (warning: personal opinion) and provides a lot of utility methods and other features to improve your template structure. Some notable caveats to all of this: All of the discussed problems are about scaling a project to a larger scope or team size. For small projects that will never need to scale in this way, this doesn't really matter. ProcessWire's built-in markup regions seem to tackle a lot of the same problems I mentioned in this post. Can't really speak for it as I haven't tried it yet. If this all sounds interesting to you and you want to learn more, you can check out my tutorials on integrating Twig into ProcessWire and extending Twig with custom functionality for ProcessWire.
  5. InputfieldHCaptcha 2.0.0 Version 2.0.0 of InputfieldHCaptcha is now available. Feature: Add a permission to bypass hCaptcha completely. See the documentation for details. If you're upgrading from an earlier version of the module, you may need to add the permission manually. Go to Access -> Permissions -> Add new and add a permission with the name bypass-hcaptcha. Breaking change: After updating, superuser accounts won't see the hCaptcha widget anymore and be allowed to bypass it everywhere. This is a potential breaking change, but mostly relevant to know for development and debugging purposes – make sure to test your forms in a private browser window. --------------------------- @Pete I've gone ahead and released the bypass permission feature in version 2.0.0. Let me know if anything is not working right for you!
  6. @Pete I've added the permission check in a separate branch, would you mind testing it? https://github.com/MoritzLost/InputfieldHCaptcha/tree/bypass-permission For now the permission hides the widget completely, I think this makes the most sense. Server-side verification is skipped as well – for Form Builder entries, the generated field value should indicate this. Let me know if this works for you or if you're encountering any problems! Might have to re-install the module so that the new permission gets added, or add it manually.
  7. @Pete Glad the module is working for you! Yeah, the bypass permission will be a useful addition. I agree that this permission should hide the input field completely. Not sure if I can prevent the input field wrapper markup from showing up altogether, I'll have to give that a try. One thing I'm concerned about is that superusers won't see the hCaptcha input at all, so some site admins might think the module is broken if they add the inputfield to a form and don't see it in the frontend (unless they test in a private browser window). Maybe instead of hiding the inputfield, it should display a static message? Something like The captcha is hidden because you're already a verified user. Or is that too confusing for users? Anyway, I'll try to build this in a feature branch this week for testing purposes!
  8. @bernhard Including the default translations in the code has also fallen a bit out of favour for me, for that exact reason. Since we're using snake-case message IDs (e.g. contact_form_submit_label instead of Submit), it's nice to see the actual translation directly in the code alongside the ID. But if those get changed in the table field, it gets confusing indeed. Usually the message IDs are enough to understand what the message is intended for (and if not, this can be solved by writing better message IDs). Lately I don't include the default translations in the code as it's just as easy to add them in the table field after. I still like that the empty row with the message ID is created whenever a new translation is added, this makes it easy to add a bunch of translatable strings in the code and add the translations later in bulk. Absolutely, this is why I always prefer message IDs over writing translations directly in a fixed source language. This has the nice benefit that you can differentiate between contexts using the message ID - for example, contact_form_submit_label is different from order_form_submit_label. This means you don't have to built context awareness into your custom translation system, making it much easier to implement.
  9. @bernhard Thanks! We can't use the built-in translation system because it can't detect translations in Twig templates, and because the interface is a bit too technical for our average client. So we rolled our own translation system, which is built on a single ProFields Table field located on the global settings page. Doesn't scale super well and doesn't include stuff like pluralization, context translations and parameter replacements, but does the job for simple interface translations on small to medium sites. It's basically the approach outlined in my custom functionality section in my ProcessWire + Twig tutorial. Though it went through a couple of changes since then, most notably I optimized the way the table field is loaded. The current version loads all rows for each call to the translation function, which was terrible for performance. Gotta remember to update that in the tutorial at some point ^^
  10. @Tyssen What do you mean it's not working? Are you getting escaped HTML output or something else? If you turn autoescape off (though I don't recommend that), nothing else should be required. If autoescape is on, you need to use the raw filter to prevent HTML output from being escaped. Maybe you're using ProcessWire's built-in textformatter that escapes HTML?
  11. @Tyssen Hmm, difficult to say. I've had a similar error with the Select Options field. When you use the dot to access object properties / values, Twig tries a lot of things to get the correct value. For example, it will try the name both as a property and as a method, try if there's a method for that property prefixed with get/has/is etc. Sometimes, this unintentionally calls a method that does exist, but is intended for a different purpose or expects some additional parameters. In those cases, using explicit methods instead of the magic get functions usually fixes it. I can't check right now, but isn't there something like get or getField on TableRow objects? Using one of those should work around this issue. Happens to everyone πŸ˜„ Yeah, the browser may change the User-Agent to match the selected device in responsive mode, so if the session component checks for matching User-Agents you're gonna get logged out. Just in case you get this error again and it's not caused by the template caching: Twig compiles all templates to PHP files and only recompiles those automatically if the auto_reload option for the environment is turned on. I usually couple this to $config->debug so I get automatic reloads in development but not in production.
  12. Thanks everyone for the valuable comments! I think it's great that we can return to a good discussion here after the bumpy start (which I'm certainly not innocent of as mentioned above). Hopefully nobody needs to hold grudges now. Thanks to Ryan in particular for the good summary regarding those issues. I think everything has been said regarding the interpersonal stuff now so I'll leave it at that 😌 @bernhard Given the above code example, I can add everything I want to the selector in the query parameter: https://example.com/?category=5,%20include=all This will result in the following selector: template=news, category=5, include=all. So now I can look at unpublished or hidden pages that the editors may not want me to see. This can be solved by sanitizing the query parameter, in this case with $sanitizer->int() or $sanitizer->selectorValue(). But this is something I have to do manually and remember everytime and everywhere it's used. And while it's simple for integers, it gets a bit more involved with strings. Finally, all of this relies on the implementation of $sanitizer->selectorValue() not having any vulnerabilities or implementation bugs, and no software is bug-free. You can solve all this, but reducing the surface area for issues like this is still really valuable. This isn't unique to ProcessWire, the other day I read a blog post regarding a known vulnerability with filter_var that hasn't been fixed yet … Sounded like Drupal up to the last two sentences πŸ™ƒ You're right, having native support for migrations and config schemas doesn't guarantee the system itself will be usable. But on the other hand, you can have the best of both worlds and have an easy to use system with support for all those features.
  13. @ryan Hey Ryan, this thread has definitely not gone the way I intended. I didn't mean to disrespect you or your work or to offend you, but I obviously have, for that I'm sorry. Thanks for still taking the time to address my points in detail. I agree that my tone was off and I was a bit too flippant, leaving the impression that I just want to advertise Craft and dunk on PW. I don't want to convince people to use Craft, I have nothing to gain from this. This is why I posted this in dev talk where I think most people reading this post will be long-time users of ProcessWire who won't jump ship at the first mention of another CMS. This is also why I didn't bother to write out a list of everything that's great about ProcessWire (for the record, that's a long list!), because I think most people reading this already use and like ProcessWire and don't need to be convinced that it's great. The fact that a free CMS developed mostly by one person can be compared to a commercial system developed by a team in this way and hold its own at all is actually impressive! I like ProcessWire, I've used it for years and still am. And I think I have contributed to PW a fair bit; I've pitched PW to dozens of clients and made sure the agency I work for buys the bulk licenses for most Pro modules. I've been active in the forum, I've written multiple open-source modules and an entire website dedicated to telling people about PW and how to use it well. I thought all this afforded me a bit of good faith when I'm writing something critical. Obviously I was wrong. I just wanted to talk about some things that I found lacking in ProcessWire and where I thought Craft has a good solution that PW could benefit from. Admittedly, calling this a comparison was a bad idea and gave the wrong impression. It's definitely not a full comparison in this sense, it's list of good ideas and approaches that I liked in Craft and would love to see in ProcessWire. At the very least it was supposed to. I'll try to address a couple of specifics below. For most of the talking points, I think your objections are valid and you're either right or there's arguments for both sides. I will leave out the points where I feel there's nothing more to be said for either site, or it's solely a matter of preference. Honestly, at this point I feel this discussion is way more heated than the topic warranted, probably due to me missing the correct tone or making some way too sweeping statements. If you feel this post is in bad taste or in bad faith, feel free to delete this thread altogether. You're right and I'm sorry about that wording. I didn't mean to imply that you didn't put a lot of thought and hard work into those features. What I meant to say was that some features don't work in particular use-cases which were important to me and that I don't think are super unusual or niche. In those cases, this use case was either not intended or not considered. And in those cases, the comparison to other systems (like Craft) has merit, if a similar feature is build in a different way that does enable those use-cases or is generally more flexible. This applies, for example, to the translation system. It's based on file scanning, so I can't use it at all since all my projects use Twig for templating and translations in Twig aren't picked up by it. So in this case, the translation system wasn't built to support any use-cases beyond the normal PHP templating. A different design for this feature, for example separating the file scanning from the translation API and allowing developers to call the API to add translations from their own code, would have solved this and enabled more use-cases beyond the one that the feature was built for. The relaunched processwire.com website, at the time this post was made, had major accessibility concerns. And at a very basic level at that, starting with the primary color used for backgrounds which doesn't provide enough contrast against the white text. I haven't checked everything, but as far as I can tell those haven't been fixed. This is a very basic accessibility issue, so relaunching a website like that signals that accessibility isn't important to this project. Of course it's only the website and not the CMS itself, but public perception matters to. I was referring to this quote in particular: To me, this demonstrates a lack of understanding of accessibility issues and goals. So I don't think I'm wrong in saying that good accessibility isn't, right now, a strong focus of ProcessWire. Agree, that was worded way to strongly. What I meant here is that the architecture imposes limits on the extensibility and in this case, you didn't seem willing to rethink the architecture to enable use-cases that go beyond the regular interface provided by ProcessWire. I still think this issue can be solved without compromising security or anything, by decoupling the logic for security checks from the CMS routes. I argued for that in the issue and haven't gotten a reply for over half a year now. To be, this signals that extending PW user accounts in this way is disouraged and shouldn't be done. Again, though, you're right that my statement here was way too sweeping. I'm also addressing the next couple of points here. The problem is that it's very easy to shoot yourself in the foot. In fact it's trivial to create an injection vulnerabilty: $category = $input->get('category'); $newsWithCategory = $pages->get("template=news, category=$category"); Of course I can sanitize the category input to make it safe-ish, but I have to remember to do that every time and it's easy to get wrong. This is why we use prepared statements instead of string concatenation for database queries with user input, to separate data from commands. So in this regard, the string-based selectors feel like a step back to me. That's fair. My point is that the interface is very close to implementation details, translations are grouped by the file they're located in. For clients who don't understand or care about those details - for example, that don't know anything about the template structure or indeed that there are templates at all - it's harder to use than it needs to be. What I mean is that the German translations might be stored in something like site/assets/files/10942 and to find that number I have to go to the backend and check. I can't just hit CMD+P in my editor, type in 'german' and get the appropriately named translation file immediately. I meant typing in translations in the code editor, see the point above. As far as I can tell, the language alternate fields are the only way to make fields translatable that don't come with a multi language variant. Like asset fields. If I want to allow an editor to provide a different image in the same asset field in multiple language, is this possible without language alternate fields? Last time I tried this I didn't find anything. Correct me if I'm wrong. The lack of documentation is a big thing. If I have to do some Google woodoo to find a blog post or forum thread from 2019 to figure out how something works, it's not well supported in my opinion.
  14. @Robin S Absolutely, I agree with all of this! For the record, I've used ProcessWire for years and still am, and I like it for what it is. I've just noticed that after working with Craft for a while, PW falls flat in comparison in some regards. But yes, the point regarding pricing is a absolutely valid. We're also doing some PW projects that are small enough that the price tag on Craft is actually relevant and a deciding factor. One caveat to the free vs. paid argument is that some of the suggestions here (and especially in the other thread regarding version control) could actually take less time to implement than the current solution. For example, how much time did it take to create a custom module installation process, module store API etc? This time could've been saved by just using Composer, and this would be an immediate improvement since it integrates well with VC and other tools. Though to be fair, I'm saying this with the advantage of hindsight, not sure if Composer was even around when PW got started. The same goes for the sheer breadth of features and the pace of new features being added. I would prefer seeing much fewer features and instead having that development time go towards upkeep, support, bug fixing and documentation of the core feature set. So we're still talking about the same amount of time going into PW development, just focused in a different way. I thought so too until I started working with Craft. I think it comes down to the scale of the project – for small projects it's convenient, but for larger projects the downsides of this approach become more noticable. Decoupling content from navigation is healthy in the long run, the article I linked has some great points regarding this: https://cld.agency/blog/content-utopia Though it's also a matter of personal preference! Wow, that's a lot of projects πŸ˜„ You're right, if you're doing many small projects, the licensing model for Craft isn't a good fit. Though if the project is small enough, you could also use the free lite version. The only important limitation is that you can't create additional user accounts beyond the admin account, but for projects with not a lot of content editing or changes after the site is live this might be fine. That's a really good summary and sums up my points above and in the other thread really well! Somebody please put this on the PW homepage πŸ™‚ @Ivan Gretsky I'm in the same boat, the problem is that ProcessWire does't work well with version control, so it's difficult to establish a git workflow with pull requests. On my Craft projects, I have no problem letting my student trainee work on some feature, because I can review everything they've done and their work only makes it into the project if I explicity approve it. With ProcessWire, this is much more difficult. Though this discussion fits better into the other thread I wrote yesterday ^^ @AndZyk Well from my point of view the jedi are evil it's a disadvantage to NOT have a template engine built into the CMS. A matter of personal preference to a certain degree. Though I've noticed a trend that the most valiant defenders of plain PHP templates are the the ones that have never given Twig a serious try πŸ˜— Technically you can bypass Twig and write plain PHP templates with some adjustments to the routing and templating components, though admittedly, Craft isn't built for that. Unless we're talking about custom routes and controllers, those are supported and Twig is completely optional in this case. Still – having first-class support for a templating engine means the entire system works well with that templating system. Craft provides a lot of super convenient tags, functions and filters for Twig that just work out of the box. We also use Twig for our ProcessWire projects, but because it's not built into the system, it's much less convenient. And some PW features don't work with it at all, like the translation system. @roderigo I'm not affiliated with Craft or Pixel & Tonic, I've added a disclaimer above.
  15. There's been a lot of talk recently about managing ProcessWire sites in version control systems (git in particular). Currently, this is quite difficult. The most important reason is that most of the site configuration is stored in the database, which you can't track in version control save for a database dump. But beyond that, there are many features and functions that don't work well with version control. To keep it short, I'll try to list everything I would like to see in terms of version control support in a potential ProcessWire version 4. For comparison, a similar system that does fully support version control is Craft CMS, which is also the inspiration for many of the approaches and changes listed below. Goals and paradigms Version control isn't self-serving, but enables several important development practices and workflows. This includes the following: Having a complete version history for a site, and being able to revert to a specific version or restore earlier versions of specific files and features. Being able to work asynchronously on a project with multiple people, without the need for excessive coordination or a lot of work to merge parallel development work. Having a single source of truth for the entire site (excluding content) in version control. This allows you to: Set up automated deployment pipelines (dev -> staging -> production) with no manual steps required for a deployment. Set up new instances of the site (additional staging environments, new live site if the server burns down, etc) with little to no manual effort. Being able to switch between development branches and instantly sync up to the site state in that branch (especially useful for pull request reviews). Everything on following wishlist will be in service to those goals. The wishlist Declarative config. This is the most important thing. ProcessWire needs a config schema that includes all the fields, templates and site settings of the site, alongside tools to import/export this config. This allows you to modify the site configuration in a development environment and instantly apply those changes to other environments. There's been some discussion about declarative config vs migrations recently, so I'll just quote my points on this debate: More on that in this thread. For reference, Craft provides this in the form of the project config. Split ProcessWire into a starter project and a Composer dependency. ProcessWire is installed the old-fashioned way, by downloading a zip file and unzipping it in the web root. This leaves you with a problem – either include all the core files in your version control, which muddies your version history, or set up custom hooks or scripts to repeat this step during deployments. In addition, a default ProcessWire installation will include lots of directories and files where it's not clear if they belong to the core or are intended to be modified and included in your site repository. This can be solved by splitting ProcessWire into two repositories. One is the actual ProcessWire core, which can be installed as a dependency using Composer. The other is a starter project that only contains a site skeleton and is intended as a starter point for your site's directory. Other site profiles can become alternative starter projects that you can choose from. Craft comes as a starter project (craftcms/cms) which only includes the skeleton for the most important directories and files and installs the actual CMS as a dependency (craftcms/craft). Better directory structures. The current directory structure doesn't separate core files and site files well, and for many directories it's not clear whether you want them included in version control. For example, the site/assets/files folder contains uploads for assets, so you want to exclude them from version control, right? The problem is that for some file go into that folder that you do want to track, so you have to manage that manually. Translation files, for example, go into the assets folder, but you probably want those in version control. Another example are modules – custom site-specific modules go into the same directory as contributed community modules. You want the former in version control, but not the latter. Make modules installable with Composer. Pretty self-explanatory – currently you can only install modules by, again, unzipping folders manually or through the module interface. You can't install modules automatically during deployment (at least not without some custom scripting and workarounds). Making those installable with Composer would solve that issue. This goes for Pro modules as well. Better distinction between content and configuration. To support deployments that update a site configuration to a particular state while leaving content untouched, you need a clear distinction between config and content. This is important if you are developing in your local dev environment while the live site is constantly updated with content. If the site has user-generated content, you can't just import the development database from three weeks ago and call it a day. But for many settings and features it's not clear whether they are content or config. For example, what about multi-language sites – are the available sites content or config? Languages are just pages under the hood, so it looks like they're content – but there are some good arguments to be made for tracking the available languages in version control. Another example are page reference fields. Those allow you to select default pages and limit selectable pages to a specific parent page – but now you have content (specific pages) in your configuration. Those pages might not exist on the target system, so this feature might break unexpectedly in a different environment. For comparison, Craft's project config explicitly defines everything that goes into the config – everything else is content and won't be included in version control or deployments. Provide CLI tools for deployments. Many of the operations you need for a deployment can only be performed through the interface, so it doesn't work with fully automated deployments. To address this, ProcessWire needs to provide a CLI tool to automate those operations. For example, installing modules, synching up the project config (see the first point) and clearing caches need to be automatable. Craft provides a CLI with those and many other commands. Better integration with standard development tools. ProcessWire uses a lot of custom solutions instead of standard tools and practices. For example, it provides a custom interface and installation method for modules, which could be simplified a lot by just using Composer for module installations. By the way, it is possible to provide a user interface for Composer operations, Craft does exactly that. But I don't think this level of polish is necessary. Simply using standard tools and practices that are compatible with version control by default will reduce development time for those features and at the same time be easier to work with for developers. Support for environment variables. You don't want passwords and other credentials tracked in version control, and you want to be able to distinguish between environments (dev, staging, production) to toggle some features or change some settings. The standard way to solve this are environment variables. This way, you keep database configuration, SMTP servers, passwords to external services and so on out of version control and provide custom values for those depending on the environment. Craft, for example, allows you to change all configuration values based on the environment, and disables any configuration changes in production environments by default. Webroot in a subdirectory. This is a minor point and admittedly only tangentially related, but it's a no-brainer once all the other changes are made. The project root should not be the webroot because that makes all project files publicly accessible by default. To fix this, ProcessWire uses a lot of .htaccess rules and .htaccess files in subdirectories, but that only works if you're using Apache. A lot of modern server management tools use other servers, like nginx or Caddy, so those require additional work to secure a ProcessWire installation. The fix here is to point the webroot to a subdirectory (like web/ or public/) and only put files that you want to be publicly accessible in there. ------- All those changes constitute a major shift in mindset and approach, which is why I think they would need to go into a future ProcessWire v4. To be honest, at this point I don't think it's likely to happen, since Ryan has repeatedly signaled that he doesn't see the value in those changes. While there are promising attempts to solve this from the community, true support for the goals outlined above can only come from the core. As long as that doesn't happen, ProcessWire won't be an option for any projects that require more than one or two people working on it or that need continuous development and deployments – at least for me. Not as long as there are other alternatives, like Craft CMS, where I get all those benefits out of the box.
  16. Last year I started working with Craft CMS, a CMS roughly similar to ProcessWire in scope and target audience (developers). Since I like Craft a lot and found it has some great solutions for problems I had with ProcessWire, I wanted to write up a comparison between the two systems, point out similarities and differences. This is also intended to introduce some ideas that ProcessWire could benefit from. Admittedly, this comparison is a bit unfair – ProcessWire is completely free and open source, while Craft CMS requires a paid license for everything but very small projects. So while Pixel & Tonic, the developers behind Craft, can fund an entire team to work on Craft and developer support, ProcessWire is mostly maintained and developed by one person. Still, I think the comparison is worthwhile, since it could help focus development on the areas where ProcessWire (in my opinion) needs improvements the most. Disclaimer: I'm not associated with either ProcessWire or Craft CMS (Pixel & Tonic), and I'm not being paid for any of this. Pricing A regular Craft license for a single site is 299 $ in the first year, then 59 $ / year for updates (this is optional). If you don't need multiple user accounts and some optional CMS features, there's a free version as well. There are also premium plugins which come with licensing fees. There's no bulk/agency pricing, so you have to include the license fees in the calculation for every project. ProcessWire is completely free, though there are the commercial Pro modules. Some of those are required for almost every site, in particular ProFields (for the Repeater Matrix), so the average site probably won't be completely free. Those modules are available as agency licenses, so for agencies with lots of ProcessWire projects they are a pretty good deal. Content modeling Both systems are quite similar in this regard. You can create fields (chosing from a variety of field types) and create content types with any number of fields. ProcessWire calls content types templates, Craft sections – sections can further be divided into different entry types with different fieldsets. ProcessWire puts everything into a single page tree / hierarchy. This is handy in some situations, but all in all detrimental. First, you have to fit everything into a page hierarchy, even pages and content that don't have a natural place in the tree. This makes it harder for clients to find what they're looking for. Also, you're heavily incentivized to generate your site navigation based on the tree hierarchy, which is bad in the long run. Craft instead has top-level index views (in the backend) for every content type – entries, categories, users, etc. You can organize entries in a section in hierarchies or streams, depending on what makes sense for that content type. Craft also has a media library where you can upload assets independent of pages. ProcessWire doesn't allow you to do this. I'm actually fine with that, though many clients care about this for some reason. Interface Both ProcessWire and Craft have mostly nice, usable interfaces. The editing interface of Craft feels a bit cleaner and more modern due to being more restricted. ProcessWire opts to have lots of display options (and even replacable admin themes), which is handy in some situations. Craft is more streamlined, which makes it feel more coherent. Both approaches have their merit. Craft is actively working on improving accessibility and has a stated goal to be accessible. Meanwhile, ProcessWire at the moment doesn't make any claims regarding accessibility, and ryan seems to be actively opposed to the idea. Templates & Theming Both ProcessWire and Craft are developer-centric in that you have full control over your HTML output. Both systems have starter projects in lieu of themes, allowing you to start with a project template and customize it as you go, or start from a completely clean slate. A major advantage of Craft is the Twig integration which it provides by default. This is a contentious topic and I don't want this to turn into a debate of plain PHP templates vs. template systems. But Twig just has some advantages that plain PHP doesn't: Block-based inheritance is impossible in plain PHP, and without that you can't have fully extendable base templates without those base templates having knowledge of ALL extending templates or being so granular that they stop being useful. Twig forces you to separate logic and presentation, putting your business logic in services and/or controllers. All the other benefits that are always brought up in discussions about Twig, like security by default, simpler templating syntax etc. You can include Twig in ProcessWire, though it's a bit of extra work. And Craft's Twig integration goes much deeper, providing many useful functions to templates. For example, there's a cache tag that 'just works' (99% of the time) and allows you to cache parts of your template output. With ProcessWire, you have to use the $cache API manually and take care of cache keys, cache invalidation etc. Developer API & data access Both ProcessWire and Craft provide a strong API to query for content / data. They are actually quite similar in that you can query by content type or properties, search for field values, apply custom ordering etc, everything you would expect. One downside of ProcessWire's API is that it's string-based (there's an array alternative, but that isn't super reliable and not well documented). This means that you have to take care of input sanitization to prevent query injection. Craft's element queries use method chaining instead, which means you have better separation between query and data. This way, the the field type classes are responsible for sanitizing their inputs before adding them to the query, so you mostly don't have to escape user input manually. One downside of element queries in Craft is that they're quite hard to extend. Though that's to be expected because those queries support a lot more native features (multi-site support and expiration dates, for example). Multisite support Craft provides full multi-site support while ProcessWire doesn't. There is a third-party module for ProcessWire module, though that comes with zero guarantees that it will continue to work and is much less convenient than the built-in multi-site support. Not really an issue for sites that don't require multi-site capability, but a gamechanger for sites that do. Internationalization & localization Both systems support code I18N, and both support multi-language setups. Interace translations in Craft can be included as simple PHP files, though there are modules that extend this to provide an editor interface for string translations. ProcessWire does provide an interface, but it's clunky and heavily developer-centric, requiring you to chose files from a list, so you can't really ask clients to use it. ProcessWire also uses file scanning to detect translatable strings, which means you can't use it with custom setups (in particular, it's impossible to use with Twig). Finally, you can't really bypass the interface since translations are stored in JSON files in, essentially, random locations, so adding translations directly in the code editor is impossible. Field translations in Craft work with all built-in fields and all module fields and are well integrated into the developer API. ProcessWire's field translations suffer from the fact that they seem tacked on and not well thought out. For example, regular text fields can be translated inline. But if you want to translate an asset field, you need to add the second language's field manually, which feels weird and looks bad and unintuitive in the interface. Custom development Craft is built with Yii2, which most people will agree isn't the best PHP framework there is. But it does integrate nicely into Yii2, and you can also bypass them and build specific features using Yii2. The framework provides standard components for models and validation, controllers, views, etc. With ProcessWire, everything is homegrown. This makes it easier to get started at first, but hinders you in the long run, because there just aren't as many utilities, best practices and components to use for custom development. Oftentimes, you're either completely on your own, or have to dig through old forum posts and blog articles to find details (sometimes unfinished) features. Building custom features into ProcessWire isn't really encouraged or supported as ryan confirms here. Roadmap and feature policy Craft has a limited feature set, but supports those features well. Things like multi-site support, translations and live preview are built into the system from the start, provide strong APIs and all of those features work with each other. They also provide a roadmap so you can see what new features are coming next. All in all, Craft development seems organized and focused on their target audience. ProcessWire's feature approach is a bit … chaotic, to say the least. New feature seem to be added whenever ryan thinks of something cool to add. The problem is that most of those feature are never completely thought out and are quickly abandoned. There's hundreds of features that look cool but just aren't well supported – they lack documentation except for old blog posts, they have bugs that never get fixed, they don't work well with each other and so on. A much more limited feature set that can actually be supported with documentation, support, integration with other features would be preferable by a long shot. Documentation The Craft documentation is clear, well organized and provides a good overview of the system as a whole. It's It does lack a bit in depth, so for custom development you often have to look through the source code to find what you're looking for. There's also a generated and searchable class reference. ProcessWire's docs are also helpful in general, though they lack a clear structure and hierarchy, making it hard to read it back to back or find specific pieces of content. Its also missing most of the features that get added on a weekly basis – see the point above. This means the only way to discover or find info on those features is to root through old blog posts. Semantic versioning Semantic versioning is crucial for peace of mind come update time, so developers know whether an update contains bugfixes, new feature and/or backwards compatibility breaks. Craft, up until version 3, doesn't follow semver closely. The first digit (3) is reserved for marketing, while BC breaks are delegated to the second digit (e.g. Craft 3.6 -> 3.7). Bugfixes and compatible feature additions are lumped together into the third digit (though curiously, sometimes they add a fourth digit for urgent bug fixes). But starting with Craft 4, P&T will switch to full semantic versioning, so every BC break will bump the major version number. ProcessWire uses the first digit for marketing as well (ProcessWire 3). Unfortunately, it doesn't use the second digit at all, so we're still at 3.0.X years after the first PW3 release. And the third digit is just incremented whenever there's a couple of commits that need releasing, so it doesn't have any significance beyond the ordinal scale. This makes it impossible to know if a given update will cause problems for you. Ryan has stated somewhere (can't find the link) that this is because there haven't been any BC breaks since version 3.0.0 so far, which is … not true. At some point, an update broke the ability to upload SVG files to asset fields if you don't have the FileValidatorSvgSanitizer module installed. If this isn't a BC break, what is? Terminology This is a minor point, but it bugs me. Both Craft and ProcessWire have terrible terminology. Without looking, what do you think a section is in Craft? ProcessWire meanwhile overloads the term template to refer both to content types and the PHP files for those content types, making it really hard to talk about those. Have fun if you're using Twig as well, now there's three distinct things that are all called templates. ------------ I wanted to talk about version control as well, but this post is already a mile long, so that's gonna be a topic for another day.
  17. @rash Do you use Imagick or GD to convert to WebP? For GD, the imagewebp function accepts the quality level as the third parameter. For Imagick, Imageick::setImageCompressionQuality should be the equivalent. Other options can be set with Imagick::setOption, see this post on Stack Overflow. No idea if ProcessWire's image API supports those, check the documentation or read through the source code to find out … Both of those are defined in code, so it shouldn't differ between servers. However, there are some possibilities why it might depend on the server: Different PHP installations with different default settings. Or different PHP versions with different levels of optimization / encoding efficiency. Maybe ProcessWire doesn't use GD/WebP at all, but calls the underlying C-libraries directly. In this case, the default parameters used by those libraries might be different.
  18. @rash Besides filesize, how does the visual quality of both images compare? Do the WebP images look better, worse or the same as the JPEG files? If you have a slightly larger filesize for a much better image, that might still be a good trade-off. That said, uploading pre-compressed JPGs and then converting them to WebP is not a good idea. Re-encoding an already heavily compressed image will always result in a garbled image and larger filesize, regardless of format. You want to upload high-quality JPGs (Quality setting of 90~100) and then generate compressed variants in JPG / WebP based on the source image. In fact, you'll want to generate multiple variants in different resolutions for different screen sizes. See my tutorial on using responsive images in ProcessWire for details. This one only talks about JPG, but you can use a <picture> element with two <source> elements, one for WebP and one for JPEG. Another thing to consider if you're seeing wildly different results on two different servers, maybe those just use different settings. WebP can encode lossless images, maybe the 'bad' server is doing that? Or if you're doing lossy compression, maybe you just need to set the WebP encoding to a lower quality level – WebP lossy encoding also has a quality setting between 1 and 100, same as JPG. There's also a range of other options to tweak to get optimal results, so maybe play with that a bit before you give up on WebP altogether. squoosh.app has a nice interface for WebP options – open a test image, select WebP in the compress menu and play around with the settings a bit. You'll be surprised at the tradeoffs between image quality and filesize you can achieve.
  19. @wbmnfktr Thanks! Mapping different work trees to different virtual hosts is a great idea, probably what I was missing for this to make sense to me. I'll try this out when I get the chance. @dotnetic The different dependencies can just be tracked in the package.json / composer.json files (and the respective lock files) and installed with npm ci or composer install. Both tools cache packages locally, so it usually only takes a couple of seconds ... I guess this would be useful if you need to switch branches every couple of minutes, though I can't think of a workflow that would require that much switching πŸ˜„
  20. @wbmnfktr I've never really gotten the point of worktrees … and every example I've read is super theoretical. What are you using them for day-to-day? The trouble I have is that most tools can't properly deal with a copy of the project in a subdirectory … for example, PHP files in a sub-directory won't be autoloaded unless I adjust my composer.json, at which point it's more hassle than promised. Maybe I'm just used to branches. Once you're used to switching branches it takes mere seconds, so it's hard to imagine worktrees being faster still. Or maybe worktrees have more uses if you're working with a compiled language, where switching branches and recompiling might take a lot more time …
  21. @horst Well then, allow me to raise your expectations again, because your description is not how it works πŸ™ƒ In your scenario, both developers could merge their branches with zero conflicts, and as a result the main branch would incorporate all the changes from both branches. They don't even need to know what the other one is doing, and nobody needs to constantly keep up with changes from other branches / team members. That's because git is really smart in the way it performs merges. Basically, you can view every branch as a set of changes applied to the existing files. As long as those changes don't conflict, you can merge in multiple PRs back to back without any manual conflict resolution. So most of the time, you can just lean back and everything works. The only time you get a merge conflict that needs to be resolved is if there are actual conflicts that require a decision. For example, if developer A renames some_old_field to unicorns and developer B renames the same field to rainbows, that would result in a merge conflict, because a single field can't have multiple names. So someone needs to decide between unicorns and rainbows for the field name. In other words, you don't have any overhead caused by git itself – git acts as a safety net by warning you about merge conflicts so you can fix them. In a well-engineered system with good separation of concerns, it's rare to have non-trivial merge conflicts, since it's unlikely that two people working on separate features will need to touch the exact same files. And most of the time, if you do get a merge conflict it's trivial to resolve – for example, if two PRs add a new variable to our SCSS variables in the same place. This would be a merge conflicts, but it's trivial to resolve, since you know you want both changes. If you know git well, you can resolve those in under a minute, oftentimes with a single command (by specifying the appropriate merge strategy for the situation). It's the exact opposite – the larger the development team, the more you will benefit from this streamlined workflow. Everyone can focus on different features and merge their work in with the minimum amount of effort required by either them or other developers to keep in sync with each other. Regarding all the git stuff, I recommend the Git Pro book (available for free), a great resource to understanding how git works under the hood and discover some of the lesser-known features and power tools. Reading the book front to back helped me a lot to establish our feature-branch workflow (for Craft projects) at work, utilize git to work way more effectively, solve issues with simple commands instead of the xkcd 1597 route and much more. For branching and merging in particular, check out the following chapters: 3.2 Git Branching - Basic Branching and Merging 7.8 Git Tools - Advanced Merging
  22. @Kiwi Chris Migrations have their place and I definitely wouldn't do without them. I think it's best if config and migrations complement each other. I think there needs to be a distinctions between shared (community / open source) modules and site-specific modules. For Craft, this distinction is between plugins (external plugins installed via Composer and tracked in the project config) and modules (site-specific modules providing site-specific functionality). Both can provide config to be tracked in the project configuration. But they work slightly different and keeping those things separate makes it easier to talk about them. @horst I just meant that existing fields that already exist in the database and still exists in the config aren't wiped and recreated when applying the config (since that would wipe the data as well). The config always includes all fields (as well as entry types, settings, etc) that exist on the site. So if I remove a field in dev, its removed from the config. If I apply that config to other environments, the field is removed from those as well. You don't need the content in the version control. Local test environments only create data for testing purposes. So when I check out my colleagues PR, I will have all entry types for the news blog etc, but no actual data. For quick testing I can just create a couple of blog posts manually. For larger projects, you can use a data seeder plugin, or a content migration that imports some data. We've done this for a project where an existing database was imported to a new system, there we could just run the import migration to seed development/staging environments. Admittedly, it's a tiny bit of additional work. But far easier than making sure you don't put garbage data into version control, figuring out how to merge diverging content branches, dealing with assets. And I don't want content dumps muddying up my git commits anyway. Once you start working this way, it's super relaxing not having to worry about creating 'real' content in your dev environment, being able to wipe out content on a whim, try something out etc. The 'real' content editing happens in the staging/production environment anyway. How are you merging diverging branches from multiple people once the time comes to merge them? You can't really merge database dumps (and stay sane). Also, database dumps are terrible for version control, much to noisy to be readable in diffs. With a YAML config, I can just look at the diff view of the PR in Github and tell at a glance what changed in the config, I don't think you can do that with an SQL dump unless you're Cypher from The Matrix … The main branch is always the source of truth for the entire site state, the config includes all fields, entry types, general config settings, etc. Everyone working on a feature creates a new branch that modifies the config in some way – but those branches still include the entire site state, including the feature being worked on. So once you merge that feature in to the main branch and deploy in staging/production, the system can 'interpolate' between the current state and the config. That is, compare the config with the database state, then adjust the database to match the config by creating fields that are in the config but not in the database, removing fields from the database that aren't in the config anymore, applying any settings changes from the config to the database etc. Of course, there may be merge conflicts if some branches contain conflicting changes. In this case, after merging in the first branch, you'd get a merge conflict which prevents the PR from being merged. You resolve these like all regular merge conflicts in git, which is made easier since the config is just a bunch of YAML files. For simple conflicts, you can resolve those directly in the Github UI. For more complicated conflicts (which are very rare), you would do that locally by either rebasing or merging in the main branch.
  23. Yes, writing the config should always dump everything. Much easier than keeping track of changes. Of course, under the hood the actual implementation could optimize that further, for example by only writing files that have changed to reduce disk i/o. But conceptually, the config should always include the full config for the current system state. On import, you probably can't wipe out all fields since that would remove the database tables and wipe all content. When the config is applied, the appropriate process/class should read the config and apply any differences between the config and the database state to the database. I.e. create missing fields, remove fields that aren't in the config, apply all other settings etc. At least that's how Craft does it. Conceptually, the entire config is read in and the site is set to that state. In Craft, there's a clear separation of concerns between config and data. The config is tracked in version control, data isn't. That's not a problem if you don't do any 'real' content editing in your dev environment. For our projects, we usually set up a staging environment pretty early on and do all actual content editing there. Once the project is ready to go live, we either switch the live domain over to that staging environment (so staging is promoted to production, essentially). Or we install a new instance of the project and copy over the database and assets folder so we have separate production and staging environments. For projects that are already live, you just wouldn't do any real content in the dev or staging environments. If you really need a large content update alongside a code update, you could use an export/import module or migrations. Migrations complement the declarative config and most of the time, we don't need them at all. By the way, there's a discussion to be had about where you draw the line between config and content. For example, for a multilingual site, are the available languages configuration (only editable by the dev) or content (editors can create new languages)? There are many of those grey areas and I don't think this has a single right answer. Craft uses UUIDs in addition to the name. Each field also has an ID that's environment-specific, but that's an implementation detail you never have to interact with, since you can always refer to a field by name or UUID. So you can change a field handle while the UUID stays the same. This also prevents naming conflicts, since new UUIDs are pretty much guaranteed to be unique. ---- On a broader note regarding the difference between declarative config and migrations: It's important to distinguish between the 'conceptual' view (the config represents the entire site state) and implementation details. Take git as an example. Conceptual, each commit represents a snapshot of the entire codebase in a particular version. Of course, under the hood git doesn't just store a copy of the entire codebase for each commit, but optimizes that by having pointers to a database of blobs / objects. But that's an implementation detail, while the public API is inspired by treating each commit as a snapshot of the codebase, not a collections of diffs.
  24. @kongondo @szabesz @horst A completely automated deployment enables continuous deployment as well as a number of other workflows. Being able to rollback to a previous version is part of it, but it's only one of the benefits of version control, and probably not the most important one. It's all a question of how much your workflow scales with (a) the amount of work done on a project / number of deployments in a given timeframe and (b) number of people on your team. For me, the 'breaking points' would be more than maybe one deployment a week, and more than one person working on a project. There were many different approaches mentioned in the previous threads – migrations, cloning the production database before any changes, lots of custom scripting etc. But those all break down if you're starting to work in a team and get into workflows centered around version control. The key to those workflows is that each commit is self-contained, meaning you can rebuild the entire site state from it with a single command. For comparison, here's how I work on a Craft project with my team, following a feature-branch workflow. I may work on the blog section while my colleague works on the navigation. We both have our own local development environment so we can test even major changes without interfering with each other. Once my colleague has finished, they commit their changes to a separate branch, push it to Github and open a pull request – including any template changes, translations, config changes, etc. I get a notification to review the PR. I would like to review it now, but I'm working on a different feature right now, have a couple of commits and some half-finished work in my working directory that's not even working at the moment. No problem, I just stash my current changes so I have a clean working directory, then fetch and checkout my colleague's branch from Github. Now it only takes a couple of commands to get my environment to the exact state the PR is in: composer install (in case any dependencies / plugins have changed) php craft project-config/apply (Apply the project configuration in my current working directory) npm ci (install new npm dependencies, if any) npm run build (build frontend) Most of the time, you only need one or two of these commands, and of course you can put those in a tiny script so it's truly only one step. Now I can test the changes in my development environment and add my feedback to the PR. Keep in mind that the new 'blog article' entry type my colleague created with all it's fields and settings is now available in my installation, since they are included in the config that's commited in the branch. Now imagine doing that if you have to recreate all the fields my colleague has created for this PR manually, and remove them again when I'm done. Now image doing that 10 times a day. By the way, everything I was working on is savely stored in my branch/stash, but is not interfering with the branch I'm testing now. This is the benefit of a declarative config: Everything that's not in the config gets removed. So even if I left my own work in a broken state, it won't interfere with reviewing the PR. With migrations, you'd have to include an up and down migration for every change, and remember to execute them in the right order when switching branches. Any manual steps, no matter how easy or quick they are, prevent those workflows at scale. Automatic deployments also makes your deployments reproducible. Let's say you have an additional staging environment so the client can test any incoming changes before they hit production. If you do manual deployments, you may do everything right when deploying to staging but forget a field when deploying to production. With fully automated deployments in symmetric environments you'll catch any deployment errors in staging. That's not to say you can't introduce bugs or have something break unexpectedly, but by removing manual steps you're removing a major source of errors in your deployments. I can one-up that: zero clicks. Automatic deployments triggered through webhooks as soon as a PR is merged into the main branch on Github. Deployment notifications are sent to slack, so everyone sees what's going on. A branch protection rule on Github prevents any developers from pushing directly to the main branch, and requires at least one (or more) approvals on a PR before it can be merged πŸ™‚ You're clients never ask you to undo some change you did a while ago? Not because of some bug, but because of changed requirements? In any case, if your version control only includes templates, but not the state of templates/fields that those templates excect, you won't be able to reverse anything non-trivial without a lot of work. Which means you don't get a major benefit of version control. Going from commenting out chunks of code because 'you might need them later' and just deleting them, knowing you will be able to restore them at any time, is really enjoyable. Having the same security for templates, fields, etc is great. Fun story: I once implemented a change requested by a client that I knew wasn't a good idea, just because it would take less time than arguing. Once they saw it in production, they immediately asked me to revert it. One `git revert` later, this feature was back in its previous iteration.
  25. @szabesz Ryan's post in the update thread is very insightful in this regard: Main takeaways from this: Ryan always works alone, never in a team. Ryan only works on projects with sporadic, large updates, never continuous/ongoing development. With these constraints, a manual workflow is really no problem. Personally, I still wouldn't want to go without version control and automatic deployments, but I can see that if you're not used to that kind of workflow you don't see the need for it in this case. Unless you run into one of the constraints to this manual workflow: Working on the same project with multiple people at the same time without version control is near-impossible and error-prone. Working on a project with constant updates where you need to deploy not once every 3 months, but 5 times a day – in the latter case, those 5 - 10 minutes for each deployment really add up and get annoying real quick. So I can understand Ryan's point of view that version control integration is kind of a 'luxus' feature, instead of an absolute necessity for many teams/projects. I don't agree with this view – but ultimately it's up to Ryan where he wants to take ProcessWire. And it's up to developers to figure out if ProcessWire's feature-set is sufficient for each individual team or project. I agree with @szabesz that you need this in the core. Full version compatibility will require some changes in mindset and feature-set for the core, and this can only come from the core itself. @Kiwi Chris The difficulty comes from trying to use migrations, so a stream of changes, instead of declarative config. You want a config that describes the entire system so it can be built from scratch if necessary, not just a set of changes to go from one build to another. See below for details. ----- In the other thread I posted some arguments why a declarative config is better than PHP migrations, just leaving this here since @dotnetic asked to have it cross-posted:
Γ—
Γ—
  • Create New...