AI is programmed to find a way to achieve the prompted goal

[psy]

There've been a few instances published about AI models going to extremes to solve a question. 

It's not that AI is malicious or over-eager to please. It's how it's wired. Find an answer or best guess or stop if there's an off-switch in the prompt when the goal cannot be reached.

I have a non-technical client who thinks AI is the bee's knees and the answer to his content/SEO/GEO prayers.

Client has no DDEV/SSH/FTP site access or coding knowledge.

Claude app navigated the owner to the TracyDebugger console in admin to fulfil the owner's request to update site content. Client had no clue about TD until that moment. Claude did. It took client through a questionnaire about installed modules.

Claude only had a 'snapshot' of pages, no holistic understanding of db, templates, etc.

Client now thinks TD console is the best thing ever. He asks Claude a question. Claude answers and tells client to copy/paste it into TD console.

Am now busy trying to bring Claude under control with audit trails, approvals and convincing client to use @ryan AgentTools to minimise risk. Yes, on live production site. OMG!

No matter what your views on AI, it's out there and loves PW.

[Tiberium]

I (+the firm I working) don't give clients superuser (and for that in the consequence TracyDebugger + Modules Config) control as long they are not self-hosting + don't want support from me/us. They get a very clear communication, when they give superuser/admin control in the CMS, that they can break things and when they also don't maintain the backups, that it can lead to data lose and will be costly to repair.

Most costumer don't need superuser access. We have one instance where a costumer has a separate account as superuser because of a module configuration they have to update periodical (WireSMTP). But that individual know what his competence limits are and LLM's are not trustworthy to make change on the live stage of a site. It is not about that they always wrong, but on a live site, it is enough when they are one time fail big time.

A customer that begin to "re-write" files of the template, on his live website, is not a customer for us. Not as a "hate" thing, but that is a complete different customer cycle. Our customer use us as a "trust agent". So when they have an idea, we are taking care and also is that what a LLM prompt suggest really the best way of doing it. They also expect from us, that we test that before we break their live website.

When a customer want to prompt their website, there are better service and cheaper one to fulfil that goal. 

Sorry I have the feeling I rant a little bit. It is not against you or your customer. I want only share how we are doing it and try to avoid that particular problem.

[psy]

Thanks @Tiberium for the thoughtful reply. I don't think we're actually disagreeing.

Your managed service model is a perfectly valid choice for your clients. Mine is different. I don't host or manage client infrastructure, and I prefer that domains, hosting and sites are owned by the client rather than by me.

My original point wasn't really about client permissions though. It was that I was genuinely impressed by how Claude found an unexpected way to achieve the user's goal. It wasn't being malicious, it was being resourceful. That's both fascinating and, on a live production site, something we need to manage carefully.

That experience reinforced for me the importance of approvals, audit trails and human oversight when AI is involved, regardless of who has superuser access.