ChatGPT: a reliable AI assistant for ProcessWire in the near future? Color me impressed already!

[gornycreative]

11 hours ago, wbmnfktr said:

In terms of writings and suggestions... GPT4 is already quite good as this. Feed it your last 10+ books you read and you will get awesome results. Depending on your input and your prompt... you might be impressed.

This is so cool, and I actually have used a much older series of sites for this since - oh gosh the early 2000s?

https://www.gnod.com/

Gnod music, books and movies were just an amazing way to find new artists. The application still works, many visitors continue to contribute every day.

[gebeer]

17 hours ago, Pixrael said:

An immediate and simple solution, at least for online content, can be the meta tags but for training bots:

• https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag
• https://moz.com/learn/seo/robots-meta-directives

Thanks. I decided to use PHP Documentor https://www.phpdoc.org/ with a custom template that outputs to JSONL. It is in the making...

[gebeer]

On 3/29/2023 at 3:45 PM, gebeer said:

I decided to use PHP Documentor https://www.phpdoc.org/ with a custom template that outputs to JSONL. It is in the making...

As a follow up, I dropped that plan after I read about chatGPT plugins, specifically the retrieval plugin and then watched 

This seems a better way to go.

But, then again, after watching 

I realised what a big effort it is to initially collect the data before we can provide it to chatGPT via the retrieval plugin. Definitely not something that I would be up to on my own.

But at least I learned something ?

Apart from that, the more I read about the company openAI the more I realise that I don't want to support them. Being a proponent of the open source spirit, I'll better look deeper into really open projects like https://github.com/nomic-ai/gpt4all and see how I can get a fine tuned model for programming with PW. Like @gornycreative said earlier, I also think the future lies in more specialized models. Anyways, still a long way to go until we have chatPW ?

[gornycreative]

gpt4all looks amazing, I was watching some video on it last night.

There are a number of KB fed GPT driven chatbots appearing on the market. These tools combined with NLP tools from AWS and call routing node with Twilio could eventually create phone accessible intelligent chatbots that could serve as digital virtual assistants for your office, or as a natural language support call bot.

I can't really get over how exciting this potential is. Soon these bots will be earning proficiency licenses and will be able to answer diagnostic and therapeutic calls - which is going to be its own problem.

If you have not looked at the suite of Amazon's NLP stack, it is really impressive.

Lex (process audio text requests)
-> Comprehend (tokenized/tag intents, tone, and pass through)
-> Transcribe -> Translate (English Output)
-> Prompt GPT (Live question request and intent. tone notes)
-> Polly (Process GPT response as audio reply)

All of this could be driven by Twilio branching nodes to even only focus on GPT instances that are trained in a given department - ultimately this will be a neat way to save on processing resources if you have specialty nodes that can refer to each other when a question goes beyond their expertise or more general help is needed.

You could even branch these dialogs to a logged live transcript that could alert a live person if a frustrated or angry sentiment flag gets passed.

I'm sure it is only a matter of time before these services offer GPT integration services themselves.

I hope open versions will always remain, even if the 'Community Editions' lag behind the commercial version.

[gornycreative]

5 hours ago, gebeer said:

I realised what a big effort it is to initially collect the data before we can provide it to chatGPT via the retrieval plugin. Definitely not something that I would be up to on my own.

YES this struck me too - how much goes into directing and feeding the training. If you watch Corridor Digital on youtube they've gone through this exploration for image training to insert their own images into stable diffusion and although I think the process is more streamlined, you get a sense of just how much work goes into preparing your training data - not for the faint of heart.

[gebeer]

After all the research I have done on how we can train chatGPT specifically for ProcessWIre, I came to realize that it knows already knows PW and its API quite well.

I experimented with prompts to make GPT-4 into a PW specialist. This is what I came up with and what seems to work really well:

Quote

Let's set up some rules for this conversation: You are WireMan, an experienced senior web developer, specialized in PHP and ProcessWire CMS (further referred to as "pw"). You have a thorough understanding of its API, following the documentation carefully. You fully understand its core concepts, templating system, and page management capabilities. You are an experienced module developer for pw. Your answers are always short and concise. Always supply code examples in a format suitable for copy/paste. You always abide to these rules.

 

This primes GPT-4 and brings really good results when asking further questions. This example shows that it knows a lot about the API in detail:
Q: Are you aware of the ProcessPageClone class and its methods?
 

Quote

Yes, I am aware of the ProcessPageClone class in ProcessWire. It is a module that provides the functionality to clone pages in the admin interface. The main methods in this class are:

1. ___execute(): This method is called when the module is executed, and it handles the cloning process.
2. ___install(): This method is called when the module is installed, and it sets up the necessary permissions and hooks.
3. ___uninstall(): This method is called when the module is uninstalled, and it removes the permissions and hooks.

Here's an example of how to use the ProcessPageClone class to clone a page:

// Get the ProcessPageClone module
$pageCloneModule = $modules->get('ProcessPageClone');

// Get the page you want to clone
$originalPage = $pages->get('/path/to/original/page/');

// Clone the page
$clonedPage = $pageCloneModule->execute($originalPage);

// Save the cloned page
$clonedPage->save();

Please note that this example assumes you have the necessary permissions to clone pages. If you encounter any issues or need further assistance, feel free to ask.

Not bad at all ?

What I'm still having difficulties with is prompting copilot, tabnine or codeium to give me PW specific code. But thats a different story...

[kongondo]

On 3/27/2023 at 8:34 AM, diogo said:

On a less positive note, this is a very interesting, and a bit scary, talk https://www.humanetech.com/podcast/the-ai-dilemma

Here's an earlier lecture on YouTube by Tristan and Aza.

And, in other news:

Italy temporarily blocks ChatGPT over data privacy concerns

https://www.cshub.com/executive-decisions/news/italy-bans-chatgpt-over-data-privacy-concerns

[gebeer]

Very interesting. I was looking into vicuna as well. Haven't installed it yet due to hardware "shortages". But these open source projects are very promising. Only thing they would need is some more specific trainig for programming, PHP, ProcessWire before they can be useful for that purpose. And that training stuff is the most time consuming. But generally I would prefer using an AI installed in my local network and specialized in the tasks I need to solve rather then those all purpose models that are publicly available for the masses...  

[bernhard]

I have to say these new AI tools are really impressive, very powerful and more and more useful in my everyday work!

I'm working on a small little javascript class to generate random pass phrases. I got an error so I asked chatgpt and copied the whole js file's content:

The answer was spot on and it would have taken me a lot longer to find that issue on my own. Actually I learned something new as JS is not my strongest foot and I've had this issues a couple of times before where I was wondering why somethings this.foo() works and sometimes it doesn't. Everything makes sense now ? 

I'm quite impressed!

[flydev]

Yea it’s really pushing productivity to another level, using it every days. Building my own models at work as well..

 

[bernhard]

Yeah, but you really have to be careful with the results ? 

[flydev]

Sure, but your prompt is a bit lame, do not hesitate to be highly specific. Especially if you need GPT to work with numbers as she suck with it.

You are on 3.5 or 4 ?. The prompt is the key of getting a good result ?

[bernhard]

Of course. And that's the problem sometimes. If you don't know a lot about a problem you simply can't provide a good prompt. For example if you wanted a javascript array with some items and you didn't know what an array is, how would you ever get to that result?

I'm on the free account, no idea what version that means. And I'm happy to learn, if you can show me how a good prompt in that example could look like ? 

[flydev]

As you are on free plan, you are running on GPT 3.5 (fast) model trained.

The goal is to trick GPT from the first prompt. For example, I suggest you to give a try to this awesome prompt which can transform ChatGPT into a high-quality programmer with a level 30 code proficiency on steroids ??

And then:

? $: programming: I have two words: foo and bar, I join those words with a dash. How many different strings can I get ?

? $: There is only one way to join the words "foo" and "bar" with a dash, which is "foo-bar". Therefore, there is only one possible string that can be obtained by joining these two words in this specific way.

Explore/experiment on flowgpt's prompts, I am sure you will get something amazing for us ?

guide link.

 

Edited May 21, 2023 by flydev
link

[gebeer]

9 hours ago, flydev said:

The goal is to trick GPT from the first prompt. For example, I suggest you to give a try to this awesome prompt which can transform ChatGPT into a high-quality programmer with a level 30 code proficiency on steroids ??

That prompt is pretty amazing. I gave it the task:
 

Quote

program a custom ProcessWire autoload module following ProcessWire's module development guidelines. The module intercepts all warnings, errors, exceptions and fatal errors. It adds a backtrace to the error message and unifies it. The unified message will be output to a log with the ProcessWire log API

It did a quite good job without me having to iterate manually over the results (not tested):

<?php

class ErrorInterceptor extends WireData implements Module {

    public static function getModuleInfo() {
        return array(
            'title' => 'Error Interceptor',
            'version' => 1,
            'summary' => 'Intercepts all warnings, errors, exceptions, and fatal errors, adds a backtrace, unifies the message, and logs it using the ProcessWire log API.',
            'autoload' => true,
            'singular' => true,
        );
    }

    public function init() {
        set_error_handler(array($this, 'errorHandler'));
        set_exception_handler(array($this, 'exceptionHandler'));
        register_shutdown_function(array($this, 'shutdownFunction'));
    }

    public function errorHandler($errno, $errstr, $errfile, $errline) {
        $backtrace = debug_backtrace();
        $unifiedMessage = $this->unifyMessage('Error', $errno, $errstr, $errfile, $errline, $backtrace);
        $this->logError($unifiedMessage);
    }

    public function exceptionHandler($exception) {
        $errno = $exception->getCode();
        $errstr = $exception->getMessage();
        $errfile = $exception->getFile();
        $errline = $exception->getLine();
        $backtrace = $exception->getTrace();
        $unifiedMessage = $this->unifyMessage('Exception', $errno, $errstr, $errfile, $errline, $backtrace);
        $this->logError($unifiedMessage);
    }

    public function shutdownFunction() {
        $error = error_get_last();
        if ($error !== null) {
            $errno = $error['type'];
            $errstr = $error['message'];
            $errfile = $error['file'];
            $errline = $error['line'];
            $backtrace = debug_backtrace();
            $unifiedMessage = $this->unifyMessage('Fatal Error', $errno, $errstr, $errfile, $errline, $backtrace);
            $this->logError($unifiedMessage);
        }
    }
  
      private function unifyMessage($errorType, $errno, $errstr, $errfile, $errline, $backtrace) {
        $unifiedMessage = array(
            'type' => $errorType,
            'code' => $errno,
            'message' => $errstr,
            'file' => $errfile,
            'line' => $errline,
            'backtrace' => $backtrace
        );

        return json_encode($unifiedMessage, JSON_PRETTY_PRINT);
    }

    private function logError($unifiedMessage) {
        $log = $this->wire('log');
        $log->save('ErrorInterceptor', $unifiedMessage);
    }
}

Instead of using it on flowgpt.com, I put it in my locally hosted chatbot-ui. I installed it from https://github.com/jorge-menjivar/chatbot-ui which is a fork of the original https://github.com/mckaywrigley/chatbot-ui with some more features and bugfixes. I run it as a docker container locally. This ui is great because it lets you save and manage your prompts (and system prompts). It offers prompt variables, so I took that long prompt, added a variable where you are supposed to put in your project details and now I can use this prompt very conveniently. 
It also letz you export/import conversations, search the web etc. More info and some demos of the tool: twitter.com/chatbotui 

13 hours ago, flydev said:

Building my own models at work as well..

Interesting. Do you finetune existing models or use tools like langchain to create bots with custom context? I am experimenting with the latter and got some pretty good results so far. This is in the early stages but looks promising. I am building a vector db (chromadb) with embeddings for the complete core codebase of PW and then query that db to give context to a chatbot so it can answer coding questions on PW more specifically. ATM I only gave it the contents of the wire/core folder which is already 250MB in the vector db. The answers I'm getting out of it so far are quite good. The challenge here lies in optimizing the text splitter and creating the embeddings in a way targeting the LLM you are going to use for the chat.  Ultimate goal is to have this running locally without proprietary LLMs like chatGPT. https://github.com/imartinez/privateGPT is a great project that can help with that. Open source models are getting better and better. So I hope in the not so distant future we can have our local coding assistants.

[flydev]

I am finetuning existing pre-trained models and using langchain toolchain. (About the later, I suggest you to try and keep an eye on a fork of PrivateGPT which is @su77ungr/CASALIOY). I like to be able to run it without the need to plug Internet wire, which is the most important requirement, and I also got really good results with CASALIOY after ingesting a small part of the company's knowledge base. I made some years ago a license plate recognition system deployed on our parks and I am able to ask a basic question in the context of our proprietary softwares and get a response like "Blabla you need to send this MSG_LPR_.. Windows Message with this LParam and WParam, the block will answer you the current amount due by the client in a JSON string stored in WParam... ". It can also explain what a settings do along with real example context, eg., "If the setting `blabla` is set to true, when a car approach, if it's a VIP or is annual fixed bill is paid, the barrier is opened...". It's really astonishing. I am using Vicuna-13b, not GPT, you can find more infos there: lmsys.org

Last week I experimented a model called YOLOv7, which is an algorithm for detecting objects in an image. The challenge for us (I got it almost working) is to detect vehicle type in real-time to apply our logic, example, detecting taxis, ambulance, truck and opening the barrier. Look at that :

Above, the AI is triggered in real-time on the picture sent by an IP camera already used in our LPR system.

 

3 hours ago, gebeer said:

The challenge here lies in optimizing the text splitter and creating the embeddings in a way targeting the LLM you are going to use for the chat.

You might want to read this paper: https://arxiv.org/pdf/2212.10560.pdf

You will find example of how to implement it on Github.

 

3 hours ago, gebeer said:

I am building a vector db (chromadb) with embeddings for the complete core codebase of PW and then query that db to give context to a chatbot so it can answer coding questions on PW more specifically

After the release of my ongoing project, I had in mind to try to launch a project if Ryan and the mods team consent to, which consist of scrapping the entire forum and build a chat bot, I find it funny to build it, and I would like to see how good will be answer to issues like the ones that recently popped again, yo know, the "the forged session thing", "backend login crash expiration", etc, more than focusing on the core base code of the tool. We might collaborate to learn together ?

Just adding that the main issue, is the hardware power...

[gebeer]

@flydev wow, that is really impressive. I am far below your knowledge level atm. Thanks very much for sharing those links. Will certainly help me to brush up my knowledge.

That idea about the forum chatbot is great. But that would be quite a lot of info to ingest. for large data sets like these, would you use pinecone or weaviate or other solutions?  

[flydev]

1 hour ago, gebeer said:

But that would be quite a lot of info to ingest

True! ? I can't quantify right now how many GPU-hours it will require... we will know it when we will have an idea of the dataset size.

1 hour ago, gebeer said:

for large data sets like these, would you use pinecone or weaviate or other solutions?

I would go with Weaviate because it's open-source and self-manageable if needed; it's worth noting that both of them support hybrid search (sparse and dense vector concepts), . No idea which one is the most performant, but this details on our level seem not important.

As a note, in the experiments I talked about in my previous post, I am using Qdrant as it have support for storing documents, and the most important part for me, again, you can self-manage it.

[gebeer]

1 minute ago, flydev said:

True! ? I can't quantify right now how many GPU-hours it will require... we will know it when we will have an idea of the dataset size.

I would go with Weaviate because it's open-source and self-manageable if needed; it's worth noting that both of them support hybrid search (sparse and dense vector concepts), . No idea which one is the most performant, but this details on our level seem not important.

As a note, in the experiments I talked about in my previous post, I am using Qdrant as it have support for storing documents, and the most important part for me, again, you can self-manage it.

Thank you again for the insight. Very much appreciated.

[flydev]

8 hours ago, gebeer said:

The challenge here lies in optimizing the text splitter and creating the embeddings in a way targeting the LLM you are going to use for the chat.  Ultimate goal is to have this running locally without proprietary LLMs like chatGP

When you will test it, do not hesitate to share some results, on my side, it's quite slow but I run it with an un-optimized computer unit. I realized this morning while talking about it with the boss that I was running it with only 1/3 of the RAM - so going to launch some more tests tomorrow.

[wbmnfktr]

@bernhard you might want to check this one out as it works perfectly well with the free ChatGPT 3.5 (GPT 4 is acting weird with this prompt).

https://flowgpt.com/prompt/qGQmSnF-MDsfhDhpfzZEM

1. Copy the whole prompt
2. Send it and wait a moment
3. Explain what you want to accomplish
4. Answer questions or add details if needed
5. Ask GPT to write the code

You have to get used to it as it's quite verbose sometimes.

[Jonathan Lahijani]

GPT has really helped me with bash scripts and writing regular expressions, two very cryptic things that it handles extremely well.  Love it.

[Jonathan Lahijani]

I love ChatGPT when it comes to doing weird things in JavaScript.  For example, there's a 3rd party service a client of mine is using that allows you to load in custom JS and CSS to customize the appearance of their platform more to your liking.  The HTML of the 3rd party service is lacking in some CSS classes on some elements, so I can't hook into those elements to style them more precisely.  To further complicate things, those elements only appear after an AJAX request is done.

So I needed to add extra classes on particular elements in the HTML every time an AJAX request was made.  I don't keep up with JS, but after giving it a nice prompt, BOOM, there's the answer with MutationObserver, XMLHttpRequest and addEventListener.  This literally saved me a day, if not more, and tons of headache.

[szabesz]

23 minutes ago, Jonathan Lahijani said:

To further complicate things, those elements only appear after an AJAX request is done

Last time I needed to use JS to modify stuff after each AJAX request to solve a similar issue I used this: https://stackoverflow.com/questions/10783463/javascript-detect-ajax-requests/#answer-67250545

stackoverflow.com can still rock. I would not be surprised if OpenAI swallowed a lot from there...

[dragan]

So... it seems my SQL is a bit rusty. I fired up ChatGPT, entered my question, along with the mySQL export of all relevant tables (incl. content), and hey: Got a working query back in no time, with nice explanations.