ChatGPT: Prompt Engineering 101

them in a very logical way. Somehow, for example, uh, I like it here, for example, for Lux. First you need to get the necessary tools, without those tools you will not be able to change the tires, so what I said here now is that let's generate a more precise message that will indicate oh no, no, there is nothing, we are another step, sorry, I feel bad. What I did here was device message

engineering

of the device. Now I try to ask the model to generate the message that will lead me to that exact result. I said, okay, generate is fine, that will tell the last language model to return the completion without providing any of the steps in the flow, so I wanted to see if I can have a very short message that can lead to these results so I can avoid everything the above text and provided me with this.

I just copied and pasted. here and I saw that it generated the list inside the previous list, okay, and this is where I said, okay, one more request, write that list in Python code, put each step starting with a new line and add a comment on each step line where you explain how this particular step is the logical successor of the previous step and the predecessor of the next step. I just wanted to see if the model understands what the flow is to go from action one to the additional to the last action and if they are logically connected.

So basically, this is what it does here, it tells me, this is called the chain of thoughts, so you basically ask the model to reason why it generated that and that, yeah, okay, and when I was reading that I saw that He fumbled around for some stock. uh between them in some groups and the last thing I wanted to add is okay. I said now I need the tools. I need to add the tools here and I think my list is perfect so what I said here we need to add it. more details in the list of actions, for example, there are missing intermediate actions between the actions, such as before losing, before these actions, you first need to get the necessary tools, the same to verify the vehicle, you cannot verify it, no has the tools so Yeah you basically updated my list and you can see now we have a lot of great details we have a trunk removal tile open and this is the Thunder group you get the necessary tools and then you have the second group when you start loosen the lug nuts and then there's another group where you lift the vehicle, so I thought this is very, very cool.

One disclaimer I have to make here is that when I tried reverse

engineering

, this one no, no, this one generates a command, yes, yes, maybe not. work if I start now a new chat because right now the difference between GPT and gpt3 chat is the health context of the chat so everything we said before is remembered, yes every time I request it here again, everything that is in this conversation is part of that conversation, so it is used, for example, if I go now to a new chat and try, okay, let me see this, let me do that and if I do that, I'm not sure if I will get the same result because Now it doesn't have any contest so yes no I don't think you will get the same result yes the previous results yes exactly so that's important this is also a very point important.

Yes, basically, if I want to. Now, to receive the same result, I will have to give you at least one example. Yes, I need to provide context when we do it programmatically. You can call the play area. You have the API that you can call and simulate this program. So a suggestion would be when you do it programmatically, always think of a mechanism that infuses a context into each message, so we try to save the context in a separate variable and each time we ask. Again we provide the context, sometimes it is difficult because you have many tokens to use and what to do here, we take all the contacts, for example if the context has more than 500 tokens, we take all the context, we go to another gpt3 instance and ask . to summarize, yes, and then go back and use this summary, which is much smaller as context, so that you always try to keep it between the expected results.

I think that's it for the demo, so if you have any questions right now. we can discuss it like forever sure perfect uh do you want to stop sharing uh yeah yeah yeah let me stop sharing here perfect awesome yeah uh thanks for the demo uh I have a lot of questions I mean I've been playing with uh loading Beauty for quite a while uh and now we have uh gpt4, so I recently started playing with uh that a little bit uh too, so recently there was an article about uh some researchers are looking into uh why chargpt can perform some tasks very well, even though it hasn't been designed for perform those tasks, for example, the bar exam, Olympiads and other exams, which he does quite well, despite the topic of these exams.

They are completely different, one is science, one is law and you can do other tasks by writing codes and everything, which are completely different tasks, so they came up with two assumptions, one is, it's really good at understanding statistics because then a whole model language model is just a statistical model, yeah, or it's doing something new, that's something we've never seen before or something that happens inside a big language model when you train with billions and billions of parameters and then give it a try. lots of examples, something new happens, so what do you think? Where we are with your experiments or how you slant your words.

Yes. When we started with gpt3 there was this text DaVinci zero zero two. model, it has some limitations, so basically all our research is on top. We came to the conclusion that you can't do complex math, that's good, very simple math that you can't do complex math even though you provide context, more examples of information, so I was failing at that and, uh, It was understood that it is a large language model, so basically its main task was to predict the next symbolic word, but as you said during training, this language model gained some hidden capabilities that later became obvious, for example, you could do them on the same playing field, you can do a text classification summary, a named entity extraction, all you have to do is just one of these questions, for example, you wouldn't change anything you would do. don't touch any parameters, you would just say, okay, now I want to extract parameters from this text, now I want to classify it, you know, so basically it was able to do it seamlessly to switch between tasks and those were just some examples provided by openai, but then we found out that it can do even more so yeah it has some hidden capabilities that at the time when they were released were very raw and green and then along came GPT 3.5 with this text DaVinci zero zero three model that overnight became very able.

So basically all the power experiments we were doing in the old model improved dramatically with the new one and now we could use less tokens, less context and you would receive this. We would receive the same results as before with more tokens, so it was clear that uh. openai started extracting those capabilities, uh, those hidden capabilities from their model and so now we're sitting with the GPT store that can now read images seamlessly and reason on any image, so you can basically ask what's in an image, yes, and will describe it. So it basically shows again that yes, during training, when you see those billions of data getting data, it deviates from its main purpose, which was to generate text, and now they are able to do things that we couldn't imagine a couple ago. of years. is that we have to find the right way to ask it to do it, so what I like about GPT and gpt4 is that now, in addition to the playground, you have this system box where you give context, so basically you have a separate field where you give content and then you have the field that you requested and the contracts will always be there so it's your job now you don't have to use the same message because right now we were structuring like let's use the initial message at the beginning and then add our questions was pretty complicated now with this new tool you basically have the context, you can always adjust the context when during the course of action you see something going in a direction that you don't want to know about and you go back to context and say oh, okay, let's add this here or that here or let's change that, yeah, and it helps, so yeah.

I think it's still a long way from doing everything to the highest quality, but the progress, at least in the last two years, is fantastic. So now I saw a document that basically says that if you follow the step model, basically the first time you give a question, you get a reason and then you reason it, for example, you ask why two plus two. is four, so you will tell him that two plus two is four because you gave him the reason and then you asked him uh two plus three and now he will provide him with the result because you told him don't just give me the result, give me the process and this is one of the ways you can reason with the model, but there are many monetary articles that describe different engineering techniques that can lead you to the desired results even in the field of mathematics where GPT was free. pretty weak, yeah, regarding, that was one of my questions for later, but I'll ask it now, like if you experimented with simple math problems, so far, the charity results have been pretty bad.

In my experience, it can't solve simple math problems quite well, but can it be done correctly? improve uh, respond uh, we only tried it a couple of times in the beginning and we stopped there because that was not our scope of the project, so our scope was to extract from the natural language culture a list of actions that you can execute In an IT environment, for example, let's say you have a program that you develop and you want the user to write to you or talk to you and you transform it into something, for example, you said I need to go to Eugene's page.

Yeah, you know, in the internal Davao application, so you want to extract key elements, not find the steps and just execute them, so this is what we tried to do for the last 15 months or so and that's why we didn't do it. pay close attention to other, say, disadvantages of gpt3, we only take seriously the freeness of GPT and even other language models like we used Bert and Roberto a couple of times, you can't, you can't be 100 sure and what output, so everything they come out, you have to take it with a pinch of salt, so you always have to double check because there is this phenomenon of hallucination, so when the model doesn't know the result, it will make something up, yeah, and that is the and like One solution that we did was we always restrict the way that we say, "Okay, this is what you have to do," we provide, we try to provide you with a small list of actions, we have it and we say "like" if You can't find any reasonable ones. the actions in this list simply return a noun or return I don't know, so all our problems are restricted with this final phrase, please don't invent anything new, just stick with what we have this way we try to minimize. they, uh, the misleading information because sometimes, at least at the beginning of the project, we receive a lot of uh and I love that result that you couldn't verify on the spot, is that right?

Is it true or is it the models, you know an event 100 and I think that's a big problem if you're trying to use it in the industrial application, that's a big problem and another thing is the corpus that was used to train a chat The MP was probably limited to the year 2021 I guess so, yeah, yeah , and that's another problem because, if you try to explain any algorithm or anything that was invented after 2021, let's say last year, last year. or in the last few months, you will invent some new ideas that are completely false, for example, I said it.

He tried to

prompt

me to explain stable diffusion and basically explained to me how stable diffusion is just a generative adversarial network. It's not, it's just that the principles are completely different, so that's something, another problem, I'm not sure how to avoid it, although do you think it's possible, how to avoid it? the data comes from 20 before 2021 so I see here I see two solutions here Solutions uh one of them was that for example you can use this context field and provide this new information so you need to find it and paste it and provide it here and Say okay on the topic, now we are paying attention, there is no new development, so you provide information and the second is the possibility that you have the possibility to adjust any of the models and we experiment with the time frame. setting, so basically it prepares a CSV file with completion

prompt

s and you can customize it for your specific domain, for example, we use this list for commands, so we basically provide thousands of user inputs and how to extract from them the commands and while we are.

We were doing fine tuning and also discovered that this was one of the tools you can use to give the model more information, e.g.we want something new that you haven't been trained in so you can provide it. uh, but there was a little drawback to that: the generated model was very biased in the information you provided and we couldn't find a way around it because it was the information you provided, you should be very very careful about what you put in this Fountain file because after that the model, even the best of them, is in The DaVinci model, every other time it would try to respond with some information from the file and no, it doesn't do the job you expected it to do, so it's like a workaround, huh, but it's not final yet, it's still not optimal, so even if you use the context, the system context window again, how many pages of text can you put there.

I know it's fine if you just want to put the information about the stable division, but what if you want to insert all the new researchers from 2021 to present? So, it's still not optimal, yes, but I think we'll get there. I think they will have a solution where we could use a mechanism to, say, update the models' weights with the new information I saw. Some guy on Reddit I think after one day he became convinced that the President of the United States was there that day, it took him a while to provide this information, but after hundreds and hundreds of prompts after that model he became convinced that, by I think it was Trump there, so I was convinced that now that Vader is the new president of the United States, he said you can do it with a lot of trouble, but again you have to wait for the price and uh, how much you need, that you know, yeah , you're ready to pay the price, uh, for thousands of tokens, uh, to put just one new line of information into the model, so you adjusted the model to believe that Darth Vader is. uh the president of the United States, yeah, but it just worked like uh uh that day and the next day when he logged in and started a new chat, it was like going back to basics again so it's still complicated so I think which is tied to your session id so somehow they know, basically for a day or four they can load gbt.

Learn what information you want to provide, but then I think it has a mechanic for resetting the default values. It's very interesting. They are using? uh the API calls from open API to log in through its own portal, and writes code in Python and that kind of stuff, yeah, on the enterprise side that we use, we have a key, a project key, yeah , and we will use all these keys. So basically all our calls are grouped into the same collections and somehow they were connected too. Can you talk a little more about tuning a model, since you have experience tuning the model that I have?

I didn't try it before. I'm definitely going to try it. What does the fine-tuning process look like? Basically, you have to prepare the CSV file. It should contain only two columns, the message and the endings, so yes, the message is the user. Inputs and completion are the expected results and you have to do it to provide many roles so we experimented with 200 and then 600 to 1000 up to 2500 uh lines to see what is optimal and somehow our tests um the accuracy was quite high with 1500 problems and after that it ran very, very slightly, so we decided to stop there as optimal size because it costs to fine tune the tuner model, uh, it costs less than ordering it, but then when you want to order the fine tuning model, it is the double oh no, it's 10 times more expensive than the regular model, so it's still better.

Basically what you do is have this file, use the API to load it and open it. AI Cloud, yes, and then you receive a file ID and then you use this file ID, okay and you start the process of continuing, basically, it's like 25 30 minutes like that and then you get the successful responses that you mentioned the model and you get a model ID and all you have to do now is when you call your model you just replace the standard model name like it was a DaVinci text message, you replace it with this new model ID that you received, yeah, and now it always will be. square that culture model, okay, yeah, I'm definitely right, that was the process and I think there was an expression, but if you didn't use that model for I'm not sure, I think it was 90 days, something like that would automatically be something like that interesting, okay yeah have you tried any other big language models from other companies for example recently meta came up with llama and then we have other language models that have been coming out.

Everywhere, of course, Google came up with Pard, which kind of failed, but I mean it worked pretty well, it failed in industrial application, but it's still a good language model and then other language models come along. uh Stanford too, you guys played with those, but we tried a couple of them? What we tried at first was uh this uh gptj and GPT Neo with a user input and get a list of actions, maybe we could ask the model for a question and a search and knowledge base to get results, so maybe no, we don't need the list of steps, maybe we should provide it, say , a document or a FAQ document on how to do something, so yeah, at one point when we tried we experimented with semantic search and I used um there, that's semantics and they're also built into the gpt3 semantic search functionality, they have a special API yeah it's bar search and it does the embeds and then the search so they both worked pretty well and because the birth was free you can basically you can go with that but regarding the string neoj Neo GPT, GPT Neo but DaVinci text was always on top, so basically DaVinci was always around 75 80, the amount of precision of the text samples and the others were like less than 40, still, I think it was in the third month of the project where we decided to go with just gpt3 um. specifically because it was very easy to switch from one test to another, yes, you wouldn't have to use a different API, just for my message and it will do the task unit and it was very precise, yes, so I think DaVinci was the first. big language model to pass the bar exam and, uh, that was kind of monumental tasks to overcome, so that was a milestone in a big language model, uh world, sure, yeah, in the future technique of Home Tech, uh, it was, it was Benchmark at this time, so a year ago when we tried, we did some Benchmark research, it was better than Google and other languages ​​that we tried there, because we know that the zero shot technique is The desired.

You'll always want to hope to expand as few tokens as possible, but if you can't receive the results with a zero shot, there's always the possibility of going for a shot in the future technique and the results are always guaranteed to be better, so that's what we do. I find that interesting. Another question is: Recently, a job. The model works because it's now integrated into Bing, which is a search engine, so instead of connecting directly to the Internet, you now have a kind of knowledge base that has the data for everything we know about everything. documented information that we have in humanities since we started digitizing data and the authors who have compared the GPT chat with a uh with jpeg compression as in a JPEG you have the information of an image but it is so compressed that you have like certain information that you can extract, but you can't figure out a lot of things that have been compressed too much and he goes on to say that the GPT chair is a poor compression of human knowledge and it sounded a bit.

Poetic for me, do you agree that Chaturbate is like a poor understanding of human knowledge or maybe it's a little more than that with a little guidance? I agree, I agree because yes, expectations are quite high. With such a large amount of information, it is clear that the potential is still naturalized, so we are still quite far from the true potential, so I agree with you because the information is there, but the way you extract it and in its For the most part, you are limited by two factors: your imagination and what the API offers you as a tool text tag, so I think there is still room to take advantage of all this and you can extract more, but yes, sometimes we had in the situation of the project. when you tried all the things you don't know and you still don't get the results and you also say, oh come on, I know you can do that, but the question is always how and it's not always the user.

Partly like the prompting techniques, sometimes there are still system limitations, so I'm also looking forward to this GPT to see how the combination of text and visuals, yeah, can help you, you know, level up. Okay, yeah, okay, yeah, that's it. uh all the questions I have right now, uh, do you want to share your parting thoughts before we end the podcast? Oh, thanks for calling me, it's my first experience of this kind, um, uh, what can I say? The future looks very good. brilliant so I keep following all these articles even yesterday I was reading about Cosmos from Microsoft Cosmos so it's very impressive so I think it's a preview of what GPT 4 will be able to do because they have some connections with open.

AI, yeah, yeah, I think big language models, uh, I'm not sure if it's the future, but it's the present, it's safe, and we as humans not only have access to information at the click of a button. distance, so basically all you have to do is to do a proper search to receive all the information that you have now you have the tools to use that information and extract things like the more useful things the more you can you can transform it into knowledge you can transform it into uh on use it so yeah, I'm pretty depressed.

I'm like I said 15 months into this project and I'm still discovering every day things that you can do with that and that um, for example, even these. days we are trying to do reverse notice engineering, so what happens if we tell the model to do the things we want to do? Let's do the opposite. Let's do it. Hey, can you write to me? I don't know a warning. that will extract me these two actions and you will give it the actions and we also want to try to change the way we do things to see if that can help us extract even more knowledge from this model, that knowledge that somehow I think is still there hidden in there, interesting, interesting, so you're saying there's probably more we can do to extract than the knowledge that's kind of internal recruiting, a challenge, yeah, exactly amazing, that's what I'm going to end the podcast with. uh here uh everyone, if you liked our podcast today, please like, share and hit the notification bell on the YouTube channel and thank you Eugene for joining our podcast and I'll see you in the next episode, thank you very much. you for inventing me thank you good luck thank you