GPT-4 - How does it work, and how do I build apps with it? - CS50 Tech Talk

Alright, this is a CS50

tech

talk

. Thank you very much everyone for coming. About a week ago we circulated on the Google form, as you may have seen, at 10:52 a.m. m. and around 11:52 a.m. m. We had 100 RSVPs, which I think is kind of a testament to how much interest there is in this world of AI and open chat of Ai and GPT, GPT and the like and, in fact, if you're familiar with what everyone They

talk

but you haven't tried it yourself this way. is the URL where you can try this tool that you have probably heard of GPT chat.

You can sign up for a free account there and start playing what everyone else has been playing and then if you're more into the app. type of mindset you probably are if you're here with us today Open AI in particular has its own low-level APIs through which you can integrate AI into your own software but of course, as is the case with computing, there are many more abstractions and services that have been built on top of these

tech

nologies and we're very happy today to be joined by our friends from McGill University and Steamship uh sill and Ted, who you'll hear in a moment talking to us about how they're making it easier

build

, deploy and share applications using some of these same technologies, so we thank you for hosting today our friends from Plimpton Jenny Lee and the student who is here with us today, but without further ado, let me hand things over to Ted, the windowsill and pizza will be served outside shortly after 1:00 p.m. m., very good, Ted, thank you very much.

Hello everyone, it's great to be here. I think we have a very good talk for you today. I'm still going to do some research. delving into how it all

work

s, what happens inside the GPT brain, as well as other language models, and then I'll show you some examples that we're seeing on the ground of how people

build

apps

and what

apps

tend to

work

on. the real world, so our perspective is that we are building AWS for AI applications, so we can talk to many of the creators who are building and deploying their applications and, through that, see both the experimental end of the spectrum as well as see what types of applications are being promoted and turned into companies turned into side projects.

Yesterday we did a great hackathon, many thanks to Neiman, David Malin and

cs50

for helping us organize all this, to Harvard for organizing it and there were two many sessions. of people built things, if you go to the steamship.com hackathon, you will find many guides, many projects that people built and you can follow them. We have a text guide and quick add-on for that if you want. do it remotely or on your own um so to prepare you we're going to talk about basically two things today that I hope you take away and really know how to use as you develop and as you do Tinker. what GPT is and how it works, get a good idea of ​​what's going on inside of it other than it being just this magical machine that predicts things and then two is how people are building with it and then most importantly how can I build with it too if you are a developer and if you have experience in CS50 you should be able to pick things up and start creating some cool apps.

I already met some of the CS50 graduates yesterday and the things they were doing were pretty amazing, so I hope this is helpful. I'm going to leave it on the windowsill and talk about some of the theoretical background of GPT, yeah, so thanks Ted um my name is so I'm a graduate student in Digital Humanities at McGill. I study literature, computer science and linguistics. At the same time, I've published some research over the last few years exploring what's possible with language models and culture in particular, and my half or whatever of the presentation is to describe to you what GPT is, that's real, that's really hard to explain in 15 minutes and there's even a lot we don't know, but a good way to approach it is to first consider all the things that people call GPT or descriptors so you can call them big language models that you can call.

They Universal approximators of computing it can be said that it is a generative AI we know that they are neural networks we know that it is an artificial intelligence for some it is a culture simulator for others it only predicts text it is also a writing assistant if you have ever used Chachi PT , you can include a little bit of your essay and get feedback, it's awesome because it's a content generator, people use it to write jasper.ai pseudorites, etc., it's an agent, so the really cool thing right now is yes You may have seen it on Twitter.

The folks at Auto GPT baby AGI are tooling these things and allowing them to run a bit freely in the wild to interact with the world's computers, etc., we use them as Chat Bots, obviously, and the actual architecture is a Transformer, so there are many ways to describe GPT and any other of them is a really good way to start the conversation, but for our purposes we can think of it as a broad language model and, more specifically, a language model, and A language model is a language model if you allow the tautology, but in reality what it

does

is produce a probability distribution over some vocabulary, so let's imagine that we have the task of predicting the next word in the sequence.

So if I give the words to a neural network I am, of all the words in English, the next most likely word to follow and, in essence, that is what GPT is trained to respond to and how it

does

it. He has a vocabulary of 50,000 words and knows approximately the entire Internet. what words are likely to follow other words from those fifty thousand in some sequence up to two thousand words up to four thousand up to eight thousand and now up to thirty two thousand three pt4 so give it a sequence here I am and about the vocabulary of 50,000 words , it gives you the probability of each word that follows, so here I am, perhaps the happiest word, quite often, so we will get that high probability if we look at all the words, all the expressions in English, it could be I'm sad. maybe that's a little less likely.

I'm from school, that should really be at the end because I don't think anyone would say I'm Björk. That's a little bit, not very likely, but less likely than happy, sad, but there's still something. attached probability and when we say it's probable it's literally a percentage that's like I'm happy. Maybe I'm like five percent of the time. Sad photos. Maybe I'm two percent of the time or whatever. For every word we give to GPT, it tries to predict what the next word is in 50,000 words and gives each of those 50,000 words a number that reflects how likely it is and the really magical thing that happens is that you can generate text new, so if you give it GPT, I'm and it predicts happy as the most likely word out of fifty thousand, then it can add it to I'm so now it says I'm happy and feeds it back into the model, tries another word, feeds it into the model over and over and over and over again and there are so many different ways that I'm happy, I'm sad, you can go and you add a little bit of randomness and all of a sudden you have a language model that can write essays that can speak and a a lot of things, which is really unexpected and something that we didn't predict even five years ago, so all of this is relevant and if we move forward as we expand the model and give it more calculation in 2012, Alex came out and we found that we can give it the model uh, we can run the model on gpus so we can speed up the process, we can give the model a lot of information downloaded from the Internet and it learns more and more and often the probabilities it gives you improve as you See more examples of English on the Internet , so we have to train the model to be very big, very wide and we have to train it for a long time and as we do that, the model becomes better and better, expressive and capable.

It also becomes a little smart and for reasons we don't understand, but the problem is also that because it learns to replicate the Internet, it knows how to speak in many different genres of text and in many different registers if you start the conversation as GPT chat, does it? Can you explain the moon landing to a six-year-old in a few sentences? gpt3 this is an example taken from the openai PTA document instructions gpt3 would have been like okay so you're giving me a For example, explaining the moon landing to a six year old. I'm going to give you a bunch of similar things because it seems very likely that they will come in a sequence.

You don't necessarily understand that you are being asked a question. respond with an answer gpt3 did not have that gadget that interface to answer the questions and the scientists had found the solution in openai and that is, we are going to give you a lot of examples of questions and answers such that we first exchanged on the Internet and then we trained it with a lot of questions and answers so that you are internet savvy but you really know that you have to answer questions and that's when GPT chat was born and that's when it gained 100 million users in a month.

I think it would be the record ticking of 20 million in a month, it was a huge thing and for a lot of people they said oh this thing is smart, I can answer, I can ask you questions, answer, we can work together to come up with a solution and that will happen. because it's still predicting words, it's still a language model, but it knows how to protect the words in the framework of a question and answer, so that's what a message is, that's instruction setting, that's a word key, that's what rlhf is if' I've ever seen that aligning acronym reinforcement with human feedback and all of those combined means that the models that are coming out today, the types of language predictors that are coming out today work to operate in a way that gpt4 exclusively only has the Align model available and this is a really great solid foundation to build on because you can do all kinds of things.

You can ask Chachi PT. Can you do this for me? Can you do that for me? You may have seen that Openai has enabled plugin access to CPT chat, so you can access Wolfram you can search the web you can search you can do instacart for you you can search recipes once the model knows that you don't just have to predict the language but you have to solve a problem and the problem here is give me a good answer to my question, suddenly you are able to interact with the world in a really solid way and from there there have been all kinds of tools that are built about this q a way that uses Chachi PT you have Auto GPT you have Lang chain you have uh react there is a react document where many of these come from and converting the model into an agent with which to achieve any ambiguous goal is where the future is headed and this is all thanks to the instructions wrap and with that I think I'll hand it over to Ted who will give a demo or something on how to use GPT as an agent, so okay, I'm a super responsive guy.

I look at things and think, okay, how can I add this Lego add-on? that Lego and put them together and build something with it and right now you know, if you look back at the history of computing when you look at the kind of things that were done in 1970, right after computing was invented, microprocessors were invented. , uh, people Researching how to sort a list of numbers was significant work and, more importantly, it was work accessible to everyone because no one knows what we can build with this new type of oil, this new type of electricity, this new type of calculation unit.

We created and anything was a game and anyone could participate in that game to solve it and I think one of the really exciting things about GPT right now is that yes, in and of itself, it's amazing, but what could we do with it if call it again and again if we integrate it into our algorithms and start incorporating it into broader software so that the world is truly yours to solve those fundamental questions of what could you do if you could program the calculation over and over again in the way you computers can not only talk to it, but also build things on top of it, so we are a hosting company, we host applications and these are just some of the things we see.

I'll show you demos of this with code and try it out. To explain some of the thought process, but I wanted to give you a high level overview, you've probably seen them on Twitter, but when it all comes to the top, these are some of the things we're seeing built. and deployed with language models today companionship that's all from I need a friend I need a friend with a purpose I want a trainer I want someone to tell me to go to the gym and do these exercises I want someone to help me study a foreign language the answer to the question is very important, this is everything from your newsroom, having a slackbot helping you, does this article fit our newsroom style guidelines all the way and do you need help with my homework or , I have some questions? what I want you to ask Wikipedia, combine it withsomething else, synthesize the answer and give me utility functions.

I would describe it as there is a large set of things that humans can do if only or computers could do if only they had access to computing languages ​​knowledge of languages ​​an example of this would be reading every tweet on Twitter tell me which ones I should read from that way I can only read the ones that really make sense to me and I don't have to skim through the rest creativity image generation text generation storytelling proposing other ways of doing things and then these wild experiments and kind of baby AGI as people call them in the ones that the AI ​​itself decides what to do and is self-directed, so I'll show you examples of a lot of these and what the code looks like and if you were like that, I would think of these categories within which to think about what you could build and then also search initial projects on how you could build them online, so I'll dive right into the demos and code. for some of these because I know that's what's interesting to see as companion Builders with a high level diagram for some of these as to how it works, roughly, you can think of a companion robot as a friend that has a purpose to ti and there are many ways to build all of these things, but one of the ways you can build this is to simply wrap GPT or a language model in an endpoint that also injects into the message some particular perspective or some particular goal that actually you want to use.

So easy in a way, but it's also very difficult because you need to iterate and design The Prompt to make it work consistently the way you want, so a good example of this is something someone brought into the hackathon yesterday and just wanted to show you. , uh, the project they built was a Mandarin idiom trainer and first I'll show you what the code looked like. I'll show you the demo first. First I think I already got it out, here we go, so this person's friend. What I wanted to create was a friend who if you gave her a particular problem you had, she would choose a Chinese idiom, a four-character Chung you, who would describe it poetically, like here's a particular way you could say this and she would say it to her. so the person who built this was studying Chinese and wanted to learn more about this.

So I could say something like I feel very sad and I would think a little and if everything is working it will generate one of these four character phrases. and will respond with an example. Now I don't know if this is correct or not, so if anyone can call me if this is really incorrect, please call me and then it will end with something encouraging. saying hey you can do this I know this is hard go ahead so let me show you how they built this and I pulled out the code right here so this was the particular initial response that people were using at the hackathon yesterday and we pulled it out.

Basically, you have a wrapper around GPT and there are a lot of things you can do, but we're going to make it easy for you to do two things, one of them is inject some personality into the message and I'll explain what that is. The message is in one second and then the second is ADD tools that can go out and do a particular thing, search the web or generate an image or add something to a database or retrieve something from a database, so once that's done, you now have more than just GPT now you have GPT, which we all know what it is and how we can interact with it, but you've also added a particular lens through which it speaks to you and potentially some tools, so this tutor of Chinese in particular was all that was needed to build it. four lines, so here's a question that I think is frying the minds of everyone in the industry right now.

Is this something we'll all do casually and no one really knows well? We just all say in the future to the movie Hello for the In the next five minutes, speak like a teacher and maybe, but also definitely, in the meantime and maybe in the future it will make sense to summarize these personalized final points so that when you talk to GPT don't just talk to GPT, have a whole army of different friends from different peers that I can talk to, they are kind of human and talk to me interactively, but since I preloaded them with hey, by the way, I want you to be a teacher of Chinese friendly and helpful. which responds to each situation by explaining the changu that suits you, speaks in English and explains the Chung in its meaning, then gives an encouraging note about learning the language and therefore you simply add something like that programmer and it will display it in the Web.

I'll take it to a Telegram bot that you can then interact with. Hey, I'm too busy to interact with him via Telegram over the web and this is the kind of thing that is now within the reach of all CS 101 graduates. Sorry. I'm using the general purpose framework all the way down to industry professionals, which can be done with just a little manipulation on top of this raw unit of conversation and intelligence, so companionship is one of the first common types. of apps that we're looking at so a second type of app that we're looking at and this blew up if for those of you who have Twitter followers, this blew up.

I think the last few months are answers to questions and I want to unpack a couple of different ways this can work because I know many of you have probably already tried to create some of these types of applications. There are a couple of different ways it works. The general framework is a user querying GPT and maybe has general characteristics. knowledge of purpose may not have general purpose knowledge, but what you want me to tell you is something specific about a paper you wrote or something specific about your course syllabus or something specific about a particular set of documents in the United Nations on a particular topic, then what you're really looking for is what we all expected the customer service bot to be, as if we've all interacted with these customer service bots and are racking our brains over the keyboard while we do, but very soon we will start to see very high fidelity Bots that interact with us comfortably and this is roughly how to do it as an engineer, so here is your game plan as an engineer, the first step is to take the documents you want to to respond.

Step two, cut them now, if you are an engineer this will drive you crazy. Don't cut them in a way that you would expect, for example, to be able to cut them into clean sentences, clean paragraphs, or semantically coherent sections. Honestly, that would be pretty cool the way most people do it and this is a simplification that tends to be fine. In your window, you have a sliding window that loops through the document and simply extracts chunks of text after extracting them. By taking out those chunks of text, you turn them into something called an Embedding Vector, so an Embedding Vector is a list of numbers that approximate some point of meaning, so we've already dealt with embedding vectors in normal life and the reason why they have done it and me.

What you know is that everyone has ordered food on Yelp before, so when you order food on Yelp, you look at what kind of restaurant it is, is it a pizzeria, is it an Italian restaurant, is it a Korean barbecue place, look at how many stars does it have one two three four five look where it is then all of these you can think of as points in space dimensions in space korean barbecue restaurant four stars near my house it's a vector of three three numbers that's all this is so this It's a vector of a thousand numbers or a vector of ten thousand different models produce vectors of different sizes, all it involves is chunking up chunks of text into a vector that approximates the meaning and then putting it into something called a database. of vectors and a vector database is just a database that stores numbers, but having that database now, when I ask a question, I can search the database and I can say, hey, the question was what does

cs50

teach what text fragments in the database have similar vectors to the question what does cs50 teach and there are all sorts of tricks and empires being created around the refinements of this general approach, but in the end you, the developer, simply model it. like this and then when you have your query, you embed it, you find the fragments of the document and then you put them in a message and Now we come back to the personality of the Companion Bots.

Now it's just a message and the message is that you are an expert at answering questions. Answer user-supplied questions using results from source documents in the database. That's it after all these decades. engineering of these customer service points, it turns out that with a couple of lines of code you can build this, so let me show you that I made one right before class with the cs50 syllabus so we can open it up and I can say that I added the PDF. right here, so I just searched I don't know if I apologize I don't know if it's an accurate or recent syllabus.

I just searched the web for the cs50 syllabus PDF. I put the URL here. I uploaded it here. just implemented a hundred line code that will now allow me to talk to it and I can tell what cs50 is going to teach me, so under the hood now what's happening is exactly what that slide just showed you, that question is needed , what will cs50 teach? In my case, it converts it to a vector that Vector approximates without exactly representing the meaning of that question. It searches a vector database that sends a lot of fragments of that PDF and then extracts a document and then passes it to a message that says "Hello, you." You are an expert at answering questions.

Someone has asked you what the CS50 teaches. Please answer it using only the source documents and source materials that I have provided. Now those source materials are dynamically loaded into the message. It's just basic engineering and I want to keep stressing that the amazing thing about Builders right now is that a lot of things just come down to a very creative tactical reorganization of cues and then using them over and over again in an algorithm and putting them into the software so that the result can appear again and again. be lying, could be making things up, could be a hallucination, cs50 will teach students how to think algorithmically and solve problems efficiently, focusing on topics such as abstraction and then returns the source document from which it was found, so this is another big category that there are tons of potential applications because you can iterate for every context, you know you can arbitrarily create a lot of these once it's software because once it's software you can iterate over and over again for your bedroom for your club for your slack for your telegram you can start entering information and then respond to it and they don't have to be documents.

You can also upload it directly to the message. I think I have it here and if I don't. I'll just skip it, oh here we go another way, you can answer questions because I think it's healthy to always encourage the simplest possible approach to something that you don't need to design this giant system. It's great to have a database. It's great to use embeds, it's great to use this great approach, it's elegant, it scales, you can do a lot of things, but you can also get away with just squeezing everything into a message and as an engineer you know that's a This year, one of our teams always says that engineers should aim to be lazy and I couldn't agree more that you as an engineer should want to prepare yourself to be able to follow the lazy path towards something, so here's how you can do the equivalent to a question answering system just with a message, let's say you have 30 friends and each friend is good at a particular thing or you can know this isomorphic to many other problems, you can just say hello, I know certain things, these are the things I know The user is going to ask me something, how should we respond and then load it into an agent.

That agent has access to GPT. He can send it, deploy it and now he has a bot that can connect to Telegram. You can connect to Slack. that robot now won't always give you the right answer because at some level we can't control the variation of the underlying model, but it will tend to respond with respect to this list and the degree to which it tends to do so is to some extent, it's something in which that both industries are working to give everyone a capability, but you're also doing rapid engineering to adjust the error bars, so I'll show you just a few more examples, and then on about eight. minutes I'll move it to questions because I'm sure you have a lot on how to build things, so just to give you an idea of ​​where we're foreigners, this is one.

I don't have a demo for you. but if you came to me and said Ted, I want a weekend Hustle Man, what should I build? Holy cow, there are a set of applications that I would describe as utility functions. I don't like that name because it doesn't sound exciting and this is really exciting and the low hanging fruits are those that automate tasks that requirebasic understanding of the language, so examples for this are generating a unit test. I don't know how many of you have ever been writing tests and you're like ah come on I can get through this, I can get through this, if you're a person who likes to write tests, you're a lucky person looking for documentation of a function, rewrite a function, make something conform to your company guidelines, do a check mark, all of these things are relatively context-free operations or context-scoped operations on a piece of information that requires linguistic understanding and really You can think of them as something that is now available to you as a software developer. a weekend project builder like a startup builder and you just have to build the interface around it and present it to other people in a context where it is meaningful for them to consume it, so the space of this is extraordinary, I mean, it's everyone's space. human effort now with this new tool, I think that's the way to think about it, people often joke that when you're building a company, when you're building a project, you don't want to start with a hammer because you want to start with a problem and in general is true, but my goodness, we just got a really cool new hammer and to some extent I encourage you to at least casually on the weekends run around and hit things with and see what can happen from a Builder to a TinkersFrom an experimentalist's point of view, creativity is another huge mega-application.

I live mostly in the text world now, so I'm going to talk about text-based things. I think so far this has mostly been growing in the world of images because we are such visual creatures and the images you can generate are amazing with AI certainly raises a lot of questions about intellectual property and art style as well, but the template for this , if you're a builder, what we're looking at in the wild is roughly the following and what I want to point out is domain knowledge. This is really the purpose of this slide is to address the importance of domain knowledge for many people to roughly find the creative process as follows. great idea about generating possibilities edit what you overgenerated repeat well as anyone who has been a writer knows that when you write you write too much and then you have to delete a lot and then you revise and you write too much and you have to delete a lot, this particular task is fantastic for AI, one of the reasons it's fantastic for AI is because it allows the AI ​​to make mistakes, you know you've pre-agreed that you're going to eliminate a lot, so if you pre-agree hey I'm just going to build you know generate five possibilities of the story could tell five possibilities of the advertising headline five possibilities of what I could write about what I could write my thesis about you pre-agreed it's okay if it's a little long because you will be the editor who will intervene and this is what you should really bring to the table: Don't think of this as a technical activity, think of this as your chance to not put GPT in charge. instead of you gripping the wheel tighter, I think at least in Python or whatever language you're using to program because you have the domain knowledge to handle GPT in generating those, so let me show you an example of what I want say with This is a cool app that someone created for the Atlas writing project, so writing Atlas is a set of short stories and you can think of it like Goodreads for short stories, so you can go in here and explore different stories and this was something someone created a story where you can write a description that you like and this will take about a minute to generate so I'll talk while it's generated and while it's working what it's doing and I'll show you the code in a second search the stories collection stories similar and this is where the domain knowledge part comes in, then use GPT to see what it was you wanted and use the knowledge of how an editor thinks about how to generate a bookshelf. a set of suggestions specifically through the lens of that perspective with the goal of writing that beautiful handwritten note that we sometimes see in a local bookstore stuck under a book that doesn't just say hey, you might like this, this is a general purpose. reason why you might like this, but specifically here's why you might like this with respect to what you gave it, it's either stalling or it's taking a long time, here we go, so here are your suggestions and in particular , these things are things that only one human could know at least for now two humans, specifically the human who said he wanted to read a story which is the text that arrived and then the human who added domain knowledge to write a sequence of interactions with the language model so you could provide very specific reasoning about something that was informed by knowledge of that domain, so for these utility applications bring your domain knowledge, let me show you what this looks like in code because I think It's helpful to see how simple and accessible this is. a set of prompts, so why might they like a particular location?

Well, here's the prompt that says this is an open source project and it has a bunch of examples and then it says well, here's the one we're interested in, here's the audience. here are a couple of examples of why people might like a particular thing in terms of audience, it's just another message, same for the topic, same for the explanation, and if you go down here and see how it was done , suggesting that history is what this line 174 is referring to. line 203 really is and again like over and over again I want to make it clear to you that this really is within reach, it's really just what 20 odd lines of the Step one searches the database for similar stories.

Step two, since I have similar stories, I pull them out. the data, step three with my domain knowledge in Python, now run these prompts, step four, prepare them into an output so that what we are programming is an approximation of human cognition, if you are willing to go there metaphorically, we don't know . We're not sure not to weigh in on where we are on open AI, an argument about life forms, okay, very far away kind of thing and then I'll leave it ready for questions. because I know there's probably a lot of them and I also want to make sure that they have a great pizza in their bellies and that it's a baby.

AGI Auto GPT is what you may have heard them called on Twitter. I consider them multi-step planning robots. So everything I showed you so far was about One-Shot interactions with GPT, so the user says they want something and then Python mediates the interactions with GPT or GPT itself does some things with the inflection of a personality that you he added. some really useful quick engineering pretty easy to control if you want to go to production if you want to build a weekend project if you want to build a company that's a great way to do it right now this is crazy and if you haven't seen it these things on Twitter I would definitely recommend looking them up.

This is what is happening. The easiest way to say it is if you put GPT in a for loop, if you let GPT talk to itself and then tell itself what to do to make it an Emergent Behavior like and like all emergent behaviors, it starts with a few simple steps , Conway's Game of Life, many elements of reality turn out to be mathematical equations that fit on a t-shirt, but then when you play them forward in time, they generate DNA or generate human life, so this is roughly the step one, take a human objective, step two, your first task is to write a list of steps and here is the critical part, repeat now make the list of steps, now you have to embody your agent with the ability to do things, so It's really just limited to doing what you give it the tools to do and what it has the skills to do.

Obviously, this is still very much a set of experiments that are running right now and it's something we'll do. What we will see will develop over the next few years and this is the scenario where Python stops being so important because we have given him the ability to self-direct what he is doing and then he finally gives them a result and I want to give you an example. Still, again I explain to you how much of this is rapid engineering, which is crazy, how little code this is, let me show you what baby AGI looks like, so here is a baby AGI that you can connect to Telegram and this is an agent which has two tools, so I haven't explained what an agent is.

I haven't explained to you what the tools are. I'll give you a quick one-sentence description. an agent is just a word that means gpt plus a larger body it's living in maybe that body has a personality maybe it has tools maybe it has python mediating its experience with other things tools are simply ways in which the agent you can choose to do things like imagine if GPT could say order a pizza and instead of you seeing the text order a pizza that caused the pizza to be ordered that's a tool so these are two tools it has a tool that generates a list of to-do one tool is to do a web search and then down here you have a message that says hey, your goal is to build a to-do list and then make that to-do list and then this is just put into a harness that makes it a and again, so that after the next task, you discount the results of that task and continue and do it. you get this loop started where you essentially start it and then the agent talks to itself, so unless I'm wrong, I don't think this has made it to production yet in terms of what we're seeing in the field of how people deploy software , but if you want to dive into the wilder side of experimentation, this is definitely one of the places you can start and it's really within your reach, all you have to do is download one of the starter projects. and you can see right in the prompts, that's how you start that iteration process, so I know it was super high level, I hope it was helpful, I think from the field, from the bottom up, what we do.

We're looking at and what people are creating, high level categories of apps that people are creating, all of these apps are apps that are accessible to everyone, which is really exciting, uh, and I suggest Twitter is a great place. to spend the time. and build things uh there are a lot of AI builders on Twitter uh posting and if I think we have a couple of minutes before Pizza arrives, maybe 10 minutes go on, so if there are any questions, why don't we kick it? That's because I'm sure there are some questions you all have. I guess I finished a little early.

Yes. I'm giving you a physics problem from a pset and we want to do it, yes, yes, 40 of the time, raw, yes. Do you have any similar practical recommendations that we as developers should do to reduce the story or maybe even things that open up AI on the back-end we should do to reduce the artwork? Would this be something you would like to use our lhf for, so the question was how? Roughly, how do you handle the problem of hallucinations? For example, if you give him a physics lecture and ask him a question, on the one hand he appears to answer you correctly, on the other hand he appears to be wrong to an expert's eye 40 of the time. 70 of the time, 10 of the time it's a big problem and then what are some ways that developers can practically use to mitigate that?

I'll give an answer, but you may also have some specific things, so a high level answer is the same. that makes these things able to synthesize information is part of the reason they blow your mind so it's hard to have your cake and eat it too to a certain extent so this is part of the game in fact humans too They do it, as people say. You know, people who are too aggressive in their assumptions about knowledge. I don't remember the name of that phenomenon where you just say the right thing and we do it too.

Some things you can do are kind of a variety of activities depending on how much money you're willing to spend, how much technical experience you have, which can range from tweaking a model to hands-on. I'm in the applied world so I'm in a duct tape world and that's how developers do things so some of the answers I'm going to give you are very duct tape like answers, giving you examples it tends to work for sharp things if he behaves wildly, the more examples you give him the better. I'm going to solve the domain of all physics, so for the domain of all physics, I'm going to rescue it and give it to you because I think you're much more equipped than I am to talk about it, so the model doesn't.

It has no fundamental truth It knows nothing Any sense of meaning that is derived from the training process is purely by differentiation A word is not another word Words are notused in the same context Understands everything only through examples given through The language is like someone who learned English or how to speak, but grew up in a gray and featureless room, has never seen the outside world, has nothing in What a rest that tells them that something is true and something is not true, from the perspective of the models. everything he says is true, he is doing his best to give you the best possible answer and if lying a little or combining two different topics is the best way to achieve this then he will decide to do it, it is part of the architecture that I can't help it, there is a number of cheap tricks that surprisingly make you confabulate or hallucinate less one of them includes recently there was an article that is a bit funny if you prefix it to your answer, my best guess is that it will actually improve or reduce hallucinations by about a 80 percent, so he clearly has some sense that some things are true and some things aren't, but we're not quite sure what that is to add to what Ted was saying some cheap things you can do. include leaving it on Google or Bing like in Bing chat, what they are doing, quotes this information and asks you to make sure your own answer is good.

If you have ever had the opportunity to generate a program, there is some kind of problem and you ask Chachi PT I. I think there is an error, you will often locate the error by yourself. Why he didn't produce the right answer from the beginning. We're not sure yet, but we're now moving in the direction of reducing hallucinations regarding physics. I'll have to give it an external database to rest on because internally, for truly domain-specific knowledge, it won't be as deterministic as one would like. These things work in continuous spaces. These things don't know what is wrong, what is true. and as a result we have to give it to ourselves, so everything that Ted's demonstration today really strives to reduce the hallucinations and give him more abilities.

I hope that answers your question in one way. I mean, I'm a simple guy like I tend to think that everyone tends to be just a few things repeated over and over again and we have human systems for this, you know, in a team like companies, work is a team that play.Sports and we are not always right, even when we aspire to be, and so we have systems that we have developed as humans to deal with things that may be wrong, so you know, human number one comes up with an answer. Human number two checks.

His human work number three provides the following final sign. This is very common. Anyone who has worked in a company has seen this in practice. The interesting thing about the state of software right now we tend to be in this mode where we're just talking. to GPT as an entity, but once we start thinking in terms of teams, so to speak, where each team member is their own agent with their own set of goals and skills, I suspect we'll start to see a model of programming where the way to solve this might not necessarily make a single brain smarter, but draw on the collective intelligence of multiple software agents, each of which plays a role, and I think that would certainly follow the pattern human of how we deal with this to give you an analogy. space shuttles things that go into space spaceships have to be good if they are not good people die they have no margin for error and as a result regarding engineering in those systems most spaceships have three computers and they all have to agree on the unison about a particular step forward if one disagrees then they read calculate recalculate recalculate until they arrive at something the good thing is that hallucinations are generally not a systemic problem in terms of knowledge, they often are - outside the model, something tripped him up and just produced a hallucination in that case, so if there are three models working in unison, as Ted says, that will generally improve his claims of success as if he were an engineer, he was an AI. a teacher, yes, what is the mechanism by which that influences this probability location?

I'm sure I'll give you what might be an unsatisfactory answer: it tends to work, but I think we know why it tends to work and again it's because this language models approximate how we talk to each other, so if I said to you "hey, help me", I need you to do a mock interview with me, that is a direct statement that I can make that pushes you into a certain mode of interaction or if I say to you, help I am trying to apologize to my wife, she is very angry with me, Can you do a role play with me that takes you to another mode of interaction?

So it's really just an abbreviation that people have found for kick the agent to kick the LOM in a certain interaction mode that tends to work the way I as a software developer expect it to work and to add it very quickly to that, um in the digital humanities that I am, I like to think of it as a narrative a narrative will have some different characters talking to each other their roles are clearly defined two people are not the same this interaction with GPT assumes personality can simulate personalities it is not cautious by any means but you can certainly predict what a conscious being would react like in a particular situation, so when we go, you're that that The character is present and interacts and is active so yes, I think we have about five minutes until the pizza outside eight minutes yes sir no, I'm not, I'm not, yes person, but it's been kind of fun with this and I understand the kind of generation word for word and the kind of vibe, the feel, you know the narrative, some of my friends and I have tried giving it logic problems, like LSAT stuff, for example, and it doesn't work.

I'm just wondering why that would be, it will generate answers that sound very rhetorically plausible, like given this condition ask him and this would be why, but often he'll even contradict himself in his answers, but it's almost never correct, so I was wondering why would it be like this he just can't reason he can't think and I like could we get to a place where he can, so to speak? I mean, no, you know what I mean, I don't need to. I think like I'm conscious I mean I have thoughts you want to talk about react so gpt4 one from when gpt4 was released in March I think it was I was passing the LSAT it was yeah yeah it just happened from what I understand , because that's one of The strange thing is that yes, if you pay for GPT work, they give you access to the best model and one of the interesting things is that it indicates that it is very sensitive if it is very sensitive to the way that you request that there was before when gpt3. came out, some people were like, "I can pass the literacy tests or I can't, I can't pass the literacy tests," and then people who are for or against gpt would say I modified the message a little bit, suddenly it can or suddenly he can't. these things are not conscious their ability to reason is like aliens they are not us they do not think like people they are not humans but they are certainly capable of passing some things empirically which demonstrates some type of rationality or logic within the model, but we are still slowly discovering, as a quick whisperer, what exactly is the right approach.

Have you seen cases where you directly create some kind of business value in a sort of startup or company where there is real added value of having sort of lead small AI applications, yeah, I mean, we host companies on top of we, who that is their main product, the value that it adds is like that of any company, I mean, you know what the mono combinator is And to make something that people want, I mean. I wouldn't think of this as GPT inherently providing value to you as a builder, that's your product, that's the Open the Eyes product, you pay to chat with GPT to get priority access, where your product might be, that's how you take it. and combines it with your data. someone else's data some domain knowledge some interface that then helps you apply it to something are two things that are true, there are a lot of experiments going on right now, both for fun and for people trying to figure out where the economic value is, but folks.

They are also building companies that are 100 behind themselves by applying this to data. I think it's likely that today we'll call it GPT and today we'll call it llms and tomorrow it'll just slide into The Ether. I mean, imagine what the progression is. "Today there will be one of these that people are mainly playing with, there are many of them that exist, but there is one that is mainly bidding above. Tomorrow we can expect there to be many of them and the next day we can expect that they'll be on our phones and they won't even be connected to the Internet and for that reason I think that, like today, we don't call our software microprocessor tools or microprocessor applications as if the processor just exists.

I think that a useful model five years from now. ten years is even if it's only metaphorically true and not literally. I think it's useful to think of this as a second processor that we had this with before, with floating point coprocessors and graphics scope processors as recently as the 90s. where it's useful to think of the trajectory of this as just another thing that computers can do that will be incorporated into absolutely everything, hence the term Foundation model that also comes up. Sorry, the pizza is ready. more question maybe one more and then then we'll have some food in the glasses right there sorry I just got told we need two more so yeah it's hard to get it to do it reliably it's incredibly helpful . to make it work reliably, some tricks you can use are: you can give him examples, you can ask him directly, those are two common tricks, um and look at the prompts that others have used to work.

I mean, there's a lot of art to be made. To find the right message right now, a lot of it is magical incantation, another thing you can do is post-process it so you can do some checking and you can have a happy path where it's one shot and you get your answer and then a sad path where maybe you turn to other cues, then you go for the diversity of approaches where it's fast by default, it's slow but ultimately converges on a higher probability of success if it fails and then something that I'm sure I'll see what people will do later is hone in on the instruction tuning style models that are most likely to respond with the computer package, uh, output, so I guess one last question, right, so, whoever you talked to? a couple of things, one is like you. talk about domain expertise here and you're encoding a bunch of domain expertise in terms of the cues that you're putting in there, what is that, where do those cues end up, do those cues end up in the Jeep chapter model and is there a privacy issue associated with that, that's a great question, so the question was and I apologize.

I just realized that we haven't been repeating all the questions for YouTube listeners, so I feel sorry for the YouTube people if they couldn't do that. I heard some of the questions, the question was: what are the privacy implications of some of these prompts? If one of the messages depends both on its indication and the setting of this indication, what does that mean with respect to my IP? perhaps the indication It's my business. I can't give you the exact answer, but I can describe to you what the landscape roughly looks like across software and also with AI.

What we see is that it's the SAS companies where you're using someone. Someone else's API and you trust that their terms and services will be respected. There are a set of companies where they provide a model for hosting on one of the big cloud providers and this is a version of the same thing, but I think with a slightly different twist. mechanically, this tends to be thought of as the enterprise version of software and generally the industry has gone in the last 20 years from running my own servers to trusting that Microsoft, Amazon or Google can run servers for me and they say that's my private account. server even though I know they are running it and I am okay with that and you will have already started to see that Amazon with a hug face Microsoft with open AI Google 2 with their own version of Bard they are going to do it Here you will have the SAS version and then you'll also have the private VPC version and then there's a third version that I think we haven't seen practically emerge yet, but this would be the maximalist one.

I want to make sure that my IP is a maximally safe version of events where you are running your own machines, you are running your own models and then the question is whether the open source or privately available version of the model is as good as the publicly hosted one and does that matter? Me and the answer right now is that, soRealistically, it probably matters a lot. Over time you may think that any particular task you need to perform requires some fixed point of intelligence to achieve and so over time what we will see is the privately obtainable versions of these models will cross that threshold and regarding that task, yes, sure you use the open source version, run it on your own machine, but we will also see SAS intelligence become smarter, it will probably stay ahead of the curve and then The question is which I care more?

Do I want better aggregate intelligence or is my task a fixed point and can I use the open source available for which I know it will work well enough because it has crossed the threshold, so to answer your question specifically, yes, you will be happy to know if Chachi PT recently updated their privacy policy to not use prompts for the training process, but so far everything went back in the trash for retraining, okay and that's just a fact, so I think pizza now it's time to pizza, yes, okay, foreign.