How ChatGPT Works Technically For Beginners

I have been using chat GPT for the last two months and I am a software programmer and now every day eighty percent of my work in terms of coding is code generated by

chatgpt

and some other AI code generation tools that I incorporate .

chatgpt

, but mostly it's been chat apt and it's amazing how it completely transformed the way I work every day and throughout this journey I've been excited and relieved because some of the coding I do every day I just don't feel like doing. do it because it's repetitive and I've been doing it for almost 20 years.

I am so relieved, excited but at the same time scared because I could see how in some cases when I tell it to code something, it actually codes better than me and I can learn from it while using it, so with all these feelings inside me I finally I took the time to. Let's learn how it

works

, how it is built and how it

works

. Technically, I would like to know the details, but also at a very high level, like I don't want to get into math or anything like that. I just want someone to explain to me at a very high level in kind of dumb language, in layman's terms, to explain to me how it's built, how the system is built.

Scientists are building it and, um, yeah, how it's basically produced and I couldn't find anything online that did it really well, so I spent a lot of time learning it and decided to make a video of my own that's a complete beginner's guide. in terms of how chat apt is researched, how it's discovered, built, developed and released to us so that everyone can use it, so let's jump right into it, GPT chat for

beginners

and, um, I'm going to put these slides on the description of my video. and there's my YouTube channel, so you can share it if you want with someone else and use this slide if you want.

So what is GPT chat and why did it shock the world? Chat GPT is a conversational AI that is capable of carrying out an intelligent conversation with a human, so you can think of this as almost like the Iron Man scene in the Marvel movies where Iron Man says: Hey Jarvis, I need to build this and this. Can you run some diagnostics on this and this and Jarvis says? like after two seconds and then it's like here are the results and then Iron Man says oh, that's amazing, can you look into that in more detail and then Jarvis says okay, sure, and then he does more research and then come back and them? they work collectively on a conversation and then build this together, so that means that AI is now capable of having intelligent human-like conversations with another human being, so AI research has been around for almost 80 years, so which is a quite old field, but the conversational and also during this time everyone tried to figure out this conversational Ai and they all failed if they turned out to be incredibly different and difficult tasks to do and, the reasons why this is difficult is our natural language, like english.

I know I speak Korean um, I mean, any other language that you speak, it's not a precise language like mathematics, um, when you speak, um, in natural languages, they are full of nuances both in the grammatical structure of the language, so, the The sequence in which words appear in the sentence matters and then depending on the context of what you're talking about, the meaning of the same word pronounced the same way are different, so you have to really listen. to the whole context complete explanation of what you are talking about to be able to interpret each word and this is exactly why it is known that it is quite difficult even for humans to be able to fully develop their brain and to be able to continue as an intelligent conversation with another and it is because that it really takes a long time for human babies to learn to talk and have a conversation so when chatgpt was able to have that conversation and in some cases for certain topics it was able to do it better than humans uh it really gave a lot of feelings inside emotions inside um humans um like things like I mean they feel amazed by this emotion and fear and these emotions uh you can get a better kind of control over them once you know the limitations of what Shept is capable of and what is not capable because you know that even if you are excited, you don't want to get too excited and disappointed to discover the limitations later, if you are afraid it may not be necessary.

Be afraid, but you are scaring yourself unnecessarily because you don't know what he is capable of and what he is not capable of. You're just a little scared, so let's get into the details of how it works and then once we understand how it works. works, then we can understand what it is capable of and what it is not capable of doing and some of the clear differences between Ai and human beings so that then you know all these emotions become clear, so how did the AI ​​scientists decipher this conversation? AI, so it turns out that the only animal that can do this massive natural language processing and have a conversation is humans, we're really the only animals that can do this on planet Earth, so scientists decided to basically say Okay, If we can't model this manually with math, let's literally take a look at how our brain works and then simulate it on a computer and see what happens.

So this also this program is completely reactionary and I say that because once you create this program, a computer program that simulates our brain, it kind of sits there and waits on a computer server, and then it waits until it is included in a certain conversation so some user logs into the chatgpt website and then starts typing like a paragraph or paragraphs of something they want to talk about with GPT chat and then at that time it takes all the text data that the user literally typed on their keyboard, runs them through the program and then gives a response, so it's completely reactionary. sitting there and hoping to react right, then there's that input part and then there's the generated response part, input and output, so that's the whole AI program, so let's back up a little bit and see how scientists start to approach, like looking at our brain, so this is a guy who won the Nobel Prize what he did was he took the brain of a rat, he cut it into very thin slices and then he dropped the watercolor so he could better visually see the structure of the slices of the brain. brain so he just put that slice under the microscope and he was looking at different parts of the brain and what seemed to be a pretty random structure so what you see here are neurons and then between the neurons the neurons are basically like you could Think of them as cells. and between the neurons, these cells there is a thin connection through which electrical signals travel and that is why when you see MRI scans you see as if this whole complex structure lights up in the MRI scan it is because there is an electrical signal that travels through these neurons and are these neurons connected randomly?

That's what I thought at first, but it turns out that different patterns emerge, so depending on what part of the brain you know is dedicated to memory, let's say it's structured a certain way if there's a part of the brain that's dedicated to memory. logical thinking, logical processing, so it's structured in a different way, the actual neurons are the same, but their connections are seen in some parts of the brain like this and then in other parts of the brain, they look somewhat different and that's what which he discovered and uh um, I don't know exactly what exactly he did to win the Nobel Prize, but these are some of the official regions that he created, so actually neurons are connected to each other in 3D space, so when you look at our brain, it looks like a 3D object like this, and if you look at it, you know they look like this here and you have neurons connected in all directions surrounding a neuron and then the connections are like these thin connections that you can see and what you see is like let's say that a brain is like that here and then you give input signals from all parts of your body so that your eyes, your touches with your fingers, your smell from your nose, everything is converted into electrical signals. for its you know, sensing points, like the fingers or the nose, smell receptors, whatever, and then there's a visual optical receptor in our eyes, whatever, the parts of our body receive it and then it will convert it into electrical signals and then it will send them to your brain and your brain what the neurons do is when the input signals reach the brain there are initial input receiving neurons and then when the signals hit them they activate and then they propagate the next signals to the next connected neurons and then it looks like this It lights up like you look at the brain, it lights up like you send signals from one neuron to another, the input neurons fire first and then you see an animation like you know the lights go on.

They light up in your brain and light up. in different patterns even if you get the same kind of visual cues from your eyes depending on what you're looking at, whether you're looking at a peach or whether you're looking at a mountain, whatever your brain is, although ultimately they come from the same Source, your eyes send different types of signals with different intensities and your brain lights up in different ways, so how do we do it? It's too complex, it's too complex and especially too complex to program this on computers, so how? So computer scientists simplified this into a very, very simple model, so forget about all the connections, let's look at a single neuron, how can we reduce it to a simple concept?

Basically, they took this neuron and started with: Let's say an input neuron can have multiple inputs, but let's talk about this input here and here the input from which you can receive electrical signals, let's say zero to nine, zero is very weak and nine It is a very strong signal, you get a signal that is three and then this neuron, this particular neuron, each neuron behaves and activates differently, but this particular neuron when it receives three it activates and emits a 519 because it has three connections with three other neurons, so at the end of this connection there are other neurons here.

So this neuron will receive five signals five even though this neuron received a signal three, this neuron will receive five, this neuron will receive one and this neuron will receive nine, so if we have many neurons like this and they are all connected in the computer program , TRUE? Are we imitating what the brain does? Yes, and depending on that, you don't just have one input neuron, you have multiple input neurons and then you send signals to it simultaneously. They will all light up and then you will have the output neurons at the end that will also activate and then give you human understandable answers at the end, so let's talk about that, one of the most successful AI cases that AI actually became .

Image recognition is one thing and this is where we begin to see the importance of using neural networks. What the scientists basically did was they had a set of images, thousands and thousands of images of dogs, birds and cats, and then what they wanted to see is basically when they present this image of a bird and then they convert it into electrical signals in the initial input layer where they have, in this case, a very small neural network, but this is just to demonstrate to you what it would look like, but in reality, to understand an image like this, you would need thousands and hundreds of um uh, you could potentially have hundreds of neurons.

I don't know, maybe this amount of neurons is good enough to decipher the Burkhead dog. I'm not sure what it is. We have an input layer and then they will all decide to turn on and activate and then pass signals to the next layer link and then they will also decide to pass signals to the next layer, the next layer and then at the end because as humans we are only interested in three types of answers dog just tell me if it's a dog, bird or cat so we have three neurons at the end and then the question is what kind of signals given the input image what kind of signals do we look for a neuron that is responsible for a response for the dog, the bird or the cat?

Hopefully our hope is that, given the image of a bird, we get a very strong signal coming out of this neuron that is responsible for the bird and then we get very weak signals for the neurons that are responsible for the dog and the cat, then we can say with security, as a human being, okay, this AI, tell me what this is and then, oh, it's a bird, right? So what is AI training? We heard all this. At the time when people talk about chat, GPT had to be trained properly, so like I said, we have thousands of bird dog images and initially you can have these neurons that are connected, in this case one neuron has a connection to all of them. the other neurons. in the next layer and then this one, let's say, is also connected to all the other neurons in the next layer and initially when you input an image of a bird, they all travel, remember that some neurons will decide to activate and some neurons will just decideactivate maybe from here to this this and this and then it will ignore this and this.

At first everything is random, so you start with a very random neural network, you give it a picture of a bird and it gives you the answer dog, that's what usually happens when you start at random and it will give you the wrong answer every time, and the idea is that then you have a training, uh, data set of, say, thousands of dog, bird, and cat images, and then every time it gives a wrong answer, you basically tell the neural networks that it wasn't the correct answer, so you need to change your trigger behavior a little so that everyone changes. their triggering behavior and then it will feed the pictures again and then test whether it is or is giving the correct answer or not and then it will continue to do that over and over again until they start more or less giving the correct answer. in the end, then, whatever it's doing inside, uh, what seemed like purely random activations now become like um, they start to have certain patterns, so if you fit into a picture of a dog, then you'd like it to certain neurons light up. and that would eventually lead to this thing turning on and giving you the conversation response, so that's what it means to be training, you really have to start with a random set of neural networks, connections and activations and then keep reiterating and telling it that you're wrong you're wrong you're wrong until you start you start getting the right answer okay two more of that do more of that and then eventually activations and connections are the intelligence within an AI, so in this particular design of a network neural where initially you have input layers and then the next layer of neurons in the neck and then you activate the next layer of neurons, activate the next layer of neurons and eventually activate the output layer of neurons is particularly good for the image. recognition, but this particular pattern of neural network fails miserably at other tasks, especially like natural language, like if you want to have a conversation, then what scientists realize, um and there's a lot of things.

I think Google is like the most advanced AI scientist. not even the open AI that created chpt, actually the open AI scientists learn from Google papers, so what the Google people realize is that we can come up with not only this neural network pattern, but we can structure it so sometimes you can even have an output from a neural network that feeds back to the previous layer, that's a little crazy here, you just see One Direction from left to right and then finally you get a response, but instead From that, sometimes they thought that you can even look back at your own neuron or you can send the signal back and feed it to the previous layer, so the answers that you get from that start to become very interesting and they discovered that these different patterns of the type of networks of neurons um and a lot of this is also based on ours uh looking at our neurons in the brain they look good, how are the neurons in our brain connected?

Is it really unidirectional from left to right? In this simple case, that's not the case, our neurons in our brain are very complex and they make connections with other neurons and then that neuron five or seven steps later also connects back to this neuron and it's all mixed up and it's crazy, so they also simplified all those patterns, crazy patterns, and created some of the patterns that seem to work pretty well in a computer program. I don't know much about this, I mean it seems like the red are the output neurons where you get the final response and the yellow are the input neurons where you initially hit with the initial input data and whatever is in between just He does his crazy magic.

I don't know the details of these, but the important thing is that these are Many of these patterns are borrowed from our biological brain, so I just want to make sure that, depending on the network of neurons, you can have very simple behavior and also very complex behavior, that's all we need to understand from here, now, so how? Did the GPT chat come up correctly? So this is an article by a Google scientist and I don't understand it either so I'm not going to try to explain it in detail, but on the left side what they did was the initial part where when humans actually type the input as a message chat, it would come and get to the initial part of the chat gpt neural network that tries to understand what is happening and tries to understand the context of the input and then the next part is to generate the response, so they have broken it down to understand what is the input, understand the input and then the next second part is generate the response, that's how they did it, but I guess the neural networks that are involved here and here are like based on a very complex pattern, nothing like this simple It's going to be a very complex pattern, but it's also the way you train them, basically, it's very similar to how humans learn, human babies learn to speak the language even earlier.

They go through formal education at school, so you can see babies from seven years old to 80 years old. They start talking to their mom and dad, but they do it in a very unprofessional, uneducated, very rudimentary way, but they can still communicate. but how are they doing? It's not like mom and dad are sitting there teaching the kid grammar. Mom and dad just talk to them, her uncle and whatever other human being talks to that baby and they all talk to her about different things. baby, sometimes they talk about the TV show, sometimes they talk about buying a chocolate in a supermarket, whatever the topics are, the human baby is sitting there and just getting all this information from other different adults and then they start finding patterns in their brain and those are their complex neurons here that you see forming connections with other neurons and then if that was a mistake they would disconnect it and then they would form it with other neurons and then they would also change the firing behavior.

Okay, so all of that development is happening inside the baby's brain, but it's completely unsupervised. No one is guiding the child about what is right and what is wrong. The baby just takes it all in and then speaks to test the baby's flow. Current babies are in brain state and then if the feedback from adults is not what they expect then they would just feel stressed and that makes the brain like to disconnect from the current connections of neurons and then reorganize and then try it. again in the real world and that process is repeated over and over again just as the scientists are training it or what appears to be a very random simulated neural network and then they keep training that the baby is doing the same thing, so before we go to the school they go through this whole pattern basically good unsupervised in the chaos phase and then they enter the school and then they receive formal education and this is with very strict guidance and the school tells the child either by the teacher or by test scores, the school tells the child what is a best answer and what is a worst answer and this is fully supervised learning and this is the fine tuning, so most of the work has already been done with unsupervised running learning by taking in all this complex information and the baby can find rough patterns of what the conversation is about, but then school is like a cherry on top, basically taking that and then fine tuning the baby's brain with guided education based on orientation does exactly the same thing so that the part of the brain is the neural networks in chatgpt that have to understand the context of what is being talked about when they are training those neural networks there is no human participation, they simply collect all kinds of information from the Internet and then they throw it into neural networks. and then the neural networks will start to find approximate patterns, so this set of blog posts I think refers to something related to technology, I think it refers to I don't know how to travel, I think this refers to I don't know the relationships human marriages, whatever it is, start finding these patterns and grouping the blog posts across all the texts.

Outdoor claims that they removed all the text websites and blog posts and everything that is text based, they removed everything from the internet and just threw it right into this neural network, so this neural network was completely trained without monitoring and can find patterns, given any input text the chatgpt website user starts typing whatever they want to talk about. channel and when that input hits that neural network, it can find patterns and context of exactly what that person is talking about very quickly, so it's almost like what a child does well and then the actual part of responding, you know, responding to them. response to human website in chat apt that is on a task from another neural network that has been completely trained with human supervision, so what that means is that I openly literally hired thousands and thousands of people and then the results of the understanding of the context.

The neural network will be the input to the response that the neural network generates, all right, so now I understand what this person is talking about in the chat right now, that's how I understood them, so that becomes the output of which they understood because the input to the response generating neural network and then the response generating neural network will give a certain response in text because at the end of the day it is a chatbot and that text has been read, judged and rated by humans, so that this is equivalent to school teachers in schools and then human judges will basically say no, that's an illegal answer, like you can't, sometimes it's completely wrong and then the human says that's a wrong answer. , uh, like I don't know, one plus one isn't. three is two like humans are actually telling the chegebt that you are wrong and then once it corrects itself and there are also other cases where Chacha PT would give an answer on how to kill another human being and then the judge human will be like you.

I don't have any more good, so you can't give these types of answers, reject, so with this type of supervision, the response generation neural network is basically, they are able to learn ethics, learn morals, everything and start to act like a human. -as answers, so it is important to understand that there are two, generally two neural networks operating within the chatpt. Okay, the current state of GPT chat for unsupervised learning, understanding the input part that the neural network takes, like I said everything. the text Internet data until the year 2021 and to train that neural network by dumping all these blog posts and all the texts, it takes about a year, that's crazy, basically you have this training program that adjusts the connections and activations of all the neurons in this neural network that understand that they are supposed to understand the input incessantly 24/7, running on a large number of GPUs to cut it over the course of a year and then the real response that generates the part where the human judges are involved and that takes about six months just working humans I guess humans work from nine to five in that case, right, they've said because humans get tired, and then that takes about of six months, once these two parts of the neural networks are trained and complete, right, uh, it serves the customers who are the users of the chatgpt website.

We literally go there and just chat. It serves customers for a period of time until a new version is released, so they are working on gpt4 and I am not. I know how long GPT g54 I guess they will follow the same pattern so there are neural networks to understand the context of the input chat and then there are neural networks to generate the response and I don't know how many. neurons and how many connections they are going to use, they say they are going to use a lot more, by the way, the more connections and the more neurons you use in the simulated neural networks of these programs, the smarter the AI ​​becomes compared to how Many, the number of neurons and connections that humans have in our type of organic cells, we have many more, many more.

I heard something like we have like ten thousand ten thousand and one thousand ten thousand more, the number of neurons anyway is much more than what we simulate right now with GPT chat, but it is supposed to be chatgpt4, which is the next version , it has many more neurons and connections, so it is supposed to be more sophisticated, so between launches these neural networks are actually trained. which are written in stone and they are fixed, they don't change, so let's compare chatgpt and humans now that we understand what is really involved in simulating our brain and creating this artificial intelligence, we can start to see the limitations and uh, what is good and what is not good for humans needs to pass about 25 years to have a fully developed brain, which means more or less that theactivation of neurons and the connections it makes with other neurons stabilize just when.

Take a child as an example, the disconnection connection and all these changes that are happening in these neurons are very fast and change radically all the time. There is even a case where a cat had an accident and the surgeon removed half of the cat's brain because it had to be cut out and the rest of the cat's brain because it was a small kitten like a small baby, the kitten's brain was able to completely reorganize itself. to take responsibility for the other missing half of the brain with the you know half of the brain surviving, that's amazing, but they took the same case with an adult cat where the adult cat got into an accident and we had to remove a part of the brain to As far as you know, hopefully the cat survives and the mature cat's brain couldn't reorganize and died properly, so for humans it takes about 25 years.

Neurons are always changing in our human being. The body, even after it is fully developed, we constantly have these minor adjustments, so it is very fluid, our neurons disconnect, connect and change their firing behaviors and as always, it changes every second that we live and our neurons are completely autonomous and super energy efficient, which means. Does each individual neuron decide to make the necessary changes? The changes necessary for itself without any kind of governing body. You will receive an external stimulus. Some kind of signal will come and it'll be like, "Oh, I should unplug this one now, but." All of that decision making is done at the cellular level and because it's organic, the hardware neurons are just cells, so it's super energy efficient, like there are electrical signals flowing through my body right now. , but these are very small electrical signals and the transfer of this. electricity is very, very efficient in our body because our body is probably made of liquid and stuff, I don't know, and all you have to do is, I don't know, just bite into a potato, eat it, and then it fuels your body. um and then your body just runs on low energy, you know, uh, but chat with GPT, on the other hand, it takes 1.5 years to be trained and released as a version which is like an advantage compared to the human being . spend 25 years um AI right now chatgpt 1.5 years will get faster and faster but for now it's 1.5 years if you compare 1.5 years to 25 years of course you only want to spend 1.5 years and that it's because you know that's the advantage of simulation, right, it's not like that, you don't have to live your whole life, you can just simulate it and then just quickly run it on your pro in a computer program and chat, repeat, it doesn't change your network neuronal once it is built, unlike humans.

They are real brains and there is a lot of research going on right now to make that change and make it less rigid, so between the releases, the Chachi PT version, let's say one and then version two, between the releases right now, It's fixed, but. then during the course of chatgpt1 being used by users, maybe we don't want to restructure everything too much, but if the user, upon entering the conversation, says you're wrong, you know you're wrong, actually this is the right way . and then the user gives the feedback and then it becomes a feedback for training.

What it is now is a fixed version of chatgpt, but you'll be able to pick it up and make minor changes to your neural networks and that's being investigated right now, but for now, you can assume that the vast majority of that neural network is more or less fixed. and you just have to wait for the next version so that the AI ​​becomes more and more intelligent and the BT charger, of course, you have seen it a lot these days. A lot of people are using it and the website is constantly down and we are slow, that means it requires huge amounts of electricity to run these servers and it requires a lot of GPUs to train the neural networks and all of these things consume a lot of power.

At the moment, I think AI is consuming a lot of electricity. I wouldn't be surprised if they spent, I don't know, 30% to 40% of an entire US state's electricity consumption on AI in some technology-driven states or even more. percentage that's just huge, so yeah, that's basically it, what that means is, in a nutshell, current AI is very rigid, just because of the way the simulated neurons work right now, it's very fixed and rigid, it consumes a large amount of energy, so, as humans, having this brain of ours, like the hardware, what is under the hood, we have a clear advantage that we are, we can be autonomous and independent intelligent beings that do not have than being connected to a massive server, you can dump someone. to a desert island and that person will find a way out, using their intelligence to survive, make decisions, face problems, solve problems in a remote location without being connected to anything else, if that person's brain needs to work, just take a potato and eat it right and you get energy and then you just function, so if you want to send an exploration team to outer space, I don't know, humans will probably be very adaptable and can run on very low energy fuels and can be very intelligent. decisions um but maybe that will change in the future like the Prometheus movie but anyway for now those are the big differences and hopefully this video can encourage some people who thought chat was superior like some, I don't know some young computer scientists.

Out there those who generally got excited about jtbt and I started researching about jtbt and I was overwhelmed by the scientific articles and the mathematics and all this, hopefully this video really made it simple, and then encouraged some young people to give the next step in this presentation. To start learning about all the details that I explained in the high level, go deeper, learn the mathematics and everything related to neural networks and you know, improve our lives with AI, thank you.