Python Tutorial: Generators - How to use them and the benefits you receive

Hello, how is everyone doing in this Python video? We're going to go over

generators

and why you would want to use

them

and some of the advantages they have over lists, so in this example I have this function here called square numbers. and what it does is it takes a list of numbers and then we have this result variable that is set to an empty list and then we loop through all the y numbers from the list that we passed in and add the square of that number. to the results list and after we have finished looping through all the numbers, we return the result and then you can see here.

I have this variable times numbers equal to square numbers. I'm going through a list of one, two, three, four. five and then I just print or I just print my numbers here, so if I run this code, you can see that our list of one, two, three, four five that passed in our result is 1 4 9 16 25, so currently our square numbers function is returning a list now, how would we convert this to be a generator? To do this, we will no longer need this result variable so we can delete it. We don't need the return statement and this attached result point. we can take this out and all we have to do is type this yield keyword and just generate the square number here, so this yield keyword is what makes it a generator now, if I save this and run it, You will be able to see it now every time I print my numbers, I don't get the list anymore if you look at the comment here, this is the result it used to be.

I no longer get 1 4 9 16 25. Squares one through five I no longer get that result. this generator object here now the reason for this is because

generators

don't hold the entire result in memory, they produce one result at a time, so it's actually waiting for us to request the next result, so it hasn't actually computed nothing. however, now if I print below my numbers that ask for the next result, then you can see that it is one because we pass our list of one, two, three, four five and then we loop through that list, so one is the first value . so it's equal to i here and we got one by one and it gave us that result, so now if we copy this line here and print the following mynums several times here and run it, then you can see that every time we run the next it goes and gets the next value obtained, so now we have 1 squared 2 squared, which is 4 3 squared versus 9 16 25 and so on, now 25 is the last value of our result, so what would happen if I ran the following once further?

Well if I do that you can see I got an error here and the exception it threw was stop the iteration and that means the whole generator has been exhausted and stopping the iteration just means it no longer has values ​​instead of getting these values ​​one . at the same time we can still use a for loop in these generators and that's personally how I use generators most of the time, so let me comment out this line and then uncomment that one and save it, so now we say for num and my nums, that my nums is our generator, I'm going to print num, so I'm going to run it and you can see that we get all of our values ​​and we don't get the iteration stop exception because the for loop knows when to stop before that.

This happens, so an immediate advantage over a list is that I think this is much more readable here rather than having the result set to an empty list and then adding it to that result and then returning the result. This is more readable, we're saying. Okay, I'm passing these numbers for each number and that list of numbers returns the result now. For those of you more familiar with Python, you may have noticed that this entire process here from these lines of code would have been much easier to write. a list comprehension, so let me talk about it and if you don't know what a list comprehension is, don't worry too much.

I just want to show the generator example with this as well. Now this is a list comprehension here and it's going to do exactly what our square number function did, so what we're doing is we're creating a list and we're taking x multiplied by x, so the square of x for x in this list of one, two, three, four five, so if you save this here and run the code. You can see I'm still getting the same results and I can also print this list here at the top. Now you can create a generator the same way and it's as easy as removing these square brackets. and instead of putting in parentheses, so if I remove those brackets, I put

them

in parentheses now, if I run this, you can see that when I printed my numbers here I tried to print them all at once, I got that generator object and then when I ran my for loop looped through all the values ​​and gave me that result, and what if you really wanted to print all the values ​​from the generator?

Well, like I said, they're not all currently saved in memory, but you can turn them into a list and it's as easy as just putting the list in and then wrapping it and then if I run it, you can see that it ran it and it printed it as if was a list now when you convert this generator to a list then you lose the advantages that you got in terms of performance and I haven't talked about performance yet but I have a better example to show those advantages so a generator is better with performance because, like I said, it doesn't hold all the values ​​in memory, which is no big deal as long as you have a small list like this of one, two, three, four five, but let's say you had tens of thousands or even millions of elements to loop through, then having that many elements in memory will definitely be noticeable, but you don't get that with generators, so whenever you cast a generator to a list like this, if this generator had a lot of values ​​that it needed to convert to that list , then you lose that performance in terms of what I would put in all those values. in memory, so let me show you a better example here of this performance difference, so I have a file here where, um, some of these things you don't have to worry about, like these lines here, I'm just printing the memory and then these. names and specialties only these will be used to create some random values ​​so I have two different functions here one of them will make a list and another one will be a generator and both are returning the same values ​​so inside this list I have my result here and I'm looping through a number of people that I'm going to pass to this function and for each person I'm just going to recreate a people dictionary. an id and a name that is chosen randomly from the list of names at the top and a specialty that is chosen randomly from the list of specialties and then I will return that result and for the generator it is exactly the same.

I'm going to loop through the number of people that I pass and then I'm going to generate this people dictionary that has the same values ​​that the list function had now real quick just to make them equal, I'm going to make that a range x instead so that they are exactly the same, so here don't worry about these lines here, this time point clock and this time point clock t2, all I'm going to do is calculate how long it takes to run this function, which returns a list, now I'm going to pass 1 million values ​​to this function, so it should return a list of 1 million results and then down here I'm printing the memory usage and the total time it took.

If I run this, you can see here at the top of the code, so this is before I do anything, so my base memory usage was around 15 megabytes and this memory after I created that list of 1 million records, so You can see here that it jumped almost 300 megabytes and took about 1.2 seconds. If you're dealing with large amounts of data, you know it's not out of the ordinary to have a million records like that, so let's see what it looks like. like I used the generator example instead, so I'm going to comment out the function that returned the list and now I'm going to uncomment this function that returns a generator and I'm going to pass the same number of values ​​I'm going to pass 1 million values ​​here , so if I save that and run it now, you can see here after running this that the memory is almost exactly the same and that's because the generator hasn't actually done anything. however it doesn't keep those millions of values ​​in memory, it's waiting for me to grab the next one or loop through them and it would give them to me one at a time, this time it grabbed here it basically didn't take any time because as soon as it gets to the first yield statement, it stops, so if I instead did this as an integer, it would be almost zero seconds now.

Every time I said before that if you convert this to a list, then you lose that performance, so let me show you. You know what I mean here, so I'm going to convert this result, this entire result into a list and now if I run this, you can see basically that I got pretty much the same result that I got when I ran the function that returned the list, like this that if you remove them again and just do the generator, then you can see that we get our performance back, that's how you use a generator.

I think it's a little more readable and also gives you big performance gains, not just with execution. time but also with memory and you can still use all the understandings and this generating expression here so you don't lose anything in that area. Those are some of the reasons why you would use generators and also some of the advantages that come with them. With that, I hope this video was helpful to you. If you have any questions regarding this topic, just ask in the comments section below. Be sure to subscribe for future Python videos and thank you all for watching.