YTread Logo
YTread Logo

Python Tutorial: Real World Example - Parsing Names From a CSV to an HTML List

Jun 02, 2021
Hello, how are you all doing? In this video, we'll look at a

real

-

world

problem I ran into and explain how to write a Python script to solve it, so I've made videos like this. before and everyone seemed to find them useful now, the difference between these videos and my normal videos is that I'm not going to go into so much step by step detail of each little step, I'm just going to explain how I came up with it. a solution and you can follow it, so this is what I want to write a script for, so some of you may not know, but for anyone who contributes to this channel through Patreon, I

list

everyone on the page of contributors to my website as a small way of saying thank you, well the problem I'm running into and it's a big problem is that the contributors are increasing in numbers where it's hard to keep track of who I've added to the site and who does not.
python tutorial real world example   parsing names from a csv to an html list
I want to automate this process with Python so I don't miss anyone and luckily Patreon provides a downloadable CSV file from all contributors that will make it easy to automate this process so if you don't know what CSV files are. means comma separated values ​​and basically CSV files allow us to put data in a plain text file and use some kind of delimiter which is usually a comma to separate the different fields, so in this video we will learn how to use the CSV module . to parse the CSV file, count the contributors and then put their

names

into an unordered HTML

list

that I can place on my website, so let's go ahead and start now, first of all.
python tutorial real world example   parsing names from a csv to an html list

More Interesting Facts About,

python tutorial real world example parsing names from a csv to an html list...

I don't want to expose anyone's information here, so the CSV file I'm going to use for this video. It removes all of everyone's personal information and just has fake

names

instead of

real

names, but other than the names being fake, this is almost identical to the file I downloaded from Patreon, so I'm going to open this file called pattern . CSV and when I first open this file, there are a couple of things that jump out at me when I first see it, so first of all, our first row is our headers and we can see that it says that the information in this file is it's going to be first name last name email commitment lifetime status country and start now I don't really know what all those fields mean, but I'm basically just concerned with first and last name, so that's fine.
python tutorial real world example   parsing names from a csv to an html list
I also noticed that there are a couple of lines here after the header that are not actual data, it's just a line explaining, the people below this line are the ones who have said they don't mind appearing on the website as contributors and then it looks like real people start on line five here now on Patreon, you can also opt out of rewards, so there's probably a line in this CSV file that's a cutoff for people who said they just want contribute but do not want to appear on the list. on the website and in fact, if we look here at line 35, we can see that breakpoint where it says that the people listed below this point don't want the reward and they don't want to be listed on the website.
python tutorial real world example   parsing names from a csv to an html list
Okay, now We have a basic idea of ​​the data that we want to capture, so now let's go ahead and start encoding this, so in the same directory I have a blank file here called parse CSV dopy and I'm going to open it first thing now. What I'm going to do is import the CSV module and you may have looked at that CSV file and thought that doesn't look hard to parse, so why not use the split method on each line of the file to get that information? and it's true that you can do that, but the CSV module makes

parsing

these files much easier, so for

example

, if someone puts a comma or something in their name for some reason, we wouldn't want to split that and also the CSV.
The module will handle new lines and all that and it just takes all the guesswork out of working with things like this, so we'll use the CSV module. Okay now I know that my end goal is to generate an unordered HTML list so I'm going to create an HTML output variable and I'm going to set this to an empty string for now and we'll fill it in as we go and I also know that I want to capture all the names of all the people I want to add to that output. so I'm going to create an empty list of names.
Okay, now let's open our CSV file just like we would open any other file, so we'll use a context manager here and we'll say with open and this is called a pattern. CSV and we want to read this file, so we'll pass an R in there and we'll just call it data file. Now I will show you two different ways to parse the CSV file. you are the most common and then I'll show you my preferred method so the first way we're going to do it is with a CSV V reader, so I'll say CSV data equals csvreader and now we'll pass that data file. and in fact, let me enlarge this text a little bit more so that everyone can see it as we go along.
I think it's better, since the read method should have parsed the CSV file and put the data into our CSV data variable, so let's print. what we have so far to make sure it looks good, so I'm going to go down a couple of lines here and print the CSV data and run it fine, so now we're just going to get this CSV reader object. You may have done it. waiting for all our CSV data now the data is there but this object is iterable and it behaves like a generator and what that means is that we have to loop through it to get each line so you can do it line by line or you can just convert it to a list and get all that data at once, so if we convert this to a list and print it, we can see that it prints a lot of information here in list form.
Now it's not the easiest to read, but it looks like our data is a good start, so now let's print it line by line so we can see it a little better. To do this, we can say for line in CSV data and now we're just going to print. each line and we run that and when we run that we can see that it's a lot easier to read so if we scroll up and look at the first two lines we can see that the first line has the headers and I don't really need them other than for know what index each field is in, so the first name is at index zero and the last name is at index one and it looks like the second line is the line that tells us which names we want to put on the website and then the third line is the first person with the name John, so we don't really need these first two lines here, we just want to get the names of the people, so if anyone has seen my video on generators.
We can actually step over the values ​​and an iterable by calling next, so let's call next on our CSV data twice before printing this loop, so we'll just say next CSV data, then copy and paste them again now that we don't do it. We need to capture the result of these and any variables, we just want to discard them, so now if we run this and scroll up, we can see that now our first line is John Doe's first person, um, okay, cool. now let's remove our print statement before we do that, it's not obvious why we run the following twice with the CSV data here, so it's important to comment on non-obvious things like this, as we go along, not only for other people, but for you as well , so you can come back to this code in a few weeks and have no idea why we ran these two lines here, so let's go ahead and make a comment that says: we don't want headers or first line. of bad data, okay, inside our loop here.
We are looping through each person in the CSV file. Now remember that first name is index zero and last name is index one, so let's go ahead and add each name to our list of names. We created at the top, now to do this we will say names. add now we want to add a string from the name space last name and to do this I'm just going to use a string F and then these curly braces for a placeholder and we'll say line and then zero index for the name, then a space, then another square bracket uh for the placeholder and then an index of one for the last name, now like I said, if you've never seen a string with an F in front like this, this is called an F string and they're new in Python 3.6, so if you're not using 3.6 or later this won't work for you, you'll have to use a normal uh string. format now I'm really liking these F strings so far and basically it's a much simpler way to format strings, so if you want to see more about them, you can check out my video on strings where I go into more depth. all different ways to format strings, but basically all we're saying here is that we want a string with the value at index zero of the line, which is the first name and then a space and then the value at index one of the line. what's the last name, okay and now that we've added them, let's print all the names that were added to that list, since it's a global variable, we can print it out of our context manager down here, so Let's go down to the end and we'll say name and names and then we'll just print the name, okay, so this looks good.
It looks like we now have first and last names if we scroll. Through our names here, we can see that one stands out and that is that it has no reward value, so if you remember that there are names on this list that did not want to be included, so all the names after this have no reward value. reward. here, uh, it shouldn't be added to our list. If we look back at our original CSV file, here we can see that this no reward line has a comma after no reward, so it should be parsed as a name, so let's add a tick. for a non-reward name and then we'll exit our loop as soon as we see that value.
So inside our loop here, we're going to say that if index zero, which should be the first name, is equal to no reward, then we're I'm going to exit that loop now before we run this, we need to keep in mind that the name before of no reward here in our file is Maggie Jefferson, so when we run this again, hopefully this will be the last name in our name list, so I'm going to go ahead and run this again and when we run it again we can see that this fake name down here of Maggie Jefferson is the last name in the list, so that works fine, now that we've tested to make sure that our names are right, we can go ahead and just delete that list where we're printing or that loop where we are printing all the names.
Now we're pretty close to finishing here, so the hard part is over. Now we just have to get started. these names in an unordered HTML list so we can place them on the site, so first on the site I would like to list how many followers there are, so first we will add it to our HTML output with paragraph tags and count how many There are many, uh, we can use the length of our list of names, so I'm going to say HTML output uh plus equals because we want to add to this, so I'm just going to use another format string here, um, to put these in. the values ​​are fine, so like I said, I used another F string here and we're using these HTML paragraph tags here that we're adding now, the only Python data I'm adding is the length of the names of those names. list so when this prints you need to substitute the actual number in there so just to make sure let's go ahead and print this HTML output so print the HTML output and I'll run it fine so apparently there are 30 people on the list of that name, so now let's create our unordered HTML list with each name, so we'll add an unordered list to our HTML output and I'll do this above where we're printing that output, so I'll say HTML output plus equals and then an unordered list. in Python now I'm going to put a new line in there first, an unordered list is this UL tag.
Now that new line I added will make it much easier to read when we actually print this, so now let's loop it. all our names and add each one to an HTML list element, um, so if you're not familiar with HTML, then don't worry too much, it's more about the process of automating this process that we're looking for here, so uh, here we'll say name and names and now we want to add each one to that HTML output, so we'll say HTML output plus equals, and this is another FST string here, so first we're going to put in a new line. with a post sln and then we put a tab with the post SLT and then the list item is this Li tag here and then we put the name.
This is the Python variable um that we're using and it's going to replace. this and then in HTML these Ford forward slashes close an HTML element, so we'll close that list element. Now, after we have all of those list elements out of our loop, we'll need to close the entire list. together, then we'll say HTML output plus equal and we'll close that list element with a slul forge and let's not forget to put a new line in there at the beginning just to clean up how it prints, okay,so now let's print. Pull this out and see what everything looks like, so I'm going to increase our output here, okay this looks good, so at this point I think this is the exact output that we wanted, so at this point we could be done, but I wanted to show one more thing, um, I told you that I would show you one more way to parse the CSV file which I prefer more than using the read method and what I prefer is to use the dictionary reader and we can use this by saying dict reader.
Now the difference between the reader and the dictionary reader is that the dictionary reader converts each line into a dictionary instead of a list and the dictionary has each field as a key and then the data as values, so let me loop through it and print it. these values ​​so you can see what this looks like, so first let me comment out these lines where we're doing all of our loops and everything and I'm also going to comment out the HTML output for now and we'll see what the output is. This dictionary reader looks and to do this we'll say for the element in CSV data and then we'll just print each element, so run it right now, at first glance, this looks a little messy, especially since my text is very large here. but each of these lines is a sorted dictionary, so the first line with the field names is no longer here.
They are now being used as keys for dictionaries, so the first throwaway value is still here as far as the reward description is concerned. line, so this first order dictionary here is our first line and if we look at our second element now, this is our first person because we can see that this is John's person, so now instead of accessing index zero for the first name and index one. for last name we can now access them directly through the first name and last name keys and I think that's much more readable to anyone looking at your code, so now to make this work again, let's get rid of this loop where we just printed everything and now we're going to remove all the logic here, okay, and now that the headers are no longer included in the output, we just want to skip that first value and let's see and We can fix our comment here and just say we don't want the first bad data line and now instead of using the index where we access the elements we can now use the first name and last name keys so now we are going to say if first name equals no reward and then we want to add the first name and add the last name, okay and now let's go down here and extract this HTML output and see if it works, so now I'm going to go ahead and run this again. and if we scroll up we have Maggie Jefferson here at the bottom and if we scroll up we can see that there are still 30 contributors and John Do is the first one, so that seems to be correct, so it seems that our results are good. so it took me a little while to write the script, but now it's going to save me a lot of time automating it in the future and it's also going to keep me from making mistakes, so one of the reasons I like to show you all these quick scripts that can automate a repetitive task is just to show how you can save a lot of time by writing a very simple script.
I mean, this script here is only 26 lines and it will save us a lot of time and I want to show that too. There's no need to think too much about these one-off scripts, so I could probably go into the script and add error checking and also some kind of object-oriented approach to this, but what I want to use the script for I don't really need it. It's too complicated, so if you have some problems that you think you can automate, then give it a try and don't think too much about performance or cleanliness of everything.
It's just a great way to learn, just by doing and experimenting Now, if any of you are interested in a more detailed look at how to parse CSV files and write CSV files, then I'm putting together a video specifically on reading and writing CSV files that I'll be recording very soon, so stay tuned. I'm on the lookout for that, okay, so I think I'll do it for this video. I hope everyone found it helpful and if anyone has any questions about what we covered in this video, feel free to ask in the comments section below. I will do my best to answer them and if you enjoy these

tutorial

s and would like to support them there are several ways to do so, the easiest way is to simply like the video and give it a thumbs up, and it is also a great help to share these videos with anyone who you think will find them useful and if you have the means you can contribute via patreon and there is a link to that page in the description section below be sure to subscribe for future videos and thank you all for watching

If you have any copyright issue, please Contact