Python Tutorial: Working with JSON Data using the json Module

Hello, how are you all doing? In this video we will learn how to work with JSON

data

in Python. If you have never used JSON, it is basically a very common

data

format for storing information and you will surely come across this. At some point, if you haven't already, you'll see that JSON is used a lot when getting data from online APIs, but it's also used for configuration files and different types of data that can be saved on your local machine, so which JSON means JavaScript object. notation, but don't get caught up in the JavaScript part of the name, it was inspired by JavaScript but is now independent of any language, so virtually every language today has libraries for parsing and generating JSON data, so let's go ahead and let's start. and see what this looks like in Python.

First of all, I imported the

json

library here at the top of the file and this is part of the standard library, so there is no need to install anything and I have a multi-line string here. that's valid JSON and you can see it almost looks like a Python dictionary, so this JSON here has a key called people and the people value is an array of more objects, so in this case it's just two more objects and each object has a name key a phone emails key and it has license which is just a boolean value of true or false so right now this is just a

python

string that happens to be valid JSON so let's see how to load this into a

python

object so that we can work with the data more easily, so to load this in python from a string we can use

json

dot load s method and to do this we can say that data is equal to json dot load s and we want to pass that string to that load s function, so I'm going to save it and now let's print this, so I'm just going to print the data and run it okay so we can see that it prints it, but it's not very clean, it's a little bit clustered, but It looks like a Python dictionary. so if I check the type of this data variable and just print the data type and save it and run it then we can see that it is a dictionary so when we load JSON into a Python object it uses the following conversion table , let me extract this here to convert this to a Python object, so when it parses that string, if it finds a JSON object, it converts it to a Python dictionary, converts JSON arrays to a Python list, a string to a string of Python, integers in integers and these real ones. numbers and floats, the true values ​​you can see are converted to uppercase, true to uppercase, false and null values ​​are converted to a Python none value, this is how these conversions happen, so now let me open a copy of security of our Python, so if I print this data variable again, then I run this and we can see that locate shows that we have a boolean value here and you can see that it is now in upper case and we also had a null value here for the second objects sent by email as null and I can see that that was converted to none, so those conversions took place, since our top level key here is an array of values ​​which should now equal a list, since this is a dictionary, we should be able to access this people key and that should be a list, so let's say we print the type and access the people key of that dictionary, so if I save that and print it, then you can see that it is a list, so now that I've converted that JSON to Python objects, it's going to be a lot easier to work with, so now we can loop through all of these people and access each one individually, since it's a list, I could say person in data and then access it. people key at that time for each of them, we could just print the person, so if I save it and run it, you can see that it ran each of these people individually and given that each of these people in our original JSON They are objects in themselves. then those should have been converted to dictionaries as well, so now we can access those values ​​inside our loop, so if we want to access the name of each person, then we can come here inside our loop and say person and then access that variable by name, so If I save that and run it, you can see that we are now accessing the names of each person in that JSON file.

Okay, great, now that we've seen how to load a JSON string into a Python object, let's now do the opposite. of that and dump a Python object into a JSON string and to do this we will use the json dot dump s method, so in our example let's say we want to remove the phone numbers of each person and then convert them back to a json. string to delete the phone number key and the value of each person inside our loop, we can just delete that value, so inside our loop here we can just say delft or delete person and then access that phone key and that will remove the key and value. from that dictionary, so now that we've removed those phone numbers from those objects, we dump all of that back into a string

using

that dump method, so I'll say the new string is equal to and is the JSON dump and we want to dump it. this data dictionary we just modified, so now let's print this, so I'll print that new string, save it and run it.

Let me make this a little bit larger here so you can see that we have a new JSON string that no longer contains. each person's phone numbers now, since this is a string, it would be nice if we could format this in a way that is easier to read, to do this we can pass an indent argument to our dump function to indent this correctly so we can say that when we dump this we can pass indent is equal to and then the number of indents per element in the string, so if we save that and run it, we can see that for each level it indents it twice. so this nested level ends at x and then this one at 6 see exactly what is happening now.

Another thing you can do to clean up your json when you dump it into a string is sort the keys, so if we pass an argument sortkeys equals true, then at the end here are the sort keys under underscore. I will set it to true. If we save it and run it, we can see that now all these keys are sorted alphabetically, where the email comes first and then it has the license and then the name which may or may not be what you want, but you have the option there if you want it. Now that we've seen how to convert strings to Python objects and vice versa, let's now look at how to load JSON files into Python objects and then write those objects.

Let's go back to the JSON files, so I'm going to go ahead and delete everything we have so far except our imports and save it and delete it. Now I have a JSON file and the same directory as my Python script here and this one is called States JSON and this is a JSON file with a list of all the US states, their abbreviations and their area codes, so if you would like to load this file into a Python object, then we can use the JSON load method, so remember that the load method loads. a file in a Python object and the load s method that we saw before loads a string, so to load this file we first have to open it so we can open this file with a with statement so we can say with open and that file is in the same directory so we don't have to pass a path, it's just called States dot JSON and we're just going to read it, so we'll leave it as default and just say like F and now with this file open, we can load it into a Python object saying that the data equals json dot load and load that file now if you've never worked with file objects before and you don't really know what's going on here with this. open statement and what we're doing there, so I have a separate video on how to work with file objects if you're interested and I'll leave a link to that in the description section below, okay, now that we've loaded this JSON file into an object Python, we should be able to loop through our data like we did before.

If I look back at the JSON file, we can see that there's a state key here and then that's a list of objects and all of these objects. I have a name abbreviation and an area code so if I go back to my list to loop through them remember it was a state key so we can say state in data then access the key of that state and then for each of them, let's print the entire file. object and that object we just called state for the state variable, so if I save it and run it, it should print that object for each state and now that this is a Python object, it's very easy to access this information and from any way you want. you want for example if you want to print only the name and abbreviation then you can say the name of the state then we will put a comma and also print the abbreviation of the state so copy and paste there save it and run it and you can see that now we have the state and shorthand access of that JSON file.

Okay, now that we have loaded this JSON from a file, let's write this Python object to a JSON file like we did and the smaller example just remove one of the keys from the data and then write it to a new file, so in this example let's go ahead and just remove the area codes, as we saw a moment ago, to remove them from a dictionary, we can just say del and then what. we want to delete, so we'll delete, let's see, I think they're area underscore codes, yeah, I'm just going to copy and paste them so I don't write anything wrong, so save it, so now we've deleted the area codes from that data. and now to write this file as JSON we will use JSON dump method, so just like with load and load s method, dump method converts the data to a JSON file and dump method converts the data to a JSON string like so as we saw before, so first we have to open the file that we want to write to, just like before, when we read the data, we will say with open and we can call it whatever we want, so I will call this new underscore States dot.

JSON and we want to write this file so we need to pass a W as an argument there and we'll just say like F and now we can dump that data to the file by just saying json dot dump and at first we want to pass in the data that we want to dump so that's this data variable here and then what we want to dump this to, in this case we want to dump this to the file that we just opened, which is this variable F, so if I save that and run it now, let me open my sidebar here and here, in the same directory, we can see that it created this new underscore states JSON, so if we open this we can see that it's kind of compressed, but we have the states key here and then. within this list, we've named Alabama as a short summary of Asian Alabama and then it goes straight to Alaska, so we removed those phone numbers in there to do what we wanted, but it's not really easy to read, so as we did with the dump s method we can also pass an indent argument to our dump method here to clean this up a little bit, so if I say indent is equal to 2 and save it and we run it if we open a backup copy of the JSON file of our new state here, so As I can see, it's much easier to read now.

Now that we've seen how to work with strings and JSON files and how to go back and forth between Python objects, let's look at a real-world example of

using

JSON data so we can get an idea of ​​how this might be used in practice, so It's quite common for websites to return JSON from their APIs to make it easy to parse, so let's look at what capturing JSON data from a public API would look like and how we could parse it. that data, so I found a Yahoo Finance API that converts US dollars to another currency, so let's see if we can extract this data, convert it, convert the JSON to a Python object and then parse some information, so I already have a Python file here.

To start, to make the request to the Web API, I am using the built-in URL Lib

module

and specifically I am importing open URL from the URL Lib dot request. Now you can also use the requests library to do this if you feel more comfortable using that, so you can see I already have the URL, paste it here in the URL opener function and then we set this source variable here equal to that response and then we use the point reading method. in that response now it will only get the response from the website and right now this will just be a string so if I run this it should print all the content of that website.

You can see it's not very readable. We have new line characters here and things like that, so now let's load this response into a Python object using the JSON

module

, so I'll create a variable here called data and we'll just say JSON dot load s since it's a string and we'll pass thatsource variable there. I'm going to go ahead and get rid of that print statement. Now that this is a Python object, let's immediately dump it back into a string with the indent argument passed so we can clean up. Turn this up a little bit and see a little bit more what it looks like, so I'm just going to print json dot dump s and we want to discard that Python object that we just created, but we're going to set an indent equal to two, so if you save it and run it, then we can see that we have some clean JSON data here and I'll scroll to the top if I can, okay, we can see we have some JSON data so we can see the first key. is the list and then inside the list we have this meta key which has some metadata so we can see here it says the count is 188 elements and also inside this list we have resources and inside resources there is another list here and inside from that list. we only have the individual resources that have the conversion information, so we have another resource key, then we can see that it has fields and this is where we get to know the name of the conversion and the price of that conversion.

Okay, so let's start

working

with this data so first of all, let's see if the count of 188 is correct, so let's take a look, we have this list and then we have these resources, so let's see if there are actually 188 resources there for be able to print. I'm going to comment on our printout. JSON for now, so I'm just going to print the length of that data object and we're going to access that list and within that list we're going to access resources and to not misspell, I'm going to copy that and paste it in there, so let me save it and run it and you can see that we got 188 resources in that list, so that's correct.

Okay, now let's loop through all of those resources so we can say the element in the data. and I should have copied and pasted that, but okay, it's inside the list and then access the resources, so for the item in that list, just print each of those items, okay, so it's a little bit difficult to see because my text is a little big here, but we can see that each of these elements has a resource key and inside that resource key we have a class name and then the fields and inside those fields that's where we have the name and the price, so let's say I wanted to print all the conversion names and all the prices, so to do this we can go up here and delete this line.

I'm just going to say the name is the same as our article and then dig deeper into this article. We're going to access this resource key and then inside there we're going to access the fields and then inside the fields we're going to access the name, so now I'm going to save it and paste it here because now we want to access it. the price, so I'm going to say that the price is equal to all of that same information, but now instead of the name in those fields, we want to access the price in those fields and this may seem a little complicated here, but you'll end up doing this a lot when you work with JSON data some of this JSON data when you're

working

you know with real world examples you have to dig a little deeper until you get exactly what you're looking for so now we should have the name and price of those conversions so if I print name and price, I save them and run them, now we have the name of each conversion and then the actual price of that conversion, okay, now let's use these conversion rates to convert US dollars to specific currency, so I'm going to create a new dictionary that has all of these names as a key and all of the conversion rates as a value so we can access them quickly, so above our for loop here I'm just going to create a really empty dictionary, so I'm going to say call to this USD rates and set it to an empty dictionary there and then inside our loop we can add each of these names as a key saying USD rates and we'll use the name as the key and then the values ​​will be the prices, like this that we will set the price there and save it.

Oh, and it looks like I actually have a typo here. These are supposed to be USD rates, not our rate. Okay, now that we have that. New dictionary setup, we can access a specific value by just accessing that key so we can get the USD to Euro rate by just printing let's say just print the USD rate and then we want to access the USD to Euro key so let's save that and run it and we can see that this says that the USD is equal to approximately zero point eight four six five euros now these conversions here I just have them written down so it's not like I've memorized them if you pull this data out yourself so you can see any conversion you want.

Okay, now let's do a conversion. If we wanted to convert, say, 50 US dollars to euros, then we could say 50 times and now this is actually a float value, but it comes as a string right now, so let's convert it to a float and then we can save it and run it like this that 50 US dollars would be right now about forty-two point three to five euros and changed this conversion, you could just pass it on. a different currency, so let's say we want to convert to Indian rupees, then that would simply be USD to I in R, so if we save that and run it, we can see that the conversion is about 50 USD to three thousand two hundred and forty.

Now if you wanted to, you could write a script that pulls this conversion information from the API once a day and then save it to a local JSON file for faster access and then write a function that accesses this information in a more repeatable way . We can see how understanding how to parse and work with this JSON information can be extremely beneficial for capturing data from the internet and using it to meet our needs and there are countless other JSON APIs available and it is essential to know how to work with them. this information if you want to get data from external sources, that's fine, so I think it will be enough for this video.

Hopefully you now understand how to work with JSON data and convert between it and Python objects, but if anyone has any. If you have questions about what we covered in this video, feel free to ask them in the comments section below and I'll do my best to answer them. If you enjoyed these

tutorial

s and would like to support them, there are several ways you can do so. The easiest ways to just like the video and give it a thumbs up and it's also very helpful to share these videos with anyone you think will find them useful and if you have the means you can contribute via Patreon and there is a link to that page. in the description section below be sure to subscribe for future videos and thank you all for watching.