{"id":28998634,"url":"https://github.com/luminati-io/parse-json-in-python","last_synced_at":"2026-04-28T15:39:42.529Z","repository":{"id":272769960,"uuid":"917696844","full_name":"luminati-io/Parse-json-in-python","owner":"luminati-io","description":"Learn to convert JSON to Python dictionaries and objects, use API responses, handle files, and explore limitations.","archived":false,"fork":false,"pushed_at":"2025-01-16T13:44:07.000Z","size":12,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-03-22T07:02:02.490Z","etag":null,"topics":["json","proxy","python","web-scraper-api","web-scraping"],"latest_commit_sha":null,"homepage":"https://brightdata.com/blog/how-tos/parse-json-data-with-python","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/luminati-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-16T13:33:49.000Z","updated_at":"2025-01-16T13:44:09.000Z","dependencies_parsed_at":"2025-01-16T15:18:08.980Z","dependency_job_id":null,"html_url":"https://github.com/luminati-io/Parse-json-in-python","commit_stats":null,"previous_names":["luminati-io/parse-json-in-python"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/luminati-io/Parse-json-in-python","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luminati-io%2FParse-json-in-python","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luminati-io%2FParse-json-in-python/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luminati-io%2FParse-json-in-python/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luminati-io%2FParse-json-in-python/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/luminati-io","download_url":"https://codeload.github.com/luminati-io/Parse-json-in-python/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luminati-io%2FParse-json-in-python/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261823776,"owners_count":23215150,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["json","proxy","python","web-scraper-api","web-scraping"],"created_at":"2025-06-25T07:09:21.847Z","updated_at":"2026-04-28T15:39:37.503Z","avatar_url":"https://github.com/luminati-io.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Parsing JSON Data with Python\n\n[![Promo](https://github.com/luminati-io/LinkedIn-Scraper/raw/main/Proxies%20and%20scrapers%20GitHub%20bonus%20banner.png)](https://brightdata.com/) \n\nThis guide covers parsing JSON data with the `json` module in Python and transforming it into a Python dictionary and vice versa.\n\n- [An Introduction to JSON in Python](#an-introduction-to-json-in-python)\n- [Parsing JSON Data With Python](#parsing-json-data-with-python-1)\n- [Converting a JSON String to a Python Dictionary](#converting-a-json-string-to-a-python-dictionary)\n  - [Transforming a JSON API Response Into a Python Dictionary](#transforming-a-json-api-response-into-a-python-dictionary)\n  - [Loading a JSON File Into a Python Dictionary](#loading-a-json-file-into-a-python-dictionary)\n  - [From JSON Data to Custom Python Object](#from-json-data-to-custom-python-object)\n- [Converting Python Data to JSON](#python-data-to-json)\n- [Limitations of the `json` Standard Module](#limitations-of-the-json-standard-module)\n- [Conclusion](#conclusion)\n\n## An Introduction to JSON in Python\n\nJavaScript Object Notation, or JSON, is a lightweight data-interchange format commonly used for transmitting data between servers and web applications via APIs. JSON data consists of key-value pairs where each key is a string, and each value can be a string, number, boolean, null, array, or object.\n\nHere is an example of JSON:\n\n```json\n{\n  \"name\": \"Maria Smith\",\n  \"age\": 32,\n  \"isMarried\": true,\n  \"hobbies\": [\"reading\", \"jogging\"],\n  \"address\": {\n    \"street\": \"123 Main St\",\n    \"city\": \"San Francisco\",\n    \"state\": \"CA\",\n    \"zip\": \"12345\"\n  },\n  \"phoneNumbers\": [\n    {\n      \"type\": \"home\",\n      \"number\": \"555-555-1234\"\n    },\n    {\n      \"type\": \"work\",\n      \"number\": \"555-555-5678\"\n    }\n  ],\n  \"notes\": null\n}\n```\n\nPython natively supports [JSON](https://docs.python.org/3/library/json.html) through the `json` module, which is part of the [Python Standard Library](https://docs.python.org/3/library/index.html). This means that you do not need to install any additional library to work with JSON in Python. You can import `json` as follows:\n\n```python\nimport json\n```\n\nThe built-in Python `json` library exposes a complete API to deal with JSON. In particular, it has two key functions: [`loads`](https://docs.python.org/3/library/json.html#json.loads) and [`load`](https://docs.python.org/3/library/json.html#json.load). The `loads` function is for parsing JSON data from a string, while the `load` function is for parsing JSON data into bytes.\n\nThrough those two methods, `json` allows you to convert JSON data to equivalent Python objects like [dictionaries](https://docs.python.org/3/tutorial/datastructures.html#dictionaries) and [lists](https://docs.python.org/3/tutorial/datastructures.html#more-on-lists), and vice versa. Plus, the `json` module allows you to create custom encoders and decoders to handle specific data types.\n\n## Parsing JSON Data With Python\n\n## Converting a JSON String to a Python Dictionary\n\nAssume that you have some JSON data stored in a string and you want to convert it to a Python dictionary. This is what the JSON data looks like:\n\n```json\n{\n  \"name\": \"iPear 23\",\n  \"colors\": [\"black\", \"white\", \"red\", \"blue\"],\n  \"price\": 999.99,\n  \"inStock\": true\n}\n```\n\nAnd this is its string representation in Python:\n\n```python\nsmartphone_json = '{\"name\": \"iPear 23\", \"colors\": [\"black\", \"white\", \"red\", \"blue\"], \"price\": 999.99, \"inStock\": true}'\n```\n\n\u003e **Note**\\\n\u003e Consider using the Python triple quotes convention to store long multi-line JSON strings.\n\nYou can verify that `smartphone` contains a valid Python string with the line below:\n\n```python\nprint(type(smartphone))\n```\n\nThis will print:\n\n```\n\u003cclass 'str'\u003e\n```\n\n[`str`](https://docs.python.org/3/library/stdtypes.html#text-sequence-type-str) stands for “string” and means that the smartphone variable has the text sequence type.\n\nParse the JSON string contained in smartphone into a Python dictionary with the json.loads() method as follows:\n\n```python\nimport json\n\n# JSON string\nsmartphone_json = '{\"name\": \"iPear 23\", \"colors\": [\"black\", \"white\", \"red\", \"blue\"], \"price\": 999.99, \"inStock\": true}'\n# from JSON string to Python dict\nsmartphone_dict = json.loads(smartphone_json)\n\n# verify the type of the resulting variable\nprint(type(smartphone_dict)) # dict\n```\n\nIf you run this snippet, you would get:\n\n```\n\u003cclass 'dict'\u003e\n```\n\nNow `smartphone_dict` contains a valid Python dictionary.\n\nNext, pass a valid JSON string to `json.loads()` to convert a JSON string to a Python dictionary.\n\nYou can now access the resulting dictionary fields as usual:\n\n```python\nproduct = smartphone_dict['name'] # smartphone\npriced = smartphone['price'] # 999.99\ncolors = smartphone['colors'] # ['black', 'white', 'red', 'blue']\n```\n\nThe `json.loads()` function will not always return a dictionary. Specifically, the returning data type depends on the input string. For example, if the JSON string contains a flat value, it will be converted to the equivalent Python primitive value:\n\n```python\nimport json\n \njson_string = '15.5'\nfloat_var = json.loads(json_string)\n\nprint(type(float_var)) # \u003cclass 'float'\u003e\n```\n\nSimilarly, a JSON string containing an array list will become a Python list:\n\n```python\nimport json\n \njson_string = '[1, 2, 3]'\nlist_var = json.loads(json_string)\nprint(json_string) # \u003cclass 'list'\u003e\n```\n\nThe [conversion table](https://docs.python.org/3/library/json.html#json-to-py-table) below explains how JSON values are converted to Python data by `json`:\n\n| **JSON Value** | **Python Data** |\n| - | - | - |\n| `string` | `str` |\n| `number (integer)` | `int` |\n| `number (real)` | `float` |\n| `true` | `True` |\n| `false` | `False` |\n| `null` | `None` |\n| `array` | `list` |\n| `object` | `dict` |\n\n### Transforming a JSON API Response Into a Python Dictionary\n\nConsider that you need to make an API and convert its JSON response to a Python dictionary. In the example below, we will call the following API endpoint from the [{JSON} Placeholder](https://jsonplaceholder.typicode.com/) project to get some fake JSON data:\n\n```\nhttps://jsonplaceholder.typicode.com/todos/1\n```\n\nThat RESTFul API returns the JSON response below:\n\n```json\n{\n  \"userId\": 1,\n  \"id\": 1,\n  \"title\": \"delectus aut autem\",\n  \"completed\": false\n}\n```\n\nYou can call that API with the [`urllib`](https://docs.python.org/3/library/urllib.html) module from the Standard Library and convert the resulting JSON to a Python dictionary as follows:\n\n```python\nimport urllib.request\nimport json\n\nurl = \"https://jsonplaceholder.typicode.com/todos/1\"\n\nwith urllib.request.urlopen(url) as response:\n     body_json = response.read()\n\nbody_dict = json.loads(body_json)\nuser_id = body_dict['userId'] # 1\n```\n\n[`urllib.request.urlopen()`](https://docs.python.org/3/library/urllib.request.html#urllib.request.urlopen) peforms the API call and returns an [`HTTPResponse`](https://docs.python.org/3/library/http.client.html#http.client.HTTPResponse) object. Its [`read()`](https://docs.python.org/3/library/http.client.html#http.client.HTTPResponse.read) method is then used to get the response body body\\_json, which contains the API response as a JSON string. Finally, that string can be parsed into a Python dictionary through `json.loads()` as explained earlier.\n\nSimilarly, you can achieve the same result with [`requests:`](https://requests.readthedocs.io/en/latest/).\n\n```python\nimport requests\nimport json\n\nurl = \"https://jsonplaceholder.typicode.com/todos/1\"\nresponse = requests.get(url)\n\nbody_dict = response.json()\nuser_id = body_dict['userId'] # 1\n```\n\n\u003e **Note**\\\n\u003e The [`.json()`](https://requests.readthedocs.io/en/latest/user/quickstart/?highlight=json#json-response-content) method automatically transforms the response object containing JSON data into the respective Python data structure.\n\n### Loading a JSON File Into a Python Dictionary\n\nSuppose you have some JSON data stored in a `smartphone.json` file as below:\n\n```json\n{\n  \"name\": \"iPear 23\",\n  \"colors\": [\"black\", \"white\", \"red\", \"blue\"],\n  \"price\": 999.99,\n  \"inStock\": true,\n  \"dimensions\": {\n    \"width\": 2.82,\n    \"height\": 5.78,\n    \"depth\": 0.30\n  },\n  \"features\": [\n    \"5G\",\n    \"HD display\",\n    \"Dual camera\"\n  ]\n}\n```\n\nYour goal is to read the JSON file and load it into a Python dictionary. Achieve that with the snippet below:\n\n```python\nimport json\n\nwith open('smartphone.json') as file:\n  smartphone_dict = json.load(file)\n\nprint(type(smartphone_dict)) # \u003cclass 'dict'\u003e\nfeatures = smartphone_dict['features'] # ['5G', 'HD display', 'Dual camera']\n```\n\nThe built-in [`open()`](https://docs.python.org/3/library/functions.html#open) library allows you to load a file and get its corresponding [file object](https://docs.python.org/3/glossary.html#term-file-object). The `json.read()` method then deserializes the [text file](https://docs.python.org/3/glossary.html#term-text-file) or [binary file](https://docs.python.org/3/glossary.html#term-binary-file) containing a JSON document to the equivalent Python object. In this case, `smartphone.json` becomes a Python dictionary.\n\n### From JSON Data to Custom Python Object\n\nNow, let's parse some JSON data into a custom Python class. This is what your custom `Smartphone` Python class looks like:\n\n```python\nclass Smartphone:\n    def __init__(self, name, colors, price, in_stock):\n        self.name = name    \n        self.colors = colors\n        self.price = price\n        self.in_stock = in_stock\n```\n\nHere, the goal is to convert the following JSON string to a `Smartphone` instance:\n\n```json\n{\n  \"name\": \"iPear 23 Plus\",\n  \"colors\": [\"black\", \"white\", \"gold\"],\n  \"price\": 1299.99,\n  \"inStock\": false\n}\n```\n\nCreate a custom decoder to accomplish this task. To do that, extend the [`JSONDecoder`](https://docs.python.org/3/library/json.html#json.JSONDecoder) class and set the `object_hook` parameter in the `__init__` method. Assign it with the name of the class method containing the custom parsing logic. In that parsing method, you can use the values contained in the standard dictionary returned by `json.read()` to instantiate a `Smartphone` object.\n\nDefine a custom `SmartphoneDecoder` as below:\n\n```python\nimport json\n \nclass SmartphoneDecoder(json.JSONDecoder):\n    def __init__(self, object_hook=None, *args, **kwargs):\n        # set the custom object_hook method\n        super().__init__(object_hook=self.object_hook, *args, **kwargs)\n\n    # class method containing the \n    # custom parsing logic\n    def object_hook(self, json_dict):\n        new_smartphone = Smartphone(\n            json_dict.get('name'), \n            json_dict.get('colors'), \n            json_dict.get('price'),\n            json_dict.get('inStock'),            \n        )\n\n        return new_smartphone\n```\n\nUse the `get()` method to read the dictionary values within the custom `object_hook()` method. This will ensure that no `KeyError`s are raised if a key is missing from the dictionary. Instead, `None` values will be returned.\n\nNow pass the `SmartphoneDecoder` class to the `cls` parameter in `json.loads()` to convert a JSON string to a `Smartphone` object:\n\n```python\nimport json\n\n# class Smartphone:\n# ...\n\n# class SmartphoneDecoder(json.JSONDecoder): \n# ...\n\nsmartphone_json = '{\"name\": \"iPear 23 Plus\", \"colors\": [\"black\", \"white\", \"gold\"], \"price\": 1299.99, \"inStock\": false}'\n\nsmartphone = json.loads(smartphone_json, cls=SmartphoneDecoder)\nprint(type(smartphone)) # \u003cclass '__main__.Smartphone'\u003e\nname = smartphone.name # iPear 23 Plus\n```\n\nSimilarly, you can use `SmartphoneDecoder` with `json.load()`:\n\n```\nsmartphone = json.load(smartphone_json_file, cls=SmartphoneDecoder)\n```\n\n## Python Data to JSON\n\nYou can also go the other way around and convert Python data structures and primitives to JSON. This is possible thanks to the [`json.dump()`](https://docs.python.org/3/library/json.html#json.dump) and [`json.dumps()`](https://docs.python.org/3/library/json.html#json.dumps) functions, which follows the [conversion table](https://docs.python.org/3/library/json.html#py-to-json-table) below:\n\n| **Python Data** | **JSON Value** |\n| - | - | - |\n| `str` | `string` |\n| `int` | `number (integer)` |\n| `float` | `number (real)` |\n| `True` | `true` |\n| `False` | `false` |\n| `None` | `null` |\n| `list` | `array` |\n| `dict` | `object` |\n| `Null` | None  |\n\n`json.dump()` allows you to write a JSON string to a file, as in the following example:\n\n```python\nimport json\n\nuser_dict = {\n    \"name\": \"John\",\n    \"surname\": \"Williams\",\n    \"age\": 48,\n    \"city\": \"New York\"\n}\n\n# serializing the sample dictionary to a JSON file\nwith open(\"user.json\", \"w\") as json_file:\n    json.dump(user_dict, json_file)\n```\n\nThis snippet will serialize the Python `user_dict` variable into the `user.json` file.\n\nSimilarly, `json.dumps()` converts a Python variable to its equivalent JSON string:\n\n```python\nimport json\n\nuser_dict = {\n    \"name\": \"John\",\n    \"surname\": \"Williams\",\n    \"age\": 48,\n    \"city\": \"New York\"\n}\n\nuser_json_string = json.dumps(user_dict)\n\nprint(user_json_string)\n```\nRun this snippet and you will get:\n\n```json\n{\"name\": \"John\", \"surname\": \"Williams\", \"age\": 48, \"city\": \"New York\"}\n```\n\n\u003e **Note**\\\n\u003e Follow the [official documentation](https://docs.python.org/3/library/json.html#json.JSONEncoder) to learn how to specify a custom encoder.\n\n### Limitations of the `json` Standard Module\n\nJSON [data parsing](https://brightdata.com/blog/web-data/what-is-data-parsing) comes with challenges that cannot be overlooked.\n\nTwo commons examples are:\n\n- The Python `json` module would fall short in case of invalid, broken, or non-standard JSON.\n- Parsing JSON data from untrusted sources is dangerous because a malicious JSON string can cause your parser to break or consume a large amount of resources.\n\nThese limitations can be worked around, but it's best to use a commercial tool that makes JSON parsing easier, such as [Web Scraper API](https://brightdata.com/products/web-scraper).\n\n## Conclusion\n\nWhile natively parsing JSON data through the `json` standard module in Python, you will need reliable proxy servers to bypass restrictions imposed by websites. Try a cutting-edge, fully-featured, commercial solution for data parsing, such as Bright Data's data and [proxy products](https://brightdata.com/proxy-types).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fluminati-io%2Fparse-json-in-python","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fluminati-io%2Fparse-json-in-python","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fluminati-io%2Fparse-json-in-python/lists"}