{"id":20287302,"url":"https://github.com/nelsonbittencourt/get_open_data","last_synced_at":"2025-10-11T18:47:16.085Z","repository":{"id":206234711,"uuid":"716150000","full_name":"NelsonBittencourt/get_open_data","owner":"NelsonBittencourt","description":"Python function to get csv/xls/xlsx files from Ckan site and convert it to Pandas dataframe.","archived":false,"fork":false,"pushed_at":"2024-02-02T17:12:03.000Z","size":17,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-04T04:14:51.957Z","etag":null,"topics":["aneel","ccee","ckan","energia","ons","open-data","python","scripts"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NelsonBittencourt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-08T14:42:27.000Z","updated_at":"2024-02-02T17:33:38.000Z","dependencies_parsed_at":null,"dependency_job_id":"c13a588a-5fc5-4f45-a134-d17237e026eb","html_url":"https://github.com/NelsonBittencourt/get_open_data","commit_stats":null,"previous_names":["nelsonbittencourt/get_open_data"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/NelsonBittencourt/get_open_data","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NelsonBittencourt%2Fget_open_data","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NelsonBittencourt%2Fget_open_data/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NelsonBittencourt%2Fget_open_data/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NelsonBittencourt%2Fget_open_data/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NelsonBittencourt","download_url":"https://codeload.github.com/NelsonBittencourt/get_open_data/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NelsonBittencourt%2Fget_open_data/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264923586,"owners_count":23683769,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aneel","ccee","ckan","energia","ons","open-data","python","scripts"],"created_at":"2024-11-14T14:39:17.767Z","updated_at":"2025-10-11T18:47:16.011Z","avatar_url":"https://github.com/NelsonBittencourt.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# get_open_data:\r\n\r\nPython function to get a csv file from Ckan site and convert it to Pandas dataframe.\r\n\r\n\r\n# Basic usage:\r\n\r\nThis example uses open data available by CCEE (https://www.ccee.org.br/), ONS (https://www.ons.org.br/) and ANEEL (https://www.gov.br/).\r\n\r\nSee full function declaration (get_open_data.py) to another optional arguments like res_encoding, res_type, res_sep, etc.\r\n\r\n\r\n\r\n```Python\r\n\r\nimport get_open_data as od\r\n\r\n# URLs with Open Data in Brazilian Electric Sector\r\nccee_open_data_url  = 'https://dadosabertos.ccee.org.br'              \r\nons_open_data_url   = 'https://dados.ons.org.br'          \r\naneel_open_data_url = 'https://dadosabertos.aneel.gov.br'\r\n   \r\n# 1) Gets CCEE data\r\nccee_test1 = od.get_ckan_data(ccee_open_data_url, 'encargo_importacao_geracao_2023')\r\n\r\n# 2) Gets ONS data\r\n# 2.1)\r\n# The 'ENERGIA_VERTIDA_TURBINAVEL-2023' resource has two files (csv and xlsx).\r\n# Use 'res_item' to select csv file. Xlsx format is not allowed in this script.\r\n# In this example, 'resource_item' isn't necessary since csv file is in zero position. \r\nons_test1 = od.get_ckan_data(ons_open_data_url, 'ENERGIA_VERTIDA_TURBINAVEL-2023', res_item=0)\r\n\r\n# 2.2)\r\n# Resource 'ImportacaoEnergia_Comercial-2023' has two files: xlsx and csv.\r\n# Line below will donwload 'xlsx' version (resource = 0).\r\n# Warning:  converting Excel files with Pandas can be very slow!\r\nons_test2 = od.get_ckan_data(ons_open_data_url,'ImportacaoEnergia_Comercial-2023',res_type='xlsx')\r\n\r\n# 3) Gets ANEEL data\r\n# The 'empreendimento-gd-informacoes-tecnicas-hidreletrica.csv' resource contains non UTF-8 characters.\r\n# In this case, to avoid errors in csv to dataframe convertion, use 'csv_encoding'.\r\naneel_test1 = od.get_ckan_data(aneel_open_data_url,'empreendimento-gd-informacoes-tecnicas-hidreletrica.csv',res_encoding='latin-1')\r\n\r\n\r\n```\r\n\r\n\r\n# Personal site:\r\nhttp://www.nrbenergia.somee.com/  \r\n\r\nor\r\n\r\nhttps://nrbenergia.azurewebsites.net/\r\n\r\n# More info:\r\nhttps://www.linkedin.com/in/nelsonrossibittencourt/\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnelsonbittencourt%2Fget_open_data","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnelsonbittencourt%2Fget_open_data","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnelsonbittencourt%2Fget_open_data/lists"}