{"id":13569059,"url":"https://github.com/Pecan-Street/DataPort-Examples","last_synced_at":"2025-04-04T05:31:24.366Z","repository":{"id":96344646,"uuid":"249732759","full_name":"Pecan-Street/DataPort-Examples","owner":"Pecan-Street","description":"DataPort Python Examples","archived":false,"fork":false,"pushed_at":"2023-05-19T12:03:57.000Z","size":14,"stargazers_count":10,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-08-02T14:05:58.814Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://dataport.pecanstreet.org/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Pecan-Street.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-03-24T14:38:46.000Z","updated_at":"2024-07-01T09:40:30.000Z","dependencies_parsed_at":"2024-01-16T20:24:15.466Z","dependency_job_id":"2b46283c-9572-4200-9225-ada2dfb5dab4","html_url":"https://github.com/Pecan-Street/DataPort-Examples","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Pecan-Street%2FDataPort-Examples","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Pecan-Street%2FDataPort-Examples/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Pecan-Street%2FDataPort-Examples/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Pecan-Street%2FDataPort-Examples/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Pecan-Street","download_url":"https://codeload.github.com/Pecan-Street/DataPort-Examples/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223100106,"owners_count":17087386,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T14:00:35.566Z","updated_at":"2024-11-05T01:31:39.421Z","avatar_url":"https://github.com/Pecan-Street.png","language":"Python","funding_links":[],"categories":["Datasets"],"sub_categories":["Electricity-Demand"],"readme":"# DataPort-Examples\nA collection of examples of how to use Pecan Street Inc. Data.\n\nhttps://dataport.pecanstreet.org/\n\n## Requirements\n- Python 3\n- A DataPort account\n\n## Installation / Setup\n### Python Modules\nThe requirements.txt file has the Python modules needed for these examples.\nIf you are using pip then issue the command:\n\n`pip3 install -r requirements.txt`\n\n### Database Credentials\nIf you have direct database access, you'll need to enter the connection parameters and authentication credentials\ninto the config/config.txt file. You can find the necessary info at https://dataport.pecanstreet.org/access . \n\n### Flat File Download\nIf you are a university free account, the data can be found on this page:\nhttps://dataport.pecanstreet.org/academic\nIn particular the flat_file/read_csv.py example uses the `Austin 15-min (22.4 MB)` download.\n\n## Examples\nThe examples directory has subdirectories containing Python 3 scripts for reading the csv\nflat files downloaded from the DataPort portal as well as loading data by connecting to the \nDataPort Postgres database directly. \n\n### examples/flat_file\nThe flat_file directory has an example that uses the flat csv files downloaded from the DataPort portal.\nTo run this example, download the 15 minute Austin bundle, then untar and unzip its contents into the flat_file\ndirectory. You'll need the `metadata.csv` file and the `15minute_data_austin.csv` to be in the same directory as \nthe `read_csv.py` file.\n\nAt the top of the file there's a constant set to limit the size of the dataset that is read.\n\n`LIMIT = 10000`\n\nThis is there because the dataset is quite large, and can take a long time to load, but this will limit \ntime covered to a few months. Change it to `LIMIT = None` to read the entire dataset.\n\nTo run the example execute from the `examples/flat_file` directory:\n\n`python3 read_csv.py`\n\n### examples/database\nThe database directory has an example of reading the metadata and data directly from the Postgres database.\nThe `eclipse.py` script reads and plots solar panel generation over a few days surrounding the August 17, 2017 \nsolar eclipse. \n\nBe sure you configured your database connection parameters and credentials as outlined above.\n\nTo run this example execute from the `examples/database` directory:\n\n`python3 eclipse.py`\n\n## Dataport CSV to HDF Conversion\nThe kind folks at the NILMTK [repo](https://github.com/nilmtk/nilmtk) have contributed a converter for our Dataport CSV data to HDF. You can find the converter \n[here](https://github.com/nilmtk/nilmtk/tree/master/nilmtk/dataset_converters/dataport). \n\n## Contribute:\nWe're happy to take contributions! Submit a pull request to this repo!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPecan-Street%2FDataPort-Examples","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FPecan-Street%2FDataPort-Examples","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPecan-Street%2FDataPort-Examples/lists"}