{"id":14065004,"url":"https://github.com/socrata/socrata-py","last_synced_at":"2026-01-23T05:27:52.941Z","repository":{"id":25798478,"uuid":"83365653","full_name":"socrata/socrata-py","owner":"socrata","description":"socrata data-pipeline python library","archived":false,"fork":false,"pushed_at":"2025-12-22T22:49:34.000Z","size":2505,"stargazers_count":99,"open_issues_count":13,"forks_count":29,"subscribers_count":44,"default_branch":"main","last_synced_at":"2025-12-24T08:57:19.774Z","etag":null,"topics":["engineering"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/socrata.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2017-02-27T22:57:14.000Z","updated_at":"2025-12-22T20:34:09.000Z","dependencies_parsed_at":"2024-06-19T16:24:27.673Z","dependency_job_id":"59cbb5ae-53a5-4089-a4ef-cfe588581d58","html_url":"https://github.com/socrata/socrata-py","commit_stats":{"total_commits":217,"total_committers":14,"mean_commits":15.5,"dds":"0.25806451612903225","last_synced_commit":"acfae68a53a8c1849cb3942f7d3ac35b0c21b8ad"},"previous_names":["socrata/publish-py"],"tags_count":12,"template":false,"template_full_name":null,"purl":"pkg:github/socrata/socrata-py","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/socrata%2Fsocrata-py","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/socrata%2Fsocrata-py/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/socrata%2Fsocrata-py/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/socrata%2Fsocrata-py/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/socrata","download_url":"https://codeload.github.com/socrata/socrata-py/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/socrata%2Fsocrata-py/sbom","scorecard":{"id":835691,"data":{"date":"2025-08-11","repo":{"name":"github.com/socrata/socrata-py","commit":"acfae68a53a8c1849cb3942f7d3ac35b0c21b8ad"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":2.8,"checks":[{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Code-Review","score":7,"reason":"Found 12/17 approved changesets -- score normalized to 7","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: pipCommand not pinned by hash: bin/test-local:5","Warn: pipCommand not pinned by hash: bin/test-rc:5","Warn: pipCommand not pinned by hash: bin/test-staging:5","Info:   0 out of   3 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":3,"reason":"7 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: PYSEC-2020-73","Warn: Project is vulnerable to: GHSA-5vgj-ggm4-fg62","Warn: Project is vulnerable to: PYSEC-2014-14 / GHSA-652x-xj99-gmcc","Warn: Project is vulnerable to: GHSA-9hjg-9r4m-mvj7","Warn: Project is vulnerable to: GHSA-9wx4-h78v-vm56","Warn: Project is vulnerable to: PYSEC-2014-13 / GHSA-cfj3-7x9c-4p3h","Warn: Project is vulnerable to: PYSEC-2018-28 / GHSA-x84v-xcm2-53pg"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 25 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-23T18:59:08.143Z","repository_id":25798478,"created_at":"2025-08-23T18:59:08.143Z","updated_at":"2025-08-23T18:59:08.143Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28680693,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-23T04:33:33.518Z","status":"ssl_error","status_checked_at":"2026-01-23T04:33:30.433Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["engineering"],"created_at":"2024-08-13T07:04:13.911Z","updated_at":"2026-01-23T05:27:52.915Z","avatar_url":"https://github.com/socrata.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# socrata-py\nPython SDK for the Socrata Data Management API. Use this library to call into publishing and ETL functionality offered when writing to Socrata datasets.\n\n```python\nwith open('cool_dataset.csv', 'rb') as file:\n    (revision, output) = Socrata(auth).create(\n        name = \"cool dataset\",\n        description = \"a description\"\n    ).csv(file)\n\n    revision.apply(output_schema = output)\n```\n\n\u003c!-- toc --\u003e\n\n  * [Installation](#installation)\n  * [Example](#example)\n  * [Using](#using)\n    + [Boilerplate](#boilerplate)\n    + [Simple usage](#simple-usage)\n      - [Create a new Dataset from a csv, tsv, xls or xlsx file](#create-a-new-dataset-from-a-csv-tsv-xls-or-xlsx-file)\n      - [Create a new Dataset from Pandas](#create-a-new-dataset-from-pandas)\n      - [Updating a dataset](#updating-a-dataset)\n      - [Generating a config and using it to update](#generating-a-config-and-using-it-to-update)\n  * [Advanced usage](#advanced-usage)\n    + [Create a revision](#create-a-revision)\n    + [Create an upload](#create-an-upload)\n    + [Upload a file](#upload-a-file)\n    + [Transforming your data](#transforming-your-data)\n    + [Wait for the transformation to finish](#wait-for-the-transformation-to-finish)\n    + [Errors in a transformation](#errors-in-a-transformation)\n    + [Validating rows](#validating-rows)\n    + [Do the upsert!](#do-the-upsert)\n    + [Metadata only revisions](#metadata-only-revisions)\n- [Development](#development)\n  * [Testing](#testing)\n  * [Generating docs](#generating-docs)\n  * [Releasing](#releasing)\n- [Library Docs](#library-docs)\n    + [Socrata](#socrata)\n      - [create](#create)\n      - [new](#new)\n      - [using_config](#using_config)\n    + [Authorization](#authorization)\n      - [live_dangerously](#live_dangerously)\n    + [Revisions](#revisions)\n      - [create_delete_revision](#create_delete_revision)\n      - [create_replace_revision](#create_replace_revision)\n      - [create_update_revision](#create_update_revision)\n      - [create_using_config](#create_using_config)\n      - [list](#list)\n      - [lookup](#lookup)\n    + [Revision](#revision)\n      - [apply](#apply)\n      - [create_upload](#create_upload)\n      - [discard](#discard)\n      - [list_operations](#list_operations)\n      - [open_in_browser](#open_in_browser)\n      - [plan](#plan)\n      - [set_notes](#set_notes)\n      - [set_output_schema](#set_output_schema)\n      - [source_as_blob](#source_as_blob)\n      - [source_from_agent](#source_from_agent)\n      - [source_from_dataset](#source_from_dataset)\n      - [source_from_url](#source_from_url)\n      - [ui_url](#ui_url)\n      - [update](#update)\n    + [Sources](#sources)\n      - [create_upload](#create_upload-1)\n      - [lookup](#lookup-1)\n    + [Source](#source)\n      - [add_to_revision](#add_to_revision)\n      - [blob](#blob)\n      - [change_parse_option](#change_parse_option)\n      - [csv](#csv)\n      - [df](#df)\n      - [geojson](#geojson)\n      - [kml](#kml)\n      - [list_operations](#list_operations-1)\n      - [load](#load)\n      - [open_in_browser](#open_in_browser-1)\n      - [shapefile](#shapefile)\n      - [tsv](#tsv)\n      - [ui_url](#ui_url-1)\n      - [wait_for_finish](#wait_for_finish)\n      - [wait_for_schema](#wait_for_schema)\n      - [xls](#xls)\n      - [xlsx](#xlsx)\n    + [Configs](#configs)\n      - [create](#create-1)\n      - [list](#list-1)\n      - [lookup](#lookup-2)\n    + [Config](#config)\n      - [change_parse_option](#change_parse_option-1)\n      - [create_revision](#create_revision)\n      - [delete](#delete)\n      - [list_operations](#list_operations-2)\n      - [update](#update-1)\n    + [InputSchema](#inputschema)\n      - [get_latest_output_schema](#get_latest_output_schema)\n      - [latest_output](#latest_output)\n      - [list_operations](#list_operations-3)\n      - [transform](#transform)\n      - [wait_for_schema](#wait_for_schema-1)\n    + [OutputSchema](#outputschema)\n      - [add_column](#add_column)\n      - [any_failed](#any_failed)\n      - [build_config](#build_config)\n      - [change_column_metadata](#change_column_metadata)\n      - [change_column_transform](#change_column_transform)\n      - [drop_column](#drop_column)\n      - [list_operations](#list_operations-4)\n      - [rows](#rows)\n      - [run](#run)\n      - [schema_errors](#schema_errors)\n      - [schema_errors_csv](#schema_errors_csv)\n      - [set_row_id](#set_row_id)\n      - [set_sort_by](#set_sort_by)\n      - [validate_row_id](#validate_row_id)\n      - [wait_for_finish](#wait_for_finish-1)\n    + [Job](#job)\n      - [is_complete](#is_complete)\n      - [list_operations](#list_operations-5)\n      - [submitted_for_approval](#submitted_for_approval)\n      - [wait_for_finish](#wait_for_finish-2)\n\n\u003c!-- tocstop --\u003e\n\n## Installation\nThis only supports python3.\n\nInstallation is available through pip. Using a virtualenv is advised. Install\nthe package by running\n\n```\npip3 install socrata-py\n```\n\nThe only hard dependency is `requests` which will be installed via pip. Pandas is not required, but creating a dataset from a Pandas dataframe is supported. See below.\n\n\n## Example\nTry the command line example with\n```bash\npython -m examples.create 'Police Reports' ~/Desktop/catalog.data.gov/Seattle_Real_Time_Fire_911_Calls.csv 'pete-test.test-socrata.com' --username $SOCRATA_USERNAME --password $SOCRATA_PASSWORD\n```\n## Using\n\n### Boilerplate\n```python\n# Import some stuff\nfrom socrata.authorization import Authorization\nfrom socrata import Socrata\nimport os\n\n# Boilerplate...\n# Make an auth object\nauth = Authorization(\n  \"pete-test.test-socrata.com\",\n  os.environ['SOCRATA_USERNAME'],\n  os.environ['SOCRATA_PASSWORD']\n)\n\n```\n\n### Simple usage\n\n\n#### Create a new Dataset from a csv, tsv, xls or xlsx file\nTo create a dataset, you can do this:\n\n```python\nwith open('cool_dataset.csv', 'rb') as file:\n    # Upload + Transform step\n\n    # revision is the *change* to the view in the catalog, which has not yet been applied.\n    # output is the OutputSchema, which is a change to data which can be applied via the revision\n    (revision, output) = Socrata(auth).create(\n        name = \"cool dataset\",\n        description = \"a description\"\n    ).csv(file)\n\n    # Transformation step\n    # We want to add some metadata to our column, drop another column, and add a new column which will\n    # be filled with values from another column and then transformed\n    output = output\\\n        .change_column_metadata('a_column', 'display_name').to('A Column!')\\\n        .change_column_metadata('a_column', 'description').to('Here is a description of my A Column')\\\n        .drop_column('b_column')\\\n        .add_column('a_column_squared', 'A Column, but times itself', 'to_number(`a_column`) * to_number(`a_column`)', 'this is a column squared')\\\n        .run()\n\n\n    # Validation of the results step\n    output = output.wait_for_finish()\n    # The data has been validated now, and we can access errors that happened during validation. For example, if one of the cells in `a_column` couldn't be converted to a number in the call to `to_number`, that error would be reflected in this error_count\n    assert output.attributes['error_count'] == 0\n\n    # If you want, you can get a csv stream of all the errors\n    errors = output.schema_errors_csv()\n    for line in errors.iter_lines():\n        print(line)\n\n    # Update step\n\n    # Apply the revision - this will make it public and available to make\n    # visualizations from\n    job = revision.apply(output_schema = output)\n\n    # This opens a browser window to your revision, and you will see the progress\n    # of the job\n    revision.open_in_browser()\n\n    # Application is async - this will block until all the data\n    # is in place and readable\n    job.wait_for_finish()\n```\n\nSimilar to the `csv` method are the `xls`, `xlsx`, and `tsv` methods, which upload\nthose files.\n\nThere is a `blob` method as well, which uploads blobby data to the source. This means the data will not be parsed, and will be displayed under \"Files and Documents\" in the catalog once the revision is applied.\n\n#### Create a new Dataset from Pandas\nDatasets can also be created from Pandas DataFrames\n```python\nimport pandas as pd\ndf = pd.read_csv('socrata-py/test/fixtures/simple.csv')\n# Do various Pandas-y changes and modifications, then...\n(revision, output) = Socrata(auth).create(\n    name = \"Pandas Dataset\",\n    description = \"Dataset made from a Pandas Dataframe\"\n).df(df)\n\n# Same code as above to apply the revision.\n\n```\n\n#### Updating a dataset\nA Socrata `update` is actually an upsert. Rows are updated or created based on the row identifier. If the row-identifer doesn't exist, all updates are just appends to the dataset.\n\nA `replace` truncates the whole dataset and then inserts the new data.\n\n#### Generating a config and using it to update\n```python\n# This is how we create our view initially\nwith open('cool_dataset.csv', 'rb') as file:\n    (revision, output) = Socrata(auth).create(\n        name = \"cool dataset\",\n        description = \"a description\"\n    ).csv(file)\n\n    revision.apply(output_schema = output)\n\n# This will build a configuration using the same settings (file parsing and\n# data transformation rules) that we used to get our output. The action\n# that we will take will be \"update\", though it could also be \"replace\"\nconfig = output.build_config(\"cool-dataset-config\", \"update\")\n\n# Now we need to save our configuration name and view id somewhere so we\n# can update the view using our config\nconfiguration_name = \"cool-dataset-config\"\nview_id = revision.view_id()\n\n# Now later, if we want to use that config to update our view, we just need the view and the configuration_name\nsocrata = Socrata(auth)\nview = socrata.views.lookup(view_id) # View will be the view we are updating with the new data\n\nwith open('updated-cool-dataset.csv', 'rb') as my_file:\n    (revision, job) = socrata.using_config(\n        configuration_name,\n        view\n    ).csv(my_file)\n    print(job) # Our update job is now running\n```\n\n\n## Advanced usage\n\n### Create a revision\n\n```python\n# This is our socrata object, using the auth variable from above\nsocrata = Socrata(auth)\n\n# This will make our initial revision, on a view that doesn't yet exist\nrevision = socrata.new({'name': 'cool dataset'})\n\n# revision is a Revision object, we can print it\nprint(revision)\nRevision({'created_by': {'display_name': 'rozap',\n                'email': 'chris.duranti@socrata.com',\n                'user_id': 'tugg-ikce'},\n 'fourfour': 'ij46-xpxe',\n 'id': 346,\n 'inserted_at': '2017-02-27T23:05:08.522796',\n 'metadata': None,\n 'update_seq': 285,\n 'upsert_jobs': []})\n\n# We can also access the attributes of the revision\nprint(revision.attributes['metadata']['name'])\n'cool dataset'\n```\n\n### Create an upload\n```python\n# Using that revision, we can create an upload\nupload = revision.create_upload('foo.csv')\n\n# And print it\nprint(upload)\nSource({'content_type': None,\n 'created_by': {'display_name': 'rozap',\n                'email': 'chris.duranti@socrata.com',\n                'user_id': 'tugg-ikce'},\n 'source_type': {\n    'filename': 'foo.csv',\n    'type': 'upload'\n },\n 'finished_at': None,\n 'id': 290,\n 'inserted_at': '2017-02-27T23:07:18.309676',\n 'schemas': []})\n```\n### Upload a file\n```python\n# And using that upload we just created, we can put bytes into it\nwith open('test/fixtures/simple.csv', 'rb') as f:\n    source = upload.csv(f)\n```\n### Transforming your data\nTransforming data consists of going from input data (data exactly as it appeared in the source)\nto output data (data as you want it to appear).\n\nTransformation from input data to output data often has problems. You might, for example, have a column\nfull of numbers, but one row in that column is actually the value `hehe!` which cannot be transformed into\na number. Rather than failing at each datum which is dirty or wrong, transforming your data allows you to\nreconcile these issues.\n\nWe might have a dataset called `temps.csv` that looks like\n```\ndate, celsius\n8-24-2017, 22\n8-25-2017, 20\n8-26-2017, 23\n8-27-2017, hehe!\n8-28-2017,\n8-29-2017, 21\n```\n\nSuppose we uploaded it in our previous step, like this:\n\n```python\nwith open('temps.csv', 'rb') as f:\n    source = upload.csv(f)\n    input_schema = source.get_latest_input_schema()\n```\n\nOur `input_schema` is the input data exactly as it appeared in the CSV, with all values of type `string`.\n\nOur `output_schema` is the output data as it was *guessed* by Socrata. Guessing may not always be correct, which is why we have import configs to \"lock in\" a schema for automation. We can get the `output_schema`\nlike so:\n\n```python\noutput_schema = input_schema.get_latest_output_schema()\n```\n\nWe can now make changes to the schema, like so\n\n```python\nnew_output_schema = output\n    # Change the field_name of date to the_date\n    .change_column_metadata('date', 'field_name').to('the_date')\\\n    # Change the description of the celsius column\n    .change_column_metadata('celsius', 'description').to('the temperature in celsius')\\\n    # Change the display name of the celsius column\n    .change_column_metadata('celsius', 'display_name').to('Degrees (Celsius)')\\\n    # Change the transform of the_date column to to_fixed_timestamp(`date`)\n    .change_column_transform('the_date').to('to_fixed_timestamp(`date`)')\\\n    # Make the celsius column all numbers\n    .change_column_transform('celsius').to('to_number(`celsius`)')\\\n    # Add a new column, which is computed from the `celsius` column\n    .add_column('fahrenheit', 'Degrees (Fahrenheit)', '(to_number(`celsius`) * (9 / 5)) + 32', 'the temperature in celsius')\\\n    .run()\n```\n\n`change_column_metadata(column_name, column_attribute)` takes the field name used to\nidentify the column and the column attribute to change (`field_name`, `display_name`, `description`, `position`)\n\n`add_column(field_name, display_name, transform_expression, description)` will create a new column\n\nWe can also call `drop_column(celsius)` which will drop the column.\n\n`.run()` will then make a request and return the new output_schema, or an error if something is invalid.\n\nTransforms can be complex SoQL expressions. Available functions are listed [here](http://docs.socratapublishing.apiary.io/#reference/0/inputschema). You can do lots of stuff with them;\n\n\nFor example, you could change all `null` values into errors (which won't be imported) by doing\nsomething like\n```python\nnew_output_schema = output\n    .change_column_transform('celsius').to('coalesce(to_number(`celsius`), error(\"Celsius was null!\"))')\n    .run()\n```\n\nOr you could add a new column that says if the day was hot or not\n```python\nnew_output_schema = output\n    .add_column('is_hot', 'Was the day hot?', 'to_number(`celsius`) \u003e= 23')\n    .run()\n```\n\nOr you could geocode a column, given the following CSV\n```\naddress,city,zip,state\n10028 Ravenna Ave NE, Seattle, 98128, WA\n1600 Pennsylvania Avenue, Washington DC, 20500, DC\n6511 32nd Ave NW, Seattle, 98155, WA\n```\n\nWe could transform our first `output_schema` into a single column dataset, where that\nsingle column is a `Point` of the address\n\n```python\noutput = output\\\n    .add_column('location', 'Incident Location', 'geocode(`address`, `city`, `state`, `zip`)')\\\n    .drop_column('address')\\\n    .drop_column('city')\\\n    .drop_column('state')\\\n    .drop_column('zip')\\\n    .run()\n```\n\n\nComposing these SoQL functions into expressions will allow you to validate, shape, clean and extend your data to make it more useful to the consumer.\n\n### Wait for the transformation to finish\nTransformations are async, so if you want to wait for it to finish, you can do so\n```ok output_schema) = new_output_schema.wait_for_finish()\n```\n\n### Errors in a transformation\nTransformations may have had errors, like in the previous example, we can't convert `hehe!` to a number. We can see the count of them like this:\n```python\nprint(output_schema.attributes['error_count'])\n```\n\nWe can view the detailed errors like this:\n```python\nerrors = output_schema.schema_errors()\n```\n\nWe can get a CSV of the errors like this:\n```python\ncsv_stream = output_schema.schema_errors_csv()\n```\n\n### Validating rows\nWe can look at the rows of our schema as well\n```python\nrows = output_schema.rows(offset = 0, limit = 20)\n\nself.assertEqual(rows, [\n    {'b': {'ok': ' bfoo'}},\n    {'b': {'ok': ' bfoo'}},\n    {'b': {'ok': ' bfoo'}},\n    {'b': {'ok': ' bfoo'}}\n])\n```\n\n### Do the upsert!\n```python\n# Now we have transformed our data into the shape we want, let's do an upsert\njob = revision.apply(output_schema = output_schema)\n\n# This will complete the upsert behind the scenes. If we want to\n# re-fetch the current state of the upsert job, we can do so\njob = job.show()\n\n# To get the progress\nprint(job.attributes['log'])\n[\n    {'details': {'Errors': 0, 'Rows Created': 0, 'Rows Updated': 0, 'By RowIdentifier': 0, 'By SID': 0, 'Rows Deleted': 0}, 'time': '2017-02-28T20:20:59', 'stage': 'upsert_complete'},\n    {'details': {'created': 1}, 'time': '2017-02-28T20:20:59', 'stage': 'columns_created'},\n    {'details': {'created': 1}, 'time': '2017-02-28T20:20:59', 'stage': 'columns_created'},\n    {'details': None, 'time': '2017-02-28T20:20:59', 'stage': 'started'}\n]\n\n\n# So maybe we just want to wait here, printing the progress, until the job is done\njob.wait_for_finish(progress = lambda job: print(job.attributes['log']))\n\n# So now if we go look at our original four-four, our data will be there\n```\n\n### Metadata only revisions\nWhen there is an existing Socrata view that you'd like to update the metadata of, you can do so by creating a Source which is the Socrata view.\n\n```python\nview = socrata.views.lookup('abba-cafe')\n\nrevision = view.revisions.create_replace_revision()\nsource = revision.source_from_dataset()\noutput_schema = source.get_latest_input_schema().get_latest_output_schema()\nnew_output_schema = output_schema\\\n    .change_column_metadata('a', 'description').to('meh')\\\n    .change_column_metadata('b', 'display_name').to('bbbb')\\\n    .change_column_metadata('c', 'field_name').to('ccc')\\\n    .run()\n\n\nrevision.apply(output_schema = new_output_schema)\n```\n# Development\n\n## Testing\nInstall test deps by running `pip install -r requirements.txt`. This will install `pdoc` and `pandas` which are required to run the tests.\n\nConfiguration is set in `test/auth.py` for tests. It reads the domain, username, and password from environment variables. If you want to run the tests, set those environment variables to something that will work.\n\nIf I wanted to run the tests against my local instance, I would run:\n```bash\nSOCRATA_DOMAIN=localhost SOCRATA_USERNAME=$SOCRATA_LOCAL_USER SOCRATA_PASSWORD=$SOCRATA_LOCAL_PASS bin/test\n```\n\n## Generating docs\nmake the docs by running\n```bash\nmake docs\n```\n\n## Releasing\nYou will need to have [twine](https://pypi.org/project/twine/) installed (`pip3 install twine`), and a `.pypirc` file in your home directory.\nFor help, read [this](https://web.archive.org/web/20180523182143/http://peterdowns.com/posts/first-time-with-pypi.html)\n\nAn example of a pypirc file looks like:\n```\n[distutils]\nindex-servers =\n  local\n  pypi\n\n[local]\nrepository=https://repo.socrata.com/artifactory/api/pypi/pypi\nusername=shared-engr\npassword=\u003cREDACTED\u003e\n\n[pypi]\nrepository=https://upload.pypi.org/legacy/\nusername=socrata\npassword=\u003cREDACTED\u003e\n```\n\nMake sure the version in setup.py is new and makes sense for the change you're releasing. Then run:\n\n```\npython3 setup.py sdist\ntwine upload dist/\u003cyour distribution file\u003e\n```\n\n\u003c!-- doc --\u003e\n# Library Docs\n\n### [Socrata](https://github.com/socrata/socrata-py/blob/master//socrata/__init__.py#L9)\n```\nArgSpec\n    Args: auth\n```\n\nTop level publishing object.\n\nAll functions making HTTP calls return a result tuple, where the first element in the\ntuple is whether or not the call succeeded, and the second element is the returned\nobject if it was a success, or a dictionary containing the error response if the call\nfailed. 2xx responses are considered successes. 4xx and 5xx responses are considered failures.\nIn the event of a socket hangup, an exception is raised.\n\n#### [create](https://github.com/socrata/socrata-py/blob/master//socrata/__init__.py#L61)\n\n\nShortcut to create a dataset. Returns a `Create` object,\nwhich contains functions which will create a view, upload\nyour file, and validate data quality in one step.\n\nTo actually place the validated data into a view, you can call .apply()\non the revision\n```\n(revision, output_schema) Socrata(auth).create(\n    name = \"cool dataset\",\n    description = \"a description\"\n).csv(file)\n\njob = revision.apply(output_schema = output_schema)\n```\n\nArgs:\n```\n   **kwargs: Arbitrary revision metadata values\n```\n\nReturns:\n```\n    result (Revision, OutputSchema): Returns the revision that was created and the\n        OutputSchema created from your uploaded file\n```\n\nExamples:\n```python\nSocrata(auth).create(\n    name = \"cool dataset\",\n    description = \"a description\"\n).csv(open('my-file.csv'))\n```\n\n#### [new](https://github.com/socrata/socrata-py/blob/master//socrata/__init__.py#L100)\n```\nArgSpec\n    Args: metadata, deleted_at\n```\n\nCreate an empty revision, on a view that doesn't exist yet. The\nview will be created for you, and the initial revision will be returned.\n\nArgs:\n```\n    metadata (dict): Metadata to apply to the revision\n```\n\nReturns:\n```\n    Revision\n```\n\nExamples:\n```python\n    rev = Socrata(auth).new({\n        'name': 'hi',\n        'description': 'foo!',\n        'metadata': {\n            'view': 'metadata',\n            'anything': 'is allowed here'\n\n        }\n    })\n```\n\n#### [using_config](https://github.com/socrata/socrata-py/blob/master//socrata/__init__.py#L28)\n```\nArgSpec\n    Args: config_name, view\n```\n\nUpdate a dataset, using the configuration that you previously\ncreated, and saved the name of. Takes the `config_name` parameter\nwhich uniquely identifies the config, and the `View` object, which can\nbe obtained from `socrata.views.lookup('view-id42')`\n\nArgs:\n```\n    config_name (str): The config name\n    view (View): The view to update\n```\n\nReturns:\n```\n    result (ConfiguredJob): Returns the ConfiguredJob\n```\n\nNote:\n    Typical usage would be in a context manager block (as demonstrated in the example\n    below). In this case, the `ConfiguredJob` is created and immediately launched by way of\n    the call to the `ConfiguredJob.csv` method.\n\nExamples:\n```\n    with open('my-file.csv', 'rb') as my_file:\n        (rev, job) = p.using_config(name, view).csv(my_file)\n```\n\n### [Authorization](https://github.com/socrata/socrata-py/blob/master//socrata/authorization.py#L3)\n```\nArgSpec\n    Args: domain, username, password, request_id_prefix\n    Defaults: domain=\n```\n\nManages basic authorization for accessing the socrata API.\nThis is passed into the `Socrata` object once, which is the entry\npoint for all operations.\n\n    auth = Authorization(\n        \"data.seattle.gov\",\n        os.environ['SOCRATA_USERNAME'],\n        os.environ['SOCRATA_PASSWORD']\n    )\n    publishing = Socrata(auth)\n\n#### [live_dangerously](https://github.com/socrata/socrata-py/blob/master//socrata/authorization.py#L28)\n\n\nDisable SSL checking. Note that this should *only* be used while developing\nagainst a local Socrata instance.\n\n### [Revisions](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L9)\n```\nArgSpec\n    Args: fourfour, auth\n```\n\n\n\n#### [create_delete_revision](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L100)\n```\nArgSpec\n    Args: metadata, permission\n    Defaults: metadata={}, permission=public\n```\n\nCreate a revision on the view, which when applied, will delete rows of data.\n\nThis is an upsert; a row id must be set.\n\nArgs:\n```\n    metadata (dict): The metadata to change; these changes will be applied when the revision is applied\n    permission (string): 'public' or 'private'\n```\n\nReturns:\n```\n    Revision The new revision, or an error\n```\n\nExamples:\n```python\n    view.revisions.create_delete_revision(metadata = {\n        'name': 'new dataset name',\n        'description': 'description'\n    })\n```\n\n#### [create_replace_revision](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L50)\n```\nArgSpec\n    Args: metadata, permission\n    Defaults: metadata={}, permission=public\n```\n\nCreate a revision on the view, which when applied, will replace the data.\n\nArgs:\n```\n    metadata (dict): The metadata to change; these changes will be applied when the revision\n        is applied\n    permission (string): 'public' or 'private'\n```\nReturns:\n```\n    Revision The new revision, or an error\n```\nExamples:\n```\n    \u003e\u003e\u003e view.revisions.create_replace_revision(metadata = {'name': 'new dataset name', 'description': 'updated description'})\n```\n\n#### [create_update_revision](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L71)\n```\nArgSpec\n    Args: metadata, permission\n    Defaults: metadata={}, permission=public\n```\n\nCreate a revision on the view, which when applied, will update the data\nrather than replacing it.\n\nThis is an upsert; if there is a rowId defined and you have duplicate ID values,\nthose rows will be updated. Otherwise they will be appended.\n\nArgs:\n```\n    metadata (dict): The metadata to change; these changes will be applied when the revision is applied\n    permission (string): 'public' or 'private'\n```\n\nReturns:\n```\n    Revision The new revision, or an error\n```\n\nExamples:\n```python\n    view.revisions.create_update_revision(metadata = {\n        'name': 'new dataset name',\n        'description': 'updated description'\n    })\n```\n\n#### [create_using_config](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L165)\n```\nArgSpec\n    Args: config\n```\n\nCreate a revision for the given dataset.\n\n#### [list](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L35)\n\n\nList all the revisions on the view\n\nReturns:\n```\n    list[Revision]\n```\n\n#### [lookup](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L146)\n```\nArgSpec\n    Args: revision_seq\n```\n\nLookup a revision within the view based on the sequence number\n\nArgs:\n```\n    revision_seq (int): The sequence number of the revision to lookup\n```\n\nReturns:\n```\n    Revision The Revision resulting from this API call, or an error\n```\n\n### [Revision](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L178)\n```\nArgSpec\n    Args: auth, response, parent\n```\n\nA revision is a change to a dataset\n\n#### [apply](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L383)\n```\nArgSpec\n    Args: output_schema\n```\n\nApply the Revision to the view that it was opened on\n\nArgs:\n```\n    output_schema (OutputSchema): Optional output schema. If your revision includes\n        data changes, this should be included. If it is a metadata only revision,\n        then you will not have an output schema, and you do not need to pass anything\n        here\n```\n\nReturns:\n```\n    Job\n```\n\nExamples:\n```\njob = revision.apply(output_schema = my_output_schema)\n```\n\n#### [create_upload](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L183)\n```\nArgSpec\n    Args: filename, parse_options\n    Defaults: filename={}\n```\n\nCreate an upload within this revision\n\nArgs:\n```\n    filename (str): The name of the file to upload\n```\nReturns:\n```\n    Source: Returns the new Source The Source created by this API call, or an error\n```\n\n#### [discard](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L329)\n\n\nDiscard this open revision.\n\nReturns:\n```\n    Revision The closed Revision or an error\n```\n\n#### [list_operations](https://github.com/socrata/socrata-py/blob/master//socrata/resource.py#L146)\n\n\nGet a list of the operations that you can perform on this\nobject. These map directly onto what's returned from the API\nin the `links` section of each resource\n\n#### [open_in_browser](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L447)\n\n\nOpen this revision in your browser, this will open a window\n\n#### [plan](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L340)\n\n\nReturn the list of operations this revision will make when it is applied\n\nReturns:\n```\n    dict\n```\n\n#### [set_notes](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L301)\n```\nArgSpec\n    Args: notes\n```\n\nSet any notes on the revision. The notes is displayed in the dataset\nchangelog if the dataset is enrolled in archiving. If it is not enrolled\nin archiving, this has no effect.\n\nArgs:\n```\n    notes (string): The change notes\n```\n\nReturns:\n```\n    Revision The updated Revision as a result of this API call, or an error\n```\n\nExamples:\n```python\n    revision = revision.set_notes(\"Just updating my dataset\")\n```\n\n#### [set_output_schema](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L279)\n```\nArgSpec\n    Args: output_schema_id\n```\n\nSet the output schema id on the revision. This is what will get applied when\nthe revision is applied if no ouput schema is explicitly supplied\n\nArgs:\n```\n    output_schema_id (int): The output schema id\n```\n\nReturns:\n```\n    Revision The updated Revision as a result of this API call, or an error\n```\n\nExamples:\n```python\n    revision = revision.set_output_schema(42)\n```\n\n#### [source_as_blob](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L240)\n```\nArgSpec\n    Args: filename, parse_options\n    Defaults: filename={}\n```\n\nCreate a source from a file that should remain unparsed\n\n#### [source_from_agent](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L228)\n```\nArgSpec\n    Args: agent_uid, namespace, path, parse_options, parameters\n    Defaults: agent_uid={}, namespace={}\n```\n\nCreate a source from a connection agent in this revision\n\n#### [source_from_dataset](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L219)\n```\nArgSpec\n    Args: parse_options\n    Defaults: parse_options={}\n```\n\nCreate a dataset source within this revision\n\n#### [source_from_url](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L201)\n```\nArgSpec\n    Args: url, parse_options\n    Defaults: url={}\n```\n\nCreate a URL source\n\nArgs:\n```\n    url (str): The URL to create the dataset from\n```\nReturns:\n```\n    Source: Returns the new Source The Source created by this API call, or an error\n```\n\n#### [ui_url](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L432)\n\n\nThis is the URL to the landing page in the UI for this revision\n\nReturns:\n```\n    url (str): URL you can paste into a browser to view the revision UI\n```\n\n#### [update](https://github.com/socrata/socrata-py/blob/master//socrata/revisions.py#L352)\n```\nArgSpec\n    Args: body\n```\n\nSet the metadata to be applied to the view\nwhen this revision is applied\n\nArgs:\n```\n    body (dict): The changes to make to this revision\n```\n\nReturns:\n```\n    Revision The updated Revision as a result of this API call, or an error\n```\n\nExamples:\n```python\n    revision = revision.update({\n        'metadata': {\n            'name': 'new name',\n            'description': 'new description'\n        }\n    })\n```\n\n### [Sources](https://github.com/socrata/socrata-py/blob/master//socrata/sources.py#L15)\n```\nArgSpec\n    Args: auth\n```\n\n\n\n#### [create_upload](https://github.com/socrata/socrata-py/blob/master//socrata/sources.py#L40)\n```\nArgSpec\n    Args: filename\n```\n\nCreate a new source. Takes a `body` param, which must contain a `filename`\nof the file.\n\nArgs:\n```\n    filename (str): The name of the file you are uploading\n```\n\nReturns:\n```\n    Source: Returns the new Source\n```\n\nExamples:\n```python\n    upload = revision.create_upload('foo.csv')\n```\n\n#### [lookup](https://github.com/socrata/socrata-py/blob/master//socrata/sources.py#L21)\n```\nArgSpec\n    Args: source_id\n```\n\nLookup a source\n\nArgs:\n```\n    source_id (int): The id\n```\n\nReturns:\n```\n    Source: Returns the new Source The Source resulting from this API call, or an error\n```\n\n### [Source](https://github.com/socrata/socrata-py/blob/master//socrata/sources.py#L122)\n```\nArgSpec\n    Args: auth, response, parent\n```\n\n\n\n#### [add_to_revision](https://github.com/socrata/socrata-py/blob/master//socrata/sources.py#L466)\n```\nArgSpec\n    Args: revision\n```\n\nAssociate this Source with the given revision.\n\n#### [blob](https://github.com/socrata/socrata-py/blob/master//socrata/sources.py#L237)\n```\nArgSpec\n    Args: file_handle\n```\n\nUploads a Blob dataset. A blob is a file that will not be parsed as a data file,\nie: an image, video, etc.\n\n\nReturns:\n```\n    Source: Returns the new Source\n```\n\nExamples:\n```python\n    with open('my-blob.jpg', 'rb') as f:\n        upload = upload.blob(f)\n```\n\n#### [change_parse_option](https://github.com/socrata/socrata-py/blob/master//socrata/builders/parse_options.py#L15)\n```\nArgSpec\n    Args: name\n```\n\nChange a parse option on the source.\n\nIf there are not yet bytes uploaded, these parse options will be used\nin order to parse the file.\n\nIf there are already bytes uploaded, this will trigger a re-parsing of\nthe file, and consequently a new InputSchema will be created. You can call\n`source.latest_input()` to get the newest one.\n\nParse options are:\nheader_count (int): the number of rows considered a header\ncolumn_header (int): the one based index of row to use to generate the header\nencoding (string): defaults to guessing the encoding, but it can be explicitly set\ncolumn_separator (string): For CSVs, this defaults to \",\", and for TSVs \"       \", but you can use a custom separator\nquote_char (string): Character used to quote values that should be escaped. Defaults to \"\"\"\n\nArgs:\n```\n    name (string): One of the options above, ie: \"column_separator\" or \"header_count\"\n```\n\nReturns:\n```\n    change (ParseOptionChange): implements a `.to(value)` function which you call to set the value\n```\n\nFor our example, assume we have this dataset\n\n```\nThis is my cool dataset\nA, B, C\n1, 2, 3\n4, 5, 6\n```\n\nWe want to say that the first 2 rows are headers, and the second of those 2\nrows should be used to make the column header. We would do that like so:\n\nExamples:\n```python\n    source = source            .change_parse_option('header_count').to(2)            .change_parse_option('column_header').to(2)            .run()\n```\n\n#### [csv](https://github.com/socrata/socrata-py/blob/master//socrata/sources.py#L261)\n```\nArgSpec\n    Args: file_handle\n```\n\nUpload a CSV, returns the new input schema.\n\nArgs:\n```\n    file_handle: The file handle, as returned by the python function `open()`\n\n    max_retries (integer): Optional retry limit per chunk in the upload. Defaults to 5.\n    backoff_seconds (integer): Optional amount of time to backoff upon a chunk upload failure. Defaults to 2.\n```\n\nReturns:\n```\n    Source: Returns the new Source\n```\n\nExamples:\n```python\n    with open('my-file.csv', 'rb') as f:\n        upload = upload.csv(f)\n```\n\n#### [df](https://github.com/socrata/socrata-py/blob/master//socrata/sources.py#L438)\n```\nArgSpec\n    Args: dataframe\n```\n\nUpload a pandas DataFrame, returns the new source.\n\nArgs:\n```\n    file_handle: The file handle, as returned by the python function `open()`\n\n    max_retries (integer): Optional retry limit per chunk in the upload. Defaults to 5.\n    backoff_seconds (integer): Optional amount of time to backoff upon a chunk upload failure. Defaults to 2.\n```\n\nReturns:\n```\n    Source: Returns the new Source\n```\n\nExamples:\n```python\n    import pandas\n    df = pandas.read_csv('test/fixtures/simple.csv')\n    upload = upload.df(df)\n```\n\n#### [geojson](https://github.com/socrata/socrata-py/blob/master//socrata/sources.py#L412)\n```\nArgSpec\n    Args: file_handle\n```\n\nUpload a geojson file, returns the new input schema.\n\nArgs:\n```\n    file_handle: The file handle, as returned by the python function `open()`\n\n    max_retries (integer): Optional retry limit per chunk in the upload. Defaults to 5.\n    backoff_seconds (integer): Optional amount of time to backoff upon a chunk upload failure. Defaults to 2.\n```\n\nReturns:\n```\n    Source: Returns the new Source\n```\n\nExamples:\n```python\n    with open('my-geojson-file.geojson', 'rb') as f:\n        upload = upload.geojson(f)\n```\n\n#### [kml](https://github.com/socrata/socrata-py/blob/master//socrata/sources.py#L386)\n```\nArgSpec\n    Args: file_handle\n```\n\nUpload a KML file, returns the new input schema.\n\nArgs:\n```\n    file_handle: The file handle, as returned by the python function `open()`\n\n    max_retries (integer): Optional retry limit per chunk in the upload. Defaults to 5.\n    backoff_seconds (integer): Optional amount of time to backoff upon a chunk upload failure. Defaults to 2.\n```\n\nReturns:\n```\n    Source: Returns the new Source\n```\n\nExamples:\n```python\n    with open('my-kml-file.kml', 'rb') as f:\n        upload = upload.kml(f)\n```\n\n#### [list_operations](https://github.com/socrata/socrata-py/blob/master//socrata/resource.py#L146)\n\n\nGet a list of the operations that you can perform on this\nobject. These map directly onto what's returned from the API\nin the `links` section of each resource\n\n#### [load](https://github.com/socrata/socrata-py/blob/master//socrata/sources.py#L206)\n\n\nForces the source to load, if it's a view source.\n\nReturns:\n```\n    Source: Returns the new Source\n```\n\n#### [open_in_browser](https://github.com/socrata/socrata-py/blob/master//socrata/sources.py#L547)\n\n\nOpen this source in your browser, this will open a window\n\n#### [shapefile](https://github.com/socrata/socrata-py/blob/master//socrata/sources.py#L361)\n```\nArgSpec\n    Args: file_handle\n```\n\nUpload a Shapefile, returns the new input schema.\n\nArgs:\n```\n    file_handle: The file handle, as returned by the python function `open()`\n\n    max_retries (integer): Optional retry limit per chunk in the upload. Defaults to 5.\n    backoff_seconds (integer): Optional amount of time to backoff upon a chunk upload failure. Defaults to 2.\n```\n\nReturns:\n```\n    Source: Returns the new Source\n```\n\nExamples:\n```python\n    with open('my-shapefile-archive.zip', 'rb') as f:\n        upload = upload.shapefile(f)\n```\n\n#### [tsv](https://github.com/socrata/socrata-py/blob/master//socrata/sources.py#L336)\n```\nArgSpec\n    Args: file_handle\n```\n\nUpload a TSV, returns the new input schema.\n\nArgs:\n```\n    file_handle: The file handle, as returned by the python function `open()`\n\n    max_retries (integer): Optional retry limit per chunk in the upload. Defaults to 5.\n    backoff_seconds (integer): Optional amount of time to backoff upon a chunk upload failure. Defaults to 2.\n```\n\nReturns:\n```\n    Source: Returns the new Source\n```\n\nExamples:\n```python\n    with open('my-file.tsv', 'rb') as f:\n        upload = upload.tsv(f)\n```\n\n#### [ui_url](https://github.com/socrata/socrata-py/blob/master//socrata/sources.py#L530)\n\n\nThis is the URL to the landing page in the UI for the sources\n\nReturns:\n```\n    url (str): URL you can paste into a browser to view the source UI\n```\n\n#### [wait_for_finish](https://github.com/socrata/socrata-py/blob/master//socrata/sources.py#L515)\n```\nArgSpec\n    Args: progress, timeout, sleeptime\n    Defaults: progress=\u003cfunction noop at 0x7fa34fb57040\u003e, timeout=43200, sleeptime=1\n```\n\nWait for this data source to finish transforming and validating. Accepts a progress function\nand a timeout.\n\nDefault timeout is 12 hours\n\n#### [wait_for_schema](https://github.com/socrata/socrata-py/blob/master//socrata/sources.py#L500)\n```\nArgSpec\n    Args: progress, timeout, sleeptime\n    Defaults: progress=\u003cfunction noop at 0x7fa34fb57040\u003e, timeout=43200, sleeptime=1\n```\n\nWait for this data source to have at least one schema present. Accepts a progress function\nand a timeout.\n\nDefault timeout is 12 hours\n\n#### [xls](https://github.com/socrata/socrata-py/blob/master//socrata/sources.py#L286)\n```\nArgSpec\n    Args: file_handle\n```\n\nUpload an XLS, returns the new input schema\n\nArgs:\n```\n    file_handle: The file handle, as returned by the python function `open()`\n\n    max_retries (integer): Optional retry limit per chunk in the upload. Defaults to 5.\n    backoff_seconds (integer): Optional amount of time to backoff upon a chunk upload failure. Defaults to 2.\n```\n\nReturns:\n```\n    Source: Returns the new Source\n```\n\nExamples:\n```python\n    with open('my-file.xls', 'rb') as f:\n        upload = upload.xls(f)\n```\n\n#### [xlsx](https://github.com/socrata/socrata-py/blob/master//socrata/sources.py#L311)\n```\nArgSpec\n    Args: file_handle\n```\n\nUpload an XLSX, returns the new input schema.\n\nArgs:\n```\n    file_handle: The file handle, as returned by the python function `open()`\n\n    max_retries (integer): Optional retry limit per chunk in the upload. Defaults to 5.\n    backoff_seconds (integer): Optional amount of time to backoff upon a chunk upload failure. Defaults to 2.\n```\n\nReturns:\n```\n    Source: Returns the new Source\n```\n\nExamples:\n```python\n    with open('my-file.xlsx', 'rb') as f:\n        upload = upload.xlsx(f)\n```\n\n### [Configs](https://github.com/socrata/socrata-py/blob/master//socrata/configs.py#L8)\n```\nArgSpec\n    Args: auth\n```\n\n\n\n#### [create](https://github.com/socrata/socrata-py/blob/master//socrata/configs.py#L14)\n```\nArgSpec\n    Args: name, data_action, parse_options, columns\n```\n\nCreate a new ImportConfig. See http://docs.socratapublishing.apiary.io/\nImportConfig section for what is supported in `data_action`, `parse_options`,\nand `columns`.\n\n#### [list](https://github.com/socrata/socrata-py/blob/master//socrata/configs.py#L41)\n\n\nList all the ImportConfigs on this domain\n\n#### [lookup](https://github.com/socrata/socrata-py/blob/master//socrata/configs.py#L32)\n```\nArgSpec\n    Args: name\n```\n\nObtain a single ImportConfig by name\n\n### [Config](https://github.com/socrata/socrata-py/blob/master//socrata/configs.py#L50)\n```\nArgSpec\n    Args: auth, response, parent\n```\n\n\n\n#### [change_parse_option](https://github.com/socrata/socrata-py/blob/master//socrata/builders/parse_options.py#L15)\n```\nArgSpec\n    Args: name\n```\n\nChange a parse option on the source.\n\nIf there are not yet bytes uploaded, these parse options will be used\nin order to parse the file.\n\nIf there are already bytes uploaded, this will trigger a re-parsing of\nthe file, and consequently a new InputSchema will be created. You can call\n`source.latest_input()` to get the newest one.\n\nParse options are:\nheader_count (int): the number of rows considered a header\ncolumn_header (int): the one based index of row to use to generate the header\nencoding (string): defaults to guessing the encoding, but it can be explicitly set\ncolumn_separator (string): For CSVs, this defaults to \",\", and for TSVs \"       \", but you can use a custom separator\nquote_char (string): Character used to quote values that should be escaped. Defaults to \"\"\"\n\nArgs:\n```\n    name (string): One of the options above, ie: \"column_separator\" or \"header_count\"\n```\n\nReturns:\n```\n    change (ParseOptionChange): implements a `.to(value)` function which you call to set the value\n```\n\nFor our example, assume we have this dataset\n\n```\nThis is my cool dataset\nA, B, C\n1, 2, 3\n4, 5, 6\n```\n\nWe want to say that the first 2 rows are headers, and the second of those 2\nrows should be used to make the column header. We would do that like so:\n\nExamples:\n```python\n    source = source            .change_parse_option('header_count').to(2)            .change_parse_option('column_header').to(2)            .run()\n```\n\n#### [create_revision](https://github.com/socrata/socrata-py/blob/master//socrata/configs.py#L68)\n```\nArgSpec\n    Args: fourfour\n```\n\nCreate a new Revision in the context of this ImportConfig.\nSources that happen in this Revision will take on the values\nin this Config.\n\n#### [delete](https://github.com/socrata/socrata-py/blob/master//socrata/configs.py#L51)\n\n\nDelete this ImportConfig. Note that this cannot be undone.\n\n#### [list_operations](https://github.com/socrata/socrata-py/blob/master//socrata/resource.py#L146)\n\n\nGet a list of the operations that you can perform on this\nobject. These map directly onto what's returned from the API\nin the `links` section of each resource\n\n#### [update](https://github.com/socrata/socrata-py/blob/master//socrata/configs.py#L57)\n```\nArgSpec\n    Args: body\n```\n\nMutate this ImportConfig in place. Subsequent revisions opened against this\nImportConfig will take on its new value.\n\n### [InputSchema](https://github.com/socrata/socrata-py/blob/master//socrata/input_schema.py#L7)\n```\nArgSpec\n    Args: auth, response, parent\n```\n\nThis represents a schema exactly as it appeared in the source\n\n#### [get_latest_output_schema](https://github.com/socrata/socrata-py/blob/master//socrata/input_schema.py#L53)\n\n\nNote that this does not make an API request\n\nReturns:\n    output_schema (OutputSchema): Returns the latest output schema\n\n#### [latest_output](https://github.com/socrata/socrata-py/blob/master//socrata/input_schema.py#L25)\n\n\nGet the latest (most recently created) OutputSchema\nwhich descends from this InputSchema\n\nReturns:\n    OutputSchema\n\n#### [list_operations](https://github.com/socrata/socrata-py/blob/master//socrata/resource.py#L146)\n\n\nGet a list of the operations that you can perform on this\nobject. These map directly onto what's returned from the API\nin the `links` section of each resource\n\n#### [transform](https://github.com/socrata/socrata-py/blob/master//socrata/input_schema.py#L11)\n```\nArgSpec\n    Args: body\n```\n\nTransform this InputSchema into an Output. Returns the\nnew OutputSchema. Note that this call is async - the data\nmay still be transforming even though the OutputSchema is\nreturned. See OutputSchema.wait_for_finish to block until\nthe\n\n#### [wait_for_schema](https://github.com/socrata/socrata-py/blob/master//socrata/input_schema.py#L38)\n```\nArgSpec\n    Args: progress, timeout, sleeptime\n    Defaults: progress=\u003cfunction noop at 0x7fa34fb57040\u003e, timeout=43200, sleeptime=1\n```\n\nWait for this data source to have at least one schema present. Accepts a progress function\nand a timeout.\n\nDefault timeout is 12 hours\n\n### [OutputSchema](https://github.com/socrata/socrata-py/blob/master//socrata/output_schema.py#L51)\n\n\nThis is data as transformed from an InputSchema\n\n#### [add_column](https://github.com/socrata/socrata-py/blob/master//socrata/output_schema.py#L223)\n```\nArgSpec\n    Args: field_name, display_name, transform_expr, description\n```\n\nAdd a column\n\nArgs:\n```\n    field_name (str): The column's field_name, must be unique\n    display_name (str): The columns display name\n    transform_expr (str): SoQL expression to evaluate and fill the column with data from\n    description (str): Optional column description\n```\n\nReturns:\n```\n    output_schema (OutputSchema): Returns self for easy chaining\n```\n\nExamples:\n```python\nnew_output_schema = output\n    # Add a new column, which is computed from the `celsius` column\n    .add_column('fahrenheit', 'Degrees (Fahrenheit)', '(to_number(`celsius`) * (9 / 5)) + 32', 'the temperature in celsius')\n    # Add a new column, which is computed from the `celsius` column\n    .add_column('kelvin', 'Degrees (Kelvin)', '(to_number(`celsius`) + 273.15')\n    .run()\n```\n\n#### [any_failed](https://github.com/socrata/socrata-py/blob/master//socrata/output_schema.py#L78)\n\n\nThis is probably not the function you are looking for.\n\nThis returns whether or not any transform in this output schema has failed. \"Failed\" in this\ncase means an internal error (which is unexpected), not a data error (which is expected). This\nfunction will wait for processing to complete if it hasn't yet.\n\nFor data errors:\n\n    Tell whether or not there are data errors\n        output_schema.any_errors()\n    Get the count of data errors\n        output_schema.attributes['error_count']\n    Get the errors themselves\n        output_schema.schema_errors(offset = 0, limit = 20)\n\n#### [build_config](https://github.com/socrata/socrata-py/blob/master//socrata/output_schema.py#L63)\n```\nArgSpec\n    Args: name, data_action\n```\n\nCreate a new ImportConfig from this OutputSchema. See the API\ndocs for what an ImportConfig is and why they're useful\n\n#### [change_column_metadata](https://github.com/socrata/socrata-py/blob/master//socrata/output_schema.py#L286)\n```\nArgSpec\n    Args: field_name, attribute\n```\n\nChange the column metadata. This returns a ColumnChange,\nwhich implements a `.to` function, which takes the new value to change to\n\nArgs:\n```\n    field_name (str): The column to change\n    attribute (str): The attribute of the column to change\n```\n\nReturns:\n```\n    change (TransformChange): The transform change, which implements the `.to` function\n```\n\nExamples:\n```python\n    new_output_schema = output\n        # Change the field_name of date to the_date\n        .change_column_metadata('date', 'field_name').to('the_date')\n        # Change the description of the celsius column\n        .change_column_metadata('celsius', 'description').to('the temperature in celsius')\n        # Change the display name of the celsius column\n        .change_column_metadata('celsius', 'display_name').to('Degrees (Celsius)')\n        .run()\n```\n\n#### [change_column_transform](https://github.com/socrata/socrata-py/blob/master//socrata/output_schema.py#L316)\n```\nArgSpec\n    Args: field_name\n```\n\nChange the column transform. This returns a TransformChange,\nwhich implements a `.to` function, which takes a transform expression.\n\nArgs:\n```\n    field_name (str): The column to change\n```\n\nReturns:\n```\n    change (TransformChange): The transform change, which implements the `.to` function\n```\n\nExamples:\n```python\n    new_output_schema = output\n        .change_column_transform('the_date').to('to_fixed_timestamp(`date`)')\n        # Make the celsius column all numbers\n        .change_column_transform('celsius').to('to_number(`celsius`)')\n        # Add a new column, which is computed from the `celsius` column\n        .add_column('fahrenheit', 'Degrees (Fahrenheit)', '(to_number(`celsius`) * (9 / 5)) + 32', 'the temperature in celsius')\n        .run()\n```\n\n#### [drop_column](https://github.com/socrata/socrata-py/blob/master//socrata/output_schema.py#L262)\n```\nArgSpec\n    Args: field_name\n```\n\nDrop the column\n\nArgs:\n```\n    field_name (str): The column to drop\n```\n\nReturns:\n```\n    output_schema (OutputSchema): Returns self for easy chaining\n```\n\nExamples:\n```python\n    new_output_schema = output\n        .drop_column('foo')\n        .run()\n```\n\n#### [list_operations](https://github.com/socrata/socrata-py/blob/master//socrata/resource.py#L146)\n\n\nGet a list of the operations that you can perform on this\nobject. These map directly onto what's returned from the API\nin the `links` section of each resource\n\n#### [rows](https://github.com/socrata/socrata-py/blob/master//socrata/output_schema.py#L140)\n```\nArgSpec\n    Args: offset, limit\n    Defaults: offset=0, limit=500\n```\n\nGet the rows for this OutputSchema. Acceps `offset` and `limit` params\nfor paging through the data.\n\n#### [run](https://github.com/socrata/socrata-py/blob/master//socrata/output_schema.py#L372)\n\n\nRun all adds, drops, and column changes.\n\n\nReturns:\n```\n    OutputSchema\n```\n\nExamples:\n```python\n    new_output_schema = output\n        # Change the field_name of date to the_date\n        .change_column_metadata('date', 'field_name').to('the_date')\n        # Change the description of the celsius column\n        .change_column_metadata('celsius', 'description').to('the temperature in celsius')\n        # Change the display name of the celsius column\n        .change_column_metadata('celsius', 'display_name').to('Degrees (Celsius)')\n        # Change the transform of the_date column to to_fixed_timestamp(`date`)\n        .change_column_transform('the_date').to('to_fixed_timestamp(`date`)')\n        # Make the celsius column all numbers\n        .change_column_transform('celsius').to('to_number(`celsius`)')\n        # Add a new column, which is computed from the `celsius` column\n        .add_column('fahrenheit', 'Degrees (Fahrenheit)', '(to_number(`celsius`) * (9 / 5)) + 32', 'the temperature in celsius')\n        .run()\n```\n\n#### [schema_errors](https://github.com/socrata/socrata-py/blob/master//socrata/output_schema.py#L148)\n```\nArgSpec\n    Args: offset, limit\n    Defaults: offset=0, limit=500\n```\n\nGet the errors that resulted in transforming into this output schema.\nAccepts `offset` and `limit` params\n\n#### [schema_errors_csv](https://github.com/socrata/socrata-py/blob/master//socrata/output_schema.py#L155)\n\n\nGet the errors that results in transforming into this output schema\nas a CSV stream.\n\nNote that this returns a Reponse, where Reponse\nis a python requests Reponse object\n\n#### [set_row_id](https://github.com/socrata/socrata-py/blob/master//socrata/output_schema.py#L196)\n```\nArgSpec\n    Args: field_name\n```\n\nSet the row id. Note you must call `validate_row_id` before doing this.\n\nArgs:\n```\n    field_name (str): The column to set as the row id\n```\n\nReturns:\n```\n    OutputSchema\n```\n\nExamples:\n```python\nnew_output_schema = output.set_row_id('the_id_column')\n```\n\n#### [set_sort_by](https://github.com/socrata/socrata-py/blob/master//socrata/output_schema.py#L344)\n\n\nReplace the columns used to sort the dataset. This returns a SortChange,\nwhich implements a `.on` function to add a sort and a `.end_sort` function\nto finish.\n\nIf you do not call this, the OutputSchema will try to preserve any existing\nsorts, which means it will remove sorts on deleted columns or on columns\nwhose transforms are changed.\n\nReturns:\n```\n    change (SortChange): The sort change, which implements the `.on` and `.end_sort` functions\n```\n\nExamples:\n```python\n    new_output_schema = output\n        .set_sort_by()\n        .on('column_one', ascending = True)\n        .on('column_two', ascending = False)\n        .on('column_three') # ascending = True is the default\n        .end_sort()\n        .run()\n```\n\n#### [validate_row_id](https://github.com/socrata/socrata-py/blob/master//socrata/output_schema.py#L170)\n```\nArgSpec\n    Args: field_name\n```\n\nSet the row id. Note you must call `validate_row_id` before doing this.\n\nArgs:\n```\n    field_name (str): The column to validate as the row id\n```\n\nReturns:\n```\n    boolean\n```\n\n#### [wait_for_finish](https://github.com/socrata/socrata-py/blob/master//socrata/output_schema.py#L108)\n```\nArgSpec\n    Args: progress, timeout, sleeptime\n    Defaults: progress=\u003cfunction noop at 0x7fa34fb57040\u003e, timeout=10800, sleeptime=1\n```\n\nWait for this dataset to finish transforming and validating. Accepts a progress function\nand a timeout.\n\nDefault timeout is 3 hours\n\n### [Job](https://github.com/socrata/socrata-py/blob/master//socrata/job.py#L4)\n```\nArgSpec\n    Args: auth, response, parent\n```\n\n\n\n#### [is_complete](https://github.com/socrata/socrata-py/blob/master//socrata/job.py#L10)\n\n\nHas this job finished or failed (or been submitted for approval)\n\n#### [list_operations](https://github.com/socrata/socrata-py/blob/master//socrata/resource.py#L146)\n\n\nGet a list of the operations that you can perform on this\nobject. These map directly onto what's returned from the API\nin the `links` section of each resource\n\n#### [submitted_for_approval](https://github.com/socrata/socrata-py/blob/master//socrata/job.py#L33)\n\n\nHas this job entered the approval queue (rather than finishing or failing)\n\n#### [wait_for_finish](https://github.com/socrata/socrata-py/blob/master//socrata/job.py#L18)\n```\nArgSpec\n    Args: progress, timeout, sleeptime\n    Defaults: progress=\u003cfunction noop at 0x7fa34fb57040\u003e, sleeptime=1\n```\n\nWait for this dataset to finish transforming and validating. Accepts a progress function\nand a timeout.\n\u003c!-- docstop --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsocrata%2Fsocrata-py","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsocrata%2Fsocrata-py","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsocrata%2Fsocrata-py/lists"}