{"id":14959502,"url":"https://github.com/streamlit/files-connection","last_synced_at":"2025-04-19T21:20:45.834Z","repository":{"id":156796658,"uuid":"632632968","full_name":"streamlit/files-connection","owner":"streamlit","description":null,"archived":false,"fork":false,"pushed_at":"2024-06-28T08:01:39.000Z","size":37,"stargazers_count":32,"open_issues_count":5,"forks_count":3,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-10T02:16:06.286Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/streamlit.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-25T20:14:45.000Z","updated_at":"2025-04-07T20:08:15.000Z","dependencies_parsed_at":null,"dependency_job_id":"460e55a4-b298-4551-b20d-4e9574a98560","html_url":"https://github.com/streamlit/files-connection","commit_stats":{"total_commits":23,"total_committers":3,"mean_commits":7.666666666666667,"dds":0.08695652173913049,"last_synced_commit":"4cfffa022c15f00087a101d3c057fc901cb69c79"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamlit%2Ffiles-connection","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamlit%2Ffiles-connection/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamlit%2Ffiles-connection/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamlit%2Ffiles-connection/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/streamlit","download_url":"https://codeload.github.com/streamlit/files-connection/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249807315,"owners_count":21328050,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-24T13:19:51.704Z","updated_at":"2025-04-19T21:20:45.784Z","avatar_url":"https://github.com/streamlit.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Streamlit FilesConnection\n\nConnect to cloud (or local) file storage from your Streamlit app. Powered by `st.experimental_connection()` and [fsspec](https://filesystem-spec.readthedocs.io/en/latest/). Works with Streamlit \u003e= 1.22.\n\nAny fsspec compatible protocol should work, it just needs to be installed. Read more about Streamlit Connections in the\n[official docs](https://docs.streamlit.io/library/api-reference/connections).\n\n## Quickstart\n\nSee the example directory for a fuller example using S3 and/or GCS.\n\n```sh\npip install streamlit\npip install st-files-connection\n```\n\n**Note:** Install from pypi coming soon\n\n```python\nimport streamlit as st\nfrom st_files_connection import FilesConnection\n\n\"# Minimal FilesConnection example\"\nwith st.echo():\n    conn = st.experimental_connection('my_connection', type=FilesConnection)\n\n# Write a file to local directory if it doesn't exist\ntest_file = \"test.txt\"\ntry:\n    _ = conn.read(test_file, input_format='text')\nexcept FileNotFoundError:\n    with conn.open(test_file, \"wt\") as f:\n        f.write(\"Hello, world!\")\n\nwith st.echo():\n    # Read back the contents of the file\n    st.write(conn.read(test_file, input_format='text'))\n```\n\n## Using for cloud file storage\n\nYou can pass the protocol name into `st.experimental_connection()` as the first argument, for any known fsspec protocol:\n\n```python\n# Create an S3 connection\nconn = st.experimental_connection('s3', type=FilesConnection)\n\n# Create a GCS connection\nconn = st.experimental_connection('gcs', type=FilesConnection)\n\n# Create a Weights \u0026 Biases connection\nconn = st.experimental_connection('wandb', type=FilesConnection)\n```\n\nFor cloud file storage tools (or anything that needs config / credentials) you can specify it in two ways:\n\n- Using the native configuration / credential approach of the underlying library (e.g. config file or environment variables)\n- Using [Streamlit secrets](https://docs.streamlit.io/library/advanced-features/secrets-management).\n\nFor Streamlit secrets, create a section called `[connections.\u003cname\u003e]` in your `.streamlit/secrets.toml` file, and add parameters\nthere. You can pass in anything you would pass to an fsspec file system constructor. Additionally:\n\n- For GCS, the contents of secrets are assumed to be the keys to a token file (e.g. it is passed as a `{\"token\":{\u003csecrets\u003e}}` dict)\n\n## Main methods\n\n### read()\n\n`conn.read(\"path/to/file\", input_format=\"text|csv|parquet|json|jsonl\" or None, ttl=None) -\u003e pd.DataFrame`\n\nSpecify a path to file and input format. Optionally specify a TTL for caching.\n\nValid values for `input_format=`:\n\n- `text` returns a string\n- `json` returns a dict or list (depending on the JSON object) - only one object per file is supported\n- `csv`, `parquet`, `jsonl` return a pandas DataFrame\n- `None` will attempt to infer the input format from file extension of `path`\n- Anything else (or unrecognized inferred type) raises a `ValueError`\n\n```python\nconn = st.experimental_connection(\"s3\", type=FilesConnection)\ndf = conn.read(f\"my-s3-bucket/path/to/file.parquet\", input_format='parquet')\nst.dataframe(df)\n```\n\n**Note:** We want to add a `format=` argument to specify output format with more options, contributions welcome!\n\n### open()\n\n`conn.open(\"path/to/file\", mode=\"rb\", *args, **kwargs) -\u003e Iterator[TextIOWrapper | AbstractBufferedFile]`\n\nWorks just like fsspec [AbstractFileSystem.open()](https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem.open).\n\n### fs\n\nUse `conn.fs` to access the [underlying FileSystem object API](https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem).\n\n## Contributing\n\nContributions to this repo are welcome. We are still figuring it out and expect it may get some usage over time. We want to keep the API pretty simple\nand not increase the maintenance surface area too much. If you are interested in helping to maintain it, reach out to us. Thanks for your patience if\nit takes a few days to respond.\n\nThe best way to submit ideas you might want to work on is to open an issue and tag `@sfc-gh-jcarroll` and/or any other listed contributors.\nPlease don't spend a bunch of time working on a PR without checking with us first, since it risks the work being wasted and leaving you frustrated.\n\nAlso note, the Streamlit `experimental_connection()` interface is open for 3rd party packages and we look forward to promoting high quality ones in\nthe ecosystem. If you have an idea that differs from our direction here, we would love for you to fork / clone, build it, and share it with us and\nthe wider community. Thank you!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstreamlit%2Ffiles-connection","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstreamlit%2Ffiles-connection","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstreamlit%2Ffiles-connection/lists"}