{"id":21128759,"url":"https://github.com/frictionlessdata/tableschema-sql-py","last_synced_at":"2025-10-04T01:19:42.138Z","repository":{"id":36609673,"uuid":"40915848","full_name":"frictionlessdata/tableschema-sql-py","owner":"frictionlessdata","description":"Generate SQL tables, load and extract data, based on JSON Table Schema descriptors.","archived":false,"fork":false,"pushed_at":"2023-07-19T17:43:48.000Z","size":328,"stargazers_count":62,"open_issues_count":1,"forks_count":17,"subscribers_count":16,"default_branch":"main","last_synced_at":"2025-09-09T04:57:58.753Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/frictionlessdata.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-08-17T17:50:23.000Z","updated_at":"2024-12-29T01:13:22.000Z","dependencies_parsed_at":"2024-06-18T21:17:45.959Z","dependency_job_id":"b517cc51-4b08-44d1-96a7-97e265d4e082","html_url":"https://github.com/frictionlessdata/tableschema-sql-py","commit_stats":{"total_commits":231,"total_committers":14,"mean_commits":16.5,"dds":0.4329004329004329,"last_synced_commit":"eee264d1f90c30a8dcf1871b1a55238bb4c855f1"},"previous_names":["frictionlessdata/jsontableschema-sql-py"],"tags_count":37,"template":false,"template_full_name":null,"purl":"pkg:github/frictionlessdata/tableschema-sql-py","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frictionlessdata%2Ftableschema-sql-py","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frictionlessdata%2Ftableschema-sql-py/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frictionlessdata%2Ftableschema-sql-py/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frictionlessdata%2Ftableschema-sql-py/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/frictionlessdata","download_url":"https://codeload.github.com/frictionlessdata/tableschema-sql-py/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frictionlessdata%2Ftableschema-sql-py/sbom","scorecard":{"id":411443,"data":{"date":"2025-08-11","repo":{"name":"github.com/frictionlessdata/tableschema-sql-py","commit":"eee264d1f90c30a8dcf1871b1a55238bb4c855f1"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.9,"checks":[{"name":"Code-Review","score":1,"reason":"Found 5/30 approved changesets -- score normalized to 1","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/general.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/general.yml:41: update your workflow using https://app.stepsecurity.io/secureworkflow/frictionlessdata/tableschema-sql-py/general.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/general.yml:43: update your workflow using https://app.stepsecurity.io/secureworkflow/frictionlessdata/tableschema-sql-py/general.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/general.yml:57: update your workflow using https://app.stepsecurity.io/secureworkflow/frictionlessdata/tableschema-sql-py/general.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/general.yml:65: update your workflow using https://app.stepsecurity.io/secureworkflow/frictionlessdata/tableschema-sql-py/general.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/general.yml:67: update your workflow using https://app.stepsecurity.io/secureworkflow/frictionlessdata/tableschema-sql-py/general.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/general.yml:78: update your workflow using https://app.stepsecurity.io/secureworkflow/frictionlessdata/tableschema-sql-py/general.yml/main?enable=pin","Warn: pipCommand not pinned by hash: .github/workflows/general.yml:49","Warn: pipCommand not pinned by hash: .github/workflows/general.yml:72","Warn: pipCommand not pinned by hash: .github/workflows/general.yml:73","Info:   0 out of   4 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   2 third-party GitHubAction dependencies pinned","Info:   0 out of   3 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE.md:0","Info: FSF or OSI recognized license: MIT License: LICENSE.md:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 9 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-18T22:48:20.055Z","repository_id":36609673,"created_at":"2025-08-18T22:48:20.055Z","updated_at":"2025-08-18T22:48:20.055Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278252444,"owners_count":25956300,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-03T02:00:06.070Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-20T05:02:25.112Z","updated_at":"2025-10-04T01:19:42.102Z","avatar_url":"https://github.com/frictionlessdata.png","language":"Python","readme":"# tableschema-sql-py\n\n[![Travis](https://img.shields.io/travis/frictionlessdata/tableschema-sql-py/master.svg)](https://travis-ci.org/frictionlessdata/tableschema-sql-py)\n[![Coveralls](http://img.shields.io/coveralls/frictionlessdata/tableschema-sql-py/master.svg)](https://coveralls.io/r/frictionlessdata/tableschema-sql-py?branch=master)\n[![PyPi](https://img.shields.io/pypi/v/tableschema-sql.svg)](https://pypi.python.org/pypi/tableschema-sql)\n[![Github](https://img.shields.io/badge/github-master-brightgreen)](https://github.com/frictionlessdata/tableschema-sql-py)\n[![Gitter](https://img.shields.io/gitter/room/frictionlessdata/chat.svg)](https://gitter.im/frictionlessdata/chat)\n\nGenerate and load SQL tables based on [Table Schema](http://specs.frictionlessdata.io/table-schema/) descriptors.\n\n## Features\n\n- implements `tableschema.Storage` interface\n- provides additional features like indexes and updating\n\n## Contents\n\n\u003c!--TOC--\u003e\n\n  - [Getting Started](#getting-started)\n    - [Installation](#installation)\n  - [Documentation](#documentation)\n  - [API Reference](#api-reference)\n    - [`Storage`](#storage)\n  - [Contributing](#contributing)\n  - [Changelog](#changelog)\n\n\u003c!--TOC--\u003e\n\n## Getting Started\n\n### Installation\n\nThe package use semantic versioning. It means that major versions  could include breaking changes. It's highly recommended to specify `package` version range in your `setup/requirements` file e.g. `package\u003e=1.0,\u003c2.0`.\n\n```bash\npip install tableschema-sql\n```\n\n## Documentation\n\n```python\nfrom datapackage import Package \nfrom tableschema import Table\nfrom sqlalchemy import create_engine\n\n# Create sqlalchemy engine\nengine = create_engine('sqlite://')\n\n# Save package to SQL\npackage = Package('datapackage.json')\npackage.save(storage='sql', engine=engine)\n\n# Load package from SQL\npackage = Package(storage='sql', engine=engine)\npackage.resources\n```\n\n## API Reference\n\n### `Storage`\n```python\nStorage(self, engine, dbschema=None, prefix='', reflect_only=None, autoincrement=None)\n```\nSQL storage\n\nPackage implements\n[Tabular Storage](https://github.com/frictionlessdata/tableschema-py#storage)\ninterface (see full documentation on the link):\n\n![Storage](https://i.imgur.com/RQgrxqp.png)\n\n\u003e Only additional API is documented\n\n__Arguments__\n- __engine (object)__: `sqlalchemy` engine\n- __dbschema (str)__: name of database schema\n- __prefix (str)__: prefix for all buckets\n- __reflect_only (callable)__:\n        a boolean predicate to filter the list of table names when reflecting\n- __autoincrement (str/dict)__:\n        add autoincrement column at the beginning.\n          - if a string it's an autoincrement column name\n          - if a dict it's an autoincrements mapping with column\n            names indexed by bucket names, for example,\n            `{'bucket1': 'id', 'bucket2': 'other_id}`\n\n\n#### `storage.create`\n```python\nstorage.create(self, bucket, descriptor, force=False, indexes_fields=None)\n```\nCreate bucket\n\n__Arguments__\n- __indexes_fields (str[])__:\n        list of tuples containing field names, or list of such lists\n\n\n#### `storage.write`\n```python\nstorage.write(self, bucket, rows, keyed=False, as_generator=False, update_keys=None, buffer_size=1000, use_bloom_filter=True)\n```\nWrite to bucket\n\n__Arguments__\n- __keyed (bool)__:\n        accept keyed rows\n- __as_generator (bool)__:\n        returns generator to provide writing control to the client\n- __update_keys (str[])__:\n        update instead of inserting if key values match existent rows\n- __buffer_size (int=1000)__:\n        maximum number of rows to try and write to the db in one batch\n- __use_bloom_filter (bool=True)__:\n        should we use a bloom filter to optimize DB update performance\n        (in exchange for some setup time)\n\n\n## Contributing\n\n\u003e The project follows the [Open Knowledge International coding standards](https://github.com/okfn/coding-standards).\n\nRecommended way to get started is to create and activate a project virtual environment.\nTo install package and development dependencies into active environment:\n\n```bash\n$ make install\n```\n\nTo run tests with linting and coverage:\n\n```bash\n$ make test\n```\n\n## Changelog\n\nHere described only breaking and the most important changes. The full changelog and documentation for all released versions could be found in nicely formatted [commit history](https://github.com/frictionlessdata/tableschema-sql-py/commits/master).\n\n#### v1.3\n\n- Implemented constraints loading to a database\n\n#### v1.2\n\n- Add option to configure buffer size, bloom filter use (#77)\n\n#### v1.1\n\n- Added support for the `autoincrement` parameter to be a mapping\n- Fixed autoincrement support for SQLite and MySQL\n\n#### v1.0\n\n- Initial driver implementation.\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrictionlessdata%2Ftableschema-sql-py","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffrictionlessdata%2Ftableschema-sql-py","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrictionlessdata%2Ftableschema-sql-py/lists"}