{"id":13609084,"url":"https://github.com/plasticityai/supersqlite","last_synced_at":"2025-04-05T04:14:09.027Z","repository":{"id":97491611,"uuid":"156831686","full_name":"plasticityai/supersqlite","owner":"plasticityai","description":"A supercharged SQLite library for Python","archived":false,"fork":false,"pushed_at":"2020-05-09T09:06:06.000Z","size":25912,"stargazers_count":719,"open_issues_count":8,"forks_count":24,"subscribers_count":13,"default_branch":"master","last_synced_at":"2024-04-22T12:32:56.713Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/plasticityai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-11-09T08:20:26.000Z","updated_at":"2024-04-02T17:40:28.000Z","dependencies_parsed_at":"2024-01-05T21:29:05.272Z","dependency_job_id":null,"html_url":"https://github.com/plasticityai/supersqlite","commit_stats":null,"previous_names":[],"tags_count":58,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plasticityai%2Fsupersqlite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plasticityai%2Fsupersqlite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plasticityai%2Fsupersqlite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plasticityai%2Fsupersqlite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/plasticityai","download_url":"https://codeload.github.com/plasticityai/supersqlite/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247284951,"owners_count":20913704,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T19:01:32.347Z","updated_at":"2025-04-05T04:14:09.008Z","avatar_url":"https://github.com/plasticityai.png","language":"Python","funding_links":[],"categories":["数据库 Drivers","Python","Database Drivers","Database Drivers [🔝](#readme)"],"sub_categories":["Automatic Plotting"],"readme":"\u003cdiv align=\"center\"\u003e\u003cimg src=\"https://gitlab.com/Plasticity/supersqlite/raw/master/images/SuperSQLite.png\" alt=\"supersqlite\" height=\"70\"\u003e\u003c/div\u003e\n\n## \u003cdiv align=\"center\"\u003eSuperSQLite: a supercharged SQLite library for Python\u003cbr /\u003e\u003cbr /\u003e[![pipeline status](https://gitlab.com/Plasticity/supersqlite/badges/master/pipeline.svg)](https://gitlab.com/Plasticity/supersqlite/commits/master)\u0026nbsp;\u0026nbsp;\u0026nbsp;[![Build Status](https://travis-ci.org/plasticityai/supersqlite.svg?branch=master)](https://travis-ci.org/plasticityai/supersqlite)\u0026nbsp;\u0026nbsp;\u0026nbsp;[![Build status](https://ci.appveyor.com/api/projects/status/72lwh2g7a9ddbnt2/branch/master?svg=true)](https://ci.appveyor.com/project/plasticity-admin/supersqlite/branch/master)\u003cbr/\u003e[![PyPI version](https://badge.fury.io/py/supersqlite.svg)](https://pypi.python.org/pypi/supersqlite/)\u0026nbsp;\u0026nbsp;\u0026nbsp;[![license](https://img.shields.io/github/license/mashape/apistatus.svg?maxAge=2592000)](https://gitlab.com/Plasticity/supersqlite/blob/master/LICENSE.txt)\u0026nbsp;\u0026nbsp;\u0026nbsp;[![Python version](https://img.shields.io/pypi/pyversions/supersqlite.svg)](https://pypi.python.org/pypi/supersqlite/)\u003c/div\u003e\nA feature-packed Python package and for utilizing SQLite in Python by [Plasticity](https://www.plasticity.ai/). It is intended to be a drop-in replacement to Python's built-in [SQLite API](https://docs.python.org/3/library/sqlite3.html), but without any limitations. It offers unique features like [remote streaming over HTTP](#remote-streaming-over-http) and [bundling of extensions like JSON, R-Trees (geospatial indexing), and Full Text Search](#extensions). SuperSQLite is also packaged with pre-compiled native binaries for SQLite and all of its extensions for nearly every platform as to avoid any C/C++ compiler errors during install.\n\n## Table of Contents\n- [Installation](#installation)\n- [Motivation](#motivation)\n- [Using the Library](#using-the-library)\n    * [Connecting](#connecting)\n    * [Querying](#querying)\n- [Remote Streaming over HTTP](#remote-streaming-over-http)\n- [Other Documentation](#other-documentation)\n- [Other Programming Languages](#other-programming-languages)\n- [Contributing](#contributing)\n- [Roadmap](#roadmap)\n- [Other Notable Projects](#other-notable-projects)\n- [LICENSE and Attribution](#license-and-attribution)\n\n## Installation\nYou can install this package with `pip`:\n```python\npip install supersqlite # Python 2.7\npip3 install supersqlite # Python 3\n```\n\n## Motivation\n[SQLite](http://www.sqlite.org), is a fast, popular embedded database, used by [large enterprises](https://www.sqlite.org/famous.html). It is the [most widely-deployed database](https://www.sqlite.org/mostdeployed.html) and has billions of deployments. It has a [built-in](https://docs.python.org/3/library/sqlite3.html) binding in Python.\n\nThe Python bindings, however, often are compiled against an out-of-date copy of SQLite or may be compiled with [limitations](https://www.sqlite.org/limits.html) set to low levels. Moreover, it is difficult to load extremely useful extensions like [JSON1](https://www.sqlite.org/json1.html) that adds JSON functionality to SQLite or [FTS5](https://www.sqlite.org/fts5.html) that adds full-text search functionality to SQLite since they must be compiled with a C/C++ compiler on each platform before being loaded.\n\nSuperSQLite aims to solve these problems by packaging a newer version of SQLite natively pre-compiled for every platform along with natively pre-compiled SQLite extensions. SuperSQLite also adds useful unique new features like [remote streaming over HTTP](#remote-streaming-over-http) to read from a centralized SQLite database.\n\nMoreover, by default, SQLite does not enable some optimizations that can result in speedups. SuperSQLite compiles SQLite with various [optimizations](#optimizations) and allows you to select your [workload at runtime](#workload-optimizations) to further automatically configure the connection to be optimized for your workload.\n\n## When to use SuperSQLite?\n\nSQLite is [extremely reliable and durable](https://www.sqlite.org/hirely.html) for large amounts of data ([up to 140TB](https://www.sqlite.org/limits.html)). It is considered one of the most [well-engineered and well-tested](https://www.sqlite.org/testing.html) software solutions today, with 711x more test code than implementation code. \n\nSQLite is [faster than nearly every other database](https://www.sqlite.org/speed.html) at read-heavy use cases (especially compared to databases that may use a client-server model with network latency like MySQL, PostgreSQL, MongoDB, DynamoDB, etc.). You can also instantiate SQLite completely in-memory to remove disk latency, if your data will fit within RAM. For key/value use cases, you can get comparable or better [read/write performance to key/value databases like LevelDB](https://sqlite.org/src4/doc/trunk/www/lsmperf.wiki) with the [LSM1 extension](#extensions).\n\nWhen you have a write-heavy workload with *multiple* servers that need to write concurrently to a shared database (backend to a website), you would probably want to choose something that has a client-server model instead like PostgreSQL, although SQLite can handle processing write requests fast enough that it is sufficient for most concurrent write loads. In fact, Expensify uses [SQLite for their entire backend](https://blog.expensify.com/2018/01/08/scaling-sqlite-to-4m-qps-on-a-single-server/). If you need the database to be automatically replicated or automatically sharded across machines or other distributed features, you probably want to use something else.\n\nSee [Appropriate Uses For SQLite](https://www.sqlite.org/whentouse.html) for more information and [Well-Known Users of SQLite](https://www.sqlite.org/famous.html) for example use cases.\n\n## Using the Library\n\nInstead of 'import sqlite3', use:\n\n    from supersqlite import sqlite3\n\nThis retains compatibility with the sqlite3 package, while adding the various\nenhancements.\n\n### Connecting\n\nGiven the above import, connect to a sqlite database file using:\n\n    conn = sqlite3.connect('foo.db')\n\n### Querying\n\n### Remote Streaming over HTTP\n\n### Workload Optimizations\n\n### Extensions\n#### JSON1\n#### FTS3, FTS4, FTS5\n#### LSM1\n#### R\\*Tree\n#### Other\n#### Custom\n\n\n### Export SQLite Resources\n\n### Optimizations\n\n## Other Documentation\nSuperSQLite extends the [apsw](https://github.com/rogerbinns/apsw) Python SQLite wrapper and adds on to its functionality. You can find the full documentation for that library [here](https://rogerbinns.github.io/apsw/), which in turn attempts to implement [PEP 249 (DB API)](https://www.python.org/dev/peps/pep-0249/). The connection object, cursor object, etc. are all [`apsw.Connection`](https://rogerbinns.github.io/apsw/connection.html), [`apsw.Cursor`](https://rogerbinns.github.io/apsw/cursor.html). Note, however, that some monkey-patching has been done to make the library more in-line and compatible as a drop-in replacement for Python's built-in `sqlite3` module.\n\nOther documentation is not available at this time. See the source file directly (it is well commented) if you need more information about a method's arguments or want to see all supported features.\n\n## Other Programming Languages\nCurrently, this library only supports Python. There are no plans to port it to any other languages, but since SQLite has a native C implementation and has bindings in most languages, you can use the [export functions](#export-sqlite-resources) to load SuperSQLite's SQLite extensions in the SQLite bindings of other programming languages or link SuperSQLite's version of SQLite to a native binary.\n\n## Contributing\nThe main repository for this project can be found on [GitLab](https://gitlab.com/Plasticity/supersqlite). The [GitHub repository](https://github.com/plasticityai/supersqlite) is only a mirror. Pull requests for more tests, better error-checking, bug fixes, performance improvements, or documentation or adding additional utilties / functionalities are welcome on [GitLab](https://gitlab.com/Plasticity/supersqlite).\n\nYou can contact us at [opensource@plasticity.ai](mailto:opensource@plasticity.ai).\n\n## Roadmap\n\n* Out of the box, \"fast-write\" configuration option that makes the connection optimized for fast-writing.\n* Out of the box, \"fast-read\" configuration option that makes the conenction optimized for\nfast-reading.\n* Optimize streaming cache behavior\n\n## Other Notable Projects\n* [pysqlite](https://github.com/ghaering/pysqlite) - The built-in `sqlite3` module in Python.\n* [apsw](https://github.com/rogerbinns/apsw) - Powers the main API of SuperSQLite, aims to port all of SQLite's API functionality (like VFSes) to Python, not just the query APIs.\n* [Magnitude](https://github.com/plasticityai/magnitude/) - Another project by Plasticity that uses SuperSQLite's unique features for machine learning embedding models.\n\n\n## LICENSE and Attribution\n\nThis repository is licensed under the license found [here](LICENSE.txt).\n\nThe SQLite \"feather\" icon is taken from the [SQLite project](https://www.sqlite.org) which is released as [public domain](https://www.sqlite.org/copyright.html).\n\nThis project is *not* affiliated with the official SQLite project.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplasticityai%2Fsupersqlite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fplasticityai%2Fsupersqlite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplasticityai%2Fsupersqlite/lists"}