{"id":13513453,"url":"https://github.com/digital-fabric/extralite","last_synced_at":"2025-10-19T19:27:32.543Z","repository":{"id":45218289,"uuid":"369477678","full_name":"digital-fabric/extralite","owner":"digital-fabric","description":"Ruby on SQLite","archived":false,"fork":false,"pushed_at":"2024-03-21T09:20:20.000Z","size":7321,"stargazers_count":240,"open_issues_count":3,"forks_count":7,"subscribers_count":12,"default_branch":"main","last_synced_at":"2024-04-14T06:07:41.759Z","etag":null,"topics":["database","ruby","sqlite","sqlite3"],"latest_commit_sha":null,"homepage":"http://www.rubydoc.info/gems/extralite","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/digital-fabric.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null},"funding":{"github":"noteflakes"}},"created_at":"2021-05-21T09:06:39.000Z","updated_at":"2024-04-15T16:37:22.779Z","dependencies_parsed_at":"2024-04-15T16:51:51.848Z","dependency_job_id":null,"html_url":"https://github.com/digital-fabric/extralite","commit_stats":{"total_commits":160,"total_committers":5,"mean_commits":32.0,"dds":0.05625000000000002,"last_synced_commit":"61669ffe961157472a8006c3381a381851ff05bb"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digital-fabric%2Fextralite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digital-fabric%2Fextralite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digital-fabric%2Fextralite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digital-fabric%2Fextralite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/digital-fabric","download_url":"https://codeload.github.com/digital-fabric/extralite/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247128753,"owners_count":20888235,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","ruby","sqlite","sqlite3"],"created_at":"2024-08-01T05:00:26.116Z","updated_at":"2025-10-19T19:27:32.532Z","avatar_url":"https://github.com/digital-fabric.png","language":"C","funding_links":["https://github.com/sponsors/noteflakes"],"categories":["C"],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n  \u003cbr\u003e\n  Extralite\n\u003c/h1\u003e\n\n\u003ch4 align=\"center\"\u003eRuby on SQLite\u003c/h4\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"http://rubygems.org/gems/extralite\"\u003e\n    \u003cimg src=\"https://badge.fury.io/rb/extralite.svg\" alt=\"Ruby gem\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/digital-fabric/extralite/actions\"\u003e\n    \u003cimg src=\"https://github.com/digital-fabric/extralite/actions/workflows/test.yml/badge.svg\" alt=\"Tests\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/digital-fabric/extralite/blob/master/LICENSE\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/license-MIT-blue.svg\" alt=\"MIT License\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://www.rubydoc.info/gems/extralite\"\u003eAPI reference\u003c/a\u003e\n\u003c/p\u003e\n\n## What is Extralite?\n\nExtralite is a fast and innovative SQLite wrapper for Ruby with a rich set of\nfeatures. It provides multiple ways of retrieving data from SQLite databases,\nmakes it possible to use SQLite databases in multi-threaded and multi-fibered\nRuby apps, and includes a comprehensive set of tools for managing SQLite\ndatabases.\n\nExtralite comes in two flavors: the `extralite` gem which uses the\nsystem-installed sqlite3 library, and the `extralite-bundle` gem which bundles\nthe latest version of SQLite\n([3.50.3](https://sqlite.org/releaselog/3_50_3.html)), offering access to the\nlatest features and enhancements.\n\n## Features\n\n- Best-in-class [performance](#performance) (up to 4.5X the performance of the\n  [sqlite3](https://github.com/sparklemotion/sqlite3-ruby) gem).\n- Support for [concurrency](#concurrency) out of the box for multi-threaded\n  and multi-fibered apps.\n- A variety of ways to [retrieve data](#query-modes) - hashes, arrays, single\n  columns, single rows, [transforms](#value-transforms).\n- Support for [external iteration](#iterating-over-records-in-a-prepared-query),\n  allowing iterating through single records or batches of records.\n- [Prepared queries](#prepared-queries).\n- [Parameter binding](#parameter-binding).\n- [Batch execution](#batch-execution-of-queries) of queries.\n- [transactions and savepoints](#transactions-and-savepoints).\n- Advanced features: load [SQLite extensions](#loading-extensions), create\n  [backups](#creating-backups), retrieve [status\n  information](#retrieving-status-information), work with\n  [changesets](#working-with-changesets), interrogate [database\n  limits](#working-with-database-limits),  [trace](#tracing-sql-statements)\n  queries.\n- [Sequel](#usage-with-sequel) adapter.\n\n## Table of Content\n\n- [Installing Extralite](#installing-extralite)\n- [Getting Started](#getting-started)\n- [Query Modes](#query-modes)\n- [Parameter binding](#parameter-binding)\n- [Value Transforms](#value-transforms)\n- [Data Types](#data-types)\n- [Prepared Queries](#prepared-queries)\n- [Batch Execution of Queries](#batch-execution-of-queries)\n- [Transactions and Savepoints](#transactions-and-savepoints)\n- [Database Information](#database-information)\n- [Error Handling](#error-handling)\n- [Concurrency](#concurrency)\n- [Advanced Usage](#advanced-usage)\n- [Usage with Sequel](#usage-with-sequel)\n- [Performance](#performance)\n- [License](#license)\n- [Contributing](#contributing)\n\n## Installing Extralite\n\nUsing bundler:\n\n```ruby\ngem 'extralite'\n```\n\nOr manually:\n\n```bash\n$ gem install extralite\n```\n\n__Note__: Extralite supports Ruby 3.0 and newer.\n\n### Installing the Extralite-SQLite Bundle\n\nIf you don't have the sqlite3 lib installed on your system, do not want to use\nthe system-installed version of SQLite, or would like to use the latest version\nof SQLite, you can install the `extralite-bundle` gem, which integrates the\nSQLite source code.\n\nUsage of the `extralite-bundle` gem is identical to the usage of the normal\n`extralite` gem, using `require 'extralite'` to load the gem.\n\n## Getting Started\n\nThe following example shows how to open an SQLite database and run some queries:\n\n```ruby\ndb = Extralite::Database.new('mydb.sqlite')\ndb.execute('create table foo (x, y, z)')\ndb.execute('insert into foo values (?, ?, ?)', 1, 2, 3)\ndb.execute('insert into foo values (?, ?, ?)', 4, 5, 6)\ndb.query('select * from foo') #=\u003e [{x: 1, y: 2, z: 3}, {x: 4, y: 5, z: 6}]\n```\n\nThe `#execute` method is used to make changes to the database, such as creating\ntables, or inserting records. It returns the number of records changed by the\nquery.\n\nThe `#query` method is used to read data from the database. It returns an array\ncontaining the resulting records, represented as hashes mapping column names (as\nsymbols) to individual values.\n\nYou can also iterate on records by providing a block to `#query`:\n\n```ruby\ndb.query 'select * from foo' do |r|\n  p record: r\nend\n```\n\n## Query Modes\n\nExtralite allows you to retrieve data from SQLite database in the form that most\na particular context. For some use cases you'll want to work with rows as\nhashes. In other cases, you'll want to work with rows as arrays, or even as\nsingle values, if you're just reading one column.\n\nFor that purpose, Extralite offers three different ways, or modes, of retrieving\nrecords:\n\n- `:hash`: retrieve each row as a hash (this is the default mode).\n- `:array`: retrieve each row as an array.\n- `:splat`: retrieve each row as one or more splatted values, without wrapping\n  them in a container (see [below](#the-splat-query-mode)).\n\nExtralite provides separate methods for the different modes:\n\n```ruby\n# alias #query_hash\ndb.query('select 1') #=\u003e [{ \"1\" =\u003e 1 }]\n\ndb.query_array('select 1') #=\u003e [[1]]\n\ndb.query_splat('select 1') #=\u003e [1]\n```\n\nNotice how all the return values above are arrays. This is because the different\n`#query_xxx` methods are designed to return multiple rows. If you want to just\nget back a single row, use one of the `query_single_xxx` methods:\n\n```ruby\n# alias #query_single_hash\ndb.query_single('select 1') #=\u003e { \"1\" =\u003e 1 }\n\ndb.query_single_array('select 1') #=\u003e [1]\n\ndb.query_single_splat('select 1') #=\u003e 1\n```\n\n### Iterating Over Query Results with a Block\n\nIn addition to getting query results as an array of rows, you can also directly\niterate over the query results by providing a block to the different\n`#query_xxx` methods:\n\n```ruby\ndb.query('select * from foo') { |row| handle_row(row) }\n```\n\n### The Splat Query Mode\n\nThe splat query mode allows you to retrieve column values for each row without\nwrapping them in a container. This is useful especially when performing queries\nthat return a single column:\n\n```ruby\n# When using the array mode we need to take unwrap the values\nids = db.query_array('select id from tasks where active = 1').map { |r| r.first }\n\n# In splat mode we don't need to do that\nids = db.query_splat('select id from tasks where active = 1')\n```\n\nThe splat mode is also useful when iterating over records with a block. The\ncolumn values are provided as splatted arguments to the given block:\n\n```ruby\ndb.query_splat('select a, b, c from foo') do |a, b, c|\n  do_this_with(a, b)\n  do_that_with(c)\nend\n```\n\nWhen iterating over records in this manner, the splat mode is slightly faster\nthan the array mode, and also reduces pressure on the Ruby GC since you avoid\nallocating arrays or hashes to hold the column values.\n\n## Parameter Binding\n\nThe `#execute` and `#query_xxx` methods accept parameters that can be bound to\nthe query, which means that their values will be used for each corresponding\nplace-holder (expressed using `?`) in the SQL statement:\n\n```ruby\ndb.query('select x from my_table where y = ? and z = ?', 'foo', 'bar')\n```\n\nYou can also express place holders by specifying their index (starting from 1)\nusing `?IDX`:\n\n```ruby\n# use the same value for both place holders:\ndb.query('select x from my_table where y = ?1 and z = ?1', 42)\n```\n\nAnother possibility is to use named parameters, which can be done by expressing\nplace holders as `:KEY`, `@KEY` or `$KEY`:\n\n```ruby\ndb.query('select x from my_table where y = $y and z = $z', y: 'foo', z: 'bar')\n```\n\nExtralite supports specifying named parameters using `Struct` or `Data` objects:\n\n```ruby\nMyStruct = Struct.new(:x, :z)\nparams = MyStruct.new(42, 6)\ndb.execute('update foo set x = $x where z = $z', params)\n\nMyData = Data.define(:x, :z)\nparams = MyData.new(43, 3)\ndb.execute('update foo set x = $x where z = $z', params)\n```\n\nParameter binding is especially useful for preventing [SQL-injection\nattacks](https://en.wikipedia.org/wiki/SQL_injection), but is also useful when\ncombined with [prepared queries](#prepared-queries) when repeatedly running the\nsame query over and over.\n\n## Data Types\n\nExtralite supports the following data types for either bound parameters or row\nvalues:\n\n- `Integer`\n- `Float`\n- `Boolean` (see below)\n- `String` (see below)\n- nil\n\n### Boolean Values\n\nSQLite does not have a boolean data type. Extralite will automatically translate\nbound parameter values of `true` or `false` to the integer values `1` and `0`,\nrespectively. Note that boolean values stored in the database will be fetched as\nintegers.\n\n### String Values\n\nString parameter values are translated by Extralite to either `TEXT` or `BLOB`\nvalues according to the string encoding used. Strings with an `ASCII-8BIT` are\ntreated as blobs. Otherwise they are treated as text values.\n\nLikewise, when fetching records, Extralite will convert a `BLOB` column value to\na string with `ASCII-8BIT` encoding, and a `TEXT` column value to a string with\n`UTF-8` encoding.\n\n```ruby\n# The following calls will insert blob values into the database\nsql = 'insert into foo values (?)'\ndb.execute(sql, File.binread('/path/to/file'))\ndb.execute(sql, Extralite::Blob.new('Hello, 世界!'))\ndb.execute(sql, 'Hello, 世界!'.force_encoding(Encoding::ASCII_8BIT))\n```\n\n## Value Transforms\n\nExtralite allows you to transform rows to any value your application may need by\nproviding a transform proc that takes the raw row values and returns the\ntransformed data. The transform proc is passed each resulting row either as a\nhash or as a list of values.\n\nTransforms are useful when you need to transform rows into ORM model instances,\nor when you need to do some other transformation on the values retrieved from\nthe database.\n\nTo transform results, pass a transform proc as the first parameter to one of the\n`#query_xxx` methods:\n\n```ruby\ntransform = -\u003e(h) { MyModel.new(h) }\ndb.query(transform, 'select * from foo')\n#=\u003e rows as instances of MyModel\n```\n\nWhen using the `splat` mode, the different column values are passed as splatted\nvalues to the transform proc:\n\n```ruby\ntransform = -\u003e(a, b, c) { { a:a, b: b, c: JSON.parse(c) } }\ndb.query_splat(transform, 'select a, b, c from foo')\n#=\u003e transformed rows\n```\n\nValue transforms can also be done with [prepared\nqueries](#value-transforms-in-prepared-queries).\n\n## Prepared Queries\n\nPrepared queries (also known as prepared statements) allow you to maximize\nperformance and reduce memory usage when running the same query repeatedly. They\nalso allow you to create parameterized queries that can be repeatedly executed\nwith different parameters:\n\n```ruby\nquery = db.prepare('select * from foo where x = ?')\n\n# bind parameters and get results as an array of hashes\nquery.bind(1).to_a\n#=\u003e [{ x: 1, y: 2, z: 3 }]\n```\n\n### Binding Values to Prepared Queries\n\nTo bind parameter values to the query, use the `#bind` method. The parameters\nwill remain bound to the query until `#bind` is called again.\n\n```ruby\nquery.bind(1)\n\n# run the query any number of times\n3.times { query.to_a }\n\n# bind other parameter values\nquery.bind(4)\n```\n\nYou can also bind parameters when creating the prepared query by passing\nadditional parameters to the `Database#prepare` method:\n\n```ruby\nquery = db.prepare('select * from foo where x = ?', 1)\n```\n\n### Fetching Records from a Prepared Query\n\nJust like the `Database` interface, prepared queries support getting data using\nthree different modes: as a hash, an array or as individual column values. To\nset the mode, you can use one of the `#prepare_xxx` methods:\n\n```ruby\n# hash mode (alias #prepare_hash)\ndb.prepare('select * from foo').to_a\n#=\u003e [{ x: 1, y: 2, z: 3}]\n\n# splat mode\ndb.prepare_splat('select x from foo').to_a\n#=\u003e [1]\n\n# array mode\ndb.prepare_array('select * from foo').to_a\n#=\u003e [[1, 2, 3]]\n```\n\nYou can also set the query mode by getting or setting `#mode`:\n\n```ruby\nq = db.prepare('select * from foo')\nq.to_a #=\u003e [{ x: 1, y: 2, z: 3}]\n\nq.mode #=\u003e :hash\nq.mode = :array\nq.to_a \"=\u003e [[1, 2, 3]]\n```\n\n### Fetching Single Records or Batches of Records\n\nPrepared queries let you iterate over records one by one, or by batches. For\nthis, use the `#next` method:\n\n```ruby\nquery = db.prepare('select * from foo')\n\nquery.next\n#=\u003e { x: 1, y: 2, z: 3 }\nquery.next\n#=\u003e { x: 4, y: 5, z: 6 }\n\n# Fetch the next 10 records\nquery.reset # go back tpo the beginning\nquery.next(10)\n#=\u003e [{ x: 1, y: 2, z: 3 }, { x: 4, y: 5, z: 6 }]\n\n# Fetch the next row as an array\nquery = db.prepare_array('select * from foo')\nquery.next\n#=\u003e [1, 2, 3]\n\n# Fetch the next row as a single column\ndb.prepare_splat('select z from foo').next\n#=\u003e 3\n```\n\nTo detect the end of the result, you can use `#eof?`. To go back to the\nbeginning of the result set, use `#reset`. The following example shows how to\nread the query results in batches of 10:\n\n```ruby\nquery.reset\nwhile !query.eof?\n  records = query.next(10)\n  process_records(records)\nend\n```\n\n### Iterating over Records in a Prepared Query\n\nIn addition to the `#next` method, you can also iterate over query results by\nusing the familiar `#each` method:\n\n```ruby\n# iterate over records as hashes\nquery = db.prepare('select * from foo')\nquery.each { |r| ... }\n\n# iterate over records as arrays\nquery = db.prepare_array('select * from foo')\nquery.each { |r| ... }\n\n# iterate over records as single values\nquery = db.prepare_splat('select a, b, c from foo')\nquery.each { |a, b, c| ... }\n```\n\n### Prepared Query as an Enumerable\n\nYou can also use a prepared query as an enumerable, allowing you to chain\nenumerable method calls while iterating over the query result set. This is done\nby calling `#each` without a block:\n\n```ruby\niterator = query.each\n#=\u003e Returns an Extralite::Iterator instance\n\niterator.map { |r| r[:x] *100 + r[:y] * 10 + r[:z] }\n#=\u003e [123, 345]\n```\n\nThe Iterator class includes the\n[`Enumerable`](https://rubyapi.org/3.3/o/enumerable) module, with all its\nmethods, such as `#map`, `#select`, `#inject`, `#lazy` etc. You can also\ninstantiate an iterator explicitly:\n\n```ruby\n# You need to pass the query to iterate over:\niterator = Extralite::Iterator.new(query)\niterator.each { |r| ... }\n```\n\n### Value Transforms in Prepared Queries\n\nPrepared queries can automatically transform their result sets by setting a\ntransform block. The transform block receives values according to the query mode\n(hash, array or splat). To set a transform you can pass a block to one of the\n`Database#prepare_xxx` methods, or use `Query#transform`:\n\n```ruby\nq = db.prepare('select * from items where id = ?') { |h| Item.new(h) }\nq.bind(42).next #=\u003e Item instance\n\n# An equivalent\nq = db.prepare('select * from items where id = ?')\nq.transform { |h| Item.new(h) }\n```\n\nThe same can be done for queries in `splat` or `array` mode:\n\n```ruby\ndb.prepare_splat('select * from foo') { |a, b, c| a + b + c }\n\ndb.prepare_array('select * from foo') { |a| a.map(\u0026:to_s).join }\n```\n\n## Batch Execution of Queries\n\nExtralite provides methods for batch execution of queries, with multiple sets of\nparameters. The `#batch_execute` method lets you insert or update a large number\nof records with a single call:\n\n```ruby\nvalues = [\n  [1, 11, 111],\n  [2, 22, 222],\n  [3, 33, 333]\n]\n# insert the above records in one fell swoop, and returns the total number of\n# changes:\ndb.batch_execute('insert into foo values (?, ?, ?)', values)\n#=\u003e 3\n```\n\nParameters to the query can also be provided by any object that is an\n`Enumerable` or has an `#each` method, or any *callable* object that responds to\n`#call`:\n\n```ruby\n# Take parameter values from a Range\ndb.batch_execute('insert into foo values (?)', 1..10)\n#=\u003e 10\n\n# Insert (chomped) lines from a file\nFile.open('foo.txt') do |f|\n  source = f.each_line.map(\u0026:chomp)\n  db.batch_execute('insert into foo values (?)', source)\nend\n\n# Insert items from a queue\nparameters = proc do\n  item = queue.shift\n  # when we're done, we return nil\n  (item == :done) ? nil : item\nend\ndb.batch_execute('insert into foo values (?)', parameters)\n#=\u003e number of rows inserted\n```\n\nLike its cousin `#execute`, the `#batch_execute` returns the total number of\nchanges to the database (rows inserted, deleted or udpated).\n\n### Batch Execution of Queries that Return Rows\n\nExtralite also provides a `#batch_query` method that like `#batch_execute` takes\na parameter source and returns an array containing the result sets for all query\ninvocations. If a block is given, the result sets are passed to the block\ninstead.\n\nThe `#batch_query` method is especially useful for batch queries with a\n`RETURNING` clause:\n\n```ruby\nupdates = [\n  { id: 3, price: 42 },\n  { id: 5, price: 43 }\n]\nsql = 'update foo set price = $price where id = $id returning id, quantity'\ndb.batch_query(sql, updates)\n#=\u003e [[{ id: 3, quantity: 4 }], [{ id: 5, quantity: 5 }]]\n\n# The same with a block (returns the total number of changes)\ndb.batch_query(sql, updates) do |rows|\n  p rows\n  #=\u003e [{ id: 3, quantity: 4 }]\n  #=\u003e [{ id: 5, quantity: 5 }]\nend\n#=\u003e 2\n```\n\nThe `#batch_query` method, like other row fetching methods, changes the row\nrepresentation according to the query mode.\n\n### Batch Execution of Prepared Queries\n\nBatch execution can also be done using prepared queries, using the same methods\n`#batch_execute` and `#batch_query`:\n\n```ruby\nquery = db.prepare 'update foo set x = ? where z = ? returning *'\n\nquery.batch_execute([[42, 3], [43, 6]])\n#=\u003e 2\n\nquery.batch_query([[42, 3], [43, 6]])\n#=\u003e [{ x: 42, y: 2, z: 3 }, { x: 43, y: 5, z: 6 }]\n```\n\n## Transactions and Savepoints\n\nAll reads and writes to SQLite databases occur within a\n[transaction](https://www.sqlite.org/lang_transaction.html). If no explicit\ntransaction has started, the submitted SQL statements passed to `#execute` or\n`#query` will all run within an implicit transaction:\n\n```ruby\n# The following two SQL statements will run in a single implicit transaction:\ndb.execute('insert into foo values (42); insert into bar values (43)')\n\n# Otherwise, each call to #execute runs in a separate transaction:\ndb.execute('insert into foo values (42)')\ndb.execute('insert into bar values (43)')\n```\n\n### Explicit Transactions\n\nWhile you can issue `BEGIN` and `COMMIT` SQL statements yourself to start and\ncommit explicit transactions, Extralite provides a convenient `#transaction`\nmethod that manages starting, commiting and rolling back of transactions\nautomatically:\n\n```ruby\ndb.transaction do\n  db.execute('insert into foo values (42)')\n  raise 'Something bad happened' if something_bad_happened\n  db.execute('insert into bar values (43)')\nend\n```\n\nIf no exception is raised in the transaction block, the changes are commited. If\nan exception is raised, the changes are rolled back and the exception is\npropagated to the application code. You can prevent the exception from being\npropagated by calling `#rollback!`:\n\n```ruby\ndb.transaction do\n  db.execute('insert into foo values (42)')\n  rollback! if something_bad_happened\n  db.execute('insert into bar values (43)')\nend\n```\n\n### Transaction Modes\n\nBy default, `#transaction` starts an `IMMEDIATE` transaction. To start a\n`DEFERRED` or `EXCLUSIVE` transaction, pass the desired mode to `#transaction`:\n\n```ruby\n# Start a DEFERRED transaction\ndb.transaction(:deferred) do\n  ...\nend\n\n# Start a EXCLUSIVE transaction\ndb.transaction(:exclusive) do\n  ...\nend\n```\n\nNote that running an `IMMEDIATE` or `EXCLUSIVE` transaction blocks the database\nfor writing (and also reading in certain cases) for the duration of the\ntransaction. This can cause queries to the same database on a different\nconnection to fail with a `BusyError` exception. This can be mitigated by\nsetting a [busy timeout](#dealing-with-a-busy-database).\n\n### Savepoints\n\nIn addition to transactions, SQLite also supports the use of\n[savepoints](https://www.sqlite.org/lang_savepoint.html), which can be used for\nmore fine-grained control of changes within a transaction, and to be able to\nrollback specific changes without abandoning the entire transaction:\n\n```ruby\ndb.transaction do\n  db.execute 'insert into foo values (1)'\n  \n  db.savepoint :my_savepoint\n  db.execute 'insert into foo values (2)'\n  \n  # the following cancels the last insert\n  db.rollback_to :my_savepoint\n  db.execute 'insert into foo values (3)'\n\n  db.release :my_savepoint\nend\n```\n\n## Database Information\n\n### Getting the List of Tables\n\nTo get the list of tables in a database, use the `#tables` method:\n\n```ruby\ndb.tables\n#=\u003e [...]\n```\n\nTo get the list of tables in an attached database, you can pass the database name to `#tables`:\n\n```ruby\ndb.execute \"attach database 'foo.db' as foo\"\ndb.tables('foo')\n#=\u003e [...]\n```\n\n### Getting the Last Insert Row Id\n\n```ruby\ndb.execute 'insert into foo values (?)', 42\ndb.last_insert_rowid\n#=\u003e 1\n```\n\n### Getting the Column Names for a Given Query\n\n```ruby\ndb.columns('select a, b, c from foo')\n#=\u003e [:a, :b, :c]\n\n# Columns a prepared query:\nquery = db.prepare('select x, y from foo')\nquery.columns\n#=\u003e [:x, :y]\n```\n\n### Pragmas\n\nYou can get or set pragma values using `#pragma`:\n\n```ruby\n# get a pragma value:\ndb.pragma(:journal_mode)\n#=\u003e 'delete'\n\n# set a pragma value:\ndb.pragma(journal_mode: 'wal')\ndb.pragma(:journal_mode)\n#=\u003e 'wal'\n```\n\nYou can also pass pragmas when opening the database:\n\n```ruby\ndb = Extralite::Database.new('path/to/db', pragma: { foreign_keys: true })\n```\n\n## Error Handling\n\nExtralite defines various exception classes that are raised when an error is\nencountered while interacting with the underlying SQLite library:\n\n- `Extralite::SQLError` - raised when SQLite encounters an invalid SQL query.\n- `Extralite::BusyError` - raised when the underlying database is locked for use\n  by another database connection.\n- `Extralite::InterruptError` - raised when a query has been interrupted.\n- `Extralite::ParameterError` - raised when an invalid parameter value has been\n  specified.\n- `Extralite::Error` - raised on all other errors.\n\nIn addition to the above exceptions, further information about the last error\nthat occurred is provided by the following methods:\n\n- `#errcode` - the [error code](https://www.sqlite.org/rescode.html) returned by\n  the underlying SQLite library.\n- `#errmsg` - the error message for the last error. For most errors, the error\n  message is copied into the exception message.\n- `#error_offset` - for SQL errors, the offset into the SQL string where the\n  error was encountered.\n\n## Concurrency\n\nExtralite provides a comprehensive set of tools for dealing with concurrency\nissues, and for making sure that running queries on SQLite databases does not\ncause the app to freeze.\n\n**Note**: In order to allow concurrent access your the database, it is highly\nrecommended that you set your database to use [WAL journaling\nmode](https://www.sqlite.org/wal.html) for *all* database connections.\nOtherwise, you risking running into performance problems and having queries fail\nwith `BusyError` exceptions. You can easily open your database in WAL journaling\nmode by passing a `wal: true` option:\n\n```ruby\n# This will set PRAGMA journal_mode=1 and PRAGMA synchronous=1\ndb = Extralite::Database.new('path/to/db', wal: true)\n```\n\n### The Ruby GVL\n\nIn order to support multi-threading, Extralite releases the [Ruby\nGVL](https://www.speedshop.co/2020/05/11/the-ruby-gvl-and-scaling.html)\nperiodically while running queries. This allows other threads to run while the\nunderlying SQLite library is busy preparing queries, fetching records and\nbacking up databases. By default, the GVL is when preparing the query, and once\nfor every 1000 iterated records. The GVL release threshold can be set separately\nfor each database:\n\n```ruby\n# release the GVL on preparing the query, and every 10 records\ndb.gvl_release_threshold = 10 \n\n# release the GVL only when preparing the query\ndb.gvl_release_threshold = 0 \n\n# never release the GVL (for single-threaded apps only)\ndb.gvl_release_threshold = -1\n\ndb.gvl_release_threshold = nil # use default value (currently 1000)\n```\n\nFor most applications, there's no need to tune the GVL threshold value, as it\nprovides [excellent](#performance) performance characteristics for both\nsingle-threaded and multi-threaded applications.\n\nIn a heavily multi-threaded application, releasing the GVL more often (lower\nthreshold value) will lead to less latency (for threads not running a query),\nbut will also hurt the throughput (for the thread running the query). Releasing\nthe GVL less often (higher threshold value) will lead to better throughput for\nqueries, while increasing latency for threads not running a query. The following\ndiagram demonstrates the relationship between the GVL release threshold value,\nlatency and throughput:\n\n```\nless latency \u0026 throughput \u003c\u003c\u003c GVL release threshold \u003e\u003e\u003e more latency \u0026 throughput\n```\n\n### Dealing with a Busy Database\n\nWhen multiple threads or processes access the same database, the database may be\nlocked for writing by one process, which will block other processes wishing to\nwrite to the database. When attempting to write to a locked database, a\n`Extralite::BusyError` will be raised:\n\n```ruby\nready = nil\nlocker = Thread.new do\n  db1 = Extralite::Database.new('my.db')\n  # Lock the database for 3 seconds\n  db1.transaction do\n    ready = true\n    sleep(3)\n  end\nend\n\ndb2 = Extralite::Database.new('my.db')\n# wait for writer1 to enter a transaction\nsleep(0) while !ready\n# This will raise a Extralite::BusyError\ndb2.transaction { }\n# Extralite::BusyError!\n```\n\nYou can mitigate this by setting a busy timeout. This will cause SQLite to wait\nfor the database to become unlocked up to the specified timeout period:\n\n```ruby\n# Wait for up to 5 seconds before giving up\ndb2.busy_timeout = 5\n# Now it'll work!\ndb2.transaction { }\n```\n\nAs stated above, setting the database to use WAL journaling mode greatly reduces\ncontention between different process/threads accessing the same database. For\nmost use cases, setting the busy timeout solves the problem of failing to run\nqueries because of a busy database, as normally transactions are short-lived.\n\nHowever, in some cases, such as when running a multi-fibered app or when\nimplementing your own timeout mechanisms, you'll want to set a [progress\nhandler](#the-progress-handler).\n\n### Interrupting a Query\n\nTo interrupt an ongoing query, use the `#interrupt` method. Normally this is\ndone from a separate thread. Here's a way to implement a timeout using\n`#interrupt`:\n\n```ruby\ndef run_query_with_timeout(sql, timeout)\n  timeout_thread = Thread.new do\n    t0 = Time.now\n    sleep(timeout)\n    @db.interrupt\n  end\n  result = @db.query(sql)\n  timeout_thread.kill\n  result\nend\n\nrun_query_with_timeout('select 1 as foo', 5)\n#=\u003e [{ foo: 1 }]\n\n# A timeout will cause a Extralite::InterruptError to be raised\nrun_query_with_timeout(slow_sql, 5)\n#=\u003e Extralite::InterruptError!\n```\n\nYou can also call `#interrupt` from within the [progress\nhandler](#the-progress-handler).\n\n### The Progress Handler\n\nExtralite also supports setting up a progress handler, which is a piece of code\nthat will be called while a query is in progress, or while the database is busy.\nThis is useful especially when you want to implement a general purpose timeout\nmechanism that deals with both a busy database and with slow queries.\n\nThe progress handler can also be used for performing any kind of operation while\na query is in progress. Here are some use cases:\n\n- Interrupting queries that take too long to run.\n- Interrupting queries on an exceptional condition, such as a received signal.\n- Updating the UI while a query is running.\n- Switching between fibers in multi-fibered apps.\n- Switching between threads in multi-threaded apps.\n- Instrumenting the performance of queries.\n\nSetting the progress handler requires that Extralite hold the GVL while running\nall queries. Therefore, it should be used with care. In a multi-threaded app,\nyou'll need to call `Thread.pass` from the progress handler in order for other\nthreads to be able to run while the query is in progress. The progress handler\nis set per-database using `#on_progress`. This method takes a single parameter\nthat specifies the approximate number of SQLite VM instructions between\nsuccessive calls to the progress handler:\n\n```ruby\ndb.on_progress do\n  check_for_timeout\n  # Allow other threads to run\n  Thread.pass\nend\n```\n\nThe progress handler can be used to interrupt queries in progress. This can be\ndone by either calling `#interrupt`, or by raising an exception. As discussed\nabove, calling `#interrupt` causes the query to raise a\n`Extralite::InterruptError` exception:\n\n```ruby\ndb.on_progress(period: 1) { db.interrupt }\ndb.query('select 1')\n#=\u003e Extralite::InterruptError!\n```\n\nYou can also interrupt queries in progress by raising an exception. The query\nwill be stopped, and the exception will propagate to the call site:\n\n```ruby\ndb.on_progress(period: 1) do\n  raise 'BOOM!'\nend\n\ndb.query('select 1')\n#=\u003e BOOM!\n```\n\nHere's how a timeout might be implemented using the progress handler:\n\n```ruby\ndef setup_progress_handler\n  @db.on_progress do\n    raise TimeoutError if Time.now - @t0 \u003e= @timeout\n    Thread.pass\n  end\nend\n\n# In this example, we just return nil on timeout\ndef run_query_with_timeout(sql, timeout)\n  @t0 = Time.now\n  @db.query(sql)\nrescue TimeoutError\n  nil\nend\n\nrun_query_with_timeout('select 1 as foo', 5)\n#=\u003e [{ foo: 1 }]\n\nrun_query_with_timeout(slow_sql, 5)\n#=\u003e nil\n```\n\n**Note**: you must not issue any query from within the progress handler.\n\n### Dealing with a Busy Database in the Progress Handler\n\nAs mentioned above, the progress handler is also called when the database is\nbusy, regardless of the progress period given to `#on_progress`. You can detect\nif the database is busy by checking the first argument passed to the progress\nhandler, which will be true when busy:\n\n```ruby\ndb.on_progress do |busy|\n  if busy\n    foo\n  else\n    bar\n  end\nend\n```\n\nThis allows you to implement separate logic to deal with busy states, for\nexample sleeping for a small period of time, or implementing a different timeout\nperiod.\n\n### Advanced Progress Handler Settings\n\nYou can further tune the behaviour of the progress handler with the following\noptions passed to `#on_progress`:\n\n- `:mode`: the following modes are supported:\n  - `:none` : the progress handler is disabled.\n  - `:normal`: the progress handler is called on query progress (this is the\n    default mode).\n  - `:once`: the progress handler is called once before running the query.\n  - `:at_least_once`: the progress handler is called once before running the\n    query, and on query progress.\n- `:period`: controls the approximate number of SQLite VM instructions executed\n  between consecutive calls to the progress handler. Default value: 1000.\n- `:tick`: controls the granularity of the progress handler. This is the value\n  passed internally to the SQLite library. Default value: 10.\n\n```ruby\ndb.on_progress(mode: :at_least_once, period: 640, tick: 5) { snooze }\n```\n\n### Global Progress Handler Settings\n\nYou can set the global progress handler behaviour by calling\n`Extralite.on_progress`. You can use this API to set the global progress\nsettings, without needing to set a progress handler individually for each\n`Database` instance. This method takes the same options as\n`Database#on_progress`:\n\n```ruby\nExtralite.on_progress(mode: :at_least_once, period: 640, tick: 5) { snooze }\n\n# the new database instance uses the global progress handler settings\ndb = Database.new(':memory:')\n```\n\n### Extralite and Fibers\n\nThe progress handler can also be used to switch between fibers in a\nmulti-fibered Ruby app, based on libraries such as\n[Async](https://github.com/socketry/async) or\n[Polyphony](https://github.com/digital-fabric/polyphony). A general solution\n(that also works for multi-threaded apps) is to call `sleep(0)` in the progress\nhandler. This will work for switching between fibers using either Polyphony or\nany fiber scheduler gem, such as Async et al:\n\n```ruby\ndb.on_progress { sleep(0) }\n```\n\nFor Polyphony-based apps, you can also call `snooze` to allow other fibers to\nrun while a query is progressing. If your Polyphony app is multi-threaded,\nyou'll also need to call `Thread.pass` in order to allow other threads to run:\n\n```ruby\ndb.on_progress do\n  snooze\n  Thread.pass\nend\n```\n\nNote that with Polyphony, once you install the progress handler, you can just\nuse the regular `#move_on_after` and `#cancel_after` methods to implement\ntimeouts for queries:\n\n```ruby\ndb.on_progress { snooze }\n\ncancel_after(3) do\n  db.query(long_running_query)\nend\n```\n\n### Thread Safety\n\nA single database instance can be safely used in multiple threads simultaneously\nas long as the following conditions are met:\n\n- No explicit transactions are used.\n- Each thread issues queries by calling `Database#query_xxx`, or uses a separate\n  `Query` instance.\n- The GVL release threshold is not `0` (i.e. the GVL is released periodically\n  while running queries.)\n\n### Use with Ractors\n\nExtralite databases can safely be used inside ractors. A ractor has the benefit\nof using a separate GVL from the maine one, which allows true parallelism for\nRuby apps. So when you use Extralite to access SQLite databases from within a\nractor, you can do so without any considerations for what's happening outside\nthe ractor when it runs queries.\n\n**Note**: Ractors are considered an experimental feature of Ruby. You may\nencounter errors or inconsistent behaviour when using ractors.\n\n## Advanced Usage\n\n### Loading Extensions\n\nExtensions can be loaded by calling `#load_extension`:\n\n```ruby\ndb.load_extension('/path/to/extension.so')\n```\n\nA pretty comprehensive set of extensions can be found here:\n\nhttps://github.com/nalgeon/sqlean\n\n### Creating Backups\n\nYou can use `Database#backup` to create backup copies of a database. The\n`#backup` method takes either a filename or a database instance:\n\n```ruby\n# with a filename\ndb.backup('backup.db')\n\n# with an instance\ntarget = Extralite::Database.new('backup.db')\ndb.backup(target)\n```\n\nFor big databases, you can also track the backup progress by providing a block\nthat takes two arguments - the number of remaining pages, and the total number pages:\n\n```ruby\ndb.backup('backup.db') do |remaining, total|\n  puts \"backup progress: #{(remaining.to_f/total * 100).round}%\"\nend\n```\n\n### Working with Changesets\n\n__Note__: as the session extension is by default disabled in SQLite\ndistributions, support for changesets is currently only available withthe\nbundled version of Extralite, `extralite-bundle`.\n\nChangesets can be used to track and persist changes to data in a database. They\ncan also be used to apply the same changes to another database, or to undo them.\nTo track changes to a database, use the `#track_changes` method:\n\n```ruby\n# track changes to the foo and bar tables:\nchangeset = db.track_changes(:foo, :bar) do\n  insert_a_bunch_of_records(db)\nend\n\n# to track changes to all tables, pass nil:\nchangeset = db.track_changes(nil) do\n  insert_a_bunch_of_records(db)\nend\n```\n\nYou can then apply the same changes to another database:\n\n```ruby\nchangeset.apply(some_other_db)\n```\n\nTo undo the changes, obtain an inverted changeset and apply it to the database:\n\n```ruby\nchangeset.invert.apply(db)\n```\n\nYou can also save and load the changeset:\n\n```ruby\n# save the changeset\nIO.write('my.changes', changeset.to_blob)\n\n# load the changeset\nchangeset = Extralite::Changeset.new\nchangeset.load(IO.read('my.changes'))\n```\n\n### Retrieving Status Information\n\nExtralite provides methods for retrieving status information about the sqlite\nruntime, database-specific status and prepared statement-specific status,\n`Extralite.runtime_status`, `Database#status` and `Query#status` respectively.\nYou can also reset the high water mark for the specific status code by providing\ntrue as the reset argument. The status codes mirror those defined by the SQLite\nAPI. Some examples:\n\n```ruby\n# The Extralite.runtime_status returns a tuple consisting of the current value\n# and the high water mark value.\ncurrent, high_watermark = Extralite.runtime_status(Extralite::SQLITE_STATUS_MEMORY_USED)\n\n# To reset the high water mark, pass true as a second argument:\ncurrent, high_watermark = Extralite.runtime_status(Extralite::SQLITE_STATUS_MEMORY_USED, true)\n\n# Similarly, you can interrogate a database's status (pass true as a second\n# argument in order to reset the high watermark):\ncurrent, high_watermark = db.status(Extralite::SQLITE_DBSTATUS_CACHE_USED)\n\n# The Query#status method returns a single value (pass true as a\n# second argument in order to reset the high watermark):\nvalue = query.status(Extralite::SQLITE_STMTSTATUS_RUN)\n```\n\n### Working with Database Limits\n\nThe `Database#limit` can be used to get and set various database limits, as\n[discussed in the SQLite docs](https://www.sqlite.org/limits.html):\n\n```ruby\n# get limit\nvalue = db.limit(Extralite::SQLITE_LIMIT_ATTACHED)\n\n# set limit\ndb.limit(Extralite::SQLITE_LIMIT_ATTACHED, new_value)\n```\n\n### Tracing SQL Statements\n\nTo trace all SQL statements executed on the database, pass a block to\n`Database#trace`. To disable tracing, call `Database#trace` without a block:\n\n```ruby\n# enable tracing\ndb.trace { |sql| p sql: sql }\n\n# disable tracing\ndb.trace\n```\n\nAny bound parameters will also be passed to the trace block:\n\n```ruby\ndb.trace { |sql, *args| p sql: sql, args: args }\ndb.query('select ?, ?', 1, 2)\n# { sql: \"select ?, ?\", args: [1, 2] }\n```\n\n## Usage with Sequel\n\nExtralite includes an adapter for\n[Sequel](https://github.com/jeremyevans/sequel). To use the Extralite adapter,\njust use the `extralite` scheme instead of `sqlite`:\n\n```ruby\nDB = Sequel.connect('extralite://blog.db')\narticles = DB[:articles]\np articles.to_a\n```\n\n(Make sure you include `extralite` as a dependency in your `Gemfile`.)\n\n## Performance\n\nA benchmark script is included, creating a table of various row counts, then\nfetching the entire table using either `sqlite3` or `extralite`. This benchmark\nshows Extralite to be up to ~4.5 times faster than `sqlite3` when fetching a\nlarge number of rows.\n\n### Rows as Hashes\n\n[Benchmark source\ncode](https://github.com/digital-fabric/extralite/blob/main/test/perf_hash.rb)\n\n|Row count|sqlite3 2.6.0|Extralite 2.12|Advantage|\n|-:|-:|-:|-:|\n|10|629.0K rows/s|950.4K rows/s|__1.51x__|\n|1K|1770.5K rows/s|4321.5K rows/s|__2.44x__|\n|100K|1028.8K rows/s|4088.7K rows/s|__3.97x__|\n\n### Rows as Arrays\n\n[Benchmark source\ncode](https://github.com/digital-fabric/extralite/blob/main/test/perf_array.rb)\n\n|Row count|sqlite3 2.6.0|Extralite 2.12|Advantage|\n|-:|-:|-:|-:|\n|10|889.4K rows/s|1000.1K rows/s|__1.13x__|\n|1K|4518.1K rows/s|5381.5K rows/s|__1.19x__|\n|100K|4454.0K rows/s|5083.8K rows/s|__1.14x__|\n\n### Prepared Queries (Prepared Statements)\n\n[Benchmark source\ncode](https://github.com/digital-fabric/extralite/blob/main/test/perf_hash_prepared.rb)\n\n|Row count|sqlite3 2.6.0|Extralite 2.12|Advantage|\n|-:|-:|-:|-:|\n|10|783.1K rows/s|1115.1K rows/s|__1.42x__|\n|1K|1782.5K rows/s|4635.5K rows/s|__2.60x__|\n|100K|1018.1K rows/s|4599.4K rows/s|__4.52x__|\n\nAs those benchmarks show, Extralite is capabale of reading up to 4.5M rows per\nsecond, and can be more than 4 times faster than the `sqlite3` gem.\n\nNote that the benchmarks above were performed on synthetic data, in a\nsingle-threaded environment, with the GVL release threshold set to -1, which\nmeans that both Extralite and the `sqlite3` gem hold the GVL for the duration of\nthe query.\n\n## License\n\nThe source code for Extralite is published under the [MIT license](LICENSE). The\nsource code for SQLite is in the [public\ndomain](https://sqlite.org/copyright.html).\n\n## Contributing\n\nContributions in the form of issues, PRs or comments will be greatly\nappreciated!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdigital-fabric%2Fextralite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdigital-fabric%2Fextralite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdigital-fabric%2Fextralite/lists"}