{"id":13615999,"url":"https://github.com/proullon/ramsql","last_synced_at":"2025-10-05T01:53:04.685Z","repository":{"id":23412157,"uuid":"26774602","full_name":"proullon/ramsql","owner":"proullon","description":"In-memory SQL engine in Go sql/driver for testing purpose","archived":false,"fork":false,"pushed_at":"2024-09-21T08:35:18.000Z","size":513,"stargazers_count":908,"open_issues_count":20,"forks_count":68,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-10-05T01:53:02.920Z","etag":null,"topics":["database","go","imdb","memory-sql-engine","sql"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/proullon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-11-17T20:00:45.000Z","updated_at":"2025-10-02T06:54:10.000Z","dependencies_parsed_at":"2022-08-08T02:00:05.355Z","dependency_job_id":"080db11a-6ca9-450a-99ce-1139fafa0328","html_url":"https://github.com/proullon/ramsql","commit_stats":{"total_commits":299,"total_committers":17,"mean_commits":17.58823529411765,"dds":"0.14046822742474918","last_synced_commit":"8f4341e71a224efdd798ea647d228b0fd38f8a0b"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/proullon/ramsql","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/proullon%2Framsql","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/proullon%2Framsql/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/proullon%2Framsql/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/proullon%2Framsql/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/proullon","download_url":"https://codeload.github.com/proullon/ramsql/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/proullon%2Framsql/sbom","scorecard":{"id":747381,"data":{"date":"2025-08-11","repo":{"name":"github.com/proullon/ramsql","commit":"2cf07310d51d8df66a1d8c08a27966cabb72c7f4"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.5,"checks":[{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/go.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Code-Review","score":2,"reason":"Found 3/15 approved changesets -- score normalized to 2","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/go.yml:17: update your workflow using https://app.stepsecurity.io/secureworkflow/proullon/ramsql/go.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/go.yml:20: update your workflow using https://app.stepsecurity.io/secureworkflow/proullon/ramsql/go.yml/master?enable=pin","Info:   0 out of   2 GitHub-owned GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: BSD 3-Clause \"New\" or \"Revised\" License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 28 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":5,"reason":"5 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GO-2024-2567 / GHSA-fqpg-rq76-99pq","Warn: Project is vulnerable to: GO-2024-2606 / GHSA-mrww-27vc-gghv","Warn: Project is vulnerable to: GO-2023-2402 / GHSA-45x7-px36-x8w8","Warn: Project is vulnerable to: GO-2024-3321 / GHSA-v778-237x-gjrc","Warn: Project is vulnerable to: GO-2025-3487 / GHSA-hcg3-q754-cr77"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-22T19:14:02.164Z","repository_id":23412157,"created_at":"2025-08-22T19:14:02.164Z","updated_at":"2025-08-22T19:14:02.164Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278399611,"owners_count":25980332,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-04T02:00:05.491Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","go","imdb","memory-sql-engine","sql"],"created_at":"2024-08-01T20:01:22.030Z","updated_at":"2025-10-05T01:53:04.669Z","avatar_url":"https://github.com/proullon.png","language":"Go","readme":"# RamSQL\n\n[![Build Status](https://travis-ci.org/proullon/ramsql.svg)](https://travis-ci.org/proullon/ramsql)\n\n## Disposable SQL engine\n\nRamSQL has been written to be used in your project's test suite.\n\nUnit testing in Go is simple, create a foo_test.go import testing and run `go test ./...`.\nBut then there is SQL queries, constraints, CRUD...and suddenly you need a PostgresSQL, setup scripts and nothing is easy anymore.\n\nThe idea is to avoid setup, DBMS installation and credentials management as long as possible.\nA unique engine is tied to a single sql.DB with as much sql.Conn as needed providing a unique DataSourceName.\nBottom line : One DataSourceName per test and you have full test isolation in no time.\n\n## Installation\n\n```\n  go get github.com/proullon/ramsql\n```\n\n## Usage\n\nLet's say you want to test the function LoadUserAddresses :\n\n```go\nfunc LoadUserAddresses(db *sql.DB, userID int64) ([]string, error) {\n\tquery := `SELECT address.street_number, address.street FROM address \n\t\t\t\t\t\t\tJOIN user_addresses ON address.id=user_addresses.address_id \n\t\t\t\t\t\t\tWHERE user_addresses.user_id = $1;`\n\n\trows, err := db.Query(query, userID)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tvar addresses []string\n\tfor rows.Next() {\n\t\tvar number int\n\t\tvar street string\n\t\tif err := rows.Scan(\u0026number, \u0026street); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\taddresses = append(addresses, fmt.Sprintf(\"%d %s\", number, street))\n\t}\n\n\treturn addresses, nil\n}\n\n```\n\nUse RamSQL to test it in a disposable isolated in-memory SQL engine :\n\n```go\npackage myproject \n\nimport (\n\t\"database/sql\"\n\t\"fmt\"\n\t\"testing\"\n\n\t_ \"github.com/proullon/ramsql/driver\"\n)\n\n\nfunc TestLoadUserAddresses(t *testing.T) {\n\tbatch := []string{\n\t\t`CREATE TABLE address (id BIGSERIAL PRIMARY KEY, street TEXT, street_number INT);`,\n\t\t`CREATE TABLE user_addresses (address_id INT, user_id INT);`,\n\t\t`INSERT INTO address (street, street_number) VALUES ('rue Victor Hugo', 32);`,\n\t\t`INSERT INTO address (street, street_number) VALUES ('boulevard de la République', 23);`,\n\t\t`INSERT INTO address (street, street_number) VALUES ('rue Charles Martel', 5);`,\n\t\t`INSERT INTO address (street, street_number) VALUES ('chemin du bout du monde ', 323);`,\n\t\t`INSERT INTO address (street, street_number) VALUES ('boulevard de la liberté', 2);`,\n\t\t`INSERT INTO address (street, street_number) VALUES ('avenue des champs', 12);`,\n\t\t`INSERT INTO user_addresses (address_id, user_id) VALUES (2, 1);`,\n\t\t`INSERT INTO user_addresses (address_id, user_id) VALUES (4, 1);`,\n\t\t`INSERT INTO user_addresses (address_id, user_id) VALUES (2, 2);`,\n\t\t`INSERT INTO user_addresses (address_id, user_id) VALUES (2, 3);`,\n\t\t`INSERT INTO user_addresses (address_id, user_id) VALUES (4, 4);`,\n\t\t`INSERT INTO user_addresses (address_id, user_id) VALUES (4, 5);`,\n\t}\n\n\tdb, err := sql.Open(\"ramsql\", \"TestLoadUserAddresses\")\n\tif err != nil {\n\t\tt.Fatalf(\"sql.Open : Error : %s\\n\", err)\n\t}\n\tdefer db.Close()\n\n\tfor _, b := range batch {\n\t\t_, err = db.Exec(b)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"sql.Exec: Error: %s\\n\", err)\n\t\t}\n\t}\n\n\taddresses, err := LoadUserAddresses(db, 1)\n\tif err != nil {\n\t\tt.Fatalf(\"Too bad! unexpected error: %s\", err)\n\t}\n\n\tif len(addresses) != 2 {\n\t\tt.Fatalf(\"Expected 2 addresses, got %d\", len(addresses))\n\t}\n\n}\n```\n\nDone. No need for a running PostgreSQL or a setup. Your tests are isolated, and compliant with go tools.\n\n## RamSQL binary\n\nLet's say you have a SQL describing your application structure:\n\n```sql\nCREATE TABLE IF NOT EXISTS address (id BIGSERIAL PRIMARY KEY, street TEXT, street_number INT);\nCREATE TABLE IF NOT EXISTS user_addresses (address_id INT, user_id INT);\n```\n\nYou may want to test its validity:\n\n```console\n$ go install github.com/proullon/ramsql\n$ ramsql \u003c schema.sql\nramsql\u003e Query OK. 1 rows affected\nramsql\u003e Query OK. 1 rows affected\n$ echo $?\n0\n```\n\n## Features\n\nFind bellow all objectives for `v1.0.0`\n\n| Name           | Category      | Parsing                  | Implementation           |\n| -------------- | ------------- | ------------------------ | ------------------------ |\n| Table          | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| Schema         | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| CREATE         | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| PRIMARY_KEY    | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| DEFAULT        | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| INSERT         | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| UNIQUE         | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| FOREIGN KEY    | SQL           | :heavy_multiplication_x: | :heavy_multiplication_x: |\n| SELECT         | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| backtick       | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| quote          | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| double quote   | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| COUNT          | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| MAX            | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| ORDER BY       | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| UPDATE         | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| DELETE         | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| DROP           | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| INNER JOIN     | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| OUTER JOIN     | SQL           | :heavy_check_mark:       | :heavy_multiplication_x: |\n| timestamp      | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| now()          | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| OFFSET         | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| Transactions   | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| BEGIN          | SQL           | :heavy_multiplication_x: | :heavy_multiplication_x: |\n| COMMIT         | SQL           | :heavy_multiplication_x: | :heavy_multiplication_x: |\n| Index          | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| Hash index     | SQL           | :heavy_check_mark:       | :heavy_check_mark:       |\n| B-Tree index   | SQL           | :heavy_check_mark:       | :heavy_multiplication_x: |\n| JSON           | SQL           | :heavy_multiplication_x: | :heavy_multiplication_x: |\n| AS             | SQL           | :heavy_multiplication_x: | :heavy_multiplication_x: |\n| CLI            | Testing       | :heavy_check_mark:       | :heavy_check_mark:       |\n| Breakpoint     | Testing       | :heavy_multiplication_x: | :heavy_multiplication_x: |\n| Query history  | Testing       | :heavy_multiplication_x: | :heavy_multiplication_x: |\n| Size limit     | Testing       | :heavy_multiplication_x: | :heavy_multiplication_x: |\n| Autogeneration | Testing       | :heavy_multiplication_x: | :heavy_multiplication_x: |\n| TTL            | Caching       | :heavy_multiplication_x: | :heavy_multiplication_x: |\n| LFRU           | Caching       | :heavy_multiplication_x: | :heavy_multiplication_x: |\n| Gorm           | Compatibility | :heavy_check_mark:       | :heavy_check_mark:       |\n\n### Unit testing\n\n- Full isolation between tests\n- No setup (either file or databases)\n- Good performance\n\n### SQL parsing\n\n- Database schema validation\n- ALTER file validation\n\n### Stress testing\n\n- File system full error with configurable maximum database size\n- Random configurable slow queries\n- Random connection error\n\n## Compatibility\n\n### GORM\n\nIf you intend to use ramsql with the GORM ORM, you should use the GORM Postgres driver. A working example would be:\n\n```go\nimport (\n\t\"database/sql\"\n\t\"testing\"\n\n\t\"gorm.io/driver/postgres\"\n\t\"gorm.io/gorm\"\n)\n\ntype Product struct {\n\tgorm.Model\n\tCode       string\n\tPrice      uint\n\tTestBigint uint64 `gorm:\"test_bigint;type:BIGINT UNSIGNED AUTO_INCREMENT\"`\n}\n\n// From https://gorm.io/docs/connecting_to_the_database.html\n// and  https://gorm.io/docs/\nfunc main() {\n\tramdb, err := sql.Open(\"ramsql\", \"TestGormQuickStart\")\n\n\tdb, err := gorm.Open(postgres.New(postgres.Config{\n\t\tConn: ramdb,\n\t}),\n\t\t\u0026gorm.Config{})\n\n\t// Migrate the schema\n\terr = db.AutoMigrate(\u0026Product{})\n\n\t// Create\n\terr = db.Create(\u0026Product{Code: \"D42\", Price: 100}).Error\n\n\t// Read\n\tvar product Product\n\terr = db.First(\u0026product, 1).Error // find product with integer primary key\n\terr = db.First(\u0026product, \"code = ?\", \"D42\").Error // find product with code D42\n\terr = db.First(\u0026product, \"Code = ?\", \"D42\").Error // find product with code D42\n\n\t// Update - update product's price to 200\n\terr = db.Model(\u0026product).Update(\"Price\", 200).Error\n\t// Update - update multiple fields\n\terr = db.Model(\u0026product).Updates(Product{Price: 200, Code: \"F42\"}).Error // non-zero fields\n\terr = db.Model(\u0026product).Updates(map[string]interface{}{\"Price\": 200, \"Code\": \"F42\"}).Error\n\n\t// Delete - delete product\n\terr = db.Delete(\u0026product, 1).Error\n\n    _ = err\n}\n```\n\n## Architecture\n\n### Rows storage and garbage collector\n\nWhat options do we have to store objects:\n\n- unsafe memory paging\n- map\n- slice\n- linked list\n\nThe issue with having a lot of objects (rows here) in memory is Garbage Collector pause. The processus will lock objects to determine if there is any pointers to it. This becomes an issue with `map[any]*Something` since GC will lock the map to check all pointers.\n\nUnsafe memory paging is a bit tricky to keep portable.\n\nSlices are nice, and could grow non linearly capped with available RAM.\n\nThe simplest option regarding GC pause and rows storage is linked list. Easy to update and remove rows without overhead, while keeping GC functioning properly.\n\n### Indexes\n\nWe want Hash index to fetch rows in `O(1)` time with `=` operator. This means we need to use a map, without using pointers. That's where `uintptr` comes to play. Hash index uses `map[string]uintptr` or `map[int64]uintptr` to keep track of pointer to linked list elements, while discarding GC checks.\n\nWe also want Binary Tree index to fetch rows in `O(log(n))` time with `\u003c, \u003c=, \u003e, \u003e=` operators.\n\n### Transactions\n\n`RamSQL` only uses table level lock transactions. In case of error or call to `Rollback()`, changes will be reverted back into modified relation.\n\n`Commit()` releases the locks.\n\n## TODO\n\n- `agnostic` -\u003e `memstore`\n- `executor` -\u003e `sql`\n","funding_links":[],"categories":["Misc","Go"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fproullon%2Framsql","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fproullon%2Framsql","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fproullon%2Framsql/lists"}