{"id":20285444,"url":"https://github.com/mramshaw/restful-couchbase","last_synced_at":"2026-05-03T12:36:28.651Z","repository":{"id":92906180,"uuid":"172590188","full_name":"mramshaw/RESTful-Couchbase","owner":"mramshaw","description":"An experiment with using Couchbase as a drop-in replacement for PostgreSQL","archived":false,"fork":false,"pushed_at":"2020-11-09T14:33:58.000Z","size":1152,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-07-26T10:01:21.805Z","etag":null,"topics":["couchbase","docker","docker-compose","document-store","go","golang","n1ql","nosql","nosql-database","restful","tdd"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mramshaw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-25T21:45:47.000Z","updated_at":"2020-11-09T14:34:01.000Z","dependencies_parsed_at":"2023-04-29T00:55:15.885Z","dependency_job_id":null,"html_url":"https://github.com/mramshaw/RESTful-Couchbase","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/mramshaw/RESTful-Couchbase","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mramshaw%2FRESTful-Couchbase","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mramshaw%2FRESTful-Couchbase/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mramshaw%2FRESTful-Couchbase/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mramshaw%2FRESTful-Couchbase/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mramshaw","download_url":"https://codeload.github.com/mramshaw/RESTful-Couchbase/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mramshaw%2FRESTful-Couchbase/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32569714,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-03T06:36:36.687Z","status":"ssl_error","status_checked_at":"2026-05-03T06:36:09.306Z","response_time":103,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["couchbase","docker","docker-compose","document-store","go","golang","n1ql","nosql","nosql-database","restful","tdd"],"created_at":"2024-11-14T14:26:43.847Z","updated_at":"2026-05-03T12:36:28.630Z","avatar_url":"https://github.com/mramshaw.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# RESTful Couchbase\n\n[![Build status](http://travis-ci.org/mramshaw/RESTful-Couchbase.svg?branch=master)](http://travis-ci.org/mramshaw/RESTful-Couchbase)\n[![Coverage Status](http://codecov.io/github/mramshaw/RESTful-Couchbase/coverage.svg?branch=master)](http://codecov.io/github/mramshaw/RESTful-Couchbase?branch=master)\n[![Go Report Card](http://goreportcard.com/badge/github.com/mramshaw/RESTful-Couchbase?style=flat-square)](http://goreportcard.com/report/github.com/mramshaw/RESTful-Couchbase)\n[![GitHub release](http://img.shields.io/github/v/release/mramshaw/RESTful-Couchbase?style=flat-square)](http://github.com/mramshaw/RESTful-Couchbase/releases)\n\nAn experiment with using [Couchbase](http://docs.couchbase.com/home/) as\na drop-in replacement for PostgreSQL.\n\n## Contents\n\nThe contents are as follows:\n\n* [Rationale](#rationale)\n* [Features](#features)\n* [Couchbase](#couchbase)\n    * [Eventually Consistent](#eventually-consistent)\n    * [Views](#views)\n    * [Caveats](#caveats)\n    * [Transactions, Sagas and locking](#transactions-sagas-and-locking)\n    * [Getting familiar with Couchbase](02-Couchbase-Introduction.md)\n* [Couchbase Performance Tips](03-Couchbase-Performance-Tips.md)\n    * [Query by KEYS rather than by id](03-Couchbase-Performance-Tips.md#query-by-keys-rather-than-by-id)\n    * [Specify \"AdHoc(false)\" to cache queries](03-Couchbase-Performance-Tips.md#specify-adhocfalse-to-cache-queries)\n    * [Track prepared statement performance](03-Couchbase-Performance-Tips.md#track-prepared-statement-performance)\n* [Operations](#operations)\n    * [To Build](#to-build)\n    * [To Run](#to-run)\n    * [For testing](#for-testing)\n    * [See what's running](#see-whats-running)\n    * [View the build and/or execution logs](#view-the-build-andor-execution-logs)\n    * [To Shutdown](#to-shutdown)\n    * [Clean up](#clean-up)\n    * [Results](#results)\n* [Versions](#versions)\n* [Reference](#reference)\n    * [Couchbase BLOG](#couchbase-blog)\n* [To Do](#to-do)\n\n## Rationale\n\nThis builds on my [RESTful-Recipes](http://github.com/mramshaw/RESTful-Recipes) repo,\nwhich stores data in [PostgreSQL](http://www.postgresql.org/).\n\nAll dependencies are handled via [Docker](http://www.docker.com/products/docker) and [docker-compose](https://github.com/docker/compose).\n\nTDD (Test-Driven Development) is implemented; the build will fail if the tests do not pass.\n\nLikewise the build will fail if either __golint__ or __go vet__ fails.\n\nEnforces industry-standard __gofmt__ code formatting.\n\nAll testing can be done with [curl](CURLs.txt).\n\nWe will use parameterized N1QL to prevent SQL injection.\n\n## Features\n\n- uses [Gorilla MUX](http://github.com/Gorilla/mux)\n- uses [Go couchbase driver](http://blog.couchbase.com/go-sdk-1.0-ga/)\n\n## Couchbase\n\nCouchbase is a document-oriented database based on the JSON document model.\n\nWhat would be referred to as a __table__ or __row__ in a relational database\nis referred to as a __document__ in Couchbase. Like other NoSQL databases,\ndocuments may have embedded elements. In this regard, Couchbase documents are\nmore akin to object storage than relational database rows or records.\n\nWhat would be referred to as a __database__ in a relational database seems\nto be referred to as a __bucket__ in Couchbase.\n\nLikewise, what would normally be referred to as a __server__ seems to be\nreferred to as a __cluster__ in Couchbase.\n\nAs with other NoSQL databases (such as DynamoDB), schemas are flexible.\n\nUnusually, features master-master replication (all nodes are identical).\n\nN1QL (the SQL-like Couchbase query language) operates on JSON documents,\nreturning JSON documents.\n\nSimiliar to __redis__ and __Cassandra__, data may be assigned arbitrary\nexpiry times.\n\nUnlike __Cassandra__, Couchbase has support for __joins__.\n\nCouchbase is packaged with an Admin Console GUI. Other NoSQL solutions\n(such as __MongoDB__ and __Cassandra__) apparently are not packaged with\nadministrative consoles (although third-party consoles are available).\n\n#### Eventually Consistent\n\nThe Couchbase documentation seem to be somewhat inconsistent. For instance:\n\n\u003e The basic storage and indexing sequence is:\n\u003e\n\u003e 1. A document is stored within the cluster. Initially the document is stored only in RAM.\n\u003e\n\u003e 2. The document is communicated to the indexer through replication to be indexed by views.\n\u003e\n\u003e This sequence means that the view results are eventually consistent with what is stored in\n\u003e memory based on the latency in replication of the change to the indexer. It is possible to\n\u003e write a document to the cluster and access the index without the newly written document\n\u003e appearing in the generated view.\n\u003e\n\u003e Conversely, documents that have been stored with an expiry may continue to be included\n\u003e within the view until the document has been removed from the database by the expiry pager.\n\u003e\n\u003e Couchbase Server supports the Observe command, which enables the current state of a document\n\u003e and whether the document has been replicated to the indexer or whether it has been considered\n\u003e for inclusion in an index.\n\u003e\n\u003e When accessing a view, the contents of the view are asynchronous to the stored documents.\n\u003e In addition, the creation and updating of the view is subject to the `stale` parameter.\n\u003e This controls how and when the view is updated when the view content is queried.\n\n    http://docs.couchbase.com/server/6.0/learn/views/views-store-data.html#document-storage-and-indexing-sequence\n\n[Note that the above makes no mention of the whether or not the document has been persisted to disk.]\n\nFor more on the `stale` parameter:\n\n    http://docs.couchbase.com/server/6.0/learn/views/views-operation.html#index-stale\n\nNote the following:\n\n\u003e * stale=false\n\u003e\n\u003e The index is updated before you execute the query, making sure that any documents updated and\n\u003e persisted to disk are included in the view. The client will wait until the index has been\n\u003e updated before the query has executed and, therefore, the response will be delayed until the\n\u003e updated index is available.\n\n[This means that it is possible to request __only__ data that has been persisted to disk.]\n\n#### Views\n\nCouchbase __views__ are described as \"eventually consistent\".\n\n\u003e Views are eventually consistent compared to the underlying stored documents.\n\u003e Documents are included in views when the document data is persisted to disk.\n\n    http://docs.couchbase.com/server/6.0/learn/views/views-intro.html\n\nContrast the above with the following:\n\n\u003e Views are updated as the document data is updated in memory. There may be a delay between the\n\u003e document being created or updated and the document being updated within the view depending\n\u003e on the client-side query parameters.\n\n    http://docs.couchbase.com/server/6.0/learn/views/views-operation.html\n\n[Presumably this reflects how the `stale` parameter is set.]\n\nWhat this ___may___ mean in practice is that Couchbase views are not updated\nuntil the underlying documents they reference are persisted to disk - and ___not___\nwhen the document updates are acknowledged and stored in memory (see [Caveats](#caveats)\nbelow). [Presumably this is also when the indexes the views reference that track\nthese documents are updated.]\n\nThe exception to this is __DCP__ (Database Change Protocol) - which is available for stream-based views:\n\n\u003e With DCP, data does not need to be persisted to disk before retrieving it with a view query.\n\n    http://docs.couchbase.com/server/6.0/learn/views/views-streaming.html\n\n#### Caveats\n\nCouchbase has a __memory-first__ architecture, which means that the results\nof write operations are acknowledged when stored in memory (they are then\nqueued to be asynchronously written to disk and/or then replicated to another\nnode). So if an operation is written to memory, and the system shuts down\nimmediately afterwards, then that operation may not persist.\n\nThis is the __default__ behaviour, and potentially violates the __D__ of\nACID transactions. However, this behaviour can be over-ridden (at a small\nperformance cost) if durability requirements are critical.\n\nCouchbase has a maximum capacity of 20 MB per document (probably not an\nissue, but worth bearing in mind).\n\nDocument ids must be unique for the bucket in Couchbase (different document\ntypes must have unique document ids). Also, each Document id (key) must be\na string in Couchbase (i.e. `\"1\"` instead of `1`). For this reason, it\nseems to be normal practice to use a compound id (such as `\"user::5\"`); a\nvariation on this seems to be to embed a __type__ field within the document\n(such as `\"type\": \"user\"`).\n\n[The document id may be accessed via the document metadata: `META().id`.\n If using N1QL this value may be returned, however it is unclear to me\n how this value may be accessed from the Couchbase GOCB driver API.]\n\n#### Transactions, Sagas and locking\n\nCouchbase provides ACID transactions for single document operations, but multi-document transactions (or __sagas__) are not yet\nnatively supported (alpha third-party solutions are available).\n\nCouchbase offers both [optimistic and pessimistic locking](http://docs.couchbase.com/go-sdk/1.5/concurrent-mutations-cluster.html).\n\nLocking in Couchbase is implemented by CAS (Compare And Swap). This is basically a hash or digest, and will indicate if the document\nin question has been mutated (if it __has__ and the CAS has been supplied, then the update will fail). Alternatively, there are\n__lock__ and __unlock__ primitives (as well as __get\\_and\\_lock__). The lock time may be specified. Mutating the document will\nalso serve to unlock it.\n\n#### Getting familiar with Couchbase\n\nRefer to [Couchbase Introduction](02-Couchbase-Introduction.md) for a quick guide to getting started with Couchbase.\n\n## Couchbase Performance Tips\n\nRefer to [Couchbase Performance Tips](03-Couchbase-Performance-Tips.md) for some general performance tips.\n\n## Operations\n\nWe will use __Docker__ and __docker-compose__ to build and test our application.\n\n#### To Build:\n\nThe command to run:\n\n    $ docker-compose up\n\nThis should look as follows:\n\n```bash\n$ docker-compose up\nCreating network \"restfulcouchbase_couchnet\" with the default driver\nCreating restfulcouchbase_couchbase_1_d6da7719d982 ... done\nCreating restfulcouchbase_golang_1_cb1241403038    ... done\nAttaching to restfulcouchbase_couchbase_1_84f81eb2a871, restfulcouchbase_golang_1_6b5fa034bf5a\ncouchbase_1_84f81eb2a871 | + set -m\ncouchbase_1_84f81eb2a871 | + sleep 10\ncouchbase_1_84f81eb2a871 | + /entrypoint.sh couchbase-server\ncouchbase_1_84f81eb2a871 | Starting Couchbase Server -- Web UI available at http://\u003cip\u003e:8091\ncouchbase_1_84f81eb2a871 | and logs available in /opt/couchbase/var/lib/couchbase/logs\ncouchbase_1_84f81eb2a871 | + /opt/couchbase/bin/couchbase-cli cluster-init -c localhost --cluster-username halcouch --cluster-password couchpass --services data,index,query\ncouchbase_1_84f81eb2a871 | SUCCESS: Cluster initialized\ncouchbase_1_84f81eb2a871 | + /opt/couchbase/bin/couchbase-cli bucket-create -c localhost --username halcouch --password couchpass --bucket recipes --bucket-type couchbase --bucket-ramsize 100 --enable-flush=1\ncouchbase_1_84f81eb2a871 | SUCCESS: Bucket created\ncouchbase_1_84f81eb2a871 | + fg 1\ncouchbase_1_84f81eb2a871 | /entrypoint.sh couchbase-server\ngolang_1_6b5fa034bf5a | GOPATH=/go GOOS=linux GOARCH=amd64 gofmt -d -e -s -w *.go\ngolang_1_6b5fa034bf5a | GOPATH=/go GOOS=linux GOARCH=amd64 gofmt -d -e -s -w application/*.go\ngolang_1_6b5fa034bf5a | GOPATH=/go GOOS=linux GOARCH=amd64 gofmt -d -e -s -w recipes/*.go\ngolang_1_6b5fa034bf5a | GOPATH=/go GOOS=linux GOARCH=amd64 gofmt -d -e -s -w test/*.go\ngolang_1_6b5fa034bf5a | GOPATH=/go GOOS=linux GOARCH=amd64 golint -set_exit_status *.go\ngolang_1_6b5fa034bf5a | GOPATH=/go GOOS=linux GOARCH=amd64 golint -set_exit_status ./...\ngolang_1_6b5fa034bf5a | GOPATH=/go GOOS=linux GOARCH=amd64 go tool vet *.go\ngolang_1_6b5fa034bf5a | GOPATH=/go GOOS=linux GOARCH=amd64 go tool vet application/*.go\ngolang_1_6b5fa034bf5a | GOPATH=/go GOOS=linux GOARCH=amd64 go tool vet recipes/*.go\ngolang_1_6b5fa034bf5a | GOPATH=/go GOOS=linux GOARCH=amd64 go tool vet test/*.go\ngolang_1_6b5fa034bf5a | GOPATH=/go GOOS=linux GOARCH=amd64 go test -v test\ngolang_1_6b5fa034bf5a | === RUN   TestEmptyTables\ngolang_1_6b5fa034bf5a | --- PASS: TestEmptyTables (0.55s)\ngolang_1_6b5fa034bf5a | === RUN   TestGetNonExistentRecipe\ngolang_1_6b5fa034bf5a | --- PASS: TestGetNonExistentRecipe (0.41s)\ngolang_1_6b5fa034bf5a | === RUN   TestCreateRecipe\ngolang_1_6b5fa034bf5a | --- PASS: TestCreateRecipe (0.41s)\ngolang_1_6b5fa034bf5a | === RUN   TestGetRecipe\ngolang_1_6b5fa034bf5a | --- PASS: TestGetRecipe (0.39s)\ngolang_1_6b5fa034bf5a | === RUN   TestUpdatePutRecipe\ngolang_1_6b5fa034bf5a | --- PASS: TestUpdatePutRecipe (0.37s)\ngolang_1_6b5fa034bf5a | === RUN   TestUpdatePatchRecipe\ngolang_1_6b5fa034bf5a | --- PASS: TestUpdatePatchRecipe (0.34s)\ngolang_1_6b5fa034bf5a | === RUN   TestDeleteRecipe\ngolang_1_6b5fa034bf5a | --- PASS: TestDeleteRecipe (0.37s)\ngolang_1_6b5fa034bf5a | === RUN   TestAddRating\ngolang_1_6b5fa034bf5a | --- PASS: TestAddRating (0.39s)\ngolang_1_6b5fa034bf5a | === RUN   TestSearch\ngolang_1_6b5fa034bf5a | --- PASS: TestSearch (4.43s)\ngolang_1_6b5fa034bf5a | PASS\ngolang_1_6b5fa034bf5a | ok  \ttest\t9.691s\ngolang_1_6b5fa034bf5a | GOPATH=/go GOOS=linux GOARCH=amd64 go build -v -o restful_couchbase main.go\ngolang_1_6b5fa034bf5a | restful_couchbase has been compiled\ngolang_1_6b5fa034bf5a | ./restful_couchbase\ngolang_1_6b5fa034bf5a | type 'make serve' to run\n```\n\nOnce `make` indicates that `restful_couchbase` has been compiled, can change [docker-compose.yml](docker-compose.yml) as follows:\n\n1) Comment `command: bash -c \"sleep 15; make\"`\n\n2) Uncomment `#command: bash -c \"sleep 15; ./restful_couchbase\"`\n\n#### To Run\n\nThe command to run:\n\n    $ docker-compose up -d\n\nFor the first run, there will be a warning if `mramshaw4docs/golang-couchbase:1.15.4` has not been built.\n\nThis image will contain all of the Go dependencies and should only need to be built once.\n\nA successful `golang` startup should show the following as the last line of `docker-compose logs golang`:\n\n    golang_1    | 2019/03/01 19:05:05 Now serving recipes ...\n\nIf this line does not appear, repeat the `docker-compose up -d` command (there is no penalty for this).\n\n#### For testing:\n\n[Optional] Start couchbase:\n\n    $ docker-compose up -d couchbase\n\nStart golang [if couchbase is not running, this step will start it]:\n\n    $ docker-compose run --publish 80:8100 golang make\n\nSuccessful startup will be indicated as follows:\n\n    2019/03/01 19:05:05 Now serving recipes ...\n\nThis should make the web service available at:\n\n    http://localhost/v1/recipes\n\nOnce the service is running, it is possible to `curl` it. Check [CURLs.txt](CURLs.txt) for examples.\n\n#### See what's running:\n\nThe command to run:\n\n    $ docker ps\n\n#### View the build and/or execution logs\n\nThe command to run:\n\n    $ docker-compose logs golang\n\n#### To Shutdown:\n\nThe command to run:\n\n    $ docker-compose down\n\nClean up docker volumes as follows:\n\n\t$ docker volume prune\n\n#### Clean Up\n\nThe command to run:\n\n    $ docker-compose run golang make clean\n\nClean up docker image as follows:\n\n\t$ docker rmi mramshaw4docs/golang-couchbase:1.15.4\n\n#### Results\n\nDue to the ad-hoc nature of NoSQL documents, the code is somewhat more complicated\nthan would be the case with relational databases; however as I am using Couchbase\nas a drop-in replacement for PostgreSQL this is hardly a fair comparison. But for\nlearning the ins and outs of Couchbase, it's been a worthwhile exercise.\n\n## Versions\n\n* Couchbase - Community Edition - version __6.0.0__\n* Docker __18.09.7__\n* Docker-Compose __1.25.4__\n* Go __1.15.4__\n\nMore recent versions of these components should be fine.\n\n## Reference\n\nQuery Optimization Using Prepared (Optimized) Statements:\n\n    http://docs.couchbase.com/go-sdk/1.5/n1ql-query.html#prepare-stmts\n\n[My feeling is that using ___named___ parameters is more self-documenting (and may\n therefore result in fewer bugs) than using ___positional___ parameters.]\n\nConcurrent Document Mutations:\n\n    http://docs.couchbase.com/go-sdk/1.5/concurrent-mutations-cluster.html\n\nViews:\n\n    http://docs.couchbase.com/server/6.0/learn/views/views-intro.html\n\n#### Couchbase BLOG\n\nFor general articles on Couchbase, their [BLOG](http://blog.couchbase.com/) is the place to start.\n\nI found the following articles useful:\n\n    http://blog.couchbase.com/moving-from-sql-server-to-couchbase-part-1-data-modeling/\n\n[To a large extent, the use of JSON in Couchbase removes the need for OR/M tooling.]\n\n    http://blog.couchbase.com/sql-server-couchbase-data-migration/\n\n[The article states there is no __date__ primitive in JSON. While this may be technically\n true, for most uses the __time__ primitive will suffice instead.]\n\n    http://blog.couchbase.com/moving-sql-server-couchbase-app-migration/\n\n[The article focuses on migrating from SQL Server but is useful for other databases.]\n\n    http://blog.couchbase.com/comparing-couchbase-views-couchbase-n1ql-indexing/\n\n[Compares and contrasts Couchbase Views with Couchbase N1QL \u0026 Indexing using GSI.]\n\n## To Do\n\n- [x] Learn [N1QL](http://docs.couchbase.com/server/6.0/getting-started/try-a-query.html)\n- [ ] Investigate the use of UUIDs as well as replication consensus procedures\n- [ ] Investigate using views to enforce constraints (indexes are a performance nightmare)\n- [x] Upgrade to latest release of Golang (__1.15.4__ as of the time of writing)\n- [x] Upgrade to latest release of Couchbase (__6.6.0__ as of the time of writing)\n- [x] Upgrade `release` badge to conform to new Shields.io standards\n- [ ] Investigate the use of `n1qlResp.Metrics.MutationCount`\n- [x] Add Travis CI build process \u0026 code coverage reporting\n- [x] Add pessimistic locking to updates\n- [ ] Update build process to `vgo`\n- [ ] Add tests for data TTL\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmramshaw%2Frestful-couchbase","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmramshaw%2Frestful-couchbase","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmramshaw%2Frestful-couchbase/lists"}