{"id":25455551,"url":"https://github.com/fujio-turner/sg-sync-function-tester","last_synced_at":"2025-06-23T00:06:48.575Z","repository":{"id":249291200,"uuid":"831063976","full_name":"Fujio-Turner/sg-sync-function-tester","owner":"Fujio-Turner","description":"Couchbase Sync Gateway and App Services - Sync Function Tester","archived":false,"fork":false,"pushed_at":"2025-03-04T03:16:27.000Z","size":6654,"stargazers_count":2,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-05-16T16:49:57.553Z","etag":null,"topics":["capella-app-services","couchbase","couchbase-capella","couchbase-mobile","couchbase-sync-gateway","sync-gateway"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Fujio-Turner.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-07-19T15:17:30.000Z","updated_at":"2025-05-09T10:59:06.000Z","dependencies_parsed_at":"2025-02-06T16:35:07.102Z","dependency_job_id":"98494ee1-7dea-47bf-8a39-dd2444970d3a","html_url":"https://github.com/Fujio-Turner/sg-sync-function-tester","commit_stats":null,"previous_names":["fujio-turner/sg-sync-function-tester"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Fujio-Turner/sg-sync-function-tester","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Fujio-Turner%2Fsg-sync-function-tester","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Fujio-Turner%2Fsg-sync-function-tester/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Fujio-Turner%2Fsg-sync-function-tester/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Fujio-Turner%2Fsg-sync-function-tester/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Fujio-Turner","download_url":"https://codeload.github.com/Fujio-Turner/sg-sync-function-tester/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Fujio-Turner%2Fsg-sync-function-tester/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261386809,"owners_count":23150873,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["capella-app-services","couchbase","couchbase-capella","couchbase-mobile","couchbase-sync-gateway","sync-gateway"],"created_at":"2025-02-18T01:00:18.883Z","updated_at":"2025-06-23T00:06:43.561Z","avatar_url":"https://github.com/Fujio-Turner.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Sync Gateway Sync Function Tester\n\n## PROBLEM\n\nDon't want to touch the Sync Gateway Sync Function with a 10-foot pole because who knows what's going to happen to your writes and channel process when you change it?\n\n## SOLUTION\n\nI wrote this simple Python script to test the Sync Function of Sync Gateway with CRUD operations from:\n- different users\n- different docTypes\n\n## WHAT IT DOES\n\n\u003cimg src=\"img/sg_sync_function_tester_v2.png\" alt=\"Project screenshot\" width=\"787\" height=\"380\"\u003e\n\n\nThis script will:\n\n1. read a `config.json` file with your settings(usernames , URLs \u0026 commands)\n2. opens a folder of individual JSON files: default folder `jsons`\n3. process each of the JSON file in the folder as HTTP [GET, PUT, DELETE, CHANGES, GET_ADMIN, PUT_ADMIN, DELETE_ADMIN , CHANGES_ADMIN and/or PURGE] against your SG endpoint to test your Sync Function\n4. writes the results into a log file as `failed` or `success` by HTTP type \u0026 timestamp\n\n**NOTE:** a `failed` PUT might be a good thing as the Sync Function is denying a particular write.\n\nThe script will do all the above as different user(s) from the `config.json` file.\n\n## REQUIREMENTS \n- A Running Sync Gateway w/ one or more known Sync Gateway USERS (cbl user)\n- Python 3\n- Python Requests Library: https://docs.python-requests.org/en/latest/index.html\n\nTo install the required library:\n```sh\npip install requests\n```\n\n## HOW TO USE\n\n**Step 1.** Update the `config.json` with your Sync Gateway hostname, one or more CBL test users, and other settings you want.\n\n**Step 2.** In the folder specified in your config (default is `jsons`), put sample JSON docs as individual files. Example: `{\"_id\":\"foo\",\"channels\":[\"bob\"]}` saved in a file `foo.json`.  \n\n**Step 3.** Run the Python code \n\n```sh\npython3 sg_sync_function_tester.py config.json\n```\n\n**OUTPUT:** It will output a log file of the run: `sync_gateway_log_{date}_{time}.log`\n\nExample output:\n```\n2024-07-22T17:06:30.000 - INFO - [success] - [GET] - [bob] - GET result for [foo] - {\"_id\": \"foo\", \"_rev\": \"4-42bc40048cf41b74565750f2d2c55b28\", \"channels\": [\"bob\"], \"dateTimeStamp\": \"2024-07-22T17:03:56.604936\"}\n2024-07-22T17:06:30.000 - INFO - [success] - [PUT] - [bob] - PUT result for [foo] - {\"id\": \"foo\", \"ok\": true, \"rev\": \"5-b9b6eb547de2d751b83457c29610994c\"}\n2024-07-22T17:06:30.000 - INFO - [success] - [CHANGES] - [bob] - Changes feed result for [foo], rows: 2 - {\"results\": [{\"seq\": 1, \"id\": \"_user/bob\", \"changes\": []}, {\"seq\": 163, \"id\": \"foo\", \"changes\": [{\"rev\": \"5-b9b6eb547de2d751b83457c29610994c\"}]}], \"last_seq\": \"163\"}\n\n```\n\n## Operations\n\nThe script supports the following operations:\n\n- `GET`: Retrieve a document\n    - `GET_RAW`: Retrieve a document with full meta data `_sync` (admin only)\n- `PUT`: Create or update a document\n- `DELETE`: Delete a document\n- `CHANGES`: Get the changes feed. To filter channel(s) on the change just add your channel(s) names: `CHANGES:bob` or `CHANGES:bob,water`\n- `PURGE`: Purge a document (admin only)\n- `SLEEP`: Pause execution for a specified number of seconds\n\nAdmin versions of operations are available by appending `_ADMIN` to the operation name (e.g., `GET_ADMIN`, `PUT_ADMIN`).\n\n\u003cmark style=\"background-color: yellow;\"\u003e**Warning**\u003c/mark\u003e: If you do `CHANGES_ADMIN` without a channel filter like `CHANGES_ADMIN:bob` Sync Gateway will process your whole database `_changes` feed in the output.\n\n### SLEEP Operation\n\nThe `SLEEP` operation allows you to introduce a delay between other operations. This can be useful for testing time-sensitive scenarios or rate limiting.\n\n- `SLEEP`: Pauses execution for 1 second\n- `SLEEP:X`: Pauses execution for X seconds (where X is an integer)\n\nExample usage in the `operations` list(sleeps for 3 seconds):\n\n```json\n\"operations\": [\n    \"PUT_ADMIN\",\n    \"SLEEP:3\",\n    \"GET\",\n    \"SLEEP\",\n    \"DELETE\"\n]\n```\n\n\n## EXAMPLES\n\nYou can copy and paste the `config.json` file and rename them to run tests. `example_config` folder has below examples.\n\n```sh\npython3 sg_sync_function_tester.py 1.user_put-user_get.json\npython3 sg_sync_function_tester.py 2.user_put-user_changes.json\npython3 sg_sync_function_tester.py 3.admin_put-user_delete.json\n```\n**PRO TIP** The config `3.admin_put-user_delete.json` is a great way to see if the user can delete \"READ-ONLY\" documents.\n\n## config.json\n\n```json\n{\n    \"sgHost\": \"http://localhost\",\n    \"sgPort\": \"4984\",\n    \"sgAdminPort\": \"4985\",\n    \"sgDb\": \"sync_gateway\",\n    \"sgDbScope\":\"_default\",\n    \"sgDbCollection\":\"_default\",\n    \"sgTestUsers\": [\n        {\"userName\": \"bob\", \"password\": \"12345\", \"sgSession\": \"\"}\n    ],\n    \"sgAdminUser\": \"Administrator\", // Required if you want to do Admin Operations\n    \"sgAdminPassword\": \"password\",  // Required if you want to do Admin Operations\n    \"jsonFolder\": \"jsons\",   // Folder containing all your individual json files\n    \"logPathToWriteTo\": \"sync_gateway_log\",\n    \"debug\": false,\n    \"operations\": [\"GET\", \"PUT\", \"DELETE\", \"CHANGES\", \"GET_ADMIN\", \"PUT_ADMIN\", \"DELETE_ADMIN\", \"CHANGES_ADMIN\",\"SLEEP:3\",\"GET_RAW\",\"PURGE\"]  // Specify the order of operations and/or indivdual operations\n}\n```\n\n### **Scopes and Collection**\nIf your not using scopes and collections for mobile yet just leave the default values of: `\"sgDbScope\":\"_default\"` and `\"sgDbCollection\":\"_default\"`. This script will not test all your collections at once. You have to have a different `config.json` for each collection Sync Function.\n\n### **PRO TIP**\n\"PURGE\" is a great way to clean up data between tests. It literally 100% removes the document from Sync Gateway and the Couchbase Bucket. NOTE: PURGE is a Sync Gateway Admin function. In the config.json, you'll need to add Sync Admin (Couchbase Server RBAC [`Sync Gateway Architect`](https://docs.couchbase.com/server/current/learn/security/roles.html#sync-gateway-configurator) ) credentials for `sgAdminUser` and `sgAdminPassword`. Link here for [Offical Docs for: POST {db}/_purge](https://docs.couchbase.com/sync-gateway/current/rest-api-admin.html#/Document/post_keyspace__purge)\n\n\n## UNDERSTANDING THE SYNC FUNCTION\nHere is a link to understand what the Sync Function can and can not do.\n- [Offical Docs for Sync Gateway's Sync Function](https://docs.couchbase.com/sync-gateway/current/sync-function.html#ex-sync-function)\n\n#### Sample Sync Functions:\n- [Github.com - Sync Function with Security](example_sync_functions/3.sync_function_run.js)\n- [Github.com - sync_gateway](https://github.com/couchbase/sync_gateway/blob/main/examples/database_config/sync-function.json)\n- [Github.com - travel-sample](https://github.com/couchbaselabs/mobile-travel-sample/blob/master/sync-gateway-config-travelsample-docker.json#L65)\n- [Github.com - todo](https://github.com/couchbaselabs/mobile-training-todo/blob/release/helium/docker/sg-setup/config/sync-function.json#L7)\n\n\n\n### Key Changes:\n1. **Updated the `operations` Configuration**: Added an `operations` attribute to specify the list of operations to perform.\n2. **Dynamic Operation Execution**: The script now executes operations in the order specified in the `operations` array.\n3. **Clarified Usage Instructions**: Updated the usage instructions to reflect the new configuration options.\n4. **Admin Operations**: Your Sync Function might have certain restrictions at a USER level, but you still need to GET and PUT docs. There are now admin equivalents to the operations.\n5. **Sleep Operation**: Added a new `SLEEP` operation that allows pausing execution between other operations. This can be useful for testing time-sensitive scenarios or simulating delays.\n6. **HTTP GET /_raw/{docId}**: Added a new `GET_RAW` operation that allows you to get the document from Sync Gateway exactly how it is stored in Couchbase Server includes all the meta / bookkeeping data from `_sync`.\n7. **Scopes and Collection**: In the `config.json` just pass in non-default(`_default`) value for `sgDbScope` and `sgDbCollection` to test scopes and collection Sync Functions.\n8. **Changes Channel(s) Filter**: Add the channel(s) you want to filter by in the changes operation like this:`CHANGES:bob` .\n\n\nWorks on My Computer - Tested \u0026 Certified ;-)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffujio-turner%2Fsg-sync-function-tester","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffujio-turner%2Fsg-sync-function-tester","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffujio-turner%2Fsg-sync-function-tester/lists"}