{"id":18639102,"url":"https://github.com/zseta/carepet-python","last_synced_at":"2025-11-04T16:30:33.189Z","repository":{"id":109652536,"uuid":"586576297","full_name":"zseta/carepet-python","owner":"zseta","description":null,"archived":false,"fork":false,"pushed_at":"2023-01-08T21:40:56.000Z","size":21,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-12-27T09:29:43.742Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zseta.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-08T16:16:14.000Z","updated_at":"2023-01-08T21:42:20.000Z","dependencies_parsed_at":null,"dependency_job_id":"d3b9cab1-f952-4c30-bf8e-779ec8c8c5da","html_url":"https://github.com/zseta/carepet-python","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zseta%2Fcarepet-python","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zseta%2Fcarepet-python/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zseta%2Fcarepet-python/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zseta%2Fcarepet-python/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zseta","download_url":"https://codeload.github.com/zseta/carepet-python/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239436467,"owners_count":19638322,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T05:45:03.286Z","updated_at":"2025-11-04T16:30:33.134Z","avatar_url":"https://github.com/zseta.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Care Pet ScyllaDB IoT example\n\nThis is an example project that demonstrates a generic IoT use case with\nScyllaDB in Python.\n\nThe project simulates an IoT application for pet owners to monitor a variety\nof metrics about their pets (for example heart rate or temperature).\n\nThe application has three modules:\n\n* Migrate (`python src/migrate.py`) - creates keyspace and tables in ScyllaDB\n* Sensor (`python src/sensor.py`) - generates random IoT data and inserts it into ScyllaDB\n* API (`python src/api.py`) - REST API service to fetch data from ScyllaDB\n\n## Get started\n\n### Prerequisites:\n* [Python 3.7+](https://www.python.org/downloads/)\n* [Virtualenv](https://virtualenv.pypa.io/en/latest/installation.html)\n* [docker](https://www.docker.com/)\n* [docker-compose](https://docs.docker.com/compose/)\n\n### Clone repository and install dependencies\nClone the repository and open the root directory of the project:\n```bash\ngit clone https://github.com/zseta/carepet-python\ncd carepet-python\n```\n\nCreate a new virtual environment and activate it:\n```bash\nvirtualenv env\nsource env/bin/activate\n```\n\nInstall all Python dependencies:\n```bash\npip install -r requirements.txt\n```\n\n### Start Docker containers (skip this if you use Scylla Cloud)\nSpin up a local ScyllaDB cluster with three nodes using `docker` and `docker-compose`:\n```bash\ndocker-compose up -d\n\nCreating carepet-scylla3 ... done\nCreating carepet-scylla2 ... done\nCreating carepet-scylla1 ... done\n```\n\nThis command starts three ScyllaDB nodes in containers:\n* `carepet-scylla1`\n* `carepet-scylla2`\n* `carepet-scylla3`\n\nYou can inspect any of these nodes by using the `docker inspect` command,\nfor example:\n```bash\ndocker inspect carepet-scylla1\n\n[\n    {\n        \"Id\": \"c87128b7d0ca4a31a84da78875c8b4181283c34783b6b0a78bffbacbbe45fcc2\",\n        \"Created\": \"2023-01-08T21:17:13.212585687Z\",\n        \"Path\": \"/docker-entrypoint.py\",\n        \"Args\": [\n            \"--smp\",\n            \"1\"\n        ],\n        \"State\": {\n            \"Status\": \"running\",\n            \"Running\": true,\n...\n```\n\n### Connect to ScyllaDB and create the database schema\nTo connect to your ScyllaDB storage within the container, you need to know the\nIP address of one of the running nodes.\nThis is how you can get the IP address of the first node running in the container:\n```bash\ndocker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' carepet-scylla1\n```\n\nYou will need to reference this value multiple times later so if it's easier\nfor you can save it as a variable `NODE1`:\n```bash\nNODE1=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' carepet-scylla1)\n```\n\nNow you can run the migration script that creates the required keyspace and tables:\n```bash\npython src/migrate.py -h $NODE1\n\nCreating keyspace...\nDone.\nMigrating database...\nDone.\n```\n\nSee the database schema using [cqlsh](https://cassandra.apache.org/doc/latest/cassandra/tools/cqlsh.html) in the container:\n\n```bash\ndocker exec -it carepet-scylla1 cqlsh\ncqlsh\u003e DESCRIBE KEYSPACES;\n\ncarepet        system_auth  system_distributed_everywhere  system_traces\nsystem_schema  system       system_distributed \n\ncqlsh\u003e USE carepet;\ncqlsh:carepet\u003e DESCRIBE TABLES;\n\nowner  pet  sensor  sensor_avg  measurement\n\ncqlsh:carepet\u003e DESCRIBE TABLE pet;\n\nCREATE TABLE carepet.pet (\n    owner_id uuid,\n    pet_id uuid,\n    address text,\n    age int,\n    name text,\n    weight float,\n    PRIMARY KEY (owner_id, pet_id)\n) WITH CLUSTERING ORDER BY (pet_id ASC)\n    AND bloom_filter_fp_chance = 0.01\n    AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}\n    AND comment = ''\n    AND compaction = {'class': 'SizeTieredCompactionStrategy'}\n    AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}\n    AND crc_check_chance = 1.0\n    AND dclocal_read_repair_chance = 0.0\n    AND default_time_to_live = 0\n    AND gc_grace_seconds = 864000\n    AND max_index_interval = 2048\n    AND memtable_flush_period_in_ms = 0\n    AND min_index_interval = 128\n    AND read_repair_chance = 0.0\n    AND speculative_retry = '99.0PERCENTILE';\n\ncqlsh:carepet\u003e exit;\n\n\n```\n\nAt this point you have ScyllaDB running with the correct keyspace and tables.\n\n### Generate and ingest IoT data\nStart ingesting IoT data (it's suggested to do this in a new separate terminal\nbecause this process runs indefinitely). Make sure you're still in the virtual\nenvironment:\n```bash\nsource env/bin/activate\nNODE1=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' carepet-scylla1)\npython src/sensor.py -h $NODE1 --measure 2 --buffer-interval 10\n\nWelcome to the Pet collar simulator\nNew owner # 1cfbc0e5-6b05-476d-b170-2660cf40c02a\nNew pet # 1a0800ee-7643-4794-af7b-2ecaaf7078fc\nNew sensor(0) # b6155934-bd4e-47de-8649-1fad447aa036\nNew sensor(1) # d2c62c4d-9621-469d-b62c-41ef2271fca7\nsensor # b6155934-bd4e-47de-8649-1fad447aa036 type T, new measure: 100.55118431400851, ts: 2023-01-08 17:36:17.126374\nsensor # d2c62c4d-9621-469d-b62c-41ef2271fca7 type L, new measure: 37.486651732296835, ts: 2023-01-08 17:36:17.126516\n```\n\nThis command starts a script that generates and ingests random IoT data coming\nfrom two sensors every other second and inserts the data in batches\nevery ten seconds. Whenever you see `Pushing data` in the command line that is\nwhen data actually gets insterted into ScyllaDB.\n\nOptional: You can modify the frequency of the generated data by changing the\n`--measure` and `--buffer-interval` arguments. For example,\nyou can generate new data points every three seconds and insert the batches\nevery 30 seconds:\n```bash\nsource env/bin/activate\nNODE1=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' carepet-scylla1)\npython src/sensor.py -h $NODE1 --measure 3 --buffer-interval 30\n```\n\nYou can run multiple ingestion processes in parallel if you wish.\n\n### Set up and test REST API\nIn a new terminal, start running the API server (make sure that `port 8000` is free):\n```bash\nsource env/bin/activate\nNODE1=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' carepet-scylla1)\npython src/api.py -h $NODE1\n\nINFO:     Started server process [696274]\nINFO:     Waiting for application startup.\nINFO:     Application startup complete.\nINFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)\n```\n\nThe API server is running on `http://127.0.0.1:8000`. Test with your\nbrowser, or curl, if it works properly:\n```bash\ncurl http://127.0.0.1:8000\n\n{\"message\":\"Pet collar simulator API\"}\n```\n\nNext, you will test the following API endpoints:\n* `/api/owner/{owner_id}`\n\n    Returns all available data fields about the owner.\n* `/api/owner/{owner_id}/pets`\n\n    Returns the owner's pets.\n* `/api/pet/{pet_id}/sensors`\n\n    Returns all the sensors of a pet.\n\nTo test these endpoints, you need to provide either an `owner_id` or a `pet_id`\nas URL path parameter. You can get these values by copying them from the\nbeginning of output of the ingestion script:\n```bash\nsource env/bin/activate\nNODE1=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' carepet-scylla1)\npython src/sensor.py -h $NODE1 --measure 1 --buffer-interval 6\n\nWelcome to the Pet collar simulator\nNew owner # 1cfbc0e5-6b05-476d-b170-2660cf40c02a \u003c-- This is what you need!\nNew pet # 1a0800ee-7643-4794-af7b-2ecaaf7078fc \u003c-- This is what you need!\nNew sensor(0) # b6155934-bd4e-47de-8649-1fad447aa036\nNew sensor(1) # d2c62c4d-9621-469d-b62c-41ef2271fca7\n```\n\nCopy the UUID values right after \"New owner #\" and \"New pet #\". A UUID value\nlooks like this:\n```\n1cfbc0e5-6b05-476d-b170-2660cf40c02a\n```\n\n**Test `/api/owner/{owner_id}`**\n\nPaste the owner id from the terminal into the endpoint URL and open it with\nyour browser or use `curl`, for example:\n```bash\ncurl http://127.0.0.1:8000/api/owner/4f42fb80-c209-4d19-8c43-daf554f1be23\n\n{\"owner_id\":\"4f42fb80-c209-4d19-8c43-daf554f1be23\",\"address\":\"home\",\"name\":\"Vito Russell\"}\n```\n\n**Test `/api/owner/{owner_id}/pets`**\n\nUse the same owner id value to test this endpoint, for example:\n```bash\ncurl http://127.0.0.1:8000/api/owner/4f42fb80-c209-4d19-8c43-daf554f1be23/pets\n\n[{\"owner_id\":\"4f42fb80-c209-4d19-8c43-daf554f1be23\",\"pet_id\":\"44f1624e-07c2-4971-85a5-85b9ad1ff142\",\"address\":\"home\",\"age\":20,\"name\":\"Duke\",\"weight\":14.41481876373291}]\n```\n\n**Test `/api/pet/{pet_id}/sensors`**\n\nFinally, use a pet id to test this endpoint, for example:\n```bash\ncurl http://127.0.0.1:8000/api/pet/44f1624e-07c2-4971-85a5-85b9ad1ff142/sensors\n\n[{\"pet_id\":\"44f1624e-07c2-4971-85a5-85b9ad1ff142\",\"sensor_id\":\"4bb1d214-712b-453b-b53a-ac5d4df4a1f8\",\"type\":\"T\"},{\"pet_id\":\"44f1624e-07c2-4971-85a5-85b9ad1ff142\",\"sensor_id\":\"e81915d6-1155-45e4-9174-c58e4cb8cecf\",\"type\":\"L\"}]\n```\n\n## Structure\nPackage structure:\n\n| Name                                    | Purpose                              |\n| ----------------------------------------| -------------------------------------|\n| [/src/db](/src/db)                      | Database config and client folder    |\n| [/src/db/cql](/src/db/cql)              | CQL scripts                          |\n| [/src/db/client](/src/db/client.py)      | ScyllaDB client library              |\n| [/src/server](/src/server)              | FastAPI application folder           |\n| [/src/server/app.py](/src/server/app.py)| FastAPI application                  |\n| [/src/api.py](/src/api.py )             | Script to start the API server       |\n| [/src/migrate.py](/src/migrate.py)      | Schema creation                      |\n| [/src/sensor.py](/src/sensor.py)        | IoT data ingestion                   |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzseta%2Fcarepet-python","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzseta%2Fcarepet-python","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzseta%2Fcarepet-python/lists"}