{"id":25397428,"url":"https://github.com/overhash/supermarket-tracker","last_synced_at":"2025-04-11T02:45:16.908Z","repository":{"id":101033425,"uuid":"410258588","full_name":"OverHash/supermarket-tracker","owner":"OverHash","description":"A supermarket aggregator for price information at New Zealand supermarkets","archived":false,"fork":false,"pushed_at":"2025-03-05T05:02:05.000Z","size":220,"stargazers_count":6,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-05T06:18:50.490Z","etag":null,"topics":["data-science","new-zealand","nz","prices","rust-lang","supermarket"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OverHash.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-09-25T11:49:08.000Z","updated_at":"2025-03-05T05:02:09.000Z","dependencies_parsed_at":null,"dependency_job_id":"fec4ce32-baaf-4447-890d-accc747e377f","html_url":"https://github.com/OverHash/supermarket-tracker","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OverHash%2Fsupermarket-tracker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OverHash%2Fsupermarket-tracker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OverHash%2Fsupermarket-tracker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OverHash%2Fsupermarket-tracker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OverHash","download_url":"https://codeload.github.com/OverHash/supermarket-tracker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248331860,"owners_count":21085992,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-science","new-zealand","nz","prices","rust-lang","supermarket"],"created_at":"2025-02-15T21:47:56.288Z","updated_at":"2025-04-11T02:45:16.884Z","avatar_url":"https://github.com/OverHash.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Supermarket Tracker\n\nTracks supermarket prices across New Zealand online supermarkets.\n\nCurrently Countdown prices can be fetched, with the application fetching both\nCountdown Mt Eden and Woolsworth Hornby store prices.\n\n## Starting the application\n\n- Setup relevant environment variables (a `.env` file can be used for convenience, see [`.env.example`](env.example))\n- Run a Postgres instance with a blank `supermarket_tracker` database created (e.g., with `docker compose up -d`)\n  - The app will initialize all tables for you when first run\n\n## CLI Usage\n\n```\nsupermarket-tracker\n\nUsage:\n    supermarket-tracker [OPTIONS] [SUBCOMMAND]\n\nOptions:\n    --supermarket \u003cSUPERMARKET\u003e     The supermarket to run price tracking on [countdown]\n    --no-insert                     Optionally skips insertion of new products/prices to database\n```\n\n### Architecture\n\nCore application is written in Rust. Read more in the [ARCHITECTURE.md](./ARCHITECTURE.md) document.\n\n### Using with Docker\n\n[Docker](https://www.docker.com/) can be used to host the Postgres database, and perform all the initial work of setup.\n\nTo use, ensure you have a `.env` file (if you don't, simply `cp .env.example .env`) and run `docker compose up -d` to start the services. This will expose a Postgres database on port 5432 of the host machine.\n\nTo stop, use `docker compose down` to stop all the containers, and `docker compose down --volumes` to delete the volumes as well.\n\n### How much data is tracked?\n\nI currently have around ~800,000 price points from ~23,000 products tracked since October 2022.\n\nIf this data would be of use, please [contact me](https://x.com/OverHashDev). I have mostly used it for my own fun statistical analysis,\nand comparing data against what Stats NZ produces.\n\n### Migrating Docker Containers Between Hosts\n\nI've had some trouble finding resources for this online, so I thought posting these instructions would be helpful.\n\nReplace `$CONTAINER_NAME`, `$DATABASE_NAME` and `$DOCKER_FILENAME` accordingly for your system.\n\n1. On machine A, run `docker exec -it $CONTAINER_NAME bash`\n2. Once in a bash terminal in the docker container, run `pg_dumpall -c -U postgres | gzip \u003e ./tmp/dump_$(date +\"%Y-%m-%d_%H_%M_%S\").gz`\n3. Verify file looks good in the docker container, and note the location of it (noted as `$DOCKER_FILENAME`).\n4. Transfer from docker container to host with `docker cp $CONTAINER_NAME:/tmp/$DOCKER_FILENAME /tmp/$DOCKER_FILENAME`\n5. Now transfer the file over to the host, using whatever means you prefer.\n6. We now use `$CONTAINER_NAME` to refer to the fresh postgres instance created. This instance must have an empty `$DATABASE_NAME` created (e.g., with `docker compose up -d`).\n7. Use `docker cp ./$DOCKER_FILENAME$ $CONTAINER_NAME$:/tmp/$DOCKER_FILENAME$`\n8. `docker exec -it $CONTAINER_NAME bash`\n9. `gunzip /tmp/$DOCKER_FILENAME | psql -U postgres -d $DATABASE_NAME`\n10. All done!\n\nCheers to https://stackoverflow.com/questions/24718706/backup-restore-a-dockerized-postgresql-database for the suggestions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foverhash%2Fsupermarket-tracker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foverhash%2Fsupermarket-tracker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foverhash%2Fsupermarket-tracker/lists"}