{"id":22191236,"url":"https://github.com/simonskodt/covid-columnar-database","last_synced_at":"2025-03-24T20:43:01.711Z","repository":{"id":265221653,"uuid":"895494287","full_name":"simonskodt/covid-columnar-database","owner":"simonskodt","description":"Part of the Data Design and Modeling Course","archived":false,"fork":false,"pushed_at":"2024-12-08T19:29:29.000Z","size":71052,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-30T01:15:33.467Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/simonskodt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-28T10:09:34.000Z","updated_at":"2024-12-08T19:29:33.000Z","dependencies_parsed_at":"2024-11-28T11:41:17.942Z","dependency_job_id":null,"html_url":"https://github.com/simonskodt/covid-columnar-database","commit_stats":null,"previous_names":["simonskodt/covid-columnar-database"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonskodt%2Fcovid-columnar-database","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonskodt%2Fcovid-columnar-database/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonskodt%2Fcovid-columnar-database/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonskodt%2Fcovid-columnar-database/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/simonskodt","download_url":"https://codeload.github.com/simonskodt/covid-columnar-database/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245351760,"owners_count":20601088,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-02T12:14:59.131Z","updated_at":"2025-03-24T20:43:01.673Z","avatar_url":"https://github.com/simonskodt.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Columnar Database: Cassandra\n\nIn this repository, we use the columnar database Apache Cassandra on a COVID dataset to extract and compare the performance of different queries.\n\nCassandra is a NoSQL column-oriented database, ideal for horizontal scalability. With today's amount of big data, it is important being able to scale out in a distributed fashion.\n\n## Covid-19 Dataset\n\nThe dataset we use is available on [Kaggle](https://www.kaggle.com/datasets/gauravduttakiit/covid-19).\n\nThe dataset contains 8 CSV files, of which we will primarily work with the `countries-aggregated.csv` file. This file contains time series data tracking COVID-19 cases, including confirmed, recovered, and death counts.\n\n## Setup Test Environment\n\nThe [test-setup](./test-setup) folder allows you to set up a small `.cql` database and inspect it.\n\nCassandra can either be set up with Docker or using the tarball from the official Cassandra website. The Docker setup is explained in the above directory, but for the tarball setup, you need to export the following in the `.zshrc` or `bash_profile` on a Mac:\n\n```sh\nexport=CASSANDRA_HOME=\u003cpath_to_cassandra_root_folder\u003e\nexport PATH=$PATH:CASSANDRA_HOME/bin\nexport JAVA_HOME=\u003cpath_to_jdk_root_folder\u003e # most likely already there\n```\n\nWe found the Docker setup is the easiest to configure for M1-M4 Macs. After executing `docker pull cassandra:latest`, three shell scripts: `run.sh`, `run-db-in-shell.sh`, and `stop.sh` orchestrate the Docker commands to run the Cassandra container, start an interactive CQL session in the terminal, and when finished, stop and remove the Docker container and network.\n\nThe `run.sh` and `run-db-in-shell.sh` script run the covid `.cql` file by default. To run the test case instead, add the `-t` flag to the command.\n\n```sh\nsh run.sh              # default covid case\nsh run.sh -t           # run the test setup mode\n```\n\n## Environments\n\nThe Cassandra Query Language (CQL) can be run from either the terminal, VSCode, or other external DB software providers. We use a mix of the two mentioned, but also RazorSQL, which has native support for Cassandra.\n\nThe connection string for these services uses IP 127.0.0.1 and port 9042. When running `run-db-in-shell.sh`, you might encounter this error message:\n\n```\nConnection error: ('Unable to connect to any servers', {'127.0.0.1:9042': ConnectionRefusedError(111, \"Tried connecting to [('127.0.0.1', 9042)]. Last error: Connection refused\")})\n```\n\nThis occurs because the Cassandra container needs time to initialize. You will typically need to wait 10-30 seconds, and you can check if it is running either through the Docker Desktop app or by using this terminal command:\n\n```sh\ndocker ps -f name=cassandra\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonskodt%2Fcovid-columnar-database","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimonskodt%2Fcovid-columnar-database","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonskodt%2Fcovid-columnar-database/lists"}