{"id":19140823,"url":"https://github.com/codait/covid-notebooks","last_synced_at":"2025-04-16T01:19:57.019Z","repository":{"id":43674229,"uuid":"257959838","full_name":"CODAIT/covid-notebooks","owner":"CODAIT","description":"Jupyter notebooks that analyze COVID-19 time series data","archived":false,"fork":false,"pushed_at":"2022-02-24T19:03:18.000Z","size":47367,"stargazers_count":106,"open_issues_count":4,"forks_count":38,"subscribers_count":25,"default_branch":"master","last_synced_at":"2025-03-29T03:23:05.498Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CODAIT.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-04-22T16:36:35.000Z","updated_at":"2025-03-11T15:48:15.000Z","dependencies_parsed_at":"2022-09-21T09:44:14.040Z","dependency_job_id":null,"html_url":"https://github.com/CODAIT/covid-notebooks","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CODAIT%2Fcovid-notebooks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CODAIT%2Fcovid-notebooks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CODAIT%2Fcovid-notebooks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CODAIT%2Fcovid-notebooks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CODAIT","download_url":"https://codeload.github.com/CODAIT/covid-notebooks/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249178980,"owners_count":21225468,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T07:18:56.279Z","updated_at":"2025-04-16T01:19:57.001Z","avatar_url":"https://github.com/CODAIT.png","language":"Jupyter Notebook","readme":"# Analyzing COVID-19 time series data\n\nThis repository provides a set of [Jupyter Notebooks](https://jupyter.org) that augment and \nanalyze COVID-19 time series data.\n\nWhile working on this scenario, we identified that building a pipeline would help organize the\nnotebooks and simplify running the full workflow to process and analyze new data. For this, we leveraged\n[Elyra's](https://github.com/elyra-ai/elyra) ability to build \n[notebook pipelines](https://elyra.readthedocs.io/en/latest/getting_started/overview.html#notebook-pipelines-visual-editor)\nto orchestrate the running of the full scenario on a [Kubeflow Pipeline](https://www.kubeflow.org/docs/pipelines/overview/pipelines-overview/)\nruntime.\n\n![COVID-19 Analytics Pipeline](docs/source/images/covid-19-pipeline.png)\n\n### Configuring the local development environment\n\n**WARNING: Do not run these notebooks from your system Python environment.**\n\nUse the following steps to create a consistent Python environment for running the\nnotebooks in this repository:\n\n1. Install [Anaconda](https://docs.anaconda.com/anaconda/install/)\n   or [Miniconda](https://docs.conda.io/en/latest/miniconda.html)\n1. Navigate to your local copy of this repository.\n1. Run the script `env.sh` to create an Anaconda environment in the directory `./env`:\n   ```console\n   $ bash env.sh\n   ```\n   Note: This script takes a while to run.\n1. Activate the new environment and start JupyterLab:\n   ```console\n   $ conda activate ./env\n   $ jupyter lab --debug\n   ```\n\n#### Configuring a local Kubeflow Pipeline runtime\n\n[Elyra's Notebook pipeline visual editor](https://elyra.readthedocs.io/en/latest/getting_started/overview.html#notebook-pipelines-visual-editor)\ncurrently supports running these pipelines in a Kubeflow Pipeline runtime.  If required, these are\n[the steps to install a local deployment of KFP](https://elyra.readthedocs.io/en/latest/recipes/deploying-kubeflow-locally-for-dev.html).\n\nAfter installing your Kubeflow Pipeline runtime, use the command below (with proper updates) to configure the new\nKFP runtime with Elyra.\n\n```bash\nelyra-metadata install runtimes --replace=true \\\n       --schema_name=kfp \\\n       --name=kfp-local \\\n       --display_name=\"Kubeflow Pipeline (local)\" \\\n       --api_endpoint=http://[host]:[api port]/pipeline \\\n       --cos_endpoint=http://[host]:[cos port] \\\n       --cos_username=[cos username] \\\n       --cos_password=[cos password] \\\n       --cos_bucket=covid\n``` \n\n**Note:** The cloud object storage above is a local minio object storage but other cloud-based object storage \nservices could be configured and used in this scenario.\n\n## Elyra Notebook pipelines\n\nElyra provides a visual editor for building Notebook-based AI pipelines, simplifying the conversion of \nmultiple notebooks into batch jobs or workflows. By leveraging cloud-based resources to run their \nexperiments faster, the data scientists, machine learning engineers, and AI developers are then more productive,\nallowing them to spend their time using their technical skills.\n\n![Notebook pipeline](https://raw.githubusercontent.com/elyra-ai/community/master/resources/blog-announcement/elyra-pipelines.gif)\n\n### Running the Elyra pipeline\n\nThe Elyra pipeline `us_data.pipeline`, which is located in the `pipeline` directory, can be run by clicking\non the `play` button as seen on the image above. The `submit` dialog will request two inputs from the user: a name \nfor the pipeline and a runtime to use while executing the pipeline. The list of available runtimes comes from\nthe registered Kubeflow Pipelines runtimes documented above. After submission, Elyra will show a dialog with a direct\nlink to where the experiment is being executed on Kubeflow Piplines. \n\nThe user can access the pipelines, and respective experiment runs, via the `api_endpoint` of the Kubeflow Pipelines\nruntime (e.g. `http://[host]:[port]/pipeline`)\n\n![Pipeline experiment run](docs/source/images/kfp-experiment.png)\n\nThe output from the executed experiments are then available in the associated `object storage`\nand the executed notebooks are available as native ipynb notebooks and also in html format\nto facilitate the visualization and sharing of the results.\n\n![Pipeline experiment results in object storage](docs/source/images/object-storage-results.png)\n\n### References\n\nFind more project details on [Elyra's GitHub](https://github.com/elyra-ai/elyra) or watching the\n[Elyra's demo](https://www.youtube.com/watch?v=Nj0yga6T4U8).\n    \n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodait%2Fcovid-notebooks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcodait%2Fcovid-notebooks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodait%2Fcovid-notebooks/lists"}