{"id":15464315,"url":"https://github.com/nitya/pydata-analysis-workshop","last_synced_at":"2025-09-05T13:31:47.834Z","repository":{"id":205016582,"uuid":"713185488","full_name":"nitya/pydata-analysis-workshop","owner":"nitya","description":"Step-by-step workshop for the \"Simplifying Data Analysis\" talk","archived":false,"fork":false,"pushed_at":"2024-03-20T04:44:00.000Z","size":17127,"stargazers_count":28,"open_issues_count":0,"forks_count":32,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-05T01:11:17.522Z","etag":null,"topics":["data-science","data-visualization","python","workshop"],"latest_commit_sha":null,"homepage":"https://nitya.github.io/pydata-analysis-workshop/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nitya.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-02T02:17:58.000Z","updated_at":"2024-11-27T11:25:25.000Z","dependencies_parsed_at":null,"dependency_job_id":"13352bbc-4725-471b-af9f-3e876940a439","html_url":"https://github.com/nitya/pydata-analysis-workshop","commit_stats":{"total_commits":18,"total_committers":2,"mean_commits":9.0,"dds":0.2222222222222222,"last_synced_commit":"12ede45d461fe8272aad9eb1a0fe605bc460532c"},"previous_names":["nitya/pydata-workshop-2023","nitya/pydata-analysis-workshop"],"tags_count":0,"template":false,"template_full_name":"github/codespaces-jupyter","purl":"pkg:github/nitya/pydata-analysis-workshop","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nitya%2Fpydata-analysis-workshop","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nitya%2Fpydata-analysis-workshop/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nitya%2Fpydata-analysis-workshop/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nitya%2Fpydata-analysis-workshop/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nitya","download_url":"https://codeload.github.com/nitya/pydata-analysis-workshop/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nitya%2Fpydata-analysis-workshop/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273765686,"owners_count":25164182,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-05T02:00:09.113Z","response_time":402,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-science","data-visualization","python","workshop"],"created_at":"2024-10-02T00:33:05.944Z","updated_at":"2025-09-05T13:31:42.815Z","avatar_url":"https://github.com/nitya.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Python Data Analysis | Workshop\n\n## 1 | Introduction\n\nThis repository provides the source code and documentation for a **Python Data Analysis** workshop based on some version of these talks:\n\n| Talk | Description | Slides |\n| ---- | ----------- | ------ |\n| [Simplifying Data Analysis with GitHub Codespaces, Jupyter Notebooks \u0026 Open AI](https://www.youtube.com/watch?v=wVMwKy1QmTM) | PyData NYC (Nov 2023) | [Slides](https://speakerdeck.com/nitya/simplifying-data-analysis-with-github-codespaces-jupyter-notebooks-and-open-ai) |\n| [Simplifying Data Analysis  (with Developer Tools \u0026 AI (for non-Python devs)](https://www.youtube.com/watch?v=scvDXfCMHYU) | Data Science Day (Mar 2024) | [Slides](https://speakerdeck.com/nitya/simplifying-data-analysis-and-visualization-with-developer-tools-and-ai) |\n| ||\n\n## 2 | Workshop Roadmap\n\nThe slide shows a high-level roadmap for what we want to cover in this workshop. The goal is to help someone who is familiar with development - but not with data science or Python - skill up quickly on topics related to data science, analysis and visualization using developer tools and AI assistance.\n\n![2024 Roadmap](https://30daysof.github.io/data-science-day/_astro/DataScienceDay-Roadmap.ixBAXxDK_1SuePL.webp)\n\nThe actual roadmap for the workshop may switch steps up in sections, or combine exercises for efficiency to align with the Week 2 schedule on [#14DaysOfDataScience](https://30daysof.github.io/data-science-day/week-2/). Click the \"Day\" entry to read the post, and the \"Details\" entry to visit the exercise, once series is complete.\n\n| Day | Description | Details |\n| ---- | ----------- | ------- |\n| [1️⃣](https://30daysof.github.io/data-science-day/week-2/1-codespaces/) | Setup a Consistent Development Environment with Dev Containers (GitHub Codespaces) + Jupyter Notebooks. Run sample notebooks. | [Setup Dev Environment](./2-jupyter/README.md) |\n| [2️⃣](https://30daysof.github.io/data-science-day/week-2/23️⃣-vscode/) | Create a Data Science driven Visual Studio Code Profile for productivity. Do the VS Code Data Science Tutorial, use Data Wrangler. | [Setup Custom VS Code Profile](./4-vscode/README.md)|\n| [3️⃣](https://30daysof.github.io/data-science-day/week-2/3-github-copilot/) | Install the Copilot extension and use it to create Notebooks, explain code, or debug issues inline (or in chat mode). | [Install \u0026 explore GitHub Copilot](./4-vscode/README.md) |\n| [4️⃣](https://30daysof.github.io/data-science-day/week-2/4-open-datasets/) | Learn about open datasets on Kaggle, Hugging Face, Azure, Open ML etc. Explore sites to learn from community \u0026 experiment.| Pick a dataset, do EDA, build Model|\n| [5️⃣](https://30daysof.github.io/data-science-day/week-2/5-responsible-ai/) | Learn about Responsible AI principles \u0026 explore the components of the Responsible AI Toolbox for assessing these in practice. | Debug Models \u0026 Decision-Making |\n| [6️⃣](https://30daysof.github.io/data-science-day/week-2/6-project-lida/) | Learn about Project LIDA and how you can use Large Language Models to summarize and visualize data, using natural language prompts | Visualize Data using LLM \u0026 NLP|\n| [7️⃣](https://30daysof.github.io/data-science-day/week-2/7-azure-ai/) | Learn about the paradigm shift from MLOps to LLM Ops and how to use the Azure AI SDK (Python) to build, evaluate, \u0026 deploy, AI apps. | Explore LLM Ops with Azure AI|\n| | |\n\n## 3 | Pre-Requisites\n  - Familiarity with GitHub and Visual Studio Code\n  - Familiarity with at least one high-level programming language\n  - Access to an OpenAI or Azure OpenAI key - optional, for Project LIDA\n  - Access to a GitHub Copilot enabled account - optional, for AI-assisted learning\n  - Knowledge of Python and Jupyter Notebooks is a plus\n  - Having a specific data set or data analysis problem to work on is a plus\n\n## 4 | Learning Objectives\n\nBy the end of this workshop, you will be able to:\n\n - **Describe** what the various tools do, and why they matter\n - **Setup** a reusable development environment - with GitHub Codespaces\n - **Document** your learning with code - with Jupyter Notebooks\n - **Generate** notebooks and explore unfamiliar code - with GitHub Copilot\n - **Discover**, analyze and visualize open datasets - from Kaggle, Hugging Face or Azure\n - **Train \u0026 debug** models for responsible AI practices - with Responsible AI Toolbox\n - **Build intuition** and explore data visualizations - with Project LIDA\n - **Explore paradigm shift** to LLM Ops - with Azure AI Studio\n\n## 5 | Developer Tools\n\nHere are the main tools we will cover, with links to relevant documentation and a short description on why they are useful.\n\n| Tool | Description |\n| ---- | ----------- |\n| [Jupyer Notebooks](https://jupyter.org/) | Interactive development environment for sharing code with documentation, for collaborative learning \u0026 debugging |\n| [GitHub Codespaces](https://docs.github.com/codespaces) | Reusable and reproducible development environments based on development containers for configuration-as-code. |\n| [Visual Studio Code](https://code.visualstudio.com/docs/datascience/overview) |  Visual Studio Code extensions for Data Science, with [Data Science Profile Template](https://code.visualstudio.com/docs/editor/profiles#_data-science-profile-template)|\n| [GitHub Copilot](https://github.com/features/copilot) | AI-based assistance for developers - from code explainers to code creation and debugging|\n| Open Datasets | Jumpstart exploration with curated public datasets for machine learning - from [Kaggle](https://www.kaggle.com/datasets), [Hugging Face](https://huggingface.co/datasets), and [Azure](https://learn.microsoft.com/azure/open-datasets/dataset-catalog)|\n| [Responsible AI](https://responsibleaitoolbox.ai/) | Responsible AI Toolbox includes tools for identifying, diagnosing, and mitigating, issues in data. |\n| [Project LIDA](https://microsoft.github.io/lida/) | Automated generation of visualizations with LLM, with user customization and suggestion options |\n| [Azure AI Studio](https://learn.microsoft.com/azure/ai-studio/) | Unified platform for building generative AI applications from model exploration to API deployment |\n| | |\n\n## 6 | Python Libraries\n\nDeveloper tools will streamline your learning journey, but you will need to skill up on a few core Python libraries, to be productive. Start with these libraries, in the recommended order. For each, start by creating a new notebook in your fork of this repo - then try to walk through the quickstart tutorial step-by-step, to get a sense of what the library does and how it works.\n\n\u003e **Important** | To start using these libraries, you will first need to install them in your development environment. Look for installation instructions that use `pip` - and add those libraries to the [requirements.txt](requirements.txt) file if absent. Rebuild container - and you should be ready to go.\n\n| Library | Description | Quickstart |\n| ------- | ----------- | ---------- |\n| [NumPy](https://numpy.org/) | Numerical computing with Python | [Quickstart Tutorial](https://numpy.org/doc/stable/user/quickstart.html) |\n| [Pandas](https://pandas.pydata.org/) | Data manipulation and analysis | [Getting Started](https://pandas.pydata.org/pandas-docs/stable/getting_started/index.html#getting-started) |\n| [Matplotlib](https://matplotlib.org/) | Data visualization (static + interactive) | [Quickstart](https://matplotlib.org/stable/users/explain/quick_start.html) - [Depth](https://matplotlib.org/stable/tutorials/index.html) |\n| [Seaborn](https://seaborn.pydata.org/) | Data visualization (matplotlib-based) | [Quickstart](https://seaborn.pydata.org/installing.html#quickstart) - [Depth](https://seaborn.pydata.org/tutorial.html) |\n| [Scikit-learn](https://scikit-learn.org/stable/) | Machine learning in Python (Core) | [Quickstart](https://scikit-learn.org/stable/getting_started.html) - [Depth](https://scikit-learn.org/stable/user_guide.html) |\n| [TensorFlow](https://www.tensorflow.org/) | Machine learning in Python (E2E) | [Quickstart](https://www.tensorflow.org/tutorials/quickstart/beginner)  - [Depth](https://www.tensorflow.org/tutorials)|\n| [PyTorch](https://pytorch.org/) | Machine learning in Python (CV, NLP) | [Quickstart](https://pytorch.org/tutorials/beginner/basics/quickstart_tutorial.html) - [Msft](https://learn.microsoft.com/training/paths/pytorch-fundamentals/) |\n| [Bokeh](https://bokeh.org/)  |  Interactive visualizations in Python | [Quickstart](https://docs.bokeh.org/en/latest/docs/first_steps.html) - [Depth](https://docs.bokeh.org/en/latest/docs/user_guide.html)|\n| | |\n\nThis is not a comprehensive list - feel free to add more to the list as you skill up, based on the goals you are driving towards in usage.\n\n\n## 7 | Learning Resources\n\nThese are resources I recommend for skilling up on these topics. Note that _open access_ editions of published books are often made available by the authors, on their sites, for supporting learners - and not for commercial use. Please read and honor the author's usage requirements when exploring content or code.\n\n| Resource | Description |\n| -------- | ----------- |\n| [2024: Data Science Day Collection](https://bit.ly/2024-datasci-collection) | My curated collection of Microsoft Resources for Data Science development |\n| [2024: Responsible AI for Developers](https://aka.ms/rai-hub/collection) |My curated collection of Microsoft Resources for Responsible AI usage|\n| [2024: Generative AI Code-First on Azure](https://aka.ms/ai-studio/collection) |My curated collection of Microsoft Resources for Generative AI with the Azure AI Platform |\n| [Python Data Science Handbook - Jake VanderPlas](https://jakevdp.github.io/PythonDataScienceHandbook/) | O'Reilly book made available online with MIT license and [tutorial notebooks](https://github.com/jakevdp/PythonDataScienceHandbook) on GitHub|\n| [Python for Data Analysis 3E - Wes Mckinney](https://wesmckinney.com/book/) | O'Reilly book made available online with MIT-licensed [code examples](https://github.com/wesm/pydata-book/tree/3rd-edition) on GitHub |\n| [Introduction to Machine Learning with Python -  Andreas C. Müller \u0026 Sarah Guido](https://www.oreilly.com/library/view/introduction-to-machine/9781449369880/) | O'Reilly book covering `scikit-learn` usage with notebooks on [GitHub](https://github.com/amueller/introduction_to_ml_with_python). Book content is not online.|\n| | |\n\n## 8 | Feedback Welcome\n\nHave questions or comments, or found areas for improvement in the coede or documentation? [File an issue](https://github.com/nitya/pydata-analysis-workshop/issues/new) to let me know.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnitya%2Fpydata-analysis-workshop","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnitya%2Fpydata-analysis-workshop","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnitya%2Fpydata-analysis-workshop/lists"}