https://github.com/eugeneyan/papermill-mlflow

🧪 Simple data science experimentation & tracking with jupyter, papermill, and mlflow.
https://github.com/eugeneyan/papermill-mlflow

experimentation jupyter mlflow papermill python

Last synced: 8 months ago
JSON representation

🧪 Simple data science experimentation & tracking with jupyter, papermill, and mlflow.

• Host: GitHub
• URL: https://github.com/eugeneyan/papermill-mlflow
• Owner: eugeneyan
• Created: 2020-03-14T03:24:38.000Z (about 6 years ago)
• Default Branch: master
• Last Pushed: 2024-07-09T17:43:09.000Z (almost 2 years ago)
• Last Synced: 2025-04-03T01:12:27.006Z (about 1 year ago)
• Topics: experimentation, jupyter, mlflow, papermill, python
• Language: Jupyter Notebook
• Homepage: https://eugeneyan.com/writing/experimentation-workflow-with-jupyter-papermill-mlflow/
• Size: 1.13 MB
• Stars: 180
• Watchers: 3
• Forks: 36
• Open Issues: 4
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# papermill-mlflow

Simple data science experimentation with `jupyter`, `papermill`, and `mlflow`

#### Associated blog post: [A simpler experimentation workflow with Jupyter, Papermill, and MLflow](https://eugeneyan.com/writing/experimentation-workflow-with-jupyter-papermill-mlflow/)

---

# Quick-start

- Clone this repo

```

git clone git@github.com:eugeneyan/papermill-mlflow.git

```

- Set up virtualenv

```

cd papermill-mlflow

# Create virtualenv based on requirements.txt

python3 -m venv venv

source venv/bin/activate

pip install -r requirements.txt

# Install kernelspec for Jupyter notebooks (the name argument must be identical)

python -m ipykernel install --user --name=papermill-mlflow

```

- Start Jupyter notebook

```

cd notebooks

jupyter notebook

```

- Run the cells in `runner.ipynb`

![image of runner notebook](https://raw.githubusercontent.com/eugeneyan/papermill-mlflow/master/assets/runner.png)

- Start MLflow (in another terminal)

```

# Open another terminal

# Activate the virtualenv

cd papermill-mlflow

source venv/bin/activate

# Start the mlflow server

cd notebooks

mlflow server

```

- Access the MLflow UI opening this in a browser: [http://127.0.0.1:5000](http://127.0.0.1:5000/#/experiments/1)

	- Navigate to "indices" in the experiment tab if necessary

![image of mlflow](https://raw.githubusercontent.com/eugeneyan/papermill-mlflow/master/assets/mlflow.png)