Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/neptune-ai/neptune-r

📒 The MLOps stack component for experiment tracking (R interface)
https://github.com/neptune-ai/neptune-r

compare language log management metadata metrics mlops models monitoring organize parameters r store tracker visualization

Last synced: 3 days ago
JSON representation

📒 The MLOps stack component for experiment tracking (R interface)

Awesome Lists containing this project

README 

        


    

 



## What is neptune.ai?



Neptune is a lightweight experiment tracker for ML teams that struggle with debugging and reproducing experiments, sharing results, and messy model handover. **It offers a single place to track, compare, store, and collaborate on experiments and models.**



With Neptune, Data Scientists can develop production-ready models faster, and ML Engineers can access model artifacts instantly in order to deploy them to production.

 



Watch a 3min explainer video →

 



Watch a 20min product demo →

 



# Getting started



## Register

[Sign up for Neptune](https://neptune.ai).



You can use Neptune for free for work, research, and personal projects. Individual accounts are free, and if you're interested in using Neptune as a team, [check available plans](https://neptune.ai/pricing).



## Install Neptune R package



If you don't have Python installed and just want to use Neptune, simply run the following code:



```R

install.packages("reticulate")

library(reticulate)

install_miniconda()



install.packages("neptune")

library(neptune)

neptune_install()

```



This code will install [miniconda](https://docs.conda.io/en/latest/miniconda.html) (a minimalistic Python environment) and set it up for you. This is a one-time thing and after that, you only need to run the last two lines when you want to work with Neptune:



```R

library(neptune)

neptune_install()

```



If you have a Python virtual environment already set up (conda, miniconda, or virtualenv), you can point to it instead of creating a new one:



```R

# If you are using virtualenv

install.packages("neptune")

library(neptune)

neptune_install(method="virtualenv", envname = "PATH/TO/YOUR/VIRTUALENV")



# If you are using conda or miniconda

install.packages("neptune")

library(neptune)

neptune_install(method="conda", envname = "PATH/TO/YOUR/CONDA/ENVIRONMENT")

```



## Create a tracked run



```R

run <- neptune_init(project="MY_WORKSPACE/MY_PROJECT",

                    api_token="NEPTUNE_API_TOKEN")

```



This code creates a run in the specified project. This will be your gateway to logging metadata to Neptune.



You need to pass your credentials (project and API token) to the `neptune_init()` method. You can also set the API token globally:



```R

neptune_set_api_token(token = "NEPTUNE_API_TOKEN")

```



### API token



To find your API token:

* Go to the Neptune UI

* Open the **User menu** toggle in the upper right

* Click **Get your API token**

* Copy your API token 



or [get your API token directly from here](https://app.neptune.ai/get_my_api_token).



### Project



The project argument has the format `WORKSPACE_NAME/PROJECT_NAME`.



To find it:

* Go to the Neptune UI 

* Go to your project 

* Open **Settings** > **Properties**

* Copy the project name



### Stop tracking



Once you are finished with tracking metadata you need to stop the tracking for that particular run:



```R

neptune_stop(run)



# Note that you can also use reticulate based syntax:

run$stop()

```



If you are running a script it will stop tracking automatically at the end. However, in an interactive environment such as RStudio, you need to stop it explicitly.



## Track metadata



### Log hyperparameters



```R

params <- list(

  "dense_units"= 128,

  "activation"= "relu",

  "dropout"= 0.23,

  "learning_rate"= 0.15,

  "batch_size"= 64,

  "n_epochs"= 30

)

run["parameters"] <- params

```



If you have parameters in form of a dictionary you can log them to Neptune in batch. It will create a field with the appropriate type for each dictionary entry. 

You can update the hyperparameters or add new ones later in the code:



```R

# Add additional parameters 

run["model/parameters/seed"] <- .Random.seed



# Update parameters e.g. after triggering an early stopping

run["model/parameters/n_epochs"] <- epoch

```



### Log training metrics



```R

for (i in 1:epochs) {

  [...] # My training loop

  neptune_log(run["train/epoch/loss"], loss)

  neptune_log(run["train/epoch/accuracy"], acc)

}



# Note that you can also use reticulate based syntax:

run["train/epoch/loss"]$log(loss)

```



You can log training metrics to Neptune using series fields. In Neptune, there are three types of series - float series, string series, and file series. Each `neptune_append()` will add a new value at the end of the series. 



### Log evaluation results



```R

run["evaluation/accuracy"] <- eval_acc

run["evaluation/loss"] <- eval_loss

```



To log evaluation metrics simply assign them to a field of your choice. Using the snippet above, both evaluation metrics will be stored in the same evaluation namespace.



```R

neptune_upload(run["evaluation/ROC"], "roc.png")



# You can upload ggplot plots directly without saving them to a file

neptune_upload(run["evaluation/ROC"], ggplot_roc)



# If you want to control additional parameters like size of the plot you can pass the same arguments as to ggsave

neptune_upload(run["evaluation/ROC"], ggplot_roc, width=20, height=20, units="cm")



# Note that you can also use reticulate based syntax:

run["evaluation/ROC"]$upload("roc.png")

run["evaluation/ROC"]$upload(ggplot_roc)

```



You can log plots and charts easily using the `neptune_upload()` function. In the case of a ggplot object, it gets converted to an image file and uploaded, but you can also upload images from the local disc.



### Upload model file



You can upload any binary file (e.g. model weights) from disk using the `neptune_upload()` method. If your model is saved as multiple files you can upload a whole folder as a `FileSet` using `neptune_upload_files()`.



```R

# Upload a single fingle sile

neptune_upload(run["model"], "model.Rdata")



# You can also upload folders in batch if you don't need access to the separate files

neptune_upload_files(run["model"], "models")



# Note that you can also use reticulate based syntax:

run["model"]$upload("model.Rdata")

run["model"]$upload_files("models")

```



# Getting help



If you got stuck or simply want to talk to us, here are your options:



* Check the [Neptune R client package documentation](https://docs.neptune.ai/integrations/r/)

* Check our [FAQ page](https://docs.neptune.ai/getting_help)

* You can submit bug reports, feature requests, or contributions directly to the repository.

* Chat! When in the Neptune application, click on the blue message icon in the bottom-right corner and send a message. A real person will talk to you ASAP (typically very ASAP),

* You can just shoot us an email at [email protected]