Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/skorch-dev/skorch

A scikit-learn compatible neural network library that wraps PyTorch
https://github.com/skorch-dev/skorch

hacktoberfest huggingface machine-learning pytorch scikit-learn

Last synced: 3 days ago
JSON representation

A scikit-learn compatible neural network library that wraps PyTorch

Awesome Lists containing this project

README 

        
.. image:: https://github.com/skorch-dev/skorch/blob/master/assets/skorch_bordered.svg

   :width: 30%



------------



|build| |coverage| |docs| |huggingface| |powered|



A scikit-learn compatible neural network library that wraps PyTorch.



.. |build| image:: https://github.com/skorch-dev/skorch/workflows/tests/badge.svg

    :alt: Test Status

    :scale: 100%



.. |coverage| image:: https://github.com/skorch-dev/skorch/blob/master/assets/coverage.svg

    :alt: Test Coverage

    :scale: 100%



.. |docs| image:: https://readthedocs.org/projects/skorch/badge/?version=latest

    :alt: Documentation Status

    :scale: 100%

    :target: https://skorch.readthedocs.io/en/latest/?badge=latest



.. |huggingface| image:: https://github.com/skorch-dev/skorch/actions/workflows/test-hf-integration.yml/badge.svg

    :alt: Hugging Face Integration

    :scale: 100%

    :target: https://github.com/skorch-dev/skorch/actions/workflows/test-hf-integration.yml



.. |powered| image:: https://github.com/skorch-dev/skorch/blob/master/assets/powered.svg

    :alt: Powered by

    :scale: 100%

    :target: https://github.com/ottogroup/



=========

Resources

=========



- `Documentation `_

- `Source Code `_

- `Installation `_



========

Examples

========



To see more elaborate examples, look `here

`__.



.. code:: python



    import numpy as np

    from sklearn.datasets import make_classification

    from torch import nn

    from skorch import NeuralNetClassifier



    X, y = make_classification(1000, 20, n_informative=10, random_state=0)

    X = X.astype(np.float32)

    y = y.astype(np.int64)



    class MyModule(nn.Module):

        def __init__(self, num_units=10, nonlin=nn.ReLU()):

            super().__init__()



            self.dense0 = nn.Linear(20, num_units)

            self.nonlin = nonlin

            self.dropout = nn.Dropout(0.5)

            self.dense1 = nn.Linear(num_units, num_units)

            self.output = nn.Linear(num_units, 2)

            self.softmax = nn.Softmax(dim=-1)



        def forward(self, X, **kwargs):

            X = self.nonlin(self.dense0(X))

            X = self.dropout(X)

            X = self.nonlin(self.dense1(X))

            X = self.softmax(self.output(X))

            return X



    net = NeuralNetClassifier(

        MyModule,

        max_epochs=10,

        lr=0.1,

        # Shuffle training data on each epoch

        iterator_train__shuffle=True,

    )



    net.fit(X, y)

    y_proba = net.predict_proba(X)



In an `sklearn Pipeline `_:



.. code:: python



    from sklearn.pipeline import Pipeline

    from sklearn.preprocessing import StandardScaler



    pipe = Pipeline([

        ('scale', StandardScaler()),

        ('net', net),

    ])



    pipe.fit(X, y)

    y_proba = pipe.predict_proba(X)



With `grid search `_:



.. code:: python



    from sklearn.model_selection import GridSearchCV



    # deactivate skorch-internal train-valid split and verbose logging

    net.set_params(train_split=False, verbose=0)

    params = {

        'lr': [0.01, 0.02],

        'max_epochs': [10, 20],

        'module__num_units': [10, 20],

    }

    gs = GridSearchCV(net, params, refit=False, cv=3, scoring='accuracy', verbose=2)



    gs.fit(X, y)

    print("best score: {:.3f}, best params: {}".format(gs.best_score_, gs.best_params_))



skorch also provides many convenient features, among others:



- `Learning rate schedulers `_ (Warm restarts, cyclic LR and many more)

- `Scoring using sklearn (and custom) scoring functions `_

- `Early stopping `_

- `Checkpointing `_

- `Parameter freezing/unfreezing `_

- `Progress bar `_ (for CLI as well as jupyter)

- `Automatic inference of CLI parameters `_

- `Integration with GPyTorch for Gaussian Processes `_

- `Integration with Hugging Face 🤗 `_



============

Installation

============



skorch requires Python 3.9 or higher.



conda installation

==================



You need a working conda installation. Get the correct miniconda for

your system from `here `__.



To install skorch, you need to use the conda-forge channel:



.. code:: bash



    conda install -c conda-forge skorch



We recommend to use a `conda virtual environment `_.



**Note**: The conda channel is *not* managed by the skorch

maintainers. More information is available `here

`__.



pip installation

================



To install with pip, run:



.. code:: bash



    python -m pip install -U skorch



Again, we recommend to use a `virtual environment

`_ for this.



From source

===========



If you would like to use the most recent additions to skorch or

help development, you should install skorch from source.



Using conda

-----------



To install skorch from source using conda, proceed as follows:



.. code:: bash



    git clone https://github.com/skorch-dev/skorch.git

    cd skorch

    conda create -n skorch-env python=3.10

    conda activate skorch-env

    conda install -c pytorch pytorch

    python -m pip install -r requirements.txt

    python -m pip install .



If you want to help developing, run:



.. code:: bash



    git clone https://github.com/skorch-dev/skorch.git

    cd skorch

    conda create -n skorch-env python=3.10

    conda activate skorch-env

    conda install -c pytorch pytorch

    python -m pip install -r requirements.txt

    python -m pip install -r requirements-dev.txt

    python -m pip install -e .



    py.test  # unit tests

    pylint skorch  # static code checks



You may adjust the Python version to any of the supported Python versions.



Using pip

---------



For pip, follow these instructions instead:



.. code:: bash



    git clone https://github.com/skorch-dev/skorch.git

    cd skorch

    # create and activate a virtual environment

    python -m pip install -r requirements.txt

    # install pytorch version for your system (see below)

    python -m pip install .



If you want to help developing, run:



.. code:: bash



    git clone https://github.com/skorch-dev/skorch.git

    cd skorch

    # create and activate a virtual environment

    python -m pip install -r requirements.txt

    # install pytorch version for your system (see below)

    python -m pip install -r requirements-dev.txt

    python -m pip install -e .



    py.test  # unit tests

    pylint skorch  # static code checks



PyTorch

=======



PyTorch is not covered by the dependencies, since the PyTorch version

you need is dependent on your OS and device. For installation

instructions for PyTorch, visit the `PyTorch website

`__. skorch officially supports the last four

minor PyTorch versions, which currently are:



- 2.2.2

- 2.3.1

- 2.4.1

- 2.5.1



However, that doesn't mean that older versions don't work, just that

they aren't tested. Since skorch mostly relies on the stable part of

the PyTorch API, older PyTorch versions should work fine.



In general, running this to install PyTorch should work:



.. code:: bash



    # using conda:

    conda install pytorch pytorch-cuda -c pytorch

    # using pip

    python -m pip install torch



==================

External resources

==================



- @jakubczakon: `blog post

  `_

  "8 Creators and Core Contributors Talk About Their Model Training

  Libraries From PyTorch Ecosystem" 2020

- @BenjaminBossan: `talk 1

  `_ "skorch: A

  scikit-learn compatible neural network library" at PyCon/PyData 2019

- @githubnemo: `poster `_

  for the PyTorch developer conference 2019

- @thomasjpfan: `talk 2 `_

  "Skorch: A Union of Scikit learn and PyTorch" at SciPy 2019

- @thomasjpfan: `talk 3 `_

  "Skorch - A Union of Scikit-learn and PyTorch" at PyData 2018

- @BenjaminBossan: `talk 4 `_ "Extend your

  scikit-learn workflow with Hugging Face and skorch" at PyData Amsterdam 2023

  (`slides 4 `_)



=============

Communication

=============



- `GitHub discussions `_: 

  user questions, thoughts, install issues, general discussions.



- `GitHub issues `_: bug

  reports, feature requests, RFCs, etc.



- Slack: We run the #skorch channel on the `PyTorch Slack server

  `_, for which you can `request access

  here `_.