https://github.com/lucacappelletti94/keras_mixed_sequence

Lazily loading mixed sequences using Keras Sequence, focused on multi-task models.
https://github.com/lucacappelletti94/keras_mixed_sequence

Last synced: 3 months ago
JSON representation

Lazily loading mixed sequences using Keras Sequence, focused on multi-task models.

• Host: GitHub
• URL: https://github.com/lucacappelletti94/keras_mixed_sequence
• Owner: LucaCappelletti94
• License: mit
• Created: 2020-01-23T20:35:35.000Z (almost 6 years ago)
• Default Branch: master
• Last Pushed: 2024-02-03T09:40:49.000Z (almost 2 years ago)
• Last Synced: 2025-04-02T22:42:41.762Z (9 months ago)
• Language: Python
• Size: 26 MB
• Stars: 0
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.rst
  • License: LICENSE

Awesome Lists containing this project

README

          
keras_mixed_sequence

=========================================================================================

|pip| |downloads|

Lazily loading mixed sequences using Keras Sequence,

focused on multi-task models.

How do I install this package?

----------------------------------------------

As usual, just download it using pip:

.. code:: shell

    pip install keras_mixed_sequence

Usage examples

----------------------------------------------

Example for traditional single-task models

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

First of all let's create a simple single-task model:

.. code:: python

    from tensorflow.keras.layers import Dense

    from tensorflow.keras.models import Sequential

    model = Sequential([

        Dense(1, activation="relu")

    ])

    model.compile(

        optimizer="nadam",

        loss="relu"

    )

Then we proceed to load or otherwise create the training data.

Here there will be listed, in the future, some custom

Sequence objects that have been created for the purpose

of being used alongside this library.

.. code:: python

    X = either_a_numpy_array_or_sequence_for_input

    y = either_a_numpy_array_or_sequence_for_output

Now we combine the training data using the MixedSequence

object.

.. code:: python

    from keras_mixed_sequence import MixedSequence

    sequence = MixedSequence(

        X, y,

        batch_size=batch_size

    )

Finally, we can train the model:

.. code:: python

    from multiprocessing import cpu_count

    model.fit_generator(

        sequence,

        steps_per_epoch=sequence.steps_per_epoch,

        epochs=2,

        verbose=0,

        use_multiprocessing=True,

        workers=cpu_count(),

        shuffle=True

    )

Example for multi-task models

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

First of all let's create a simple multi-taks model:

.. code:: python

    from tensorflow.keras.models import Model

    from tensorflow.keras.layers import Dense, Input

    inputs = Input(shape=(10,))

    output1 = Dense(

        units=10,

        activation="relu",

        name="output1"

    )(inputs)

    output2 = Dense(

        units=10,

        activation="relu",

        name="output2"

    )(inputs)

    model = Model(

        inputs=inputs,

        outputs=[output1, output2],

        name="my_model"

    )

    model.compile(

        optimizer="nadam",

        loss="MSE"

    )

Then we proceed to load or otherwise create the training data.

Here there will be listed, in the future, some custom

Sequence objects that have been created for the purpose

of being used alongside this library.

.. code:: python

    X = either_a_numpy_array_or_sequence_for_input

    y1 = either_a_numpy_array_or_sequence_for_output1

    y2 = either_a_numpy_array_or_sequence_for_output2

Now we combine the training data using the MixedSequence

object.

.. code:: python

    from keras_mixed_sequence import MixedSequence

    sequence = MixedSequence(

        x=X,

        y={

            "output1": y1,

            "output2": y2

        },

        batch_size=batch_size

    )

Finally, we can train the model:

.. code:: python

    from multiprocessing import cpu_count

    model.fit_generator(

        sequence,

        steps_per_epoch=sequence.steps_per_epoch,

        epochs=2,

        verbose=0,

        use_multiprocessing=True,

        workers=cpu_count(),

        shuffle=True

    )

.. |pip| image:: https://badge.fury.io/py/keras-mixed-sequence.svg

    :target: https://badge.fury.io/py/keras-mixed-sequence

    :alt: Pypi project

.. |downloads| image:: https://pepy.tech/badge/keras-mixed-sequence

    :target: https://pepy.tech/badge/keras-mixed-sequence

    :alt: Pypi total project downloads