{"id":21461157,"url":"https://github.com/marcpinet/neuralnetlib","last_synced_at":"2025-07-15T02:32:31.455Z","repository":{"id":206471321,"uuid":"716764629","full_name":"marcpinet/neuralnetlib","owner":"marcpinet","description":"🧠 A convolutional neural network library written in python with only numpy","archived":false,"fork":false,"pushed_at":"2024-05-18T15:56:17.000Z","size":4642,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-05-18T20:50:34.332Z","etag":null,"topics":["ai","artificial-intelligence","densenet","machine-learning","neural-network","numpy","python"],"latest_commit_sha":null,"homepage":"https://marcpinet.me","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/marcpinet.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-09T20:32:00.000Z","updated_at":"2024-05-18T15:56:01.000Z","dependencies_parsed_at":"2023-12-01T13:54:08.523Z","dependency_job_id":"f825dee8-891c-4308-a2f6-39e6724802f6","html_url":"https://github.com/marcpinet/neuralnetlib","commit_stats":null,"previous_names":["marcpinet/handmade-neuralnetwork","marcpinet/neuralnetlib"],"tags_count":28,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcpinet%2Fneuralnetlib","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcpinet%2Fneuralnetlib/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcpinet%2Fneuralnetlib/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcpinet%2Fneuralnetlib/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/marcpinet","download_url":"https://codeload.github.com/marcpinet/neuralnetlib/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225934411,"owners_count":17547740,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","artificial-intelligence","densenet","machine-learning","neural-network","numpy","python"],"created_at":"2024-11-23T07:07:46.889Z","updated_at":"2025-07-15T02:32:31.442Z","avatar_url":"https://github.com/marcpinet.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Neuralnetlib\n\n## 📝 Description\n\nThis is a handmade machine and deep learning framework library, made in python, **using numpy as its only external dependency**.\n\nI made it to challenge myself and to learn more about deep neural networks, how they work _in depth_.\n\nThe big part of this project, meaning the [Multilayer Perceptron (MLP)](https://en.wikipedia.org/wiki/Multilayer_perceptron) part, was made in a week.\n\nI then decided to push it even further by adding [Convolutional Neural Networks (CNN)](https://en.wikipedia.org/wiki/Convolutional_neural_network),  [Recurrent Neural Networks (RNN)](https://en.wikipedia.org/wiki/Recurrent_neural_network), [Autoencoders](https://en.wikipedia.org/wiki/Autoencoder), [Variational Autoencoders (VAE)](https://en.wikipedia.org/wiki/Variational_autoencoder), [GANs](https://en.wikipedia.org/wiki/Generative_adversarial_network) and [Transformers](https://en.wikipedia.org/wiki/Transformer_(machine_learning_model)).\n\nRegarding the Transformers, I just basically reimplement the [Attention is All You Need](https://arxiv.org/abs/1706.03762) paper. It theorically works but needs a huge amount of data that can't be trained on a CPU. You can however see what each layers produce and how the attention weights are calculated [here](examples/models-usages/generation/transformer-text-generation/transformer-debug.ipynb).\n\nThis project will be maintained as long as I have ideas to improve it, and as long as I have time to work on it.\n\n## 📦 Features\n\n- Many models architectures (sequential, functional, autoencoder, transformer, gan) 🏗\n- Many layers (dense, dropout, conv1d/2d, pooling1d/2d, flatten, embedding, batchnormalization, textvectorization, lstm, gru, attention and more) 🧠\n- Many activation functions (sigmoid, tanh, relu, leaky relu, softmax, linear, elu, selu) 📈\n- Many loss functions (mean squared error, mean absolute error, categorical crossentropy, binary crossentropy, huber loss) 📉\n- Many optimizers (sgd, momentum, rmsprop, adam) 📊\n- Supports binary classification, multiclass classification, regression and text generation 📚\n- Preprocessing tools (tokenizer, pca, ngram, standardscaler, pad_sequences, one_hot_encode and more) 🛠\n- Machine learning tools (isolation forest, kmeans, pca, t-sne, k-means) 🧮\n- Callbacks and regularizers (early stopping, l1/l2 regularization) 📉\n- Save and load models 📁\n- Simple to use 📚\n\n## ⚙️ Installation\n\nYou can install the library using pip:\n\n```bash\npip install neuralnetlib\n```\n\n## 💡 How to use\n\n## Basic usage\n\nSee [this file](examples/models-usages/mlp-classification-regression/mnist_multiclass.ipynb) for a simple example of how to use the library.\u003cbr\u003e\nFor a more advanced example, see [this file](examples/models-usages/cnn-classification/cnn_classification_mnist.ipynb) for using CNN.\u003cbr\u003e\nYou can also check [this file](examples/models-usages/mlp-classification-regression/sentiment_analysis.ipynb) for text classification using RNN.\u003cbr\u003e\n\n## Advanced usage\n\nSee [this file](examples/models-usages/generation/autoencoder_vae_example.ipynb) for an example of how to use VAE to generate new images.\u003cbr\u003e\nAlso see [this file](examples/models-usages/generation/gan_mnist_convolutional.ipynb) for an example of how to use GAN to generate new images.\u003cbr\u003e\nAnd [this file](examples/models-usages/rnn-text-generation/dinosaur_names_generator.ipynb) for an example of how to generate new dinosaur names.\u003cbr\u003e\n\nMore examples in [this folder](examples).\n\nYou are free to tweak the hyperparameters and the network architecture to see how it affects the results.\n\n## 🚀 Quick training examples (more [here](examples/models-usages/))\n\n### Binary Classification\n\n```python\nfrom neuralnetlib.models import Sequential\nfrom neuralnetlib.layers import Input, Dense\nfrom neuralnetlib.activations import Sigmoid\nfrom neuralnetlib.losses import BinaryCrossentropy\nfrom neuralnetlib.optimizers import SGD\nfrom neuralnetlib.metrics import accuracy_score\n\n# ... Preprocess x_train, y_train, x_test, y_test if necessary (you can use neuralnetlib.preprocess and neuralnetlib.utils)\n\n# Create a model\nmodel = Sequential()\nmodel.add(Input(10))  # 10 features\nmodel.add(Dense(8))\nmodel.add(Dense(1))\nmodel.add(Activation(Sigmoid()))  # many ways to tell the model which Activation Function you'd like, see the next example\n\n# Compile the model\nmodel.compile(loss_function='bce', optimizer='sgd')\n\n# Train the model\nmodel.fit(X_train, y_train, epochs=10, batch_size=32, metrics=['accuracy'])\n```\n\n### Multiclass Classification\n\n```python\nfrom neuralnetlib.activations import Softmax\nfrom neuralnetlib.losses import CategoricalCrossentropy\nfrom neuralnetlib.optimizers import Adam\nfrom neuralnetlib.metrics import accuracy_score\n\n# ... Preprocess x_train, y_train, x_test, y_test if necessary (you can use neuralnetlib.preprocess and neuralnetlib.utils)\n\n# Create and compile a model\nmodel = Sequential()\nmodel.add(Input(28, 28, 1)) # For example, MNIST images\nmodel.add(Conv2D(32, kernel_size=3, padding='same'), activation='relu')  # activation supports both str...\nmodel.add(BatchNormalization())\nmodel.add(MaxPooling2D(pool_size=2))\nmodel.add(Dense(64, activation='relu'))\nmodel.add(Dense(10, activation=Softmax()))  # ... and ActivationFunction objects\nmodel.compile(loss_function='categorical_crossentropy', optimizer=Adam())\n\n\nmodel.compile(loss_function='categorical_crossentropy', optimizer=Adam())  # same for loss_function and optimizer\n\n# Train the model\nmodel.fit(X_train, y_train_ohe, epochs=5, metrics=['accuracy'])\n```\n\n### Regression\n\n```python\nfrom neuralnetlib.losses import MeanSquaredError\nfrom neuralnetlib.metrics import accuracy_score\n\n# ... Preprocess x_train, y_train, x_test, y_test if necessary (you can use neuralnetlib.preprocess and neuralnetlib.utils)\n\n# Create and compile a model\nmodel = Sequential()\nmodel.add(Input(13))\nmodel.add(Dense(64, activation='leakyrelu'))\nmodel.add(Dense(1), activation=\"linear\")\n\nmodel.compile(loss_function=\"mse\", optimizer='adam')  # you can either put acronyms or full name\n\n# Train the model\nmodel.fit(X_train, y_train, epochs=100, batch_size=128, metrics=['accuracy'])\n```\n\n### Image Compression\n\n```python\nX, y = fetch_openml('Fashion-MNIST', version=1, return_X_y=True, as_frame=False)\nX = X.astype('float32') / 255.\n\nX = X.reshape(-1, 28, 28, 1)\n\nX_train, X_test = train_test_split(X, test_size=0.2, random_state=42)\n\nautoencoder = Autoencoder(random_state=42, skip_connections=True)\n\nautoencoder.add_encoder_layer(Input((28, 28, 1)))\nautoencoder.add_encoder_layer(Conv2D(16, kernel_size=(3, 3), strides=(2, 2), activation='relu', padding='same'))\nautoencoder.add_encoder_layer(Conv2D(32, kernel_size=(3, 3), strides=(2, 2), activation='relu', padding='same'))\n\nautoencoder.add_encoder_layer(Flatten())\nautoencoder.add_encoder_layer(Dense(64, activation='relu'))  # Bottleneck\n\nautoencoder.add_decoder_layer(Dense(7 * 7 * 32, activation='relu'))\nautoencoder.add_decoder_layer(Reshape((7, 7, 32)))\n\nautoencoder.add_decoder_layer(UpSampling2D(size=(2, 2)))  # Output: 14x14x32\nautoencoder.add_decoder_layer(Conv2D(16, kernel_size=(3, 3), activation='relu', padding='same'))\n\nautoencoder.add_decoder_layer(UpSampling2D(size=(2, 2)))  # Output: 28x28x16\nautoencoder.add_decoder_layer(Conv2D(1, kernel_size=(3, 3), activation='sigmoid', padding='same'))  # Output: 28x28x1\n\nautoencoder.compile(encoder_loss='mse', decoder_loss='mse', encoder_optimizer='adam', decoder_optimizer='adam', verbose=True)\n\nhistory = autoencoder.fit(X_train, epochs=5, batch_size=256, validation_data=(X_test,), verbose=True,)\n```\n\n### Image Generation\n\n```python\n# Load the MNIST dataset\n(x_train, y_train), (x_test, y_test) = mnist.load_data()\nn_classes = np.unique(y_train).shape[0]\n\n# Concatenate train and test data\nX = np.concatenate([x_train, x_test])\ny = np.concatenate([y_train, y_test])\n\n# Flatten images\nX = X.reshape(X.shape[0], -1)\n\n# Normalize pixel values\nX = X.astype('float32') / 255\n\n# Labels to categorical \ny = one_hot_encode(y, n_classes)\n\nnoise_dim = 32\n\ngenerator = Sequential()\ngenerator.add(Input(noise_dim))\ngenerator.add(Dense(128, input_dim=noise_dim + n_classes, activation='leakyrelu'))\ngenerator.add(Dense(784, activation='sigmoid'))\n\ndiscriminator = Sequential()\ndiscriminator.add(Input(784 + n_classes))\ndiscriminator.add(Dense(128, input_dim=784 + n_classes, activation='leakyrelu'))\ndiscriminator.add(Dense(1, activation='sigmoid'))\n\ngan = GAN(latent_dim=noise_dim, n_classes=n_classes)\n\ngan.compile(generator, discriminator, generator_optimizer='adam', discriminator_optimizer='adam', loss_function='bce', verbose=True)\n\nhistory = gan.fit(X, y, epochs=40, batch_size=128, plot_generated=True)   \n```\n\n### Text Generation (example here is for translation)\n\n```python\ndf = pd.read_csv(\"dataset.tsv\", sep=\"\\t\")\ndf.iloc[:, 1] = df.iloc[:, 1].apply(lambda x: re.sub(r'\\\\x[a-fA-F0-9]{2}|\\\\u[a-fA-F0-9]{4}|\\xa0|\\u202f', ' ', x))  # remove unicode characters\n\nLIMIT = 1000\nfr_sentences = df.iloc[:, 1].values.tolist()[0:LIMIT]\nen_sentences = df.iloc[:, 3].values.tolist()[0:LIMIT]\n\nfr_tokenizer = Tokenizer(filters=\"\", mode=\"word\")  # else the tokenizer would remove the special characters including ponctuation\nen_tokenizer = Tokenizer(filters=\"\", mode=\"word\")  # else the tokenizer would remove the special characters including ponctuation\n\nfr_tokenizer.fit_on_texts(fr_sentences, preprocess_ponctuation=True)\nen_tokenizer.fit_on_texts(en_sentences, preprocess_ponctuation=True)\n\nX = fr_tokenizer.texts_to_sequences(fr_sentences, preprocess_ponctuation=True, add_special_tokens=True)\ny = en_tokenizer.texts_to_sequences(en_sentences, preprocess_ponctuation=True, add_special_tokens=True)\n\nmax_len_x = max(len(seq) for seq in X)\nmax_len_y = max(len(seq) for seq in y)\nmax_seq_len = max(max_len_x, max_len_y)\n\nvocab_size_fr = len(fr_tokenizer.word_index)\nvocab_size_en = len(en_tokenizer.word_index)\nmax_vocab_size = max(vocab_size_fr, vocab_size_en)\n\nX = pad_sequences(X, max_length=max_seq_len, padding='post', pad_value=fr_tokenizer.PAD_IDX)\ny = pad_sequences(y, max_length=max_seq_len, padding='post', pad_value=en_tokenizer.PAD_IDX)\n\nx_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2, shuffle=False)\n\nmodel = Transformer(src_vocab_size=vocab_size_fr, tgt_vocab_size=vocab_size_en, d_model=512, n_heads=8, n_encoder_layers=8, n_decoder_layers=10, d_ff=2048, dropout_rate=0.1, max_sequence_length=max_seq_len, random_state=42)\n\nmodel.compile(loss_function=\"cels\", optimizer=Adam(learning_rate=5e-5, beta_1=0.9, beta_2=0.98, epsilon=1e-9, clip_norm=1.0, ), verbose=True)\n\nhistory = model.fit(x_train, y_train, epochs=50, batch_size=32, verbose=True, callbacks=[EarlyStopping(monitor='loss', patience=20), LearningRateScheduler(schedule=\"warmup_cosine\", initial_learning_rate=5e-5, verbose=True)],validation_data=(x_test, y_test), metrics=['bleu_score'])\n```\n\n\u003e [!NOTE]\n\u003e You can also save and load models using the `save` and `load` methods.\n\n```python\n# Save a model\nmodel.save('my_model.json')\n\n# Load a model\nmodel = Model.load('my_model.json')\n```\n\n## 📜 Some outputs and easy usages\n\n### Here is the decision boundary on a Binary Classification (breast cancer dataset):\n\n![decision_boundary](resources/img/decision_boundary.gif)\n\n\u003e [!NOTE]\n\u003e PCA (Principal Component Analysis) was used to reduce the number of features to 2, so we could plot the decision boundary.\n\u003e Representing n-dimensional data in 2D is not easy, so the decision boundary may not be *always* accurate.\n\u003e I also tried with t-SNE, but the results were not good.\n\n### Here is an example of a model training on the mnist using the library\n\n![cli](resources/img/cli.gif)\n\n### Here is an example of a loaded model used with Tkinter:\n\n![gui](resources/img/gui.gif)\n\n### Here, I replaced Keras/Tensorflow with this library for my [Handigits](https://github.com/marcpinet/handigits) project...\n\n![plot](resources/img/handigits.gif)\n\n### Here is the generated dinosaur names using a simple RNN and a list of existing dinosaur names.\n\n![dino](resources/img/dino.png)\n\n### Here are some MNIST generated images using a cGAN.\n\n![mnist_generated](resources/img/mnist_generated.gif)\n\n**You can __of course__ use the library for any dataset you want.**\n\n## ✏️ Edit the library\n\nYou can pull the repository and run:\n\n```bash\npip install -e .\n```\n\nAnd test your changes on the examples.\n\n## 🎯 TODO\n\n- [ ] Add support for stream dataset loading to allow loading large datasets (larger than your RAM)\n- [ ] Visual updates (tabulation of model.summary() parameters calculation, colorized progress bar, etc.)\n- [ ] Better save format (like h5py)\n- [ ] Add cuDNN support to allow the use of GPUs\n\n## 🐞 Know issues\n\nNothing yet! Feel free to open an issue if you find one.\n\n## ✍️ Authors\n\n- Marc Pinet - *Initial work* - [marcpinet](https://github.com/marcpinet)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcpinet%2Fneuralnetlib","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarcpinet%2Fneuralnetlib","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcpinet%2Fneuralnetlib/lists"}