{"id":16162938,"url":"https://github.com/ipitio/bronte","last_synced_at":"2025-08-14T19:20:56.510Z","repository":{"id":214557559,"uuid":"731813740","full_name":"ipitio/bronte","owner":"ipitio","description":"Deep Learning Playground: A modular and extensible framework","archived":false,"fork":false,"pushed_at":"2024-01-17T00:43:59.000Z","size":56611,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-13T08:52:22.134Z","etag":null,"topics":["deep-learning","etl","framework","pytorch"],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ipitio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-15T00:08:48.000Z","updated_at":"2023-12-15T00:13:33.000Z","dependencies_parsed_at":"2024-11-02T08:32:36.358Z","dependency_job_id":null,"html_url":"https://github.com/ipitio/bronte","commit_stats":null,"previous_names":["ipitio/bronte"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipitio%2Fbronte","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipitio%2Fbronte/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipitio%2Fbronte/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipitio%2Fbronte/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ipitio","download_url":"https://codeload.github.com/ipitio/bronte/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247590393,"owners_count":20963136,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","etl","framework","pytorch"],"created_at":"2024-10-10T02:33:56.673Z","updated_at":"2025-04-07T04:40:19.547Z","avatar_url":"https://github.com/ipitio.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Bronte\n\n![thunder](thunder.png)\n\n[![License: AGPL](https://img.shields.io/badge/License-AGPL-blue.svg)](https://www.gnu.org/licenses/agpl-3.0)\n\n`bronte` is a modular and extensible Deep Learning framework; It views a model not as layers, but a trainer of layers, whose preprocessing and evaluation are task-specific. Like with `Pytorch Lightning`, this abstracts away training and allows for a clean separation of concerns, making it easy to modify, add, and experiment with different tasks and architectures. If you'd like to add a new one, you can do so by creating a new class in the appropriate module and adding it in `bronte`.\n\nIt is composed of the following modules:\n\n- `bronte`: Factory and Driver\n- `arch`: Layers and Forward Pass\n- `task`: Preprocessing and Evaluation\n- `base`: Training and Inference\n- `data`: Datasets\n- `loss`: Loss calculations\n- `tune`: Hyperparameter tuning\n\n`Bronte` the class takes a dictionary of options, including the names of a task and an arch, and creates a model. When data is passed to `Bronte`, it splits it into features X and target(s) y, and passes these to the model's `fit` method, which then initializes the layers, optimizer, scheduler, criterion, scaler, datasets, and dataloaders, and starts training. Please look at the notebook for a list of all currently supported options (under Deep Learning \u003e Options).\n\n\u003e **Note**\n\u003e\n\u003e You must initialize the layers not in `__init__`, but in `init_layers`, as this is used to (re)initialize the model's layers when (resuming) training.\n\n## Usage\n\n### Training\n\n    import bronte\n\n    data = [df]\n    models = [task | arch]\n\n    # load data into tables\n    for df in data:\n      bronte.load(df)\n\n    # start tensorboard\n    bronte.track()\n\n    # train models on tables, returning list of Bronte objects\n    trainers = bronte.fit(models)\n\n    # call again to stop tensorboard\n    bronte.track()\n\n    # flush db\n    bronte.flush()\n\n### Inference\n\n    import bronte\n\n    XX = [X]\n    paths = [\"models/.../model.pt\"]\n\n    # predict on list of new data, returning dict: {path: {str(XX.index(X)): y}}\n    predictions = bronte.predict(XX, paths)\n\n## Supports\n\n- Training:\n  - (C/G/T)PU\n  - Persistence\n  - Mixed Precision\n  - Multi input and output\n  - Model and state checkpointing\n  - Learning Rate scheduling\n  - Transfer Learning\n  - Gradient accumulation and scaling\n  - Parallel and Distributed with `dask`\n  - Hyperparameter tuning with `optuna`\n  - Calculating feature importances with `shap`\n  - Monitoring/Logging with `tensorboard`\n- Tasks:\n  - Regression\n  - Classification\n- Architectures:\n  - FFN\n  - RNN with Attention\n\n## TODO\n\n- [ ] Frontend + Flask\n- [ ] More archs, tasks\n- [ ] Tests, Typing, Documentation\n\n## Example\n\nThe notebook `basketball.ipynb` runs an ETL Pipeline for a sample dataset of Basketball statistics and performs Deep Learning using `Bronte`.\n\n### ETL Pipeline\n\nFirst, the data is extracted (from CSVs in this case), merged, and examined (ie. EDA) with `ydata-profiling`. Then it's transformed with some standard cleaning and dataset-specific feature engineering, before being partitioned into small chunks and loaded into a database.\n\n### Deep Learning\n\nThis database is then read table-by-table, for each task and arch specified, and passed to `Bronte`. Over the course of training, checkpoints of state and visuals of metrics and importances will be saved to `models/`. Once training is complete, `Bronte` can be used to load the trained models and make predictions on new data.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fipitio%2Fbronte","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fipitio%2Fbronte","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fipitio%2Fbronte/lists"}