{"id":19478146,"url":"https://github.com/decathlon/decavision","last_synced_at":"2025-04-25T14:32:58.708Z","repository":{"id":37678853,"uuid":"318210393","full_name":"Decathlon/decavision","owner":"Decathlon","description":"Python package to help train image classification deep neural networks for generic datasets. Especially powerful when using with TPUs. The main advantages of the package is that is leverages the tfrecords format for the data along with transfer learning and hyperparameter optimization for model training.","archived":false,"fork":false,"pushed_at":"2023-10-03T23:52:17.000Z","size":7049,"stargazers_count":4,"open_issues_count":10,"forks_count":3,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-03T23:11:18.226Z","etag":null,"topics":["deep-learning","hyperparameter-optimization","image-classification","keras-tensorflow","tfrecord","transfer-learning"],"latest_commit_sha":null,"homepage":"https://decavision-doc.herokuapp.com/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Decathlon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"contributing.md","funding":null,"license":"LICENSE","code_of_conduct":"code-of-conduct.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-12-03T13:56:37.000Z","updated_at":"2023-10-17T14:28:07.000Z","dependencies_parsed_at":"2023-12-26T09:06:29.242Z","dependency_job_id":"289b6e95-1299-4656-a7b9-e902cf1ca7c8","html_url":"https://github.com/Decathlon/decavision","commit_stats":{"total_commits":152,"total_committers":5,"mean_commits":30.4,"dds":0.5263157894736843,"last_synced_commit":"ea673ab9dfa34c9c904c2b3b3c285c7688a66b46"},"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Decathlon%2Fdecavision","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Decathlon%2Fdecavision/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Decathlon%2Fdecavision/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Decathlon%2Fdecavision/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Decathlon","download_url":"https://codeload.github.com/Decathlon/decavision/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250834267,"owners_count":21494940,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","hyperparameter-optimization","image-classification","keras-tensorflow","tfrecord","transfer-learning"],"created_at":"2024-11-10T19:47:21.568Z","updated_at":"2025-04-25T14:32:55.823Z","avatar_url":"https://github.com/Decathlon.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Image classification\n\nThis repository contains the code that is used to create the decavision package published on PyPI.\nThe package can be found on [PyPI](www.pypi.com/decavision) and is compatible with python versions 3.6 and above.\n\nThe complete documentation can be found [here](https://decavision-doc.herokuapp.com/).\n\n## Improve the library\n\nIf you want to modify something in the code, it is as simple as cloning this repository\n```\ngit clone https://github.com/Decathlon/decavision.git\ncd decavision\n```\nmodifying the scripts and using the code as if you were using the package itself. This can be done locally or in a \nColab notebook if you want to exploit the free GPUs and TPUs. In that case, an example of notebook is provided in the \ndocumentation. If you use the repository locally, make sure that all the requirements are installed and it is \nrecommended to work in a clean environment:\n```\npip install -r requirements.txt\n```\n\n## Pull request\n\nWhen working on the source code you have to do it in a new branch. Only the code on the master branch is considered \nfor deployment so a PR must be made and accepted to push the changes online.\n\n### pep8speaks\n\nA [bot](https://github.com/OrkoHunter/pep8speaks) is used in the repository to analyse PRs and check if the code follows \nPEP8 conventions. The bot comments all the problems and nothing forces the changes to be done. Reviewers can use the \ncomment to request for modifications. To modify the errors used by pep8speaks, take a look at the `.pep8speaks.yml` file\n\n### PR checklist\n\nWhen evaluating a pull request, here is a list of the things to check\n- `setup.py` file must be updated with a new version number and explanations of the modifications in the long description\n- major changes must be explained in the documentation and in that case make sure that the `index.php` and `composer.json` files exist\n- make sure the code works locally and on colab with a GPU and a TPU\n- make sure that all the functions have docstrings and that they are all up to date\n- `conf.py` file of the docs must show the most recent version\n\n## Deploy package\n\nOnce you have made modifications that you are satisfied with, you can push a new version of the package online. \nMore info about the procedure can be found [here](https://packaging.python.org/tutorials/packaging-projects/).\n\nYou will have to update the version number in the setup.py file. Increase the first digit if you made MAJOR changes, \nthe second digit for MINOR changes and the third digit for BUG FIXES.\n\nAlso make sure that you create a new release on github at the commit that you deploy. To do so you have to create a tag \nwith the version number and add it to a release. The command to create a tag is\n```\ngit tag -a vX.X.X commit_id -m \"Your message here\"\n```\nwith the `commit_id` being the id of the commit that you are deploying. Then you just use\n```\ngit push origin vX.X.X\n```\nto send the tag to github.\n\n### Process (dev)\n\nIf you make modifications to the code and want to test what it will look like in the packaged form you can push your \ncode to github and install the package from the repo to use it:\n```\npip install git+https://github.com/Decathlon/decavision.git@branch_name\n```\n\n### Process (prod)\n\nAfter testing to make sure everything works you can deploy to PyPI. To do so you have to install a few dependencies:\n```\npip install setuptools wheel twine\n```\nYou can then build the distribution by running the following line in the root directory\n```\npython setup.py sdist bdist_wheel\n```\nThis will create two files (a .whl and a .tar.gz) in a 'dist' directory, with reference to the package version number\n in their name. Make sure to delete the older files if you have any. Finally you can push the package to PyPI with\n```\npython -m twine upload dist/*\n```\nThis command will ask you to login to your PyPI account and you need to have the rights to the package.\n\n## Build documentation\n\nThis project uses [sphinx](https://www.sphinx-doc.org/en/master/usage/quickstart.html) to generate the documentation. \nA large part of the docs is generated automatically from the docstrings of the functions and classes in the code. \nThese are all written using the google style. Here is an example of this style:\n ```\ndef function(arg1, arg2):\n    \"\"\"\n    description of the function\n\n    Arguments:\n        arg1 (type of arg1): description of arg1\n        arg2 (type of arg2): description of arg2\n\n    Returns:\n        type of output: description of output\n```\nUsing this style with sphinx is possible because of the [napoleon](https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html) extension.\n\n### Process\n\nIn order to modify the documentation, the first thing to do is to clone the repository and install the requirements.\nYou additionally have to install sphinx and the template that we use:\n```\npip install sphinx\npip install sphinx-rtd-theme\n```\nThen change to the 'docs' directory. This contains two types of files:\n- conf.py\n- .rst files\n\nThe conf.py file contains all the specifications about the project, like the extensions and their properties.\nThis is also where you can change the template that is used. For simple modifications, this doesn't need to be modified.\n\nThe .rst (reStructuredText) files contain the bulk of the text. The index.rst file is where the table of content is \ndefined, using the following type of code:\n```\n.. toctree::\n   :maxdepth: 3\n\n   code\n```\nThis takes what is written in the file code.rst and makes an element in the table of content. The rest of the files \nare built using plain reStructuredText (the sphinx doc contains a good primer on this language). To include docstrings \nfrom a specific script, use the following code:\n```\n.. automodule:: path.to.script\n   :members:\n```\n\nAfter doing your modifications, you just have to type the following line in the command prompt, still from the 'docs' directory:\n```\nmake html\n```\nThe files in the folder _build will be updated with your documentation documentation. You can open \n_build/html/index.html in a browser to explore the documentation. Sometimes there may be conflicts when modifying the\npages. If you see something strange, just delete the whole content of _build and build the documentation once again.\n\nMore detail about how to use sphinx can be found in this [blog post](https://medium.com/@richdayandnight/a-simple-tutorial-on-how-to-document-your-python-project-using-sphinx-and-rinohtype-177c22a15b5b)\n\n### Deploy documentation\n\nFor now the documentation is deployed on heroku. The deployment there is simple. Just follow the steps. This assumes\nthat you have the [Heroku CLI](https://devcenter.heroku.com/articles/heroku-cli) and that your heroku account has \naccess to the decavision-doc app.\n\n1. Make sure that you have a file called 'index.php' in the folder _build/html and that it contains the code:\n   ```\n   \u003c?php include_once(\"index.html\"); ?\u003e\n   ```\n   You also need to have a file called ```composer.json``` that contains only `{}`.\n2. Commit all your modifications using git\n3. Login to your heroku account using the command line:\n   ```\n   heroku login\n   ```\n4. Add the heroku project as a remote for git:\n   ```\n   heroku git:remote -a decavision-doc\n   ```\n5. Push the modifications to heroku with the following line:\n   ```\n   git subtree push --prefix docs/_build/html heroku master\n   ```\n   This has to be done from the main directory and it pushes only the code in the docs/_build/html subfolder to heroku. If it fails you can try\n   ```\n   cd docs/_build/html\n   git init\n   git add .\n   git commit -am \"deploy\"\n   heroku git:remote -a decavision-doc\n   git push --force heroku master\n   rm -rf .git\n   cd ../../..\n   ```\n\n### To do\n\n- use tensorflow's efficientnets\n- add other pretrained models (mobilenet, B4)\n- save best model when optimizing\n- choose which hyperparameters to search\n- use data from bucket to generate tfrecords\n- use local data to generate tfrecords with TPU\n- generate tfrecords in batches\n- allow to unfreeze everything when training\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdecathlon%2Fdecavision","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdecathlon%2Fdecavision","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdecathlon%2Fdecavision/lists"}