{"id":30523998,"url":"https://github.com/lightonai/transfer-learning-opu","last_synced_at":"2025-08-26T20:52:24.614Z","repository":{"id":40969946,"uuid":"241848379","full_name":"lightonai/transfer-learning-opu","owner":"lightonai","description":"Optical Transfer Learning","archived":false,"fork":false,"pushed_at":"2022-11-22T07:38:00.000Z","size":900,"stargazers_count":26,"open_issues_count":2,"forks_count":3,"subscribers_count":5,"default_branch":"master","last_synced_at":"2023-03-04T05:22:24.135Z","etag":null,"topics":["deep-learning","machine-learning","photonic-computing","tensorrt","transfer-learning"],"latest_commit_sha":null,"homepage":"https://medium.com/@LightOnIO/au-revoir-backprop-bonjour-optical-transfer-learning-5f5ae18e4719","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lightonai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-02-20T09:51:54.000Z","updated_at":"2022-09-18T04:37:18.000Z","dependencies_parsed_at":"2022-08-31T13:21:35.134Z","dependency_job_id":null,"html_url":"https://github.com/lightonai/transfer-learning-opu","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"purl":"pkg:github/lightonai/transfer-learning-opu","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightonai%2Ftransfer-learning-opu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightonai%2Ftransfer-learning-opu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightonai%2Ftransfer-learning-opu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightonai%2Ftransfer-learning-opu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lightonai","download_url":"https://codeload.github.com/lightonai/transfer-learning-opu/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightonai%2Ftransfer-learning-opu/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272254542,"owners_count":24901064,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-26T02:00:07.904Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","machine-learning","photonic-computing","tensorrt","transfer-learning"],"created_at":"2025-08-26T20:52:24.119Z","updated_at":"2025-08-26T20:52:24.602Z","avatar_url":"https://github.com/lightonai.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Au Revoir Backprop! Bonjour Optical Transfer Learning!\nCode used to produce data for our blog post [Au Revoir Backprop! Bonjour Optical Transfer Learning!](https://medium.com/@LightOnIO/au-revoir-backprop-bonjour-optical-transfer-learning-5f5ae18e4719)\n\n## Access to Optical Processing Units\n\nTo request access to LightOn Cloud and try our photonic co-processor, please visit: https://cloud.lighton.ai/\n\nFor researchers, we also have a LightOn Cloud for Research program, please visit https://cloud.lighton.ai/lighton-research/ for more information.\n\n## How to install\n\nWe advise creating a `virtualenv` before running these commands. You can create one with `python3 -m venv \u003cvenv_name\u003e`. \nActivate it with source `\u003cpath_to_venv\u003e/bin/activate`  before proceeding. We used `python 3.5` and `pytorch 1.2` \nfor all the simulations.\n\n- Clone the repository and then do `pip install \u003cpath_to_repo\u003e`.\n\n- (optional) Should you wish to replicate the results with `TensorRT` in `int8` you need to download the appropriate version from the\n [official NVIDIA website](https://developer.nvidia.com/tensorrt). We tested the code with `TensorRT 6.0.1.5` with `CUDA 10.1`.\n\n- Finally download the dataset from the [Kaggle page](https://www.kaggle.com/alessiocorrado99/animals10). You should put the dataset \nin the  same folder as the repo, but all scripts have an option to change the path with `-dataset_path`. \n\n**NOTE**: we had problems with the `Pillow` package because this combination of Pytorch and TensorRT requires version\n `Pillow 6.1` in the `onnx` conversion of the model. If you have the same problems, uninstall `Pillow` and then retry with\n `pip install Pillow==6.1`. \n\n## Replicate the OPU/backprop results\n\nUse the script `multiple_block.sh` in the `bash` folder. Open it in a text editor and then:\n- set the OPU/backprop flags at the top to `true`, depending on which simulation you want to run\n- Set the dtype to `float32`/`float16`. This affects only the OPU simulation.\n\n- (optional) change the path to the script/dataset/save folder if you want to deviate from the \ndefaults; \n\n- launch `./multiple_block.sh`. You might need to run `chmod +x multiple_block.sh` to make the script executable.\n\n#### Jupyter notebook option\nThe notebook `TL_OPU.ipynb` in the `notebooks` folder does largely the same thing as the OPU script. It is a good way \nto get an idea of the general pipeline on the full DenseNet model.\n\n## Replicate the TensorRT results  \n\nNavigate to the `script` folder and then launch the following command: \n\n```\npython3 tensorrt_training.py densenet169 Saturn -dtype_train int8 -dtype_inf int8 -block 10 -layer 12 \n-n_components 2 -encode_type plain_th -encode_thr 0 -alpha_exp_min 6 -alpha_exp_max 8 \n-save_path ~/dummy/int8/ -features_path ~/datasets_conv_features/int8_features/\n``` \n\nSubstitute the `save_path` with your desired destination folder. In the above example I had pre-extracted the features \non a GPU which supported `int8` (RTX 2080) and then moved them to the OPU machine. If your machine already supports \n`int8` just drop the `-features_path` argument.\n\nIf you want to just extract the dataset features you can use the `tensorrt_extract_features.py`. Example call:\n\n```\npython3 tensorrt_extract_features.py densenet169 32 -block 10 -layer 12 \n-dtype_train int8 -dtype_inf int8 -dataset_path ~/datasets/animals10/\n```\n\nObviously change the dataset path with the correct one on your machine.\n\n## Hardware specifics\n\nAll the simulations have been run on a Tesla P100 GPU with 16GB memory and a Intel(R) Xeon(R) Gold 6128 CPU @ 3.40GHz with 12 cores. \nFor the int8 simulations we use an RTX 2080 with 12GB memory.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flightonai%2Ftransfer-learning-opu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flightonai%2Ftransfer-learning-opu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flightonai%2Ftransfer-learning-opu/lists"}