{"id":13442713,"url":"https://github.com/chiehpower/Setup-deeplearning-tools","last_synced_at":"2025-03-20T15:30:24.214Z","repository":{"id":41157604,"uuid":"243426358","full_name":"chiehpower/Setup-deeplearning-tools","owner":"chiehpower","description":"Set up CI in DL/ cuda/ cudnn/ TensorRT/ onnx2trt/ onnxruntime/ onnxsim/ Pytorch/ Triton-Inference-Server/ Bazel/ Tesseract/ PaddleOCR/ NVIDIA-docker/ minIO/ Supervisord on AGX or PC from scratch.","archived":false,"fork":false,"pushed_at":"2023-09-27T02:44:32.000Z","size":4929,"stargazers_count":43,"open_issues_count":1,"forks_count":7,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-10-28T05:59:38.302Z","etag":null,"topics":["agx","ci","cuda","cudnn","deep-learning","docker","installation","minio","nvidia","onnx-simplifier","onnx2trt","onnxruntime","paddleocr","pytorch","supervisord","tensorrt","tensorrt-inference-server","tesseract-ocr","triton-inference-server","triton-server"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chiehpower.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-02-27T03:59:27.000Z","updated_at":"2024-08-17T07:27:52.000Z","dependencies_parsed_at":"2024-10-28T04:00:45.043Z","dependency_job_id":"8a71f92f-d53b-44de-8948-e940b3da2d45","html_url":"https://github.com/chiehpower/Setup-deeplearning-tools","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chiehpower%2FSetup-deeplearning-tools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chiehpower%2FSetup-deeplearning-tools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chiehpower%2FSetup-deeplearning-tools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chiehpower%2FSetup-deeplearning-tools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chiehpower","download_url":"https://codeload.github.com/chiehpower/Setup-deeplearning-tools/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244639815,"owners_count":20485932,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agx","ci","cuda","cudnn","deep-learning","docker","installation","minio","nvidia","onnx-simplifier","onnx2trt","onnxruntime","paddleocr","pytorch","supervisord","tensorrt","tensorrt-inference-server","tesseract-ocr","triton-inference-server","triton-server"],"created_at":"2024-07-31T03:01:49.553Z","updated_at":"2025-03-20T15:30:24.208Z","avatar_url":"https://github.com/chiehpower.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"[![](https://img.shields.io/badge/Lauguage-English-blue)](./) [![](https://img.shields.io/badge/CUDA-v10.0-lightgrey)](https://developer.nvidia.com/cuda-10.0-download-archive?target_os=Linux\u0026target_arch=x86_64\u0026target_distro=Ubuntu\u0026target_version=1804\u0026target_type=deblocal) [![](https://img.shields.io/badge/cuDNN-v7.6.5-red)](https://developer.nvidia.com/rdp/cudnn-download) [![](https://img.shields.io/badge/TensorRT-v7.0.0.11-orange)](https://developer.nvidia.com/nvidia-tensorrt-7x-download) [![](https://img.shields.io/badge/Status-Updating-yellow)](./)\n\n[English](./) | [中文](./Mandarin/README.md)\n\n---\n# The list \n\n- Install onnx2trt: [Check here](./onnx2trt/README.md)\n- Install onnxruntime: [Check here](./onnxruntime/README.md)\n- Install onnx-simplifier: [Check here](./onnx-simplifier/README.md)\n- Install Pytorch: [Check here](./Pytorch/README.md)\n- Install NVIDIA-docker: [Check here](./NVIDIA-docker/README.md)\n- Set up AGX: [Check here](./AGX/README.md)\n- Install Tesseract: [Check here](./Tesseract/README.md)\n- Install Bazel: [Check here](./Bazel/README.md)\n- Test Triton-Inference-Server: [Check here](./Triton-inference-server/README.md)\n- Docker_Installation : [Check here](./Docker_Installation/README.md)\n- Install NVIDIA Apex : [Check here](./NVIDIA-Apex/README.md)\n- Set up the docker container for the GPUs of Ampere archittecture : [Check here](./Ampere_docker/README.md)\n- Set up CI (Continuous integration) with Deep Learning / Python : [Check here](https://github.com/chiehpower/Gitlab-CI-For-Deep-Learning/tree/b1854f1431b880b11e9d4e58bb73b12481ed2f9c)\n- Upgrade GPU driver : [Check here](./Upgrade-GPU-Driver/README.md)\n- Setup Ray (including examples) : [Check here](./Ray/README.md)\n- Integrate Supervisor in to your services : [Check here](./supervisord/README.md)\n- Install PaddleOCR in (AGX) ARM devices : [Check here](./PaddleOCR/README.md)\n- Set up minIO database : [Check here](./minIO/README.md)\n\n#### ▍Git Clone\n\n```bash\ngit clone https://github.com/chiehpower/Setup-deeplearning-tools.git\ncd Setup-deeplearning-tools\ngit submodule update --init --recursive\n```\n\n---\n# Table of Contents\n- Install / Remove CUDA and cudnn\n  - Remove\n  - Install CUDA\n  - Install cudnn\n  - Check it\n- Install TensorRT\n- onnx2trt\n- onnxruntime\n- onnx-simplifier\n- Pytorch\n- AGX\n- NVIDIA-docker\n- Tesseract\n- Bazel\n- Triton-Inference-Server\n- Docker-Installation \n- NVIDIA Apex\n- Set up the docker container for the GPUs of Ampere archittecture\n- Set up CI (Continuous integration) with Deep Learning / Python\n- Upgrade GPU Driver\n- Supervisor\n- Install PaddleOCR in (AGX) ARM devices\n- Set up minIO\n\n---\n# Install NVIDIA GPU Driver \n\nFirst, we need to add repository into your system.\n```\nsudo add-apt-repository ppa:graphics-drivers/ppa\n```\nSecond, update it.\n```\nsudo apt update\n```\nThen we can check our gpu which one driver is suitable for us to install.\n```\nubuntu-drivers devices\n```\nThe info should be like below:\n\n![gpu-driver](./assets/gpu-driver.png)\n\nSo I chose 450 this one driver. \n```\nsudo apt install nvidia-driver-450\n```\nDone~\n\n# Install / Remove CUDA and cudnn\n\n**If your GPU is using Ampere architecture, you have to notice the version of CUDA what you choose.**\n\n## Remove\n\n(My original version is 10.1, so I remove the folder cuda-10.1)\n\n```\nsudo apt-get remove cuda-10.1 \nsudo apt autoremove\n```\n\nAfter the steps above, go to `/etc/apt/sources.list.d` and remove those files which are relevant with the words of CUDA.\n\n```\nsudo rm cuda.list \n```\n\n## Install CUDA\n\n1. First, download the .deb file from here. \n\thttps://developer.nvidia.com/cuda-10.0-download-archive?target_os=Linux\u0026target_arch=x86_64\u0026target_distro=Ubuntu\u0026target_version=1804\u0026target_type=deblocal\n2. sudo dpkg -i cuda-repo-ubuntu1804-10-0-local-10.0.130-410.48_1.0-1_amd64.deb\n3. sudo apt-key add /var/cuda-repo-10-0-local-10.0.130-410.48/7fa2af80.pub \n4. sudo apt-get update\n5. sudo apt-get install cuda![meta-package](./assets/cuda.png)\n6. sudo apt-get install cuda-libraries-dev-10-0 \n\u003e Other installation options are available in the form of meta-packages. For example, to install all the library packages, replace \"cuda\" with the \"cuda-libraries-10-0\" meta package. For more information on all the available meta packages click [here](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#package-manager-metas).\n7. sudo apt-get install cuda-libraries-10-0 \n8. sudo apt-get install cuda-runtime-10-0\n9. sudo apt-get install cuda-toolkit-10-0\n10. sudo apt-get install cuda-10-0\n\n**Please don't forget to add the path of CUDA in .zshrc or .bashrc.**\n\n```\nexport LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-10.0/lib64\nexport CUDA_INSTALL_DIR=/usr/local/cuda-10.0\nexport PATH=$PATH:/usr/local/cuda-10.0/bin\nexport CUDA_HOME=$CUDA_HOME:/usr/local/cuda-10.0\nexport PATH=/usr/local/cuda-10.0/bin${PATH:+:${PATH}}\nexport LD_LIBRARY_PATH=/usr/local/cuda-10.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}\n```\n\n## Install cudnn\nDownload from : https://developer.nvidia.com/rdp/cudnn-download\n(Recommend to download the .tar file.)\n\nCopy these files to the folder of `/cuda/include/` and `/cuda/lib64/`\n\n```\n\u003e sudo cp cuda/include/cudnn.h /usr/local/cuda/include/\n\u003e sudo cp cuda/lib64/lib* /usr/local/cuda/lib64/\n```\nGo to the folder of `/usr/local/cuda/lib64/` \n```\ncd /usr/local/cuda/lib64/\n```\nBuild the symbolic link (You need to use your version number instead of 7.6.5 if your version is different with mine.)\n```\nsudo chmod +r libcudnn.so.7.6.5\nsudo ln -sf libcudnn.so.7.3.1 libcudnn.so.7\nsudo ln -sf libcudnn.so.7 libcudnn.so\nsudo ldconfig\n```\n\n## Check it\n\n```\nnvidia-smi\n```\n\n```\nnvcc -V\n```\n\n### NOTE: CUDA Version 10.2\n\n\nAfter you install the cuda and cudnn as followed official instructions, you meet some problems about cuda libs while you are running some python packages.\nFor example, `OSError: libcublas.so.10.0: cannot open shared object file: No such file or directory`\n\nPlease try this.\n\n```\nsudo ln -sf /usr/local/cuda/lib64/libcudnn.so.8.0. /usr/local/cuda/lib64/libcudnn.so.8\n\nsudo ln -s /usr/local/cuda-10.2/lib64/libcudart.so.10.2 /usr/lib\nsudo ln -s /usr/local/cuda-10.2/lib64/libcudart.so.10.2 /usr/lib/libcudart.so.10.1\nsudo ln -s /usr/local/cuda-10.2/lib64/libcudart.so.10.2 /usr/lib/libcudart.so.10.0\nsudo ldconfig\n\nsudo apt-get install cuda-libraries-10-0\n```\n\n---\n# Install TensorRT\n\nThe latest version is TensorRT 7.\nWe download the TensorRT version 7 from [here](https://developer.nvidia.com/nvidia-tensorrt-7x-download). (You need to login your account.)\n\nMy system is `Ubunty 18.04`, `cuDNN version 7.6.5` and `CUDA version 10.0`. I recommend you to install by tar package.\n\nSo I chose this tar package. If your version setting is same with me. You can download from here directly.\n[TensorRT 7.0.0.11 for Ubuntu 18.04 and CUDA 10.0 tar package](https://developer.nvidia.com/compute/machine-learning/tensorrt/secure/7.0/7.0.0.11/tars/TensorRT-7.0.0.11.Ubuntu-18.04.x86_64-gnu.cuda-10.0.cudnn7.6.tar.gz)\n\nExtract it.\n```\ntar -zxvf TensorRT-7.0.0.11.Ubuntu-18.04.x86_64-gnu.cuda-10.0.cudnn7.6.tar.gz\n```\n\nPlease follow this [instruction](https://docs.nvidia.com/deeplearning/sdk/tensorrt-install-guide/index.html). \nAccording to my case, I followed [this part](https://docs.nvidia.com/deeplearning/sdk/tensorrt-install-guide/index.html#installing-tar).\n\nFirst, we go into your TensorRT folder.\n```\ncd TensorRT7\n```\n\n## Install the Python TensorRT wheel file\n\n```\ncd ./python\nsudo pip3 install tensorrt-*-cp3x-none-linux_x86_64.whl\ncd ..\n```\n\n## Install the Python UFF wheel file. This is only required if you plan to use TensorRT with TensorFlow\n\n```\ncd ./uff\nsudo pip3 install uff-0.6.5-py2.py3-none-any.whl\nwhich convert-to-uff\ncd ..\n```\n\n## Install the Python `graphsurgeon` wheel file.\n\n```\ncd ./graphsurgeon\nsudo pip3 install graphsurgeon-0.4.1-py2.py3-none-any.whl\ncd ..\n```\n\n## Export your path of TensorRT lib\n\n1. Open your .bashrc / .zshrc\n\t```\n\tvim ~/.bashrc\n\t```\n\tor \n\t```\n\tvim ~/.zshrc\n\t```\n2. Add it by your location\n\t```\n\tLD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/(your location)/TensorRT7/lib\n\texport TRT_RELEASE=/home/(your location)/TensorRT7_cuda100\n\t```\n3. Source it\n\t```\n\tsource ~/.bashrc\n\t```\n\tor \n\t```\n\tsource ~/.zshrc\n\t```\n\n## Check it\n\nYou can use python version 3 to test on your command line.\n```\nimport tensorrt\n```\nIt should not happen any wrong.\n\nYou can also check by those samples on `~/TensorRT7/samples/` which are C++ samples and `~/TensorRT7/samples/python` which are python samples.\n\n# Update Cmake version\n\n```\nwget https://github.com/Kitware/CMake/releases/download/v3.14.4/cmake-3.14.4-Linux-x86_64.sh\nchmod +x cmake-3.14.4-Linux-x86_64.sh\nsudo ./cmake-3.14.4-Linux-x86_64.sh --prefix=/usr/local --exclude-subdir --skip-license\nrm ./cmake-3.14.4-Linux-x86_64.sh\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchiehpower%2FSetup-deeplearning-tools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchiehpower%2FSetup-deeplearning-tools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchiehpower%2FSetup-deeplearning-tools/lists"}