{"id":15661263,"url":"https://github.com/laggui/pytorch-cpp","last_synced_at":"2025-08-12T06:13:04.134Z","repository":{"id":80591075,"uuid":"156898576","full_name":"laggui/pytorch-cpp","owner":"laggui","description":"Just messing around with PyTorch 1.0's JIT compiler and their new C++ API Libtorch.","archived":false,"fork":false,"pushed_at":"2019-12-19T17:52:35.000Z","size":22,"stargazers_count":19,"open_issues_count":0,"forks_count":8,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-05-05T21:49:12.446Z","etag":null,"topics":["libtorch","pytorch","pytorch-cpp"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/laggui.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-11-09T17:56:14.000Z","updated_at":"2023-09-08T17:47:03.000Z","dependencies_parsed_at":null,"dependency_job_id":"2210bbd1-efb4-4d7a-ab18-81349eb6072f","html_url":"https://github.com/laggui/pytorch-cpp","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/laggui/pytorch-cpp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/laggui%2Fpytorch-cpp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/laggui%2Fpytorch-cpp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/laggui%2Fpytorch-cpp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/laggui%2Fpytorch-cpp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/laggui","download_url":"https://codeload.github.com/laggui/pytorch-cpp/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/laggui%2Fpytorch-cpp/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270011188,"owners_count":24511902,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-12T02:00:09.011Z","response_time":80,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["libtorch","pytorch","pytorch-cpp"],"created_at":"2024-10-03T13:26:44.219Z","updated_at":"2025-08-12T06:13:04.106Z","avatar_url":"https://github.com/laggui.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pytorch-cpp\nIn this repo I experiment with PyTorch 1.0 and their new JIT compiler, as well as their C++ API Libtorch.\n\nCurrently, the repo contains a VGG16 based network implementation in PyTorch for CIFAR-10 classification (based on my [previous experiment](https://github.com/laggui/NN_compress)), and the C++ source for inference.\n\n**Note:** timings may vary. In my previous experiments, I found that the traced TorchScript model does not bring any significant improvements when using it with the Python API, but the Libtorch inference time was much faster in C++. This is pretty cool because it means you can easily run your experiments and training in Python, and then bring your models over to your C++ project for serving.\n\n## pytorch/\nThis subdirectory includes the network's [architecture definition](pytorch/vgg.py), the [training script](pytorch/train.py), the [test script](pytorch/test.py) on the CIFAR-10 dataset, a [prediction script](pytorch/predict.py) for inference and, most importantly, the [script to convert the model to Torch Script](pytorch/to_torch_script.py).\n\n## libtorch/\nThis is where you'll find the source for the network's inference in C++. In [predict.cpp](libtorch/predict.cpp), we load the Torch Script module generated in PyTorch, read the input image and pre-process it in order to feed it to our network for inference.\n\n## Example Usage\n\n### PyTorch Predict\n\n```sh\npytorch$ python predict.py pytorch --model=../data/VGG16model.pth --image=../data/dog.png\n==\u003e Building model...\n==\u003e Loading PyTorch model...\nPredicted: dog | 10.056212425231934\nForward pass time: 0.0043811798095703125 seconds\nTotal time: 0.0052343260031193495 seconds\n```\n\n```sh\npytorch$ python predict.py torch-script --model=../data/VGG16-traced-eval.pt --image=../data/dog.png \n==\u003e Building model...\n==\u003e Loading Torch Script model...\nPredicted: dog | 10.056212425231934\nForward pass time: 0.01126241683959961 seconds\nTotal time: 0.012680109008215368 seconds\n```\n\nPredictions were done using a 1080 Ti GPU. Interestingly, the traced (static) network has slower inference time. Further investigation on a more realisitc application needs to be done, since this sample example is using CIFAR-10 images (32x32 RGB, which is a very small input size), and only predicting for one sample instead of continuously predicting in real-time.\n\n#### Further Testing\n\nIn order to realistically test the traced (static) network versus its standard (dynamic) PyTorch model counterpart, I trained the same VGG16 network (with depthwise separable convolutions) for a single epoch, and used the saved model to predict multiple times on the same input (upscaled 224x224 image of a dog from CIFAR-10).\n\n**Standard Model (Dynamic)**\n\n```sh\npytorch$ python predict.py pytorch --model=../data/VGG16model-224.pth --image=../data/dog-224.png --input=224 --test_timing=1\n==\u003e Building model...\n==\u003e Loading PyTorch model...\nPredicted: dog | 1.722057580947876\nForward pass time: 0.005976676940917969 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.004324197769165039 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.00431060791015625 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.0046079158782958984 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.0043218135833740234 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.004750728607177734 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.00461125373840332 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.0052700042724609375 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.004312992095947266 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.004832744598388672 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.004314422607421875 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.004302263259887695 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.0047190189361572266 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.005443096160888672 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.004314899444580078 seconds\nAvg forward pass time (excluding first): 0.00460256849016462 seconds\nTotal time: 0.0730239039985463 seconds\n```\n\n**Torch Script Model (Static)**\n\n```sh\npytorch$ python predict.py torch-script --model=../data/VGG16model-224-traced-eval.pt --image=../data/dog-224.png --input=224 --test_timing=1\n==\u003e Building model...\n==\u003e Loading Torch Script model...\nPredicted: dog | 1.722057580947876\nForward pass time: 0.014840841293334961 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.0043413639068603516 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.0043256282806396484 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.005699634552001953 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.004336118698120117 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.004330635070800781 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.0050067901611328125 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.00433039665222168 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.0043239593505859375 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.0047681331634521484 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.004338264465332031 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.004318952560424805 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.004320621490478516 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.004678487777709961 seconds\nPredicted: dog | 1.722057580947876\nForward pass time: 0.004454374313354492 seconds\nAvg forward pass time (excluding first): 0.004540954317365374 seconds\nTotal time: 0.08327161299530417 seconds\n```\n\nAs you can see, the difference in timing (averaged) is very slim. In both cases, the first forward pass takes longer than the following (and actually, the Torch Script model takes a lot longer).\n\n### Libtorch\nBefore running our prediction, we need to compile the source. In your `libtorch` directory, create a build directory and compile+build the application from source.\n\n```sh\nlibtorch$ mkdir build \nlibtorch$ cd build\nlibtorch/build$ cmake -DCMAKE_PREFIX_PATH=/path/to/libtorch ..\n-- The C compiler identification is GNU 5.4.0\n-- The CXX compiler identification is GNU 5.4.0\n-- Check for working C compiler: /usr/bin/cc\n-- Check for working C compiler: /usr/bin/cc -- works\n-- Detecting C compiler ABI info\n.\n.\n.\n-- Configuring done\n-- Generating done\n-- Build files have been written to: libtorch/build\nlibtorch/build$ make\nScanning dependencies of target vgg-predict\n[ 50%] Building CXX object CMakeFiles/vgg-predict.dir/predict.cpp.o\n[100%] Linking CXX executable vgg-predict\n[100%] Built target vgg-predict  \n```\n\nYou're now ready to run the application.\n\n```sh\nlibtorch/build$ ./vgg-predict ../../data/VGG16model.pth ../../data/dog.png\nModel loaded\nMoving model to GPU\nPredicted: dog | 10.0562\nTime: 0.009481 seconds\n```\n\n### TO-DO\n\n- Update experiments timings for the latest PyTorch release (1.3)\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flaggui%2Fpytorch-cpp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flaggui%2Fpytorch-cpp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flaggui%2Fpytorch-cpp/lists"}