{"id":18527828,"url":"https://github.com/zhangjun/tensorrt-server","last_synced_at":"2025-05-14T21:33:14.754Z","repository":{"id":49799290,"uuid":"491832190","full_name":"zhangjun/TensorRT-Server","owner":"zhangjun","description":"TensorRT Server","archived":false,"fork":false,"pushed_at":"2023-05-21T10:20:51.000Z","size":70,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-17T06:30:04.096Z","etag":null,"topics":["inference-engine","inference-server","onnx","tensorrt"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zhangjun.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-05-13T09:13:23.000Z","updated_at":"2024-02-29T04:57:47.000Z","dependencies_parsed_at":"2024-11-06T17:57:03.212Z","dependency_job_id":"d49229ab-b585-4ea4-a522-7cc9b0f3f39b","html_url":"https://github.com/zhangjun/TensorRT-Server","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhangjun%2FTensorRT-Server","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhangjun%2FTensorRT-Server/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhangjun%2FTensorRT-Server/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhangjun%2FTensorRT-Server/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zhangjun","download_url":"https://codeload.github.com/zhangjun/TensorRT-Server/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254231290,"owners_count":22036341,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["inference-engine","inference-server","onnx","tensorrt"],"created_at":"2024-11-06T17:56:04.803Z","updated_at":"2025-05-14T21:33:14.719Z","avatar_url":"https://github.com/zhangjun.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# TensorRT-Server\n## Build\n```\nmkdir build \u0026\u0026 cd build\ncmake ..\n```\n\n# TensorRT Engine\n## engine\n- API\n    ```\n    getNBBindings()\n    getBindingDimensions(index)\n    getBindingName(index)\n    bindingIsInput(index)\n    getBindingIndex(name)\n    ```\n## network\n- API\n    ```\n    markOutput()\n\n    ```\n## ILayer and ITensor\n### ILayer\n### ITensor\n\n## custom plugin\n    ```\n    auto creator = getPluginRegistry()-\u003egetPluginCreator(\"mish_trt\", \"1\");\n    const PluginFieldCollection *pluginData = creator-\u003egetFieldNames();\n    IPluginV2 *pluginObj = creator-\u003ecreatePlugin((\"mish\" + std::to_string(linx)).c_str(), pluginData);\n    ITensor *inputTensors[] = {bn1-\u003egetOutput(0)};\n    auto mish = network-\u003eaddPluginV2(\u0026inputTensors[0], 1, *pluginObj);\n    return mish;\n    ```\n- IPluginV2IOExt\n    ```\n    int initialize()\n    void terminate()\n    void destroy()\n    Dims getOutputDimensions(int index, const Dims* inputs, int nbInputDims)\n    DataType getOutputDataType(int index, const nvinfer1::DataType* inputTypes, int nbInputs)\n    IPluginV2IOExt* clone()\n    size_t getWorkspaceSize(int maxBatchSize)\n    Dims getOutputDimensions(int index, const Dims* inputs, int nbInputDims)\n    int enqueue(int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream)\n    bool supportsFormatCombination(int pos, const PluginTensorDesc* inOut, int nbInputs, int nbOutputs)\n    void configurePlugin(const PluginTensorDesc* in, int nbInput, const PluginTensorDesc* out, int nbOutput)\n    void setPluginNamespace(const char* pluginNamespace)\n    const char* getPluginNamespace() const\n    size_t getSerializationSize()\n    void serialize(void* buffer)\n    const char* getPluginType()\n    const char* getPluginVersion()\n\n    ```\n- IPluginCreator\n    ```\n    IPluginV2IOExt* createPlugin(const char* name, const PluginFieldCollection* fc)\n    IPluginV2IOExt* deserializePlugin(const char* name, const void* serialData, size_t serialLength)\n    const char* getPluginName() const\n    const char* getPluginVersion()\n    const PluginFieldCollection* getFieldNames()\n    void setPluginNamespace(const char* libNamespace)\n    const char* getPluginNamespace() const\n\n    ```\n\n# TensorRT Optimization\n\n# code\n\nhttps://github.com/jkjung-avt/tensorrt_demos/blob/master/yolo/onnx_to_tensorrt.py\n\nhttps://github.com/jkjung-avt/tensorrt_demos/blob/master/yolo/yolo_to_onnx.py\n\nhttps://github.com/jkjung-avt/tensorrt_demos/blob/master/plugins/yolo_layer.h\n\nhttps://github.com/tjuskyzhang/Scaled-YOLOv4-TensorRT/blob/master/yolov4-csp-tensorrt/yololayer.h\n\nhttps://github.com/tjuskyzhang/Scaled-YOLOv4-TensorRT/blob/master/yolov4-csp-tensorrt/yolov4-csp.cpp\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhangjun%2Ftensorrt-server","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzhangjun%2Ftensorrt-server","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhangjun%2Ftensorrt-server/lists"}