{"id":18554250,"url":"https://github.com/oracle/hiq","last_synced_at":"2025-04-04T19:15:44.602Z","repository":{"id":42480322,"uuid":"466051631","full_name":"oracle/hiq","owner":"oracle","description":"HiQ - Observability And Optimization In Modern AI Era","archived":false,"fork":false,"pushed_at":"2025-01-27T18:04:29.000Z","size":19535,"stargazers_count":70,"open_issues_count":3,"forks_count":9,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-28T18:16:12.727Z","etag":null,"topics":["logging","monitoring","observability","python","tracing"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oracle.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-03-04T08:59:02.000Z","updated_at":"2025-03-21T23:53:51.000Z","dependencies_parsed_at":"2024-06-19T00:06:14.531Z","dependency_job_id":"dee43271-e4cb-437c-a5cc-92e579ad630b","html_url":"https://github.com/oracle/hiq","commit_stats":{"total_commits":114,"total_committers":6,"mean_commits":19.0,"dds":"0.39473684210526316","last_synced_commit":"afe2a1d3bb9d4f21cc2c5bfc6e291f9770b8b4f0"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oracle%2Fhiq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oracle%2Fhiq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oracle%2Fhiq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oracle%2Fhiq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oracle","download_url":"https://codeload.github.com/oracle/hiq/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247234923,"owners_count":20905854,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["logging","monitoring","observability","python","tracing"],"created_at":"2024-11-06T21:20:39.241Z","updated_at":"2025-04-04T19:15:44.584Z","avatar_url":"https://github.com/oracle.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![](docs/../hiq/docs/source/_static/hiq.png) 🦉  Observability And Optimization In Modern AI Era\n----\n[![Documentation Status](https://readthedocs.org/projects/hiq/badge/?version=latest)](https://hiq.readthedocs.io/en/latest/?badge=latest)\n[![CodeCov][cov-img]][cov]\n[![Github release][release-img]][release]\n[![lic][license-img]][license]\n[![arXiv](https://img.shields.io/badge/arXiv-2304.13302-red.svg)](https://arxiv.org/abs/2304.13302)\n\n\u003e 🔥 HiQ now supports GPU profiling, DNN model visualization and tracing for DNN libraries like pyTorch, `transformers`, LAVIS, and LLMs like LLaMA, OPT, Bloom, T5 and GPT2 in addition to Onnxruntime, FastAPI and Flask.\n\nHiQ is a `declarative`, `non-intrusive`, `dynamic` and `transparent` tracking system for both **monolithic** application and **distributed** system. It brings the runtime information tracking and optimization to a new level without compromising with speed and system performance, or hiding any tracking overhead information. HiQ applies for both I/O bound and CPU bound applications. In addition to latency tracking, HiQ provides memory, disk I/O and Network I/O tracking out of the box. The output can be saved in form of normal line by line log file, or HiQ tree, or span graph.\n\nHiQ's philosophy is to **decouple `observability logic` from `business logic`**. We don't have to enter the black hole to observe it. Do you like the idea? Leave a ⭐ if you enjoy the project and welcome to say Hi to us on [Slack 👋](https://join.slack.com/t/hiq-myo2317/shared_invite/zt-17ejh6ybo-51IX6G1lHMXgLbq2HKIO_Q)\n\n[📜HiQ Pape: A Declarative, Non-intrusive, Dynamic and Transparent Observability and Optimization System](https://arxiv.org/abs/2304.13302)\n\n![Observability of DNN Model](https://raw.githubusercontent.com/henrywoo/hiq/main/hiq/docs/medium/all.png)\n\n## Installation\n\n- Basic Installation\n\n```bash\npip install hiq-python\n```\n\n- HiQ also supports extra installation\n\n```bash\npip install hiq-python[fastapi] # To support fastapi web server online tracing\npip install hiq-python[gpu]     # To support GPU tracing, which will install pynvml\npip install hiq-python[lavis]   # To support Salesforce LAVIS Vision Language models\npip install hiq-python[transformers] # To support tracing Hugging Face's transformers library\npip install hiq-python[full]         # To support all the cases, and this will install all the dependency libraries\n```\n\n## Get Started\n\nLet start with a simplest example by running HiQ against a simple monolithic python code [📄 `main.py`](hiq/examples/quick_start/main.py):\n\n```python\n# this is the main.py python source code\nimport time\n\ndef func1():\n    time.sleep(1.5)\n    print(\"func1\")\n    func2()\n\ndef func2():\n    time.sleep(2.5)\n    print(\"func2\")\n\ndef main():\n    func1()\n\nif __name__ == \"__main__\":\n    main()\n```\n\nIn this code, there is a simple chain of function calls: `main()` -\u003e `func1` -\u003e `func2`.\n\nNow we want to trace the functions without modifying its code. Let's run the following:\n\n\n```python\ngit clone https://github.com/oracle-samples/hiq.git\ncd hiq/examples/quick_start\npython main_driver.py\n```\n\nIf everything is fine, you should be able to see the output like this:\n\n![HiQ Simplest Example](https://github.com/oracle/hiq/raw/main/hiq/docs/source/img/main_driver.jpg)\n\nFrom the screenshot we can see the timestamp and the latency of each function:\n\n\n|   | main  | func1  |  func2 |  tracing overhead |\n|---|---|---|---|---|\n| latency(second)  | 4.0045  | 4.0044  | 2.5026  | 0.0000163  |\n\n\nHiQ just traced the `main.py` file running without touching one line of its code.\n\n## Documentation\n\n**HTML**: [🔗 HiQ Online Documents](https://hiq.readthedocs.io/en/latest/index.html)  | **PDF**: Please check [🔗 HiQ User Guide](https://github.com/oracle/hiq/blob/main/hiq/docs/hiq.pdf).\n\n----\n\nLogging: https://hiq.readthedocs.io/en/latest/4_o_advanced.html#log-monkey-king  \nTracing: https://hiq.readthedocs.io/en/latest/5_distributed.html  \n- Zipkin: https://hiq.readthedocs.io/en/latest/5_distributed.html#zipkin   \n- Jaeger: https://hiq.readthedocs.io/en/latest/5_distributed.html#jaeger   \n\nMetrics:  \n- Prometheus: https://hiq.readthedocs.io/en/latest/7_integration.html#prometheus  \n\nStreaming:  \n- Kafka: https://hiq.readthedocs.io/en/latest/7_integration.html#oci-streaming \n\n## DNN Model Observability \u0026 Visualization\n\nHiQ can visualize DNN model. To get the following BERT model's structure, you can just run:\n\n```\npython -m hiq.vis\n```\n\n![BERT](https://raw.githubusercontent.com/henrywoo/hiq/main/hiq/docs/medium/vis_bert.png)\n\nThe graph is self-explantory. There are several conventions:\n\n- ❄️ means frozen layer, where `requires_grad` is false.\n- 📈 means gradient exists for that model parameter, which usually happens after backpopulation.\n- `+`, bold font, and underscored dotted line mean the displayed layer is a folded version of multiple layers with the same structure.\n\nWhat you need to do is just calling `print_model(model)` in your code. Refer to: [here](https://github.com/henrywoo/hiq/tree/main/hiq/examples/vis) for how to use it.\n\n## HiQ Web UI\n\n- Main Page\n\n![HiQ UI Main Page](https://github.com/oracle/hiq/raw/main/hiq/docs/source/img/hiq-ui-1.png)\n\n- Latency Details\n\n![HiQ UI Latency Details](https://github.com/oracle/hiq/raw/main/hiq/docs/source/img/hiq-ui-2.png)\n\n## Jupyter NoteBook\n\nHiQ was originally developed to find Onnxruntime performance bottleneck in DNN inference, and it works well for other computation intensive applications too. The following are two examples.\n\n### Add Observability to PaddlePaddle (PaddleOCR)\n\n- [Latency](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/paddle/demo.ipynb)\n- [Memory](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/paddle/demo_memory.ipynb)\n\n- Latency Gantt Chart\n\n![Latency Gantt Chart](https://raw.githubusercontent.com/oracle/hiq/main/hiq/docs/medium/hiq-gantt.png)\n\n- HiQ Call Graph\n\n![HiQ Call Graph](https://raw.githubusercontent.com/oracle/hiq/main/hiq/docs/medium/hiq-call-graph.png)\n\n### Add Observability to Onnxruntime (AlexNet)\n\n- [Latency](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/onnxruntime/demo.ipynb)\n- [Intrusive](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/onnxruntime/demo_intrusive.ipynb)\n\n## Examples\n\nPlease check [🔗 examples](https://github.com/oracle/hiq/blob/main/hiq/examples) for usage examples.\n\n## Contributing\n\nHiQ welcomes contributions from the community. Before submitting a pull request, please review our contribution guide](./CONTRIBUTING.md).\n\n## Security\n\nPlease consult the [🔗 security guide](https://github.com/oracle/hiq/blob/main/SECURITY.md) for our responsible security vulnerability disclosure process.\n\n## License\n\nCopyright (c) 2022, 2023 Oracle and/or its affiliates. Released under the Universal Permissive License v1.0 as shown at \u003chttps://oss.oracle.com/licenses/upl/\u003e.\n\n## Presentation and Demos\n\n- [Introduction to Observability with HiQ](https://github.com/oracle-samples/hiq/blob/main/hiq/docs/Introduction-To-Observability-With-HiQ.pdf)\n\n[cov-img]: https://codecov.io/gh/uber/athenadriver/branch/master/graph/badge.svg\n[cov]: https://hiq.readthedocs.io/en/latest/index.html\n\n[release-img]: https://img.shields.io/badge/release-v1.1.13-red\n[release]: https://github.com/oracle-samples/hiq\n\n[license-img]: https://img.shields.io/badge/License-UPL--1.0-red\n[license]: https://github.com/oracle-samples/hiq/blob/main/LICENSE.txt\n\n[release-policy]: https://golang.org/doc/devel/release.html#policy\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foracle%2Fhiq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foracle%2Fhiq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foracle%2Fhiq/lists"}