{"id":13628474,"url":"https://github.com/awslabs/mxboard","last_synced_at":"2025-04-17T04:31:49.866Z","repository":{"id":31291156,"uuid":"120534294","full_name":"awslabs/mxboard","owner":"awslabs","description":"Logging MXNet data for visualization in TensorBoard.","archived":true,"fork":false,"pushed_at":"2021-11-30T10:46:24.000Z","size":111,"stargazers_count":324,"open_issues_count":15,"forks_count":47,"subscribers_count":24,"default_branch":"master","last_synced_at":"2025-03-13T02:37:53.425Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/awslabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-02-06T23:03:51.000Z","updated_at":"2025-02-22T15:28:49.000Z","dependencies_parsed_at":"2022-08-07T16:15:43.443Z","dependency_job_id":null,"html_url":"https://github.com/awslabs/mxboard","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awslabs%2Fmxboard","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awslabs%2Fmxboard/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awslabs%2Fmxboard/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awslabs%2Fmxboard/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/awslabs","download_url":"https://codeload.github.com/awslabs/mxboard/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249315941,"owners_count":21249860,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T22:00:52.749Z","updated_at":"2025-04-17T04:31:49.488Z","avatar_url":"https://github.com/awslabs.png","language":"Python","readme":"# Logging MXNet Data for Visualization in TensorBoard\n\n\n## Overview\n\nMXBoard provides a set of APIs for logging\n[MXNet](http://mxnet.incubator.apache.org/) data for visualization in\n[TensorBoard](https://www.tensorflow.org/programmers_guide/summaries_and_tensorboard).\nThe idea of this project comes from discussions with [Zihao Zheng](https://github.com/zihaolucky),\nthe author of\n[dmlc/tensorboard](https://github.com/dmlc/tensorboard),\non delivering a visualization solution for MXNet users.\nWe aim at providing the logging APIs that can process MXNet data efficiently\nand supporting most of the data types for visualization in the TensorBoard GUI.\nWe adapted the following low-level logging components from their Python and C++\nimplementations in [TensorFlow](https://github.com/tensorflow/tensorflow): `FileWriter`, `EventFileWriter`,\n`EventsWriter`, `RecordWriter`,\nand `_EventLoggerThread`. We also adapted the user-level logging APIs defined in `SummaryWriter` from\n[tensorboard-pytorch](https://github.com/lanpa/tensorboard-pytorch).\nThe encoding algorithm used in writing protobuf objects into event files\nis directly borrowed from\n[TeamHG-Memex/tensorboard_logger](https://github.com/TeamHG-Memex/tensorboard_logger).\n\nMXBoard supports a set of Python APIs for logging the following data types\nfor TensorBoard to render. Logging APIs for other languages may be added in the future.\n\n![mxboard_cover](https://github.com/dmlc/web-data/blob/master/mxnet/tensorboard/mxboard_cover.png)\n\nThe corresponding Python APIs are accessible through a class called `SummaryWriter` as follows:\n\n```python\n    mxboard.SummaryWriter.add_graph\n    mxboard.SummaryWriter.add_scalar\n    mxboard.SummaryWriter.add_histogram\n    mxboard.SummaryWriter.add_embedding\n    mxboard.SummaryWriter.add_image\n    mxboard.SummaryWriter.add_text\n    mxboard.SummaryWriter.add_pr_curve\n    mxboard.SummaryWriter.add_audio\n```\n\n\n## Installation\n\n### Install MXBoard from PyPI\n\n```bash\npip install mxboard\n```\n\n### Install MXBoard Python package from source\n\n```bash\ngit clone https://github.com/awslabs/mxboard.git\ncd mxboard/python\npython setup.py install\n```\n\n\n### Install TensorBoard from PyPI\nMXBoard is a logger for writing MXNet data to event files.\nTo visualize those data in browsers, users still have to install\n[TensorBoard](https://www.tensorflow.org/versions/r1.1/get_started/summaries_and_tensorboard)\nseparately.\n\n```bash\npip install tensorboard\n```\n\nUse the following to verify that the TensorBoard binary has been installed correctly.\n\n```bash\ntensorboard --help\n```\n\n\n### Other required packages\nMXBoard relies on the following packages for data logging.\n- [MXNet](https://pypi.python.org/pypi/mxnet)\n- [protobuf3](https://pypi.python.org/pypi/protobuf)\n- [six](https://pypi.python.org/pypi/six)\n- [Pillow](https://pypi.python.org/pypi/Pillow)\n\nPlease note that you need to install MXNet manually before using MXBoard.\nThe other packages will be installed automatically when you install MXBoard\nvia pip or building from source. If you want to build from source, please make sure\nthat protobuf compiler is installed. Check [this page](https://github.com/google/protobuf/releases)\nfor downloading the protobuf compiler whose file name starts with \"protoc\".\n\n\n## Visualizing MXNet data in 30 seconds\nNow that you have installed all of the required packages, let's walk through a simple visualization example. You will see how\nMXBoard enables visualizing MXNet `NDArray`s with histograms.\n\n**Step 1.** Logging event data to a file.\n\nPrepare a Python script for writing data generated by the `normal` operator to an event file.\nThe data is generated ten times with decreasing standard deviation and written to the event\nfile each time. It's expected to see the data distribution gradually become more centered around\nthe mean value. Note that here we specify creating the event file in the folder `logs`\nunder the current directory. We will need to pass this folder path to the TensorBoard binary.\n```python\nimport mxnet as mx\nfrom mxboard import SummaryWriter\n\n\nwith SummaryWriter(logdir='./logs') as sw:\n    for i in range(10):\n        # create a normal distribution with fixed mean and decreasing std\n        data = mx.nd.normal(loc=0, scale=10.0/(i+1), shape=(10, 3, 8, 8))\n        sw.add_histogram(tag='norml_dist', values=data, bins=200, global_step=i)\n```\n\n**Step 2.** Launch TensorBoard to load the event file generated above.\n\nUse the following command to start the TensorBoard server. It will use the logs that were generated in the current directory's `logs` folder.\n```bash\ntensorboard --logdir=./logs --host=127.0.0.1 --port=8888\n```\nNote that in some situations,\nthe port number `8888` may be occupied by other applications and launching TensorBoard\nmay fail. You may choose a different available port number.\n\n**Step 3.** Open TensorBoard in your browser.\n\nIn the browser, enter the address `127.0.0.1:8888`, and click the tab **HISTOGRAMS**\nin the TensorBoard GUI. You will see data distribution changing as time progresses.\n\n![summary_histogram_norm](https://github.com/dmlc/web-data/blob/master/mxnet/tensorboard/doc/summary_histogram_norm.png)\n\n## More tutorials\n- [Quick start for logging data of various types](https://github.com/awslabs/mxboard/blob/master/demo.md)\n- [Monitoring training an MNIST model with MXBoard](https://github.com/awslabs/mxboard/blob/master/examples/mnist)\n- [Visualizing filters of ConvNets](https://github.com/reminisce/mxboard-demo#visualizing-filters-of-convnets)\n- [Visualizing ConvNet codes as embeddings](https://github.com/reminisce/mxboard-demo#visualizing-convnet-codes-as-embeddings)\n\n## References\n1. https://github.com/TeamHG-Memex/tensorboard_logger\n2. https://github.com/lanpa/tensorboard-pytorch\n3. https://github.com/dmlc/tensorboard\n4. https://github.com/tensorflow/tensorflow\n5. https://github.com/tensorflow/tensorboard\n\n## License\nThis library is licensed under the Apache 2.0 License.\n","funding_links":[],"categories":["Model Explanation","\u003ca name=\"Tools\"\u003e\u003c/a\u003e13. Tools","工作流程和实验跟踪","Python","\u003ca name=\"Tools\"\u003e\u003c/a\u003e9. Tools"],"sub_categories":["NLP","13.3 Visualization"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fawslabs%2Fmxboard","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fawslabs%2Fmxboard","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fawslabs%2Fmxboard/lists"}