{"id":19900096,"url":"https://github.com/m-lab/traceroute-caller","last_synced_at":"2025-05-02T23:30:29.607Z","repository":{"id":34410863,"uuid":"174017867","full_name":"m-lab/traceroute-caller","owner":"m-lab","description":"A sidecar service which runs traceroute after a connection closes","archived":false,"fork":false,"pushed_at":"2024-10-16T21:14:50.000Z","size":41985,"stargazers_count":18,"open_issues_count":24,"forks_count":5,"subscribers_count":13,"default_branch":"main","last_synced_at":"2024-10-29T14:50:09.231Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/m-lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-03-05T20:44:21.000Z","updated_at":"2024-10-16T21:14:03.000Z","dependencies_parsed_at":"2024-11-06T16:35:13.652Z","dependency_job_id":"463471c4-a3be-4cdf-8f20-be6d334af1e5","html_url":"https://github.com/m-lab/traceroute-caller","commit_stats":{"total_commits":240,"total_committers":14,"mean_commits":"17.142857142857142","dds":0.5041666666666667,"last_synced_commit":"fd359ff60229676e04bed52b1a71b8e6b0b72cbd"},"previous_names":[],"tags_count":37,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m-lab%2Ftraceroute-caller","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m-lab%2Ftraceroute-caller/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m-lab%2Ftraceroute-caller/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m-lab%2Ftraceroute-caller/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/m-lab","download_url":"https://codeload.github.com/m-lab/traceroute-caller/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224003270,"owners_count":17239426,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T20:11:09.412Z","updated_at":"2024-11-12T20:11:10.066Z","avatar_url":"https://github.com/m-lab.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# traceroute-caller\n[![Version](https://img.shields.io/github/tag/m-lab/traceroute-caller.svg)](https://github.com/m-lab/traceroute-caller/releases) [![Build Status](https://travis-ci.com/m-lab/traceroute-caller.svg?branch=master)](https://travis-ci.com/m-lab/traceroute-caller) [![Coverage Status](https://coveralls.io/repos/m-lab/traceroute-caller/badge.svg?branch=master)](https://coveralls.io/github/m-lab/traceroute-caller?branch=master) [![GoDoc](https://godoc.org/github.com/m-lab/traceroute-caller?status.svg)](https://godoc.org/github.com/m-lab/traceroute-caller) [![Go Report Card](https://goreportcard.com/badge/github.com/m-lab/traceroute-caller)](https://goreportcard.com/report/github.com/m-lab/traceroute-caller)\n\n## Local Development\n\nUsing `docker-compose` you can run a local instance of traceroute-caller that\noperates in concert with events from `measurementlab/tcpinfo` and using\nannotation from `measurement-lab/uuid-annotator`.\n\nYou must have a recent version of the Docker server configured and running in\nyour local environment. As well, your local environment must include a recent\nversion of `docker-compose`.\n\n```sh\n$ docker-compose version\ndocker-compose version 1.27.4, build 40524192\ndocker-py version: 4.3.1\nCPython version: 3.7.7\nOpenSSL version: OpenSSL 1.1.1g  21 Apr 2020\n```\n\nIn the root directory of traceroute-caller, start a local build using\nsample files in `./testdata`.\n\n```sh\ndocker-compose up\n```\n\nThis will create and run three containers.  Container names are prefixed by the\ncurrent working directory name (i.e., traceroute-caller).  After the containers\nare running, trigger a network connection from within one of those containers.\nFor example:\n\n```sh\ndocker exec -it traceroute-caller_traceroute-caller_1 apt-get update\n```\n\nThe logs from traceroute-caller should indicate that files are being saved\nunder `./local/*`.\n\n```sh\nls -lR ./local\n```\n\nUse `docker-compose down` to stop the containers and remove resources before\nrestarting your docker-compose environment.\n\n```sh\ndocker-compose down\ndocker-compose up\n```\n\n## Traceroute Examiner Tool: trex\n\nThe `trex` command line tool in this repo can examine `scamper` MDA\ntraceroutes that are in `.jsonl` format and do the following:\n\n1. Extract single-path traceroutes from an MDA traceroute.\n2. List traceroutes that took longer than a specified duration.\n3. List complete and incomplete traceroutes.\n\n\nNote:\n* Not all traceroutes are complete.  That is, not all traceroutes\ntrace all the way to the destination IP address.\n* Different hops associated with the same flow ID constitute a single path.\n* The order of hops in a path is determined by the TTL.\n* Unresponsive hops are marked as an asterisk (\"*\").\n* It is possible for a hop to return multiple replies to a probe.\nTherefore, for the same flow ID and TTL, there may be zero, one, or more\nthan one replies.\n* When showing single-paths, only complete paths (if any) are printed.\n* If you need to see all paths, use the \"-v\" flag to enable the verbose\nmode.\n\nThe easiest way to get started with `trex` is to first fetch an archive\nof M-Lab's MDA traceroutes to examine.  This can be done as shown below:\n\n```sh\n$ mkdir ~/traceroutes\n$ cd ~/traceroutes\n$ gsutil cp gs://archive-measurement-lab/ndt/scamper1/2021/10/01/20211001T003000.005106Z-scamper1-mlab1-lis02-ndt.tgz .\n$ tar xzf 20211001T003000.005106Z-scamper1-mlab1-lis02-ndt.tgz\n```\n\nThe above command extracts individual traceroute files to a directory\ncalled `2021`.  Now build the `trex` tool as shown below:\n\n```sh\n$ git clone https://github.com/m-lab/traceroute-caller\n$ cd traceroute-caller/cmd/trex\n$ go build\n```\n\nThe above command builds `trex` and now you can use it to examine the\ntraceroute files that you extracted.  If `trex` examines more than\none file, it prints statistics on how many files were found, how many\nwere skipped because they were not `.jsonl` files, how many errors, etc.\n\n\n```sh\n# Show usage message.\n$ ./trex -h\nUsage: ./trex [-cehv] [-d \u003cseconds\u003e] path [path...]\npath  a pathname to a file or directory (if directory, all files are processed recursively)\n-h    print usage message and exit\n-c    print flow IDs and file names of traceroutes that completed (\"--\" for incomplete traceroutes)\n-d    print times and file names of traceroutes that took more than the specified duration\n-e    print examples how to use this tool and exit\n-v    enable verbose mode (mostly for debugging)\n\n# Show examples.\nExamples:\n# Extract and print a single-path traceroute (if it exists) from a traceroute file\n$ trex /traceroutes/2022/04/01/20220401T001905Z_ndt-qqvlt_1647967485_000000000009379D.jsonl\n\nfile: /traceroutes/2022/04/01/20220401T001905Z_ndt-qqvlt_1647967485_000000000009379D.jsonl\nsrc: 209.170.110.216\ndst: 199.19.248.6\nscamper start: 1648772345\ntracelb start: 1648772345 (0 seconds after scamper start)\nscamper stop:  1648772346 (1 seconds after scamper start)\nflowid: 1\nTTL    TX(ms)   RX(ms)    RTT(ms)  IP address\n  1       N/A      N/A      0.000  209.170.110.193\n  2       150      151      0.653  213.248.100.57\n  3      1055     1062      7.244  199.19.248.6  \u003c=== destination\n\nThe TX and RX columns are elapsed transmit and receive times since the tracelb\ncommand was started.\n\n\n# Same command as above but enable the verbose mode (useful for debugging).\n$ trex -v /traceroutes/2022/04/01/20220401T001905Z_ndt-qqvlt_1647967485_000000000009379D.jsonl\n\n/traceroutes/2022/04/01/20220401T001905Z_ndt-qqvlt_1647967485_000000000009379D.jsonl\nTracelb.Src: 209.170.110.216\nTracelb.Dst: 199.19.248.6\nTracelb.Nodes[0] 209.170.110.193\n  Tracelb.Nodes[0].Links[0][0] 213.248.100.57\n    Tracelb.Nodes[0].Links[0][0].Probes[0].Flowid: 1\n    Tracelb.Nodes[0].Links[0][0].Probes[1].Flowid: 2\n    Tracelb.Nodes[0].Links[0][0].Probes[2].Flowid: 3\n    Tracelb.Nodes[0].Links[0][0].Probes[3].Flowid: 4\n    Tracelb.Nodes[0].Links[0][0].Probes[4].Flowid: 5\n    Tracelb.Nodes[0].Links[0][0].Probes[5].Flowid: 6\nTracelb.Nodes[1] 213.248.100.57\n  Tracelb.Nodes[1].Links[0][0] 199.19.248.6\n    Tracelb.Nodes[1].Links[0][0].Probes[0].Flowid: 1\n\nfile: /traceroutes/2022/04/01/20220401T001905Z_ndt-qqvlt_1647967485_000000000009379D.jsonl\nsrc: 209.170.110.216\ndst: 199.19.248.6\nscamper start: 1648772345\ntracelb start: 1648772345 (0 seconds after scamper start)\nscamper stop:  1648772346 (1 seconds after scamper start)\nflowid: 1\nTTL    TX(ms)   RX(ms)    RTT(ms)  IP address\n  1       N/A      N/A      0.000  209.170.110.193\n  2       150      151      0.653  213.248.100.57\n  3      1055     1062      7.244  199.19.248.6  \u003c=== destination\n\nflowid: 2\nTTL    TX(ms)   RX(ms)    RTT(ms)  IP address\n  1       N/A      N/A      0.000  209.170.110.193\n  2       301      302      0.644  213.248.100.57\n\nflowid: 3\nTTL    TX(ms)   RX(ms)    RTT(ms)  IP address\n  1       N/A      N/A      0.000  209.170.110.193\n  2       452      453      0.707  213.248.100.57\n\nflowid: 4\nTTL    TX(ms)   RX(ms)    RTT(ms)  IP address\n  1       N/A      N/A      0.000  209.170.110.193\n  2       603      604      0.608  213.248.100.57\n\nflowid: 5\nTTL    TX(ms)   RX(ms)    RTT(ms)  IP address\n  1       N/A      N/A      0.000  209.170.110.193\n  2       754      754      0.621  213.248.100.57\n\nflowid: 6\nTTL    TX(ms)   RX(ms)    RTT(ms)  IP address\n  1       N/A      N/A      0.000  209.170.110.193\n  2       904      905      0.673  213.248.100.57\n\n\n# Print all traceroute files in a directory hierarchy that took longer than 5 minutes\n$ trex -d 300 /traceroutes/2021\n 428 /traceroutes/2021/10/01/20211001T000053Z_ndt-292jb_1632518393_00000000000516D4.jsonl\n 386 /traceroutes/2021/10/01/20211001T000151Z_ndt-292jb_1632518393_000000000005160D.jsonl\n...\n\nfiles found:                          425\nfiles skipped (not .jsonl):             0\nfiles that could not be read:           0\nfiles that could not be parsed:         0\nfiles successfully parsed:            425\nfiles with no traceroute data:          0\n\nminimum duration:                       4 seconds\nmaximum duration:                     456 seconds\naverage duration:                     220 seconds\n\n\n# Print flow ID of complete traceroutes (\"--\" if incomplete) in a directory hierarchy\n$ ./trex -c /traceroutes/2021\n 1 /traceroutes/2021/10/01/20211001T000014Z_ndt-292jb_1632518393_00000000000516C8.jsonl\n 1 /traceroutes/2021/10/01/20211001T000015Z_ndt-292jb_1632518393_00000000000516C9.jsonl\n-- /traceroutes/2021/10/01/20211001T000023Z_ndt-292jb_1632518393_00000000000516C4.jsonl\n...\n\nfiles found:                          425\nfiles skipped (not .jsonl):             0\nfiles that could not be read:           0\nfiles that could not be parsed:         0\nfiles successfully parsed:            425\nfiles with no traceroute data:          0\nfiles with complete traceroutes:      149  (35%)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fm-lab%2Ftraceroute-caller","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fm-lab%2Ftraceroute-caller","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fm-lab%2Ftraceroute-caller/lists"}