{"id":13752110,"url":"https://github.com/linuxerwang/goroutine-inspect","last_synced_at":"2025-04-06T11:07:28.192Z","repository":{"id":42792214,"uuid":"96541335","full_name":"linuxerwang/goroutine-inspect","owner":"linuxerwang","description":"An interactive tool to analyze Golang goroutine dump.","archived":false,"fork":false,"pushed_at":"2021-12-06T15:48:46.000Z","size":21,"stargazers_count":467,"open_issues_count":1,"forks_count":18,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-03-30T10:06:29.908Z","etag":null,"topics":["analyze","dump","golang","goroutine","inspect"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/linuxerwang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-07-07T13:31:03.000Z","updated_at":"2025-03-14T20:18:03.000Z","dependencies_parsed_at":"2022-09-01T16:04:04.027Z","dependency_job_id":null,"html_url":"https://github.com/linuxerwang/goroutine-inspect","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linuxerwang%2Fgoroutine-inspect","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linuxerwang%2Fgoroutine-inspect/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linuxerwang%2Fgoroutine-inspect/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linuxerwang%2Fgoroutine-inspect/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/linuxerwang","download_url":"https://codeload.github.com/linuxerwang/goroutine-inspect/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247471517,"owners_count":20944158,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analyze","dump","golang","goroutine","inspect"],"created_at":"2024-08-03T09:00:59.604Z","updated_at":"2025-04-06T11:07:28.164Z","avatar_url":"https://github.com/linuxerwang.png","language":"Go","readme":"# goroutine-inspect\n\nAn interactive tool to analyze Golang goroutine dump.\n\n## Build and Run\n\n```bash\ngo get github.com/linuxerwang/goroutine-inspect\n$GOPATH/bin/goroutine-inspect\n```\n\n## Workspace\n\nWorkspace is the place to hold imported goroutine dumps. Instructions are\nprovided to maintain these dumps.\n\nIn the interactive shell, two kinds of instructions can be issued: commands\nand statements.\n\n## Commands\n\nAt present, the following commands are supported.\n\n| command | function                          |\n| ------- | --------------------------------- |\n| cd      | Change current working directory. |\n| clear   | Clear the workspace.              |\n| exit    | Exit the interactive shell.       |\n| help    | Show help.                        |\n| ls      | Show files in current directory.  |\n| pwd     | Show present working directory.   |\n| quit    | Quit the interactive shell.       |\n| whos    | Show all varaibles in workspace.  |\n\n## Statements\n\n### Load Goroutine Dump From Files\n\nLoad the dump and assign to a variable:\n\n```bash\n\u003e\u003e original = load(\"pprof-goroutines-20170510-170245.dump\")\n\u003e\u003e whos\noriginal\n```\n\n### Show the Summary of a Dump Var\n\nSimply type the variable name:\n\n```bash\n\u003e\u003e original\n# of goroutines: 2217\n\n        running: 1\n        IO wait: 533\n        syscall: 2\n   chan receive: 50\n         select: 1504\n       runnable: 38\n     semacquire: 85\n      chan send: 4\n\n```\n\n### Copy a Dump Var\n\nTo copy the whole dump, simply assign it to a different var:\n\n```bash\n\u003e\u003e copy1 = original\n\u003e\u003e whos\ncopy        original\n```\n\nIt's equivalent to using a copy() function:\n\n```bash\n\u003e\u003e copy2 = original.copy()\n\u003e\u003e whos\ncopy        copy1        copy2        original\n```\n\nThe copy() function allows passing a conditional so that only those meeting\nthe cariteria will be copied:\n\n```bash\n\u003e\u003e copy3 = original.copy(\"id\u003e900 \u0026\u0026 id\u003c2000\")\n```\n\n### Modify the Dump Goroutine Items\n\nFunction delete() accepts a conditional to delete goroutine items in a dump\nvar. Function keep() do the reversed conditional.\n\n```bash\n\u003e\u003e copy\n# of goroutines: 2217\n\n        running: 1\n        IO wait: 533\n        syscall: 2\n   chan receive: 50\n         select: 1504\n       runnable: 38\n     semacquire: 85\n      chan send: 4\n\n\u003e\u003e copy.delete(\"id\u003e100 \u0026\u0026 id\u003c1000\")\nDeleted 118 goroutines, kept 2099.\n\u003e\u003e copy.keep(\"id\u003e200\")\nDeleted 12 goroutines, kept 2087.\n\u003e\u003e copy\n# of goroutines: 2087\n\n        running: 1\n         select: 1411\n        IO wait: 500\n     semacquire: 85\n       runnable: 37\n   chan receive: 49\n      chan send: 4\n\n```\n\n### Display Goroutine Dump Items\n\nFunction show() displays goroutine dump items with optional offset and limit.\nThe default offset is 0, and default limit is 10.\n\n```bash\n\u003e\u003e original.show() # offset 0, limit 10\n\ngoroutine 1803 [select, 10 minutes]:\ngoogle.golang.org/grpc/transport.(*http2Server).keepalive(0xc420e59ce0)\n        google.golang.org/grpc/transport/http2_server.go:919 +0x488\ncreated by google.golang.org/grpc/transport.newHTTP2Server\n        google.golang.org/grpc/transport/http2_server.go:226 +0x97c\n\n...\n...\n\n\u003e\u003e original.show(15) # offset 15, limit 10\n\ngoroutine 6455709 [running]:\nruntime/pprof.writeGoroutineStacks(0xe9a080, 0xc4216f0088, 0x1d, 0x40)\n        go1.8.1.linux-amd64/src/runtime/pprof/pprof.go:603 +0x79\nruntime/pprof.writeGoroutine(0xe9a080, 0xc4216f0088, 0x2, 0x1d, 0xc4217cede0)\n        go1.8.1.linux-amd64/src/runtime/pprof/pprof.go:592 +0x44\nruntime/pprof.(*Profile).WriteTo(0xed3780, 0xe9a080, 0xc4216f0088, 0x2, 0xc4217cef80, 0x1)\n        go1.8.1.linux-amd64/src/runtime/pprof/pprof.go:302 +0x3b5\nwww.test.com/bagel/runtime.dumpToFile(0xed0f0ba5e, 0xae05027, 0xee1780, 0xc425bd2060, 0x5, 0x5)\n        www.test.com/bagel/runtime/dump.go:58 +0x3f3\ncreated by www.test.com/bagel/runtime.EnableGoroutineDump.func1\n        www.test.com/bagel/runtime/dump.go:30 +0x2d6\n\n...\n...\n\n\u003e\u003e original.show(15, 1) # offset 15, limit 1\n\ngoroutine 6455709 [running]:\nruntime/pprof.writeGoroutineStacks(0xe9a080, 0xc4216f0088, 0x1d, 0x40)\n        go1.8.1.linux-amd64/src/runtime/pprof/pprof.go:603 +0x79\nruntime/pprof.writeGoroutine(0xe9a080, 0xc4216f0088, 0x2, 0x1d, 0xc4217cede0)\n        go1.8.1.linux-amd64/src/runtime/pprof/pprof.go:592 +0x44\nruntime/pprof.(*Profile).WriteTo(0xed3780, 0xe9a080, 0xc4216f0088, 0x2, 0xc4217cef80, 0x1)\n        go1.8.1.linux-amd64/src/runtime/pprof/pprof.go:302 +0x3b5\nwww.test.com/bagel/runtime.dumpToFile(0xed0f0ba5e, 0xae05027, 0xee1780, 0xc425bd2060, 0x5, 0x5)\n        www.test.com/bagel/runtime/dump.go:58 +0x3f3\ncreated by www.test.com/bagel/runtime.EnableGoroutineDump.func1\n        www.test.com/bagel/runtime/dump.go:30 +0x2d6\n```\n\n### Search Goroutine Dump Items\n\nSimilar to show(), but with a conditional to only show items meeting certain\ncriteria:\n\n```bash\n\u003e\u003e original.search(\"id \u003c 2000\", 15, 1) # offset 15, limit 1\n\ngoroutine 6455896 [select]:\nnet.(*netFD).connect.func2(0xea1980, 0xc424bca540, 0xc422c1af50, 0xc424bca600, 0xc424bca5a0)\n        go1.8.1.linux-amd64/src/net/fd_unix.go:133 +0x1d5\ncreated by net.(*netFD).connect\n        go1.8.1.linux-amd64/src/net/fd_unix.go:144 +0x239\n```\n\nOne useful ability is to filter goroutines by running time:\n\n```bash\n\u003e\u003e original.search(\"duration \u003e 10\") # duration larger than 10 minutes\n\ngoroutine 72 [select, 25 minutes]: 119 times: [72, 54755, 76757, 299, 201, 286, 283, 296, 204, 302, 207, 305, 338, 356, 359, 362, 365, 372, 375, 368, 378, 328, 331, 387, 381\n, 390, 384, 403, 393, 334, 406, 396, 399, 337, 418, 341, 436, 344, 439, 421, 424, 409, 427, 452, 430, 433, 442, 455, 445, 458, 448, 461, 464, 468, 483, 471, 499, 486, 502, 5\n05, 489, 76462, 76773, 54530, 54572, 55194, 54824, 54481, 42719, 54691, 54859, 55023, 75593, 76750, 55202, 54885, 79006, 54468, 55212, 54473, 54462, 54931, 54864, 55133, 550\n97, 54882, 54901, 55209, 54499, 55114, 54564, 76653, 54416, 54527, 75588, 55034, 54868, 54791, 54813, 54698, 54579, 55111, 54443, 54486, 76467, 54654, 54537, 54456, 55126, 5\n5117, 54622, 55199, 54556, 54477, 54871, 79498, 76601, 76735, 76996]\ngoogle.golang.org/grpc/transport.(*http2Server).keepalive(0xc4202f0420)\n        google.golang.org/grpc/transport/http2_server.go:919 +0x488\ncreated by google.golang.org/grpc/transport.newHTTP2Server\n        google.golang.org/grpc/transport/http2_server.go:226 +0x97c\n```\n\nNote that the above is after a dedup operation, so it shows the same stack trace\nexisting 119 times. See the \"Dedup goroutines\" section.\n\n### Diff Two Goroutine Dumps\n\n```bash\n\u003e\u003e l, c, r = x.diff(y)\n\u003e\u003e l\n# of goroutines: 574\n\n        IO wait: 147\n   chan receive: 1\n       runnable: 3\n         select: 421\n        syscall: 2\n\n\u003e\u003e c\n# of goroutines: 651\n\n        IO wait: 157\n       runnable: 4\n         select: 489\n     semacquire: 1\n\n\u003e\u003e r\n# of goroutines: 992\n\n        IO wait: 229\n   chan receive: 49\n      chan send: 4\n       runnable: 31\n        running: 1\n         select: 594\n     semacquire: 84\n```\n\nIt returns three values: the dump var containing goroutines only appear in\nx (the left side), the dump var containing goroutines appear in both x and y,\nthe dump var containing goroutines only appear in y (the right side).\n\n### Dedup goroutines\n\nNormally goroutine dump files contain thousands of goroutine entries, but\nthere are many duplicated traces. Function dedup() helps to identify these\nduplicated traces by comparing the trace lines, and only keep one copy of\nthem. It greatly reduces the information explosion and make developers much\neasier to focus on their problems.\n\n```bash\n\u003e\u003e a\n# of goroutines: 2217\n\n        IO wait: 533\n   chan receive: 50\n      chan send: 4\n       runnable: 38\n        running: 1\n         select: 1504\n     semacquire: 85\n        syscall: 2\n\n\u003e\u003e a.dedup()\nDedupped 2217, kept 46\n\u003e\u003e\n\u003e\u003e a\n# of goroutines: 46\n\n        IO wait: 6\n   chan receive: 2\n      chan send: 2\n       runnable: 18\n        running: 1\n         select: 12\n     semacquire: 3\n        syscall: 2\n```\n\nTo show goroutines with 5+ duplicates:\n\n```bash\n\u003e\u003e a.search(\"dups \u003e= 5\")\n   ...\n```\n\n### Save the Modified Goroutine Dump to a File\n\nAfter a dump var is modified, it can be saved to a file:\n\n```bash\n\u003e\u003e a.save(\"pprof-deduped.log\")\n```\n\n## Properties of a Goroutine Dump Item\n\nEach dump item has 5 properties which can be used in conditionals:\n\n| property | type    | meaning                                             |\n| -------- | ------- | --------------------------------------------------- |\n| id       | integer | The goroutine ID.                                   |\n| dups     | integer | The number of duplicate traces.                     |\n| duration | integer | The waiting duration (in minutes) of a goroutine.   |\n| lines    | integer | The number of lines of the goroutine's stack trace. |\n| state    | string  | The running state of the goroutine.                 |\n| trace    | string  | The concatenated text of the goroutine stack trace. |\n\n## Functions in Conditionals\n\nThe following functions can be used in defining conditionals:\n\n| function | args           | return value | meaning                                               |\n| -------- | -------------- | ------------ | ----------------------------------------------------- |\n| contains | string, string | bool         | Returns true if the first arg contains the second arg |\n| lower    | string         | string       | Returns the lowercased string of the input.           |\n| upper    | string         | string       | Returns the uppercased string of the input.           |\n\nExample:\n\n```bash\n\u003e\u003e original.search(\"contains(lower(trace), 'handlestream')\")\n```\n","funding_links":[],"categories":["Go","Programming Languages"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinuxerwang%2Fgoroutine-inspect","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flinuxerwang%2Fgoroutine-inspect","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinuxerwang%2Fgoroutine-inspect/lists"}