{"id":28946281,"url":"https://github.com/amadeusitgroup/process-watcher","last_synced_at":"2026-03-05T09:02:57.322Z","repository":{"id":285736880,"uuid":"955939687","full_name":"AmadeusITGroup/process-watcher","owner":"AmadeusITGroup","description":"A tool to efficiently monitor and measure the peak memory usage of process trees to optimize build performance and resource allocation.","archived":false,"fork":false,"pushed_at":"2025-05-27T11:27:59.000Z","size":37,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-08-04T19:36:37.335Z","etag":null,"topics":["c","linux","memory-monitoring","monitoring","monitoring-tool","process-management","process-tree","resource-usage","resource-usage-tracking"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AmadeusITGroup.png","metadata":{"files":{"readme":"README.md","changelog":"ChangeLog","contributing":"CONTRIBUTING.md","funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-03-27T12:49:55.000Z","updated_at":"2025-05-27T11:28:03.000Z","dependencies_parsed_at":"2025-05-27T12:26:59.537Z","dependency_job_id":"12b17363-09e7-4473-a3a4-9d2ac4cac528","html_url":"https://github.com/AmadeusITGroup/process-watcher","commit_stats":null,"previous_names":["amadeusitgroup/process-watcher"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/AmadeusITGroup/process-watcher","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AmadeusITGroup%2Fprocess-watcher","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AmadeusITGroup%2Fprocess-watcher/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AmadeusITGroup%2Fprocess-watcher/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AmadeusITGroup%2Fprocess-watcher/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AmadeusITGroup","download_url":"https://codeload.github.com/AmadeusITGroup/process-watcher/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AmadeusITGroup%2Fprocess-watcher/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30117482,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-05T08:19:04.902Z","status":"ssl_error","status_checked_at":"2026-03-05T08:17:37.148Z","response_time":93,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c","linux","memory-monitoring","monitoring","monitoring-tool","process-management","process-tree","resource-usage","resource-usage-tracking"],"created_at":"2025-06-23T08:05:35.198Z","updated_at":"2026-03-05T09:02:57.311Z","avatar_url":"https://github.com/AmadeusITGroup.png","language":"C","readme":"Copyright (C) 2025 Amadeus s.a.s.\nSee the end of the file for license conditions.\n\nprocess-watcher\n===============\n\nMeasure the peak (envelope) memory consumption of trees of processes\nover a time window identified by PID.\n\nThe process-watcher is a tool that helps measure the memory usage of\nany process (and its subprocesses, recursively).  For example, it\ncan measure how much memory is needed to run \"make\".\n\nIn Amadeus, it is used by an internal build tool to report the memory\nenvelope used by each part of the build of a project, so that\ndevelopers can know and request the right-sized amount of memory for\ntheir needs, and tune the parallel options of their build to reduce\nthe build duration as much as possible without exceeding the available\nmemory (and avoid being killed by the Linux OOM killer).  Basically,\nour build tool spawns a process-watcher at startup and kills it at\nexit time; it queries it after each possibly-large subjob.\n\n    Buildtool    Make#1     Make#2\n        |      (and subprocesses thereof)\n        |\n        |--------\u003e|\n        |         |-\u003e|\n        |         |---\u003e|\n        |         |  | |\n        |------------------\u003e|\n        |         |  | |    |\n        |         |\u003c-|-|    |-\u003e|\n        |         |  |      |  |\n        |         |  |      |\u003c-|\n        |\u003c------------------|\n        |         |  |\n        |         |\u003c-|\n        |         |\n        |\u003c--------|\n\n    Question: how much memory was needed for make#1?  for make#2?\n    Incidently, what is the maximum parallelization we can do without\n    exceeding the limits of the system memory?\n\n    In the above case, our Buildtool can ask process-watcher about the\n    memory envelope of Make#1 and Make#2 independently.\n\nAs process-watcher identifies a process tree by the PID of the top\nprocess, it is suitable to identify a particular process tree and not\nsomething like \"all processes that are named gcc on the system\", which\nwould be problematic if several similar jobs may be running at the\nsame time.\n\nprocess-watcher is designed to have a very small disk footprint,\nmemory footprint and cpu footprint, which makes its installation and\nuse unobtrusive even in very constrained or saturated environments.\n\nHow process-watcher works\n=========================\n\nThe \"process-watcher capture\" continuously monitors all processes of\nthe system (specifically by watching /proc), and records their parent\nPID, and their memory counters, every 2 seconds (the sampling rate).\nIt keeps the whole history of this information; on my desktop\ncomputer, each snapshot adds about 30 KB to the history file, but you\ncan reduce it by removing unwanted stats from the \"fields\" file and\nrecompiling.  It consumes very little memory and CPU (though it should\nbe possible to optimize it even more, see TODO.md).  It needs to be\nstopped (e.g. kill -TERM) when the monitoring is not needed anymore.\n\nIt also provides a query endpoint \"process-watcher get\".  This\nendpoint allows querying the maximum memory usage of a particular\nprocess tree over a particular time window (start and end time).  It\ncan be run while the capture is running.  On my laptop, it is able\nto process a 700+ MB history file in under 880 ms, so about 800 MB/s.\n\nLimitations\n===========\n\nNote that if a process starts and terminates within a sampling period,\nthen it will go undetected by process-watcher.  However, the main use\nof process-watcher is to find processes that consume a lot of memory,\nand those processes generally last much longer than the sampling\nperiod.\n\nSimilar works\n=============\n\nhttps://github.com/ncabatoff/process-exporter (a Prometheus exporter)\nis a Go program that regularly checks the statistics of processes and\nexports them to a Prometheus server.  It cannot measure a specific\nprocess tree if there are multiple process trees matching your\nmatchers, or measuring the memory envelope of a process tree, and\nprobably consumes more RAM than process-watcher, but is useful for\ngetting statistics over a long time.\n\nhttps://github.com/astrofrog/psrecord is a program that can collect\nmemory and CPU stats of a process tree identified by its top PID.  It\ndoes not compute the envelope memory statistics, and has a larger\noverall footprint than process-watcher as it currently relies on\nCPython.  However, it has features that process-tree does not have,\nsuch as the CPU stats.\n\nBuilding\n========\n\nPrerequisites:\n\n- autoconf, automake\n- some C compiler (gcc...)\n- some awk implementation.\n\nTo build:\n\n    ./autogen.sh\n    ./configure\n    make check\n\nTo install:\n\n    make install\n\nLicense\n=======\n\nThis file is part of process-watcher.\n\nprocess-watcher is free software: you can redistribute it and/or\nmodify it under the terms of the Apache 2.0 License as published by\nthe Apache Software Foundation.\n\nprocess-watcher is distributed in the hope that it will be useful, but\nWITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the License\nfor more details.\n\nYou should have received a copy of the Apache 2.0 License\nalong with process-watcher.\nIf not, see \u003chttps://www.apache.org/licenses/LICENSE-2.0\u003e.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famadeusitgroup%2Fprocess-watcher","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Famadeusitgroup%2Fprocess-watcher","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famadeusitgroup%2Fprocess-watcher/lists"}