{"id":13814203,"url":"https://github.com/shish/pgosquery","last_synced_at":"2025-04-07T06:11:32.384Z","repository":{"id":22628162,"uuid":"25970775","full_name":"shish/pgosquery","owner":"shish","description":"Like Facebook's OSQuery, but for Postgres","archived":false,"fork":false,"pushed_at":"2016-07-12T16:30:32.000Z","size":15,"stargazers_count":448,"open_issues_count":0,"forks_count":9,"subscribers_count":17,"default_branch":"master","last_synced_at":"2025-04-06T06:11:39.755Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shish.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-10-30T12:58:48.000Z","updated_at":"2024-11-28T16:30:14.000Z","dependencies_parsed_at":"2022-08-21T09:30:34.491Z","dependency_job_id":null,"html_url":"https://github.com/shish/pgosquery","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shish%2Fpgosquery","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shish%2Fpgosquery/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shish%2Fpgosquery/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shish%2Fpgosquery/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shish","download_url":"https://codeload.github.com/shish/pgosquery/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247601448,"owners_count":20964864,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-04T04:01:46.908Z","updated_at":"2025-04-07T06:11:32.357Z","avatar_url":"https://github.com/shish.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"TL;DR Example\n-------------\n\n```sql\n--------------------------------------------------------\n-- get the name, pid and attached port of all processes\n-- which are listening on localhost interfaces\n--------------------------------------------------------\nSELECT DISTINCT\n    process.name,\n    listening.port,\n    process.pid\nFROM processes AS process\nJOIN listening_ports AS listening\nON process.pid = listening.pid\nWHERE listening.address = '127.0.0.1';\n```\n\n```psql\n   name   | port | pid\n----------+------+------\n postgres | 5432 | 6932\n\n(1 row)\n```\n\n\nAbout\n-----\n\nSo I saw Facebook's [OSQuery][1], and thought \"That looks awesome, but\ncomplicated to build on top of SQLite. Postgres' Foreign Data Wrappers seem\nlike a much better foundation. How long would it take to write the same app\non top of Postgres?\". Turns out it takes about 15 minutes, for someone who's\nnever written an FDW before :-)\n\nThis approach does have the downside that it runs as the postgres user rather\nthan as root, so it can't see the full details of other people's processes,\nbut I'm sure that could be worked around if you really want to.\n\nCurrently this is just a proof-of-concept to see how useful Postgres' foreign\ndata wrappers are, and how easy they are to create with the Multicorn python\nlibrary. Seems the answers are \"very useful\" and \"very easy\". If people want\nto make this more useful by adding more virtual tables, pull requests are\nwelcome~\n\n[1]: https://github.com/facebook/osquery\n\n\nInstallation\n------------\n\nTL;DR \n```\ngit clone https://github.com/shish/pgosquery.git\ncd pgosquery\nsudo apt-get install pgxnclient\nsudo pgxn install multicorn\nsudo pip install psutil\nsudo python setup.py develop\npsql -h localhost -U postgres -w \u003c queries.sql\n```\n\nLet your system python install know about this module:\n```bash\n$ sudo python setup.py develop\n```\n\"setup.py develop\" will link the current directory so you can modify it; \"setup.py install\" will copy a snapshot of current code to the OS folder.\n\nNote that either way, you need to restart the postgres server to pick up python code changes.\n\n\nCreate a database with multicorn loaded (See http://multicorn.org/#installation for multicorn installation)\n```sql\nCREATE DATABASE pgosquery;\n\\c pgosquery;\n\nCREATE EXTENSION multicorn;\n```\n\nCreate a FDW table for PgOSQuery:\n```sql\nCREATE SERVER pgosquery_srv foreign data wrapper multicorn options (\n    wrapper 'pgosquery.PgOSQuery'\n);\n\nCREATE FOREIGN TABLE processes (\n    pid integer,\n    name character varying,\n\tusername character varying\n) server pgosquery_srv options (\n    tabletype 'processes'\n);\n\nCREATE FOREIGN TABLE listening_ports (\n    pid integer,\n    address character varying,\n\tport integer\n) server pgosquery_srv options (\n    tabletype 'listening_ports'\n);\n```\n\nSelect data:\n```sql\n--------------------------------------------------------\n-- get the name, pid and attached port of all processes\n-- which are listening on all interfaces\n--------------------------------------------------------\nSELECT DISTINCT\n    process.name,\n    listening.port,\n    process.pid\nFROM processes AS process\nJOIN listening_ports AS listening\nON process.pid = listening.pid\nWHERE listening.address = '127.0.0.1';\n```\n\n```psql\n   name   | port | pid\n----------+------+------\n postgres | 5432 | 6932\n\n(1 row)\n```\n\n\nTable Types\n-----------\n\n`processes`: Columns are based on psutil's Process attributes, see http://pythonhosted.org/psutil/#psutil.Process\n\n`listening_ports`: pid, address, port\n\n`net_connections`: pid, address, port, type, status\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshish%2Fpgosquery","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshish%2Fpgosquery","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshish%2Fpgosquery/lists"}