{"id":19388434,"url":"https://github.com/mk-fg/conntrack-logger","last_synced_at":"2025-04-23T23:31:39.642Z","repository":{"id":9363171,"uuid":"11217497","full_name":"mk-fg/conntrack-logger","owner":"mk-fg","description":"Tool to log conntrack flows and associated process/service info","archived":false,"fork":false,"pushed_at":"2017-07-13T08:15:09.000Z","size":23,"stargazers_count":16,"open_issues_count":0,"forks_count":7,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-02T22:32:53.274Z","etag":null,"topics":["auditing","cffi","conntrack","libnetfilter-conntrack","linux","netlink-sockets","network-monitoring","python"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"wtfpl","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mk-fg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-07-06T11:48:21.000Z","updated_at":"2024-06-17T16:49:49.000Z","dependencies_parsed_at":"2022-09-25T14:51:10.855Z","dependency_job_id":null,"html_url":"https://github.com/mk-fg/conntrack-logger","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mk-fg%2Fconntrack-logger","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mk-fg%2Fconntrack-logger/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mk-fg%2Fconntrack-logger/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mk-fg%2Fconntrack-logger/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mk-fg","download_url":"https://codeload.github.com/mk-fg/conntrack-logger/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250532064,"owners_count":21446107,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["auditing","cffi","conntrack","libnetfilter-conntrack","linux","netlink-sockets","network-monitoring","python"],"created_at":"2024-11-10T10:12:40.545Z","updated_at":"2025-04-23T23:31:39.346Z","avatar_url":"https://github.com/mk-fg.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"conntrack-logger\n--------------------\n\nTool to make best effort to log conntrack flows along with associated pids,\nwhich service cgroup they belong to and misc other info about them.\n\nThink of it as an auditd extension to log network connections.\n\nMain purpose is to keep track of what (if anything) in the system tries to\nestablish fishy or unauthorized connections.\n\nFor example, imagine your IDS spots a occasional (e.g. 1 per day/week)\nconnections to known botnet hosts from one of the intranet machines.\nYou get a dump of some encrypted traffic that gets passed, but looking at the\nmachine in question, you've no idea which pid initiated these at the time - only\nclue is transient port numbers, which are useful only while connection lasts.\n\nThis tool allows to attribute such logged connections to pids (which might be\ne.g. forked curl, hence not useful by itself) and services they belong to,\nassuming proper service-pid-tracking system (i.e. systemd cgroups) is in place.\n\nUnlike e.g. [netstat-monitor](https://github.com/stalexan/netstat-monitor/), it\ndoesn't poll `/proc/net/*` paths (though still uses them to map flow back to\npid), getting \"new flow\" events via libnetfilter_conntrack (netlink socket)\ninstead, in a bit more efficient manner.\n\n\n\nUsage\n--------------------\n\nJust run nfct_logger.py and get the entries from its stdout (lines wrapped for readability):\n\n```console\n# ./nfct_logger.py -p tcp\n1373127181: ipv6/tcp 2001:470:1f0b:11de::12/55446 \u003e 2607:f8b0:4006:802::1010/80 ::\n\t2354 1000:1000 /user/1000.user/1.session/systemd-1196/enlightenment.service ::\n\tcurl -s -o /dev/null ipv6.google.com\n1373127199: ipv4/tcp 192.168.0.12/34870 \u003e 195.24.232.208/80 ::\n\t28865 1000:1000 /user/1000.user/1.session/systemd-1196/dbus.service ::\n\tpython /usr/libexec/dbus-lastfm-scrobbler\n1373127220: ipv4/tcp 127.0.0.1/59047 \u003e 127.0.0.1/1234 ::\n\t2387 1000:1000 /user/1000.user/1.session/systemd-1196/enlightenment.service ::\n\tncat -v cane 1234\n```\n\nDefault log format (can be controlled via --format, timestamp format via --format-ts) is (wrapped):\n\n\t{ts}: {ev.proto} {ev.src}/{ev.sport} \u003e {ev.dst}/{ev.dport} ::\n\t\t{info.pid} {info.uid}:{info.gid} {info.service} :: {info.cmdline}\n\nInfo about pid might not be available for transient connections, like one-way\nudp packets, as these don't seem to end up in /proc/net/udp (or udp6) tables,\nhence it's hard to get socket \"inode\" number.\n\nAs netfilter, conntrack and netlink sockets are linux-specific things (afaik),\nscript should not work on any other platforms, unless there is some\ncompatibility layer in place.\n\n\n### nfct_cffi\n\nTool is based on bundled nfct_cffi module, which can be used from any python\ncode:\n\n```python\nfrom nfct_cffi import NFCT\n\nsrc = NFCT().generator()\nprint 'Netlink fd: {} (to e.g. integrate into eventloop)'.format(next(src))\nfor data in src:\n\tprint 'Got event: {}'.format(data)\n```\n\nModule uses libnetfilter_conntrack via CFFI.\n\n\n\nInstallation\n--------------------\n\nIt's a regular package for Python 2.7 (not 3.X), but not in pypi, so can be\ninstalled from a checkout with something like that:\n\n\t% python setup.py install\n\nBetter way would be to use [pip](http://pip-installer.org/) to install all the\nnecessary dependencies as well:\n\n\t% pip install 'git+https://github.com/mk-fg/conntrack-logger.git#egg=conntrack-logger'\n\nNote that to install stuff in system-wide PATH and site-packages, elevated\nprivileges are often required.\nUse \"install --user\",\n[~/.pydistutils.cfg](http://docs.python.org/install/index.html#distutils-configuration-files)\nor [virtualenv](http://pypi.python.org/pypi/virtualenv) to do unprivileged\ninstalls into custom paths.\n\nAlternatively, `./nfct_logger.py` can be run right from the checkout tree\nwithout any installation.\n\n### Requirements\n\n* Python 2.7 (not 3.X)\n* [CFFI](http://cffi.readthedocs.org) (for libnetfilter_conntrack bindings)\n* [libnetfilter_conntrack](http://www.netfilter.org/projects/libnetfilter_conntrack)\n* nf_conntrack_netlink kernel module (e.g. `modprobe nf_conntrack_netlink`)\n\nCFFI uses C compiler to generate bindings, so gcc (or other compiler) should be\navailable if module is being built from source or used from checkout tree.\n\nTo install these requirements on Debian/Ubuntu (tested on Ubuntu \"Xenial\"\n16.04), use:\n\n\t# apt install build-essential libnfnetlink-dev python-cffi libnetfilter-conntrack-dev libpython2.7-dev\n\n\nLimitations\n--------------------\n\nWhen new flow event is received from libnetfilter_conntrack, it\n[doesn't have \"pid\" attribute](https://git.netfilter.org/libnetfilter_conntrack/tree/include/libnetfilter_conntrack/libnetfilter_conntrack.h#n62)\nassociated with it, so script looks up corresponding line in `/proc/net/*` to\npick \"inode\" number for connection from there, then does\n`glob('/proc/[0-9]*/fd/[0-9]*')`, readlink() on each to find which one leads to\nsocket matching that inode and then grabs/prints info for the pid from there on\nmatch.\n\nSo for super-quick connections, slow pid context switching, lots of pids or\nsomething, it might fail to match socket/pid in time, while both are still\naround, printing only connection info instead.\n\nRunning curl on even the fastest url probably won't ever slip by the logging,\nbut some fast app opening socket, sending a packet, then closing it immediately\nafterwards can do that.\n\n[auditd](https://people.redhat.com/sgrubb/audit) is probably a tool to track\nsuch things in a more dedicated way.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmk-fg%2Fconntrack-logger","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmk-fg%2Fconntrack-logger","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmk-fg%2Fconntrack-logger/lists"}