{"id":21532057,"url":"https://github.com/digitalegesellschaft/anonip","last_synced_at":"2025-05-10T22:50:51.662Z","repository":{"id":41322297,"uuid":"106590511","full_name":"DigitaleGesellschaft/Anonip","owner":"DigitaleGesellschaft","description":"Anonip is a tool to anonymize IP-addresses in log-files.","archived":false,"fork":false,"pushed_at":"2023-01-13T14:27:49.000Z","size":108,"stargazers_count":79,"open_issues_count":11,"forks_count":17,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-05-01T09:23:27.684Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DigitaleGesellschaft.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-10-11T18:03:59.000Z","updated_at":"2025-03-17T09:52:25.000Z","dependencies_parsed_at":"2023-02-09T15:50:14.719Z","dependency_job_id":null,"html_url":"https://github.com/DigitaleGesellschaft/Anonip","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DigitaleGesellschaft%2FAnonip","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DigitaleGesellschaft%2FAnonip/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DigitaleGesellschaft%2FAnonip/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DigitaleGesellschaft%2FAnonip/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DigitaleGesellschaft","download_url":"https://codeload.github.com/DigitaleGesellschaft/Anonip/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253492589,"owners_count":21916963,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-24T02:18:37.695Z","updated_at":"2025-05-10T22:50:51.644Z","avatar_url":"https://github.com/DigitaleGesellschaft.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# anonip\n\n[![PyPI](https://img.shields.io/pypi/v/anonip.svg)](https://pypi.org/project/anonip/)\n[![Python versions](https://img.shields.io/pypi/pyversions/anonip.svg)](https://pypi.org/project/anonip/)\n[![Build Status](https://github.com/DigitaleGesellschaft/Anonip/workflows/Tests/badge.svg)](https://github.com/DigitaleGesellschaft/Anonip/actions?query=workflow%3ATests)\n[![Coverage](https://img.shields.io/badge/coverage-100%25-brightgreen.svg)](https://github.com/DigitaleGesellschaft/Anonip/blob/master/setup.cfg#L58)\n[![Black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/DigitaleGesellschaft/Anonip)\n[![License](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)\n\nDigitale Gesellschaft\nhttps://www.digitale-gesellschaft.ch\n\n\nFormerly\nSwiss Privacy Foundation\nhttps://www.privacyfoundation.ch/\n\n\n## Description\n\nAnonip is a tool to anonymize IP addresses in log files.\n\nIt masks the last bits of IPv4 and IPv6 addresses. That way most of the\nrelevant information is preserved, while the IP-address does not match a\nparticular individuum anymore.\n\nDepending on your webserver software, the log entries may directly get piped\nto Anonip. The unmasked IP addresses will never be written to any file.\n\nUsing shell redirects, it's also possible to rewrite existing log files.\n\n## Features\n\n - Masks IP addresses in log files\n - Configurable amount of masked bits\n - The column containing the IP address can freely be chosen\n - Alternatively use a regex to point anonip to the location(s) of the IP(s). See [this RFC](https://github.com/DigitaleGesellschaft/Anonip/issues/44) for more information.\n - Works for both access.log- and error.log files\n\n## Officially supported python versions\n\n - 2.7\n - 3.7\n - 3.8\n - 3.9\n - 3.10\n - 3.11\n - 3.12\n\n## Dependencies\nIf you're using python version \u003e=3.3, there are no external\ndependencies.\n\nFor python versions \u003c3.3:\n - [ipaddress module](https://github.com/kwi-dk/py2-ipaddress)\n\n## Invocation\n\n```\nusage: anonip.py [-h] [-4 INTEGER] [-6 INTEGER] [-i INTEGER] [-o FILE]\n                 [--input FILE] [-c INTEGER [INTEGER ...]] [-l STRING]\n                 [--regex STRING [STRING ...]] [-r STRING] [-p] [-d] [-v]\n\nAnonip is a tool to anonymize IP-addresses in log files.\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -4 INTEGER, --ipv4mask INTEGER\n                        truncate the last n bits (default: 12)\n  -6 INTEGER, --ipv6mask INTEGER\n                        truncate the last n bits (default: 84)\n  -i INTEGER, --increment INTEGER\n                        increment the IP address by n (default: 0)\n  -o FILE, --output FILE\n                        file to write to\n  --input FILE          File or FIFO to read from (default: stdin)\n  -c INTEGER [INTEGER ...], --column INTEGER [INTEGER ...]\n                        assume IP address is in column n (1-based indexed;\n                        default: 1)\n  -l STRING, --delimiter STRING\n                        log delimiter (default: \" \")\n  --regex STRING [STRING ...]\n                        regex for detecting IP addresses (use instead of -c)\n  -r STRING, --replace STRING\n                        replacement string in case address parsing fails\n                        (Example: 0.0.0.0)\n  -p, --skip-private    do not mask addresses in private ranges. See IANA\n                        Special-Purpose Address Registry.\n  -d, --debug           print debug messages\n  -v, --version         show program's version number and exit\n\nExample-usage in apache-config:\nCustomLog \"| /path/to/anonip.py [OPTIONS] --output /path/to/log\" combined\n```\n\n## Usage\n\n``` shell\n/path/to/anonip.py [OPTIONS] \u003c /path/to/orig_log --output /path/to/log\n```\nor using shell redirects only (mind the redirected output is appending):\n``` shell\n/path/to/anonip.py [OPTIONS] \u003c /path/to/orig_log \u003e\u003e /path/to/log\n```\n\n### With Apache\n\nIn the Apache configuration (or the one of a vhost) the log output needs to\nget piped to anonip like this:\n```\nCustomLog \"|/path/to/anonip.py [OPTIONS] --output /path/to/log\" combined\nErrorLog  \"|/path/to/anonip.py [OPTIONS] --output /path/to/error_log\"\n```\nThat's it! All the IP addresses will be masked in the log now.\n\n\n### With nginx\n\nnginx does not support spawning a process it then pipes to. Thus\nyou need to create a named pipe (file-based FIFO) and start the\nprocesses yourself, along this lines:\n``` shell\nmkfifo /path/to/log.fifo /path/to/error_log.fifo\n/path/to/anonip.py [OPTIONS] --output /path/to/log \u003c /path/to/log.fifo \u0026\n/path/to/anonip.py [OPTIONS] --output /path/to/error_log \u003c /path/to/error_log.fifo \u0026\n```\nAs you can see, you need to start a separate process for each access-log\nfile and for each error-log file.\n\nIn the nginx configuration (or the one of a vhost) the log output\nneeds to be set to the named pipe like this:\n```\naccess_log /path/to/log.fifo;\nerror_log  /path/to/error_log.fifo;\n```\n\n### As a python module\n\nRead from stdin:\n``` python\nfrom anonip import Anonip\n\nanonip = Anonip()\nfor line in anonip.run():\n    print(line)\n\n```\n\nManually feed lines:\n``` python\nfrom anonip import Anonip\n\ndata = ['1.1.1.1', '2.2.2.2', '3.3.3.3']\nanonip = Anonip()\nfor line in data:\n    print(anonip.process_line(line))\n\n```\n\n### Python 2 or 3?\nFor compatibility reasons, anonip uses the shebang `#! /usr/bin/env python`.\nThis will default to python2 on all Linux distributions except for Arch Linux.\nThe performance of anonip can be improved by running it with python3. If\npython3 is available on your system, you should preferrably invoke anonip\nlike this:\n\n``` shell\npython3 -m anonip [OPTIONS]\n```\n\nor\n\n``` shell\npython3 /path/to/anonip.py [OPTIONS]\n```\n\n## Motivation\n\nIn most cases IP addresses are personal data as they refer to individuals (or at least\ntheir Internet connection). IP addresses - and the data associated with them - may\ntherefore only be lawfully processed in accordance with the principles of the\napplicable data protection laws.\n\nStorage of log files from web servers, for example, is only permitted within close time\nlimits or with the voluntary consent of the persons concerned (as long as the\ninformation about the IP address is linkable to a person).\n\nAnonip tries to avoid exactly that, but without losing the benefit of those log files.\n\nWith the masking of the last bits of IP addresses, we're still able to distinguish the\nlog entries up to a certain degree. Compared to the entire removal of the IP-adresses,\nwe're still able to make a rough geolocating as well as a reverse DNS lookup. But the\notherwise distinct IP addresses do not match a particular individuum anymore.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdigitalegesellschaft%2Fanonip","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdigitalegesellschaft%2Fanonip","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdigitalegesellschaft%2Fanonip/lists"}