{"id":19900112,"url":"https://github.com/m-lab/uuid-annotator","last_synced_at":"2025-10-30T04:06:31.264Z","repository":{"id":39719224,"uuid":"231651554","full_name":"m-lab/uuid-annotator","owner":"m-lab","description":"Produces metadata locally for every connection on each server.","archived":false,"fork":false,"pushed_at":"2024-10-07T13:18:45.000Z","size":4639,"stargazers_count":0,"open_issues_count":13,"forks_count":0,"subscribers_count":13,"default_branch":"main","last_synced_at":"2024-10-29T14:37:41.345Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/m-lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-01-03T19:27:17.000Z","updated_at":"2024-10-07T13:18:36.000Z","dependencies_parsed_at":"2024-11-06T15:39:00.027Z","dependency_job_id":"b3a6a645-2f71-4b65-b850-730e1d0e8f34","html_url":"https://github.com/m-lab/uuid-annotator","commit_stats":{"total_commits":48,"total_committers":8,"mean_commits":6.0,"dds":0.6458333333333333,"last_synced_commit":"c2d823b34305a69443d5aeda1f2df273e9dacb2e"},"previous_names":[],"tags_count":26,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m-lab%2Fuuid-annotator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m-lab%2Fuuid-annotator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m-lab%2Fuuid-annotator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m-lab%2Fuuid-annotator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/m-lab","download_url":"https://codeload.github.com/m-lab/uuid-annotator/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223842246,"owners_count":17212321,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T20:11:13.266Z","updated_at":"2025-10-30T04:06:31.176Z","avatar_url":"https://github.com/m-lab.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# UUID-Annotator\n\n[![Version](https://img.shields.io/github/tag/m-lab/uuid-annotator.svg)](https://github.com/m-lab/uuid-annotator/releases)  [![Build Status](https://travis-ci.com/m-lab/uuid-annotator.svg?branch=master)](https://travis-ci.com/m-lab/uuid-annotator)  [![Coverage Status](https://coveralls.io/repos/m-lab/uuid-annotator/badge.svg?branch=master)](https://coveralls.io/github/m-lab/uuid-annotator?branch=master)  [![GoDoc](https://godoc.org/github.com/m-lab/uuid-annotator?status.svg)](https://godoc.org/github.com/m-lab/uuid-annotator)  [![Go Report Card](https://goreportcard.com/badge/github.com/m-lab/uuid-annotator)](https://goreportcard.com/report/github.com/m-lab/uuid-annotator)\n\nA system for generating and saving per-connection metadata in real-time on\nM-Lab's edge systems.\n\n## Design\n\nIt generates a JSON file for every connection containing the geolocation and\nnetwork location metadata for the IP addresses in the connection, and eventually\nadds in all other annotations concerning the \"local environment\" as well.\n\nThe datatype it generates will be \"annotation\" and it will generate filenames\nlike:\n\n```txt\n    /ndt/annotation/2009/03/18/${UUID}.json\n```\n\nwhere `${UUID}` is the actual UUID of the connection under consideration. It will follow both our uniform names best-practices and pusher best-practices.\n\nThe columns in the JSON file will initially be a subset of our standard columns:\n\n- `client.Geo.*`\n- `server.Geo.*`\n- `client.Network.ASNumber`\n- `server.Network.ASNumber`\n\nLater versions can (and should!) add columns that include real-time switch\ncounters, local machine load, and other indicators of measurement quality,\nbut v1 will concentrate on location data. Each new column added to the\nannotator output should be added to our set of standard columns.\n\nThe location annotation service will read from a MaxMind file served up via a\nfile stored in a GCS bucket. It will periodically poll (in a memoryless manner)\nto discover whether the file has changed.\n\n## Performance\n\nThis service will depend on tcp-info's UUID notification service, but no\nlocal service should depend on the annotator. As such, we do not need to\nworry about the annotator slowing down an integrated service, we only need to\nworry about the annotator keeping up with the creation rate of TCP\nconnections. We do not anticipate that being too difficult.\n\n## Availability\n\nThis service is a core service and needs to be highly available, just like\ntcp-info, packet-headers, traceroute-caller, and DISCO. It represents our one\nchance to annotate UUIDs with metadata. As such, the health of the experiment\nservice should depend on the health of the UUID annotation service, just like it\nshould depend on the other core services.\n\n## Usage\n\n### Stand-alone\n\nIf only the local ipservice socket is needed to provide annotations for specific\nIPs, the uuid-annotator may be run in a \"stand-alone\" mode. This mode does not\nrequire the tcp-info `-tcpinfo.eventsocket`, `-siteinfo.url`, or `-datadir`\nflags.\n\n```sh\ndocker build -t local-annotator .\ndocker run -v $PWD/testdata:/testdata -it local-annotator   \\\n    -ipservice.sock=/local/uuid-annotator.sock \\\n    -maxmind.url=file:///testdata/GeoLite2-City-real.tar.gz \\\n    -routeview-v4.url=file:///testdata/RouteViewIPv4.pfx2as.gz \\\n    -routeview-v6.url=file:///testdata/RouteViewIPv6.pfx2as.gz\n```\n\n### Generate Schemas\n\nIf using uuid-annotator data as part of the autoloader pipeline, you may\ngenerate the data type schemas using the `generate-schemas` command:\n\n```sh\ndocker run -v $PWD:/schemas --entrypoint /generate-schemas -it local-annotator \\\n    -ann2 /schemas/ann2.json -hop2 /schemas/hop2.json\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fm-lab%2Fuuid-annotator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fm-lab%2Fuuid-annotator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fm-lab%2Fuuid-annotator/lists"}