{"id":13583435,"url":"https://github.com/Dieterbe/anthracite","last_synced_at":"2025-04-06T18:32:19.643Z","repository":{"id":5453007,"uuid":"6646939","full_name":"Dieterbe/anthracite","owner":"Dieterbe","description":"an event / change logging/managament app","archived":false,"fork":false,"pushed_at":"2016-12-13T18:16:07.000Z","size":1629,"stargazers_count":295,"open_issues_count":12,"forks_count":32,"subscribers_count":31,"default_branch":"master","last_synced_at":"2025-04-06T09:08:34.855Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Dieterbe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-11-12T01:26:33.000Z","updated_at":"2024-10-15T09:11:22.000Z","dependencies_parsed_at":"2022-08-03T16:15:20.282Z","dependency_job_id":null,"html_url":"https://github.com/Dieterbe/anthracite","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dieterbe%2Fanthracite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dieterbe%2Fanthracite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dieterbe%2Fanthracite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dieterbe%2Fanthracite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Dieterbe","download_url":"https://codeload.github.com/Dieterbe/anthracite/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247531229,"owners_count":20953918,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T15:03:28.632Z","updated_at":"2025-04-06T18:32:19.621Z","avatar_url":"https://github.com/Dieterbe.png","language":"JavaScript","readme":"## Anthracite event manager ##\n\n* what: track and manage all changes and events that can have a business and/or operational impact.  \n  (deploys, manual changes, outages, press releases, etc)\n* why: to increase operational visibility and collaboration\n\nsome use cases:\n\n* changelog for troubleshooting and keeping people informed\n* enriching monitoring dashboards with markers and annotation text, for visual interactive analysis\n* generating reports of operational outage response metrics (see further down)\n\n## Design goals ##\n\n* do one thing and do it well.  aim for simplicity, flexibility and integration.\n* accept and deliver events in various ways and support querying for tags and text (full text search)\n* support arbitrary tags, allow events with multiple lines, even rich text and hyperlinks.\n\n![Screenshot](https://raw.github.com/Dieterbe/anthracite/master/screenshots/screenshot-table.png)\n\n## Components ##\n\n* anthracite-web.py is the web app (interface for humans, and HTTP POST event receiver)\n* anthracite-compose-submit.sh to interactively compose and submit events from the CLI.\n* anthracite-submit-github.sh to send messages with git log from a code checkout\n* ElasticSearch is used as database.\n\n\n## Methods of submitting events ##\n\n* HTTP POST receiver in the web app (so you can use something like curl)\n* manually, in the web interface\n* manually, with the `anthracite-compose-submit.sh` CLI script\n* `anthracite-submit-github.sh` goes into a code checkout, generates a nice message with the commits/author info from a given\n  commit range, and submits it, along with a given list of tags\n\nSee the integration-examples directory for shell scripts (which also demonstrate how to use curl) and a \npython function.\n\n\n## Integration ##\n\n* [Timeserieswidget](https://github.com/Dieterbe/timeserieswidget) shows graphite graphs with anthracite's events\n* The [Graph-Explorer graphite dashboard](https://github.com/Dieterbe/graph-explorer) uses that.  You can search for events using ES/Lucene's\n  powerfull fulltext search, and you can also add new events by clicking on graphs where anomalies appear.\n\n![Screenshot2](https://raw.github.com/Dieterbe/anthracite/master/screenshots/flot-annotated-event.png)\n\n## Dependencies ##\n\n* python2\n* elasticsearch\n* java \u003e=1.6 for elasticsearch\n\n\n## Extensible schema to suit your business ##\n\n_standard event_ has:\n* date\n* description\n* 0-N arbitrary tags (words or 'key=value' tags)\n\nthis works fine in a lot of cases, but many environments require enhancements.\nYou can enhance quite a bunch via config.py options.  The forms adapt as needed,\nand the extra fields will be stored like regular fields.\n\n* `recommended_tags`: promote the use of specific tags in forms (they still get stored with other tags)\n* `extra_attributes`: extend on the default schema by specifying attributes, with these properties:\n\n    * key: the field name\n    * label: label to use in forms\n    * mandatory: does this option need to be filled in on forms or can it be left blank?\n    * choices: list of values. or False to enable freeform text input. list with 1 element to enforce a specific value.\n    * select_many: whether to allow the user to select N of the choices, or just one.\n\n* `helptext`: override/add help messages for specific fields in forms\n* `plugins`: enable plugins by filename (should match with what's in the `plugins` folder)\n\nThe default config.py demonstrates how to use them.\n\n## Plugins ##\n\nplugins expose new functionality by providing functions and decorating them with routes to bind them to a http path/method.\n\nThey can also add handler functions to handle incoming events (i.e. to validate according to a custom schema)\nThey provide `add_urls` to specify which urls should get added to the menu,\n`remove_urls` to denote which existing urls they replace/deprecate.\nplugins can have their own template files.\nAll options mentioned above (except `plugins`) can be specified by plugins, i.e. you can have plugins\nthat promote certain tags, change the schema in a certain way, make certain fields mandatory, etc.\n\nAnthracite comes with 2 plugins that we use at Vimeo, and that serve as examples for you:\n\n* vimeo_analytics : tabular and csv outputs for events relevant to our analytics team.\n* vimeo_add_forms : specialized forms with different schema's for different teams, and handlers\n  to validate accordingly\n\n\n## Handy ElasticSearch commands ##\n\nempty database/start from scratch (requires anthracite-web restart)\n```\ncurl -X DELETE \"http://localhost:9200/anthracite\"\n```\n\n## Installation ##\n\nInstall dependencies, and just get a code checkout and initialize all git submodules, like so:\n\n```\ngit clone --recursive https://github.com/Dieterbe/anthracite.git\n```\n\n* for Elasticsearch:\n\nsuper easy, see [elasticsearch docs](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup.html)   \njust set a unique cluster name, like `\u003ccompany\u003e-anthracite`.\nThis avoids ES accidentally joining other running ES instances on the same network and forming an undesired replicating cluster.\nNo need for any further configuration, schema setup, etc, anthracite-web takes care of that.\n\n\n## Deployment ##\n\nStart the web application and point your browser to http://0.0.0.0:8081/\n\n```\n\u003cpath_to_anthracite\u003e/anthracite-web.py\n\u003cpath_to_elasticsearch\u003e/bin/elasticsearch\n```\n\n## About \"relevant events\" ##\n\nI recommend you submit any event that **has** or **might have** a **relevant** effect on:\n* your application behavior\n* monitoring itself (for example you fixed a bug in metrics reporting. it shouldn't look like the app behavior changed)\n* the business (press coverage, viral videos, etc), because this also affects your app usage and metrics.\n\n\n## Formats and conventions ##\n\nThe format is very loose.  I recommend to use tags for categorisation, and ultimately there'll be full-text search so you don't have to worry too much\nabout formatting or additional tags, as long as the info is within the event.\n\nHowever, I recommend to try to use some \"standardized\" nomenclature, such as 'deploy', 'manual' (for manual changes), 'outage', ...\nYou can use tags like `author=\u003cperson\u003e` but this usually doesn't give any benefit over just tagging `\u003cperson\u003e`.\n\n\n## Operational metrics report ##\n\n![Screenshot](https://raw.github.com/Dieterbe/anthracite/master/screenshots/screenshot-opsreport.png)\n\nnew: use the optional outage field for a key (in default config) + start/detected/recovered tag\n\n\nThe event format and its tags are very loose.  However, you can use specific tags to enable the ops reporting:\n\n* give outage related events an identifier key (`20130225_switch_broke`) and tag outage related events like `outage=\u003ckey\u003e`.\n* add tags like `start`, `detected` (issue noticed) and `resolved` to mark resolution (actual service restoration). TODO metric for 'cause identified'.\n* add these tags to existing events (such as code deploys) or create new events to mark the points in time where an outage started,\n  got detected, got resolved, or changed impact level (i.e. temporary partial fixes)\n* Optional: add a tag like `impact=50`: scale of 0 to 100 to denote the extent to which users are affected during the outage. (100 being full outage for all)\n  this helps in assessing the severity of the event but don't obsess over it, it doesn't need to be too accurate.\n  note: nothing stops you from using a value like 1000 to mark an unrecoverable loss (i.e. dataloss)\n\nThe report will look for these tags and give you a report of your operational metrics: (note that the metrics are not weighted for impact yet)\n\nper event, mean, and total:\n\n* TTF (time to failure)\n* TTD (time to detection)\n* TTR (time to resolution)\n\naverage (TODO per-year)\n* Uptime\n\nThe [ops metametrics slidedeck](http://www.slideshare.net/jallspaw/ops-metametrics-the-currency-you-pay-for-change) give you more information.\n\n\n## TODO ##\n\n* plugins for puppet, chef to automatically submit their relevant events (or logstash filter to create anthracite events from logs)\n* auto-update events on web interface to make semi-realtime\n* on graphs in dashboards, show timeframs from start to end, and start to \"cause found\", to \"resolved\" etc.\n* concurrent webserver to make sure all http requests can get served\n* better MTBF\n","funding_links":[],"categories":["Observability \u0026 Monitoring","JavaScript"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FDieterbe%2Fanthracite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FDieterbe%2Fanthracite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FDieterbe%2Fanthracite/lists"}