{"id":20331120,"url":"https://github.com/comcast/money","last_synced_at":"2025-05-06T20:03:07.413Z","repository":{"id":2775984,"uuid":"43500617","full_name":"Comcast/money","owner":"Comcast","description":"Dapper Style Distributed Tracing Instrumentation Libraries","archived":false,"fork":false,"pushed_at":"2024-09-13T17:13:20.000Z","size":2399,"stargazers_count":67,"open_issues_count":21,"forks_count":33,"subscribers_count":25,"default_branch":"master","last_synced_at":"2025-03-03T00:09:23.332Z","etag":null,"topics":["dapper","distributed","distributed-traces","instrumentation","span","tracing"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Comcast.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-10-01T14:18:57.000Z","updated_at":"2024-09-13T17:05:43.000Z","dependencies_parsed_at":"2024-11-14T20:18:56.751Z","dependency_job_id":"d6ff385a-0e3b-4af8-965d-b87f980f90ce","html_url":"https://github.com/Comcast/money","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Comcast%2Fmoney","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Comcast%2Fmoney/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Comcast%2Fmoney/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Comcast%2Fmoney/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Comcast","download_url":"https://codeload.github.com/Comcast/money/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243565416,"owners_count":20311704,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dapper","distributed","distributed-traces","instrumentation","span","tracing"],"created_at":"2024-11-14T20:18:52.292Z","updated_at":"2025-03-14T11:09:11.206Z","avatar_url":"https://github.com/Comcast.png","language":"Scala","readme":"## Development Status\nReleased 0.9.0!  A lot of updates here.  Many things are not backward compatible\nwith 0.8.x, in particular the configuration.  See the `sample` for information\non usage and configuration changes.\n\n**In-progress**\nCurrently updating the documentation to be current.  A lot of out of data material in the wiki.\n\n## What it has\n\n- *http-client* - support for apache http clients, to inject money and/or B3 http headers seamlessly\n- *java-servlet* - support for java servlet filters, to automatically parse money and/or B3 http headers from HTTP requests, and inject them into your process to continue the trace\n- *akka* - support for instrumenting akka actor applications for trace continuations\n- *avro* - support for serialization of money data to avro\n- *spring* - easily instrument your applications using Spring AOP / spring beans.  Sample MVC application included under `samples` directory\n- *aspectj* - use java annotations and aspectj to non-intrusively add distributed tracing to your application\n- *kafka* - we do provide support for sending money data directly to kafka.  **Note: this module is dated and could use some love**\n\nIn addition to what you find in this repository, there are also other repositories:\n\n- [Go-lang](https://github.com/Comcast/golang-money) support as well in a separate repository.\n- [Cedi-Dtrace - Functional Money](https://github.com/ccadllc/cedi-dtrace) is a library that supports tracing for principled functional programming in Scala.  If this repository is too side effecting for you to stomach, this is where you should go.\n\n\n## Distributed Tracing Reads\n[Distributed Trace For Video Systems](http://www.nctatechnicalpapers.com/Paper/2015/2015-distributed-trace-for-video-systems/download)\nwritten by Michael Bevilacqua-Linn discusses our experiences at Comcast implementing distributed traces.\n\n## Money: Distributed Tracing Made Simple\n### Money makes it simple to trace across threads and systems\nMoney is a modular distributed tracing platform that can be seamlessly incorporated into modern applications.  *It's\npurpose is to provide a foundation for operational analytics through distributed tracing.*\n\nMoney is built on Scala and Akka, to be non-blocking from the core. It is purposefully un-opinionated, keeping undesired pull-through dependencies to a minimum.\n\nMoney modules build on the core, so implementing tracing is a snap. From Apache Http Components to Spring 4, from thread pools to Scala Futures, Money has modules to support a wide range of architectures.\n\n## Why is it different?\nMoney was inspired by inspired by [Google Dapper](http://research.google.com/pubs/pub36356.html)\nand [Twitter Zipkin](https://zipkin.apache.org/); however there are some subtle yet fundamental differences\nbetween those systems and Money, the biggest one being...\n\n### Spans start and end within a single process\nIn Dapper, a Span can encompass the communication between a client and a server.  Let's use an example of an\n*Order System* calling an *Inventory Management System*.  With Dapper, you could have the following:\n\n- *span-name* - GetInventory\n- *start-time*\n- *client-send* - the time that the client sent the request\n- *server-recv* - the time that the server received the request\n- *server-send* - the time that the server responded to the request\n- *client-recv* - the time that the client received the request\n- *foo* - any arbitrary annotation (note) made by either the client or the server\n\nThe idea being that everything can be calculated when the data is at rest.\n\nIn Money, it is theoretically possible to do the same, but by default we always extend a span when the server\nstarts processing.  We do this because we like to record important notes by default, namely the *span-duration* and\nthe *span-success*.  Using the example above, with Money we would get:\n\n- *app-name* - OrderSystem\n- *span-name* - GetInventory\n- *start-time* - time that the get inventory request started\n- *span-duration* - duration in Microseconds the entire request took\n- *span-success* - a boolean indicating success or failure of the operation from the Order System\n- *host* - the machine name generating the data\n- *foo* - any arbitrary bit of data\n\n... and on the server we would get...\n\n- *app-name* - InventoryManagement\n- *span-name* - GetInventory\n- *start-time* - time that the server began processing the request\n- *span-duration* - how long it took for the server to process the request\n- *span-success* - a success or failure indicator of the operation\n- *host* - the machine name generating the data\n- *bar* - any arbitrary bit of data\n\n#### Why do that?\nThere are a tradeoffs with any decision.  \n\nHere are some disadvantages:\n\n- More data, we get twice as much data for the same RPC, this can be a problem for extremely large volume systems as much\nmore data will be generated\n- Memory pressure - as we calculate duration and success, we need to maintain some state in process, which will add some memory overhead\n\nHere are some advantages:\n\n- You can use Money as a basis for operational analytics.  Money generates 3 important stats: latency, throughput and error rate.\nWe find that these are the most important stats in analyzing system performance and capacity.  You could calculate\nthese at rest in Dapper; but we believe that these can be understood more quickly to support real-time monitoring\n- You can very quickly integrate with monitoring tools.  As opposed to having to calculate metrics at rest in Dapper and Zipkin,\nyou can publish span data directly to Graphite (for instance), and create charts immediately.  This is a very strong advantage\nto calculating metrics at the source.  Money actually comes with a *Graphite Emtitter* out-of-the box.  Given that\nmoney is pluggable, it is quite trivial to integrate with many other systems.\n- In distributed system calls, it is important to know that the server \"succeeded\" or \"failed\", allowing you to more quickly\nfocus your troubleshooting efforts when things go wrong.  For example, if the server succeeded, then we can direct our\nefforts toward the client or the network\n- Less calculation is needed on the data - we have basic measures that we can use for calculations, allowing us\nto process data and perform analytics faster\n\n### We don't sample...yet\nI can see rocks starting to fly here, and I understand.  Money was not built in its present incarnation to support\nsystems which generate 10s of millions of events per second.  Money was built to use distributed tracing as a foundation\nfor operational analytics.  \n\nWe were much more interested in creating a standard set of metrics that we could use as a basis to perform operational analytics.\nAs such, for us, every event does matter as we can build aggregates very easy (even success).\n\nWe have been able to instrument systems that do generate many millions of events per hour with success, but Money did\nnot have the same considerations that went into the Dapper design.  Being able to process our base metrics gets us\ncloser to real-time understanding of processing; distributing the calculations to the origin systems gave us a lot of\nflexibility and speed in processing the data at rest.\n\n\u003e We are committed to support sampling and are evaluating designs...ideas are welcome.  Look for basic sampling to be added shortly\n\n### We do not provide a backend like Zipkin\nZipkin comes with an entire infrastructure built around Scribe and Cassandra that actually allows you to see data.  This\nis super cool, and something we aspire to complete.  We have looked at Spark Streaming and Akka Cluster Sharding as\nimplementation mechanisms (and have some prototype / experimental code to that end), but we have not yet gotten our act\ntogether.\n\nOne of the main advantages of Money is that it provides usable operational analytics out-of-the-box; whether\nreporting to Graphite, exposing data via JMX, and/or aggregating logs in Logstash.  As such, we have been able to gain\nkey insight into traces and performance using standard open source tools; here are some examples:\n\n- Splunk / LogStash - we use log aggregators to perform metrics across systems at rest.  The nice thing about log\naggregators is that you can not only see your metrics, but you can also look at all of the log entries that happend\nfor a single Span\n- Graphite / Prometheus - we use common analytic systems for monitoring and altering on Money data\n\n## Should I use Money?\nThis depends on the scale of your implementation.  Money _tries_ to serve a wide range of implementations.\n\nCertainly, if you want to implement an application that is serving 1000s or 10000s of request per second per JVM, Money\nshould work for you.  You can easily funnel data into your log aggregator or other reporting system and start\ngetting the benefits immediately.\n\nIf your implementation is in the order of 50000+ RPS with lots of spans, then things will get difficult\nas you will have to manage a lot of data.  Spooling span events to disk and sending them as you can is one approach.\nYou can use FluentD, Heka, PipeD or something else to eventually get the data off of disk.  Theoretically it is possible,\nbut without sampling, Money is generating a ton of data.  If you are not using that data for analytics, you can\nfilter it out (or contribute back a sampling feature); either way, it becomes a challenge.\n\n## I don't need no docs, gimme Money!\n\nAdd a dependency as follows for maven:\n\n```xml\n    \u003cdependency\u003e\n        \u003cgroupId\u003ecom.comcast.money\u003c/groupId\u003e\n        \u003cartifactId\u003emoney-core_2.12\u003c/artifactId\u003e\n        \u003cversion\u003e0.9.0\u003c/version\u003e\n    \u003c/dependency\u003e\n```\n\n## User Guide\n[Money Documentation](https://github.com/Comcast/money/wiki)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcomcast%2Fmoney","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcomcast%2Fmoney","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcomcast%2Fmoney/lists"}