{"id":15919931,"url":"https://github.com/awslabs/disco","last_synced_at":"2025-07-01T10:33:06.274Z","repository":{"id":41174405,"uuid":"208317627","full_name":"awslabs/disco","owner":"awslabs","description":"A suite of tools including a framework for creating Java Agents, for aspect-oriented tooling for distributed systems.","archived":false,"fork":false,"pushed_at":"2023-03-15T17:58:01.000Z","size":33942,"stargazers_count":65,"open_issues_count":7,"forks_count":12,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-05-20T11:06:47.578Z","etag":null,"topics":["agent","agents","aspect-oriented-programming","context-propagation","java","jvm","jvm-bytecode","jvm-languages"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/awslabs.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-09-13T17:53:10.000Z","updated_at":"2025-05-13T17:16:44.000Z","dependencies_parsed_at":"2024-10-28T23:50:26.130Z","dependency_job_id":null,"html_url":"https://github.com/awslabs/disco","commit_stats":{"total_commits":103,"total_committers":14,"mean_commits":7.357142857142857,"dds":0.7475728155339806,"last_synced_commit":"7b0b70f50554269d57bf7b15ea54a6ac47d59e4e"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/awslabs/disco","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awslabs%2Fdisco","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awslabs%2Fdisco/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awslabs%2Fdisco/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awslabs%2Fdisco/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/awslabs","download_url":"https://codeload.github.com/awslabs/disco/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awslabs%2Fdisco/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262944254,"owners_count":23388722,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent","agents","aspect-oriented-programming","context-propagation","java","jvm","jvm-bytecode","jvm-languages"],"created_at":"2024-10-06T19:06:54.485Z","updated_at":"2025-07-01T10:33:06.228Z","avatar_url":"https://github.com/awslabs.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n[![Actions Status](https://github.com/awslabs/disco/workflows/Java%20CI/badge.svg)](https://github.com/awslabs/disco/actions)\n\n## The Amazon Disco Toolkit\n\nA suite of tools including a framework for creating Java Agents, for aspect-oriented\ntooling for distributed systems.\n\nDisco is a sort-of acronym of **Di**stributed **S**ystems **Co**mprehension\nand may be styled as DiSCo, Disco or disco. We really don't mind.\n\nRight now, although intended eventually to encompass other tooling in a\nsimilar space, Disco includes a library/framework/toolkit for creating\nJava Agents aimed at solving operational tooling for distributed systems \nin service-oriented architectures.\n\n### Pre-release software\n| :stop_sign: \u0026nbsp; Disco is pre-release software. As an author of agents or plugins, you may be subject to some churn or rework while we finalize the APIs and features. |\n| --- |\n\nPlease note that whilst on versions less than semantic version 1.0, Disco is \npre-release or preview software. Some APIs may be subject to change or removal until\na stable v1.0 release occurs.\n\n\n### What does it do?\n\nThe Disco Java Agent toolkit provides 2 key runtime primitives, as an\nextension on top of Java (or other JVM languages such as Kotlin, but Java\nis currently the only language with robust tests).\n\n1. An in-process Event Bus, which advertises moments in the lifetime of a\n*transaction* or *activity* in service-oriented software. For example, a\nservice receiving a request via HTTP will begin a timeline of such events\nat that time, concluding when it has finally delivered a response to its caller.\n1. A Transactionally Scoped data dictionary called the TransactionContext.\nBy *Transactionally Scoped* we mean that this data store will be created\nat the beginning of the activity lifetime as described above, and survive \nuntil it concludes. Extensions are added to the Java runtime such that this\ndata store will be consistent during thread handoffs.\n\nThese are explored in more detail below.\n\nIn future, more languages will be supported.\n\n#### Event Bus\n\nThink about a straw man API on a service-oriented system. Let's say we have\na service CityInfoService, with an API \"getInfoForZipCode\". It delegates to two downstream\nservices, WeatherService and TrafficService, whose responses it aggregates into a\nresponse containing weather and traffic information for a given zip code. Since these\ntwo services are orthogonal and not interdependent, it calls them in parallel.\n\nThe timeline of a call to getInfoForZipCode might be as follows:\n\n1. At time T in Thread 0, a Servlet.service() call is made upon receipt of the getInfoForZipCode\nAPI being called.\n1. At time T+1 in Thread 0, the service submits two tasks to a pooled ExecutorService, to call\nthe two downstream services.\n1. At time T+2 in Thread 1, a call to WeatherService is made.\n1. At time T+2 in Thread 2, a call to TrafficService is made.\n1. At time T+3 in Thread 1, the response from WeatherService is received.\n1. At time T+4 in Thread 2, the response from TrafficService is received.\n1. At time T+5 in Thread 0, Threads 1 and 2 are join()'d.\n1. Finally, at time T+6 in Thread 0, a response is given to the service's caller.\n\nThe disco event bus issues events at all the key moments on this timeline of events.\nAt the time of each event, the TransactionContext (see below) is available to pass\ndata between the points at which each event is received and throughout this timeline\nof events, the Transaction ID (again, see below) is consistent and unique. If two calls\nto getInfoForZipCode were happening concurrently, and each had the same sequence of events\nthose events would agree on the content of the TransactionContext *within each activity*\nwith no crosstalk or confusion across the two.\n\nEvents are published for the upstream request and response, the downstream requests and responses\nand each time a Thread is forked or joined.\n\n#### Transaction Context\n\nConsider when a service activity might call 3 *downstream* services,\nparallelized in 3 threads from a thread pool. It may do this, for example,\nby dispatching work to an ExecutorService.\n\nOne problem we have observed in tooling such as AWS X-Ray, due to deficiencies\nin Java, is that these worker threads have no clear concept of 'caller' or 'parent'. \nUsing Disco, the Java runtime is extended to ensure that the concept of caller/parent\nis passed from thread to thread at the time of thread handoff (e.g. when calling Thread.start(),\nor someExecutor.submit(), or when using a Java 8 parallel stream), by the 'forking' thread\ngiving the 'forked' thread access to its Transaction Context data store.\n\nBy default, upon creation, the Transaction Context always contains a 96 bit random number\nformatted as a hexadecimal string as a Transaction ID. This can be overridden by plugins or\nclient code if desirable, and any other arbitrary data may also be added at\nany point in the lifetime of the service activity. Once the data is placed\nin the Transaction Context, it becomes available across the activity's family\nof threads thereafter.\n\n## What kind of uses does an Agent built on disco have?\n\nLet's start with a simple example of a logger. In our CityInfoService above, it may be\nchallenging to produce really good 'joined-up' logs due to the concurrent behavior of the service.\nWhen logging the calls to WeatherService and TrafficService, the threads appear\n'orphaned'. If you've ever tried to make sense of logs by looking at 'nearby timestamps'\ninstead of having unambiguous IDs available, this will be a familiar problem.\n\nSo the simplest example, is to create a Listener on the disco Event Bus which logs\nevery event, along with the ID from the Transaction Context. Now without taking any\nparticular special action in the service's business logic itself, all these lines of log\ncan be tied together via this ID.\n\nAnother common requirement is to pass metadata (perhaps via HTTP headers) from service to \nservice, when creating tracing our routing tools in service-oriented architectures.\nIncoming request events and outgoing request events provide a facility to inspect, and\nmanipulate HTTP headers. The AWS X-Ray agent uses this feature to propagate its segments\nacross service call hops, without user-intervention.\n\n## Creating a Java Agent\n\nThere are two basic ways to create a Java Agent using Disco. As a self-contained\nartifact, or using a plugin-based system to allow multi-tenancy.\n\nSee the subproject disco-java-agent-example as a simple example of building\na self contained agent, along with the associated tests for it in disco-java-agent-example-test\n\nAlternatively, the plugin-based approach may be seen in the disco-java-agent-web-plugin\nproject, which uses the canonical agent found in the disco-java-agent/disco-java-agent package\nas a substrate for plugin discovery.\n\n## How to install a Java Agent once created\n\nAs can be see in the build.gradle.kts files of several subprojects (e.g. the tests for\ndisco-java-agent-example), a single argument needs to be passed to the Java process\nat start-up. See AgentConfigParser.java in the Core libraries for details on the command line\narguments for agents, and see PluginDiscovery.java for details on how a substrate agent\nmay load plugins.\n\n## Using Java Agents on managed runtimes such as Lambda\n\nOne complexity for some managed runtimes is that the user does not have complete\nauthority over the arguments passed to Java. To remedy this, it is possible to 'Inject'\na Disco agent at runtime, using a tiny (1 or 2 lines) amount of boilerplate code.\nAn example of this is given in the disco-java-agent-example-inject-test project.\nIf using this method, care must be taken to perform the injection as early as possible\nin the target software's lifetime (first line of Main() is ideal, as soon as possible\nafter that is the next best). Disco works on the basis of extending the Java runtime\nvia aspect-oriented bytecode manipulation, and some of these treatments are unable to work\nif the instrumented class has already been used.\n\n## License\n\nThis library is licensed under the Apache 2.0 Software License.\n\n## Building Disco\n\nThe simplest way to start is to run the following command in the root directory.\n``./gradlew build``\n\nThis will build all the code, and run all the tests (functional tests and integration tests).\n\nThe final tests which are executed are tests for the 'thread handoff' TransactionContext\npropogation mentioned elsewhere in this README, which deserve a mention. Some of the tests are naturally\n'flaky'. This is true because when we request, for example, someCollection.parallelStream(), and then perform\nwork, the test is not in control of *whether that will actually be parallel or actually be serial*. The\nJava runtime is in charge and is not easily manipulated. If the Java runtime elects not to parallelize,\nour test becomes meaningless - we cannot assert that disco's thread hand-off behavior is\ncorrect if no thread hand-off occurred at all. So these tests are designed to retry. To be clear they don't\nstubbornly \"retry until they succeed\". They retry until *preconditions are met which they do not control*.\n\nUnfortunately this can mean that they *sometimes* fail and require restarting. We don't know a better way, sorry.\n\n### Including Disco as a dependency in your product\n\nDisco is available in Maven Central. A Bill of Materials (BOM) package is vended to make depending on multiple\nDisco packages easier.\n\n#### Using Maven coordinates\n```xml\n\u003cdependencyManagement\u003e\n    \u003cdependencies\u003e\n        \u003cdependency\u003e\n            \u003cgroupId\u003esoftware.amazon.disco\u003c/groupId\u003e\n            \u003cartifactId\u003edisco-toolkit-bom\u003c/artifactId\u003e\n            \u003cversion\u003e0.13.0\u003c/version\u003e\n            \u003ctype\u003epom\u003c/type\u003e\n            \u003cscope\u003eimport\u003c/scope\u003e\n        \u003c/dependency\u003e\n    \u003c/dependencies\u003e\n\u003c/dependencyManagement\u003e\n\u003cdependencies\u003e\n    \u003cdependency\u003e\n      \u003cgroupId\u003esoftware.amazon.disco\u003c/groupId\u003e\n      \u003cartifactId\u003edisco-java-agent-api\u003c/artifactId\u003e\n    \u003c/dependency\u003e\n    \u003c!-- Other Disco dependencies --\u003e\n\u003c/dependencies\u003e\n```\n\n#### Using Gradle's default DSL\n```groovy\nimplementation platform('software.amazon.disco:disco-toolkit-bom:0.13.0')\nimplementation 'software.amazon.disco:disco-java-agent-api'\n// Other disco dependencies\n```\n\n#### Using Gradle's Kotlin DSL\n```kotlin\nimplementation(platform(\"software.amazon.disco:disco-toolkit-bom:0.13.0\"))\nimplementation(\"software.amazon.disco:disco-java-agent-api\")\n// Other disco dependencies\n```\n\n### Troubleshooting\n\nIf you receive the following error\n```\nCould not determine the dependencies of task ':disco-java-agent-example:shadowJar'.\n\u003e Could not resolve all files for configuration ':disco-java-agent-example:runtimeClasspath'.\n   \u003e path may not be null or empty string. path='null'\n```\n\nPlease ensure that the default Java binary that is ran is Java 8. As a workaround, you may specify the `JAVA_HOME` to point to another version of Java.\n\nExample: `export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_202.jdk/Contents/Home`\n\n## What's with all the subprojects?\n\nDisco is quite componentized, added to which there are quite a few projects which serve as examples and tests.\n\nBrowse through the READMEs included with the subprojects to make sense of it all, but in summary there are\na few layers and families of projects in here:\n\n1. Public API contained in disco-java-agent-api\n1. Special API for implementors of interception plugins, in disco-java-agent-plugin-api\n1. Core library, for consumption by plugin authors and agent authors (not client code) in disco-java-agent-core\n1. Canonical Pluggable agent, capable of plugin discovery, in disco-java-agent/disco-java-agent\n1. A facility to 'Inject' a Disco Agent into managed runtimes like AWS Lambda\n1. A Plugin to support Servlets and Apache HTTP clients, in disco-java-agent-web-plugin\n1. A Plugin to support SQL connections \u0026 queries using JDBC, in disco-java-agent-sql-plugin\n1. A Plugin to support requests made with the AWS SDK for Java, in disco-java-agent-aws-plugin\n1. Example code in anything with '-example' in the project name.\n1. Tests in anything with '-test' in the project name.\n\nSubprojects themselves also encapsulate their own tests.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fawslabs%2Fdisco","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fawslabs%2Fdisco","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fawslabs%2Fdisco/lists"}