{"id":22812638,"url":"https://github.com/nastel/remoraj","last_synced_at":"2025-04-22T16:20:32.498Z","repository":{"id":40271462,"uuid":"212433643","full_name":"Nastel/remoraj","owner":"Nastel","description":"Extensible, low overhead Java Bytecode instrumentation agent for optimizing Java app performance","archived":false,"fork":false,"pushed_at":"2023-09-05T21:59:18.000Z","size":14808,"stargazers_count":61,"open_issues_count":6,"forks_count":6,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-04-22T16:20:24.708Z","etag":null,"topics":["agent-trace","bytebuddy","bytecode","bytecode-instrumentation","devops-pipeline","devops-tools","java","java-apps","java-performance","java-transaction","jkool","nastel-xray","performance-metrics","remora-java-agent","streams-agent","tracking"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Nastel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-10-02T20:19:56.000Z","updated_at":"2025-02-04T15:50:37.000Z","dependencies_parsed_at":"2022-07-24T18:02:16.983Z","dependency_job_id":null,"html_url":"https://github.com/Nastel/remoraj","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nastel%2Fremoraj","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nastel%2Fremoraj/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nastel%2Fremoraj/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nastel%2Fremoraj/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Nastel","download_url":"https://codeload.github.com/Nastel/remoraj/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250275553,"owners_count":21403680,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent-trace","bytebuddy","bytecode","bytecode-instrumentation","devops-pipeline","devops-tools","java","java-apps","java-performance","java-transaction","jkool","nastel-xray","performance-metrics","remora-java-agent","streams-agent","tracking"],"created_at":"2024-12-12T12:13:21.307Z","updated_at":"2025-04-22T16:20:32.419Z","avatar_url":"https://github.com/Nastel.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# RemoraJ: Extensible Java Bytecode Agent for Optimizing Java App Performance\n\nRemoraJ is an extensible java profiling agent which uses bytecode instrumentation to intercept java IPC calls with minimal overhead. RemoraJ monitors java apps by tracking IPC calls such as: HTTP, WebServices, JMS, JDBC, Sockets, WebSockets, Kafka, I/O Streams and more. The goal is to capture inter-JVM calls, I/O, messages, exceptions, timings which are valuable during troubleshooting and performance optimization of java apps. \n\nRemoraJ consists of two main runtime components:\n\n* `Remora Java Agent`: loaded into JVM, app server, instruments bytecode, emits traces and metrics via pluggable advices\n* `Remora Streams Agent`: forwards the traces generated by the java agent to one of the supported analytics platforms below\n\n**Platforms for Analyzing RemoraJ Traces**\n* [Nastel XRay](https://xray.nastel.com/xray/Nastel/login.jsp) -- SaaS Platform for AIOps \u0026 transaction tracking\n* Integrate traces into ELK, Splunk or other platform via [agent output class implementation](#agent-trace-output)\n\nBoth agents communicate via a memory mapped queue stored on a filesystem see `tnt-data-source.xml` for details. The Streams Agent sends traces using JSON over HTTPS.\n\n**NOTE:** Streams Agent is based on [tnt4j-streams](https://github.com/Nastel/tnt4j-streams) project designed to process, parse and stream time series data over a number of transports such as: HTTPS, Kafka, JMS, MQTT, MQ, files, etc.\n\n![RemoraJ Architecture](https://github.com/Nastel/remoraj/blob/master/remoraj-arch.png)\n\n## RemoraJ Intercepts (Advices)\nRemoraJ supports the following intercepts (advices):\n```\n  \"ApacheHttpClientAdvice\",\n  \"ApacheLegacyHttpClientAdvice\",\n  \"BankBenchmarkAdvice\",\n  \"RemoraControlAdvice\",\n  \"EjbRemoteAdvice\",\n  \"HttpUrlConnectionAdvice\",\n  \"JavaxServletAdvice\",\n  \"JBossAdvice\",\n  \"JBossServletAdvice\",\n  \"JDBCConnectionAdvice\",\n  \"JDBCStatementAdvice\",\n  \"JDBCCallableStatementAdvice\",\n  \"JMSCreateConnectionAdvice\",\n  \"JMSReceiveAdvice\",\n  \"JMSSendAdvice\",\n  \"KafkaConsumerAdvice\",\n  \"KafkaProducerAdvice\",\n  \"KafkaConsumerClientAdvice\",\n  \"SimpleTest\",\n  \"SimpleTestConstructor\",\n  \"SpringServiceAdvice\",\n  \"SpringExceptionAdvice\",\n  \"SpringTransactionAdvice\",\n  \"WebLogicAdvice\",\n  \"WebsocketSendAdvice\",\n  \"WebsocketEndpointAdvice\",\n  \"WebsocketReceiveAdvice\",\n  \"WebsocketSessionAdvice\",\n  \"IBMAdapterRSA\",\n  \"WASAdvice\"\n```\n# Installing RemoraJ\n\nYou must configure the Remora Java agent and Streams agent to run RemoraJ. Your should have a working data repository and a streaming access token. Below is the outline of RemoraJ setup:\n\n* Configure the Remora Java Agent for your JVM (captures java call traces, metrics)\n* Start your application or app server (with remora agent)\n\nComplete the following steps for Nastel XRay integration:\n* Obtain your account with [Nastel XRay](https://xray.nastel.com/xray/Nastel/login.jsp). Make sure to get your streaming access token\n* Configure the Streams Agent (you will need your streaming access token here)\n* Start the Streams Agent (forwards traces to your data repository associated with your streaming access token)\n* Run your Java apps with RemoraJ Agent enabled\n* Login to your [Nastel XRay Dashboard](https://xray.nastel.com/xray/Nastel/login.jsp) dashboard to analyze your traces \n\n## Prerequisites\n\nJava 8+ runtime (IBM WebSphere 8.5.5+, Tomcat 8+, jBoss 7.2+ should be already running Java 8. Make sure JVM is Java 1.8 or above).\n\nThe following IBM WebSphere runtime jar files are required to build `remora-websphere` advices:\n\n* `com.ibm.jaxws.thinclient_8.5.0.jar`\n* `com.ibm.ws.runtime.jar`\n* `com.ibm.ws.webcontainer.jar`\n* `rsahelpers.jar`\n* `com.ibm.ws.admin.core.jar`\n\nPlace these jar files in `remora-websphere/lib` folder before running a build.\n\n# Remora Java Agent \n## Using -javaagent option\n\n### IBM WebSphere\n\n* Option 1: using IBM WebSphere console.\n\n    * Step 1:    Navigate to `Application servers \u003e [Your server name] \u003e Process definition \u003e Java Virtual Machine`\n    * Step 2:    Edit field \"Generic JVM arguments\"\n    * Step 3:    Add `-javaagent:[\u003cinstall_dir\u003e/remora-\u003cversion\u003e]/remora.jar`\n    * Step 4:    Restart IBM WebSphere \n    * Step 5:    Run and configure Streams forwarding agent\n\n* Option 2: editing `server.xml` properties manually\n\n    * Step 1:    Navigate to `\u003cUSER_DIR\u003e/IBM/WebSphere/\u003cServer\u003e/profiles/\u003cApp Server\u003e/config/cells/\u003cCell\u003e/nodes/\u003cNode\u003e/servers/\u003cServer\u003e/server.xml`\n    * Step 2:    Edit node `/process:Server/processDefinitions/jvmEntries` parameter `@genericJvmArguments`\n    * Step 3:    Edit the path to where your `remora.jar` situated\n    ```xml\n    \u003cjvmEntries xmi:id=\"JavaVirtualMachine_1183122130078\" verboseModeClass=\"false\" verboseModeGarbageCollection=\"false\" verboseModeJNI=\"false\" initialHeapSize=\"512\" maximumHeapSize=\"2056\" runHProf=\"false\" hprofArguments=\"\" genericJvmArguments=\"-javaagent:\u003cinstall_dir\u003e/remora-\u003cversion\u003e/remora.jar\" executableJarFileName=\"\" disableJIT=\"false\"\u003e\n    ```\n\n### IBM WAS Liberty\n\n* Step 1:    Edit or create `jvm.options` file in the folder `wlp/usr/servers/\u003cserverName\u003e/`\n* Step 2:    Add lines:\n```\n-javaagent:\u003cinstall_dir\u003e/remora-\u003cversion\u003e/remora.jar -Dremora.path\n```\n* Step 3:    Edit the path to where your `remora.jar` situated\n* Step 4:    Edit or create `bootstrap.properties` in the folder `wlp/usr/servers/\u003cserverName\u003e/`\n* Step 5:    Add line:\n```\norg.osgi.framework.bootdelegation=com.jkoolcloud.remora.*\n```\n\n### JBoss Application Server\n\n#### Standalone mode\n\n* Step 1:    Edit `bin/standalone.bat[.sh]`\n* Step 2:    Add line: \n```\n    winx: set \"JAVA_OPTS=%JAVA_OPTS% -javaagent:\u003cinstall_dir\u003e/remora-\u003cversion\u003e/remora.jar\"\n    unix: JAVA_OPTS=\"$JAVA_OPTS -javaagent:\u003cinstall_dir\u003e/remora-\u003cversion\u003e/remora.jar\"\n```\n* Step 3:    Edit the path to where your `remora.jar` is located\n\n#### Domain mode\n\n* Step 1:    Edit `domain/configuration/host.xml`\n* Step 2:    Edit tag `\u003cservers\u003e\u003cjvm\u003e`\n* Step 3:    Add `\u003coption value=\"-javaagent:\u003cinstall_dir\u003e/remora-\u003cversion\u003e/remora.jar\"/\u003e`\n```xml\n       \u003cjvm name=\"default\"\u003e\n           \u003cjvm-options\u003e\n           \u003c/jvm-options\u003e\n               \u003coption value=\"-agentlib:jdwp=transport=dt_socket,address=5007,server=y,suspend=n\"/\u003e\n               \u003coption value=\"-javaagent:\u003cinstall_dir\u003e/remora-\u003cversion\u003e/remora.jar\"/\u003e\n       \u003c/jvm\u003e\n```\n* Step 4:    Edit the path to where your `remora.jar` is located\n\n### Standalone Java Application\n\nTo run you standalone application with RemoraJ add option `-javaagent:\u003cinstall_dir\u003e/remora-\u003cversion\u003e/remora.jar` to your run script or command line:\n\n```\njava -javaagent:\u003cinstall_dir\u003e/remora-\u003cversion\u003e/remora.jar -jar \u003cjar-file-name\u003e.jar\n```\n\n## Configure Streams Agent (Optional)\n\nAs for version 0.1.9 Streams will be configured automatically on first run. For more sophisticated control you can follow these instructions:\n\n* Step 1:    (Optional) Go to `\u003cinstall_dir\u003e/remora-\u003cversion\u003e/tnt4j-streams/config`\n* Step 2:    (Optional) Edit `tnt4j-streams.properties` and setup your access token (`event.sink.factory.EventSinkFactory.prod.Token`)\n* Step 3:    (Optional) More settings in `\u003cinstall_dir\u003e/remora-\u003cversion\u003e/tnt4j-streams/remora-streamer/tnt-data-source.xml` \n* Step 4:    (Optional) Edit ```\u003cproperty name=\"FileName\" value=\"../../queue\"/\u003e``` to point to your RemoraJ queue directory.\n\n**NOTE:** RemoraJ queue directory hosts files containing java traces produced by remora java agent. The traces are read by the Streams Agent and forwarded to your data repository. \n\n# Remora Agent Configuration\n\nRemoraJ configuration file is located `config` folder, file named `remora.properties`.\nSee relevant comments in the file for advanced config.\n\n## Agent VM Id\n\nTo identify your VM: set system property (`java -Dremora.vmid=MyVmName`) to any readable token identifying the process remora agent is \nattached to. `remora.vmid` is set to `ManagementFactory.getRuntimeMXBean().getName()` by default.\n\n**NOTE:** `remora.vmid` value might be not available on every VM's implementation and may be platform dependent.\n\n## Application Id\n \nTo identify your application: set system property (`java -Dremora.appl.name=MyApp1`). All emmited traces are tagged with the specified app name. Application name is automatically set based on deployed app name when remora agent is running in the application server context such as jBoss. \n\n**NOTE:** Default `remora.appl.name` is set to `java` if not explicitly specified.\n\n## Agent Trace Output\n\nBy default, RemoraJ java agent writes its traces to a high-performance memmory mapped persistent store (file) backed by `com.jkoolcloud.remora.core.output.ChronicleOutput`. This store is used as a communication channel between RemoraJ java agent and the Streams Agent.\n\nThere are several trace outputs available below:\n\n* `com.jkoolcloud.remora.core.output.NullOutput` -- no output (null output)\n* `com.jkoolcloud.remora.core.output.SysOutOutput` -- `System.out` (console)\n* `com.jkoolcloud.remora.core.output.ChronicleOutput` -- persistent memory mapped file-based output **(default)**\n\nFile system queue is created only with `com.jkoolcloud.remora.core.output.ChronicleOutput`. Set system property (`java -Dremora.output`) with the trace output class reference (e.g. `-Dremora.output=com.jkoolcloud.remora.core.output.SysOutOutput`).\n\n## Advice Trace Filters\nAdvices can have trace filters. Filters determine which trace are include vs. excluded. You configure the filter by defining one in `remora.properties`.\n```properties\n\u003cprefix for filter definition - filter\u003e.\u003cunique filter name\u003e.\u003cfilter property\u003e\n```\nfor example:\n```properties\nfilter.myDefinedFilter.type=com.jkoolcloud.remora.filters.ClassFilter\nfilter.myDefinedFilter.mode=INCLUDE/EXCLUDE\nfilter.myDefinedFilter.classes=java.net.SocketInputStream\n```\nto apply the filter(s) to advice, add as `advice` property:\n\n```\ncom.jkoolcloud.remora.advices.\u003cadvice name\u003e.filters=\u003cunique filter name\u003e;\u003cunique filter name2\u003e;\u003cunique filter name3\u003e\n```\n\n# Running Streams Agent\n\nOnce the Remora Java Agent is running and the Streams Agent configured you can run `\u003cinstall_dir\u003e/remora-\u003cversion\u003e/tnt4j-streams/remora-streamer/run.sh(run.bat)` to start the Streams Agent and begin forwarding java traces to your trace output. Running first time you will be asked for a repository token. This tokes is latter written in file \"token\", if you'd like to change.\n\n# Troubleshooting and Logging\n\nIf there is some problems running you can always check the logs. Logging might cause seriuos overhead, so by default the logging option is turned off.\nIn order to turn on the logging you need to change configuration file, or you can turn it on remotelly if you are using `remora-control` module.\n\nTo turn on logging you need to change option `com.jkoolcloud.remora.advices.BaseTransformers.logging` to `true` in the file `remora.properties`. You can also turn on logging for individual advices.\n\nEach advice creates it own logging file. You can find all files in your remora's folder under `log` -- `\u003cinstall_dir\u003e/remora-\u003cversion\u003e/log`.\n\n# Supported Analytics Platforms\n* [Nastel XRay](https://xray.nastel.com/xray/Nastel/login.jsp) -- AIOps \u0026 transaction tracking. Supports SaaS/On-prem/Cloud\n* Integrate traces into ELK, Splunk or other platform via [agent output class implementation](#agent-trace-output) \n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnastel%2Fremoraj","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnastel%2Fremoraj","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnastel%2Fremoraj/lists"}