Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/ashutoshiwnl/monarchjavaagent

Monarch-Java-Agent is a powerful Java agent for method monitoring and analysis. It offers various features to track method execution time, print stack traces, log method arguments and return values, capture heap dumps, and gather JVM-related information.
https://github.com/ashutoshiwnl/monarchjavaagent

application-monitoring bytecode-manipulation instrumentation-agent java-agent

Last synced: 22 days ago
JSON representation

Monarch-Java-Agent is a powerful Java agent for method monitoring and analysis. It offers various features to track method execution time, print stack traces, log method arguments and return values, capture heap dumps, and gather JVM-related information.

Awesome Lists containing this project

README 

          
# MonarchJavaAgent



![Monarch Logo](jekyll/source/monarch-logo-v2.png)



MonarchJavaAgent is a hybrid Java agent for production diagnostics and JVM observation. It lets you attach targeted bytecode instrumentation to a live JVM, expose Prometheus-compatible JVM metrics, and use runtime configuration to move between instrumenter, observer, and hybrid workflows without changing the application code.



This is useful when you need to:



- inspect a hot method in production without redeploying

- inject a temporary diagnostic probe to validate a runtime hypothesis

- expose JVM health signals to Prometheus and Grafana

- combine runtime traces with monitoring data during incident response



## Features



- **Stack Trace Printing**: Print the stack trace when a certain method is invoked. Supports conditional printing only if a specific class/method appears in the current stack trace.

- **Return Value Logging**: Log the return value of a method.

- **Custom code addition**: Add custom code to a method.

- **Trace macro for custom code**: Use `MLOG(...)` inside `ADD` rules to route diagnostic output into Monarch trace logs instead of app stdout/app logger.

- **Class replacement**: Replace already loaded classes using external bytecode from `.class` files or `.jar` entries via JVM redefine.

- **Heap Dump Capture**: Take a heap dump when a method is invoked or exits.

- **Method Execution Time**: Print the time taken for a method's execution (Bit buggy currently and needs bytecode verification to be turned off).

- **System Flags Printing**: Print system flags of the target application.

- **JVM Options Printing**: Print JVM options of the target application.

- **JVM Heap Usage Details**: Print JVM heap usage details of the target application.

- **JVM CPU Usage Details**: Print JVM CPU usage details of the target application.

- **JVM GC Stats**: Print JVM Garbage Collection stats including collection count and time.

- **JVM Thread Stats**: Print JVM thread stats including current thread count and peak thread count.

- **JVM Classloader Stats**: Print JVM class loading stats including loaded/unloaded classes, class load rate, and Metaspace usage.

- **Metrics HTTP Endpoint**: Exposes JVM metrics (heap, GC, threads, CPU, classloader) via a built-in `/metrics` endpoint in Prometheus text format, with OpenMetrics negotiation support and JSON compatibility at `/metrics.json`.

- **Alert Emails**: Sends alert emails for high heap usage, high CPU usage, thread deadlock, high classloading, etc. in the target application. 



## Production Use Cases



- Attach to a live JVM and trace method arguments, return values, or stack frames on a hot path.

- Add a temporary runtime debug probe with `ADD` or `CODEPOINT` rules to confirm an application hypothesis.

- Run in `observer` mode to expose JVM metrics to Prometheus without enabling bytecode instrumentation.

- Run in `hybrid` mode to combine targeted instrumentation and scrapeable metrics during incident investigation.



## Architecture (Current)



After refactoring, startup wiring is split into focused bootstrap components:



- `AgentConfigurator`: thin facade used by `Agent.premain` and `Agent.agentmain`.

- `AgentStartupOrchestrator`: startup sequence orchestration.

- `LoggingBootstrap`, `ConfigBootstrap`, `TraceBootstrap`, `SmtpBootstrap`, `TransformerBootstrap`, `InstrumentationManagerBootstrap`: focused startup stages.



This keeps behavior unchanged while reducing coupling in the startup path.



Transformer internals are also split using a handler strategy:



- `GlobalTransformer`: orchestration (rule selection, backup, dispatch)

- `ActionExecution`: per-rule execution context

- `transformer.handlers.*`: isolated action handlers (`ARGS`, `RET`, `STACK`, `HEAP`, `ADD`, `PROFILE`)



## Agent Arguments



| Argument          | Description                                                               |

|-------------------|---------------------------------------------------------------------------|

| `configFile`      | Path to the configuration file specifying agent behavior **[Mandatory]**. |

| `agentLogFileDir` | Directory where initialization logs will be written.                      |

| `agentLogLevel`   | Log verbosity level (`DEBUG`, `INFO`, `WARN`, `ERROR`).                   |

| `smtpProperties`  | Path to SMTP configuration for sending alert emails.                      |

| `agentJarPath`    | Path to the MonarchJavaAgent jar **[Mandatory for startup attach]**.      |



## Usage



You can attach MonarchJavaAgent either during startup or during runtime.



**For startup:**



1. Download the latest agent JAR file from releases page.

2. Start your Java application using the `-javaagent` option.

3. Specify the configuration file and other options as needed.



Example command to attach the agent:



```bash 

java -Xverify:none -javaagent:/path/to/MonarchJavaAgent.jar=configFile=/path/to/config.yaml,agentLogFileDir=/path/to/log/dir,agentLogLevel=DEBUG,smtpProperties=/path/to/smtpProperties.props,agentJarPath=/path/to/MonarchJavaAgent.jar YourMainClass

```



**For Runtime:**



1. Download the latest agent JAR file from releases page.

2. Start your application.

3. Run "monarchAgentStart.bat"/"monarchAgentStart.sh" and provide the requested details.



Example command run:

```bash 

C:\Users\ashut\monarch-java-agent\attachScript> .\monarchAgentStart.bat

Enter path to the agent JAR file: C:\Users\ashut\monarch-java-agent\target\MonarchJavaAgent-1.2.jar

Enter path to the agent config file: C:\Users\ashut\monarch-java-agent\sampleConfig\mConfig.yaml

Enter arguments to pass to the agent: agentLogFileDir=C:\Users\ashut\manualTesting,agentLogLevel=DEBUG,smtpProperties=/path/to/smtpProperties.props

Enter PID of the target JVM (press Enter to use current JVM): 24300

Agent attached successfully to PID 24300

```



Preflight validation mode (no startup side effects):



```bash

java -javaagent:/path/to/MonarchJavaAgent.jar=configFile=/path/to/config.yaml,preflight=true,agentLogFileDir=/path/to/log/dir,agentLogLevel=INFO,agentJarPath=/path/to/MonarchJavaAgent.jar YourMainClass

```



When `preflight=true` is provided, Monarch validates config and rules, prints a validation summary, and exits without starting instrumentation/observer components.



## Configuration



MonarchJavaAgent now supports a nested, mode-aware YAML structure. Supported modes are:



- `instrumenter`: enable bytecode instrumentation only

- `observer`: enable JVM observation/metrics only

- `hybrid`: enable both instrumentation and observer features



Canonical sample configuration:



```yaml

mode: hybrid



instrumentation:

  enabled: true

  configRefreshInterval: 15

  traceFileLocation: C:\\TraceFileDumps

  agentRules:

    - ClassA::methodA@INGRESS::STACK

    - ClassA::methodA@INGRESS::ARGS

    - ClassA::methodA@EGRESS::RET

    - ClassA::methodB@INGRESS::ARGS

    - ClassA::methodB@INGRESS::STACK::[com.asm]

    - ClassA::methodB@EGRESS::STACK

    - ClassA::methodB@EGRESS::RET

    - ClassB::methodC@PROFILE

    - ClassB::methodC@INGRESS::HEAP

    - ClassB::methodC@INGRESS::ADD::[System.out.println(20);]

    - ClassA::methodA@INGRESS::ADD::[System.out.println(this.getClass().getName());]

    - ClassA::methodA@CODEPOINT(11)::ADD::[System.out.println(499);]

    - ClassA::methodA@CODEPOINT(11)::ADD::[System.out.println(499 + "," + "Ashutosh Mishra");]

    - com.example.MyService@CHANGE::FILE::[/opt/patches/MyService.class]

    - com.example.*@CHANGE::JAR::[/opt/patches/hotfix.jar]



observer:

  enabled: true

  printClassLoaderTrace: true

  printJVMSystemProperties: true

  printEnvironmentVariables: true

  metrics:

    exposeHttp: true

    port: 9090

    heapUsage: true

    cpuUsage: true

    threadUsage: true

    gcStats: true

    classLoaderStats: true



alerts:

  enabled: true

  maxHeapDumps: 3

  emailRecipientList:

    - abc@example.com

    - ashutosh@asm.com

```



Backward compatibility:



- Legacy flat keys such as `shouldInstrument`, `printJVMHeapUsage`, `exposeMetrics`, and `metricsPort` are still supported.

- Legacy flat keys are deprecated and now emit warnings during config parsing.

- If both nested and legacy forms are present for the same setting, the nested value wins.



Current implementation note:



- `instrumentation.traceFileLocation` is still used as the shared trace output root for both instrumentation and observer trace output.

- For `observer` mode, set `instrumentation.traceFileLocation` as well until trace output is moved to a common/shared config section in a future cleanup.



Launch args remain separate from YAML:



- `configFile`

- `agentLogFileDir`

- `agentLogLevel`

- `smtpProperties`

- `agentJarPath`



See the checked-in example at [sampleConfig/mConfig.yaml](sampleConfig/mConfig.yaml).



Legacy flat example still accepted:



```yaml

shouldInstrument: true

configRefreshInterval: 15

traceFileLocation: C:\\TraceFileDumps

agentRules:

  - ClassA::methodA@INGRESS::STACK

printJVMHeapUsage: true

printJVMCpuUsage: true

printJVMThreadUsage: true

printJVMGCStats: true

printJVMClassLoaderStats: true

exposeMetrics: true

metricsPort: 9090

maxHeapDumps: 3

sendAlertEmails: true

emailRecipientList:

  - abc@example.com

```



## Rule Syntax



The rule syntax for MonarchJavaAgent follows the format:



```plaintext

::@::

```



Where:



- ``: Fully Qualified Class Name.

- ``: Name of the method.

- ``: Event at which the action should be performed. Possible values are:

    - INGRESS

    - EGRESS

    - CODEPOINT

    - PROFILE (Note: PROFILE is a special case and no ACTION is required along with it.)

- ``: Action to be performed. Possible values are:

    - STACK: Print stack trace.

    - HEAP: Capture heap dump.

    - ARGS: Log method arguments.

    - RET: Log method return value.

    - ADD: Add custom code.



Optional `ADD` trace macro:



- `MLOG()`: Writes `String.valueOf()` to Monarch trace file via `TraceFileLogger`.

- `MLOG(...)` is optional and backward-compatible; existing `ADD` payloads still work.



Examples:



```plaintext

ClassA::methodA@INGRESS::ADD::[System.out.println("legacy add path");]

ClassA::methodA@INGRESS::ADD::[MLOG("Money: " + objA.getMoney());]

ClassA::methodA@INGRESS::ADD::[if (objA != null) { MLOG("objA=" + objA); }]

```



Class replacement rule syntax:



```plaintext

@CHANGE::FILE::[]

@CHANGE::JAR::[]

```



Where:



- `` is either an exact FQCN (for example `com.example.MyClass`) or a wildcard suffix pattern (for example `com.example.*`).

- `CHANGE` rules apply only to classes that are already loaded.

- `FILE` uses raw bytes from the class file.

- `JAR` resolves `` matches to class entries inside the jar.



Rule validation diagnostics:



- Startup now emits a rule validation summary with accepted/rejected/skipped counts.

- Rejected rules include reason + suggestion.

- `CHANGE::FILE` / `CHANGE::JAR` rules are prevalidated for path existence/readability and source-type expectations.



## Metrics Endpoint



If `observer.metrics.exposeHttp: true` is enabled in the config, MonarchJavaAgent starts a lightweight HTTP server exposing JVM metrics.



Default endpoints:



- `http://localhost:9090/metrics` (Prometheus text format by default)

- `http://localhost:9090/metrics.json` (legacy JSON compatibility)



`/metrics` supports OpenMetrics content negotiation. If the request includes:



- `Accept: application/openmetrics-text; version=1.0.0`



the endpoint responds with OpenMetrics content type and EOF marker.



Exported metrics include:



- Heap usage (bytes and usage percentages)

- CPU load and cores

- GC interval stats (per-GC labels) plus GC totals/rates for long-window analysis

- Thread stats

- Classloader stats

- Agent info + scrape timestamp



Sampling cadence transparency:



- Monarch updates metric values on monitor loops, not on every scrape.

- Current monitor loop cadences are:

  - CPU: every 10s

  - Heap: every 30s

  - GC / Thread / Classloader: every 60s

- If Prometheus scrapes faster than these cadences (for example every 2s), repeated samples are expected between monitor updates. This is correct behavior, but adds little extra signal for those metrics.

- For GC trends and alerting over time, prefer counter-based PromQL such as:

  - `rate(monarch_jvm_gc_collection_count_total[5m])`

  - `rate(monarch_jvm_gc_collection_time_seconds_total[5m])`



## Prometheus And Grafana



MonarchJavaAgent fits cleanly into a standard Prometheus and Grafana stack. Prometheus scrapes Monarch's `/metrics` endpoint, and Grafana visualizes the exported JVM signals without requiring a separate exporter.



Minimal Prometheus scrape config:



```yaml

global:

  scrape_interval: 10s



scrape_configs:

  - job_name: "monarch-java-agent"

    static_configs:

      - targets: ["127.0.0.1:9090"]

```



Scrape interval guidance:



- `10s` is a practical default because CPU metrics update every 10s.

- Scraping faster than 10s is allowed, but usually returns repeated values for most series.

- If your focus is only slow-moving JVM health metrics, `15s`-`30s` can reduce Prometheus load with minimal loss.



Common Prometheus queries:



- `monarch_jvm_heap_used_bytes`

- `monarch_jvm_cpu_process_percent`

- `monarch_jvm_threads_current`

- `monarch_jvm_classloader_loaded`



Example Grafana dashboard using Monarch metrics:



![Grafana dashboard for MonarchJavaAgent metrics](jekyll/source/grafana-dashboard.png.jpg)



## Documentation



- Jekyll source for the project site lives under `jekyll/source`.

- GitHub Pages is built and deployed from workflow automation (not from committed `docs/` output).



## Releasing



- Release process runbook: [RELEASING.md](RELEASING.md)

- Release workflow is tag-driven and validates tag version against `pom.xml`.



## Building from Source



```bash

git clone https://github.com/AshutoshIWNL/MonarchJavaAgent.git

cd MonarchJavaAgent

mvn clean package

```



## Smoke Testing (Startup + Runtime Attach)



The project now includes an integration smoke harness that validates:



- `-javaagent` startup mode

- Runtime attach mode via Attach API

- Instrumentation markers for `ARGS`, `RET`, `STACK`, `PROFILE`, `ADD`, `CODEPOINT`, `HEAP`

- Class replacement success/failure coverage for `CHANGE::FILE` and `CHANGE::JAR`

- Metrics endpoint availability

- Heap dump generation

- Runtime config reload behavior (rule removal stops instrumentation effect without restart)

- Invalid rule behavior (startup failure for malformed action rules)

- Nested-vs-legacy config precedence behavior at runtime

- Boundary safety behavior where startup/attach/transform failures are contained and do not crash the host JVM flow



### Prerequisites



- Java + Maven available in `PATH`

- For runtime attach tests, set `JAVA_HOME` to a JDK (not JRE)

- Profiling/instrumentation currently runs with bytecode verification disabled (`-Xverify:none`) in smoke scripts



### Windows (PowerShell)



```powershell

powershell -ExecutionPolicy Bypass -File scripts\smoke-all.ps1

```



To run individual suites:



```powershell

# full startup + attach coverage

powershell -ExecutionPolicy Bypass -File scripts\smoke-javaagent.ps1

powershell -ExecutionPolicy Bypass -File scripts\smoke-attach.ps1



# quick partial checks (fast signal)

powershell -ExecutionPolicy Bypass -File scripts\smoke-quick.ps1



# deep runtime config reload checks

powershell -ExecutionPolicy Bypass -File scripts\smoke-config-reload.ps1



# invalid rule behavior check

powershell -ExecutionPolicy Bypass -File scripts\smoke-invalid-rule.ps1



# nested config overrides legacy flat keys

powershell -ExecutionPolicy Bypass -File scripts\smoke-config-precedence.ps1



# class replacement via redefine from class file and jar

powershell -ExecutionPolicy Bypass -File scripts\smoke-class-replace.ps1

```



### Linux/macOS (Shell)



```bash

chmod +x scripts/*.sh

export JAVA_HOME=/path/to/jdk

bash scripts/smoke-all.sh

```



To run individual modes:



```bash

# full startup + attach coverage

bash scripts/smoke-javaagent.sh

bash scripts/smoke-attach.sh



# quick partial checks (fast signal)

bash scripts/smoke-quick.sh



# deep runtime config reload checks

bash scripts/smoke-config-reload.sh



# invalid rule behavior check

bash scripts/smoke-invalid-rule.sh



# nested config overrides legacy flat keys

bash scripts/smoke-config-precedence.sh



# class replacement via redefine from class file and jar

bash scripts/smoke-class-replace.sh

```



Smoke run artifacts (trace/log/config) are written to temp run directories under:



- Windows: `%TEMP%\mja-smoke\...`

- Linux/macOS: `/tmp/mja-smoke/...`



## Author



- **Ashutosh Mishra** (https://github.com/AshutoshIWNL)



## License



This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.