Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/aws-samples/kubernetes-log4j-cve-2021-44228-node-agent


https://github.com/aws-samples/kubernetes-log4j-cve-2021-44228-node-agent

Last synced: 3 months ago
JSON representation

Awesome Lists containing this project

README 

        
#    !!!!    DEPRECATION NOTICE     !!!!

This project is being deprecated as outlined below.    

- 2023-01-01 - This repository will no longer be updated

- 2023-04-01 - This repository will be permanently deprecated



The project will cease to accept issues and will not be updated as of 2023-01-01. After this date, this project will be unsupported and no new releases or artifacts associated with this project will be generated. 



The project repository will permanenetly archived (made read-only) as of 2023-04-01.



As noted in the original project documentation, this was always meant to be a temporary measure to aid Kubernetes users, and it only ever provided a partial mitigation of the security exposure from [Log4j2 CVE-2021-44228](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228) and [CVE-2021-45046](https://lists.apache.org/thread/83y7dx5xvn3h5290q1twn16tltolv88f). It is strongly recommended that all users of Log4j update to a patched version of the project to fully mitigate any risk associated with these CVEs. 

## What if I still want to use this? 

The release artifacts associated with the latest release will remain permanently available for consumers of this project that continue to deploy it in to their Kubernetes environments. Specifically, the [DaemonSet manifest](https://raw.githubusercontent.com/aws-samples/kubernetes-log4j-cve-2021-44228-node-agent/main/daemonset.yaml) and ECR Public [conainer image](https://gallery.ecr.aws/aws-containers/kubernetes-log4j-cve-2021-44228-mitigation) will remain available after deprecation, but will no longer be supported or updated. 

# Apache Log4j2 CVE-2021-44228 node agent



AWS has developed an [RPM](https://github.com/corretto/hotpatch-for-apache-log4j2) that performs a JVM-level hot-patch which disables JNDI lookups from the Log4j2 library, mitigating [Log4j2 CVE-2021-44228](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228) and [CVE-2021-45046](https://lists.apache.org/thread/83y7dx5xvn3h5290q1twn16tltolv88f).



The Apache Log4j2 CVE-2021-44228 node agent is an open source project built by the Kubernetes team at AWS. It is designed to run as a DaemonSet and mitigate the impact of Log4j2 CVE-2021-44228, which affects applications running Apache Log4j2 versions < 2.15.0 when processing inputs from untrusted sources. Running this DeamonSet will patch JVMs running in containers as well as on the host.



**What it does:** A cron entry will be installed on every worker node that runs a process looking for running JVMs and injects an agent which mitigates the Log4J2 CVE. All JVMs, including those running in containers will be hot-patched in this way. Currently, the hot-patch process is configured to run every 30min with a 15min jitter. The effective window can range from 15 to 45min between runs.



**Note:** You can find additional information about the Log4j2 vulnerability and AWS response in the [AWS Security Bulletin](https://aws.amazon.com/security/security-bulletins/AWS-2021-006/).



**Note:** On 4/20/22, the image (v0.0.16) was updated with the latest patch for Amazon Linux 1 and 2. If you installed a previous version of this DaemonSet and it is still running, you should re-apply the daemonset.yaml file to install the latest RPM. If you no longer need the log4j patch, you should delete the DaemonSet and uninstall the RPM as older versions of the RPM (< 1.1-16) had a vulnerability that could allow an attacker to escape from a container and gain access to the underlying host. 



**Note:** version 1.3-5 addresses [ALAS2-2022-1806](https://alas.aws.amazon.com/AL2/ALAS-2022-1806.html) and [ALAS-2022-1601](https://alas.aws.amazon.com/ALAS-2022-1601.html). See [Amazon Linux "log4j hotpatch" <1.3-5 local privilege escalation to root](https://github.com/justinsteven/advisories/blob/main/2022_amazon_log4j-cve-2021-44228-hotpatch_local_privesc.md) for additional information. 



## Installation instructions

1. Apply the manifest:

```bash

kubectl apply -f https://raw.githubusercontent.com/aws-samples/kubernetes-log4j-cve-2021-44228-node-agent/main/daemonset.yaml

```



2. Check the logs for one or more of your DaemonSet pods.



Spot check a single pod:

```bash

kubectl get pods -l job=node-patch-installer -n node-configuration-daemonset

```

```bash

kubectl logs  -c node-patch-installer -n node-configuration-daemonset

```

or get confirmation for every node:

```bash

kubectl logs -l job=node-patch-installer -c node-patch-installer -n node-configuration-daemonset | grep 'installed'

```

If the installation was successful, the last line in the log file will read `installed and verified` otherwise an error will be logged.



### Considerations



* This project is meant to act as a temporary, best effort mitigation until you can update the Log4j2 dependency in all of your Java based Kubernetes applications to at least Log4j version 2.15.0. Do not rely on this agent as a long-term mitigation. This tool may help you mitigate the risk when updating dependencies is not immediately possible.

* The DaemonSet is supported on Amazon Linux (AL), Amazon Linux 2 (AL2), and Debian Linux distributions like Ubuntu. [Bottlerocket](https://github.com/bottlerocket-os/bottlerocket) is not supported as it does not include a shell or a package manager. If you are running log4j on Bottlerocket, the CVE will be addressed by [Hotdog](https://github.com/bottlerocket-os/hotdog). 

* Because DaemonSets do not run on AWS Fargate, any pods running on Fargate in Amazon EKS clusters will not be patched by this agent.

* With the default runtime frequency of 30 mins, the agent is better suited for long-running containers.

* The RPM only works with the following Java distributions:

    * Credo

    * Corretto

    * OpenJDK

 * If your worker nodes are in a sanboxed environment with no Internet connectivity, you can pull the [image](https://gallery.ecr.aws/aws-containers/kubernetes-log4j-cve-2021-44228-mitigation) from ECR and push it to a private registry. Alternatively, you can modify the makefile in the image directory to build and push the resulting image to your private registry. Edit the DaemonSet spec.template.spec.initContainers.image in the `daemonset.yaml` file to reflect the new URI:tag of the image, then apply the manifest to your cluster.  



## About CVE-2021-44228



Apache Log4j2 < 2.15.0 JNDI features used in configuration, log messages, and parameters do not protect against attacker controlled LDAP and other JNDI related endpoints. An attacker who can control log messages or log message parameters can execute arbitrary code loaded from LDAP servers when message lookup substitution is enabled. From Log4j2 versions < 2.15.0, this behavior has been disabled by default. Full details can be found in the [CVE bulletin](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228).



You can find additional information about the Log4j2 vulnerability in the [AWS Security Bulletin](https://aws.amazon.com/security/security-bulletins/AWS-2021-006/).



## How this works



AWS has developed an RPM that performs a JVM-level hotpatch disabling JNDI lookups from the Log4j2 library, mitigating the Log4j2 issue for that applies to JVMs on the host as well as JVMs running in containers. This project packages up the RPM as a Kubernetes DaemonSet.



When installed, a process will run on every worker node that looks for running JVMs and injects an agent into the JVM to mitigate the Log4j2 vulnerability. The agent attempts to patch the `lookup()` method of all loaded `org.apache.logging.log4j.core.lookup.JndiLookup` instances to unconditionally return the string `Patched JndiLookup::lookup()`. This is designed to address the [CVE-2021-44228](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228/) remote code execution vulnerability in Log4j2 without restarting the Java process.



This process by default is configured to run every 30 mins, and will add a layer of protection in clusters where applications have yet to be patched with an updated Log4j2 dependency.



You can find more details on the hotpatch in the [AWS open source blog](https://aws.amazon.com/blogs/opensource/hotpatch-for-apache-log4j/).



The container image is hosted in the aws-containers [ECR Public repository](https://gallery.ecr.aws/aws-containers/kubernetes-log4j-cve-2021-44228-mitigation).