{"id":25439576,"url":"https://github.com/brthor/docker-layer2-icc","last_synced_at":"2025-06-26T06:35:11.780Z","repository":{"id":74586968,"uuid":"122146348","full_name":"brthor/docker-layer2-icc","owner":"brthor","description":"Demonstrating that disabling ICC in docker does not block raw packets between containers.","archived":false,"fork":false,"pushed_at":"2018-02-21T19:52:02.000Z","size":250,"stargazers_count":65,"open_issues_count":0,"forks_count":6,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-08T10:54:04.903Z","etag":null,"topics":["docker"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/brthor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-02-20T02:20:17.000Z","updated_at":"2024-09-21T16:57:19.000Z","dependencies_parsed_at":"2023-02-27T03:01:09.196Z","dependency_job_id":null,"html_url":"https://github.com/brthor/docker-layer2-icc","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/brthor/docker-layer2-icc","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brthor%2Fdocker-layer2-icc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brthor%2Fdocker-layer2-icc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brthor%2Fdocker-layer2-icc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brthor%2Fdocker-layer2-icc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/brthor","download_url":"https://codeload.github.com/brthor/docker-layer2-icc/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brthor%2Fdocker-layer2-icc/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262016205,"owners_count":23245490,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker"],"created_at":"2025-02-17T10:19:48.347Z","updated_at":"2025-06-26T06:35:11.772Z","avatar_url":"https://github.com/brthor.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Docker Layer 2 ICC Bug\n\nQuick Start:\n```bash\ngit clone https://github.com/brthor/docker-layer2-icc.git \u0026\u0026 cd docker-layer2-icc \u0026\u0026 ./run.sh\n```\n\n## Explanation (What and How)\n\nWhen you create a docker container using `docker run`, it is automatically connected to a bridge network. Unless inter-container communication (ICC) was disabled in the docker daemon, every container on that bridge network can communicate with one another via sockets.\n\nDocker allows you to restrict ICC in two ways:\n1. Restricting it in the kernel with `\"icc\": false` in `daemon.json`\n2. Creating a network like `docker network create -o com.docker.network.bridge.enable_icc=false noicc` then connecting containers to it with `docker run --network=noicc ...`\n\nIn both cases you expect that all network communications will be blocked between the containers themselves. As demonstrated below, this is not the case. Docker puts in iptables rules that block communications on layer 3, but layer 2 communications are allowed. What this means, is I can still send data between the containers over a socket.\n\n**Disabling ICC doesn't block raw ethernet frames between containers.**\n\nThis behavior is highly unexpected, and in highly secure environments, likely to be an issue.\n\n## Repro Steps\n\nI reproed this using `Docker-CE for Mac Version 17.09.0-ce-rc3-mac30 (19329)`.\nAnd `Docker-CE on CentOS 7.3 Version 18.02.0-ce`\n\n### Automatic Repro\n\nUse [./run.sh](/run.sh) to run an automatic repro. Your output will look like:\n\n```bash\n$ ./run.sh\nSending build context to Docker daemon  602.1kB\nStep 1/4 : FROM python:3\n ---\u003e 336d482502ab\nStep 2/4 : RUN apt-get update \u0026\u0026 apt-get install -y nano\n ---\u003e Using cache\n ---\u003e 5cee00913b09\nStep 3/4 : COPY ./ethListen.py /ethListen.py\n ---\u003e Using cache\n ---\u003e a2667cd58e69\nStep 4/4 : CMD python -u /ethListen.py\n ---\u003e Using cache\n ---\u003e 9743c680cc72\nSuccessfully built 9743c680cc72\nSuccessfully tagged eth-listener:latest\nSending build context to Docker daemon  602.1kB\nStep 1/4 : FROM python:3\n ---\u003e 336d482502ab\nStep 2/4 : RUN apt-get update \u0026\u0026 apt-get install -y nano\n ---\u003e Using cache\n ---\u003e 5cee00913b09\nStep 3/4 : COPY ./ethSender.py /ethSender.py\n ---\u003e Using cache\n ---\u003e e528077d48da\nStep 4/4 : CMD python -u /ethSender.py\n ---\u003e Using cache\n ---\u003e 46061f2a53ca\nSuccessfully built 46061f2a53ca\nSuccessfully tagged eth-sender:latest\neth-listener\neth-sender\n8391d06550a63fb8423be86876cd906a79720f6d52c0d8885ce2e102b8015768\nSent 35-byte Ethernet packet on eth0\nXX:XX:XX:XX:XX:XX\n^ Mac Address ^\nListening for packets\nReceived: Len: 78 bytes time: 1519123406.9721925 message: `:\n                                                            $[8\n                                                               $\nReceived: Len: 90 bytes time: 1519123407.0323486 message: `$B:|{\nReceived: Len: 90 bytes time: 1519123407.2033708 message: `$B:|{\nReceived: Len: 35 bytes time: 1519123407.2230816 message: HELLO from the SENDER\n^C\n```\n\nIf you see the line\n```\n\nReceived: Len: 35 bytes time: 1519123407.2230816 message: HELLO from the SENDER\n\n```\n\nThen the sender successfully sent a raw ethernet frame, despite icc being disabled.\n\n### Manual Repro Steps\n\n1. Build the Listener image: `docker build -t \"eth-listener\" -f listener-Dockerfile .`\nThe [listener](/ethListen.py) listens for raw ethernet frames and prints any received data.\nIt also finds and prints it's layer 2 address. We will need this value to send it data.\n\n2. Build the Sender image: `docker build -t \"eth-sender\" -f sender-Dockerfile .`\nThe [sender](/ethSender.py) sends the string \"HELLO from the SENDER\" to the listener. \nWe will look for this string amongst the listener output.\n\n3. Create the ICC Disabled network: `docker network create -o com.docker.network.bridge.enable_icc=false noicc`\n`noicc` is the name of the network.\n\n4. Make sure you have two shells open, one for the listening container, and one for the sender.\n\n5. Start your listening container: \n```bash\n$ docker run -it --network noicc --name eth-listener eth-listener\nXX:XX:XX:XX:XX:XX\n^ Mac Address ^\nListening for packets\n...\n```\n\n**Copy the mac address value for the next step.**\n\n6. Run the sending container in your second terminal\n```bash\n$ docker run -it --network noicc --name eth-sender eth-sender\n\"Enter the Destination Mac Address.\"\n```\n\n7. Paste the mac address from the listening container in the sending container terminal.\n\n8. Observe the Listening container output for `HELLO from the SENDER`\n```\n$ docker run -it --network noicc --name eth-listener eth-listener\nXX:XX:XX:XX:XX:XX\n^ Mac Address ^\nListening for packets\nReceived: Len: 78 bytes time: 1519123587.8931577 message: `:vgG\\vg\nReceived: Len: 78 bytes time: 1519123587.9433937 message: `:]qB\nReceived: Len: 90 bytes time: 1519123588.222616 message: `$:\nReceived: Len: 35 bytes time: 1519123588.2736714 message: HELLO from the SENDER\nReceived: Len: 32 bytes time: 1519123588.9437985 message:\n```\n\n**Disabling ICC on the bridge network didn't block the raw socket communication as we would have expected**\n\nThese contain more explanation to break it down.\n\n## Bug Resolution (Workarounds)\n\nIf you keep containers on the same network bridge, create ebtables rules between containers. For each pair of containers, create a pair of rules, like those below, but replacing the mac addresses with the addresses of your containers.\n```\nsudo ebtables -A FORWARD -d 02:42:ac:13:00:02 -s 02:42:ac:13:00:03 -j DROP\nsudo ebtables -A FORWARD -s 02:42:ac:13:00:02 -d 02:42:ac:13:00:03 -j DROP\n```\n\nIt can be worked around by placing containers on different network bridges. That means using `docker network create` for every container. By default you can only create 31 networks. [Set the subnet on each network manually to get around this limitation](https://loomchild.net/2016/09/04/docker-can-create-only-31-networks-on-a-single-machine/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrthor%2Fdocker-layer2-icc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbrthor%2Fdocker-layer2-icc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrthor%2Fdocker-layer2-icc/lists"}