{"id":20388291,"url":"https://github.com/libp2p/gossipsub-hardening","last_synced_at":"2025-04-12T10:36:45.000Z","repository":{"id":39793029,"uuid":"259375360","full_name":"libp2p/gossipsub-hardening","owner":"libp2p","description":"testground plans for evaluating gossipsub under attack scenarios","archived":false,"fork":false,"pushed_at":"2024-01-31T16:49:37.000Z","size":135,"stargazers_count":13,"open_issues_count":2,"forks_count":6,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-03-26T05:33:09.161Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/libp2p.png","metadata":{"files":{"readme":"README-shared-environment.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2020-04-27T15:36:50.000Z","updated_at":"2024-03-06T13:40:24.000Z","dependencies_parsed_at":"2024-01-31T18:03:41.148Z","dependency_job_id":null,"html_url":"https://github.com/libp2p/gossipsub-hardening","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/libp2p%2Fgossipsub-hardening","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/libp2p%2Fgossipsub-hardening/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/libp2p%2Fgossipsub-hardening/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/libp2p%2Fgossipsub-hardening/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/libp2p","download_url":"https://codeload.github.com/libp2p/gossipsub-hardening/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248554488,"owners_count":21123682,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T03:08:35.426Z","updated_at":"2025-04-12T10:36:44.973Z","avatar_url":"https://github.com/libp2p.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Shared jupyterhub Workflow\n\nThis doc describes how we (the gossipsub-hardening team at Protocol Labs) have been running the tests in this repo.\n\n## Connecting to the Jupyterhub server\n\nWe have an EC2 instance running Ubuntu 18.04, with [The Littlest Jupyterhub][tljh] installed. It doesn't\nhave a persistent domain or SSL cert, so we've been connecting to it using an SSH tunnel.\n\nThe current incantation is:\n\n```shell\nssh -A -L 8666:localhost:80 jupyter-protocol@ec2-3-122-216-37.eu-central-1.compute.amazonaws.com\n```\n\nThis will open a shell as the `jupyter-protocol` user, and tunnel traffic from port 80 on the remote\nmachine to port 8666 on localhost.\n\nIf your ssh key isn't authorized, ping @yusefnapora (or someone else with access, if I'm sleeping or something)\nto get added to the `authorized_keys` file.\n\nThe `-A` flag enables ssh-agent forwarding, which will let you pull from this repo while you're shelled in, assuming \nyour SSH key is linked to your github account \u0026 you have read access to this repo. Note that the agent forwarding\ndoesn't seem to work if you're inside a tmux session on the remote host. There's probably a way to\nget it working, but I've just been doing `git pull` outside of tmux.\n\nOnce the tunnel is up, you can go to `http://localhost:8666`, where you'll be asked to sign in. Sign in as\nuser `protocol` with an empty password.\n\n## Server Environment\n\nThere are some things specific to the environment that are worth mentioning.\n\nThe testground daemon is running inside a tmux session owned by the `jupyter-protocol` user. Running `tmux attach`\nwhile shelled in should open it for you - you may have to switch to a different pane - it's generally running in\nthe first pane.\n\nIf for some reason testground isn't running, (e.g. `ps aux | grep testground` comes up empty), you can start the\ndaemon with:\n\n```shell\ntestground --vv daemon\n```\n\nThe testground that's on the `$PATH` is a symlink to `~/repos/testground/testground`, so if you pull in changes\nto testground and rebuild, it should get picked up by the runner scripts, etc automatically.\n\nThis repo is checked out to `~/repos/gossipsub-hardening`, and there's a symlink to it in `~/testground/plans`, so that\nthe daemon can find it and run our plans.\n\n## Cluster setup\n\nThe [testground/infra](https://github.com/testground/infra) repo is checked out at `~/repos/infra`. It contains\nthe scripts for creating and deleting the k8s cluster. The infra README has more detail and some helpful commands,\nbut here are some of the most relevant, plus some things to try if things break.\n\nBefore running any of the commands related to the cluster, you'll need to source some environment vars:\n\n```\nsource ~/k8s-env.sh\n```\n\n\nTo see the current status of the cluster:\n\n```shell\nkops validate cluster\n```\n\nIf that command can't connect to the cluster VMS at all, it either means the cluster has been deleted, \nor you need to export the kubectl config:\n\n```shell\nkops export kubecfg --state $KOPS_STATE_STORE --name=$NAME\n```\n\nIf `kops validate cluster` still can't connect to anything, someone probably deleted the cluster when they were\ndone with it. To create it:\n\n```shell\ncd ~/repos/infra/k8s\n./install.sh cluster.yaml\n```\n\nThis will take a few minutes, and the newly created cluster will only have 4 workers. To resize it:\n\n```shell\nkops edit ig nodes\n```\n\nand edit the `maxSize` and `minSize` params - set both to the desired node count. Then, apply the changes with\n\n```shell\nkops update cluster $NAME --yes\n```\n\nAfter a few minutes, `kops validate cluster` should show all the instances up, and the cluster will be ready.\n\n## Running Tests\n\nEverything in the [main README](./README.md) should apply when running tests on the server, but you can ignore\nthe parts that tell you to run `jupyter notebook` manually.\n\nWhen you log into the Jupyterhub server, you should see a file browser interface. Navigate to \n`repos/gossipsub-hardening/scripts` to open the `Runner.ipynb` notebook.\n\nThere are a bunch of `config-*.json` files next to the runner notebook - these are a good starting point for\nconfiguring test variations - the `config-1k.json` is the baseline test described in the report.\n\nAt the moment, some of the config json files may still be targeting the `feat/hardening` branch and will give an\nerror right away if you run them - change the branch in the Pubsub config panel to `master` and it should be all good.\n\nIf you want to target \"vanilla\" gossipsub (v1.0), you can set the branch to `release-v0.2` and uncheck the\n\"target hardened API\" checkbox in the UI.\n\nAfter a successful run, you should see the path to the analysis notebook printed. Nav there with the Jupyter\nfile browser to run the analysis notebook and generate the charts, etc.\n\n## Troubleshooting\n\nSometimes, especially if you're running with lots of instances, `weave` (the thing that manages the k8s data network)\nwill give up the ghost, and one or more test instances will get stuck and be unable to communicate with the others.\n\nIf you never see the `All networks initialized` message in the testground output, or if it takes several minutes to\nget to `All networks initialized` after all instances are in the Running state, it's likely that you've hit this issue.\n\nIf weave has failed, you may see some weave pods stuck in a \"not ready\" state if you run\n\n```shell\nkubectl validate cluster\n```\n\nYou can try forcibly removing the stuck weaves, although I can't find the magic commands for that at the moment.\nWhat I've been doing instead is scaling the cluster down to a single worker and then back up, to start with a clean\nslate. Scaling down to zero would probably be better, now that I think of it...\n\nIf you do hit the weave issue, you can try lowering the # of connections for the attacker, having fewer attackers,\nor packing the attacker peers less tightly into their containers by adjusting the number of attacker nodes and\nthe number of attack peers per container. Spreading the attackers out over more containers may help, but you may\nalso need to resize the cluster and add more worker VMs. \n\nIf you don't have enough resources, testground will fail you right away and helpfully tell you what the limit it.\n\nYou can control the CPU and RAM allocated for each test container by editing `~/testground/.env.toml` and restarting\nthe testground daemon.\n\n[tljh]: http://tljh.jupyter.org/en/latest/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flibp2p%2Fgossipsub-hardening","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flibp2p%2Fgossipsub-hardening","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flibp2p%2Fgossipsub-hardening/lists"}