{"id":23072477,"url":"https://github.com/doches/clusterfuck","last_synced_at":"2025-10-11T14:49:57.483Z","repository":{"id":694415,"uuid":"338934","full_name":"doches/clusterfuck","owner":"doches","description":"A subversive distributed systems tool","archived":false,"fork":false,"pushed_at":"2010-09-06T13:30:51.000Z","size":108,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-10-02T23:21:07.952Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/doches.png","metadata":{"files":{"readme":"README.rdoc","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2009-10-15T22:33:32.000Z","updated_at":"2015-01-24T05:46:44.000Z","dependencies_parsed_at":"2022-07-07T13:42:05.826Z","dependency_job_id":null,"html_url":"https://github.com/doches/clusterfuck","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/doches/clusterfuck","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/doches%2Fclusterfuck","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/doches%2Fclusterfuck/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/doches%2Fclusterfuck/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/doches%2Fclusterfuck/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/doches","download_url":"https://codeload.github.com/doches/clusterfuck/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/doches%2Fclusterfuck/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278526239,"owners_count":26001325,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-05T02:00:06.059Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-16T07:19:55.925Z","updated_at":"2025-10-11T14:49:57.468Z","avatar_url":"https://github.com/doches.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"= Clusterfuck\n==== A Subversive Distributed-Systems Tool\n\nClusterfuck is a tool for automating the process of SSH-ing into remote machines and kickstarting a large number\nof jobs. It's probably best explained by an example, so here's what I use it for:\n\nAs part of my research I need to compute the distance between each pair of objects in a set of about 70,000 items.\nComputing the distance between each pair takes a few seconds; running the entire job on a single machine generally takes over a day.\nHowever, as a member of the University I have a ssh login that works on quite a few machines, so I found myself breaking the job up into smaller, quicker chunks and running each chunk on a different machine.\nClusterfuck was born out of my frustration with that method -- \"surely,\" I said to myself, \"this can be automated.\"\n\nIf you have a lot of jobs to run and access to multiple machines on which to run them, Clusterfuck is for you!\n\n== Usage\nTo use Clusterfuck you'll first need to create a configuration file (a \"clusterfile\"). An example clusterfile might look something like this:\n  \n  Clusterfuck::Task.new do |task|\n    task.hosts = %w{clark asimov}\n    task.jobs = (0..3).map { |x| Clusterfuck::Job.new(\"host{x}\",\"sleep 0.5 \u0026\u0026 hostname\") }\n    task.temp = \"fragments\"\n    task.username = \"SSHUSERNAME\"\n    task.password = \"SSHPASSWORD\"\n    task.debug = true\n  end\n\nThis creates a new clusterfuck task and distributes the jobs across two hosts, +clark+ and +asimov+. \nThe jobs to be run in this case are pretty trivial; we basically ssh into each machine, sleep for a little bit, then get the hostname.\nWhatever each job prints to stdout is saved in +task+.+temp+ (under the current working directory); running\nthis clusterfile will create 4 files in \u003ccode\u003e./fragments/\u003c/code\u003e: host0.[hostname], host1.[hostname], host2.[hostname], and host3.[hostname] (where [hostname] is the name of the machine on which the job was run). \n+task+.+username+ and +task+.+password+ are the SSH credentials used to log into the maching -- currently, Clusterfuck\ncan only use one global set of credentials. There's no technical reason for this, other than the fact that I don't\nreally need to use machine-specific logins, so it'll probably appear in future releases.\n+task+.+verbose+ turns on verbose output (messages to stdout each time a job is started, skipped, or canceled).\n\nOnce you have a clusterfile you can kick off your jobs by running the command +clusterfuck+ in the same directory.\n\n== Note on Patches/Pull Requests\n \n* Fork the project.\n* Add something cool or fix a nefarious bug. Documentation wins extra love.\n* Add tests for it. I'd really like this, but since I haven't written any tests myself yet I can't really blame you if you skip it...\n* Commit, but do not mess with rakefile, version, or history.\n  (if you want to have your own version that's ok -- but\n  bump the version in a separate commit that I can ignore when I pull)\n* Send me a pull request. \n\n== Copyright\n\nCopyright (c) 2009 Trevor Fountain. See LICENSE for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdoches%2Fclusterfuck","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdoches%2Fclusterfuck","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdoches%2Fclusterfuck/lists"}