{"id":20358763,"url":"https://github.com/weld-project/clamor","last_synced_at":"2025-03-04T17:50:31.726Z","repository":{"id":98920698,"uuid":"424517465","full_name":"weld-project/clamor","owner":"weld-project","description":null,"archived":false,"fork":false,"pushed_at":"2021-11-04T18:00:46.000Z","size":54267,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-01-15T01:46:42.899Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/weld-project.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-04T08:11:31.000Z","updated_at":"2024-05-02T19:48:46.000Z","dependencies_parsed_at":"2023-03-27T13:18:00.554Z","dependency_job_id":null,"html_url":"https://github.com/weld-project/clamor","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weld-project%2Fclamor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weld-project%2Fclamor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weld-project%2Fclamor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weld-project%2Fclamor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/weld-project","download_url":"https://codeload.github.com/weld-project/clamor/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241895432,"owners_count":20038512,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T23:28:48.273Z","updated_at":"2025-03-04T17:50:31.697Z","avatar_url":"https://github.com/weld-project.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Clamor\n\nClamor is a framework that augments parallel data analytics frameworks with\nfine-grained access to large global variables.\nThis allows significant performance improvements for workloads with sparse accesses,\nwhere a Spark-based solution would require broadcasting the entire variable.\n\nClamor was published in SoCC 2021. The [paper](https://dl.acm.org/doi/abs/10.1145/3472883.3486996)\ndescribes Clamor's architecture and demonstrates its performance improvements over comparable frameworks.\n\n## Disclaimer\n\nWe appreciate your interest in using Clamor. This is a research prototype and as such there may be issues\npreventing it from being used out of the box for new applications. We cannot promise to provide extensive\ntechnical support, but we appreciate issues, pull requests, or comments/questions to prthaker@stanford.edu.\n\n## Dependencies\n\nThe script `install.sh` downloads the required repositories and compiles gRPC, Weld, and Clamor.\n\nClamor uses the `spdlog` logging library, which we package with the repository.\n\nClamor depends on the following libraries, which are available through `apt-get`:\n```\nbuild-essential autoconf libtool pkg-config libgflags-dev libgtest-dev clang libc++-dev unzip libcurl4-openssl-dev libboost-dev \n```\n\nWeld (and the Clamor-enabled Rust libraries) additionally require the following:\n```\ncargo llvm-6.0 llvm-6.0-dev clang-6.0 zlib1g-dev\n```\n\nFinally, Clamor uses `grpc` and `protobuf` for RPCs:\n```\ngit clone -b v1.15.1 https://github.com/grpc/grpc\ncd grpc\ngit submodule update --init\ncd third_party/protobuf\n./autogen.sh\n./configure CFLAGS=\"-m64 -fPIC -DNDEBUG\" CXXFLAGS=\"-m64 -fPIC -DNDEBUG\" LDFLAGS=-m64\nmake\nsudo make install\ncd ../../\nmake -j\nsudo make install\ncd ../\n```\nNote the compiler flags for `./configure`.\n\n## Installation\n\nClone Weld from the Clamor fork:\n```\ngit clone https://github.com/pratiksha/weld/tree/clamor-ll\ncd weld/\ngit checkout clamor-ll\nexport WELD_HOME=`pwd`\ncd ..\n```\n\nClone the Clamor repository:\n```\ngit clone https://github.com/pratiksha/clamor\ncd clamor\nexport CLAMOR_HOME=`pwd`\n```\n\nIf running locally with small memory for testing, use the `LOCAL` environment variable\nto allocate a smaller shared address space at compile time:\n```\nexport LOCAL=true\n```\n\nBuild the memory allocation library (needed to link with Weld):\n```\ncd smalloc/\nmake\ncd ../..\n```\n\nBuild Weld:\n```\ncd weld/\ncargo build --release\ncd ..\n```\n\nBuild Clamor:\n```\ncd clamor/clamor\nmake\ncd ..\n```\n\nClamor requires global configuration changes to run:\n```\n./scripts/disable-aslr.sh # disable ASLR\nulimit -s unlimited # disable stack space limits\n```\n\n## Running Clamor\n\nTo build a simple example:\n```\ncd experiments/cpp/simple_sum\nmake\n```\n\nThis example downloads some integers from an Amazon S3 bucket and sums them.\nWe can try to run the simple example locally with a single worker.\n\nTo run the simple example we first need to start the memory manager:\n```\n./simple_sum -i 0.0.0.0 -m manager -r 40000 -l simple_sum.weld -t 1 -w 0.0.0.0\n```\n\nWe then connect a worker:\n```\n./simple_sum -i 0.0.0.0 -m worker -p 50000 -d 0.0.0.0 -l simple_sum.weld\n```\n\nFinally, we can start the driver, which runs the sum program:\n```\n./simple_sum -i 0.0.0.0 -m master -p 70000 -d 0.0.0.0 -l simple_sum.weld -t 1 -w 0.0.0.0\n```\n\nFor a more complex example involving lookups into a large hash table, see `experiments/cpp/hashmap`.\n\n## Clamor cluster management\n\n[Cloister](https://github.com/pratiksha/cloister) provides some rudimentary functionality\nto set up clusters using Amazon EC2 and a script to launch Clamor experiments using Cloister-created clusters.\n\nClamor does not require Cloister to run. Other options for launching clusters,\nsuch as [Flintrock](https://github.com/nchammas/flintrock), may be better maintained.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fweld-project%2Fclamor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fweld-project%2Fclamor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fweld-project%2Fclamor/lists"}