{"id":50878345,"url":"https://github.com/radlab/SCADS","last_synced_at":"2026-07-03T02:01:15.879Z","repository":{"id":863249,"uuid":"599568","full_name":"radlab/SCADS","owner":"radlab","description":"Scalable Consistency Adjustable Data Storage","archived":false,"fork":false,"pushed_at":"2020-10-13T09:49:28.000Z","size":25870,"stargazers_count":45,"open_issues_count":22,"forks_count":10,"subscribers_count":23,"default_branch":"v2.1.3","last_synced_at":"2024-03-25T23:07:51.858Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/radlab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2010-04-07T20:29:31.000Z","updated_at":"2024-03-25T23:07:51.859Z","dependencies_parsed_at":"2022-08-16T11:15:12.242Z","dependency_job_id":null,"html_url":"https://github.com/radlab/SCADS","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/radlab/SCADS","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radlab%2FSCADS","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radlab%2FSCADS/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radlab%2FSCADS/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radlab%2FSCADS/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/radlab","download_url":"https://codeload.github.com/radlab/SCADS/tar.gz/refs/heads/v2.1.3","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radlab%2FSCADS/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35069183,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-03T02:00:05.635Z","response_time":110,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-15T12:00:25.787Z","updated_at":"2026-07-03T02:01:15.866Z","avatar_url":"https://github.com/radlab.png","language":"Scala","funding_links":[],"categories":["大数据"],"sub_categories":["微服务框架"],"readme":"SCADS (Scalable Consistency Adjustable Data Storage) is a research prototype distributed storage system used in the [RAD Lab](http://radlab.cs.berkeley.edu/) and the [AMP Lab](https://amplab.cs.berkeley.edu/) at [UC Berkeley](http://berkeley.edu/). The goals of the system were first described in [our vision paper from CIDR2009](http://radlab.cs.berkeley.edu/publication/185)\n\nSCADS Sub-projects\n=================\nSCADS is composed of the following sub-projects:\n\nSCADS Core\n----------\n* [config](https://github.com/radlab/SCADS/wiki/SCADS-Config) - Configuration file parsing. [scaladoc](http://radlab.github.com/SCADS/config/latest/api/) [source](http://radlab.github.com/SCADS/config/latest/sxr/)\n* [avro-plugin](https://github.com/radlab/SCADS/wiki/Avro-Plugin) - A scala compiler plugin allowing case classes to be efficiently serialized using [Avro](http://avro.apache.org/) encoding. [scaladoc](http://radlab.github.com/SCADS/avro-plugin/latest/api/) [source](http://radlab.github.com/SCADS/avro-plugin/latest/sxr/)\n* [deploylib](https://github.com/radlab/SCADS/wiki/Deploylib) - A parallel ssh library for deploying jvm-based experiments on remote servers and EC2. [scaladoc](http://radlab.github.com/SCADS/deploylib/latest/api/) [source](http://radlab.github.com/SCADS/deploylib/latest/sxr/)\n* communication - [Netty](http://www.jboss.org/netty)-based message passing of Avro encoded messages.  Scala library for using [Apache ZooKeeper](http://zookeeper.apache.org/). [scaladoc](http://radlab.github.com/SCADS/modeling/latest/api/) [source](http://radlab.github.com/SCADS/modeling/latest/sxr/)\n* scala-engine - K/V storage optionally using [BDB](http://www.oracle.com/technetwork/database/berkeleydb/overview/index.html) for persistence. [scaladoc](http://radlab.github.com/SCADS/scala-engine/latest/api/) [source](http://radlab.github.com/SCADS/scala-engine/latest/sxr/)\n* [piql](https://github.com/radlab/SCADS/wiki/PIQL) - The performance-insightful query language (PIQL) and scale-independent relational optimizer. [scaladoc](http://radlab.github.com/SCADS/piql/latest/api/) [source](http://radlab.github.com/SCADS/piql/latest/sxr/)\n* perf - A library of helpers for writing SCADS performance tests using deploylib. [scaladoc](http://radlab.github.com/SCADS/perf/latest/api/) [source](http://radlab.github.com/SCADS/perf/latest/sxr/)\n\nOther Experiments\n-----------------\n* scadr - PIQL queries for a clone of [Twitter](http://www.twitter.com). [scaladoc](http://radlab.github.com/SCADS/scadr/latest/api/) [source](http://radlab.github.com/SCADS/scadr/latest/sxr/)\n* matheon = [scaladoc](http://radlab.github.com/SCADS/matheon/latest/api/) [source](http://radlab.github.com/SCADS/matheon/latest/sxr/)\n* tpcw - PIQL queries for the [TPC-W](http://www.tpc.org/tpcw/) benchmark. [scaladoc](http://radlab.github.com/SCADS/tpcw/latest/api/) [source](http://radlab.github.com/SCADS/tpcw/latest/sxr/)\n* axer - Alternate Avro encoding optimized for random field access. [scaladoc](http://radlab.github.com/SCADS/axer/latest/api/) [source](http://radlab.github.com/SCADS/axer/latest/sxr/)\n* modeling - experiments to characterize the performance of PIQL queries. [scaladoc](http://radlab.github.com/SCADS/modeling/latest/api/) [source](http://radlab.github.com/SCADS/modeling/latest/sxr/)\n\nDeprecated Projects\n-------------------\nThe following sub-projects are no longer actively maintained:\n\n* [demo](https://github.com/radlab/SCADS/wiki/Demo) - The [RAD Lab final demo](http://radlab.cs.berkeley.edu/media-news/345) used SCADS along with other projects from the RAD Lab to scale web applications written by novice developers over a weekend to hundreds of servers on Amazon EC2.\n* director - The director ensures SLO compliance for storage operations by using machine learning models to dynamically re-provision a SCADS storage cluster based on current and projected workload.  More details can be found in the paper from [FAST2011](http://www.eecs.berkeley.edu/~franklin/Papers/fast11director.pdf).\n\nThird Party Components\n----------------------\n* optional - The optional command line parsing library from [paulp](https://github.com/paulp/optional) with added support for default arguments.\n\nBuilding\n========\nSCADS is built using [SBT](https://github.com/harrah/xsbt).  The SBT launcher is included in the distribution (bin/sbt) and is responsible for downloading all other required jars (scala library/compiler and dependencies).\n\nSBT commands can be invoked from the command line.  For example, to clean and build jar files for the entire SCADS project, you would run the following command:\n\n    scads/$ sbt clean package\n\nYou can also execute commands on specific sub-projects by specifying `\u003csubproject\u003e/\u003ccommand\u003e`.  For example:\n\n    scads/$ sbt piql/compile\n\nAdditionally, if you are going to be running several commands, you can use SBT from an interactive console, which amortizes the cost of starting the JVM and JITing SBT and the scala compiler.  For example:\n\n    scads\u003e sbt\n    [info] Loading project definition from /Users/marmbrus/Workspace/radlab/scads/project\n    [info] Set current project to scads (in build file:/Users/marmbrus/Workspace/radlab/scads/)\n    scads:sbt09\u003e project modeling\n    [info] Set current project to modeling (in build file:/Users/marmbrus/Workspace/radlab/scads/)\n    modeling:sbt09\u003e compile\n    [success] Total time: 7 s, completed Sep 11, 2011 5:23:28 PM\n\nUseful Command Reference\n------------------------\n* `clean` - delete generated files\n* `compile` - build the current project and all its dependencies\n* `doc` - compile scalado\n* `gen-idea` - build a project for use in [InteliJ](http://www.jetbrains.com/idea/)\n* `ghpages:push-api-doc` - update the api documentation hosted on github pages\n* `package` - create the jar or war file for the current project\n* `project \u003csubproject name\u003e` - switch to the specified sub-project; subsequent commands will be run only on this project and its dependencies\n* `projects` - list all of the available sub-projects\n* `publish` - publish jars to the radlab nexus repository\n* `publish-local` - publish jars to your local ivy cache\n* `reload` - recompiles the project definition and restarts SBT\n* `run` - run the mainclass for the current sub-project; if there are multiple choices you will be prompted for which one you want to run\n* `test` - run the testcases for the current sub-project and all its dependencies\n* `test-only [test case]` - run the specified testcase or only the testcases whose source has changed since tests were last run\n* `update` - download all managed dependencies jars; note that in contrast to maven, this must be run explicitly.  This only needs to be run once unless dependencies have been added to the project.\n\nDeploy Console\n--------------\nRunning `deploy-console` in SBT brings up a scala console containing the deploylib environment for the current project.  The deploylib environment contains all jar files necessary for running the current project on a remote machine.  For example, to run [the experiment comparing two plans for the PIQL intersection query](https://github.com/radlab/SCADS/blob/sbt09/experiments/modeling/src/main/scala/experiments/PlanCompare.scala) on EC2:\n\n    scads/$ sbt modeling/deploy-console\n    [info] Loading project definition from /Users/marmbrus/Workspace/radlab/scads/project\n    [info] Set current project to scads (in build file:/Users/marmbrus/Workspace/radlab/scads/)\n    import deploylib._\n    import deploylib.ec2._\n    allJars: Seq[java.io.File] = List(/Users/marmbrus/Workspace/radlab/scads/piql/scadr/target/scala-2.9.1/scadr_2.9.1-2.1.2-SNAPSHOT.jar...\n    Welcome to Scala version 2.9.1.final (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_24).\n    Type in expressions to have them evaluated.\n    Type :help for more information.\n\n    scala\u003e import edu.berkeley.cs.scads.piql.modeling.Experiments._\n    scala\u003e import edu.berkeley.cs.scads.piql.modeling.PlanCompare._\n    scala\u003e cluster.setup(numSlaves = 3)\n    scala\u003e run\n    \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fradlab%2FSCADS","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fradlab%2FSCADS","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fradlab%2FSCADS/lists"}