{"id":15145783,"url":"https://github.com/soundcloud/cascading","last_synced_at":"2025-09-29T15:31:03.822Z","repository":{"id":66309413,"uuid":"42986879","full_name":"soundcloud/cascading","owner":"soundcloud","description":"Cascading is a feature rich API for defining and executing complex and fault tolerant data processing workflows on a Hadoop cluster. Please see https://github.com/cwensel/cascading for access to all WIP branches.","archived":false,"fork":true,"pushed_at":"2015-10-28T14:32:07.000Z","size":18353,"stargazers_count":0,"open_issues_count":0,"forks_count":3,"subscribers_count":50,"default_branch":"sc-patches","last_synced_at":"2024-09-27T11:42:42.076Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://www.cascading.org/","language":"Java","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"Cascading/cascading","license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/soundcloud.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.txt","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-09-23T07:59:55.000Z","updated_at":"2021-12-14T09:05:43.000Z","dependencies_parsed_at":"2023-03-10T23:48:36.511Z","dependency_job_id":null,"html_url":"https://github.com/soundcloud/cascading","commit_stats":null,"previous_names":[],"tags_count":95,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soundcloud%2Fcascading","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soundcloud%2Fcascading/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soundcloud%2Fcascading/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soundcloud%2Fcascading/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/soundcloud","download_url":"https://codeload.github.com/soundcloud/cascading/tar.gz/refs/heads/sc-patches","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234635025,"owners_count":18863988,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-26T11:42:50.401Z","updated_at":"2025-09-29T15:31:03.309Z","avatar_url":"https://github.com/soundcloud.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cascading\n\nThanks for using Cascading.\n\n## General Information:\n\nFor project documentation and community support, visit: [cascading.org](http://cascading.org/)\n\nTo download a pre-built distribution, visit [http://cascading.org/downloads/](http://cascading.org/downloads/),\nor use Maven (described below).\n\nThe project includes five Cascading jar files:\n\n* `cascading-core-x.y.z.jar`          - all Cascading Core class files\n* `cascading-xml-x.y.z.jar`           - all Cascading XML operations class files\n* `cascading-local-x.y.z.jar`         - all Cascading Local in-memory mode class files\n* `cascading-hadoop-x.y.z.jar`        - all Cascading Hadoop 1.x MapReduce mode class files\n* `cascading-hadoop2-mr1-x.y.z.jar`   - all Cascading Hadoop 2.x MapReduce mode class files\n\nThese class jars, along with, tests, source and javadoc jars, are all available via the\n[Conjars.org](http://conjars.org) Maven repository.\n\nHadoop 1.x mode is where the Cascading application should run on a Hadoop cluster. Hadoop 2.x is the same\nbut for Hadoop 2.x releases.\n\nLocal mode is where the Cascading application will run locally in memory without any Hadoop dependencies.\n\n## Extensions, the SDK, and DSLs\n\nThere are a number of projects based on Cascading available. Visit the\n[Cascading Extensions](http://cascading.org/extensions/) page for a current list.\n\nOr download the [Cascading SDK](http://cascading.org/sdk/) which includes pre-built binaries.\n\nOf note are three top level projects:\n\n* [Fluid](http://cascading.org/fluid/) - An alternative fluent API for Cascading \n* [Lingual](http://cascading.org/lingual/) - ANSI SQL and JDBC with Cascading\n* [Pattern](http://cascading.org/pattern/) - Machine Learning and [PMML](http://en.wikipedia.org/wiki/Predictive_Model_Markup_Language) support with Cascading\n\nAnd new languages:\n\n* [Scalding](http://cascading.org/projects/scalding/) - A Scala based DSL\n* [Cascalog](http://cascading.org/projects/cascalog/) - A Clojure based DSL\n\n## Versioning\n\nCascading stable releases are always of the form `x.y.z`, where `z` is the current maintenance release.\n\nThe source and tags for all stable releases can be found here:\n[https://github.com/Cascading/cascading](https://github.com/Cascading/cascading)\n\nWIP (work in progress) releases are fully tested builds of code not yet deemed fully stable. On every build by our\ncontinuous integration servers, the WIP build number is increased. Successful builds are then tagged and published.\n\nThe WIP releases are always of the form `x.y.z-wip-n`, where `x.y.z` will be the next stable release version the WIP\nreleases are leading up to. `n` is the current successfully tested build.\n\nThe source, working branches, and tags for all WIP releases can be found here:\n[https://github.com/cwensel/cascading](https://github.com/cwensel/cascading)\n\nOr downloaded from here:\n[http://www.cascading.org/wip/](http://cascading.org/wip/)\n\nWhen a WIP is deemed stable and ready for production use, it will be published as a `x.y.z` release, and made\navailable from the [http://cascading.org/downloads/](http://cascading.org/downloads/) page.\n\n## Reporting issues\n\nTo report an issue, first ping the [Cascading User mailing list](http://cascading.org/support/) with any questions.\n\nIf unresolved, look for a comparable test in the `cascading-platform` sub-project, reproduce your issue, then issue\na pull request with the failing test added to one of the existing suites.\n\nIf you wish to allow your test code to be added to the Cascading test suite, please sign and return this\n[contributor agreement](http://files.concurrentinc.com/agreements/Concurrent_Contributor_Agreement.doc).\n\n## Using with Maven/Ivy\n\nIt is strongly recommended developers pull Cascading from our Maven compatible jar repository\n[Conjars.org](http://conjars.org).\n\nYou can find the latest public and WIP (work in progress) releases here:\n\n*  http://conjars.org/cascading/cascading-core\n*  http://conjars.org/cascading/cascading-local\n*  http://conjars.org/cascading/cascading-hadoop\n*  http://conjars.org/cascading/cascading-hadoop2-mr1\n*  http://conjars.org/cascading/cascading-xml\n\nWhen creating tests, make sure to add any of the relevant above dependencies to your `test` scope or equivalent\nconfiguration along with the `cascading-platform` dependency.\n\n*  http://conjars.org/cascading/cascading-platform\n\nNote the `cascading-platform` compile dependency has no classes, you must pull the tests dependency with the\n`tests` classifier.\n\nSee [http://cascading.org/downloads/#maven](http://cascading.org/downloads/#maven) for example Maven pom\ndependency settings.\n\nSource and Javadoc artifacts (using the appropriate classifier) are also available through Conjars.\n\nNote that `cascading-hadoop` and `cascading-hadoop2-mr1` has a `provided` dependency on the Hadoop jars so that it\nwon't get sucked into any application packaging as a dependency, typically.\n\n## Building and IDE Integration\n\nFor most cases, building Cascading is unnecessary as it has been pre-built, tested, and published to our Maven\nrepository (above).\n\nTo build Cascading, run the following in the shell:\n\n```bash\n\u003e git clone https://github.com/cascading/cascading.git\n\u003e cd cascading\n\u003e gradle build\n```\n\nCascading requires Gradle 1.x to build.\n\nTo use an IDE like IntelliJ, run the following to create IntelliJ project files:\n\n```bash\n\u003e gradle idea\n```\n\nSimilarly for Eclipse:\n\n```bash\n\u003e gradle eclipse\n```\n\n## Using with Apache Hadoop\n\nFirst confirm you are using a supported version of Apache Hadoop by checking the\n[Compatibility](http://cascading.org/support/compatibility/) page.\n\nTo use Cascading with Hadoop, we suggest stuffing `cascading-core`, `cascading-hadoop`, (optionally) `cascading-xml`\njar files and all third-party libs into the `lib` folder of your job jar and executing your job via\n`$HADOOP_HOME/bin/hadoop jar your.jar \u003cyour args\u003e`.\n\nFor example, your job jar would look like this (via: `jar -t your.jar`)\n\n```bash\n/\u003call your class and resource files\u003e\n/lib/cascading-core-x.y.z.jar\n/lib/cascading-hadoop-x.y.z.jar\n/lib/cascading-xml-x.y.z.jar\n/lib/\u003ccascading third-party jar files\u003e\n```\n\nHadoop will unpack the jar locally and remotely (in the cluster) and add any libraries in `lib` to the classpath. This\nis a feature specific to Hadoop.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoundcloud%2Fcascading","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsoundcloud%2Fcascading","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoundcloud%2Fcascading/lists"}