{"id":18649818,"url":"https://github.com/ashwanthkumar/mrcube","last_synced_at":"2025-11-05T10:30:31.948Z","repository":{"id":10186456,"uuid":"12274827","full_name":"ashwanthkumar/mrcube","owner":"ashwanthkumar","description":"Scalding CUBE operators","archived":false,"fork":false,"pushed_at":"2015-01-04T12:27:12.000Z","size":184,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-12-27T12:44:50.080Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ashwanthkumar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-08-21T16:16:54.000Z","updated_at":"2015-01-06T11:29:17.000Z","dependencies_parsed_at":"2022-09-12T21:24:15.238Z","dependency_job_id":null,"html_url":"https://github.com/ashwanthkumar/mrcube","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashwanthkumar%2Fmrcube","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashwanthkumar%2Fmrcube/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashwanthkumar%2Fmrcube/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashwanthkumar%2Fmrcube/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ashwanthkumar","download_url":"https://codeload.github.com/ashwanthkumar/mrcube/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239456409,"owners_count":19641843,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T06:40:45.312Z","updated_at":"2025-11-05T10:30:31.897Z","avatar_url":"https://github.com/ashwanthkumar.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Build Status](https://snap-ci.com/ashwanthkumar/mrcube/branch/master/build_image)](https://snap-ci.com/ashwanthkumar/mrcube/branch/master)\n\n# Scalding MRCube\n\nScalding CUBE operators\n\nNaive `cubify` and `rollup` methods on richPipe.\n\nFor each Input Tuple\n\n- cubify generates 2^n tuples, where n is the number of fields we are cubing on\n- rollup generates n+1 tuples, where n is the number of fields we are rolling up on\n\n### Dev\n```bash\n$ git clone https://github.com/ashwanthkumar/mrcube.git\n$ sbt test\n```\n\n### Dependencies\nFor Maven,\n```xml\n\u003cdependency\u003e\n  \u003cgroupId\u003ein.ashwanthkumar\u003c/groupId\u003e\n  \u003cartifactId\u003emrcube_2.10\u003c/artifactId\u003e\n  \u003cversion\u003e0.12.0\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\nFor SBT,\n```sbt\nlibraryDependencies += \"in.ashwanthkumar\" %% \"mrcube\" % \"0.12.0\"\n```\n\n### Cubify\n\nIf the input tuple is  ``(\"ipod\", \"miami\", \"2012\", \"200000\")`` the output generated from the job is\n\n```\n(\"ipod\", \"miami\", \"2012\", \"1\", \"200000.0\")\n(\"ipod\", \"miami\", \"null\", \"1\", \"200000.0\")\n(\"ipod\", \"null\", \"null\", \"1\", \"200000.0\")\n(\"ipod\", \"null\", \"2012\", \"1\", \"200000.0\")\n(\"null\", \"null\", \"2012\", \"1\", \"200000.0\")\n(\"null\", \"miami\", \"null\", \"1\", \"200000.0\")\n(\"null\", \"miami\", \"2012\", \"1\", \"200000.0\")\n(\"null\", \"null\", \"null\", \"1\", \"200000.0\")\n```\n\nInstead of \"null\" you can pass in another custom string to cubify.\n\n```scala\nimport in.ashwanthkumar.mrcube._\n\nclass CubifyJob(args: Args) extends Job(args) {\n\n  Csv(args(\"input\"), fields = ('product, 'location, 'year, 'sales))\n    .read\n    .cubify(('product, 'location, 'year))\n    .groupBy('product, 'location, 'year) { _.size('size).sum[Int]('sales) }\n    .write(Csv(args(\"output\")))\n}\n```\n\n\n### Rollup\n\nIf the input tuple is  ``(\"ipod\", \"miami\", \"2012\", \"200000\")`` the output generated from the job is\n\n```\n(\"ipod\", \"miami\", \"2012\", \"1\", \"200000.0\")\n(\"null\", \"miami\", \"2012\", \"1\", \"200000.0\")\n(\"null\", \"null\", \"2012\", \"1\", \"200000.0\")\n(\"null\", \"null\", \"null\", \"1\", \"200000.0\")\n```\n\nSimilarly instead of \"null\" you can pass in another custom string to rollup.\n\n```scala\nimport in.ashwanthkumar.mrcube._\n\nclass RollupJob(args: Args) extends Job(args) {\n  Csv(args(\"input\"), fields = ('product, 'location, 'year, 'sales))\n    .read\n    .rollup(('product, 'location, 'year))\n    .groupBy('product, 'location, 'year) { _.size('size).sum[Int]('sales) }\n    .write(Csv(args(\"output\")))\n\n}\n```\n\n### References\n\n1. [Distributed Cube Materialization on Holistic Measures](http://arnab.org/files/mrcube.pdf) by [Dr. Arnam Nandi](http://arnab.org/) et. al\n2. CUBE Operator in Pig - [PIG 2167](https://issues.apache.org/jira/browse/PIG-2167)\n\n### License\nLicensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fashwanthkumar%2Fmrcube","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fashwanthkumar%2Fmrcube","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fashwanthkumar%2Fmrcube/lists"}