{"id":19986393,"url":"https://github.com/sourcec0de/mortar-examples","last_synced_at":"2025-05-04T07:31:16.143Z","repository":{"id":8927427,"uuid":"10657195","full_name":"sourcec0de/mortar-examples","owner":"sourcec0de","description":"Mortar Project with examples for several different public data sets and data types/formats","archived":false,"fork":false,"pushed_at":"2013-06-10T18:45:50.000Z","size":285,"stargazers_count":0,"open_issues_count":0,"forks_count":33,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-04-15T03:21:06.605Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://help.mortardata.com/","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sourcec0de.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-06-13T02:32:58.000Z","updated_at":"2019-11-12T14:57:04.000Z","dependencies_parsed_at":"2022-08-27T01:52:01.243Z","dependency_job_id":null,"html_url":"https://github.com/sourcec0de/mortar-examples","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcec0de%2Fmortar-examples","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcec0de%2Fmortar-examples/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcec0de%2Fmortar-examples/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcec0de%2Fmortar-examples/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sourcec0de","download_url":"https://codeload.github.com/sourcec0de/mortar-examples/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224390011,"owners_count":17303438,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T04:29:01.005Z","updated_at":"2024-11-13T04:29:01.612Z","avatar_url":"https://github.com/sourcec0de.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Welcome to Mortar!\n\nMortar is a platform-as-a-service for Hadoop.  With Mortar, you can run jobs on Hadoop using Apache Pig and Python without any special training.  \n\n## Getting Started\n\nHere we've included some example scripts that explore public data sets. To start using them:\n\n1. [Signup for a Mortar account](https://app.mortardata.com/signup)\n1. [Install the Mortar Development Framework](http://help.mortardata.com/#!/install_mortar_development_framework)\n1.  Clone this repository to your computer and register it as a project with Mortar:\n\n        git clone git@github.com:mortardata/mortar-examples.git\n        cd mortar-examples\n        mortar register mortar-examples\n\nOnce you've setup the project, use the `mortar illustrate` command to show data flowing through a given script.  Use `mortar run` to run the script on a Hadoop cluster.\n\nFor lots more help and tutorials on running Mortar, check out the [Mortar Help](http://help.mortardata.com/) site.\n\n## Examples\n\n### airline_travel: CSV data from Bureau of Labor Statistics\n\nThe [airline_travel](https://github.com/mortardata/mortar-examples/blob/master/pigscripts/airline_travel.pig) pigscript takes data from the [Bureau of Transportation Statistics](http://www.transtats.bts.gov/DL_SelectFields.asp?Table_ID=236) and uses it to find out how airlines perform when we normalize for the airports they fly from and to.\n\n### coffee_tweets: JSON data from Twitter\n\nThe [coffee_tweets](https://github.com/mortardata/mortar-examples/blob/master/pigscripts/coffee_tweets.pig) pigscript answers the question \"Which US state contains the highest concentration of coffee snobs?\".  It analyzes and aggregates twitter data from the [twitter-gardenhose](https://github.com/mortardata/twitter-gardenhose), looking for telltale signs of coffee snobbery in tweets.\n\n### common_crawl_trending_topics: Dataset of technology news webpages taken from the Common Crawl\n\nThe [common_crawl_trending_topics](https://github.com/mortardata/mortar-examples/blob/master/pigscripts/common_crawl_trending_topics.pig) pigscript finds single-word trending topics by month from a corpus of technology news webpages (techcrunch, gigaom, and allthingsd). It does\nthis by calculating the frequency of each word in each month, finding the \"frequency velocity\" from month to month, and selecting the words \nwith the highest frequency velocity in each month.\n\n### excite: Search log data from excite! search engine\n\nThe [excite](https://github.com/mortardata/mortar-examples/blob/master/pigscripts/excite.pig) pigscript shows an example of loading search engine logs from the excite! search engine and joining them up to a users table.  This is a common pattern for web log analysis.\n\n### millionsong: Million song dataset\n\nTwo pigscripts explore the publicly-available [Million Song Dataset](http://labrosa.ee.columbia.edu/millionsong/pages/field-list).\n\nThe first, [top_density_songs](https://github.com/mortardata/mortar-examples/blob/master/pigscripts/top_density_songs.pig) finds the songs with the most beats per second in the 1MM song dataset.  Code to ***REALLY FAST*** music!\n\nThe second, [hottest_song_of_the_decade](https://github.com/mortardata/mortar-examples/blob/master/pigscripts/hottest_song_of_the_decade.pig) figures out which song is the hottest for each decade of data in the million song dataset.\n\n### nasa_logs: Apache logs from NASA\n\nThe [nasa_logs](https://github.com/mortardata/mortar-examples/blob/master/pigscripts/nasa_logs.pig) pigscript is an example of parsing Apache logs to find the most-served resources by date. It takes a sample of two month's worth of logs from \nNASA Kennedy Space Center's web server in 1995 and finds for each date the number of requests served, the number of bytes served, and\nthe top 10 resources served (images are filtered out since most of the requests are just for icons). It can take a parameter\nORDERING equal to either 'num_requests', to rank resources by the number of requests served, or 'num_bytes', to rank resources\nby number of bytes served.\n\n### twitter_sentiment: JSON data from Twitter\n\nThe [twitter_sentiment](https://github.com/mortardata/mortar-examples/blob/master/pigscripts/twitter_sentiment.pig) pigscript finds which words are most likely to appear in tweets expressing a \"postive sentiment\" and which words are most likely to appear in tweets expressing a \"negative sentiment\". It calculates these likelihoods by looking at the frequency of a word in the corpus of positive/negative tweets diveded by the frequency of that word in the corpus of all processed tweets. The words that cause tweets to be classified as positive/negative (ex. \"awesome\", \"disappointing\") in the first place are excluded from the associations, so you can see what caused the sentiments instead of the sentiments themselves. The tweets are taken from the [twitter-gardenhose](https://github.com/mortardata/twitter-gardenhose).\n\n## Advanced Examples\n\n### Twitter Pagerank\n\nA separate Mortar project, [twitter-pagerank](https://github.com/mortardata/twitter-pagerank) shows how to embed Pig \nin a Jython controlscript.  This project runs Pagerank, an algorithm that uses several iteration steps, on a subset of \nthe Twitter follower graph. The result is a list of who influential people on Twitter tend to follow. \nThere is a [tutorial](http://help.mortardata.com/tutorials/git_projects/working_with_iterative_algorithms) \non the Mortar help site which walks through the twitter-pagerank project.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsourcec0de%2Fmortar-examples","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsourcec0de%2Fmortar-examples","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsourcec0de%2Fmortar-examples/lists"}