{"id":20875405,"url":"https://github.com/ljdursi/beyond-single-core-r","last_synced_at":"2025-05-12T15:31:31.723Z","repository":{"id":84104096,"uuid":"81374075","full_name":"ljdursi/beyond-single-core-R","owner":"ljdursi","description":"Short tour of parallel and foreach packages, and how to think about scaling data analyses","archived":false,"fork":false,"pushed_at":"2020-08-23T01:34:12.000Z","size":84547,"stargazers_count":75,"open_issues_count":0,"forks_count":14,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-07-21T07:32:41.628Z","etag":null,"topics":["parallel","parallel-computing","r","scalability"],"latest_commit_sha":null,"homepage":"https://ljdursi.github.io/beyond-single-core-R","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ljdursi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-02-08T20:43:28.000Z","updated_at":"2024-01-04T16:11:15.000Z","dependencies_parsed_at":"2023-07-18T15:31:05.194Z","dependency_job_id":null,"html_url":"https://github.com/ljdursi/beyond-single-core-R","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ljdursi%2Fbeyond-single-core-R","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ljdursi%2Fbeyond-single-core-R/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ljdursi%2Fbeyond-single-core-R/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ljdursi%2Fbeyond-single-core-R/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ljdursi","download_url":"https://codeload.github.com/ljdursi/beyond-single-core-R/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225140621,"owners_count":17427139,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["parallel","parallel-computing","r","scalability"],"created_at":"2024-11-18T06:45:02.657Z","updated_at":"2024-11-18T06:45:03.512Z","avatar_url":"https://github.com/ljdursi.png","language":"R","funding_links":[],"categories":[],"sub_categories":[],"readme":"Beyond Single Core: Parallel Analysis in R\n===================\n\nR is a great environment for interactive analysis on your desktop, but when your data needs outgrow your \npersonal computer, it's not clear what to do next.\n\nThis is material for a short overview of scalable data analysis in R.  The slides can be viewed at https://ljdursi.github.io/beyond-single-core-R .\n\nIt covers:\n\n* How to think about parallelism and scalability in data analysis\n* The standard parallel package, including what was the snow and multicore facilities, \nusing [airline data](http://stat-computing.org/dataexpo/2009/the-data.html) as an example\n* The [foreach](http://cran.r-project.org/web/packages/foreach/index.html) package, using \nairline data and simple stock data;\n* A summary of best practices.\n\nIncluded in the materials, though not in the talk, are some more advanced methods:\n* The [bigmemory](http://cran.r-project.org/web/packages/bigmemory/index.html) package for out-of-core computation on large data matrices, with a simple physical sciences example;\n* The [Rdsm](http://cran.r-project.org/web/packages/Rdsm/index.html) package for shared memory; and\n* a brief introduction to the powerful [pbdR](http://r-pbd.org) pacakges for extremely large-scale computation.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fljdursi%2Fbeyond-single-core-r","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fljdursi%2Fbeyond-single-core-r","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fljdursi%2Fbeyond-single-core-r/lists"}