{"id":16572212,"url":"https://github.com/eddelbuettel/ldlasb","last_synced_at":"2025-10-29T01:31:52.186Z","repository":{"id":47128908,"uuid":"390865194","full_name":"eddelbuettel/ldlasb","owner":"eddelbuettel","description":"Lies, Damned Lies, and Selective Benchmarks","archived":false,"fork":false,"pushed_at":"2021-08-01T13:20:39.000Z","size":72,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-01T21:05:33.280Z","etag":null,"topics":["benchmarking","performance","r","r-package","rcpp"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eddelbuettel.png","metadata":{"files":{"readme":"README.md","changelog":"ChangeLog","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-07-29T22:29:20.000Z","updated_at":"2024-01-03T19:41:50.000Z","dependencies_parsed_at":"2022-09-04T02:41:37.698Z","dependency_job_id":null,"html_url":"https://github.com/eddelbuettel/ldlasb","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eddelbuettel%2Fldlasb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eddelbuettel%2Fldlasb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eddelbuettel%2Fldlasb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eddelbuettel%2Fldlasb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eddelbuettel","download_url":"https://codeload.github.com/eddelbuettel/ldlasb/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238758171,"owners_count":19525721,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmarking","performance","r","r-package","rcpp"],"created_at":"2024-10-11T21:26:47.772Z","updated_at":"2025-10-29T01:31:51.816Z","avatar_url":"https://github.com/eddelbuettel.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Lies, Damned Lies and Selective Benchmarking\n\nThis repository contains a short package providing two benchmarks between [Rcpp][1] and [another package][2] on [CRAN][3] which we will refer to just as just 'other'.\n\nThe 'other' package comes with a motivating vignette containing a lot of discussion.\nIts merits (\"is another package needed\", \"is it better to restart fresh or extend an existing package\", etc) may well be discussed another time.\nBut it also contains two benchmarks ([one](https://cran.r-project.org/package=cpp11/vignettes/motivations.html#protection) and [two](https://cran.r-project.org/package=cpp11/vignettes/motivations.html#growing-vectors)).\n\nThis note is about these two benchmarks.\n\nOne is simply outdated (and even then somewhat cheekily chosen).\nThe other is so obviously misleading that it alone merited this note and code repository.\n\n\n### Outdated Comparison\n\nThe first of these two covers how objects are protected, and released, internally.\n[Rcpp][1] initially used simple helper functions (modeled after what R does at the C level) to set (and unset) a protection bit.\nThis was later reworked, and a further change was filed as a _desiderata_ years ago (in [issue #382](https://github.com/RcppCore/Rcpp/issues/382)).\nA helpful and detailed bug report stemming from inserting tens of thousands of _grobs_ into a plot showed that with very large number of objects, performance degraded, even crashing R.\nThis lead to a useful discussion (in [issue #1018](https://github.com/RcppCore/Rcpp/issues/1081)) in which for example R Core member Luke Tierney suggested an improved method (with O(1) performance) for this problem.\nAn initial implementation revealed side-effects and was held back from the next release.\nHowever, it has since been completed (in PRs [#1133](https://github.com/RcppCore/Rcpp/pull/1133) and [#1135](https://github.com/RcppCore/Rcpp/pull/1135) added in January 2021), and is part of the [Rcpp][1] release 1.0.7 on [CRAN][2].\n\nThe 'other' package freshly implemented the O(1) method and then used it for a benchmark.\n\nThat is a little, well, \"obviously\" dented in its favor but so be it.\nIn any event, it is now outdated.  \n\nSo running `demo(release, package=\"ldlasb\")` re-runs the benchmark (based on the functions in `src/release.cpp`) under the current [Rcpp][1] version and shows, lo and behold, that [Rcpp][1] is consistently faster---while both perform in the promised O(1) manner.\n\n![](inst/figures/release.png)\n\n\n### \"Interesting\" Comparison\n\nSince the very beginning of [Rcpp][1], it has been _clearly_ and _repeatedly_ pointed out that `push_back()` and `push_front()` were added to complete to the API with STL-alike behavior as a convenience---but one that that should not be used for real code.\nThis has been stated loud and clear in many places:\nstarting with the [original announcement in Feb 2010](https://lists.r-forge.r-project.org/pipermail/rcpp-devel/2010-February/000410.html),\nseveral StackOverflow answers in [Dec 2011](https://stackoverflow.com/a/8631853/143305) and in [Dec 2012](https://stackoverflow.com/a/13783044/143305),\na post in the the rcpp-devel list in [Jun 2013](https://lists.r-forge.r-project.org/pipermail/rcpp-devel/2013-June/006078.html),\nanother StackOverflow answer in [Nov 2013](https://stackoverflow.com/a/19829440/143305),\nan early Rcpp Gallery post in [Dec 2013](https://gallery.rcpp.org/articles/plyr-c-to-rcpp/),\nagain on StackOverflow [Dec 2014](https://stackoverflow.com/a/27585789/143305), \nas well as in the 'Advanced R' [first](http://adv-r.had.co.nz/Rcpp.html#stl) and [second](https://adv-r.hadley.nz/rcpp.html#stl) editions.\n\nFor emphasis, here is a quote from the [rcpp-devel post in 2013](https://lists.r-forge.r-project.org/pipermail/rcpp-devel/2013-June/006078.html):\n\n\u003e Those are somehow cosmetic additions. The usual suggestion is not to\n\u003e use push_front and push_back on Rcpp types.\n\u003e\n\u003e We use R's memory, and in R, resizing a vector means moving the data.\n\u003e So if you push_back 3 times, you're moving the data 3 times.\n\u003e\n\u003e Using R own memory is the best ever decision we made in Rcpp. You can\n\u003e always use your own data structures to accumulate data, perhaps using\n\u003e stl types and then convert back to R types, which is something we make\n\u003e easy to do.\n\nMany code examples and packages show exactly that approach (as _e.g._ discussed\nin the [Rcpp Gallery post](https://gallery.rcpp.org/articles/plyr-c-to-rcpp/)).\nAnybody who claims otherwise is (possibly intentionally) misleading.\n\nYes what does the 'other' package show in comparison?  \n\nJust that.\nWhile this could be taken as an honest mistake, it has the \"smell\" of yet another rather explicitly chosen comparison.\n\nNow, if and when one actually does as the more than half-dozen _very clear_ recommendations cited above suggests and _grows an STL object_ then the picture changes as seen when running `demo(grow, package=\"ldlasb\")` (based on the functions in `src/grow.cpp`):\n\n\n![](inst/figures/grow.png)\n\n[Rcpp][1] wins, again.\n\n### Repository\n\nThis repository contains all the code that is needed to run the two benchmarks, wrapped up in a small convenience package called `ldlasb`.\nIt cleanly passes `R CMD check` too. \nYou can install it the usual ways including via, _e.g._, \n\n```r\nremotes::install_github(\"eddelbuettel/ldlasb\")\n```\n\n### Run Benchmarks\n\nOnce the package is installed, you can reach the benchmarks via `demo(release, package=\"ldlasb\", ask=FALSE)` and `demo(grow, package=\"ldlasb\", ask=FALSE)`, respectively.\n\nThe results shown here were generated on my standard machine (currently) running Ubuntu 20.10 with, _inter alia_, R 4.1.0 and g++ 10.3.0, on an i7-8700K cpu.\nWe can expects to vary somewhat between systems and compilers, but the overall outcome should not change.\n\n### Author\n\nThis package was written by Dirk Eddelbuettel.\n\nCode from the 'other' package was used to reproduce / replicate the benchmarks and graphs; see [here for more](inst/AUTHORS).\n\n### License\n\nThis package is licensed under the GNU GPL, Version 2 or later.\n\n[1]: https://github.com/RcppCore/Rcpp\n[2]: https://CRAN.R-Project.org/package=cpp11\n[3]: https://CRAN.R-Project.org\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feddelbuettel%2Fldlasb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feddelbuettel%2Fldlasb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feddelbuettel%2Fldlasb/lists"}