{"id":13665073,"url":"https://github.com/oganm/dnddata","last_synced_at":"2025-12-24T22:52:33.739Z","repository":{"id":43781418,"uuid":"166397044","full_name":"oganm/dnddata","owner":"oganm","description":"Weekly updated dataset of D\u0026D characters submitted to https://oganm.com/shiny/printSheetApp and https://oganm.com/shiny/interactiveSheet. A superset of characters used in oganm/dndstats","archived":false,"fork":false,"pushed_at":"2022-08-24T01:47:44.000Z","size":42462,"stargazers_count":107,"open_issues_count":2,"forks_count":19,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-11-10T23:37:21.521Z","etag":null,"topics":["5e","dnd","dnd-characters","dnd5e","dungeons-and-dragons","ogan-dnd"],"latest_commit_sha":null,"homepage":"","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oganm.png","metadata":{"files":{"readme":"README.Rmd","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-01-18T11:48:54.000Z","updated_at":"2024-09-26T01:12:03.000Z","dependencies_parsed_at":"2022-09-02T10:50:20.560Z","dependency_job_id":null,"html_url":"https://github.com/oganm/dnddata","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oganm%2Fdnddata","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oganm%2Fdnddata/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oganm%2Fdnddata/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oganm%2Fdnddata/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oganm","download_url":"https://codeload.github.com/oganm/dnddata/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250937112,"owners_count":21510895,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["5e","dnd","dnd-characters","dnd5e","dungeons-and-dragons","ogan-dnd"],"created_at":"2024-08-02T05:03:16.982Z","updated_at":"2025-12-24T22:52:33.728Z","avatar_url":"https://github.com/oganm.png","language":"R","funding_links":[],"categories":["R","Dataset"],"sub_categories":["Web"],"readme":"---\ntitle: \"README\"\noutput: \n    github_document:\n        html_preview: false\n---\n\n```{r setup, include=FALSE}\nknitr::opts_chunk$set(echo = TRUE)\ndevtools::load_all()\n```\n\n\n\n# dnddata\n\n   * [dnddata](#dnddata)\n      * [Usage/installation](#usageinstallation)\n      * [Examples](#examples)\n      * [About the data](#about-the-data)\n         * [Column/element description](#columnelement-description)\n         * [Caveats](#caveats)\n            * [Possible Issues with data fields](#possible-issues-with-data-fields)\n            * [Possible issues with detection of unique characters](#possible-issues-with-detection-of-unique-characters)\n            * [Possible issues with selection bias](#possible-issues-with-selection-bias)\n\nThis is a weekly updated dataset of character that are submitted to my web\napplications [printSheetApp](https://oganm.com/shiny/printSheetApp) and\n[interactiveSheet](https://oganm.com/shiny/interactiveSheet). It is a superset\nof the dataset I previously released under\n[oganm/dndstats](https://oganm.github.io/dndstats) with a much larger sample\n(`r length(dnd_chars_unique_list)` characters) size and more data fields. It was inspired\nby the [FiveThirtyEight](https://fivethirtyeight.com/features/is-your-dd-character-rare/) article on race/class proportions and the data seems to correlate\nwell with those results (see my [dndstats article](https://oganm.github.io/dndstats)).\n\nAlong with a simple table (an R `data.frame` in package), the data is also\npresent in json format (an R `list` in package). In the table version some data\nfields encode complex information that are represented in a more readable manner\nin the json format. The data included is otherwise identical.\n\n## Usage/installation\n\nIf you are an R user, you can simply install this package and load it to access the dataset\n\n```r\ndevtools::install_github('oganm/dnddata')\nlibrary(dnddata)\n```\nTry `?tables`, `?lists` to see available objects and their descriptions\n\nIf you are not an R user, access the files within the [data-raw](data-raw) directory. The files\nare available as JSON and TSV. You can find the field descriptions \n[below](#columnelement-description). `dnd_chars_all` files contain all characters\nthat are submitted while `dnd_chars_unique` files are filtered to include unique\ncharacters.\n\n## Examples\n\nI will be using the list form of the dataset as a basis here.\n\nLet's replicate that plot from [fivethirtyeight](https://fivethirtyeight.com/features/is-your-dd-character-rare/)\nas I did in my [original article](https://oganm.github.io/dndstats/).\n\n```{r,fig.width=9,message=FALSE}\nlibrary(purrr)\nlibrary(ggplot2)\nlibrary(magrittr)\nlibrary(dplyr)\nlibrary(reshape2)\n\n# find all available races\nraces = dnd_chars_unique_list %\u003e% \n\tpurrr::map('race') %\u003e% \n\tpurrr::map_chr('processedRace') %\u003e% trimws() %\u003e% \n\tunique %\u003e% {.[.!='']}\n\n# find all available classes\nclasses = dnd_chars_unique_list %\u003e% \n\tpurrr::map('class') %\u003e%\n\tunlist(recursive = FALSE) %\u003e%\n\tpurrr::map_chr('class') %\u003e% trimws() %\u003e%  unique\n\n# create an empty matrix\ncoOccurenceMatrix = matrix(0 , nrow=length(races),ncol = length(classes))\ncolnames(coOccurenceMatrix) = classes\nrownames(coOccurenceMatrix) = races\n# fill the matrix with co-occurences of race and classes\nfor(i in seq_along(races)){\n\tfor(j in seq_along(classes)){\n\t\t# get characters with the right race\n\t\traceSubset = dnd_chars_unique_list[dnd_chars_unique_list %\u003e% \n                          purrr::map('race') %\u003e% \n                          purrr::map_chr('processedRace') %\u003e% {.==races[i]}]\n\t\t\n\t\t# get the characters with the right class. Weight multiclassed characters based on level\n\t\traceSubset %\u003e% purrr::map('class') %\u003e% \n\t\t\tpurrr::map_dbl(function(x){\n\t\t\t\tx  %\u003e% sapply(function(y){\n\t\t\t\t\t(trimws(y$class) == classes[j])*y$level/(sum(map_int(x,'level')))\n\t\t\t\t}) %\u003e% sum}) %\u003e% sum -\u003e coOcc\n\t\t\n\t\tcoOccurenceMatrix[i,j] = coOcc\n\t}\n}\n\n# reorder the matrix a little bit\ncoOccurenceMatrix = \n    coOccurenceMatrix[coOccurenceMatrix %\u003e% apply(1,sum) %\u003e% order(decreasing = FALSE),\n                            coOccurenceMatrix %\u003e% apply(2,sum) %\u003e% order(decreasing = TRUE)]\n\n# calculate percentages\ncoOccurenceMatrix = coOccurenceMatrix/(sum(coOccurenceMatrix))* 100\n\n# remove the rows and columns if they are less than 1%\ncoOccurenceMatrixSubset = coOccurenceMatrix[,!(coOccurenceMatrix %\u003e% apply(2,sum) %\u003e% {.\u003c1})]\ncoOccurenceMatrixSubset = coOccurenceMatrixSubset[!(coOccurenceMatrixSubset %\u003e% apply(1,sum) %\u003e% {.\u003c1}),]\n\n# add in class and race sums\nclassSums = coOccurenceMatrix %\u003e% apply(2,sum) %\u003e% {.[colnames(coOccurenceMatrixSubset)]}\nraceSums = coOccurenceMatrix %\u003e% apply(1,sum) %\u003e% {.[rownames(coOccurenceMatrixSubset)]}\ncoOccurenceMatrixSubset = cbind(coOccurenceMatrixSubset,raceSums)\ncoOccurenceMatrixSubset = rbind(Total = c(classSums,NA), coOccurenceMatrixSubset)\ncolnames(coOccurenceMatrixSubset)[ncol(coOccurenceMatrixSubset)] = \"Total\"\n\n# ggplot\ncoOccurenceFrame = coOccurenceMatrixSubset %\u003e% reshape2::melt()\nnames(coOccurenceFrame)[1:2] = c('Race','Class')\ncoOccurenceFrame %\u003c\u003e% mutate(fillCol = value*(Race!='Total' \u0026 Class!='Total'))\ncoOccurenceFrame %\u003e% ggplot(aes(x = Class,y = Race)) +\n    geom_tile(aes(fill = fillCol),show.legend = FALSE)+\n    scale_fill_continuous(low = 'white',high = '#46A948',na.value = 'white')+\n    cowplot::theme_cowplot() + \n    geom_text(aes(label = value %\u003e% round(2) %\u003e% format(nsmall=2))) + \n    scale_x_discrete(position='top') + xlab('') + ylab('') + \n    theme(axis.text.x = element_text(angle = 30,vjust = 0.5,hjust = 0)) \n\n```\n\nOr try something new. Wonder which fighting style is more popular?\n\n```{r}\n\ndnd_chars_unique_list %\u003e% purrr::map('choices') %\u003e% \n\tpurrr::map('fighting style') %\u003e% \n\tunlist %\u003e%\n\ttable %\u003e% \n\tsort(decreasing = TRUE) %\u003e% \n\tas.data.frame %\u003e% \n\tggplot(aes(x = ., y = Freq)) +\n\tgeom_bar(stat= 'identity') +\n\tcowplot::theme_cowplot() +\n\ttheme(axis.text.x= element_text(angle = 45,hjust = 1))\n```\n\n## About the data\n\n### Column/element description\n\n- **ip:** A shortened hash of the IP address of the submitter\n\n- **finger:** A shortened hash of the browser fingerprint of the submitter\n\n- **name:** A shortened hash of character names\n\n- **race:** Race of the character as coded by the app. May be unclear as the app inconsistently codes race/subrace information. See processedRace\n\n- **background:** Background as it comes out of the application.\n\n- **date:** Time \u0026 date of input. Dates before 2018-04-16 are unreliable as some has accidentally changed while moving files around.\n\n- **class:** Class and level. Different classes are separated by | when needed.\n\n- **justClass:** Class without level. Different classes are separated by | when needed.\n\n- **subclass:** Subclass. Might be missing if the character is low level. Different classes are separated by | when needed.\n\n- **level:** Total level\n\n- **feats:** Feats chosen. Mutliple feats are separated by | when needed\n\n- **HP:** Total HP\n\n- **AC:** AC score\n\n- **Str, Dex, Con, Int, Wis, Cha:** Ability score modifiers\n\n- **alignment:** Alignment free text field. Since it's a free text field, it includes alignments written in many forms. See processedAlignment, good and lawful to get the standardized alignment data.\n\n- **skills:** List of proficient skills. Skills are separated by |.\n\n- **weapons:** List of weapons, separated by |. This is a free text field. See processedWeapons for the standardized version\n\n- **spells:** List of spells, separated by |. Each spell has its level next to it separated by *s. This is a free text field. See processedSpells for the standardized version\n\n- **castingStat:** Casting stat as entered by the user. The format allows one casting stat so this is likely wrong if the character has different spellcasting classes. Also every character has a casting stat even if they are not casters due to the data format.\n\n- **choices:** Character building choices. This field information about character properties such as fighting styles and skills chosen for expertise. Different choice types are separated by | when needed. The choice data is written as name of choice followed by a / followed by the choices that are separated by *s\n\n- **country:** The origin of the submitter's IP\n\n- **countryCode:** 2 letter country code\n\n- **processedAlignment:** Standardized version of the alignment column. I have manually matched each non standard spelling of alignment to its correct form. First character represents lawfulness (L, N, C), second one goodness (G,N,E). An empty string means alignment wasn't written or unclear.\n\n- **good, lawful:** Isolated columns for goodness and lawfulness\n\n- **processedRace:** I have gone through the way race column is filled by the app and asigned them to correct races. Also includes some common races that are not natively supported such as warforged and changelings. If empty, indiciates a homebrew race not natively supported by the app.\n\n- **processedSpells:** Formatting is same as spells. Standardized version of the spells column. Spells are matched to an official list using string similarity and some hardcoded rules.\n\n- **processedWeapons:** Formatting is same as weapons. Standardized version of the weapons column. Created like the processedSpells column.\n\n- **levelGroup:** Splits levels into groups. The groups represent the common ASI levels\n\n- **alias:** A friendly alias that correspond to each uniqe name\n\nThe list version of this dataset contains all of these fields but they are organised\na little differently, keeping fields like `spells` and `processedSpells` together.\n\n### Caveats\n\n#### Possible Issues with data fields\n\nSome data fields are more reliable than others. Below is a summary of all potential problems with the data fields\n\n* **ip and browser fingerprints:** Both IP and browser fingerprints are represented as hashes.\nI keep them to have an idea of individual users but did not make use of them so far. Note\nthat same IPs can be shared by an entire region in some cases.\n\n* **processedAlignment:** Alignment is a free text field in the app and optional. Many characters do not enter their alignments. To create the standardized alignment fields, I went through every entry and manually assigned every alternative spelling to the standardized version. These include mispelled entries, abreviations, entries in different languages etc. In cases where I wasn't able\nto match (eg. what the hell is \"lawful cute\"), this field was left blank. Between automatic updates\nnew and exciting ways to describe alignment can come into play. Unless I manually added these\nnew entries, they will also appear blank.\n\n* **processedSpells:** The mobile app allows entering free text into the spell fields. Which means I have\nto deal with people writing spells in a non-standard way with typos, abbreviations or additional information such as range, damage dice. I use some heuristics to match the entered text to a list of all published spells. Shortly, I look at the Levenshtein distance between the entry and the published spells and match the entry with the top result if \n  * the spell level is correct and,\n  * there are not more than 10 substitutions/deletions/insertions or either entry or the potential match includes all words that the counterpart includes.\n  * In addition, there are special cases for Bigby's Hand, Tasha's Hideous Laughter and Melf's Acid Arrow as those spells are often written\n  in their SRD form and match to wrong spells.\n \n\n```{r, echo = FALSE}\nwithSpells = which(dnd_chars_unique$spells !='')\n\nwithSpells %\u003e% lapply(function(i){\n\trawSpells = dnd_chars_unique$spells[i] %\u003e% strsplit('\\\\|') %\u003e% {.[[1]]}\n\tpSpells =  dnd_chars_unique$processedSpells[i] %\u003e% strsplit('\\\\|') %\u003e% {.[[1]]}\n\tseq_along(rawSpells) %\u003e% sapply(function(j){\n\t\tc(i,rawSpells[j],pSpells[j])\n\t}) %\u003e% t\n}) %\u003e% do.call(rbind,.) -\u003e  spellProcessedPairs\n\nspellCount = spellProcessedPairs %\u003e% nrow\nstandardSpellCount = nrow(spellProcessedPairs[spellProcessedPairs[,3] !='*' \u0026 spellProcessedPairs[,2] == spellProcessedPairs[,3],])\nnonStandardSpellCount = nrow(spellProcessedPairs[spellProcessedPairs[,3] !='*' \u0026 spellProcessedPairs[,2] != spellProcessedPairs[,3],])\nmismatchCount = spellProcessedPairs[spellProcessedPairs[,3] =='*',-3] %\u003e% nrow\n\nnonStandardPercent = nonStandardSpellCount/spellCount * 100\nmismatchPercent = mismatchCount/spellCount * 100\nstandardPercent = standardSpellCount/spellCount * 100\n```\n \n \n`r round(standardPercent)`% of all spells parsed did not\nrequire any modification. `r round(nonStandardPercent)`% of were only able to be\nmatched through the heuristics. A manual examination of a random seleciton of\nthese matches revealed 2/200 mistakes. `r round(mismatchPercent)`% of the spell\nentries were not matched to an official spell. Manual observation of these\nentries revealed that the common reasons for a failure to match are users\nwriting the spell under the wrong spell level, writing some class/race features\nsuch as blindsight as spells or adding/removing more than 10 charters when\nwriting the spells either through abbreviation or adding additional information\nabout the spell.\n\n\n```{r echo = FALSE}\n\nwithWeapons = which(dnd_chars_unique$weapons !='')\nwithWeapons %\u003e% lapply(function(i){\n\trawWeapons = dnd_chars_unique$weapons[i] %\u003e% stringr::str_split('\\\\|') %\u003e% {.[[1]]}\n\tpWeapons =  dnd_chars_unique$processedWeapons[i] %\u003e% stringr::str_split('\\\\|') %\u003e% {.[[1]]}\n\tseq_along(rawWeapons) %\u003e% sapply(function(j){\n\t\tc(i,rawWeapons[j],pWeapons[j])\n\t}) %\u003e% t\n}) %\u003e% do.call(rbind,.) -\u003e  weaponProcessedPairs\n\nweaponCount = weaponProcessedPairs %\u003e% nrow\nstandardWeaponCount = nrow(weaponProcessedPairs[weaponProcessedPairs[,2] == weaponProcessedPairs[,3],])\nnonStandardWeaponCount = nrow(weaponProcessedPairs[weaponProcessedPairs[,2] != weaponProcessedPairs[,3] \u0026 weaponProcessedPairs[,3] !='',])\nmismatchCount = weaponProcessedPairs[weaponProcessedPairs[,3] =='',] %\u003e% nrow\n\nnonStandardPercent = nonStandardWeaponCount/weaponCount * 100\nmismatchPercent = mismatchCount/weaponCount * 100\nstandardPercent = standardWeaponCount/weaponCount * 100\n```\n\n* **processedWeapons:** Weapon names are also free text fields so a processing method similar to the one used for spells is used for weapon names. Instead of a threshold of 10 substititutions/deletions/insertions, 2 was used since weapon names typically did not include additional information like spell names did. Special cases were written for hand crossbow and heavy crossbow as they were typically mismatched to their official name (eg. \"crossbow, hand\"). Here the weapons that weren't matched were spell names or homebrew weapons.\n\n`r round(standardPercent)`% of all weapons parsed did not require any\nmodification. `r round(nonStandardPercent)`% of were only able to be matched\nthrough the heuristics. A manual examination of a random seleciton of these\nmatches revealed 1/200 mistake. `r round(mismatchPercent)`% of the weapon\nentries were not matched to an official weapon.\n\n#### Possible issues with detection of unique characters\n\nIdentification of unique characters rely on some heuristics. I assume any character\nwith the same name and class is potentially the same character. In these cases I\npick the highest level character. Race and other properties are not considered so\nsome unique characters may be lost along the way. I have chosen to be less exact\nto reduce the nubmer of possible test characters since there were examples of people\nsubmitting essentially the same character with different races, presumably to test\nthings out. For multiclassed characters, if a lower level character with the same name and a subset of classes exist, they are removed, again leaving the character with the highest\nlevel.\n\n#### Possible issues with selection bias\n\nThis data comes from characters submitted to my web applications. The\napplications are written to support a popular third party character sheet app\nfor mobile platforms. I have advertised my applications primarily on Reddit\nr/dndnext and r/dnd. I have seen them mentioned in a few other platforms by word of\nmouth. That means we are looking at subsamples of subsamples here, all of which\ncan cause some amount of selection bias. Some characters could be thought\nexperiments or for testing purposes and never see actual game play.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foganm%2Fdnddata","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foganm%2Fdnddata","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foganm%2Fdnddata/lists"}