{"id":27537968,"url":"https://github.com/mncube/idmact","last_synced_at":"2025-04-18T20:48:43.260Z","repository":{"id":156396069,"uuid":"627938399","full_name":"mncube/idmact","owner":"mncube","description":"Interpreting Differences Between Mean ACT Scores","archived":false,"fork":false,"pushed_at":"2023-05-18T17:06:31.000Z","size":124,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-14T01:47:04.019Z","etag":null,"topics":["assessment","measurement","psychometrics","r","scale"],"latest_commit_sha":null,"homepage":"","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mncube.png","metadata":{"files":{"readme":"README.Rmd","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-04-14T14:29:56.000Z","updated_at":"2023-04-20T01:41:58.000Z","dependencies_parsed_at":null,"dependency_job_id":"5b2bf875-de95-434b-8972-ceadaa739be0","html_url":"https://github.com/mncube/idmact","commit_stats":{"total_commits":55,"total_committers":1,"mean_commits":55.0,"dds":0.0,"last_synced_commit":"d80b08cf9f16757d26e622745cdcad3604cfcdc3"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mncube%2Fidmact","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mncube%2Fidmact/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mncube%2Fidmact/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mncube%2Fidmact/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mncube","download_url":"https://codeload.github.com/mncube/idmact/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249544027,"owners_count":21288850,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["assessment","measurement","psychometrics","r","scale"],"created_at":"2025-04-18T20:48:42.629Z","updated_at":"2025-04-18T20:48:43.248Z","avatar_url":"https://github.com/mncube.png","language":"R","funding_links":[],"categories":[],"sub_categories":[],"readme":"---\noutput: github_document\n---\n\n\u003c!-- README.md is generated from README.Rmd. Please edit that file --\u003e\n\n```{r, include = FALSE}\nknitr::opts_chunk$set(\n  collapse = TRUE,\n  comment = \"#\u003e\",\n  fig.path = \"man/figures/README-\",\n  out.width = \"100%\"\n)\n```\n\n\n# idmact\n\n\u003c!-- badges: start --\u003e\n\u003c!-- badges: end --\u003e\n\nThe goal of idmact is to provide an implementation of the [Schiel (1998)](https://www.act.org/content/dam/act/unsecured/documents/ACT_RR98-01.pdf) \nalgorithm for interpreting differences between mean ACT assessment scores.\n\n## Installation\n\nYou can install the released version of idmact from [CRAN](https://CRAN.R-project.org) with:\n\n```{r eval=FALSE}\ninstall.packages(\"idmact\")\n```\n\n\nYou can install the development version of idmact from [GitHub](https://github.com/) with:\n\n```{r eval=FALSE}\n# install.packages(\"devtools\")\ndevtools::install_github(\"mncube/idmact\")\n```\n\n\n## Main Algorithm\n\nThe idmact package provides tools designed to examine the significance of\ndifferences between mean ACT scale scores over time. The fundamental algorithm for\ncomposite scale scores consists of the following six steps:\n\n1. Add one unit to the raw score of one or more subjects for each student to derive\nadjusted raw scores.\n\n2. Convert adjusted raw scores to adjusted scale scores using the form's raw score\nto scale score map. Note that perfect raw scores are always converted to the\nmaximum allowable scale score, irrespective of the adjustment in step one.\n\n3. For each student, sum the adjusted scale scores across all subject areas,\ndivide this sum by the number of subject areas, and round to the nearest integer.\nThis produces each student's adjusted composite scale score.\n\n4. Calculate the adjusted mean composite scale score across all students (m_adj).\n\n5. Calculate the unadjusted mean composite scale score across all observations\n(m_unadj).\n\n6. Compute the difference between the adjusted and unadjusted mean composite scale\nscores to get the delta composite: deltac = m_adj - m_unadj\n\nWhile this algorithm was initially developed to address the challenges in\ninterpreting differences between mean ACT scale scores, it can also be beneficial\nin other contexts. These may include situations where different forms of an\nassessment have varying raw score to scale score maps, particularly when the\nrelationship between raw and scale scores is complex and/or proprietary.\n\n## Single Subject Example\n\nIn the subsequent example, we'll use the algorithm described above to interpret\ndifferences between two forms of a hypothetical assessment. Both forms share the\nsame range of raw scores and scale scores. However, the conversion map from raw\nscores to scale scores differs between the two assessments. To keep this example\nstraightforward, we'll simulate a single data set of raw scores and then use the\nalgorithm to compare mean differences in scale scores between the two assessments.\nThis initial example will concentrate on a single subject area, hence step 3 of\nthe algorithm will not be included.\n\n### Generate Data and Maps\n\n```{r example}\nlibrary(idmact)\n\n# Create 100 raw scores\nset.seed(279)\nraw_scores \u003c- as.list(sample(1:100, 100, replace = TRUE))\n\n# Map between raw scores and scale scores for each form\n\n## Each form has scale scores ranging from 1 to 12\nmap_scale \u003c- c(1:12)\n\n## Each assessment has raw scores ranging from 1 - 100\nmap_raw_formA \u003c- list(1:5, 6:20, 21:25, 26:40, 41:45, 46:50, 51:55,\n                   56:75, 76:80, 81:85, 86:90, 91:100)\n\nmap_raw_formB \u003c- list(1:10, 11:20, 21:30, 31:40, 41:50, 51:55, 56:65,\n                   66:75, 76:85, 86:90, 91:95, 96:100)\n```\n\n\n### Format Map\n\nIn the raw score to scale score map presented above, vectors/lists were utilized to\nefficiently map each of the 100 raw scores to each of the 12 scale scores. Use the\nmap_elongate function to expand the raw and scale sections of the map into a format\nwhere each portion is a list of 100. This is the map format required for the\nidmact_subj function, which implements the subject-level algorithm.\n\n```{r}\nformA \u003c- map_elongate(map_raw = map_raw_formA,\n                      map_scale = map_scale)\n\nformB \u003c- map_elongate(map_raw = map_raw_formB,\n                      map_scale = map_scale)\n```\n\n\n### Run Subject Level Algorithm\n\nUtilize the raw data and the maps stored in formA and formB to calculate and\ncompare the subject-level delta (deltas). In the algorithm below, each raw score\nis increased by 1 using the default value of the inc parameter.\n\n```{r}\nresA \u003c- idmact_subj(raw = raw_scores,\n                    map_raw = formA$map_raw,\n                    map_scale = formA$map_scale)\n\n\nresB \u003c- idmact_subj(raw = raw_scores,\n                    map_raw = formB$map_raw,\n                    map_scale = formB$map_scale)\n```\n\n\n### Compare Results\n\n```{r}\ncat(\"Form A subject level delta:\", resA$deltas, \"\\n\")\ncat(\"Form B subject level delta:\", resB$deltas)\n```\n\n\nIn this section, the function idmact_subj is used to calculate the 'delta' for\neach form (A and B). The 'delta' is the difference between the mean adjusted scale\nscore and the mean unadjusted scale score. This 'delta' value provides an estimate\nof how much the mean scale score would increase if every student were to answer\none additional item correctly on the test.\n\nIn the provided example:\n\nFor Form A, the subject level delta is 0.11, meaning the mean scale score is expected\nto increase by 0.11 if every student answers one additional item correctly.\n\nFor Form B, the subject level delta is 0.1, suggesting that the mean scale score\nwould increase by 0.1 under the same conditions.\n\nThe results indicate that Form A is more responsive to increases in raw scores,\nas a one unit increase in raw score leads to a larger increase in the mean scale\nscore for Form A compared to Form B.\n\n## Composite Example\n\nThe idmact_comp function can be used to calculate the delta for composite scores\n(deltac). In the example below, raw scores for an additional subject area ('s2')\nwill be created and combined with the data from the previous example to demonstrate\nidmact_comp.\n\n### Generate Data and Maps\n\n```{r}\n# Create 100 raw scores\nset.seed(250)\nraw_scores_s2 \u003c- as.list(sample(1:100, 100, replace = TRUE))\n\n# Subject two will use the same ranges for raw scores and scale scores as in\n# the previous example, but the map will be slightly different. \nmap_raw_formA_s2 \u003c- list(1:10, 11:25, 26:30, 31:40, 41:45, 46:50, 51:60,\n                         61:75, 76:80, 81:85, 86:90, 91:100)\n\nmap_raw_formB_s2 \u003c- list(1:10, 11:16, 17:25, 26:35, 36:45, 46:55, 56:60,\n                         61:75, 76:85, 86:90, 91:95, 96:100)\n\nformA_s2 \u003c- map_elongate(map_raw = map_raw_formA_s2,\n                         map_scale = map_scale)\n\nformB_s2 \u003c- map_elongate(map_raw = map_raw_formB_s2,\n                         map_scale = map_scale)\n```\n\n\n### Run Composite Level Algorithm\n\nIn the algorithm below, each raw score for each subject is incremented by 1.\n\n```{r}\nresA_comp \u003c- idmact_comp(raw = list(raw_scores, raw_scores_s2),\n                         inc = list(1, 1),\n                         map_raw = list(formA$map_raw, formA_s2$map_raw),\n                         map_scale = list(formA$map_scale, formA_s2$map_scale))\n\nresB_comp \u003c- idmact_comp(raw = list(raw_scores, raw_scores_s2),\n                         inc = list(1, 1),\n                         map_raw = list(formB$map_raw, formB_s2$map_raw),\n                         map_scale = list(formB$map_scale, formB_s2$map_scale))\n```\n\n\n### Compare Composite Results\n\n```{r}\ncat(\"Form A composite level delta:\", resA_comp$composite_results$deltac, \"\\n\")\ncat(\"Form B composite level delta:\", resB_comp$composite_results$deltac)\n```\n\n\nIn the composite example, two subjects are considered instead of one, and the\nidmact_comp function is used to calculate the composite level delta (deltac).\nThe composite level delta is calculated in a similar way to the subject level delta,\nbut it considers the total adjusted and unadjusted scale scores across all subjects,\nrather than just one.\n\nIn the provided example:\n\nFor both Form A and Form B, the composite level delta is 0.08. This means that if\nevery student were to answer one additional item correctly on each form, the mean\ncomposite scale score is expected to increase by 0.08.\n\nThis suggests that, when considering multiple subjects, both Form A and Form B\nrespond similarly to increases in raw scores.\n\n### See Subject Area 2 Results\n\n```{r}\ncat(\"Form A subject area 2 delta:\", resA_comp$subject_results[[2]]$deltas, \"\\n\")\ncat(\"Form B subject area 2 delta:\", resB_comp$subject_results[[2]]$deltas)\n```\n\n\nThis section presents the 'delta' for the second subject area specifically. The \n'delta' for the second subject area is calculated in the same way as the subject\nlevel delta mentioned in the first section, but it only considers the scores for\nthe second subject.\n\nIn the provided example:\n\nFor Form A, the subject area 2 delta is 0.05, suggesting that the mean scale score\nfor the second subject would increase by 0.05 if every student answered one additional\nitem correctly.\n\nFor Form B, the subject area 2 delta is 0.08, meaning that the mean scale score\nfor the second subject would increase by 0.08 under the same conditions.\n\nThese results indicate that, for the second subject area specifically, Form B is\nmore responsive to increases in raw scores.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmncube%2Fidmact","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmncube%2Fidmact","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmncube%2Fidmact/lists"}