{"id":37667654,"url":"https://github.com/albertov5/tec-data-04-dataframes","last_synced_at":"2026-01-16T12:00:32.456Z","repository":{"id":46330104,"uuid":"515334189","full_name":"AlbertoV5/tec-data-04-dataframes","owner":"AlbertoV5","description":null,"archived":false,"fork":false,"pushed_at":"2022-07-27T05:22:57.000Z","size":5185,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2023-03-04T18:11:45.984Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AlbertoV5.png","metadata":{"files":{"readme":"readme.html","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-07-18T20:37:52.000Z","updated_at":"2022-07-26T21:38:34.000Z","dependencies_parsed_at":"2022-09-02T11:00:36.735Z","dependency_job_id":null,"html_url":"https://github.com/AlbertoV5/tec-data-04-dataframes","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"purl":"pkg:github/AlbertoV5/tec-data-04-dataframes","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlbertoV5%2Ftec-data-04-dataframes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlbertoV5%2Ftec-data-04-dataframes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlbertoV5%2Ftec-data-04-dataframes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlbertoV5%2Ftec-data-04-dataframes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AlbertoV5","download_url":"https://codeload.github.com/AlbertoV5/tec-data-04-dataframes/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlbertoV5%2Ftec-data-04-dataframes/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28478397,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-16T11:59:17.896Z","status":"ssl_error","status_checked_at":"2026-01-16T11:55:55.838Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-16T12:00:22.168Z","updated_at":"2026-01-16T12:00:32.393Z","avatar_url":"https://github.com/AlbertoV5.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c?xml version=\"1.0\" encoding=\"utf-8\"?\u003e\n\u003c!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n\"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\"\u003e\n\u003chtml xmlns=\"http://www.w3.org/1999/xhtml\" lang=\"en\" xml:lang=\"en\"\u003e\n\u003chead\u003e\n\u003c!-- 2022-07-27 Wed 00:21 --\u003e\n\u003cmeta http-equiv=\"Content-Type\" content=\"text/html;charset=utf-8\" /\u003e\n\u003cmeta name=\"viewport\" content=\"width=device-width, initial-scale=1\" /\u003e\n\u003ctitle\u003eChallenge\u003c/title\u003e\n\u003cmeta name=\"author\" content=\"Alberto Valdez\" /\u003e\n\u003cmeta name=\"generator\" content=\"Org Mode\" /\u003e\n\u003clink rel=\"stylesheet\" type=\"text/css\" href=\"https://albertov5.github.io/org-html-themes/src/readtheorg_theme/css/htmlize.css\"/\u003e\n\u003clink rel=\"stylesheet\" type=\"text/css\" href=\"https://albertov5.github.io/org-html-themes/src/readtheorg_theme/css/readtheorg.css\"/\u003e\n\u003cscript src=\"https://ajax.googleapis.com/ajax/libs/jquery/2.1.3/jquery.min.js\"\u003e\u003c/script\u003e\n\u003cscript src=\"https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js\"\u003e\u003c/script\u003e\n\u003cscript type=\"text/javascript\" src=\"https://albertov5.github.io/org-html-themes/src/lib/js/jquery.stickytableheaders.min.js\"\u003e\u003c/script\u003e\n\u003cscript type=\"text/javascript\" src=\"https://albertov5.github.io/org-html-themes/src/readtheorg_theme/js/readtheorg.js\"\u003e\u003c/script\u003e\n\u003c/head\u003e\n\u003cbody\u003e\n\u003cdiv id=\"content\" class=\"content\"\u003e\n\u003ch1 class=\"title\"\u003eChallenge\u003c/h1\u003e\n\u003cdiv id=\"table-of-contents\" role=\"doc-toc\"\u003e\n\u003ch2\u003eTable of Contents\u003c/h2\u003e\n\u003cdiv id=\"text-table-of-contents\" role=\"doc-toc\"\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#orgb3dcba6\"\u003eSchool District Analysis\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#org012006d\"\u003eOverview\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#org98fe9f1\"\u003eTools and Requirements\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#orgf6b7993\"\u003eResults\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#orge75ad66\"\u003eThomas High School Difference\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#org6a4f2c2\"\u003eDifference of Scores by School Spending\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#orge3f16b6\"\u003eDifference of Scores by School Size\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#org7e67fa2\"\u003eDifference of Scores by School Type\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#orgcff017e\"\u003eDifference of Math and Reading scores by grade\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#orgd247294\"\u003eSummary\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#org2e9971c\"\u003eClosing Thoughts\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\n\u003cdiv id=\"outline-container-orgb3dcba6\" class=\"outline-2\"\u003e\n\u003ch2 id=\"orgb3dcba6\"\u003eSchool District Analysis\u003c/h2\u003e\n\u003cdiv class=\"outline-text-2\" id=\"text-orgb3dcba6\"\u003e\n\u003c/div\u003e\n\u003cdiv id=\"outline-container-org012006d\" class=\"outline-3\"\u003e\n\u003ch3 id=\"org012006d\"\u003eOverview\u003c/h3\u003e\n\u003cdiv class=\"outline-text-3\" id=\"text-org012006d\"\u003e\n\u003cp\u003e\nPreviously, we analyzed the student performance and funding of various schools. The results were very informative as we got to see the average scores by grade, school type, school size, etc. However, we need to repeat the process as we were informed that some of the data is invalid as some grades seem to be altered.\n\u003c/p\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\n\u003cdiv id=\"outline-container-org98fe9f1\" class=\"outline-3\"\u003e\n\u003ch3 id=\"org98fe9f1\"\u003eTools and Requirements\u003c/h3\u003e\n\u003cdiv class=\"outline-text-3\" id=\"text-org98fe9f1\"\u003e\n\u003cp\u003e\n\u003ccode\u003ePython 3.7.13\u003c/code\u003e\n\u003c/p\u003e\n\u003cpre class=\"example\" id=\"orged8bd80\"\u003e\nnumpy==1.21.6\npandas==1.3.5\npytest==7.1.2\n\u003c/pre\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\n\u003cdiv id=\"outline-container-orgf6b7993\" class=\"outline-3\"\u003e\n\u003ch3 id=\"orgf6b7993\"\u003eResults\u003c/h3\u003e\n\u003cdiv class=\"outline-text-3\" id=\"text-orgf6b7993\"\u003e\n\u003cp\u003e\nThe District Summary was not heavily affected by the missing data. We replaced all of the scores of the 9th grade of Thomas High School with \u003ccode\u003eNaN\u003c/code\u003e values. Even after that, the averages and percentages by school types, sizes and spending changed by 1% or less.\n\u003c/p\u003e\n\n\n\u003cdiv id=\"org4e482b0\" class=\"figure\"\u003e\n\u003cp\u003e\u003cimg src=\"./resources/district_summary.png\" alt=\"district_summary.png\" /\u003e\n\u003c/p\u003e\n\u003c/div\u003e\n\n\u003cp\u003e\nIn our previous analysis, the Average Math Score was \u003ccode\u003e79\u003c/code\u003e. The rest of the values, once formatted, stay the same. However, the School Summary was most noticeably affected.\n\u003c/p\u003e\n\n\u003cp\u003e\nWe can create a few tests to compare both DataFrames.\n\u003c/p\u003e\n\u003cdiv class=\"org-src-container\"\u003e\n\u003cpre class=\"src src-python\"\u003e\u003cspan style=\"color: #579C4C;\"\u003e# \u003c/span\u003e\u003cspan style=\"color: #579C4C;\"\u003efrom PyCitySchools_test.py\u003c/span\u003e\n\u003cspan style=\"color: #339CDB;\"\u003edef\u003c/span\u003e \u003cspan style=\"color: #D9DAA2;\"\u003ecompare_dataframes\u003c/span\u003e(original: pd.DataFrame, modified: pd.DataFrame):\n    \u003cspan style=\"color: #777778;\"\u003e\"\"\"Substract modified DF from original.\u003c/span\u003e\n\u003cspan style=\"color: #777778;\"\u003e    Check that the values in the difference are smaller than 1%.\"\"\"\u003c/span\u003e\n    \u003cspan style=\"color: #339CDB;\"\u003eassert\u003c/span\u003e original.size == modified.size\n    difference = original - modified\n    \u003cspan style=\"color: #339CDB;\"\u003efor\u003c/span\u003e col_diff, col \u003cspan style=\"color: #339CDB;\"\u003ein\u003c/span\u003e \u003cspan style=\"color: #C586C0;\"\u003ezip\u003c/span\u003e(difference, original):\n        \u003cspan style=\"color: #339CDB;\"\u003efor\u003c/span\u003e val_diff, val \u003cspan style=\"color: #339CDB;\"\u003ein\u003c/span\u003e \u003cspan style=\"color: #C586C0;\"\u003ezip\u003c/span\u003e(difference[col_diff], original[col]):\n            ratio = \u003cspan style=\"color: #C586C0;\"\u003eabs\u003c/span\u003e(val_diff / val)\n            \u003cspan style=\"color: #339CDB;\"\u003eassert\u003c/span\u003e ratio \u0026lt; \u003cspan style=\"color: #B5CEA8; font-weight: bold;\"\u003e0.01\u003c/span\u003e\n            \u003cspan style=\"color: #339CDB;\"\u003eif\u003c/span\u003e ratio != \u003cspan style=\"color: #B5CEA8; font-weight: bold;\"\u003e0\u003c/span\u003e:\n                msg = f\u003cspan style=\"color: #c5937c;\"\u003e\"\u003c/span\u003e{col_diff}\u003cspan style=\"color: #c5937c;\"\u003e DIFFERENCE: \u003c/span\u003e{100*ratio:.2f}\u003cspan style=\"color: #c5937c;\"\u003e%\"\u003c/span\u003e\n                log.info(msg)\n    log.info(difference)\n\u003c/pre\u003e\n\u003c/div\u003e\n\n\u003cp\u003e\nThis is an example of the test results. Where we expect the changes not to be larger than 1%, which is an arbitrary value based on observation. If it was higher, we should check that we didn\u0026rsquo;t alter more of the data by mistake.\n\u003cimg src=\"./resources/tests2.png\" alt=\"tests2.png\" /\u003e\n\u003c/p\u003e\n\u003c/div\u003e\n\n\u003cdiv id=\"outline-container-orge75ad66\" class=\"outline-4\"\u003e\n\u003ch4 id=\"orge75ad66\"\u003eThomas High School Difference\u003c/h4\u003e\n\u003cdiv class=\"outline-text-4\" id=\"text-orge75ad66\"\u003e\n\u003cp\u003e\nA positive value means the that original result had a higher score.\n\u003c/p\u003e\n\u003ctable border=\"2\" cellspacing=\"0\" cellpadding=\"6\" rules=\"groups\" frame=\"hsides\"\u003e\n\n\n\u003ccolgroup\u003e\n\u003ccol  class=\"org-left\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\u003c/colgroup\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth scope=\"col\" class=\"org-left\"\u003e\u0026#xa0;\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003eAverageMathScore\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003eAverageReadingScore\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003e%PassingMath\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003e%PassingReading\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003e%OverallPassing\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003eThomas High School\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.067412\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e-0.047152\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.086481\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.290129\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.317689\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n\u003cp\u003e\nWe can suspect that the original Thomas High School 9th grade scores were curved to help more students pass the classes. The difference is very small, however, when comparing the rankings of each school by all columns, we found two problems.\n\u003c/p\u003e\n\u003cdiv class=\"org-src-container\"\u003e\n\u003cpre class=\"src src-org\"\u003eThomas High School rank moved from 4 to 6 in Average Math Score.\nThomas High School rank moved from 1 to 3 in % Passing Reading.\n\u003c/pre\u003e\n\u003c/div\u003e\n\n\u003cp\u003e\nThis is the code we can use to find the ranking differences.\n\u003c/p\u003e\n\u003cdiv class=\"org-src-container\"\u003e\n\u003cpre class=\"src src-python\"\u003e\u003cspan style=\"color: #579C4C;\"\u003e# \u003c/span\u003e\u003cspan style=\"color: #579C4C;\"\u003efrom PyCitySchools_test.py\u003c/span\u003e\n\u003cspan style=\"color: #339CDB;\"\u003edef\u003c/span\u003e \u003cspan style=\"color: #D9DAA2;\"\u003etest_ths_ranks\u003c/span\u003e(\u003cspan style=\"color: #d4d4d4;\"\u003ecomplete_parameters\u003c/span\u003e: \u003cspan style=\"color: #C586C0;\"\u003edict\u003c/span\u003e, modified_parameters: \u003cspan style=\"color: #C586C0;\"\u003edict\u003c/span\u003e):\n    \u003cspan style=\"color: #777778;\"\u003e\"\"\"Find if Thomas High School rank amongst other schools changed.\"\"\"\u003c/span\u003e\n    original_df = pd.DataFrame(complete_parameters)\n    \u003cspan style=\"color: #d4d4d4;\"\u003emodified_df\u003c/span\u003e = pd.DataFrame(modified_parameters)\n    \u003cspan style=\"color: #d4d4d4;\"\u003esize\u003c/span\u003e = complete_parameters[\u003cspan style=\"color: #c5937c;\"\u003e\"Average Math Score\"\u003c/span\u003e].size\n    \u003cspan style=\"color: #d4d4d4;\"\u003erank_difference\u003c/span\u003e = pd.DataFrame()\n    \u003cspan style=\"color: #339CDB;\"\u003efor\u003c/span\u003e category \u003cspan style=\"color: #339CDB;\"\u003ein\u003c/span\u003e \u003cspan style=\"color: #d4d4d4;\"\u003ecomplete_parameters\u003c/span\u003e:\n        original = original_df.sort_values(by=category, ascending=\u003cspan style=\"color: #339CDB;\"\u003eFalse\u003c/span\u003e)\n        modified = modified_df.sort_values(by=category, ascending=\u003cspan style=\"color: #339CDB;\"\u003eFalse\u003c/span\u003e)\n        original[\u003cspan style=\"color: #c5937c;\"\u003e\"rank\"\u003c/span\u003e] = \u003cspan style=\"color: #C586C0;\"\u003erange\u003c/span\u003e(\u003cspan style=\"color: #B5CEA8; font-weight: bold;\"\u003e1\u003c/span\u003e, size + \u003cspan style=\"color: #B5CEA8; font-weight: bold;\"\u003e1\u003c/span\u003e)\n        modified[\u003cspan style=\"color: #c5937c;\"\u003e\"rank\"\u003c/span\u003e] = \u003cspan style=\"color: #C586C0;\"\u003erange\u003c/span\u003e(\u003cspan style=\"color: #B5CEA8; font-weight: bold;\"\u003e1\u003c/span\u003e, size + \u003cspan style=\"color: #B5CEA8; font-weight: bold;\"\u003e1\u003c/span\u003e)\n        original_rank = original.loc[original.index == \u003cspan style=\"color: #c5937c;\"\u003e\"Thomas High School\"\u003c/span\u003e, \u003cspan style=\"color: #c5937c;\"\u003e\"rank\"\u003c/span\u003e]\n        modified_rank = modified.loc[modified.index == \u003cspan style=\"color: #c5937c;\"\u003e\"Thomas High School\"\u003c/span\u003e, \u003cspan style=\"color: #c5937c;\"\u003e\"rank\"\u003c/span\u003e]\n        rank_shift = original_rank - modified_rank\n        \u003cspan style=\"color: #339CDB;\"\u003eif\u003c/span\u003e rank_shift.values[\u003cspan style=\"color: #B5CEA8; font-weight: bold;\"\u003e0\u003c/span\u003e] != \u003cspan style=\"color: #B5CEA8; font-weight: bold;\"\u003e0\u003c/span\u003e:\n            log.info(\nf\u003cspan style=\"color: #c5937c;\"\u003e\"\"\"Thomas High School rank moved from {original_rank.values[0]} to {modified_rank.values[0]} in {category}.\"\"\"\u003c/span\u003e\n            )\n        rank_difference[\u003cspan style=\"color: #d4d4d4;\"\u003ecategory\u003c/span\u003e] = rank_shift\n\n    rank_difference.index.name = \u003cspan style=\"color: #c5937c;\"\u003e\"\"\u003c/span\u003e\n    log.info(rank_difference)\n\u003c/pre\u003e\n\u003c/div\u003e\n\n\u003cp\u003e\nA negative value means that the original result had the school ranked higher.\n\u003c/p\u003e\n\u003ctable border=\"2\" cellspacing=\"0\" cellpadding=\"6\" rules=\"groups\" frame=\"hsides\"\u003e\n\n\n\u003ccolgroup\u003e\n\u003ccol  class=\"org-left\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\u003c/colgroup\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth scope=\"col\" class=\"org-left\"\u003e\u0026#xa0;\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003eAverageMathScore\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003eAverageReadingScore\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003e%PassingMath\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003e%PassingReading\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003e%OverallPassing\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003eThomas High School\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e-2\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e-2\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n\u003cp\u003e\nThis tells us that, in the original data, Thomas High School was \u003ccode\u003e#1\u003c/code\u003e in \u003ccode\u003e%PassingReading\u003c/code\u003e and \u003ccode\u003eTop 5\u003c/code\u003e in \u003ccode\u003eAverageMathScore\u003c/code\u003e. This is probably the most relevant change in the analysis.\n\u003c/p\u003e\n\n\u003cp\u003e\nRanking of Original % Passing Reading\n\u003c/p\u003e\n\u003ctable border=\"2\" cellspacing=\"0\" cellpadding=\"6\" rules=\"groups\" frame=\"hsides\"\u003e\n\n\n\u003ccolgroup\u003e\n\u003ccol  class=\"org-left\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\u003c/colgroup\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth scope=\"col\" class=\"org-left\"\u003e\u0026#xa0;\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003e% Passing Reading\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003erank\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003e\u003cb\u003eThomas High School\u003c/b\u003e\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e\u003cb\u003e97.308869\u003c/b\u003e\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e\u003cb\u003e1\u003c/b\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003eGriffin High School\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e97.138965\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e2\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003eCabrera High School\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e97.039828\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e3\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003eWright High School\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e96.611111\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e4\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003eWilson High School\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e96.539641\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e5\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n\u003cp\u003e\nRanking of Modified % Passing Reading\n\u003c/p\u003e\n\u003ctable border=\"2\" cellspacing=\"0\" cellpadding=\"6\" rules=\"groups\" frame=\"hsides\"\u003e\n\n\n\u003ccolgroup\u003e\n\u003ccol  class=\"org-left\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\u003c/colgroup\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth scope=\"col\" class=\"org-left\"\u003e\u0026#xa0;\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003e% Passing Reading\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003erank\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003eGriffin High School\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e97.138965\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e1\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003eCabrera High School\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e97.039828\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e2\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003e\u003cb\u003eThomas High School\u003c/b\u003e\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e\u003cb\u003e97.018739\u003c/b\u003e\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e\u003cb\u003e3\u003c/b\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003eWright High School\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e96.611111\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e4\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003eWilson High School\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e96.539641\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e5\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n\n\u003cp\u003e\nNow that we have the code to find the differences in the results, we can explore the rest of the School Analysis parts for sake of completion. Positive values mean that the original results were higher.\n\u003c/p\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\n\u003cdiv id=\"outline-container-org6a4f2c2\" class=\"outline-4\"\u003e\n\u003ch4 id=\"org6a4f2c2\"\u003eDifference of Scores by School Spending\u003c/h4\u003e\n\u003cdiv class=\"outline-text-4\" id=\"text-org6a4f2c2\"\u003e\n\u003ctable border=\"2\" cellspacing=\"0\" cellpadding=\"6\" rules=\"groups\" frame=\"hsides\"\u003e\n\n\n\u003ccolgroup\u003e\n\u003ccol  class=\"org-left\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\u003c/colgroup\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth scope=\"col\" class=\"org-left\"\u003eSchoolSpending\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003eAverageMathScore\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003eAverageReadingScore\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003e%PassingMath\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003e%PassingReading\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003e%OverallPassing\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003e\u0026lt;$586\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.00000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003e$586-630\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.00000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003e$631-645\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.016853\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e-0.011788\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.02162\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.072532\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.079422\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003e$646-675\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.00000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\n\u003cdiv id=\"outline-container-orge3f16b6\" class=\"outline-4\"\u003e\n\u003ch4 id=\"orge3f16b6\"\u003eDifference of Scores by School Size\u003c/h4\u003e\n\u003cdiv class=\"outline-text-4\" id=\"text-orge3f16b6\"\u003e\n\u003ctable border=\"2\" cellspacing=\"0\" cellpadding=\"6\" rules=\"groups\" frame=\"hsides\"\u003e\n\n\n\u003ccolgroup\u003e\n\u003ccol  class=\"org-left\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\u003c/colgroup\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth scope=\"col\" class=\"org-left\"\u003eSchoolSize\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003eAverageMathScore\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003eAverageReadingScore\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003e%PassingMath\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003e%PassingReading\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003e%OverallPassing\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003eSmall   (\u0026lt;1000)\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.00000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003eMedium  (1000-1999)\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.013482\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e-0.00943\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.017296\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.058026\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.063538\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003eLarge   (2000-5000)\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.00000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\n\u003cdiv id=\"outline-container-org7e67fa2\" class=\"outline-4\"\u003e\n\u003ch4 id=\"org7e67fa2\"\u003eDifference of Scores by School Type\u003c/h4\u003e\n\u003cdiv class=\"outline-text-4\" id=\"text-org7e67fa2\"\u003e\n\u003ctable border=\"2\" cellspacing=\"0\" cellpadding=\"6\" rules=\"groups\" frame=\"hsides\"\u003e\n\n\n\u003ccolgroup\u003e\n\u003ccol  class=\"org-left\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\n\u003ccol  class=\"org-right\" /\u003e\n\u003c/colgroup\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth scope=\"col\" class=\"org-left\"\u003eSchoolType\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003eAverageMathScore\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003eAverageReadingScore\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003e%PassingMath\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003e%PassingReading\u003c/th\u003e\n\u003cth scope=\"col\" class=\"org-right\"\u003e%OverallPassing\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003eCharter\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.008426\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e-0.005894\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.01081\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.036266\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.039711\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd class=\"org-left\"\u003eDistrict\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.00000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003ctd class=\"org-right\"\u003e0.000000\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\n\u003cdiv id=\"outline-container-orgcff017e\" class=\"outline-4\"\u003e\n\u003ch4 id=\"orgcff017e\"\u003eDifference of Math and Reading scores by grade\u003c/h4\u003e\n\u003cdiv class=\"outline-text-4\" id=\"text-orgcff017e\"\u003e\n\u003cp\u003e\nWe can\u0026rsquo;t compare the 9th grade row in the same way as we can compare the rest of the analysis results. If we try to use Python to make a judgment, we will probably run into an error as we could not compare \u003ccode\u003eNaN\u003c/code\u003e to a \u003ccode\u003efloat\u003c/code\u003e like with other tests.\n\u003c/p\u003e\n\n\u003cdiv class=\"org-src-container\"\u003e\n\u003cpre class=\"src src-python\"\u003e\u003cspan style=\"color: #339CDB;\"\u003eassert\u003c/span\u003e \u003cspan style=\"color: #C586C0;\"\u003eabs\u003c/span\u003e(value_modified / value_original) \u0026lt; \u003cspan style=\"color: #B5CEA8; font-weight: bold;\"\u003e0.01\u003c/span\u003e\n\u003c/pre\u003e\n\u003c/div\u003e\n\n\u003cdiv class=\"org-src-container\"\u003e\n\u003cpre class=\"src src-org\"\u003e\u0026gt;       assert abs(value_modified / value_original) \u0026lt; 0.01\nE       assert nan \u0026lt; 0.01\n\nPyCitySchools_test.py:159: AssertionError\n\u003c/pre\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\n\u003cdiv id=\"outline-container-orgd247294\" class=\"outline-3\"\u003e\n\u003ch3 id=\"orgd247294\"\u003eSummary\u003c/h3\u003e\n\u003cdiv class=\"outline-text-3\" id=\"text-orgd247294\"\u003e\n\u003col class=\"org-ol\"\u003e\n\u003cli\u003eThe biggest difference was that Thomas High School was Ranked \u003ccode\u003e#1\u003c/code\u003e in Passing Reading Percentage before removing the data, and moved to \u003ccode\u003e#3\u003c/code\u003e once we replaced the scores with \u003ccode\u003eNaN\u003c/code\u003e.\u003c/li\u003e\n\u003cli\u003eAnother important change was that Thomas High School was \u003ccode\u003eTop 5\u003c/code\u003e in Average Math Score before removing the data, and moved to \u003ccode\u003e#6\u003c/code\u003e once we replaced the scores with \u003ccode\u003eNaN\u003c/code\u003e. What is most interesting is that the Overall Percentage didn\u0026rsquo;t change that much, as we saw increases in both reading and math scores. We can assume that the curve applied to the reading scores helped mostly the students that didn\u0026rsquo;t pass the class initially.\u003c/li\u003e\n\u003cli\u003eThe whole District Average Math Score decreased by \u003ccode\u003e1%\u003c/code\u003e after removing the data, some of it can be the formatting of the data to string as the rest of the values that we inspected more closely had changes under 1%, which in this case was an arbitrary parameter for testing that the differences wouldn\u0026rsquo;t be too big.\u003c/li\u003e\n\u003cli\u003eThe School Summary was not heavily affected as the schools are shown individually, however, Thomas High School affects the Binned results because it is grouped with the rest of the \u003ccode\u003eCharter\u003c/code\u003e, \u003ccode\u003eMedium Sized\u003c/code\u003e and \u003ccode\u003e$631-645\u003c/code\u003e groups.\u003c/li\u003e\n\u003c/ol\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\n\u003cdiv id=\"outline-container-org2e9971c\" class=\"outline-3\"\u003e\n\u003ch3 id=\"org2e9971c\"\u003eClosing Thoughts\u003c/h3\u003e\n\u003cdiv class=\"outline-text-3\" id=\"text-org2e9971c\"\u003e\n\u003cp\u003e\nThanks to Python and Pandas we were able to process the data once more in a streamlined way. We can also run as many transformations, comparisons and tests with our DataFrames as we want. The possibility of doing so programmatically gives us more time to group and separate the results and can give us answers that help us make decisions.\n\u003c/p\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\u003cdiv id=\"postamble\" class=\"status\"\u003e\n\u003cp class=\"author\"\u003eAuthor: Alberto Valdez\u003c/p\u003e\n\u003cp class=\"date\"\u003eCreated: 2022-07-27 Wed 00:21\u003c/p\u003e\n\u003c/div\u003e\n\u003c/body\u003e\n\u003c/html\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falbertov5%2Ftec-data-04-dataframes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falbertov5%2Ftec-data-04-dataframes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falbertov5%2Ftec-data-04-dataframes/lists"}