{"id":30623487,"url":"https://github.com/snhobbs/spreadsheet-wrangler","last_synced_at":"2025-10-13T12:34:15.796Z","repository":{"id":57470315,"uuid":"358711117","full_name":"snhobbs/spreadsheet-wrangler","owner":"snhobbs","description":"Command line tool for interacting with spreadsheet data","archived":false,"fork":false,"pushed_at":"2025-07-27T18:38:58.000Z","size":291,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-08-30T16:31:11.347Z","etag":null,"topics":["bill-of-materials","command-line","command-line-tool","csv","excel","python","spreadsheet","spreadsheet-editor","spreadsheet-manipulation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/snhobbs.png","metadata":{"files":{"readme":"README.md","changelog":"HISTORY.rst","contributing":"CONTRIBUTING.rst","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-04-16T20:22:30.000Z","updated_at":"2025-07-27T18:39:02.000Z","dependencies_parsed_at":"2023-01-23T15:45:22.585Z","dependency_job_id":"4a53aeb1-aaba-4d22-8897-37f713cf3a02","html_url":"https://github.com/snhobbs/spreadsheet-wrangler","commit_stats":{"total_commits":27,"total_committers":2,"mean_commits":13.5,"dds":"0.11111111111111116","last_synced_commit":"28c24c0d0393b9044a477db4d6cb614b4b5a7f3d"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/snhobbs/spreadsheet-wrangler","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snhobbs%2Fspreadsheet-wrangler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snhobbs%2Fspreadsheet-wrangler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snhobbs%2Fspreadsheet-wrangler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snhobbs%2Fspreadsheet-wrangler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/snhobbs","download_url":"https://codeload.github.com/snhobbs/spreadsheet-wrangler/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snhobbs%2Fspreadsheet-wrangler/sbom","scorecard":{"id":834580,"data":{"date":"2025-08-11","repo":{"name":"github.com/snhobbs/spreadsheet-wrangler","commit":"9d6d898f7c2152ec0fc7c25a5c3a1b8936e7dd8c"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.6,"checks":[{"name":"Maintained","score":0,"reason":"1 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Code-Review","score":0,"reason":"Found 0/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/cd.yml:47: update your workflow using https://app.stepsecurity.io/secureworkflow/snhobbs/spreadsheet-wrangler/cd.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/cd.yml:52: update your workflow using https://app.stepsecurity.io/secureworkflow/snhobbs/spreadsheet-wrangler/cd.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/cd.yml:69: update your workflow using https://app.stepsecurity.io/secureworkflow/snhobbs/spreadsheet-wrangler/cd.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/cd.yml:74: update your workflow using https://app.stepsecurity.io/secureworkflow/snhobbs/spreadsheet-wrangler/cd.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/cd.yml:11: update your workflow using https://app.stepsecurity.io/secureworkflow/snhobbs/spreadsheet-wrangler/cd.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/cd.yml:15: update your workflow using https://app.stepsecurity.io/secureworkflow/snhobbs/spreadsheet-wrangler/cd.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/cd.yml:27: update your workflow using https://app.stepsecurity.io/secureworkflow/snhobbs/spreadsheet-wrangler/cd.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/python-app.yml:21: update your workflow using https://app.stepsecurity.io/secureworkflow/snhobbs/spreadsheet-wrangler/python-app.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/python-app.yml:23: update your workflow using https://app.stepsecurity.io/secureworkflow/snhobbs/spreadsheet-wrangler/python-app.yml/main?enable=pin","Warn: pipCommand not pinned by hash: .github/workflows/cd.yml:20","Warn: pipCommand not pinned by hash: .github/workflows/python-app.yml:28","Warn: pipCommand not pinned by hash: .github/workflows/python-app.yml:29","Warn: pipCommand not pinned by hash: .github/workflows/python-app.yml:30","Warn: pipCommand not pinned by hash: .github/workflows/python-app.yml:31","Info:   0 out of   7 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   2 third-party GitHubAction dependencies pinned","Info:   0 out of   5 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/cd.yml:1","Info: topLevel 'contents' permission set to 'read': .github/workflows/python-app.yml:13","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Packaging","score":10,"reason":"packaging workflow detected","details":["Info: Project packages its releases by way of GitHub Actions.: .github/workflows/cd.yml:32"],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":7,"reason":"3 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: PYSEC-2021-437 / GHSA-5xp3-jfq3-5q8x","Warn: Project is vulnerable to: PYSEC-2023-228 / GHSA-mq26-g339-26xf","Warn: Project is vulnerable to: PYSEC-2022-43017 / GHSA-qwmp-2cf2-g9g6"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-23T18:38:19.621Z","repository_id":57470315,"created_at":"2025-08-23T18:38:19.621Z","updated_at":"2025-08-23T18:38:19.621Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279015056,"owners_count":26085643,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-13T02:00:06.723Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bill-of-materials","command-line","command-line-tool","csv","excel","python","spreadsheet","spreadsheet-editor","spreadsheet-manipulation"],"created_at":"2025-08-30T16:21:40.430Z","updated_at":"2025-10-13T12:34:15.791Z","avatar_url":"https://github.com/snhobbs.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# spreadsheet-wrangler\nCommand line tool for interacting with spreadsheet data.\n\n## Installation\n### pypi\n```bash\npip install spreadsheet-wrangler\n```\n\n### Source\n```bash\ngit clone https://github.com/snhobbs/spreadsheet-wrangler.git\ncd spreadsheet-wrangler\npip install .\n```\n\n## Functions\n+ merge: Left merge two spreadsheets and save as .xlsx\n+ compare: Compare two spreadsheets on a column name and print discrepancies\n+ cluster: Combine the same values in specified columns as arrays under the clustered column name, retaining the first row's other data\n+ uncluster: Unpack clustered columns into individual entries, duplicating rows accordingly\n+ filter: Filter rows\n\n## Usage Examples\nThese examples follow the use case of working with a bill of materials (BOM).\nA specialized tool that uses this library for this use case is\navailable here[https://github.com/snhobbs/BOMTools].\n\n\n### Expand a list of designators and unique identifiers with additional data\nTo expand a list of reference designators and unique identifier of a part (bom.xlsx) with\nadditionally available data (data_store.xlsx) use the spreadsheet_wrangler tool:\n```\nspreadsheet_wrangler.py merge --on=\"pn\" -l bom.xlsx -r data_store.xlsx -p '{\"pn\":[\"Internal Part Number\"]}'\n```\nAny unique identifier will work, examples are: manufacturers part number, internal part number, index to a database, etc.\nEquivalent names for columns are passed to the tool in a JSON format. A file titled bom_Merged_data_store_On_pn.xlsx will be generated with this command.\n\n### Compare the data in two BOMS\nTo compare the available data of two BOMs to compare function of spreadsheet_wrangler should be used. If a BOM was exported\nand needs to be checked against another with questionable history run:\n```\nspreadsheet_wrangler.py compare --on=\"ref-des\" -l bom.xlsx -r bom_Merged_data_store_On_pn.xlsx\n```\nThis will compare the original BOM with the merged one from the first example. Comparisons are done column by column with rows matched by the value in the column passed with the argument \"--on\". This should be a unique for each instance of a part (i.e a ref des).\nThe shared columns will be checked as well as any passed in with the --columns argument. Discrepancies are printed to screen.\n\n### Generate a BOM sorted by the type of parts\nBOMs used for ordering, shipping, budgeting, or shipping to a CM are typically ordered by the type of part.\n\nThe ordering BOM sorts by reference designator and combines the BOM into unique part types. This can then be used for ordering or quoting. This can be passed to a tool like kicost or used with supplier BOM Managers or the Octopart BOM Manager (recommended).\n\nTo sort a BOM by the type of part and with a list of the reference designators run:\n```\nspreadsheet_wrangler.py cluster --column=\"ref-des\" --on=\"pn\" -s bom_clustered_on_ref-des.xlsx\n```\n\n### Compare a BOM sorted by the type of parts with a design BOM\nTo compare a BOM sorted by the part type (as shown above) with a BOM sorted by reference designator the BOM needs to be unpacked first and then compared.\nTo unpack run:\n```\nspreadsheet_wrangler.py uncluster --column=\"ref-des\" -s bom_unclustered.xlsx\n```\nThis will separate the lines like the original bom.xlsx. This BOM can now be compared using the compare function described above.\n\nNOTE: Note the data in each grouped row is duplicated for each clustered element. This is not necessarily correct if data was dissimilar and lost during the clustering step.\n\n### Clustering a partially filled in BOM\nAfter exporting a design BOM with each component in it's own line you end up with what I call a design BOM except without the useful fields included.\n\n![unclustered BOM](unclustered_bom.png)\n\nFor this I want to cluster based on \"Comment\" and \"Footprint\", that is I want all 0603 10K resistors together and not include any other 0603 or 10K parts in the cluster. The command would then be:\n\n```\nspreadsheet_wrangler.py cluster --spreadsheet BOM-1x2_tester.csv --column \"Designator\" --on \"Comment\" --on \"Footprint\" -o BOM_clustered.xlsx\n```\nThis call turns the above BOM into:\n\n![clustered on comment and footprint](clustered_on_comment_and_footprint.png)\nwo spreadsheets on a column name, prints out the discrepancies\n- cluster: Combine the same values in a specified column as an array with the same name as the clustered column. The remainder of the first rows data is kept.\n- uncluster: Unpack clustered columns into one entry for each. The row is duplicated for each entry.\n- filter\n\n## Usage Examples\nThese examples follow the use case of working with a bill of materials (BOM). A specialized tool that uses this library for this use case is\navailable here[https://github.com/snhobbs/BOMTools].\n\n### Expand a list of designators and unique identifiers with additional data\nTo expand a list of reference designators and unique identifier of a part (bom.xlsx) with\nadditionally available data (data_store.xlsx) use the spreadsheet_wrangler tool:\n```\nspreadsheet_wrangler.py merge --on=\"pn\" -l bom.xlsx -r data_store.xlsx -p '{\"pn\":[\"Internal Part Number\"]}'\n```\nAny unique identifier will work, examples are: manufacturers part number, internal part number, index to a database, etc.\nEquivalent names for columns are passed to the tool in a JSON format. A file titled bom_Merged_data_store_On_pn.xlsx will be generated with this command.\n\n### Compare the data in two BOMS\nTo compare the available data of two BOMs to compare function of spreadsheet_wrangler should be used. If a BOM was exported\nand needs to be checked against another with questionable history run:\n```\nspreadsheet_wrangler.py compare --on=\"ref-des\" -l bom.xlsx -r bom_Merged_data_store_On_pn.xlsx\n```\nThis will compare the original BOM with the merged one from the first example. Comparisons are done column by column with rows matched by the value in the column passed with the argument \"--on\". This should be a unique for each instance of a part (i.e a ref des).\nThe shared columns will be checked as well as any passed in with the --columns argument. Discrepancies are printed to screen.\n\n### Generate a BOM sorted by the type of parts\nBOMs used for ordering, shipping, budgeting, or shipping to a CM are typically ordered by the type of part.\n\nThe ordering BOM sorts by reference designator and combines the BOM into unique part types. This can then be used for ordering or quoting. This can be passed to a tool like kicost or used with supplier BOM Managers or the Octopart BOM Manager (recommended).\n\nTo sort a BOM by the type of part and with a list of the reference designators run:\n```\nspreadsheet_wrangler.py cluster --column=\"ref-des\" --on=\"pn\" -s bom_clustered_on_ref-des.xlsx\n```\n\n### Compare a BOM sorted by the type of parts with a design BOM\nTo compare a BOM sorted by the part type (as shown above) with a BOM sorted by reference designator the BOM needs to be unpacked first and then compared.\nTo unpack run:\n```\nspreadsheet_wrangler.py uncluster --column=\"ref-des\" -s bom_unclustered.xlsx\n```\nThis will separate the lines like the original bom.xlsx. This BOM can now be compared using the compare function described above.\n\nNOTE: Note the data in each grouped row is duplicated for each clustered element. This is not necessarily correct if data was dissimilar and lost during the clustering step.\n\n### Clustering a partially filled in BOM\nAfter exporting a design BOM with each component in it's own line you end up with what I call a design BOM except without the useful fields included.\n\n![unclustered BOM](unclustered_bom.png)\n\nFor this I want to cluster based on \"Comment\" and \"Footprint\", that is I want all 0603 10K resistors together and not include any other 0603 or 10K parts in the cluster. The command would then be:\n\n```\nspreadsheet_wrangler.py cluster --spreadsheet BOM-1x2_tester.csv --column \"Designator\" --on \"Comment\" --on \"Footprint\" -o BOM_clustered.xlsx\n```\nThis call turns the above BOM into:\n\n![clustered on comment and footprint](clustered_on_comment_and_footprint.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnhobbs%2Fspreadsheet-wrangler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsnhobbs%2Fspreadsheet-wrangler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnhobbs%2Fspreadsheet-wrangler/lists"}