{"id":24029912,"url":"https://github.com/pcolladosoto/elasticity-processing","last_synced_at":"2026-06-13T08:32:15.001Z","repository":{"id":166396109,"uuid":"559848255","full_name":"pcolladosoto/elasticity-processing","owner":"pcolladosoto","description":"This repository contains the data processing utilities leveraged on Gonzalo Martínez de Marco's End of Degree Project.","archived":false,"fork":false,"pushed_at":"2023-12-10T22:47:14.000Z","size":31,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-26T03:27:31.869Z","etag":null,"topics":["data-processing","excel","gnuplot","jq","python3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pcolladosoto.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-31T08:15:26.000Z","updated_at":"2023-03-13T17:19:28.000Z","dependencies_parsed_at":null,"dependency_job_id":"f8a8601a-1e84-4edf-bb51-83dd7252f782","html_url":"https://github.com/pcolladosoto/elasticity-processing","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pcolladosoto/elasticity-processing","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pcolladosoto%2Felasticity-processing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pcolladosoto%2Felasticity-processing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pcolladosoto%2Felasticity-processing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pcolladosoto%2Felasticity-processing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pcolladosoto","download_url":"https://codeload.github.com/pcolladosoto/elasticity-processing/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pcolladosoto%2Felasticity-processing/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34278153,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-13T02:00:06.617Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-processing","excel","gnuplot","jq","python3"],"created_at":"2025-01-08T17:04:51.840Z","updated_at":"2026-06-13T08:32:14.733Z","avatar_url":"https://github.com/pcolladosoto.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Parsing elasticity data from raw text files\nWe have been tasked with parsing `;` separated files in order to retrieve information\ngenerated by several elasticity experiments studying the properties of plastics suitable\nfor 3D-Printing.\n\nWe'll just parse the different raw files to then generate Excel (i.e. `*.xlsx`) file,\nthus making handling them much easier.\n\nWe'll also use [`jq(1)`](https://stedolan.github.io/jq/manual/) together with\n[`gnuplot(1)`](http://www.gnuplotting.org/manpage-gnuplot-4-6/) to generate any\ngraphs that might be requested.\n\n## Useful commands\nBoth `jq` and `gnuplot` are very powerful tools. The thing is, they are also a bit complex\nto use... This section contains several commands we can rely on for common tasks.\n\n### Converting JSON data to CSV\nWe can leverage `jq` filters to this end:\n\n\t# Option breakdown:\n\t\t# -r: We want raw output instead of compliant JSON elements.\n\t\t# \u003cfilter\u003e: The filter will access the root document member (`.data`) and then\n\t\t\t# we'll choose the series both for elgontation and tension. We'll end\n\t\t\t# up by transposing the results so that we get columns instead of just\n\t\t\t# two lines and then convert that to CSC with `@csv`.\n\t\t# Prob14.json: The JSON file we are processing.\n\t\t# Shell redirection: We'll save the result in `tensionElongation.csv`.\n\tjq -r '.data | [.elongationN, .tensionMPa] | transpose[] | @csv' Prob14.json \u003e tensionElongation.csv\n\nWe can use variations of the above for a ton of operations: `jq` is extremely powerful!\n\n### Plotting CSV files\nWe can use `gnuplot` to plot the two columns we exported just now:\n\n\t# Let's tell gnuplot to break up the lines on `,`\n\tset datafaile separator \",\"\n\n\t# We'll set the axes labels\n\tset xlabel \"Tensión [MPa]\"\n\tset ylabel \"Elongación\"\n\n\t# We can now plot the data:\n\t\t# \"tensionElongation.csv\": The file containing the data.\n\t\t# using 1:2: We'll use the first column on the X axis and the\n\t\t\t# second one on the Y axis. Note we can also use the\n\t\t\t# pseudocolumn `0` to plot a single data series too.\n\t\t# with lines: We'll use solid lines in the plot.\n\t\t# t \"Prob14\": The name to show on the graph's legend.\n\tplot \"tensionElongation.csv\" using 1:2 with lines t \"Prob14\"\n\nLike before, variations of the above can become quite useful too!\n\n## Useful stuff\nWhen doing all this we ran into some statistics we had never worked with before...\nThe following is a list of some of the links we found useful when working with the\nShapiro-Wilk test:\n\n- The main WikiPedia entry: https://en.wikipedia.org/wiki/Shapiro–Wilk_test\n- The Null hypothesis: https://en.wikipedia.org/wiki/Null_hypothesis\n- What's a *p-value*?: https://en.wikipedia.org/wiki/P-value\n- Scipy's `stats.shapiro()` doc: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.shapiro.html\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpcolladosoto%2Felasticity-processing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpcolladosoto%2Felasticity-processing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpcolladosoto%2Felasticity-processing/lists"}