{"id":17383021,"url":"https://github.com/mcommer/emtools","last_synced_at":"2025-06-30T20:04:59.715Z","repository":{"id":254342159,"uuid":"845757399","full_name":"mcommer/emtools","owner":"mcommer","description":"A toolbox for geophysical EM-simulation data- and model-file processing, analysis, plotting, and other gimmicks","archived":false,"fork":false,"pushed_at":"2024-08-28T22:12:08.000Z","size":3111,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-27T20:49:14.386Z","etag":null,"topics":["data-visualization","electromagnetics","geophysics","plotting-scripts","shell-scripts"],"latest_commit_sha":null,"homepage":"","language":"Awk","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mcommer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-21T21:49:14.000Z","updated_at":"2024-08-29T21:47:38.000Z","dependencies_parsed_at":"2024-12-06T19:12:21.887Z","dependency_job_id":"2950d330-065e-47de-9642-95005226c889","html_url":"https://github.com/mcommer/emtools","commit_stats":null,"previous_names":["mcommer/emtools"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mcommer/emtools","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcommer%2Femtools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcommer%2Femtools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcommer%2Femtools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcommer%2Femtools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mcommer","download_url":"https://codeload.github.com/mcommer/emtools/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcommer%2Femtools/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262842917,"owners_count":23373165,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-visualization","electromagnetics","geophysics","plotting-scripts","shell-scripts"],"created_at":"2024-10-16T07:40:18.637Z","updated_at":"2025-06-30T20:04:59.651Z","avatar_url":"https://github.com/mcommer.png","language":"Awk","readme":"# emtools - a toolbox for geophysical EM-simulation data and model file processing, analysis, plotting, and other gimmicks\r\n\r\n## Table of Contents\r\n  - [📖About](#about)\r\n  - [🏗️Installation🚧](#️installation)\r\n  - [📉Plotting EM-data: **emtools\\_data\\_plot**](#plotting-em-data-emtools_data_plot)\r\n  - [📉Examples for **emtools\\_data\\_plot**](#examples-for-emtools_data_plot)\r\n  - [📞Contact](#contact)\r\n\r\n## 📖About \r\nGeophysical electromagnetic (EM) simulations involve a host of pre- and post-simulation steps\r\nthat prepare, modify, process and analyze simulation inputs and outputs.\r\n**emtools** is a suite of tools for many of these tasks:\r\n1. creation and modification of EM data input/output files\r\n2. creation and modification of EM model input/output files and their corresponding grid (mesh) files\r\n3. plotting of data and models\r\n\r\n**emtools** is a derivative of Michael Commer's EMGeo toolboxes emgeo_prc, emgeo_modelcom, em3d_modelcom, and em3d_dataplot. These tools were developed at Lawrence Berkeley Nat'l Lab, California. em3d_dataplot examples can for example be seen in:\r\n*Commer,  M., and Newman, G.A., (2008). New advances in three-dimensional controlled-source electromagnetic inversion, Geophys. J. Int., 172/2, 513–535*. **emtools_data_plot** is based on em3d_dataplot. The **emtools_data_plot** extension for ModEM data files (1st version: 2023.10) was developed at Observatorio Nacional, Rio de Janeiro.\r\n\r\n### Currently available **emtools**\r\n1. **emtools_data_plot** - plotting of data, supported formats: ModEM\r\n\r\n### Data formats supported by **emtools**\r\nEM simulation codes are often proprietary and have their own specific file formats for data and\r\nmodel inputs and outputs. **emtools** currently supports the formats of these simulators:\r\n1. **ModEM** - ModEM is a modular system of computer codes for \r\ninversion of EM geophysical data developed over the past decade at Oregon State University. \r\nA stable version of the code tailored to inversion of 3D magnetotelluric data has been made freely\r\navailable for academic research. See also: *Egbert, G.D., Meqbel, N., and Ritter, O. (2014). \r\nImplementing novel schemes for inversion of 3D EM data in ModEM, the OSU modular EM inversion system. \r\nSoc. Expl. Geophys. Tech. Prog. Exp. Abstr.*\r\n\r\n## 🏗️Installation🚧\r\nInstallation of **emtools** is done from a terminal command prompt.\r\n### 1. ✅Required auxiliary tools\r\nBefore installation, ensure that the following tools are available on the system where you run **emtools**:\r\n1. git (also available under https://git-scm.com)\r\n2. csh\r\n3. gawk\r\n4. gnuplot\r\n5. ps2pdf\r\n6. sed\r\n\r\nThe above components are standard tools. If not present on your system, their installation should be straightforward using your OS-specific package handling utility, for example under Ubuntu:\r\n```shell\r\nsudo apt-get update\r\nsudo apt-get -y install gnuplot\r\n```\r\n\r\n### 2. 👭Make the **emtools** parent directory and clone\r\nThe **emtools** parent directory is where you will do \"git clone ...\".\r\n```shell\r\n# 1) Go to your home directory\r\ncd ~\r\n# 2) If not already present: create the parent dir. usr/local in your home\r\n# Note: This is the recommended installation. You can choose another location,\r\n# for example /usr/local or /opt (then the next steps might need to be su-done)\r\nmkdir -p usr/local\r\ncd usr/local\r\n# 3) Clone the repository. Afterwards, you will have ~/usr/local/emtools/\r\ngit clone https://github.com/mcommer/emtools.git\r\n# 4) make the emtools executables executable (if needed, dep. on umask) \r\nchmod u+x emtools/bin/*\r\n```\r\n\r\n### 3. 🛒Export the EMTOOLS shell variable\r\nThe EMTOOLS shell variable is needed by most tools of the **emtools** package and is set to  \r\nEMTOOLS = \\\u003cPARENT-DIR\\\u003e/emtools = **${HOME}/usr/local/emtools** (as recommended above in Step 2)  \r\nIt is further recommended to set EMTOOLS at shell startup, for example for bash-users:\r\n```shell\r\n# this is in your ~/.bashrc file\r\nexport EMTOOLS=${HOME}/usr/local/emtools\r\n# add the $EMTOOLS/bin/ directory to your executable-PATH\r\nexport PATH=${PATH}:${EMTOOLS}/bin\r\n```\r\nAlternatively, you can set EMTOOLS via the **emtools** configuration file **${HOME}/.emtools**\r\n```shell\r\n# emtools configuration file for user \"me\", resides in /home/me/.emtools\r\nEMTOOLS: /home/me/usr/local/emtools  # uncomment to activate / comment to deactivate\r\n```\r\n\r\n\r\n## 📉Plotting EM-data: **emtools_data_plot**\r\n**emtools_data_plot** is a tool for plotting data from geophysical electromagnetic (EM) input/output files.\r\n**emtools_data_plot** is part of the **emtools** suite. Currently supported EM data file formats:\r\n- ModEM\r\n\r\nAfter installation, you can run\r\n```shell\r\nemtools_data_plot -h\r\n```\r\nto get a manpage-like help screen. Type \"q\" to exit the help screen.\r\n\r\n## 📉Examples for **emtools_data_plot**\r\nDirectories with plot examples are contained in the directory\r\n**${EMTOOLS}/doc/examples/emtools_data_plot/**\r\n\r\n### 📉Example 1: MT_data_comparison/\r\n- Plots two MT files together\r\n- Output file: mt_base_anom.pdf\r\n```shell\r\nemtools_data_plot mt_base.out mt_anom.out -k base,anomaly -o mt_base_anom\r\n```\r\n\r\n### 📉Example 2: InvTest/\r\n- You can plot an arbitrary number of data sets (data files) together\r\n- Here, three data sets are combined in one plot\r\n- Output files: smallmod.pdf, smallmod.dat, smallmod.gnu\r\n- The option \"-keep\" saves the files smallmod.dat and smallmod.gnu in the same\r\ndirectory from where **emtools_data_plot** was run\r\n- Option \"-keep\" is useful if one wants to use the plotting data for other purposes, for example plot\r\ncreation in publications, other fine-tuning, etc.\r\n```shell\r\nemtools_data_plot smallmod_NLCG_000.dat smallmod_NLCG_005.dat smallmod_NLCG_010.dat -k Iter_0,Iter_5,Iter10 -o smallmod -w p_pt_5_ps_0.4,l,l -keep\r\n```\r\n\r\n### 📉Example 3: MCM/\r\nExample MCM-1:\r\n- Plot measured data, where measured_Ey_20_Err.dat is a CSEM data set and Measured_MT_2_0.5.dat is an MT data set\r\n- Output files: measured_Ey.pdf (ex. MCM-1), Measured_MT_2_0.5_plot.pdf (ex. MCM-2)\r\n- This uses the gnuplot style \"-w lp_pt_3_lt_4\" = \"with linespoints pointtype 3 linetype 4\"\r\n\r\nExample MCM-2:\r\n- Since no output file is specified, the default name *_plot is used as fileroot, where * is the fileroot of the input file name (Measured_MT_2_0.5).\r\n- Here, the with-argument \"-w p_pt_6_lt_2\" = \"with points pointtype 6 linetype 2\" causes the point symbols to be plotted with the color that corresponds to linetype 2.\r\n\r\nExample MCM-3:\r\n- Output file: Rx_y_subsets.pdf\r\n- This command will produce 884 plot pages. Each page corresponds to a data subset\r\n- In each data subset, the MT-receiver's x-coordinates represent the abscissa\r\n- All receivers of a subset further share a unique y-coordinate; the latter will be specified in the plot title\r\n```shell\r\n# Example MCM-1\r\nemtools_data_plot -w lp_pt_3_lt_4 -o measured_Ey measured_Ey_20_Err.dat\r\n# Example MCM-2\r\nemtools_data_plot -w p_pt_6_lt_2 Measured_MT_2_0.5.dat -view\r\n# Example MCM-3\r\nemtools_data_plot -w p_pt_5 -r x:y -o Rx_y_subsets Measured_MT_2_0.5.dat -view\r\n```\r\n\r\n### 📉Example 4: PauloTest1/\r\nExample PauloTest1-a:\r\n- The example involves data sets that have only small numerical differences\r\n- Demonstrates the option \"-col\" and data-difference plotting\r\n- Output file: pred_dip1d.pdf\r\n- The first option \"-col r:light-green,i:honeydew\" sets the background colors of the RE and IM plots to\r\nthe color names \"light-green\" and \"honeydew\", respectively\r\n- The y-range of each plot is set to the interval [1e-19,1e-11]. This interval applies to\r\nthe y-axis of all RE-data subplots and all IM-data subplots\r\n- The two data sets pred_dip1d_bef.dat and pred_dip1d_aft.dat are numerically very similar,\r\ntherefore, only one plot line will be visible\r\n\r\nExample PauloTest1-b:\r\n- Output files: pred_diff_Abs.pdf, pred_diff_Abs.dat, pred_diff_Abs.gnu\r\n- The example demonstrates the differencing functionality of **emtools_data_plot**\r\n- The resulting plot shows absolute differences between the data sets pred_dip1d_bef.dat and pred_em1d_bef.dat\r\n- The y-range is set to [0,1e-11] (linear y-axis) for all subplots\r\n- The background color of the whole plot page is set to light-magenta\r\n```shell\r\n# Example PauloTest1-a\r\nemtools_data_plot -col r:light-green,i:honeydew -yr 1e-19,1e-11 -k Dipole1d_before,Dipole1D_after -o pred_dip1d pred_dip1d_bef.dat pred_dip1d_aft.dat -view\r\n# Example PauloTest1-b\r\nemtools_data_plot -col b:light-magenta -da -yr 0,1e-11 -o pred_diff_Abs pred_dip1d_bef.dat pred_em1d_bef.dat -keep -view,xpdf\r\n```\r\nTo get a list of all possible color names, start gnuplot and type \"show colornames\":\r\n```shell\r\n gnuplot\u003e show colornames\r\n         There are 111 predefined color names:\r\n   white              #ffffff = 255 255 255\r\n   black              #000000 =   0   0   0\r\n   dark-grey          #a0a0a0 = 160 160 160\r\n   red                #ff0000 = 255   0   0\r\n   web-green          #00c000 =   0 192   0\r\n   web-blue           #0080ff =   0 128 255\r\n   dark-magenta       #c000ff = 192   0 255\r\n   dark-cyan          #00eeee =   0 238 238\r\n   ...\r\n```\r\n\r\n### 📉Example 5: PauloTest2/\r\nExample PauloTest2-a:\r\n- The example shows the usage of the data plotting style yerrorbars\r\n- Output file: pred_em1d_iso_on_plot.pdf\r\n- The plot uses errorbar symbols: \"with yerrorbars pointtype 4 pointsize 0.3 linetype 3\"\r\n- Here, the T-string is present and is set to \"1e-11\" (end of \"-w\"-option argument), which denotes an absolute half-errorbar length of 1e-11\r\n- The option \"-yr e\" sets all plot-y-axis ranges to the same interval, which is given by the global min and max data value\r\n- The option \"-view\" will launch the default PDF-file viewer, which you can set via the line \"PDFVIEWER: ...\" in the **emtools** configuration file\r\n\r\nExample PauloTest2-b:\r\n- Output file: pred_em1d_iso_on2_plot.pdf\r\n- The plot uses errorbar symbols: \"with yerrorbars pointtype 2 pointsize 0.7 linetype 4\"\r\n- Here, the T-string is present and is set to \"26.5%\", which denotes a relative half-yerrorbar size in percent\r\n- The yerrorbar of a given data point y thus extends over the range [y1,y2] = [y-|y|*p/100,y+|y|*p/100], where p=26.5\r\n- The option \"-view,xpdf\" will launch the PDF-file viewer \"xpdf\"\r\n\r\n```shell\r\n# Example PauloTest2-a\r\nemtools_data_plot pred_em1d_iso_on.dat -w yerr_pt_4_ps_0.3_lt_3:1e-11 -yr e -view\r\n# Example PauloTest2-b\r\nemtools_data_plot pred_em1d_iso_on.dat -w yerr_pt_2_ps_0.7_lt_4:26.5% -yr e,e -view,xpdf -o pred_em1d_iso_on2_plot\r\n```\r\n\r\n### 📉Example 6: SmallModelResponse/\r\n- Play more with colors and experiment with logo-graphic insertion\r\n- Output file: compare_em1d_dip1d.pdf\r\n- Here, the page color is set to gray90 (a light-grey tone), while the RE- and IM-subplots will have other distinct colors\r\n- A logo PNG file is included at the plot page bottom using the option \"-logo ./companylogo.png\" (PNG-file needs to be present in the run directory)\r\n- The option \"-logo\" is experimental, as one may have to tweak the logo graphic to achieve a satisfying outcome\r\n- The option \"-view,xpdf\" will launch the PDF-file viewer \"xpdf\"\r\n```shell \r\nemtools_data_plot -k EM1D,Dipole1D csem.em1d csem.di1d -o compare_em1d_dip1d -col b:gray90,r:tan1,i:gray80 -logo ./companylogo.png -view,xpdf\r\n```\r\n\r\n## 📞Contact\r\nmicha@on.br\r\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmcommer%2Femtools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmcommer%2Femtools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmcommer%2Femtools/lists"}