{"id":16396361,"url":"https://github.com/clhenrick/shell_scripts","last_synced_at":"2025-08-10T08:43:28.999Z","repository":{"id":6377062,"uuid":"7614565","full_name":"clhenrick/shell_scripts","owner":"clhenrick","description":"Bash shell scripts for batch GeoProcessing using GDAL \u0026 OGR2OGR","archived":false,"fork":false,"pushed_at":"2015-07-31T14:02:27.000Z","size":254,"stargazers_count":66,"open_issues_count":0,"forks_count":20,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-04-03T16:24:37.716Z","etag":null,"topics":["bash","foss4geo","gdal","gis","ogr2ogr","shell","shp"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/clhenrick.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-01-14T23:12:21.000Z","updated_at":"2024-02-29T14:35:20.000Z","dependencies_parsed_at":"2022-09-24T02:13:39.025Z","dependency_job_id":null,"html_url":"https://github.com/clhenrick/shell_scripts","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/clhenrick/shell_scripts","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clhenrick%2Fshell_scripts","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clhenrick%2Fshell_scripts/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clhenrick%2Fshell_scripts/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clhenrick%2Fshell_scripts/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/clhenrick","download_url":"https://codeload.github.com/clhenrick/shell_scripts/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clhenrick%2Fshell_scripts/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269698784,"owners_count":24461197,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-10T02:00:08.965Z","response_time":71,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bash","foss4geo","gdal","gis","ogr2ogr","shell","shp"],"created_at":"2024-10-11T05:07:11.233Z","updated_at":"2025-08-10T08:43:28.961Z","avatar_url":"https://github.com/clhenrick.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"shell_scripts\n=============\n\nBash [shell scripts](http://en.wikipedia.org/wiki/Shell_script) primarily for batch geoprocessing spatial data using the OGR2OGR utility, a part of the Geospatial Data Abstract Library: [GDAL.](http://www.gdal.org/ogr2ogr.html)  \n\nThese scripts are useful when open-source GIS applications such as [QGIS](http://qgis.org/en/site/) do not allow for batch processing directories of vector spatial data. Additionally, invoking the scripts from a shell (such as the [Terminal.App](http://en.wikipedia.org/wiki/Terminal_%28OS_X%29) in Mac OSX) allows for heavy data processing to be run in the background while freeing up a GIS software to be used simultaneously for visualization and analysis.\n\nThere are also a few scripts in here for doing other useful things such as a batch find and replace or renaming files to be database and GIS friendly. \n\n## The Scripts\n* [clip-extent-project.sh](#clip-extent-projectsh)\n* [clip-raster-to-shp.sh](#clip-raster-to-shpsh)\n* [clip-to-polygon.sh](#clip-to-polygonsh)\n* [geojson-to-shp.sh](#geojson-to-shpsh)\n* [get-extent.sh](#get-extentsh)\n* [gpx-to-shp.sh](#gpx-to-shpsh)\n* [grep-find-replace.sh](#grep-find-replacesh)\n* [make-hillshades.sh](#make-hillshadessh)\n* [merge.sh](#mergesh)\n* [project-google.sh](#project-googlesh)\n* [project-mercator.sh](#project-mercatorsh)\n* [project-wgs84.sh](#project-wgs84sh)\n* [rename-files-gis-friendly.sh](#rename-files-gis-friendlysh)\n* [separate-roads-by-type.sh](#separate-roads-by-typesh)\n* [shp-to-geojson.sh](#shp-to-geojsonsh)\n\n## Requirements\nYou must install [**GDAL**](http://gdal.org/) and be able to access its utilities from the command line.  \n\n###Mac OS X:\nInstall GDAL via either [**King Chaos**](http://www.kyngchaos.com/software/qgis) or [**Homebrew**](http://brew.sh/).  \n\nIf installing through **King Chaos** be sure to follow the instructions for appending the **GDAL** utilities' absolute path to the `PATH` variable in your `.bash_profile` or `.bashrc` file in your home directory.  \n\n###Linux:\n\nInstall **FWTools** (the Linux 64-bit version is [here](http://fwtools.maptools.org/linux-experimental.html)).\n\n###Test:\n\nMake sure `ogr2ogr` is working in Bash with basic command: `ogrinfo --version`\n\nYou should get back something like:\n\n    GDAL 1.11.0, released 2014/04/16\n\nYou can either add the shell_scripts directory to your `PATH` or run the commands with the full file path.\n\n##FAQ\nThe scripts I authored in this repository were written while working with Cartographic Design \u0026 GIS software that primarily uses the [ESRI Shapefile format](http://en.wikipedia.org/wiki/Shapefile) for storing vector spatial data. Thus most of these scripts assume that is your desired output (and sometimes input) data format unless otherwise specified. Use the `shp-to-geojson` and `geojson-to-shp` scripts to convert data between the ESRI Shapefile and GeoJSON data formats as needed.\n\nMany of these scripts contain variables that must be edited to in order to change their paramaters. Before running open the script of choice with a text editor and change variable assignments for parameters such as output projection (eg: `T_SRS=EPSG:4326`).\n\n**Note:** *You must set the executible permission for these scripts before you can run them.* First `cd` to the folder in which they live and do `chmod +ux *` to make them executible.\n\n## Reference Material\nThese resources have helped me out in learning to build the `ogr2ogr` shell scrips:\n\n* [Directory of Spatial Reference Systems (SRS)](http://spatialreference.org/ref/)\n\n* FYI, GitHub currently likes (i.e., requires) data in the [CRS](http://en.wikipedia.org/wiki/Spatial_reference_system) `urn:ogc:def:crs:OGC:1.3:CRS84` / `EPSG:4326` for displaying geoJSON.\n\n* [List of OGR-supported Vector Formats](http://www.gdal.org/ogr/ogr_formats.html)...*so many options!*\n\n* This [Unix shell scripting tutorial](https://supportweb.cs.bham.ac.uk/documentation/tutorials/docsystem/build/tutorials/unixscripting/unixscripting.html)\n\n## Script Descriptions\n\n### clip-extent-project.sh\nClips data given bounding box coordinates and projects to an output CRS. Change variable assignments within the script for these parameters.  \n####Supported Types  \nESRI Shapefile  \n####Usage\n**variables:**  \n\n```\nLAT_MIN # minimum latitude  (min y value)\nLON_MIN # minimum longitude ( min x value)\nLAT_MAX # maximum latitude  (max y value)\nLON_MAX # maximum longitude (max x value)\nS_SRS  #data's source coordinate system / projection\nT_SRS   #data's target coordinate system / projection\nDIR_OUT # set absolute path for output .shp files here\nPRJ # extracts EPSG code from T_SRS to append to filename\n```\n  \n`cd` to the directory of your `.shp` data and do \n`./clip-extent-project.sh`  \n####Sample Output  \n`test`\n\n### clip-raster-to-shp.sh\nClips raster datasets to a shapefile polygon. Both the raster and vector datasets must be in the same projection / CRS.  \n####Supported Types  \n`.shp` and GDAL supported raster formats.  \n####Usage  \n`./clip-raster-to-shp.sh polygon-to-clip-to.shp raster-data.tiff`\n####Sample Output  \n`test`\n\n### clip-to-polygon.sh\nClips all data to a shapefile polygon.  \n####Supported Types  \n`.shp`  \n####Usage  \n**variables:**\n\n```\nCLIP # absolute path to clipping .shp polygon\nDIR_OUT # absolute path for output .shp files\n```\n\n`cd` to directory containing `.shp` data and do\n`./clip-to-polygon.sh`  \n####Sample Output  \n`test`\n\n### geojson-to-shp.sh\nThis script will export all `.geojson` files in a directory to `.shp` files in a specified subdirectory.  \n####Supported Types  \n`.geojson`  \n####Usage  \n**variables:**  \n`NEWDIR # Path for new sub-directory for output .shp files`  \n\n`cd` to the directory of `.geojson` files you want to convert\n`./geojson-to-shp.sh`  \n####Sample Output  \nWrites new files to a subdirectory.  \n\n### get-extent.sh\nReturns bounding box coordinates for a dataset as:  \n`\u003cx_min\u003e \u003cy_min\u003e \u003cx_max\u003e \u003cy_max\u003e`\n####Supported Types  \n`.shp`  \n####Usage  \n`./get-extent.sh $DIR/$FILE`  \n####Sample Output  \n`-84.391994 33.758135 -84.376599 33.754353`\n\n### gpx-to-shp.sh\nConverts all `.gpx` files in a directory into separate `.shp` files for the following `.gpx` data layers:  \n`tracks`, `way points` and `track points` assuming the `.gpx` file contains data for these layers.  \n####Supported Types  \n`.gpx`  \n####Usage  \n**variables:**  \n\n```\nOUTDR # absolute path for .shp files here\nTSRS # target CRS EPSG code\nPRJ # extracts EPSG code from $TSRS to append to filename\n```\n`./gpx-to-shp.sh`  \n####Sample Output  \n\n```\nsome_gpx_file_tracks_2223.shp  \nsome_gpx_file_points_2223.shp\nsome_gpx_file_waypoints_2223.shp\n```\n\n### grep-find-replace.sh\nSearch through all files in a directory and perform a find and replace on text inside those files.  \n####Supported Types  \n`.txt, .html, .css, etc.`  \n####Usage  \n**variables:**  \n\n```\nstartdirectory # directory to search in\nsearchterm # file name to search for\nreplaceterm # file name to move / replace\n```\n`./grep-find-replace.sh`  \n####Sample Output  \n`test`\n\n### make-hillshades.sh\nTakes a Digital Elevation Model (DEM) raster and generates hillshades from 4 different light angles and a slope shade.These 5 files may then be composited in QGIS, TileMill, Photoshop, etc.  \n\n**Note:** Requires a `color-slope.txt` file containing the following: \n\n```\n0 255 255 255\n90 0 0 0 \n```\n**Suggestion:** Process DEM data prior to running this script (mosaic, clip, resample, reproject, etc).  \n####Supported Types  \nGDAL supported Raster files.  \n####Usage  \n**variables:**  \n`Z # vertical exaggeration factor. Default is 1`\n\n`cd` to the folder containing the DEM  \nthen do: `./make-hillshades.sh some-input-dem.flt`  \n####Sample Output  \n\n```\nhillshade_az45.tif  \nhillshade_az135.tif  \nhillshade_az225.tif  \nhillshade_az315.tif  \nslope.tif  \nslopeshade.tif\n```\n\n### merge.sh\nMerges a directory of shapefiles. Copies the first `.shp` file as the target to merge to and then appends each following `.shp` file to it.  \n**Caveats:** Files must be of same geometry. If attribute schema differs between files you will end up with a huge attribute table.\n####Supported Types  \n`.shp`\n####Usage  \n**variables:**  \n\n```\nFILE # name of .shp file to be merged to (will be created if it doesn't yet exist)\nLAYER # must be same as above but without `.shp` extension\nSSRS # source CRS\nTSRS # target CRS\n```\n\n`cd` to the folder containing files to merge\ncopy a shapefile to merge other shapefiles to by doing\n`ogr2ogr merge.shp shapefile-to-merge-everything-to.shp`\nthen do\n`./merge.sh`\n####Sample Output  \n`test`\n\n### project-web-mercator.sh\nProjects all data in a directory to web-mercator projection `EPSG:3857`\n####Supported Types  \n`.shp`\n####Usage  \n**variables:**  \n`$1 # source CRS of target data`  \n\ndo:  \n`./project-web-mercator.sh \u003cEPSG code\u003e`\n####Sample Output  \n`test`\n\n### project-mercator.sh\nProjects all data in a directory to WGS 94 / World Mercator `EPSG:3395`\n####Supported Types  \n`.shp`\n####Usage  \n**variables:**  \n`$1 # source CRS of target data`  \n\n`cd` to the folder containing your data and do\n`./project-mercator.sh \u003cEPSG code\u003e`\n####Sample Output  \n`test`\n\n### project-wgs84.sh\nProjects all data in a directory to WGS 84 `EPSG:4326`\n####Supported Types  \n`.shp`\n####Usage  \n**variables:**  \n`$1 # source CRS of target data` \n\n`./project-wgs84.sh \u003cEPSG code\u003e`\n####Sample Output  \n`test`\n\n### rename-files-gis-friendly.sh\nModifies all filenames in a directory to be GIS / database friendly by  \nreplacing white spaces with underscores and making all characters lowercase.\n####Supported Types  \nany\n####Usage  \n`./rename-files-gis-friendly.sh`\n####Sample Output  \n`test`\n\n### separate-roads-by-type.sh\nTakes a generic OpenStreetMap `roads.shp` file extracted from a Planet OSM source such as [bbbike.org](http://extract.bbbike.org) and parses into multiple shapefiles based on the aggregating values from the `type` field.  \nWill reproject all files to a desired CRS and append EPSG code to the end of the file name.  \n####Supported Types  \n`.shp`\n####Usage  \n**variables:**  \n\n```\nROADS # the name of your OSM roads layer file, must have a type field name and be named roads.shp\nTSRS # target CRS\nPRJ # extracts EPSG code from T_SRS to append to filename\nDIR_OUT # set absolute path for output .shp files here\n````\n`cd` to folder containing `roads.shp` and do: `./separate-roads-by-type.sh`\n####Sample Output  \n\n```\nroads_motorways_2227.shp  \nroads_motorway-links_2227.shp   \nroads_main-rd_2227.shp  \nroads_main-rd-links_2227.shp    \nroads_other-rd_2227.shp  \nroads_dirt-rd_2227.shp            \nroads_trails_2227.shp\n```\n\n### shp-to-geojson.sh\nConverts all shapefiles in a directory `$DIR1` into geoJSON files in `$DIR2` and reprojects to `EPSG:4326`.  \n\nIf `$DIR1` is not specified, the script looks in the present working directory.  If `$DIR2` is not specificied, the script will place the new geoJSON files in a new subdirectory called `geojson`.\n\n**Note:** if on Mac OSX you must specify input and output directories or script will fail due to stupid unix differences between Linux and OSX.  \n####Supported Types  \n`.shp`\n####Usage  \n`./shp-to-geojson.sh $DIR1 $DIR2`\n####Sample Output  \n```\n91764 bytes\nconverting file: ./roads_2277.shp into roads_2277.geojson...\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclhenrick%2Fshell_scripts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fclhenrick%2Fshell_scripts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclhenrick%2Fshell_scripts/lists"}