{"id":13412984,"url":"https://github.com/hishamkaram/gismanager","last_synced_at":"2026-05-07T09:01:06.665Z","repository":{"id":57496842,"uuid":"150863977","full_name":"hishamkaram/gismanager","owner":"hishamkaram","description":"Publish Your GIS Data(Vector Data) to PostGIS and Geoserver","archived":false,"fork":false,"pushed_at":"2026-05-04T14:12:53.000Z","size":500,"stargazers_count":54,"open_issues_count":2,"forks_count":10,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-05-04T14:18:07.240Z","etag":null,"topics":["gdal","geoserver","gis","golang","osgeo","postgis"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hishamkaram.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-09-29T12:51:37.000Z","updated_at":"2026-05-04T13:36:41.000Z","dependencies_parsed_at":"2022-09-03T02:10:33.882Z","dependency_job_id":null,"html_url":"https://github.com/hishamkaram/gismanager","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/hishamkaram/gismanager","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hishamkaram%2Fgismanager","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hishamkaram%2Fgismanager/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hishamkaram%2Fgismanager/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hishamkaram%2Fgismanager/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hishamkaram","download_url":"https://codeload.github.com/hishamkaram/gismanager/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hishamkaram%2Fgismanager/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32730282,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-07T02:14:30.463Z","status":"ssl_error","status_checked_at":"2026-05-07T02:14:29.405Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gdal","geoserver","gis","golang","osgeo","postgis"],"created_at":"2024-07-30T20:01:31.946Z","updated_at":"2026-05-07T09:01:06.655Z","avatar_url":"https://github.com/hishamkaram.png","language":"Go","funding_links":[],"categories":["Go","Geographic","Relational Databases","地理","位置信息与地理GEO处理库"],"sub_categories":["Search and Analytic Databases","Advanced Console UIs","检索及分析资料库","SQL 查询语句构建库"],"readme":"[![Go Reference](https://pkg.go.dev/badge/github.com/hishamkaram/gismanager/v2.svg)](https://pkg.go.dev/github.com/hishamkaram/gismanager/v2)\n[![Go Report Card](https://goreportcard.com/badge/github.com/hishamkaram/gismanager/v2)](https://goreportcard.com/report/github.com/hishamkaram/gismanager/v2)\n[![GitHub license](https://img.shields.io/github/license/hishamkaram/gismanager.svg)](https://github.com/hishamkaram/gismanager/blob/master/LICENSE)\n[![GitHub issues](https://img.shields.io/github/issues/hishamkaram/gismanager.svg)](https://github.com/hishamkaram/gismanager/issues)\n\n# GIS Manager\n\n**A Go tool that publishes GIS vector data to GeoServer via PostGIS.** Drop a directory of shapefiles, GeoJSON, GeoPackage, or KML files into a config, run one command, and gismanager loads each file into a PostGIS database and registers the resulting tables as GeoServer feature types — workspace creation, datastore registration, and layer publishing all idempotent.\n\nIt exists because the path from \"I have a shapefile\" to \"I have a published GeoServer layer\" otherwise involves three different tools (ogr2ogr, psql, and the GeoServer admin UI or REST API). gismanager is one CLI plus one config.\n\n## Contents\n\n- [What this tool does](#what-this-tool-does)\n- [Install](#install)\n- [Quick start](#quick-start)\n- [Worked example: publish a directory of GIS files](#worked-example-publish-a-directory-of-gis-files)\n- [Conversion](#conversion)\n- [Errors](#errors)\n- [Configuration](#configuration)\n- [Library use](#library-use)\n- [Version compatibility](#version-compatibility)\n- [Contributing](#contributing)\n- [License](#license)\n\n## What this tool does\n\nThree CLI binaries plus a Go library covering two GIS workflows:\n\n- **`gismanager`** — full publish pipeline. Walks `source.path`, picks every supported GIS file, copies its layers into PostGIS via OGR's PostgreSQL driver, then publishes each PostGIS table as a GeoServer feature type. Idempotent: workspace, datastore, and feature-type creation all check existence first via the `geoserver/v2` client's `Get` + `errors.Is(err, geoserver.ErrNotFound)` idiom.\n- **`layerSchema`** — read-only schema inspector. Walks `source.path`, opens each GIS file via GDAL, and prints the geometry column + attribute fields for every layer it finds. No PostGIS, no GeoServer.\n- **`gisconvert`** *(v1.2+)* — data-conversion CLI: vector format conversion (`ogr2ogr` equivalent), raster format conversion (`gdal_translate` equivalent), Cloud-Optimized GeoTIFF generation, and raster reprojection (`gdalwarp` equivalent). See [Conversion](#conversion).\n- **`package gismanager`** — both flows as a Go library:\n  - **Publish**: construct a `*publish.Manager` via `publish.New(opts ...Option)` (functional-options) or the YAML-driven `publish.FromConfig(yamlPath)`, then call `Walk` / `PublishAll` or the lower-level `OpenSource` / `LayerToPostgis` / `PublishGeoserverLayer` primitives.\n  - **Convert** *(v1.2+)*: stateless `ConvertVector` / `ConvertRaster` / `ToCOG` / `ReprojectRaster` package functions. No `*publish.Manager` required.\n\n### Supported source formats\n\nThe two surfaces have different reach.\n\n#### Publish pipeline (`cmd/gismanager`, `Walk`, `PublishAll`)\n\nDriven by an explicit extension allowlist in [`publish/vars.go::supportedEXT`](publish/vars.go) — only these extensions are picked up by directory walks:\n\n| Extension | OGR driver |\n|---|---|\n| `.shp`, `.zip` (zipped shapefile bundle) | ESRI Shapefile |\n| `.geojson`, `.json` | GeoJSON |\n| `.gpkg` | GeoPackage |\n| `.kml` | KML |\n\nZipped shapefile bundles are auto-extracted into a temp directory before the OGR open — see [`internal/zipx`](internal/zipx) for the stdlib `archive/zip`-based extractor (zip-slip rejection, 2 GiB per-entry cap). Adding a new extension to the publish pipeline means adding a row to `supportedEXT` and a switch case in [`publish/manager.go::GetDriver`](publish/manager.go).\n\n#### Conversion entry points *(v1.2+, `cmd/gisconvert`, `ConvertVector` / `ConvertRaster` / `ToCOG` / `ReprojectRaster`)*\n\nNo allowlist — the conversion functions delegate straight to GDAL's full driver registry via `gdal.OpenEx`. Whatever the underlying GDAL build can read or write, the conversion entry points accept. In the project's dev image (`ghcr.io/osgeo/gdal:ubuntu-small-3.12.4`) that includes the obvious workhorses:\n\n- **Vector** (OGR): Shapefile, GeoPackage, GeoJSON, FlatGeobuf, KML, GML, GPX, MapInfo TAB/MIF, CSV (with WKT/lon-lat geometry), DGN, S-57, PostgreSQL/PostGIS, plus the cloud-native paths via VFS (`/vsis3/`, `/vsicurl/`, `/vsizip/`, `/vsimem/`, …).\n- **Raster** (GDAL): GeoTIFF, Cloud-Optimized GeoTIFF, JP2/OpenJPEG, PNG, JPEG, NITF, HFA (Erdas .img), VRT, plus the same VFS paths.\n\nFor an authoritative list against any GDAL build, run `ogrinfo --formats` (vector) and `gdalinfo --formats` (raster) inside the dev container — driver compile-time inclusion varies across GDAL builds, and the dev image is the supported baseline.\n\n## Install\n\n**This project does NOT install GDAL on the host machine.** All build, test, and run work happens inside the Docker dev image (`ghcr.io/osgeo/gdal:ubuntu-small-3.12.4` base + Go 1.25.9 + tooling). If you don't have Docker, [install Docker first](https://docs.docker.com/get-docker/).\n\n```bash\ngit clone https://github.com/hishamkaram/gismanager\ncd gismanager\nmake dev          # opens an interactive bash inside the container\n# OR run targets non-interactively:\nmake build        # go build ./...\nmake test-unit    # unit tests (no live GeoServer)\nmake image        # produce a runtime image: gismanager:local\n```\n\nThe runtime image (`make image`) is a multi-stage build that ships only the binaries + libgdal at `~500 MB`. It's what you'd publish to a registry for production-ish use.\n\n## Quick start\n\nThe shortest path to a published layer assumes you have a GeoServer + PostGIS already running somewhere reachable. If you don't, the project's own integration test stack is a working example — `make compose-test-up` boots GeoServer 2.28.0 + PostGIS 16 in two containers.\n\n1. Write a config file. Example `my-config.yaml`:\n\n   ```yaml\n   geoserver:\n     url: http://localhost:8080/geoserver\n     username: admin\n     password: geoserver\n     workspace: my_workspace\n   datastore:\n     host: localhost\n     port: 5432\n     database: gis\n     username: golang\n     password: golang\n     name: my_postgis_store\n   source:\n     path: ./testdata\n   ```\n\n2. Run the CLI from the runtime image:\n\n   ```bash\n   docker run --rm --network host \\\n     -v \"$PWD/my-config.yaml:/cfg.yaml:ro\" \\\n     -v \"$PWD/testdata:/testdata:ro\" \\\n     gismanager:local --config /cfg.yaml\n   ```\n\n   gismanager scans `/testdata`, loads each supported file into PostGIS, and publishes the resulting tables as feature types in the `my_workspace` workspace.\n\n## Worked example: publish a directory of GIS files\n\nThe repo's own `testdata/` directory has a GeoJSON file and a GeoPackage. Booting the integration stack and publishing them end-to-end:\n\n```bash\n# Boot GeoServer 2.28.0 + PostGIS 16 in containers.\nmake compose-test-up\n\n# Run the CLI inside the test-runner container so it can see the\n# in-network GeoServer + PostGIS by service name.\ndocker compose -f docker-compose.test.yml run --rm test-runner \\\n  bash -c 'go run ./cmd/gismanager --config testdata/test_config.yml'\n\n# Verify via the GeoServer REST API.\ncurl -fsS -u admin:geoserver \\\n  http://localhost:8080/geoserver/rest/workspaces/golang/datastores/gismanager_data/featuretypes.json | jq\n\n# Tear down.\nmake compose-test-down\n```\n\nSame idea programmatically (the integration suite is a worked example — see [`publish/publish_integration_test.go`](publish/publish_integration_test.go) `TestPublishGeoJSON_EndToEnd_Integration`).\n\n## Conversion\n\nBeyond the publish pipeline, gismanager ships a stateless conversion subsystem that mirrors GDAL's command-line workhorses:\n\n| Function | Wraps | Use case | Since |\n|---|---|---|---|\n| `ConvertVector` | `ogr2ogr` | vector format conversion + reproject + bbox clip + attribute filter + simplify | v1.2 |\n| `ConvertRaster` | `gdal_translate` | raster format conversion + band subset + output window | v1.2 |\n| `ToCOG` | `gdal_translate -of COG` | Cloud-Optimized GeoTIFF with sane defaults pre-applied | v1.2 |\n| `ReprojectRaster` | `gdalwarp` | raster reprojection + cookie-cutter clip via cutline | v1.2 |\n| `Rasterize` | `gdal_rasterize` | vector → raster: burn polygons into a mask, or attributes into a continuous field | v1.3 |\n| `BuildVRT` | `gdalbuildvrt` | mosaic many GeoTIFFs into a Virtual Raster (tile pyramid prep, RGBA stacking) | v1.3 |\n| `DEMProcessing` | `gdaldem` | DEM analysis: hillshade, slope, aspect, color-relief, TRI, TPI, roughness | v1.3 |\n\nAll seven are top-level package functions — no `*publish.Manager` required — and accept `/vsi*/`-prefixed paths transparently (`/vsis3/`, `/vsicurl/`, `/vsimem/`, `/vsizip/`, `/vsigs/`, `/vsiaz/`). Driver names supplied via the `With*Format` helpers are pre-validated against the running GDAL build (since v1.3) — an unknown driver surfaces as a clean `ErrConvertFailed` instead of GDAL's silent fail-with-stderr-warning behavior.\n\n```go\n// Vector: 4326 GeoJSON → 3857 GeoPackage, clipped to Africa, simplified.\nerr := convert.ConvertVector(ctx, \"world.geojson\", \"africa.gpkg\",\n    convert.WithVectorFormat(\"GPKG\"),\n    convert.WithVectorOverwrite(),\n    convert.WithVectorTargetSRS(\"EPSG:3857\"),\n    convert.WithVectorBoundingBox(-25, -40, 60, 40),\n    convert.WithVectorWhere(\"CONTINENT = 'Africa'\"),\n    convert.WithVectorSimplify(100),\n)\n\n// Raster: GeoTIFF → COG with the canonical defaults.\nerr = convert.ToCOG(ctx, \"scene.tif\", \"scene.cog.tif\")\n\n// Raster reprojection: UTM → Web Mercator.\nerr = convert.ReprojectRaster(ctx, \"utm.tif\", \"wm.tif\",\n    \"EPSG:32618\", \"EPSG:3857\",\n    convert.WithRasterResamplingAlg(\"bilinear\"),\n)\n```\n\nFull reference, more examples, and the cloud-I/O matrix: [`docs/conversions.md`](docs/conversions.md).\n\n## Errors\n\nEvery error gismanager returns is a `*GISError` wrapping a sentinel from [`errs/errs.go`](errs/errs.go). Match by sentinel:\n\n```go\nimport \"github.com/hishamkaram/gismanager/v2/publish\"\n\nerr := mgr.PublishGeoserverLayer(ctx, layer)\nswitch {\ncase errors.Is(err, errs.ErrPostGISConnect):\n    // PostGIS unreachable; bring it up and retry.\ncase errors.Is(err, errs.ErrGeoServerPublish):\n    // Workspace/datastore/feature-type creation failed; the wrapped\n    // *geoserver.APIError carries the HTTP status (404, 409, 500…).\n    var apiErr *geoserver.APIError\n    if errors.As(err, \u0026apiErr) {\n        log.Printf(\"status=%d body=%s\", apiErr.StatusCode, apiErr.Body)\n    }\ncase errors.Is(err, errs.ErrUnsupportedFormat):\n    // The path's extension isn't one of the dispatched OGR drivers.\n}\n```\n\nSentinels: `ErrConfigInvalid`, `ErrUnsupportedFormat`, `ErrInvalidLayer`, `ErrInvalidDatasource`, `ErrPostGISConnect`, `ErrGeoServerPublish`, `ErrNoSourcesFound`. Never compare error strings — `errors.Is(err, sentinel)` is the only correct test.\n\n## Observability\n\nThe library emits structured logs via `*slog.Logger` throughout the publish\npipeline and the conversion subsystem. Pass a custom logger via\n`WithLogger` to control formatting, filtering, and routing.\n\nFor production deploys with trace correlation (every library log gets the\nactive span's `trace_id` / `span_id`), use the\n[OpenTelemetry slog bridge](https://pkg.go.dev/go.opentelemetry.io/contrib/bridges/otelslog).\nA runnable end-to-end example lives at\n[`examples/otel_pipeline/`](./examples/otel_pipeline) (its own Go submodule\nso the OTel SDK doesn't bloat gismanager's runtime dep surface). See\n[`docs/observability.md`](./docs/observability.md) for the architectural\npattern and a Kubernetes deployment recipe.\n\n## Configuration\n\nYAML schema (the same struct gets used by `publish.FromConfig`):\n\n```yaml\ngeoserver:\n  url: \u003cstring\u003e          # GeoServer base URL, e.g. http://geoserver:8080/geoserver\n  username: \u003cstring\u003e     # admin user\n  password: \u003cstring\u003e\n  workspace: \u003cstring\u003e    # workspace gismanager publishes into; created on first use\n\ndatastore:               # connection params for the PostGIS database\n  host: \u003cstring\u003e\n  port: \u003cuint\u003e\n  database: \u003cstring\u003e     # database name (created externally; gismanager does not provision databases)\n  username: \u003cstring\u003e\n  password: \u003cstring\u003e\n  name: \u003cstring\u003e         # GeoServer datastore name (must be unique within the workspace)\n\nsource:\n  path: \u003cstring\u003e         # directory or single file containing GIS data\n```\n\nA working config used by the integration suite lives at [`testdata/test_config.yml`](testdata/test_config.yml).\n\n## Library use\n\n### Publish pipeline\n\n```go\nimport (\n    \"context\"\n    \"github.com/hishamkaram/gismanager/v2/publish\"\n)\n\nfunc main() {\n    mgr, err := publish.FromConfig(\"my-config.yaml\")\n    if err != nil { /* ... */ }\n\n    ctx := context.Background()\n\n    // Convenience: walk + load + publish in one call.\n    if err := mgr.PublishAll(ctx); err != nil { /* ... */ }\n\n    // Or stream-iterate manually for finer control:\n    for item, err := range mgr.Walk(ctx) {\n        if err != nil { continue }\n        // item.Layer is logger-stamped via NewLayer; use it freely.\n        _ = item.Layer\n    }\n\n    // Or call the low-level primitives directly (note the defer to\n    // release CGo handles — Walk / PublishAll do this automatically):\n    target, err := mgr.OpenSource(ctx, mgr.Datastore.BuildConnectionString(), 1)\n    if err != nil { /* ... */ }\n    defer target.Destroy()\n\n    src, err := mgr.OpenSource(ctx, \"/path/to/file.geojson\", 0)\n    if err != nil { /* ... */ }\n    defer src.Destroy()\n\n    for i := 0; i \u003c src.LayerCount(); i++ {\n        layer := src.LayerByIndex(i)\n        wrapped := mgr.NewLayer(\u0026layer)\n        newLayer, err := wrapped.LayerToPostgis(target, mgr, true)\n        if err != nil || newLayer == nil { continue }\n        _ = mgr.PublishGeoserverLayer(ctx, newLayer)\n    }\n}\n```\n\n\u003e **Resource lifecycle:** `*gdal.DataSource` returned by `OpenSource` owns a CGo-side handle that must be released via `.Destroy()` to avoid leaks. `Walk` and `PublishAll` do this automatically (`defer source.Destroy()` per file). Direct callers of `OpenSource` are responsible for calling `Destroy()` themselves — the binding has no `Close()` method; `Destroy()` is the canonical release primitive in `lukeroth/gdal`.\n\n### Conversion *(v1.2+)*\n\nThe conversion entry points are top-level package functions — no `*publish.Manager` required, no GeoServer/PostGIS state held.\n\n```go\n// Reproject + clip + filter + simplify in one call.\nerr := convert.ConvertVector(ctx, \"world.geojson\", \"africa.gpkg\",\n    convert.WithVectorFormat(\"GPKG\"),\n    convert.WithVectorOverwrite(),\n    convert.WithVectorTargetSRS(\"EPSG:3857\"),\n    convert.WithVectorBoundingBox(-25, -40, 60, 40),\n    convert.WithVectorWhere(\"CONTINENT = 'Africa'\"),\n    convert.WithVectorSimplify(100),\n)\n\n// Cloud-Optimized GeoTIFF with sane defaults.\nerr = convert.ToCOG(ctx, \"scene.tif\", \"scene.cog.tif\")\n\n// UTM → Web Mercator with bilinear resampling.\nerr = convert.ReprojectRaster(ctx, \"utm.tif\", \"wm.tif\",\n    \"EPSG:32618\", \"EPSG:3857\",\n    convert.WithRasterResamplingAlg(\"bilinear\"),\n)\n\n// v1.3+: vector → raster (burn an attribute into a continuous Float32 field).\nerr = convert.Rasterize(ctx, \"countries.geojson\", \"pop.tif\",\n    convert.WithRasterizeFormat(\"GTiff\"),\n    convert.WithRasterizeOutputType(\"Float32\"),\n    convert.WithRasterizeAttribute(\"POP_EST\"),\n    convert.WithRasterizeOutputSize(360, 180),\n)\n\n// v1.3+: stack many GeoTIFFs into a single VRT for downstream Warp/Translate.\nerr = convert.BuildVRT(ctx, \"mosaic.vrt\",\n    []string{\"tile1.tif\", \"tile2.tif\", \"tile3.tif\"},\n    convert.WithVRTResolution(\"highest\"),\n)\n\n// v1.3+: hillshade from a DEM.\nerr = convert.DEMProcessing(ctx, \"dem.tif\", \"dem.hs.tif\", \"hillshade\",\n    convert.WithDEMAzimuth(315),\n    convert.WithDEMAltitude(45),\n)\n```\n\nA complete 30-line program lives in [`examples/convert_pipeline/main.go`](examples/convert_pipeline/main.go); full reference and the cloud-I/O VFS matrix in [`docs/conversions.md`](docs/conversions.md).\n\n## Version compatibility\n\ngismanager v1.0.x targets:\n\n- **Go 1.25+** — required by the transitive `geoserver/v2` dependency.\n- **GeoServer 2.27 LTS + 2.28 stable** — both validated by the matrix integration leg in CI on every PR.\n- **PostGIS 16-3.4** — the integration stack pins this; older PostGIS (≥ 2.5) should work but isn't gated in CI.\n- **GDAL 3.12.x** — pinned via the `ghcr.io/osgeo/gdal:ubuntu-small-3.12.4` base image. Bumping requires re-running the integration suite.\n\nGeoServer 3.0 (Tomcat 11 / Jakarta EE / ImageN) is parked for a future v1.x point release after the upstream migration settles.\n\nFull matrix: [`docs/version-compat.md`](docs/version-compat.md).\n\n## Contributing\n\nSee [`CONTRIBUTING.md`](CONTRIBUTING.md). Highlights:\n\n- All work happens inside the Docker dev image — no host GDAL install.\n- Branch from `master`; squash-merge with `--delete-branch`. Never push directly to `master`.\n- Both unit and integration tests are mandatory on every PR. The matrix CI runs against GeoServer 2.27.4 + 2.28.0; both legs must pass.\n- Conventional Commits.\n\n## License\n\n[MIT](LICENSE) © Hesham Karm.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhishamkaram%2Fgismanager","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhishamkaram%2Fgismanager","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhishamkaram%2Fgismanager/lists"}