{"id":15002635,"url":"https://github.com/tailormap/tailormap-data","last_synced_at":"2025-05-06T22:22:39.318Z","repository":{"id":253950093,"uuid":"844986778","full_name":"Tailormap/tailormap-data","owner":"Tailormap","description":"Provides a stack of databases (PostGIS, Oracle Free, MS SQL Server Developer) with preloaded data and a preconfigured GeoServer and Solr. This stack is meant for demonstration and unit/integration testing purposes.","archived":false,"fork":false,"pushed_at":"2025-05-05T19:18:29.000Z","size":508,"stargazers_count":1,"open_issues_count":1,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-05-05T20:36:09.945Z","etag":null,"topics":["geoserver","oracle","postgis","sqlserver","tailormap"],"latest_commit_sha":null,"homepage":"","language":"Scheme","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Tailormap.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-08-20T11:07:24.000Z","updated_at":"2025-04-22T08:10:34.000Z","dependencies_parsed_at":null,"dependency_job_id":"26d35c2d-bd69-4d9a-8ed7-fe333716441d","html_url":"https://github.com/Tailormap/tailormap-data","commit_stats":{"total_commits":11,"total_committers":2,"mean_commits":5.5,"dds":0.4545454545454546,"last_synced_commit":"ce3ba04449d770f5a9f42503ea7d0638869f3289"},"previous_names":["tailormap/tailormap-data"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tailormap%2Ftailormap-data","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tailormap%2Ftailormap-data/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tailormap%2Ftailormap-data/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tailormap%2Ftailormap-data/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Tailormap","download_url":"https://codeload.github.com/Tailormap/tailormap-data/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252777366,"owners_count":21802590,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["geoserver","oracle","postgis","sqlserver","tailormap"],"created_at":"2024-09-24T18:51:35.101Z","updated_at":"2025-05-06T22:22:39.266Z","avatar_url":"https://github.com/Tailormap.png","language":"Scheme","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Tailormap Data\n\nProvides a stack of databases (PostGIS, Oracle XE, MS SQL Server Developer and a Solr image) with preloaded data and a\npreconfigured GeoServer.\nThis compose stack is meant for demonstration and unit/integration testing purposes; it is not a production stack.\nNotably this\nstack exposes the various database TCP ports on the host (by default binds to 127.0.0.1) and uses plain text passwords\nin the GeoServer datastores.\n\nAfter starting the GeoServer admin password will be changed to one given using the `GEOSERVER_ADMIN_PASSWORD`\nenvironment variable. This incurs a config reload as well.\n\n## Building the Solr image\n\nThe Solr image is build using `docker buildx` from the `solr/Dockerfile` in a multi-platform build. Check the workflow\nfile `.github/workflows/build-publish.yml` for details.\nThis image is used to provide a Solr instance for Tailormap and is available using eg. `docker pull ghcr.io/tailormap/solr:9.7.0`.\n\n## Building and running the stack\n\n**NOTE** You need to have git-lfs installed to get the dump files, if the dump files are missing from\neg. `oracle/dumpdir/` or `tailormap-data/mssql/docker-entrypoint-initdb.d`\n\n### Build\n\n`GEOSERVER_DATA_DIR=/opt/geoserver-data docker compose -f ./docker-compose.yml build --pull --parallel --compress --no-cache`\nwill build but not run the stack\n\n### Deploy\n\n**NOTE** For integration testing `tailormap-api` do not use this stack but use the `docker-compose.yml` file in\n`build/ci/` of the Tailormap API project. Use the above command to build the images locally though.\n\n![Deployment diagram](docker-compose.VizFormats.svg)\n\nFor Bash:\n\n```bash\n# create external networks\ndocker network create tailormap-viewer\ndocker network create traefik\n# bind to all ip addresses of the host (insecure!)\nexport BIND_IP=0.0.0.0\nexport GEOSERVER_ADMIN_PASSWORD=geoserver123\nexport GEODATA_PASSWORD=07d8313e-75b1-46\nexport GEOSERVER_DATA_DIR=/opt/geoserver-data\n\ndocker compose -f ./docker-compose.yml up -d --build\n```\n\nwill (re)build what is needed and start the containers in this stack.\n\nFor Powershell:\n\n```ps1\ndocker network create tailormap-viewer\ndocker network create traefik\n\n# bind to all ip addresses of the host (insecure!)\n$env:BIND_IP=\"0.0.0.0\"\n$env:GEOSERVER_DATA_DIR=\"/opt/geoserver-data\"\n$env:GEOSERVER_ADMIN_PASSWORD=\"geoserver123\"\n$env:GEODATA_PASSWORD=\"07d8313e-75b1-46\"\n\ndocker compose -f ./docker-compose.yml up -d --build\n\n```\n\n_Note that on Windows you probably need to open TCP firewall ports for GeoServer (8080), Oracle (1521), PostgreSQL (\n5432)\nand SQL Server (1433)._\n\nTo rebuild and deploy a single service (eg. sqlserver)\n`docker compose -f ./docker-compose.yml up -d --no-deps --build --force-recreate sqlserver`\n\n### Cleanup\n\nCleanup all built images and volumes (including persistent data!) using\n`docker compose -f ./docker-compose.yml down --rmi all -v`\n\n## Testdata in these images\n\n### BGT\n\nA small area of the Dutch BGT (Basisregistratie Grootschalige Topografie) is included, the area is:\n`SRID=28992;Polygon ((130011 458031, 130011 459995, 132703 459995, 132703 458031, 130011 458031))`\nA small area of OpenStreetMap is loaded into PostGIS only (bounding box: `4.9715,52.0907,5.1237,52.1558`).\nSee below on how this data was prepared for each database to be loaded at container/volume instantiation.\n\n## PostGIS\n\nYou can connect to the PostGIS `geodata` database in the `tailormap-data` network or on the `host` network on port 5432\nusing the `geodata` user.\n\n### datasets\n\n#### BGT\n\nThe BGT dataset was loaded using the command:\n\n```bash\ndocker run -it --rm --network=host ghcr.io/b3partners/brmo-bgt-loader:snapshot download initial \\\n                    --geo-filter=\"Polygon ((130011 458031, 130011 459995, 132703 459995, 132703 458031, 130011 458031))\" \\\n                    --connection=\"jdbc:postgresql://localhost:5432/geodata?sslmode=disable\u0026reWriteBatchedInserts=true\" \\\n                    --user=geodata --password=07d8313e-75b1-45\n\n```\n\nNote that the postgis image will load any `*.sql` and `*.sql.gz` into the default (postgres) database, we add `.dump`\n\n```bash\ndocker-compose exec -u postgres postgis pg_dump --no-owner --compress=9 -xE UTF-8 \\\n      --exclude-table=spatial_ref_sys,geography_columns,geometry_columns,brmo_metadata \\\n      -S postgres --disable-triggers --dbname=geodata -f /tmp/dump.sql.gz\n# Docker Compose 2 docker compose cp postgis:/tmp/dump.sql.gz postgis/initdb/geodata.sql.gz\ndocker cp postgis:/tmp/dump.sql.gz postgis/docker-entrypoint-initdb.d/2-geodata.sql.gz.dump\n```\n\n#### OSM\n\n**Note:** OpenStreetMap-data are provided under the Open Data Commons Open Database (ODbL) licence.\n\nThe OSM dataset is derived from Geofabrik using the commands:\n\n```bash\nwget https://download.geofabrik.de/europe/netherlands/utrecht-latest.osm.pbf -o /tmp/utrecht-latest.osm.pbf\nosmium extract -b 4.9715,52.0907,5.1237,52.1558 /tmp/utrecht-latest.osm.pbf -o postgis/docker-entrypoint-initdb.d/2-geodata.osm.pbf --overwrite\n\n```\n\n## Oracle\n\nYou can connect to the Oracle `FREEPDB1` service in the `tailormap-data` network or on the `host`\nnetwork on port 1521 using the `geodata` user.\n\n### datasets\n\n#### BGT\n\nThe BGT dataset was loaded using the command:\n\n```bash\ndocker run -it --rm --network=host ghcr.io/b3partners/brmo-bgt-loader:snapshot download initial \\\n                    --geo-filter=\"Polygon ((130011 458031, 130011 459995, 132703 459995, 132703 458031, 130011 458031))\" \\\n                    --connection=\"jdbc:oracle:thin:@localhost:1521/FREEPDB1\" \\\n                    --user=geodata --password=\"07d8313e-75b1-45\"\n```\n\nand exported using datapump:\n\n```bash\ndocker exec oracle rm -f /tmp/dumpdir/*\ndocker-compose exec -u oracle oracle expdp geodata/07d8313e-75b1-45@localhost:1521/FREEPDB1 \\\n                    DUMPFILE=dumpdir:geodata.dump LOGFILE=dumpdir:geodata.log SCHEMAS=geodata\ndocker cp oracle:/tmp/dumpdir/ oracle/\n```\n\nNote that the `dumpdir` is created during database init.\n\n## SQL Server\n\nYou can connect to the SQL Server `geodata` database in the `tailormap-data` network or on the `host` network\non port 1433 using the `geodata` user.\n\n### datasets\n\n#### BGT\n\nThe BGT dataset was loaded using the command:\n\n```shell\ndocker run -it --rm --network=host ghcr.io/b3partners/brmo-bgt-loader:snapshot download initial \\\n                    --geo-filter=\"Polygon ((130011 458031, 130011 459995, 132703 459995, 132703 458031, 130011 458031))\" \\\n                    --connection=\"jdbc:sqlserver://localhost:1433;databaseName=geodata;encrypt=false\" \\\n                    --user=geodata --password=\"07d8313e-75b1-45\"\n```\n\nAfter loading the BGT a `geometry_columns` table was added using `geometry_columns.sql` and then exported using backup.\n\n```shell\ndocker exec sqlserver rm -f /tmp/geodata.bak\ndocker-compose exec -u mssql sqlserver /opt/mssql-tools18/bin/sqlcmd -S localhost \\\n              -Q \"BACKUP DATABASE geodata TO DISK='/tmp/geodata.bak'\" -U geodata -P \"07d8313e-75b1-45\"\ndocker cp sqlserver:/tmp/geodata.bak mssql/docker-entrypoint-initdb.d/geodata.backup.dump\n```\n\n## GeoServer\n\nGeoserver comes preconfigured with three `geodata` datastores from the above databases (postgis/oracle/sqlserver) and\none\n`osm` datastore from postgis only. For each database there is a workspace (postgis/oracle/sqlserver) that has a few\nlayers published.\nThe global WMS config is limited to EPSG:28992, EPSG:4326 and EPSG:3857 and creating tiled layers by default is off.\nWCS is turned off.\n\nYou can rebuild and restart the geoserver container using\n`GEOSERVER_DATA_DIR=/opt/geoserver-data GEOSERVER_ADMIN_PASSWORD=geoserver123 GEOSERVER_CSRF_WHITELIST=localhost docker-compose up -dV --no-deps --build --force-recreate geoserver`\n\nIf you want to make changes to the image start the stack and then make those changes, copy the datadir from the\ncontainer to a local directory using `docker cp geoserver:/opt/geoserver-data geoserver/maven/src/main/webapp/`.\nThen copy/update the files in the data directory after which you can create a pull request with the changes.\n\n### environment\n\n- **GEOSERVER_DATA_DIR** the location of the Geoserver data directory, normally `/opt/geoserver-data`\n- **GEOSERVER_CSRF_WHITELIST** A list of allowed domains for the CSRF tokens,\n  see: https://docs.geoserver.org/stable/en/user/security/webadmin/csrf.html\n- **GEOSERVER_CSRF_DISABLED** enable or disable CSRF tokens,\n  see: https://docs.geoserver.org/stable/en/user/security/webadmin/csrf.html\n\n## Usage\n\nYou can add the GeoServer services in Tailormap using the url\n`http://geoserver:8080/geoserver/ows?service=WMS\u0026request=GetCapabilities` if you are using a local deployment or\n`https://snapshot.tailormap.nl/geoserver/ows?service=WMS\u0026request=GetCapabilities` when using the deployment\non the h03.b3p.nl server\n\nNote that you may need to update the proxy url in the GeoServer settings and the `cookie-config` in the `web.xml` file.\n\nYou can add the attribute sources in Tailormap admin using `oracle`, `postgis` or `sqlserver` as the hostname\nand using the `geodata` user.\n\nFor Oracle you must use database `/FREEPDB1` and schema `GEODATA`,\nfor PostGIS you must use database `geodata` with schemas `public` and/or `osm`,\nfor SQL Server you must use database `geodata;encrypt=false` with schema `dbo` because the SQL Server JDBC driver\ndefaults to using TLS with and SQL Server does that with a self-signed certificate by default. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftailormap%2Ftailormap-data","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftailormap%2Ftailormap-data","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftailormap%2Ftailormap-data/lists"}