{"id":21614917,"url":"https://github.com/eea/gioland","last_synced_at":"2026-05-02T23:41:11.349Z","repository":{"id":3914026,"uuid":"5003098","full_name":"eea/gioland","owner":"eea","description":"Platform where service providers can deliver large files containing GIS data","archived":false,"fork":false,"pushed_at":"2022-07-06T19:49:37.000Z","size":10049,"stargazers_count":1,"open_issues_count":6,"forks_count":0,"subscribers_count":48,"default_branch":"master","last_synced_at":"2025-03-18T16:55:51.281Z","etag":null,"topics":["copernicus","docker","flask"],"latest_commit_sha":null,"homepage":"https://gaur.eea.europa.eu/gioland/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eea.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-07-12T12:31:11.000Z","updated_at":"2019-06-24T14:48:00.000Z","dependencies_parsed_at":"2022-08-30T12:41:37.678Z","dependency_job_id":null,"html_url":"https://github.com/eea/gioland","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/eea/gioland","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eea%2Fgioland","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eea%2Fgioland/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eea%2Fgioland/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eea%2Fgioland/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eea","download_url":"https://codeload.github.com/eea/gioland/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eea%2Fgioland/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276267851,"owners_count":25613281,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-21T02:00:07.055Z","response_time":72,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["copernicus","docker","flask"],"created_at":"2024-11-24T22:10:01.906Z","updated_at":"2025-09-21T16:21:53.433Z","avatar_url":"https://github.com/eea.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"GioLand upload service\n======================\n\nThe GioLand upload service is a platform where service providers can\ndeliver large files containing GIS data. Deliveries are tagged with:\ncountry, theme, projection, spatial resolution, extent. Each delivery\ngoes through a workflow of checks, resubmissions and enhancement. See\n[this issue](http://taskman.eionet.europa.eu/issues/2) for the initial requirements.\n\n[![Travis](https://travis-ci.org/eea/gioland.svg?branch=master)](https://travis-ci.org/eea/gioland)\n[![Coverage](https://coveralls.io/repos/github/eea/gioland/badge.svg?branch=master)](https://coveralls.io/github/eea/gioland?branch=master)\n[![Docker]( https://dockerbuildbadges.quelltext.eu/status.svg?organization=eeacms\u0026repository=gioland)](https://hub.docker.com/r/eeacms/gioland/builds)\n\n### Prerequisites\n\n* Install [Docker](https://docs.docker.com/engine/installation/)\n* Install [Docker Compose](https://docs.docker.com/compose/install/)\n\n### Installing the application\n\n1. Get the source code:\n\n       $ git clone https://github.com/eea/gioland.git\n       $ cd gioland/\n\n2. Customize env file:\n\n        $ cp docker/app.env.example docker/app.env\n        $ vim docker/app.env\n\n3. Start application stack:\n\n        $ docker-compose up -d\n        $ docker-compose logs\n\n### Configuration variables\n\nThe application expects configuration via environment variables:\n\n``DEBUG``\n    Turns on debugging behaviour if set to ``on``. Not secure for use in\n    production.\n\n``WAREHOUSE_PATH``\n    Path to folder containing the database and uploaded files.\n\n``LOCK_FILE_PATH``\n    Path to lockfile used to synchronize access to chunked file uploads.\n\n``SENTRY_DSN``\n    URL of Sentry server to report errors.\n\n``SECRET_KEY``\n    Random secret used for Flask browser sessions.\n\n``ROLE_SP``, ``ROLE_ETC``, ``ROLE_NRC``, ``ROLE_ADMIN``, ``ROLE_VIEWER``\n    Space-separated lists of principals for that role. Principals can be\n    in the format ``user_id:NAME`` or ``ldap_group:NAME``.\n\n``BASE_URL``\n    Base URL of the application. Necessary to generate correct URLs.\n\n``UNS_CHANNEL_ID``, ``UNS_LOGIN_USERNAME``, ``UNS_LOGIN_PASSWORD``\n    Credentials for sending notifications via UNS.\n\n``UNS_SUPPRESS_NOTIFICATIONS``\n    If ``on``, don't send any UNS notifications.\n\n``LDAP_SERVER``, ``LDAP_USER_DN_PATTERN``\n    Server and DN pattern for connecting to LDAP. For example\n    ``ldap://ldap3.eionet.europa.eu`` and\n    ``uid={user_id},ou=Users,o=EIONET,l=Europe``.\n\n### Development notes\n\n#### Data model\n\nEach service provider delivery goes through the following stages:\n\n* ``int`` (Service provider upload)\n* ``sch`` (Semantic check)\n* ``ver`` (Verification)\n* ``vch`` (Verification check)\n* ``enh`` (Enhancement)\n* ``ech`` (Enhancement check)\n* ``fin`` (Final integrated)\n* ``fva`` (Final validated)\n\nThe initial upload is made in a \"parcel\" (think of it as a folder).\nSubsequent steps in the workflow each have their own parcel, where more\nfiles can be uploaded. Parcels have back-forward links so each delivery\nis a chain of parcels. If a workflow step (e.g. Verification check)\nresults in a rejection, a new parcel of the previous step is created, so\nthe chain can loop back if needed.\n\nEach delivery is tagged with the following metadata fields, which are\ncopied over from parcel to parcel:\n\n* country\n* theme\n* projection\n* resolution\n* extent\n* coverage\n\n\n#### Database\n\nThe application stores data in a ZODB database in\n``$WAREHOUSE_PATH/filestorage``. The ``warehouse.py`` module is\nresponsible for connecting to the database and contains the models.\nUploaded files are stored in the filesystem, under\n``$WAREHOUSE_PATH/parcels``, where each parcel has its own folder. Since\nthe files are accessed from a remote machine via CIFS, for automated GIS\nprocessing, a tree of symlinks is maintained in\n``$WAREHOUSE_PATH/tree``, where the path is generated using the metadata\nfields of each parcel.\n\n\n#### Notifications\n\nUploads and other workflow steps trigger notifications to relevant\npeople. They are sent via UNS_.\n\n.. _UNS: http://uns.eionet.europa.eu/\n\n\n#### Large files\n\nService providers upload very large files (in the order of 20GB). This\nis done via HTTP, with the file split in 1MB chunks, and reassembled on\nthe server. The chunks are saved in a temporary folder in the parcel.\n\n\n### Contacts\n\nThe project owner is Alan Steel (alan.steel at eaa.europa.eu)\n\nOther people involved in this project are:\n\n* Andreea Dima (andreea.dima at eaudeweb.ro)\n* Diana Boiangiu (diana.boiangiu at eaudeweb.ro)\n\n\n### Resources\n\nMinimum requirements: 256MB RAM; 1 CPU\n\nThe production server needs a lot of hard disk space because raster map\nimages are uploaded there.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feea%2Fgioland","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feea%2Fgioland","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feea%2Fgioland/lists"}