{"id":21659865,"url":"https://github.com/slub/kartenforum_georeference","last_synced_at":"2025-04-11T22:40:31.810Z","repository":{"id":38249977,"uuid":"389546910","full_name":"slub/kartenforum_georeference","owner":"slub","description":"Georeference backend for the SLUB Virtual Map Forum (\"Kartenforum\").","archived":false,"fork":false,"pushed_at":"2025-01-07T12:27:52.000Z","size":9840,"stargazers_count":2,"open_issues_count":1,"forks_count":1,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-03-25T18:41:03.182Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/slub.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-07-26T07:30:22.000Z","updated_at":"2025-01-07T12:25:15.000Z","dependencies_parsed_at":"2024-11-25T09:42:57.002Z","dependency_job_id":null,"html_url":"https://github.com/slub/kartenforum_georeference","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slub%2Fkartenforum_georeference","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slub%2Fkartenforum_georeference/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slub%2Fkartenforum_georeference/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slub%2Fkartenforum_georeference/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/slub","download_url":"https://codeload.github.com/slub/kartenforum_georeference/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248493007,"owners_count":21113159,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-25T09:31:47.193Z","updated_at":"2025-04-11T22:40:31.800Z","avatar_url":"https://github.com/slub.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n\t\u003cimg width=\"400\" alt=\"The Lounge\" src=\"docs/_assets/header-logo.svg\"\u003e\n\u003c/h1\u003e\n\n\u003ch1 align=\"center\"\u003eKartenforum Georeference Service\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n    The \u003cstrong\u003eKartenforum Georeference \u003c/strong\u003e provides the georeferencing functionality for the \u003ca href=\"https://kartenforum.slub-dresden.de/\"\u003eVirtual Map Forum 2.0\u003c/a\u003e.\n    This platform allows users to explore and interact with both historical and georeferenced maps. The service is designed to efficiently manage and process georeferenced map data. Using \u003cstrong\u003eFastAPI\u003c/strong\u003e, it delivers a scalable and high-performance API, while \u003cstrong\u003ePostgreSQL\u003c/strong\u003e (and PostGIS) manages spatial data storage and queries.\n\u003c/p\u003e\n\u003cbr\u003e\u003c/br\u003e \n\n\n## 📖 Table of Contents\n\n- [Get Started](#get-started)\n  - [With Docker](#with-docker)\n  - [Without Docker](#without-docker)\n- [Configuration](#configuration)\n- [Linting and Formatting](#linting-and-formatting)\n- [Running Tests](#running-tests)\n- [PostgreSQL Database](#postgresql-database)\n  - [Migrations](#migrations)\n- [Testing with Postgres and Testcontainers](#testing-with-postgres-and-testcontainers)\n- [Debugging](#debugging)\n- [License](#-license)\n\n\n\n## Get Started\n\n### With Docker\n1. Set environment variables for the docker build: There are two environment variables required for the docker build, `USERNAME` and `UID`\n    ```shell\n     export USER=$(whoami) \n     export UID=$(id -u)\n    ```\n    Also see this script https://github.com/pikobytes/slub_ddev_kartenforum/blob/main/generate-env.sh for more\n  information.\n\u003cbr\u003e\u003c/br\u003e\n\n2. Start Docker containers:\n\n    ```shell\n    cd docker/\n    docker-compose up\n    ```\n   \n3. In your IDE make sure to use the correct python interpreter. The one in the `.venv` folder, which should have been \nautomatically added by the docker container.\n\u003cbr\u003e\u003c/br\u003e\n\n4. The server should be available under `http://localhost:8000/`\n\n### Without Docker\n1. Set up a virtual environment and install [poetry](https://python-poetry.org/docs/).\n    ```shell\n    python3 -m venv .venv\n    .venv/bin/pip install poetry==1.8.3\n    ```\n\n2. Install the required dependencies\n    ```shell\n    .venv/bin/poetry install\n    ```\n    In case you experience any issues with python-gdal, make sure the version matches your local gdal version. Also if you\nmake any changes to `pyproject.toml`. Also, if you make any changes to `pyproject.toml` you need to run `.venv/bin/poetry lock` again.\n\u003cbr\u003e\u003c/br\u003e\n3. Set Up the pre-commit hooks\n\n    ```shell\n    .venv/bin/poetry run pre-commit install\n    ```\n   \n4. Run the server:\n    ```shell\n   uvicorn georeference.main:app --reload\n    ```\n\n## Configuration\nConfiguration is managed via environment variables. They can be set in the following ways:\n1. Directly in the environment: These take precedence over everything else.\n2. In a `.env.production` file in the root of the project: This file will be automatically loaded by `pydantic-settings`\n   and will take precedence over the default values defined in `.env`. It does not need to be exhaustive, only the\n   variables that should be overwritten are required.\n3. In a `.env` file: This file is also automatically loaded by `pydantic-settings`.\n\nFor more information, check the [Pydantic documentation](https://docs.pydantic.dev/latest/concepts/pydantic_settings/#dotenv-env-support).\n\n\n## Linting and Formatting\nThis project utilizes [ruff](https://docs.astral.sh/ruff/) for both linting and code formatting, with the configuration \nspecified in the `pyproject.toml` file. To maintain consistency, please ensure that your IDE is configured to use Ruff for code formatting. Additionally, \nlinting and formatting tasks are automatically enforced as pre-commit hooks to help uphold code quality.\n\n\n## Running Tests\nIf you want to run all tests make sure to have [ddev-kartenforum](https://github.com/slub/ddev-kartenforum) locally setup and running.\n1. Please install the following dependencies and download and initialize the test data:\n    \n    ```bash\n    sudo apt install libvips libvips-tools sshpass libvips-tools\n    scripts/initialize-testdata.sh\n    ```\n2. Run all tests:\n    ```shell\n    .venv/bin/poetry run pytest \n    ```\n3. To run specific tests, use the following command structure:\n    ```shell\n    .venv/bin/poetry run pytest -rP georeference/tests/utils/proj_test.py::test_get_crs_from_request_params_use_passed_crs\n    ```\n💡 **Tips:**\n* Ensure Docker is running, as the `testcontainers` library will spin up a PostgreSQL container for the tests.\n* Make sure `DEV_MODE` is enabled to avoid SSL verification failures when querying the TYPO3 application.\n* Properly set up your local environment for testing.\n\n\n## PostgreSQL Database\ndocker can be used to set up a development infrastructure\n\n```\ncd docker/\ndocker-compose up\n```\n\n### Migrations\nTo modify the database schema, use the alembic library for versioned migrations. The database connection URL is managed \nin the settings.py file, which reads from the `.env`.\n\n#### Creating a Migration\nTo create a new migration, run the following command:\n  ```shell\n   poetry run alembic revision -m \"your message\"\n  ```\nThis will generate a new migration file in the `alembic/versions` folder.\nIf the versions folder does not exist inside the alembic directory, please create it manually:\n```shell\nsudo mkdir alembic/versions\n```\n#### Applying a Migration\nAfter making your changes, apply the migration using the command:\n```\npoetry run alembic upgrade head\n```\nThis updates the database schema to the latest version. For more information also check the [alembic documentation](https://alembic.sqlalchemy.org/en/latest/index.html).\n\n\n## Testing with Postgres and Testcontainers\nThe project uses [postgres-test-container](https://github.com/testcontainers/testcontainers-python) library.\n\n### How it works:\n- **Container Setup:** \u003cbr\u003e \n    For each test, a temporary Docker container is created, running a PostgreSQL database called `vkdb`.\n    The necessary database schema is automatically set up when the container is initialized.\n\n- **Data Initialization:** \u003cbr\u003e \n  Before running each test, we make sure to clear any existing datas in tables and\n  relationships in the\n  database. Then, we populate these tables with initial data to ensure a consistent starting point for testing.\n\n- **Data Cleanup:** \u003cbr\u003e \n  After each test, we clean up the data in the tables to maintain a clean slate for the next test.\n\n- **Automatic Cleanup:** \u003cbr\u003e \n  Once all tests are completed, the Docker container is automatically stopped and deleted to keep\n  your environment tidy.\n\n## Debugging\n\nTo facilitate debugging, you can run the application directly through your IDE’s debugger, allowing you to set breakpoints, inspect variables, and step through the code.\n\n### Debugging with PyCharm\n\n1. **Open the Run/Debug Configurations:**  \n   In the top-right corner of PyCharm, click on the dropdown next to the run button and select *Edit Configurations*.\n\n2. **Add a New Configuration:**  \n   Create a new configuration for FastAPI or Pytest:\n   - For test debugging, select the Pytest option and specify the test files or directories you wish to debug.\n\n3. **Start Debugging:**  \n   Once set up, you can start the server or run tests in debug mode. The debugger will pause at any breakpoints you've set, allowing for detailed inspection of the execution flow.\n\n\u003e 💡**Tip:**  \n\u003e If you encounter issues with relative paths, try using the absolute path to the files in the configuration.\n\n\n## 📜 License\n\nThis project is licensed under the [GNU General Public License v3.0](https://www.gnu.org/licenses/gpl-3.0.en.html) - see the [LICENSE](LICENSE) file for details.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslub%2Fkartenforum_georeference","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fslub%2Fkartenforum_georeference","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslub%2Fkartenforum_georeference/lists"}