{"id":44651418,"url":"https://github.com/nfdi4cat/pid4cat-handle-dev-server","last_synced_at":"2026-02-14T20:48:36.897Z","repository":{"id":311127276,"uuid":"1034704259","full_name":"nfdi4cat/pid4cat-handle-dev-server","owner":"nfdi4cat","description":"Containerized Handle Server development environment for PID4CAT to support local development and CI/CD.","archived":false,"fork":false,"pushed_at":"2025-12-01T22:36:41.000Z","size":122,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-04T12:54:06.320Z","etag":null,"topics":["docker","handle-net","persistent-identifiers","pid4cat"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nfdi4cat.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-08T20:51:10.000Z","updated_at":"2025-12-01T22:36:38.000Z","dependencies_parsed_at":"2025-08-22T10:36:51.667Z","dependency_job_id":null,"html_url":"https://github.com/nfdi4cat/pid4cat-handle-dev-server","commit_stats":null,"previous_names":["nfdi4cat/pid4cat-handle-dev-server"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nfdi4cat/pid4cat-handle-dev-server","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nfdi4cat%2Fpid4cat-handle-dev-server","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nfdi4cat%2Fpid4cat-handle-dev-server/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nfdi4cat%2Fpid4cat-handle-dev-server/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nfdi4cat%2Fpid4cat-handle-dev-server/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nfdi4cat","download_url":"https://codeload.github.com/nfdi4cat/pid4cat-handle-dev-server/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nfdi4cat%2Fpid4cat-handle-dev-server/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29455362,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-14T15:52:44.973Z","status":"ssl_error","status_checked_at":"2026-02-14T15:52:11.208Z","response_time":53,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","handle-net","persistent-identifiers","pid4cat"],"created_at":"2026-02-14T20:48:36.262Z","updated_at":"2026-02-14T20:48:36.886Z","avatar_url":"https://github.com/nfdi4cat.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PID4CAT Handle Development Server\n\nThis repository provides a Docker-containerized setup for running Handle Server software for development and testing purposes on your local system or in CI.\n\nThe Handle Server software itself is from \u003chttp://www.handle.net/\u003e. For official documentation, production deployment guidance, and support, please refer to the Handle.Net documentation and support channels.\n\nThis Docker setup includes:\n\n- Handle Server environment with PostgreSQL backend\n- Automated configuration scripts that load settings from environment variables  \n- Development credentials and examples\n\nDocumentation included here:\n\n- Markdown version of [HANDLE.NET (Ver. 9) Technical Manual](./handle_net_documentation/HN_Technical_Manual_9.md)\n- Markdown version of [Release Notes Handle.Net Version 9.3.2 Software](./handle_net_documentation/HN_v9.3.2_ReleaseNotes.md)\n\nOther sources of information:\n\n- Documentation for the [PyHandle](https://eudat-b2handle.github.io/PYHANDLE/) library and its [GitHub repo](https://github.com/eudat-b2handle/PYHANDLE)\n\n## Getting Started\n\n### Step 1: Generate Key Pair\n\nThe Handle Server requires PKCS#8-formatted RSA keys for encryption as documented in the Handle.Net Technical Manual, [Section 1.4.1 \"Types of Authentication\"](./handle_net_documentation/HN_Technical_Manual_9.md#141-types-of-authentication). Generate them before configuring the environment.\n\n**Note for Windows users**: Use Git Bash for OpenSSL commands, as OpenSSL is not available in PowerShell by default.\n\n```bash\n# Generate private key in default PKCS8 format\nopenssl genpkey -algorithm RSA -out mykey.pem\n\n# Extract public key\nopenssl pkey -in mykey.pem -pubout -out mykey.pem.pub\n```\n\n### Step 2: Configure Environment\n\nBefore starting the containers, configure any environment settings and provide the keys you generated in Step 1.\nAdd each on a single line to the `docker-compose.yml` file.\n\nTo convert the keys into the required format, run in Git Bash:\n\n```bash\n# Private key\n openssl pkcs8 -topk8 -in mykey.pem -nocrypt | sed ':a;N;$!ba;s/\\n/\\\\r\\\\n/g'\n\n# Public key\nopenssl pkcs8 -topk8 -in mykey.pem -nocrypt | openssl pkey -pubout | sed ':a;N;$!ba;s/\\n/\\\\r\\\\n/g'\n```\n\nFor more on overriding env variables see [Environment variables in Compose](https://docs.docker.com/compose/environment-variables/).\n\n#### Required Configuration\n\n| Config                        | Default       | Required  | Description\n| ------                        | -------       | --------  | -----------\n| SERVER_PRIVATE_KEY_PEM        | None          | **Yes**   | PEM PKCS8 format private key from Step 1\n| SERVER_PUBLIC_KEY_PEM         | None          | **Yes**   | PEM PKCS8 format public key from Step 1\n\n#### Optional Configuration\n\nAll configuration values have sensible defaults.\nYou only need to modify these if you want to customize the setup:\n\n**Interface Configuration**\n\n| Config                        | Default       | Required  | Description\n| ------                        | -------       | --------  | -----------\n| BIND_ADDRESS                  | 0.0.0.0       | No        | IP address to bind server interfaces to (HTTP, TCP, UDP)\n| NUM_THREADS                   | 15            | No        | Number of threads for TCP/UDP interfaces\n| BIND_PORT                     | 2641          | No        | Port for TCP/UDP Handle protocol\n| LOG_ACCESSES                  | yes           | No        | Log access requests to interfaces\n| HTTP_BIND_PORT                | 8000          | No        | Port for HTTP interface\n| HTTP_NUM_THREADS              | 15            | No        | Number of threads for HTTP interface\n| BACKLOG                       | None          | No        | Network backlog queue size\n| BIND_ADDRESS_V6               | None          | No        | IPv6 bind address\n| LISTEN_ADDRESS                | None          | No        | Address to listen on (different from bind)\n| ALLOW_RECURSION               | None          | No        | Allow recursive handle resolution\n\n**Server Configuration**\n\n| Config                        | Default       | Required  | Description\n| ------                        | -------       | --------  | -----------\n| SERVER_ADMINS                 | None          | No        | Space-separated list of handle admins e.g. \"ADMIN1 ADMIN2 ADMIN3\"\n| REPLICATION_ADMINS            | None          | No        | Space-separated list of handle admins for replication e.g. \"ADMIN1 ADMIN2 ADMIN3\"\n| MAX_SESSION_TIME              | 86400000      | No        | Max authenticated client session time in milliseconds\n| THIS_SERVER_ID                | 1             | No        | An identifier for this handle server\n| MAX_AUTH_TIME                 | 60000         | No        | Max time to wait for client to respond to authentication challenge in milliseconds\n| SERVER_ADMIN_FULL_ACCESS      | yes           | No        | Admins listed in SERVER_ADMINS will have full permissions over all handles on the server\n| ALLOW_NA_ADMINS               | no            | No        | Allow global handle server admins access to this handle server\n| TEMPLATE_NS_OVERRIDE          | yes           | No        | Allow template namespace override\n| CASE_SENSITIVE                | no            | No        | Are handles case-sensitive\n| AUTO_HOMED_PREFIXES           | TEST          | No        | Space-separated list of prefixes that are auto-homed e.g., \"0.NA/TEST PREFIX2\"\n| MAX_HANDLES                   | None          | No        | Maximum number of handles the server will store\n| MAX_VALUES                    | None          | No        | Maximum number of values per handle\n| TRACE_RESOLUTION              | None          | No        | Enable resolution tracing for debugging\n| ALLOW_LIST_HDLS               | None          | No        | Allow listing handles under a prefix\n\n**Protocol Settings**\n\n| Config                        | Default       | Required  | Description\n| ------                        | -------       | --------  | -----------\n| NO_UDP_RESOLUTION             | yes           | No        | Set to \"no\" to enable UDP protocol support\n\n**Logging Configuration**\n\n| Config                        | Default       | Required  | Description\n| ------                        | -------       | --------  | -----------\n| LOG_SAVE_INTERVAL             | Never         | No        | How often to save logs (Never, Weekly, Monthly, etc.)\n| LOG_SAVE_DIRECTORY            | logs          | No        | Directory to save log files\n\n**Site Information**\n\n| Config                        | Default       | Required  | Description\n| ------                        | -------       | --------  | -----------\n| HANDLE_HOST_IP                | 0.0.0.0       | No        | Public handle host IP used for siteinfo.json\n| SITE_DESCRIPTION              | Handle Server | No        | Description text for the server in siteinfo.json\n\n**Storage Configuration**\n\n| Config                        | Default       | Required  | Description\n| ------                        | -------       | --------  | -----------\n| STORAGE_TYPE                  | sql           | No        | Storage backend type - set to \"sql\" for PostgreSQL storage\n| SQL_URL                       | None          | No        | JDBC URL used to connect to the PostgreSQL database\n| SQL_DRIVER                    | org.postgresql.Driver | No | Java class that contains the driver for the JDBC connection\n| SQL_LOGIN                     | postgres      | No        | Username for database connection\n| SQL_PASSWD                    | None          | No        | Password for database connection\n| SQL_READ_ONLY                 | no            | No        | Boolean setting for allowing writes to database or not\n\n**HTTP Configuration**\n\n| Config                        | Default       | Required  | Description\n| ------                        | -------       | --------  | -----------\n| ALLOW_CORS                    | None          | No        | Enable CORS for HTTP interface\n| CORS_ORIGINS                  | None          | No        | Allowed CORS origins\n| HTTPS_ENABLED                 | None          | No        | Enable HTTPS support\n| HTTPS_PORT                    | None          | No        | Port for HTTPS interface\n| HTTP_LOG_FORMAT               | None          | No        | HTTP access log format\n| MAX_REQUEST_SIZE              | None          | No        | Maximum HTTP request size\n| SESSION_TIMEOUT               | None          | No        | HTTP session timeout\n| ADMIN_PATH                    | None          | No        | Custom path for admin interface\n\n### Step 3: Start the Handle Server\n\nStart the containers from the project root:\n\n```bash\ndocker compose up --build\n```\n\n**Note:** The handle_server container build process can take several minutes as it downloads and compiles Java components. To speed up subsequent startups, you can pre-build the handle_server image:\n\n```bash\n# Pre-build the handle_server image (recommended for faster startup)\ndocker compose build handle_server\n\n# Then start normally\ndocker compose up\n```\n\nWait for the containers to start up completely.\nYou should see log messages indicating the Handle Server is ready and listening on ports 2641 (TCP/UDP) and 8000 (HTTP/HTTPS).\n\n### Step 4: Test the Setup\n\n#### Reading Handles with curl\n\nTest read access to existing handles:\n\n```bash\n# Read admin handle\ncurl -k \"https://127.0.0.1:8000/api/handles/TEST/ADMIN?pretty=yes\"\n\n# Read any handle\ncurl -k \"https://127.0.0.1:8000/api/handles/TEST/EXAMPLE001?pretty=yes\"\n```\n\n- `-k`: Tell curl to accept self-signed SSL certificates\n- `?pretty=yes`: Query parameter to get JSON-formatted response\n\nMore options are documented in the Handle.Net Technical Manual [Section 14.1 Resources](./handle_net_documentation/HN_Technical_Manual_9.md#141-resources).\n\n#### Database Access\n\nInspect handles directly in the PostgreSQL database (for development only):\n\n```bash\n# Connect to PostgreSQL database\ndocker exec -it pid4cat-handle-dev-server-postgres-1 psql -U handleuser -d handledb\n\n# View handles in readable format\nSELECT encode(handle, 'escape') as handle_name, idx, type, encode(data, 'escape') as data_value \nFROM handles WHERE handle LIKE decode('544553542f', 'hex') || '%' ORDER BY handle;\n```\n\n## Usage\n\n### Creating Handles with curl\n\nFor creating handles, authentication is required. The simplest and most reliable method is **Basic Authentication** as documented in the Handle.Net Technical Manual [section 14.6.3 Basic Access Authentication](./handle_net_documentation/HN_Technical_Manual_9.md#1463-basic-access-authentication).\n\nThis method uses standard HTTP Basic Authentication with percent-encoded username:\n\n1. **Create handle data file** (`handle_data.json`):\n\n   ```json\n   {\n     \"values\": [\n       {\n         \"index\": 1,\n         \"type\": \"URL\",\n         \"data\": {\n           \"format\": \"string\",\n           \"value\": \"https://example.org/mydata\"\n         }\n       },\n       {\n         \"index\": 100,\n         \"type\": \"HS_ADMIN\",\n         \"data\": {\n           \"format\": \"admin\",\n           \"value\": {\n             \"handle\": \"TEST/ADMIN\",\n             \"index\": 300,\n             \"permissions\": \"011111110011\"\n           }\n         }\n       }\n     ]\n   }\n   ```\n\n2. **Create handle with curl**:\n\n   ```bash\n   curl --insecure -v -X PUT \"https://127.0.0.1:8000/api/handles/TEST/MY_NEW_HANDLE\" \\\n     -H \"Content-Type: application/json\" \\\n     -u \"300%3ATEST%2FADMIN:ASECRETKEY\" \\\n     --data-binary \"@handle_data.json\"\n   ```\n\n   **Key points:**\n   - Username `300:TEST/ADMIN` is percent-encoded as `300%3ATEST%2FADMIN`\n   - Colon (`:`) becomes `%3A`\n   - Forward slash (`/`) becomes `%2F`\n   - Password is the secret key: `ASECRETKEY`\n\n3. **Success response:**\n\n   ```json\n   {\"responseCode\":1,\"handle\":\"TEST/MY_NEW_HANDLE\"}\n   ```\n\nThis method is reliable, simple, and works consistently across different platforms and curl versions.\n\n### Using PyHandle Library\n\nThe [PyHandle](https://pypi.org/project/pyhandle/) library provides a Python interface that implements the challenge-response framework with digital signatures (using HS_PUBKEY). This complex authentication mechanism cannot be used with curl alone. It is documented in the Handle.Net Technical Manual, [Section 14.6.4 and the following sections](./handle_net_documentation/HN_Technical_Manual_9.md#1464-authentication-via-authorization-handle).\n\n```bash\n# Activate virtual environment (if using one)\n.\\.venv\\Scripts\\Activate.ps1   # Windows PowerShell\n# or\nsource .venv/bin/activate      # Unix/Linux\n\n# Run the example script\npython examples/create_handle_examples.py\n```\n\n### Accessing the Web Admin GUI\n\nThe dockerized Handle Server provides a web-based administration interface for managing handles through a graphical user interface.\n\n**Access URL:** `https://127.0.0.1:8000/admin/`\n\n**Login Credentials:**\n\n- **Username:** `300:TEST/ADMIN`\n- **Password:** `ASECRETKEY`\n\nThe web interface allows you to:\n\n- View existing handles\n- Create new handles\n- Modify handle values\n- Home/Unhome prefixes\n\n  ![Web Admin Interface](./handle_net_documentation/webadmin-screenshot.png)\n\n  **Note:** The web admin interface uses username/password authentication (HS_SECKEY) and does not require certificate files.\n\n## Architecture\n\nThe dockerized Handle Server consists of three main components:\n\n```mermaid\ngraph LR\n    A[Client\u003cbr/\u003ecurl/pyhandle/\u003cbr/\u003eweb browser] --\u003e B[Handle Server\u003cbr/\u003eHTTP API: 8000\u003cbr/\u003eTCP/UDP: 2641]\n    B --\u003e C[PostgreSQL\u003cbr/\u003eDatabase\u003cbr/\u003ePort 5432]\n```\n\n**Component Details:**\n- **PostgreSQL**: Stores handles in `handles` and `nas` tables with proper indexing\n- **Handle Server**: Java application that processes Handle protocol requests\n- **Configuration System**: Jinja2 templates generate server config from environment variables\n- **Web Interface**: Built-in admin GUI accessible via HTTPS\n\n### File Structure\n```\npid4cat-handle-dev-server/\n├── docker-compose.yml              # Container orchestration\n├── handle-server/                  # Handle server container\n│   ├── Dockerfile                  # Container build instructions\n│   ├── config/                     # Configuration templates\n│   │   ├── config.dct.jinja       # Main server config Jinja2 template\n│   │   └── siteinfo.json.jinja    # Server metadata Jinja2 template\n│   ├── scripts/                    # Build and startup scripts\n│   │   ├── create_config.py       # Jinja2 config generation from env vars\n│   │   └── handle.sh              # Container startup script\n│   └── storage/\n│       └── init_db.sql            # Database initialization\n└── examples/                       # Usage examples\n    ├── create_handle_examples.py   # PyHandle library demo\n    └── credentials.json            # PyHandle credentials\n```\n\n## Configuration System\n\nThe Handle Server configuration is generated dynamically from environment variables using Jinja2 templates:\n\n### How It Works\n1. **Environment Variables** → Set in `docker-compose.yml`\n2. **create_config.py Script** → Reads env vars and processes Jinja2 templates\n3. **Template Files** → `.jinja` files with `{{ variable }}` placeholders\n4. **Final Config** → Generated `.dct` and `.json` files used by server\n\n### Configuration Flow\n\n```mermaid\nflowchart TD\n    A[Environment Variables\u003cbr/\u003edocker-compose.yml] --\u003e C[create_config.py\u003cbr/\u003eScript]\n    B[Jinja2 Templates\u003cbr/\u003econfig.dct.jinja\u003cbr/\u003esiteinfo.json.jinja] --\u003e C\n    C --\u003e D[Generated Config\u003cbr/\u003econfig.dct\u003cbr/\u003esiteinfo.json]\n    D --\u003e E[Handle Server\u003cbr/\u003eStartup]\n```\n\n### Key Configuration Files\n\n| File | Purpose | Variables Used |\n|------|---------|----------------|\n| `config.dct.jinja` | Main server settings | `{{ BIND_ADDRESS }}`, `{{ SERVER_ADMINS }}`, `{{ SQL_URL }}`, etc. |\n| `siteinfo.json.jinja` | Server metadata for Handle protocol | `{{ HANDLE_HOST_IP }}`, `{{ SERVER_PUBLIC_KEY_DSA_BASE64 }}` |\n\n### Startup Process\n1. Container starts → `handle.sh` script runs\n2. `create_config.py` executed: `python3 create_config.py /opt/handle/bin /var/handle/svr`\n3. Jinja2 templates processed → Final config files generated\n4. Keys converted from PEM to DSA format using `hdl-convert-key`\n5. Handle Server starts with generated configuration\n\n### Testing Configuration Script\n\nThe `create_config.py` script includes unit tests that can be run locally:\n\n```bash\ncd handle-server/scripts\npython -m unittest test_create_config.py\n```\n\nTests cover configuration building, template rendering, and error handling using Python's standard library with minimal dependencies (requires `jinja2`).\n\n## Troubleshooting\n\n### Common Issues\n\n**Database Errors: `relation \"nas\" does not exist`**\n```bash\n# Solution: Reset database and restart\ndocker compose down\ndocker volume rm pid4cat-handle-dev-server_postgres_data\ndocker compose up --build\n```\n\n**File Path Errors: `No such file or directory`**  \nCheck that file paths in `docker-compose.yml` match actual file locations after recent moves.\n\n**Configuration Not Applied**  \nEnvironment variables are only read during container build. After changing `docker-compose.yml`, run:\n```bash\ndocker compose up --build\n```\n\n**SSL Certificate Warnings**  \nNormal for development. Use the `-k` flag with curl: `curl -k https://127.0.0.1:8000/...`\n\n### Quick Commands Reference\n```bash\n# Essential operations\ndocker compose up --build          # Start with rebuild\ndocker compose down                # Stop all services  \ndocker compose logs -f handle_server # View server logs\ndocker volume ls                   # List volumes\ndocker exec -it \u003ccontainer\u003e bash   # Access container shell\n\n# Testing\ncurl -k \"https://127.0.0.1:8000/api/handles/TEST/ADMIN?pretty=yes\"\npython examples/create_handle_examples.py\n```\n\n### Development Checklist\n- [ ] Docker and Docker Compose installed\n- [ ] Remove old volume: `docker volume rm pid4cat-handle-dev-server_postgres_data` (if restarting)\n- [ ] Start services: `docker compose up --build`  \n- [ ] Wait for PostgreSQL \"ready to accept connections\" message\n- [ ] Test API: `curl -k \"https://127.0.0.1:8000/api/handles/TEST/ADMIN?pretty=yes\"`\n- [ ] Access web admin: https://127.0.0.1:8000/admin/ (300:TEST/ADMIN / ASECRETKEY)\n\n## Technical Notes\n\n- **TEST Prefix**: Configured for local development and testing only. Handles created with TEST prefix are not globally resolvable. They are restricted to your local setup.\n- **Database Storage**: All handles are stored in PostgreSQL with proper indexing and transaction support.\n- **Ports**: Handle Server listens on 2641 (TCP/UDP native protocol) and 8000 (HTTP/HTTPS REST API).\n- **Security**: Uses hardcoded development keys and credentials. Generate your own keys for production use.\n\n## Acknowledgement\n\n[datacite/docker-handle](https://github.com/datacite/docker-handle) was very helpful to figure out details.\nHere we use more flexible Jinja2 templating, support only PostgreSQL for storage, and use the latest Handle.Net software (v9.3.2).\n\nCreated with assistance from Claude Sonnet 4 (Anthropic, accessed August 2025) and Gemini 2.5 Pro (Google, accessed August 2025) via the Claude Code command-line interface, GitHub Copilot in Visual Studio Code, and Warp.\nAll AI-generated code was reviewed, tested, and validated by the authors.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnfdi4cat%2Fpid4cat-handle-dev-server","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnfdi4cat%2Fpid4cat-handle-dev-server","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnfdi4cat%2Fpid4cat-handle-dev-server/lists"}