{"id":43171931,"url":"https://github.com/opencitations/oc_api","last_synced_at":"2026-02-01T02:35:38.271Z","repository":{"id":286815991,"uuid":"961824556","full_name":"opencitations/oc_api","owner":"opencitations","description":"This repository contains the API service for OpenCitations, allowing users to interact with the OpenCitations datasets through RESTful endpoints.","archived":false,"fork":false,"pushed_at":"2026-01-28T16:03:06.000Z","size":7013,"stargazers_count":1,"open_issues_count":1,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2026-01-29T06:54:20.011Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://api.opencitations.net","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/opencitations.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-04-07T08:15:52.000Z","updated_at":"2026-01-28T16:02:32.000Z","dependencies_parsed_at":"2025-10-28T18:13:55.130Z","dependency_job_id":"d5e9be23-f0ec-4d64-b8a7-189ec09a168d","html_url":"https://github.com/opencitations/oc_api","commit_stats":null,"previous_names":["opencitations/oc_api"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/opencitations/oc_api","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencitations%2Foc_api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencitations%2Foc_api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencitations%2Foc_api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencitations%2Foc_api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/opencitations","download_url":"https://codeload.github.com/opencitations/oc_api/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencitations%2Foc_api/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28965430,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-01T02:14:24.993Z","status":"ssl_error","status_checked_at":"2026-02-01T02:13:55.706Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-02-01T02:35:37.695Z","updated_at":"2026-02-01T02:35:38.265Z","avatar_url":"https://github.com/opencitations.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OpenCitations API Service\n\n[\u003cimg src=\"https://img.shields.io/badge/powered%20by-OpenCitations-%239931FC?labelColor=2D22DE\" /\u003e](http://opencitations.net)\n[![Run tests](https://github.com/opencitations/oc_api/actions/workflows/run_tests.yml/badge.svg?branch=main)](https://github.com/opencitations/oc_api/actions/workflows/run_tests.yml)\n![Coverage](test/coverage-badge.svg)\n\nThis repository contains the API service for OpenCitations, allowing users to interact with the OpenCitations datasets through RESTful endpoints.\n\n## Overview\n\nThe service provides two main API endpoints:\n\n- **Index endpoint** (`/index/v2`): For querying the OpenCitations Index database\n- **Meta endpoint** (`/meta/v1`): For querying the OpenCitations Meta database\n\n## Configuration\n\n### Environment Variables\n\nThe service requires the following environment variables. These values take precedence over the ones defined in `conf.json`:\n\n- `BASE_URL`: Base URL for the API service\n- `SPARQL_ENDPOINT_INDEX`: URL for the internal Index SPARQL endpoint (used by the API)\n- `SPARQL_ENDPOINT_META`: URL for the internal Meta SPARQL endpoint (used by the API)\n- `LOG_DIR`: Directory path where log files will be stored\n- `SYNC_ENABLED`: Enable/disable static files synchronization (default: false)\n- `REDIS_ENABLED`: Enable/disable Redis database used for storing tokens (default: false)\n- `REDIS_HOST`: Redis server hostname (default from conf.json)\n- `REDIS_PORT`: Redis server port (default: 6379)\n- `REDIS_DB`: Redis database number (default: 0)\n- `REDIS_PASSWORD`: Redis authentication password\n\nFor instance:\n```env\n# API Configuration\nBASE_URL=api.opencitations.net\nLOG_DIR=/home/dir/log/\nSPARQL_ENDPOINT_INDEX=http://qlever-service.default.svc.cluster.local:7011  \nSPARQL_ENDPOINT_META=http://virtuoso-service.default.svc.cluster.local:8890/sparql\nSYNC_ENABLED=true\n\n# Redis Configuration\nREDIS_ENABLED=true\nREDIS_HOST=redis-service.default.svc.cluster.local\nREDIS_PORT=6379\nREDIS_DB=0\nREDIS_PASSWORD=your_redis_password\n```\n\n\u003e **Note**: When running with Docker, environment variables always override the corresponding values in `conf.json`. If an environment variable is not set, the application will fall back to the values defined in `conf.json`.\n\n### Static Files Synchronization\n\nThe application can synchronize static files from a GitHub repository. This configuration is managed in `conf.json`:\n```json\n{\n  \"oc_services_templates\": \"https://github.com/opencitations/oc_services_templates\",\n  \"sync\": {\n    \"folders\": [\n      \"static\",\n      \"html-template/common\"\n    ],\n    \"files\": [\n      \"test.txt\"\n    ]\n  }\n}\n```\n\n- `oc_services_templates`: The GitHub repository URL to sync files from\n- `sync.folders`: List of folders to synchronize\n- `sync.files`: List of individual files to synchronize\n\nWhen static sync is enabled (via `--sync-static` or `SYNC_ENABLED=true`), the application will:\n1. Clone the specified repository\n2. Copy the specified folders and files\n3. Keep the local static files up to date\n\n\u003e **Note**: Make sure the specified folders and files exist in the source repository.\n\n## Running Options\n\n### Local Development\n\nFor local development and testing, the application uses the built-in web.py HTTP server:\n```bash\n# Run with default settings\npython3 api_oc.py\n\n# Run with static sync enabled\npython3 api_oc.py --sync-static\n\n# Run on custom port\npython3 api_oc.py --port 8085\n\n# Run with both options\npython3 api_oc.py --sync-static --port 8085\n```\n\nThe application supports the following command line arguments:\n\n- `--sync-static`: Synchronize static files at startup and enable periodic sync (every 30 minutes)\n- `--port PORT`: Specify the port to run the application on (default: 8080)\n\n### Production Deployment (Docker)\n\nWhen running in Docker/Kubernetes, the application uses **Gunicorn** as the WSGI HTTP server for better performance and concurrency handling:\n\n- **Server**: Gunicorn with gevent workers\n- **Workers**: 2 concurrent worker processes\n- **Worker Type**: gevent (async) for handling thousands of simultaneous requests\n- **Timeout**: 1200 seconds (to handle long-running SPARQL queries)\n- **Connections per worker**: 800 simultaneous connections\n\nThe Docker container automatically uses Gunicorn and is configured with static sync enabled by default.\n\n\u003e **Note**: The application code automatically detects the execution environment. When run with `python3 api_oc.py`, it uses the built-in web.py server. When run with Gunicorn (as in Docker), it uses the WSGI interface.\n\nYou can customize the Gunicorn server configuration by modifying the `gunicorn.conf.py` file.\n\n### Dockerfile\n```dockerfile\n# Base image: Python slim for a lightweight container\nFROM python:3.11-slim\n\n# Define environment variables with default values\n# These can be overridden during container runtime\nENV BASE_URL=\"api.opencitations.net\" \\\n    LOG_DIR=\"/mnt/log_dir/oc_api\"  \\\n    SPARQL_ENDPOINT_INDEX=\"http://qlever-service.default.svc.cluster.local:7011\" \\\n    SPARQL_ENDPOINT_META=\"http://virtuoso-service.default.svc.cluster.local:8890/sparql\" \\\n    SYNC_ENABLED=\"true\"\n  # Uncomment the following lines if you are running the application in a local development environment or any non-Kubernetes deployment scenario.\n  #  REDIS_ENABLED=\"true\" \\\n  #  REDIS_HOST=\"redis-service.default.svc.cluster.local\" \\\n  #  REDIS_PORT=\"6379\" \\\n  #  REDIS_DB=\"0\" \\\n  #  REDIS_PASSWORD=\"your_redis_password\"\n\n# Ensure Python output is unbuffered\nENV PYTHONUNBUFFERED=1\n# Install system dependencies required for Python package compilation\nRUN apt-get update \u0026\u0026 \\\n    apt-get install -y \\\n    git \\\n    python3-dev \\\n    build-essential\n\n# Set the working directory for our application\nWORKDIR /website\n\n# Copy the application code from the GitHub repo\nRUN git clone --single-branch --branch main https://github.com/opencitations/oc_api .\n\n# Install Python dependencies from requirements.txt\nRUN pip install -r requirements.txt\n\n# Expose the port that our service will listen on\nEXPOSE 8080\n\n# Start the application with gunicorn for production\nCMD [\"gunicorn\", \"-c\", \"gunicorn.conf.py\", \"api_oc:application\"]\n```\n\n## Testing\n\n### Running Tests Locally\n\nTo run the API tests locally, you'll need to have the test environment set up properly. The tests require a Virtuoso database to be running.\n\n#### Prerequisites\n\n1. Install dependencies using uv:\n```bash\n   uv sync --dev\n```\n\n2. Start the test database:\n```bash\n   ./test/start_test_db.sh\n```\n\n#### Running the Tests\n\nOnce the test database is running, you can execute the tests with coverage:\n```bash\n# Run tests with coverage report\nuv run pytest --cov=src --cov-report=term-missing --cov-report=html\n\n# Run only specific test files\nuv run pytest test/test_metaapi.py\n\n# Run tests with verbose output\nuv run pytest -v\n```\n\n#### Stopping the Test Database\n\nAfter running the tests, stop the test database:\n```bash\n./test/stop_test_db.sh\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopencitations%2Foc_api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopencitations%2Foc_api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopencitations%2Foc_api/lists"}