{"id":49632710,"url":"https://github.com/dalejandrov/sipsa","last_synced_at":"2026-05-05T13:49:48.711Z","repository":{"id":338850938,"uuid":"1093254301","full_name":"dalejandrov/sipsa","owner":"dalejandrov","description":"A REST API wrapper that exposes official Colombian agricultural price and supply data from the DANE SIPSA SOAP web service through a modern RESTful interface, built with Spring Boot","archived":false,"fork":false,"pushed_at":"2026-02-16T18:56:18.000Z","size":173,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-17T01:32:45.159Z","etag":null,"topics":["java","maven","rest-api","spring-boot"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dalejandrov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-10T05:54:11.000Z","updated_at":"2026-02-16T18:56:22.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/dalejandrov/sipsa","commit_stats":null,"previous_names":["dalejandrov/sipsa"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/dalejandrov/sipsa","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dalejandrov%2Fsipsa","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dalejandrov%2Fsipsa/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dalejandrov%2Fsipsa/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dalejandrov%2Fsipsa/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dalejandrov","download_url":"https://codeload.github.com/dalejandrov/sipsa/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dalejandrov%2Fsipsa/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32652474,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-05T11:29:49.557Z","status":"ssl_error","status_checked_at":"2026-05-05T11:29:48.587Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["java","maven","rest-api","spring-boot"],"created_at":"2026-05-05T13:49:47.830Z","updated_at":"2026-05-05T13:49:48.700Z","avatar_url":"https://github.com/dalejandrov.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SIPSA Integration Service\n\n![Java](https://img.shields.io/badge/Java-21-orange?logo=openjdk)\n![Spring Boot](https://img.shields.io/badge/Spring%20Boot-3.5.9-brightgreen?logo=springboot)\n![PostgreSQL](https://img.shields.io/badge/PostgreSQL-18.0-4169E1?logo=postgresql\u0026logoColor=white)\n![Docker](https://img.shields.io/badge/Docker-ready-2496ED?logo=docker\u0026logoColor=white)\n![License](https://img.shields.io/github/license/dalejandrov/sipsa)\n\nREST API wrapper for DANE Colombia's SIPSA SOAP web service. Access agricultural price and supply data through a modern REST interface with automatic syncing, pagination, filtering, and timezone support.\n\n## What is SIPSA?\n\nSIPSA (Sistema de Información de Precios y Abastecimiento del Sector Agropecuario) is Colombia's official agricultural market information system maintained by DANE. It tracks prices and supply across different market types:\n\n- **Ciudad**: Daily city-level pricing\n- **Parcial**: Municipality market data  \n- **Mayoristas**: Weekly and monthly wholesale market aggregates\n- **Abastecimientos**: Monthly supply data to wholesale markets\n\n## Quick Start\n\n### Docker (recommended)\n\n```bash\ngit clone https://github.com/dalejandrov/sipsa.git\ncd sipsa\ndocker-compose up -d\n```\n\nAPI available at `http://localhost:8080`\n\n### Local Development\n\nRequirements: Java 21, PostgreSQL 18, Maven 3.9+\n\n```bash\n# 1. Clone and navigate\ngit clone https://github.com/dalejandrov/sipsa.git\ncd sipsa\n\n# 2. Set up environment variables\ncp .env.example .env\n# Edit .env with your database credentials\n\n# 3. Create database (adjust credentials if needed)\ncreatedb sipsa_db\n\n# 4. Run\n./mvnw spring-boot:run\n```\n\n## API Usage\n\n### Query Endpoints (Public)\n\n**Base URL**: `/api/sipsa`\n\n- `GET /api/sipsa` - List all available endpoints\n- `GET /api/sipsa/ciudad` - Daily city-level prices\n- `GET /api/sipsa/parcial` - Municipality market data\n- `GET /api/sipsa/mayoristas/semanal` - Weekly wholesale prices\n- `GET /api/sipsa/mayoristas/mensual` - Monthly wholesale aggregates\n- `GET /api/sipsa/abastecimientos/mensual` - Monthly supply to wholesale markets\n\n### Operations Endpoints (Internal)\n\n**Base URL**: `/api/internal/ingestion`\n\n- `POST /api/internal/ingestion/run` - Trigger manual ingestion (query params: `method`, `force`)\n- `GET /api/internal/ingestion/methods` - List available ingestion methods\n- `GET /api/internal/ingestion/running` - Get currently active ingestion runs\n- `GET /api/internal/ingestion/runs` - List all ingestion runs\n- `GET /api/internal/ingestion/runs/{runId}` - Get specific run status\n- `POST /api/internal/ingestion/cancel/{runId}` - Cancel an active run\n\n### Audit Endpoints (Internal)\n\n**Base URL**: `/api/internal/audit`\n\n- `GET /api/internal/audit/request/{requestId}` - Get audit trail for specific request\n- `GET /api/internal/audit/run/{runId}` - Get audit events for specific run\n- `GET /api/internal/audit/recent` - Get most recent audit events (last 100)\n- `GET /api/internal/audit/all` - Query all audit events with pagination and filters\n\n## Features\n\n- **REST instead of SOAP**: No more XML wrangling\n- **Automatic sync**: Scheduled jobs fetch new data from DANE\n- **Timezone support**: Get dates in your timezone via `X-Timezone` header\n- **Pagination**: Standard limit/offset pagination\n- **Filtering**: Query by date ranges, cities, products\n- **Docker ready**: Includes PostgreSQL in docker-compose\n- **Health checks**: `/actuator/health` for monitoring\n- **Audit trail**: Track all data ingestion runs\n\n## Configuration\n\nAll configuration is done via environment variables. Copy `.env.example` to `.env` and adjust values:\n\n```bash\ncp .env.example .env\n```\n\nKey configurations available in [.env.example](.env.example):\n- **Database**: Connection settings (host, port, credentials)\n- **Server**: Port and active profiles\n- **SOAP Service**: DANE endpoint and timeouts\n- **Ingestion**: Batch sizes, schedules, and quality thresholds\n- **Logging**: Log levels per package\n\nFor advanced configuration, see [application.yaml](src/main/resources/application.yaml).\n\n## Tech Stack\n\n- Java 21\n- Spring Boot 3.5.9\n- PostgreSQL 18\n- Apache CXF (SOAP client)\n- Resilience4j (circuit breaker)\n\n## Architecture\n\n```\nAPI Layer (REST controllers)\n    ↓\nApplication Layer (services, business logic)\n    ↓\nDomain Layer (entities, rules)\n    ↓\nInfrastructure (repositories, SOAP client, schedulers)\n```\n\n**How it works:**\n\n1. Scheduled jobs run based on DANE's publication schedule\n2. SOAP client fetches data from DANE's web service\n3. Data is validated and stored in PostgreSQL\n4. REST API serves the data with filtering and pagination\n5. Audit system logs all operations\n\n## Documentation\n\nDetailed technical documentation and diagrams available in [docs/](docs/):\n\n- **[ER Diagram](docs/diagrams/er-diagram.puml)**: Database schema with entities and relationships\n- **[Class Diagram](docs/diagrams/class-diagram.puml)**: System architecture organized by layers\n- **[Component Diagram](docs/diagrams/component-diagram.puml)**: Component interactions and dependencies\n- **[Sequence Diagrams](docs/diagrams/sequence/)**: Process flows for ingestion and API queries\n\nView diagrams using [PlantUML Online](https://www.plantuml.com/plantuml/) or IDE plugins.\n\n## Manual Data Ingestion\n\nTrigger a manual sync via operations API:\n\n```bash\ncurl -X POST \"http://localhost:8080/api/internal/ingestion/run?method=promediosSipsaCiudad\u0026force=false\"\n```\n\nAvailable methods:\n- `promediosSipsaCiudad` - City-level pricing\n- `promediosSipsaParcial` - Municipality markets\n- `promediosSipsaSemanaMadr` - Weekly wholesale\n- `promediosSipsaMesMadr` - Monthly wholesale\n- `promedioAbasSipsaMesMadr` - Monthly supply\n\nCheck available methods:\n```bash\ncurl http://localhost:8080/api/internal/ingestion/methods\n```\n\n## Contributing\n\nPull requests welcome. For major changes, open an issue first.\n\n## License\n\n[MIT](LICENSE)\n\n---\n\nData provided by [DANE - Departamento Administrativo Nacional de Estadística](https://www.dane.gov.co/index.php/estadisticas-por-tema/agropecuario/sistema-de-informacion-de-precios-sipsa/servicio-web-para-consulta-de-la-base-de-datos-de-sipsa)\n\n*This is an unofficial wrapper. For official data access, visit DANE's website.*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdalejandrov%2Fsipsa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdalejandrov%2Fsipsa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdalejandrov%2Fsipsa/lists"}