{"id":51104227,"url":"https://github.com/fintlabs/fint-kontroll-device-catalog","last_synced_at":"2026-06-24T13:01:37.798Z","repository":{"id":324857810,"uuid":"1090301489","full_name":"FINTLabs/fint-kontroll-device-catalog","owner":"FINTLabs","description":"fint-kontroll-device-catalog","archived":false,"fork":false,"pushed_at":"2026-06-22T10:08:46.000Z","size":125,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-22T12:08:46.444Z","etag":null,"topics":["bas"],"latest_commit_sha":null,"homepage":"","language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FINTLabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-05T13:40:28.000Z","updated_at":"2026-06-22T10:08:50.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/FINTLabs/fint-kontroll-device-catalog","commit_stats":null,"previous_names":["fintlabs/fint-kontroll-device-catalog"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/FINTLabs/fint-kontroll-device-catalog","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FINTLabs%2Ffint-kontroll-device-catalog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FINTLabs%2Ffint-kontroll-device-catalog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FINTLabs%2Ffint-kontroll-device-catalog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FINTLabs%2Ffint-kontroll-device-catalog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FINTLabs","download_url":"https://codeload.github.com/FINTLabs/fint-kontroll-device-catalog/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FINTLabs%2Ffint-kontroll-device-catalog/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34733256,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-24T02:00:07.484Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bas"],"created_at":"2026-06-24T13:01:34.699Z","updated_at":"2026-06-24T13:01:37.775Z","avatar_url":"https://github.com/FINTLabs.png","language":"Kotlin","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FINT Kontroll Device Catalog\n\nFINT Kontroll Device Catalog consumes device catalog entities from Kafka, stores them in PostgreSQL, and republishes the normalized Kontroll views used by FINT Kontroll.\n\nThe application is responsible for:\n\n- Kafka consumers for devices, device groups, and device group memberships.\n- JPA persistence for the local device catalog.\n- Kafka producers for Kontroll device, device group, and membership topics.\n- REST endpoints for reading the persisted Kontroll views.\n- Swagger/OpenAPI documentation.\n\n## EntityPersistenceService\n\n`EntityPersistenceService` is the main write path for incoming Kafka entities. The Kafka listener configuration passes each consumed `KafkaEntity` to `EntityPersistenceService.handle(...)`, and the service dispatches by entity type:\n\n- `KafkaDevice`\n- `KafkaDeviceGroup`\n- `KafkaDeviceGroupMembership`\n\n### Entity flow\n\n```mermaid\nflowchart LR\n    kafkaIn[Kafka input topics] --\u003e consumer[DeviceConsumerConfiguration]\n    consumer --\u003e persistence[EntityPersistenceService.handle]\n\n    persistence --\u003e deviceOrGroup[KafkaDevice or KafkaDeviceGroup]\n    deviceOrGroup --\u003e mapEntity[EntityMappingService]\n    mapEntity --\u003e upsertEntity[DeviceRepository or DeviceGroupRepository]\n    upsertEntity --\u003e database[(PostgreSQL device schema)]\n\n    persistence --\u003e membership[KafkaDeviceGroupMembership]\n    membership --\u003e lookupRefs[Find Device and DeviceGroup by sourceId]\n    lookupRefs --\u003e refsFound{Both found?}\n    refsFound -- yes --\u003e mapMembership[EntityMappingService]\n    mapMembership --\u003e upsertMembership[DeviceGroupMembershipRepository]\n    upsertMembership --\u003e database\n    refsFound -- no --\u003e retryBuffer[DeviceGroupMembershipRetryBuffer]\n    retryBuffer --\u003e retryScheduler[DeviceGroupMembershipRetryScheduler]\n    retryScheduler --\u003e persistence\n\n    database --\u003e kontrollMapper[KontrollEntityMappingService]\n    kontrollMapper --\u003e publishers[Kontroll publishing components]\n    publishers --\u003e kafkaOut[Kafka Kontroll topics]\n```\n\nThe main path persists incoming Kafka entities before publishing the corresponding Kontroll entity. Membership events have an extra retry path because they reference both a device and a device group that may arrive on Kafka later.\n\n### Device flow\n\nFor `KafkaDevice`, the service:\n\n1. Looks up an existing `Device` by `sourceId`.\n2. Maps the Kafka payload onto a new or existing JPA entity with `EntityMappingService`.\n3. Saves the `Device` through `DeviceRepository`.\n4. Maps the saved entity to `KontrollDevice`.\n5. Publishes the Kontroll entity with `KontrollDevicePublishingComponent`.\n\nDevices are upserted by `sourceId`. Partial fields in later events preserve selected existing values according to the mapping logic.\n\n### Device group flow\n\nFor `KafkaDeviceGroup`, the service:\n\n1. Looks up an existing `DeviceGroup` by `sourceId`.\n2. Maps the Kafka payload onto a new or existing JPA entity.\n3. Saves the `DeviceGroup`.\n4. Synchronizes the group's member count with `DeviceGroupRepository.syncNoOfMembers(...)`.\n5. Maps and publishes the saved group as `KontrollDeviceGroup`.\n\nDevice groups are also upserted by `sourceId`.\n\n### Membership flow\n\nFor `KafkaDeviceGroupMembership`, the service:\n\n1. Looks up the target `DeviceGroup` by the Kafka membership `deviceGroupId`.\n2. Looks up the target `Device` by the Kafka membership `deviceId`.\n3. Buffers the membership for retry if either side is missing.\n4. Builds the composite membership ID from the persisted group ID and device ID.\n5. Upserts the `DeviceGroupMembership`.\n6. Publishes `KontrollDeviceGroupMembership`.\n7. Republishes the affected `KontrollDeviceGroup` after synchronizing the member count.\n\nMemberships depend on devices and groups already being present locally. If a membership event arrives before its device or group, it is placed in `DeviceGroupMembershipRetryBuffer`. `DeviceGroupMembershipRetryScheduler` drains the buffer every 10 seconds and sends each item back through `EntityPersistenceService.handle(...)`.\n\n## Kafka Topics\n\nThe service consumes these resource topics through `DeviceConsumerConfiguration`:\n\n- `device`\n- `device-group`\n- `device-group-membership`\n\nIt publishes normalized Kontroll entities to:\n\n- `kontroll-device`\n- `kontroll-device-group`\n- `kontroll-device-group-membership`\n\nTopic names are built with the configured Novari/FINT topic prefix parameters, including org ID and domain context.\n\n## REST API\n\nThe API is exposed under `/api`:\n\n- `GET /api/devicegroups`\n- `GET /api/devicegroups/{id}`\n- `GET /api/devices`\n- `GET /api/devices/{id}`\n- `GET /api/devicegroups/{id}/members`\n- `POST /api/devicegroups/publishAllDeviceGroupsDevicesMembership`\n\nThe `publishAllDeviceGroupsDevicesMembership` endpoint republishes all persisted device groups, devices, and memberships. It is protected with `@OnlyDevelopers`.\n\nSwagger UI is available at:\n\n```text\nhttp://localhost:\u003cport\u003e/swagger-ui/index.html\n```\n\n## Running Locally\n\nThe project requires Java 21 and PostgreSQL. The `local-staging` profile expects:\n\n- PostgreSQL\n- Kafka service\n\nRun the service with:\n\n```bash\n./gradlew bootRun --args='--spring.profiles.active=local-staging'\n```\n\nRun tests with:\n\n```bash\n./gradlew test\n```\n\n## Persistence\n\nFlyway migrations are loaded from:\n\n```text\nclasspath:db/migration\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffintlabs%2Ffint-kontroll-device-catalog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffintlabs%2Ffint-kontroll-device-catalog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffintlabs%2Ffint-kontroll-device-catalog/lists"}