{"id":23860300,"url":"https://github.com/gamussa/parkflow","last_synced_at":"2026-05-10T07:32:21.088Z","repository":{"id":268600203,"uuid":"904887478","full_name":"gAmUssA/parkflow","owner":"gAmUssA","description":"Real-time parking management system with stream processing and analytics capabilities.","archived":false,"fork":false,"pushed_at":"2024-12-23T23:16:53.000Z","size":571,"stargazers_count":0,"open_issues_count":2,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-03T04:41:47.212Z","etag":null,"topics":["duckdb","flink-stream-processing","kafka","kafka-streams","python","uv"],"latest_commit_sha":null,"homepage":"","language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gAmUssA.png","metadata":{"files":{"readme":"README.adoc","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-17T18:34:28.000Z","updated_at":"2024-12-18T23:08:43.000Z","dependencies_parsed_at":"2024-12-17T19:47:25.447Z","dependency_job_id":null,"html_url":"https://github.com/gAmUssA/parkflow","commit_stats":null,"previous_names":["gamussa/parkflow"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gAmUssA%2Fparkflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gAmUssA%2Fparkflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gAmUssA%2Fparkflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gAmUssA%2Fparkflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gAmUssA","download_url":"https://codeload.github.com/gAmUssA/parkflow/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240170070,"owners_count":19759140,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["duckdb","flink-stream-processing","kafka","kafka-streams","python","uv"],"created_at":"2025-01-03T04:39:14.267Z","updated_at":"2026-05-10T07:32:16.048Z","avatar_url":"https://github.com/gAmUssA.png","language":"Kotlin","funding_links":[],"categories":[],"sub_categories":[],"readme":"= ParkFlow\n:toc: macro\n:toc-title: Table of Contents\n:toclevels: 3\n\n[.lead]\nPARKFLOW™ (Parking Analytics \u0026 Real-time Knowledge Framework for Live Operations Window).\n\nReal-time parking management system with stream processing and analytics capabilities.\n\nWARNING: Not an actual system, just a demo of integrating a bunch of technologies of Modern Data Stack – Kafka for Streaming, DuckDB for Analytics, Plotly/Dash for visualisation.\n\n== Quick Start\n[source,bash]\n----\n# Clone and start the project\ngit clone https://github.com/yourusername/parkflow.git\ncd parkflow\nmake install  # Install dependencies\nmake start   # Start all services\n\n# Send some test events\nmake simulate EVENTS=20 DELAY=500\n\n# Open dashboard\nopen http://localhost:8050\n----\n\ntoc::[]\n\n== About ParkFlow\n\nParkFlow is a modern parking management system designed to handle real-time vehicle entry/exit events, process payments, and provide instant analytics.\n\nKey features:\n\n* Real-time vehicle detection and entry/exit management\n* Stream processing of parking events using Kafka Streams and Apache Flink\n* Analytics dashboard with DuckDB for quick insights\n* Scalable microservices architecture using Kotlin and Python\n* Modern web interface built with Plotly Dash\n\n== Project Structure\n\nEach component is designed to be independent and scalable.\nThe system follows modern microservices architecture principles.\nAll services communicate through well-defined APIs and event streams.\n\nimage::img.png[]\n\n[cols=\"3\"]\n|===\n|Module |Status |Description\n\n|`parkflow-common`\n|🟢 Ready\n|Shared models and utilities.\nThis module contains code used across multiple services.\n\n|`parkflow-entry-exit`\n|🟢 Ready\n|Gate control service.\nManages vehicle entry and exit points.\n\n|`parkflow-dashboard`\n|🟢 Ready\n|Plotly Dash UI.\nProvides real-time visualization and monitoring.\n\n|`parkflow_cli`\n|🟢 Ready\n|Command Line Interface.\nProvides development tools and utilities.\n\n|`parkflow-stream`\n|🟡 In Progress\n|Kafka Streams processor.\nHandles real-time event processing and state updates.\n\n|`parkflow-analytics`\n|🟡 In Progress\n|DuckDB analytics engine.\nProvides fast analytical queries and insights.\n\n|`parkflow-flink`\n|🔴 Not Started\n|Flink processor implementation.\nProvides advanced stream processing capabilities.\n\n|`parkflow-payment`\n|🔴 Not Started\n|Payment processing service.\nHandles all financial transactions.\n\n|`parkflow-gateway`\n|🔴 Not Started\n|API Gateway.\nRoutes requests and handles service discovery.\n|===\n\n== Technology Stack\n\n=== JVM Services\n* Kotlin with Gradle (Kotlin DSL)\n\n=== Python Services\n* Python 3.11+ with UV package manager\n\n=== Stream Processing\n* Kafka Streams 3.6.1\n* Apache Flink 1.19\n\n=== Services \u0026 Analytics\n* REST Services: FastAPI\n* Analytics: DuckDB\n* Dashboards: Plotly/Dash\n* Event Schemas: Apache Avro\n\n=== Testing\n* JVM: Kotest with TestContainers\n* Python: pytest with pytest-asyncio\n* Coverage minimum: 80%\n\n== Event Schema and Flow\n\n=== Event Types\n\n==== VehicleEntryEvent\nRecords vehicle entry into parking facility:\n\n* Unique event ID and timestamp\n* License plate and recognition confidence\n* Gate and lane identifiers\n* Optional vehicle image URL\n* Vehicle type (CAR, MOTORCYCLE, TRUCK)\n\n==== PaymentEvent\nRecords parking payment transactions:\n\n* Unique event and transaction IDs\n* License plate reference\n* Amount and currency\n* Payment method (CREDIT_CARD, DEBIT_CARD, MOBILE_PAYMENT, CASH)\n* Payment status (PENDING, COMPLETED, FAILED, REFUNDED)\n* Parking duration\n\n==== VehicleExitEvent\nRecords vehicle exit from parking facility:\n\n* Unique event ID and timestamp\n* License plate and recognition confidence\n* Gate and lane identifiers\n* Reference to entry event\n* Optional vehicle image URL\n\n=== Event Flow Logic\n\nThe system generates events following a strict sequence:\n\n. *Vehicle Entry*\n** Generated when capacity allows\n** More frequent during peak hours (8-9 AM, 4-5 PM)\n** Less frequent during quiet periods (11 PM - 5 AM)\n\n. *Payment Processing*\n** Occurs after minimum 5-minute stay\n** 85% of sessions require payment\n** Payment amount based on duration (base $2 + $3/hour, max $25)\n** Payment methods distribution:\n*** Credit Card: 70%\n*** Debit Card: 25%\n*** Cash: 5%\n\n. *Vehicle Exit*\n** Generated after payment completion (if required)\n** Must reference original entry event\n** Completes the parking session\n\n=== Stay Duration Patterns\n\n* Quick stops: 5-15 minutes (10% of sessions)\n* Shopping: 1-3 hours (60% of sessions)\n* Work/Long-term: 8-10 hours (30% of sessions)\n\n=== Traffic Patterns\n\n* Peak Hours (8-9 AM, 4-5 PM)\n** 80% chance of new entry\n** 6-12 second intervals\n\n* Normal Hours\n** 30% chance of new entry\n** 20-60 second intervals\n\n* Quiet Period (11 PM - 5 AM)\n** 10% chance of new entry\n** 10-15 minute intervals\n\n== Analytics Service (DuckDB)\n\nThe analytics service provides a REST API for querying and analyzing parking data using DuckDB.\n\n=== API Endpoints\n\n[cols=\"2,1,2\"]\n|===\n|Endpoint |Method |Description\n\n|`/health`\n|GET\n|Check service health\n\n|`/query`\n|POST\n|Execute SQL queries\n\n|`/upload`\n|POST\n|Upload CSV files to DuckDB tables\n\n|`/tables`\n|GET\n|List available tables\n\n|`/schema/{table_name}`\n|GET\n|Get schema for a specific table\n\n|`/analyze/{table_name}`\n|POST\n|Get basic statistics for a table\n|===\n\n=== Example Usage\n\n.Check service health\n[source,bash]\n----\ncurl http://localhost:3000/health\n----\n\n.Execute a SQL query\n[source,bash]\n----\ncurl -X POST http://localhost:3000/query \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"query\": \"SELECT * FROM parking_events LIMIT 5\"}'\n----\n\n.Upload a CSV file\n[source,bash]\n----\ncurl -X POST http://localhost:3000/upload \\\n  -F \"file=@data.csv\" \\\n  -F \"table_name=parking_events\"\n----\n\n.Get table schema\n[source,bash]\n----\ncurl http://localhost:3000/schema/parking_events\n----\n\n.Get table statistics\n[source,bash]\n----\ncurl -X POST http://localhost:3000/analyze/parking_events \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"columns\": [\"duration\", \"amount\"]}'\n----\n\n=== Docker Configuration\n\nThe DuckDB service is containerized using Docker with the following features:\n\n* Uses official DuckDB binary (v1.1.0)\n* FastAPI-based REST interface\n* Persistent storage in `/data/analytics.db`\n* Health monitoring\n* CORS support for web clients\n\n=== Environment Variables\n\n[cols=\"1,2\"]\n|===\n|Variable |Description\n\n|`DUCKDB_DATABASE`\n|Path to DuckDB database file (default: `/data/analytics.db`)\n\n|`PYTHONUNBUFFERED`\n|Python output buffering (set to 1 for immediate logs)\n|===\n\n== Development Setup\n\n=== Prerequisites\n\n* JDK 17+\n* Python 3.11+\n* Docker and Docker Compose\n* UV package manager\n\n=== Building the Project\n\n.JVM services\n[source,bash]\n----\n./gradlew build\n----\n\n.Python services\n[source,bash]\n----\nuv venv\nsource .venv/bin/activate\nuv pip install -r requirements.txt\n----\n\n=== Running Infrastructure\n\n[source,bash]\n----\ndocker-compose up -d\n----\n\n== Testing\n\n.JVM Services\n[source,bash]\n----\n./gradlew test\n----\n\n.Python Services\n[source,bash]\n----\npytest\n----\n\n[NOTE]\n====\nCoverage reports will be generated in:\n\n* JVM services: `build/reports`\n* Python services: `.coverage`\n====\n\n== Running the Applications\n\n=== Prerequisites\n\n* Java 17 or higher\n* Docker and Docker Compose\n* Gradle 8.x\n\n=== Running parkflow-entry-exit\n\n==== Local Development\n\nTo run the application locally with default settings:\n\n[source,bash]\n----\n./gradlew parkflow-entry-exit:run\n----\n\nThis will use the following default configuration:\n\n* Kafka Bootstrap Servers: localhost:29092\n* Schema Registry URL: http://localhost:8081\n* Application Port: 8085\n* Host: 0.0.0.0\n\n==== Environment Variables\n\nYou can customize the application behavior using environment variables:\n\n[cols=\"1,1,1\"]\n|===\n|Variable |Description |Default Value\n\n|KAFKA_BOOTSTRAP_SERVERS\n|Comma-separated list of Kafka brokers\n|localhost:29092\n\n|KAFKA_TOPIC\n|Name of the Kafka topic\n|parking.entry.events\n\n|SCHEMA_REGISTRY_URL\n|URL of the Schema Registry\n|http://localhost:8081\n\n|PORT\n|Application port\n|8085\n\n|HOST\n|Application host\n|0.0.0.0\n|===\n\n==== Running with Different Profiles\n\nUse the provided script to set environment variables for different profiles:\n\n[source,bash]\n----\n# For local development\nsource ./scripts/set-profile.sh local\n\n# For cloud deployment\nsource ./scripts/set-profile.sh cloud\n----\n\n==== Testing the Application\n\n===== Single Event\n\n[source,bash]\n----\ncurl -X POST http://localhost:8085/api/v1/entry/event\n----\n\n===== Simulation\n\n[source,bash]\n----\ncurl -X POST http://localhost:8085/api/v1/entry/simulate \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"numberOfEvents\": 10, \"delayBetweenEventsMs\": 1000}'\n----\n\n=== Infrastructure Services\n\nThe application requires the following services that are defined in docker-compose.yml:\n\n* Kafka (apache/kafka:3.8.0)\n** Running in Kraft mode\n** External port: 29092\n** Internal port: 9092\n\n* Schema Registry (confluentinc/cp-schema-registry:7.8.0)\n** Port: 8081\n** Depends on Kafka\n\n* DuckDB Analytics\n** Port: 3000\n** Mounted volume: ./data\n\n=== Troubleshooting\n\n==== Common Issues\n\n1. Port Already in Use\n+\n[source,bash]\n----\nlsof -i :8085  # Check if port 8085 is in use\nkill -9 \u003cPID\u003e  # Kill the process if needed\n----\n\n2. Kafka Connection Issues\n+\n[source,bash]\n----\n# Check if Kafka is running\ndocker compose ps\n\n# Check Kafka logs\ndocker compose logs kafka\n\n# Restart Kafka\ndocker compose restart kafka\n----\n\n3. Schema Registry Issues\n+\n[source,bash]\n----\n# Check Schema Registry status\ncurl -X GET http://localhost:8081\n\n# Check Schema Registry logs\ndocker compose logs schema-registry\n----\n\n=== Health Checks\n\nThe application provides health check endpoints:\n\n* Kafka: Check via producer metrics\n* Schema Registry: Available at http://localhost:8081/subjects\n* Application: Main endpoint at http://localhost:8085/api/v1/entry/event\n\n== Contributing\n\nWe welcome contributions!\nHere's how you can help:\n\n1. *Fork the Repository*\n* Fork the repo on GitHub\n* Clone your fork locally\n\n2. *Create a Branch*\n* Create a new branch for your feature\n* Make your changes\n* Write or update tests as needed\n\n3. *Submit a Pull Request*\n* Push your changes to your fork\n* Submit a pull request to the main repository\n* Describe your changes in detail\n* Link any relevant issues\n\n4. *Code Review*\n* Wait for review from maintainers\n* Make any requested changes\n* Once approved, your PR will be merged\n\n=== Development Guidelines\n\n* Follow existing code style and conventions\n* Write meaningful commit messages\n* Add tests for new features\n* Update documentation as needed\n* Keep PRs focused and atomic\n\n== License\n\nThis project is licensed under the MIT License.\nSee the link:LICENSE[LICENSE] file for details.\n\nCopyright (c) 2024 Viktor Gamov\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgamussa%2Fparkflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgamussa%2Fparkflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgamussa%2Fparkflow/lists"}