{"id":29908782,"url":"https://github.com/engineering-research-and-development/universal-data-connector","last_synced_at":"2025-08-02T00:44:18.850Z","repository":{"id":303379253,"uuid":"1015277419","full_name":"Engineering-Research-and-Development/universal-data-connector","owner":"Engineering-Research-and-Development","description":null,"archived":false,"fork":false,"pushed_at":"2025-07-07T09:13:10.000Z","size":156,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-07T10:28:01.647Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Engineering-Research-and-Development.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-07T09:08:18.000Z","updated_at":"2025-07-07T09:13:14.000Z","dependencies_parsed_at":"2025-07-07T10:29:00.282Z","dependency_job_id":"6723520b-82bc-4565-8624-a7647cc3b0bf","html_url":"https://github.com/Engineering-Research-and-Development/universal-data-connector","commit_stats":null,"previous_names":["engineering-research-and-development/universal-data-connector"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Engineering-Research-and-Development/universal-data-connector","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Engineering-Research-and-Development%2Funiversal-data-connector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Engineering-Research-and-Development%2Funiversal-data-connector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Engineering-Research-and-Development%2Funiversal-data-connector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Engineering-Research-and-Development%2Funiversal-data-connector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Engineering-Research-and-Development","download_url":"https://codeload.github.com/Engineering-Research-and-Development/universal-data-connector/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Engineering-Research-and-Development%2Funiversal-data-connector/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268320374,"owners_count":24231801,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-01T02:00:08.611Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-02T00:44:03.290Z","updated_at":"2025-08-02T00:44:18.765Z","avatar_url":"https://github.com/Engineering-Research-and-Development.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Universal Data Connector\n\nUn connettore dati universale configurabile per Industry 5.0 che supporta l'ingestion di dati da multiple sources.\n\n## Caratteristiche\n\n- 🔌 **Multi-Source Support**: OPC UA, MQTT, HTTP REST\n- ⚙️ **Configurazione Flessibile**: Configurazione tramite file JSON\n- 🚀 **API REST**: Monitoraggio stato e controllo\n- 📊 **Real-time Processing**: Elaborazione dati in tempo reale\n- 🔄 **Modular Architecture**: Architettura modulare ed estensibile\n- 📝 **Logging Avanzato**: Sistema di logging strutturato\n\n## Sources Supportate\n\n### OPC UA\n- Connessione a server OPC UA\n- Subscription a nodi specifici\n- Gestione autenticazione e certificati\n\n### MQTT\n- Connessione a broker MQTT\n- Subscription a topic multipli\n- Supporto QoS levels\n\n### HTTP REST\n- Polling di endpoint REST\n- Supporto autenticazione (Bearer, Basic)\n- Scheduling configurabile\n\n## Quick Start\n\n```bash\n# Installa le dipendenze\nnpm install\n\n# Avvia in modalità sviluppo\nnpm run dev\n\n# Avvia in produzione\nnpm start\n```\n\n## Configurazione\n\nIl connettore utilizza un file `config/sources.json` per definire le sources di dati.\n\nEsempio di configurazione:\n\n```json\n{\n  \"sources\": [\n    {\n      \"id\": \"plc-001\",\n      \"type\": \"opcua\",\n      \"enabled\": true,\n      \"config\": {\n        \"endpoint\": \"opc.tcp://192.168.1.100:4840\",\n        \"nodes\": [\"ns=2;s=Temperature\", \"ns=2;s=Pressure\"]\n      }\n    },\n    {\n      \"id\": \"sensor-mqtt\",\n      \"type\": \"mqtt\",\n      \"enabled\": true,\n      \"config\": {\n        \"broker\": \"mqtt://localhost:1883\",\n        \"topics\": [\"sensors/+/temperature\", \"sensors/+/humidity\"]\n      }\n    }\n  ]\n}\n```\n\n## API Endpoints\n\n- `GET /api/status` - Stato generale del connettore\n- `GET /api/sources` - Lista sources configurate\n- `GET /api/sources/:id/status` - Stato di una source specifica\n- `POST /api/sources/:id/start` - Avvia una source\n- `POST /api/sources/:id/stop` - Ferma una source\n- `GET /api/data/latest` - Ultimi dati ricevuti\n\n## Dynamic Configuration\n\nIl Universal Data Connector supporta la **configurazione dinamica** senza riavvio del servizio:\n\n### Configurazione Sources in Real-time\n```bash\n# Aggiorna la configurazione delle sources\ncurl -X POST \"http://localhost:3000/api/config/sources/configure\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"sources\": [\n      {\n        \"id\": \"new-plc\",\n        \"type\": \"opcua\", \n        \"enabled\": true,\n        \"config\": {\n          \"endpoint\": \"opc.tcp://192.168.1.100:4840\",\n          \"nodes\": [\"ns=2;s=Temperature\"]\n        }\n      }\n    ]\n  }'\n```\n\n### Configurazione Storage in Real-time\n```bash\n# Cambia storage da memory a PostgreSQL\ncurl -X POST \"http://localhost:3000/api/config/storage/configure\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"type\": \"postgresql\",\n    \"config\": {\n      \"host\": \"localhost\",\n      \"port\": 5432,\n      \"database\": \"universal_data_connector\",\n      \"username\": \"postgres\",\n      \"password\": \"password\"\n    }\n  }'\n```\n\n### API di Configurazione Dinamica\n\n- `POST /api/config/sources/configure` - Configura sources con payload completo\n- `POST /api/config/sources/reload` - Ricarica configurazione sources\n- `POST /api/config/storage/configure` - Configura storage con test connessione\n- `POST /api/config/storage/reload` - Ricarica configurazione storage\n- `GET /api/config/engine/status` - Stato engine e capacità di riconfigurazione\n\n**Caratteristiche:**\n- ✅ Nessun riavvio richiesto\n- ✅ Test automatico connessioni storage\n- ✅ Migrazione dati automatica\n- ✅ Validazione configurazioni\n- ✅ Rollback in caso di errori\n\nPer dettagli completi, vedi [Dynamic Configuration Guide](docs/DynamicConfiguration.md).\n\n## Storage Configuration\n\nIl Universal Data Connector supporta diversi backend di storage per persistere i dati raccolti:\n\n### Tipi di Storage Supportati\n\n- **Memory** - Storage temporaneo in memoria (predefinito)\n- **PostgreSQL** - Database relazionale per alta performance\n- **MariaDB/MySQL** - Database relazionale compatibile MySQL\n- **MongoDB** - Database NoSQL per dati semi-strutturati\n- **Redis** - Cache in memoria ad alte prestazioni\n\n### Configurazione Storage\n\nConfigura lo storage nel file `config/storage.json`:\n\n```json\n{\n  \"storage\": {\n    \"type\": \"postgresql\",\n    \"config\": {\n      \"host\": \"localhost\",\n      \"port\": 5432,\n      \"database\": \"universal_data_connector\",\n      \"username\": \"postgres\",\n      \"password\": \"password\",\n      \"table\": \"sensor_data\",\n      \"pool\": {\n        \"min\": 2,\n        \"max\": 10\n      }\n    }\n  }\n}\n```\n\n### API Storage\n\n- `GET /api/config/storage` - Configurazione storage corrente\n- `PUT /api/config/storage` - Aggiorna configurazione storage\n- `POST /api/config/storage/test` - Testa connessione storage\n- `GET /api/config/storage/health` - Stato e statistiche storage\n- `GET /api/config/storage/types` - Tipi storage disponibili\n\nPer dettagli completi sulla configurazione storage, vedi [Storage Configuration Guide](docs/Storage.md).\n\n## Struttura Progetto\n\n```\nsrc/\n├── server.js              # Entry point principale\n├── core/                  # Core engine\n├── connectors/            # Moduli connettori sources\n├── config/                # Sistema configurazione\n├── api/                   # REST API routes\n├── utils/                 # Utilities\n└── types/                 # Type definitions\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fengineering-research-and-development%2Funiversal-data-connector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fengineering-research-and-development%2Funiversal-data-connector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fengineering-research-and-development%2Funiversal-data-connector/lists"}