{"id":37079687,"url":"https://github.com/muxu-io/mqtt-application","last_synced_at":"2026-01-14T09:38:37.327Z","repository":{"id":309086467,"uuid":"1031525540","full_name":"muxu-io/mqtt-application","owner":"muxu-io","description":null,"archived":false,"fork":false,"pushed_at":"2025-08-17T19:25:25.000Z","size":110,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-09-23T14:28:32.281Z","etag":null,"topics":["asyncio","icsia","iot","mqtt","python"],"latest_commit_sha":null,"homepage":"https://github.com/muxu-io/mqtt-application","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/muxu-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-03T23:30:45.000Z","updated_at":"2025-08-17T19:25:27.000Z","dependencies_parsed_at":"2025-08-09T19:32:09.260Z","dependency_job_id":"00475fcc-80f4-411b-921a-717f410041fb","html_url":"https://github.com/muxu-io/mqtt-application","commit_stats":null,"previous_names":["muxu-io/mqtt-application"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/muxu-io/mqtt-application","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muxu-io%2Fmqtt-application","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muxu-io%2Fmqtt-application/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muxu-io%2Fmqtt-application/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muxu-io%2Fmqtt-application/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/muxu-io","download_url":"https://codeload.github.com/muxu-io/mqtt-application/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muxu-io%2Fmqtt-application/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28416120,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T08:38:59.149Z","status":"ssl_error","status_checked_at":"2026-01-14T08:38:43.588Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asyncio","icsia","iot","mqtt","python"],"created_at":"2026-01-14T09:38:36.641Z","updated_at":"2026-01-14T09:38:37.318Z","avatar_url":"https://github.com/muxu-io.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MQTT Application\n\nA comprehensive asynchronous MQTT client library for Python, designed for building robust IoT applications and message processing systems.\n\n## Features\n\n- **Asynchronous Architecture**: Built with `asyncio` for high-performance concurrent operations\n- **Automatic Reconnection**: Robust connection handling with configurable retry logic\n- **Message Processing**: Worker-based system for concurrent message handling\n- **Status Publishing**: Periodic device status reporting\n- **Command Handling**: Built-in command processing with acknowledgment system\n- **Configuration Management**: YAML-based configuration with environment variable support\n- **Logging Integration**: Seamless integration with mqtt-logger for distributed logging\n\n## Installation\n\n```bash\npip install muxu-io-mqtt-application\n```\n\n## Installation from Source\n\nIf you're working with the source code from this repository, you'll need to install the dependencies in the correct order:\n\n```bash\ncd ~/projects/icsia/dummy-icsia\n\n# Create and activate virtual environment (recommended)\npython3 -m venv .venv\nsource .venv/bin/activate  # On Linux/macOS\n# or\n.venv\\Scripts\\activate     # On Windows\n\n# Install the mqtt-logger package\npushd ../mqtt-logger \u0026\u0026 pip install -e \".\" \u0026\u0026 popd\n\n# Install the mqtt-application package with dev dependencies\npushd ../mqtt-application \u0026\u0026 pip install -e \".\" \u0026\u0026 popd\n```\n\n## Dependencies\n\nThis package requires the following external modules:\n- `mqtt-logger`: For MQTT-enabled logging capabilities\n- `muxu-io-mqtt-connector`: For low-level MQTT connection management (PyPI package)\n\n## Quick Start\n\n### Simplified Usage (Recommended)\n\nThe easiest way to use the library is with the `MqttApplication` class that handles everything automatically:\n\n```python\nimport asyncio\nfrom mqtt_application import MqttApplication\n\nasync def main():\n    # Everything configured from config.yaml\n    async with MqttApplication() as app:\n        await app.run()\n\nif __name__ == \"__main__\":\n    asyncio.run(main())\n```\n\nOr even simpler for standalone usage:\n\n```python\nfrom mqtt_application import MqttApplication\n\n# One-liner to run the application\nif __name__ == \"__main__\":\n    MqttApplication.run_from_config()\n```\n\n### Custom Command Handlers\n\nTo add custom business logic, register command handlers:\n\n```python\nimport asyncio\nfrom mqtt_application import MqttApplication\n\nasync def my_custom_command(data):\n    \"\"\"Handle custom command.\"\"\"\n    # Your business logic here\n    print(f\"Processing custom command: {data}\")\n    return {\"status\": \"completed\", \"result\": \"success\"}\n\nasync def main():\n    async with MqttApplication() as app:\n        # Register custom commands\n        app.register_command(\"my_command\", my_custom_command)\n        await app.run()\n\nif __name__ == \"__main__\":\n    asyncio.run(main())\n```\n\n### Message Subscriptions\n\nSubscribe to MQTT messages using config-based subscriptions or programmatic registration:\n\n#### Config-Based Subscriptions (Recommended)\n\n```yaml\n# config.yaml\nsubscriptions:\n  status_messages:\n    topic_pattern: \"icsia/+/status/current\"\n    callback_method: \"_on_status_message\"\n  ack_messages:\n    topic_pattern: \"icsia/+/status/ack\"\n    callback_method: \"_on_ack_message\"\n```\n\n```python\nclass MyApplication:\n    def __init__(self):\n        # Pass self as callback_context so config can find your methods\n        self.app = MqttApplication(callback_context=self)\n\n    async def _on_status_message(self, topic: str, payload: str, properties):\n        \"\"\"Handle status messages from any device.\"\"\"\n        print(f\"Status from {topic}: {payload}\")\n\n    async def _on_ack_message(self, topic: str, payload: str, properties):\n        \"\"\"Handle acknowledgment messages.\"\"\"\n        print(f\"ACK from {topic}: {payload}\")\n\n    async def run(self):\n        async with self.app:\n            await self.app.run()\n```\n\n#### Programmatic Registration\n\n```python\nasync def my_handler(topic: str, payload: str, properties):\n    print(f\"Message on {topic}: {payload}\")\n\nasync def main():\n    async with MqttApplication() as app:\n        # Register handler for config-based subscriptions\n        app.register_callback_handler(\"my_handler\", my_handler)\n        await app.run()\n```\n\n## Configuration\n\nCreate a `config.yaml` file in your project root:\n\n```yaml\n---\n# MQTT Broker settings\nmqtt:\n  broker: \"localhost\"\n  port: 1883\n  reconnect_interval: 5\n  max_reconnect_attempts: -1  # -1 means infinite attempts\n  throttle_interval: 0.1\n\n# Device configuration\ndevice:\n  device_id: \"my_device_01\"\n  namespace: \"icsia\"  # Configurable namespace for topic patterns\n  status_publish_interval: 30.0\n\n# Auto-generated topic patterns from namespace + device_id:\n# {namespace}/+/cmd/#\n# {namespace}/{device_id}/logs\n# {namespace}/{device_id}/status/ack\n# {namespace}/{device_id}/status/completion\n# {namespace}/{device_id}/status/current\n\n# Logger settings\nlogger:\n  log_file: \"{device_id}.log\"\n  log_level: \"INFO\"\n\n# Worker configuration\nworkers:\n  count: 3\n```\n\n## API Reference\n\n### AsyncMqttClient\n\nThe main MQTT client class for connecting to brokers and handling messages.\n\n```python\nAsyncMqttClient(\n    broker_address: str,\n    port: int,\n    topics: list[str],\n    message_queue: asyncio.Queue,\n    logger: MqttLogger,\n    reconnect_interval: int = 5,\n    max_reconnect_attempts: int = -1\n)\n```\n\n### AsyncCommandHandler\n\nHandles command processing and acknowledgments.\n\n```python\nAsyncCommandHandler(\n    logger: MqttLogger,\n    mqtt_broker: Optional[str] = None,\n    mqtt_port: Optional[int] = None,\n    ack_topic_pattern: str = \"devices/{device_id}/status/ack\",\n    completion_topic_pattern: str = \"devices/{device_id}/status/completion\"\n)\n```\n\n### PeriodicStatusPublisher\n\nPublishes device status at regular intervals.\n\n```python\nPeriodicStatusPublisher(\n    device_id: str,\n    logger: MqttLogger,\n    mqtt_broker: str,\n    mqtt_port: int,\n    publish_interval: float = 30.0,\n    status_topic_pattern: str = \"devices/{device_id}/status/current\"\n)\n```\n\n### Config\n\nConfiguration management with YAML support.\n\n```python\nfrom mqtt_application import config\n\n# Get configuration values\nmqtt_config = config.get_mqtt_config()\ndevice_id = config.get(\"device.device_id\", \"default\")\nlog_level = config.get_log_level()\n```\n\n## Command Line Usage\n\nYou can also run the library as a standalone application:\n\n```bash\nmqtt-application  # Uses config.yaml in current directory\n```\n\n## Development Setup\n\n### Virtual Environment Setup\n\nIt's recommended to use a virtual environment for development:\n\n```bash\n# Create virtual environment\npython3 -m venv .venv\n\n# Activate virtual environment\nsource .venv/bin/activate  # On Linux/macOS\n# or\n.venv\\Scripts\\activate     # On Windows\n\n# Install development dependencies\npip install -e \".[dev]\"\n```\n\n### Running Integration Tests\n\nThis project uses comprehensive integration tests that validate real-world functionality with actual MQTT brokers and network connections.\n\n**Prerequisites**: Ensure your virtual environment is activated and development dependencies are installed:\n\n```bash\n# Activate virtual environment\nsource .venv/bin/activate  # On Linux/macOS\n.venv\\Scripts\\activate     # On Windows\n\n# Install development dependencies\npip install -e \".[dev]\"\n```\n\n**Note**: If you encounter issues with `python` command not finding pytest, use the virtual environment directly:\n```bash\n# Direct virtual environment usage (Linux/macOS)\n.venv/bin/python -m pytest\n\n# Direct virtual environment usage (Windows)\n.venv\\Scripts\\python -m pytest\n```\n\n### Running Tests\n\nThis project uses comprehensive integration tests that validate real-world functionality with actual MQTT brokers and network connections.\n\n#### Prerequisites\n\nMake sure your virtual environment is activated and development dependencies are installed:\n\n```bash\n# Activate virtual environment\nsource .venv/bin/activate  # On Linux/macOS\n.venv\\Scripts\\activate     # On Windows\n\n# Install development dependencies\npip install -e \".[dev]\"\n```\n\n**Note**: If you encounter `No module named pytest` errors, make sure your virtual environment is activated:\n\n```bash\n# Activate virtual environment\nsource .venv/bin/activate  # On Linux/macOS\n.venv\\Scripts\\activate     # On Windows\n\n# Install development dependencies\npip install -e \".[dev]\"\n```\n\n**Alternative**: Use the virtual environment directly without activation:\n```bash\n# Direct virtual environment usage (Linux/macOS)\n.venv/bin/python -m pytest\n\n# Direct virtual environment usage (Windows)\n.venv\\Scripts\\python -m pytest\n```\n\n#### Basic Test Commands\n\n```bash\n# Run all tests (integration and unit tests)\npython -m pytest\n\n# Run only integration tests\npython -m pytest -m integration\n\n# Run only unit tests (non-integration)\npython -m pytest -m \"not integration\"\n\n# Run with verbose output\npython -m pytest -v\n\n# Run specific test file\npython -m pytest tests/test_integration.py\n\n# Run specific test class\npython -m pytest tests/test_integration.py::TestMqttIntegration\n\n# Run specific test method\npython -m pytest tests/test_integration.py::TestMqttIntegration::test_mqtt_logger_connection\n\n# Run tests matching a pattern\npython -m pytest -k \"mqtt_logger\"\n```\n\n#### Advanced Testing Options\n\n```bash\n# Skip slow tests during development\npython -m pytest -m \"integration and not slow\"\n\n# Run tests with coverage reporting (requires pytest-cov)\npython -m pytest --cov=src/mqtt_application --cov-report=html --cov-report=term\n\n# Generate XML coverage report for CI/CD\npython -m pytest --cov=src/mqtt_application --cov-report=xml\n\n# Run tests in parallel (requires pytest-xdist)\npython -m pytest -n auto\n\n# Stop on first failure (useful during development)\npython -m pytest -x\n\n# Run last failed tests only\npython -m pytest --lf\n\n# Show local variables in tracebacks for debugging\npython -m pytest -l\n\n# Run in quiet mode (less verbose output)\npython -m pytest -q\n\n# Show test durations (identify slow tests)\npython -m pytest --durations=10\n```\n\n#### Common Development Workflows\n\n```bash\n# Quick feedback during development (unit tests only)\npython -m pytest -m \"not integration\" -x\n\n# Fast integration tests only (skip slow network tests)\npython -m pytest -m \"integration and not slow\"\n\n# Full test suite before committing\npython -m pytest -v\n\n# Debug a specific failing test with maximum verbosity\npython -m pytest tests/test_integration.py::TestMqttIntegration::test_mqtt_logger_connection -vvv -s\n\n# Test specific functionality you're working on\npython -m pytest -k \"command_handler\" -v\n\n# Continuous testing during development (requires pytest-watch)\nptw -- -m \"not integration\"\n```\n\n#### Test Organization\n\nThis project uses pytest markers to categorize tests:\n\n- `@pytest.mark.integration`: Tests requiring network access and real MQTT connections\n- `@pytest.mark.slow`: Tests that take longer to run (network resilience, retry mechanisms)\n\nView all available markers:\n```bash\npython -m pytest --markers\n```\n\n#### CI/CD Integration\n\n```bash\n# Full test suite with coverage for CI\npython -m pytest -v --cov=src/mqtt_application --cov-report=xml --cov-report=term\n\n# Integration tests only for production validation\npython -m pytest -v -m integration\n\n# Quick validation (unit tests + fast integration tests)\npython -m pytest -m \"not slow\" -v\n```\n\n**Note**: Integration tests use real MQTT connections to `test.mosquitto.org` and test actual component interactions. This provides more reliable validation than mock-based unit tests, but requires network access.\n\n#### Troubleshooting\n\n**Virtual Environment Issues:**\n```bash\n# If 'python' command doesn't work, use the virtual environment directly\n.venv/bin/python -m pytest        # Linux/macOS\n.venv\\Scripts\\python -m pytest    # Windows\n\n# Check you're using the right Python version (3.8+)\npython --version\nwhich python  # Should point to .venv/bin/python (Linux/macOS)\n```\n\n**Common Issues and Solutions:**\n\n1. **SyntaxError with pytest**: Ensure you're using Python 3.8+ from your virtual environment, not system Python 2.7\n2. **ModuleNotFoundError**: Install development dependencies with `pip install -e \".[dev]\"`\n3. **Network timeouts**: Integration tests require internet access to `test.mosquitto.org`. Use `-m \"not integration\"` to skip them\n4. **Permission denied**: Use the full path to the virtual environment's Python executable\n5. **Tests hang**: Some integration tests make real network connections. Use `Ctrl+C` to interrupt and check your network connection\n\n**Getting Help:**\n```bash\n# Show pytest help\npython -m pytest --help\n\n# Show available fixtures\npython -m pytest --fixtures\n\n# Show available markers\npython -m pytest --markers\n\n# Collect tests without running them\npython -m pytest --collect-only\n```\n```bash\npython -m pytest --markers | grep @pytest.mark\n```\n\nThe integration test suite covers:\n- Real MQTT broker connections and message publishing/subscribing\n- End-to-end command processing workflows\n- Status publishing and periodic operations\n- Network resilience and error handling\n- Malformed message handling\n- Connection retry mechanisms\n\n### Code Formatting\n\n```bash\nblack .\nruff check .\n```\n\n## Architecture\n\nThe library is designed with a modular architecture:\n\n- **AsyncMqttClient**: Core MQTT connectivity and message routing\n- **AsyncCommandHandler**: Command processing with built-in acknowledgment system\n- **PeriodicStatusPublisher**: Regular status reporting functionality\n- **Workers**: Concurrent message processing system\n- **Config**: Centralized configuration management\n\n## Dependencies\n\n- **External modules** (must be installed separately):\n  - `mqtt-logger`: MQTT-enabled logging\n  - `muxu-io-mqtt-connector`: Low-level MQTT operations\n- **Standard dependencies**:\n  - `aiomqtt`: Async MQTT client\n  - `pyyaml`: YAML configuration parsing\n\n## License\n\nMIT License - see LICENSE file for details.\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Set up development environment:\n   ```bash\n   python3 -m venv .venv\n   source .venv/bin/activate  # On Linux/macOS\n   pip install -e \".[dev]\"\n   ```\n4. Make your changes\n5. Add tests for new functionality\n6. Run the test suite:\n   ```bash\n   # Run all tests\n   python -m pytest\n\n   # Or run quick tests during development\n   python -m pytest -m \"not integration\" -x\n   ```\n7. Submit a pull request\n\n## Support\n\nFor issues and questions:\n- GitHub Issues: [Project Issues](https://github.com/muxu-io/mqtt-application/issues)\n- Email: alex@muxu.io\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmuxu-io%2Fmqtt-application","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmuxu-io%2Fmqtt-application","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmuxu-io%2Fmqtt-application/lists"}