{"id":26822271,"url":"https://github.com/pysyslog/pysyslog-lfc","last_synced_at":"2026-04-18T07:36:39.510Z","repository":{"id":285157362,"uuid":"957238130","full_name":"pysyslog/pysyslog-lfc","owner":"pysyslog","description":"A modular, flow-based syslog processor written in Python, designed for structured log pipelines, JSON output, and modern deployments.","archived":false,"fork":false,"pushed_at":"2025-11-24T23:07:49.000Z","size":8580,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-11-28T09:46:50.618Z","etag":null,"topics":["elasticsearch","ini-configuration","log-parser","log-processing","python-log-processing","python3"],"latest_commit_sha":null,"homepage":"https://pysyslog.com","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pysyslog.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-03-29T21:54:57.000Z","updated_at":"2025-11-24T23:07:57.000Z","dependencies_parsed_at":null,"dependency_job_id":"b392e33a-f4a6-4deb-8ebe-62ff72b0e4ca","html_url":"https://github.com/pysyslog/pysyslog-lfc","commit_stats":null,"previous_names":["pysyslog/pysyslog-lfc"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pysyslog/pysyslog-lfc","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pysyslog%2Fpysyslog-lfc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pysyslog%2Fpysyslog-lfc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pysyslog%2Fpysyslog-lfc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pysyslog%2Fpysyslog-lfc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pysyslog","download_url":"https://codeload.github.com/pysyslog/pysyslog-lfc/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pysyslog%2Fpysyslog-lfc/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31961347,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-18T00:39:45.007Z","status":"online","status_checked_at":"2026-04-18T02:00:07.018Z","response_time":103,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elasticsearch","ini-configuration","log-parser","log-processing","python-log-processing","python3"],"created_at":"2025-03-30T08:18:23.729Z","updated_at":"2026-04-18T07:36:39.503Z","avatar_url":"https://github.com/pysyslog.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n[![Pylint](https://github.com/pysyslog/pysyslog-lfc/actions/workflows/pylint.yml/badge.svg)](https://github.com/pysyslog/pysyslog-lfc/actions/workflows/pylint.yml)\n![Python Versions](https://img.shields.io/badge/python-3.8%20|%203.9%20|%203.10-blue)\n\n\n\n# PySyslog LFC\n\nA lightweight, modular log processor with flow-based configuration.\n\n## Features\n\n- Flow-based log processing model\n- Dynamic component loading\n- Support for various input sources (Unix socket, file, flow chaining)\n- Multiple parser types (RFC 3164, regex, passthrough)\n- Flexible output options (file, TCP, memory for flow chaining)\n- JSON-formatted logs\n- Systemd service integration\n- Clean, modern design without legacy syslog terminology\n\n## Installation\n\n### Manual Installation\n\n#### Prerequisites\n\n- Python 3.8 or higher\n- pip3\n- git\n\n#### Linux/macOS\n\n1. Clone the repository:\n```bash\ngit clone https://github.com/pysyslog/pysyslog-lfc.git\n\n```\n\n```bash\n\ncd pysyslog-lfc\n```\n2. Run the installation script:\n```bash\nsudo ./install.sh\n```\n\n#### Windows\n\n1. Clone the repository:\n\n```cmd\n\ngit clone https://github.com/pysyslog/pysyslog-lfc.git\n\n\n```\n\n```cmd\n\n\ncd pysyslog-lfc\n```\n\n2. Run the installation script as administrator:\n```cmd\ninstall.bat\n```\n\n### Development Installation\n\n1. Clone the repository:\n```bash\ngit clone https://github.com/pysyslog/pysyslog-lfc.git\ncd pysyslog-lfc\n```\n\n2. Create a virtual environment:\n```bash\npython3 -m venv venv\nsource venv/bin/activate  # Linux/macOS\nvenv\\Scripts\\activate     # Windows\n```\n\n3. Install dependencies:\n```bash\npip install -r requirements.txt\n```\n\n4. Install in development mode:\n```bash\npip install -e .\n```\n\n## Configuration\n\nFor detailed configuration documentation, see:\n- [Main Configuration](docs/configuration/main.md)\n- [Flow Configuration](docs/configuration/flows.md)\n\n### Important Note About Current Configuration\n\nThe default `main.ini` file references components that are not yet implemented (see [MISSING_COMPONENTS.md](MISSING_COMPONENTS.md)). For testing with currently available components, use the example configuration:\n\n```bash\n# After installation, replace the config with the working example\nsudo cp /etc/pysyslog/main.ini /etc/pysyslog/main.ini.backup\nsudo cp etc/pysyslog/main.ini.example /etc/pysyslog/main.ini\nsudo systemctl restart pysyslog\n```\n\nOr use the example config directly when running manually:\n```bash\npysyslog -c etc/pysyslog/main.ini.example\n```\n\n## Usage\n\n### Quick Start (Testing on Ubuntu)\n\nFor quick testing without system installation:\n\n```bash\n# 1. Install in development mode (no sudo needed)\npython3 -m venv venv\nsource venv/bin/activate\npip install -r requirements.txt\npip install -e .\n\n# 2. Run the test script to verify everything works\npython3 test_example_config.py\n\n# 3. Test with example configuration (runs in foreground, press Ctrl+C to stop)\npython3 -m pysyslog -c etc/pysyslog/main.ini.example --log-level DEBUG\n```\n\n**Note:** In development mode, you don't need `/etc/pysyslog/` or systemd. Just use the config files from the repository directly (e.g., `etc/pysyslog/main.ini.example`).\n\n### Command Line\n\nStart PySyslog LFC:\n```bash\n# Linux/macOS (system installation)\nsudo pysyslog\n\n# Or specify a custom config file\nsudo pysyslog -c /path/to/config.ini\n\n# Windows\npysyslog\n```\n\n### Testing the Application\n\nBefore deploying to production, test with the example configuration:\n\n```bash\n# 1. Test configuration loading\npython3 -c \"import sys; sys.path.insert(0, 'src'); from pysyslog.config import ConfigLoader; loader = ConfigLoader(); config = loader.load('etc/pysyslog/main.ini.example'); print(f'Loaded {len(config.flows)} flows')\"\n\n# 2. Run the test script\npython3 test_example_config.py\n\n# 3. Test manually with example config\npython3 -m pysyslog -c etc/pysyslog/main.ini.example --log-level DEBUG\n```\n\n### Service Management\n\n#### Linux (systemd)\n```bash\nsudo systemctl start pysyslog\nsudo systemctl stop pysyslog\nsudo systemctl restart pysyslog\nsudo systemctl status pysyslog\n```\n\n#### macOS (launchd)\n```bash\nsudo launchctl start com.pysyslog\nsudo launchctl stop com.pysyslog\nsudo launchctl unload /Library/LaunchDaemons/com.pysyslog.plist\nsudo launchctl load /Library/LaunchDaemons/com.pysyslog.plist\n```\n\n#### Windows\n```cmd\nnet start pysyslog\nnet stop pysyslog\n```\n\n### Viewing Logs\n\n#### Linux (systemd service)\n```bash\n# View service logs\nsudo journalctl -u pysyslog -f\n\n# View service status\nsudo systemctl status pysyslog\n\n# View application logs (if configured to write to files)\nsudo tail -f /var/log/pysyslog/*.log\n```\n\n#### Linux (manual run)\nWhen running manually, logs appear in stdout/stderr. For testing:\n```bash\n# Run with debug logging\npython3 -m pysyslog -c etc/pysyslog/main.ini.example --log-level DEBUG\n```\n\n#### macOS\n```bash\nsudo log show --predicate 'process == \"pysyslog\"' --last 5m\n```\n\n#### Windows\n```cmd\nGet-EventLog -LogName Application -Source pysyslog\n```\n\n### Troubleshooting\n\n#### Service won't start\n1. Check the configuration file syntax:\n   ```bash\n   python3 -c \"import sys; sys.path.insert(0, 'src'); from pysyslog.config import ConfigLoader; ConfigLoader().load('/etc/pysyslog/main.ini')\"\n   ```\n\n2. Check service logs:\n   ```bash\n   sudo journalctl -u pysyslog -n 50\n   ```\n\n3. Test configuration manually:\n   ```bash\n   sudo -u pysyslog /usr/bin/pysyslog -c /etc/pysyslog/main.ini --log-level DEBUG\n   ```\n\n#### Configuration errors\n- Ensure all referenced components are implemented (see [MISSING_COMPONENTS.md](MISSING_COMPONENTS.md))\n- Use `main.ini.example` for testing with available components\n- Check that component types match registered components in `components/registry.py`\n\n## Deployment on Ubuntu\n\n### Full System Installation\n\n1. **Prerequisites:**\n   ```bash\n   sudo apt-get update\n   sudo apt-get install -y python3 python3-pip python3-dev git\n   ```\n\n2. **Clone and Install:**\n   ```bash\n   git clone https://github.com/pysyslog/pysyslog-lfc.git\n   cd pysyslog-lfc\n   sudo ./install.sh\n   ```\n\n3. **Configure for Testing:**\n   ```bash\n   # Backup original config\n   sudo cp /etc/pysyslog/main.ini /etc/pysyslog/main.ini.original\n   \n   # Use working example config\n   sudo cp etc/pysyslog/main.ini.example /etc/pysyslog/main.ini\n   \n   # Restart service\n   sudo systemctl restart pysyslog\n   \n   # Check status\n   sudo systemctl status pysyslog\n   ```\n\n4. **Verify Installation:**\n   ```bash\n   # Check service is running\n   sudo systemctl status pysyslog\n   \n   # View logs\n   sudo journalctl -u pysyslog -f\n   \n   # Test the executable\n   /usr/bin/pysyslog --help\n   ```\n\n### Development/Testing Installation (No Sudo)\n\nFor development and testing without system-wide installation:\n\n```bash\n# 1. Clone repository\ngit clone https://github.com/pysyslog/pysyslog-lfc.git\ncd pysyslog-lfc\n\n# 2. Create virtual environment\npython3 -m venv venv\nsource venv/bin/activate\n\n# 3. Install in development mode\npip install -e .\n\n# 4. Run tests\npython3 test_example_config.py\n\n# 5. Run with example config\npython3 -m pysyslog -c etc/pysyslog/main.ini.example --log-level DEBUG\n```\n\n## Development\n\n### Project Structure\n\n```\npysyslog-lfc/\n├── bin/                    # Executable scripts\n├── docs/                   # Documentation\n│   └── configuration/      # Configuration docs\n├── etc/                    # Configuration files\n│   ├── pysyslog/\n│   │   ├── main.ini\n│   │   └── conf.d/\n│   ├── systemd/           # Linux service files\n│   ├── launchd/           # macOS service files\n│   └── windows/           # Windows service files\n├── lib/                    # Python package\n│   └── pysyslog/\n│       ├── __init__.py\n│       ├── main.py\n│       ├── config.py\n│       ├── flow.py\n│       ├── components.py\n│       ├── inputs/         # Input components\n|       ├── filters/        # Filter components\n│       ├── parsers/        # Parser components\n│       └── outputs/        # Output components\n├── install.sh             # Linux/macOS installation script\n├── install.bat            # Windows installation script\n├── requirements.txt       # Python dependencies\n└── setup.py              # Python package setup\n```\n\n### Adding New Components\n\n1. Create a new component file in the appropriate directory:\n   - `src/pysyslog/inputs/` for input components\n   - `src/pysyslog/filters/` for filter components\n   - `src/pysyslog/parsers/` for parser components\n   - `src/pysyslog/outputs/` for output components\n   - `src/pysyslog/formats/` for format components\n\n2. Implement the required interface (inherit from base classes in `components/base.py`)\n\n3. Register the component in `src/pysyslog/components/registry.py`:\n   ```python\n   BUILTIN_INPUTS = {\n       \"your_component\": \"pysyslog.inputs.your_component:YourComponentClass\",\n   }\n   ```\n\n4. Update the `__init__.py` in the component's directory to export the class\n\n5. Test your component:\n   ```bash\n   python3 test_example_config.py\n   ```\n\nSee [MISSING_COMPONENTS.md](MISSING_COMPONENTS.md) for a list of components that need to be implemented.\n\n## License\n\nMIT License - see LICENSE file for details. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpysyslog%2Fpysyslog-lfc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpysyslog%2Fpysyslog-lfc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpysyslog%2Fpysyslog-lfc/lists"}