{"id":26256471,"url":"https://github.com/netcracker/qubership-graylog-obfuscation-plugin","last_synced_at":"2025-12-28T00:36:06.395Z","repository":{"id":282188110,"uuid":"905766114","full_name":"Netcracker/qubership-graylog-obfuscation-plugin","owner":"Netcracker","description":"Graylog plugin (\u003e=3.x) to obfuscate logs in Graylog","archived":false,"fork":false,"pushed_at":"2025-01-29T06:55:07.000Z","size":135,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-13T08:37:16.696Z","etag":null,"topics":["logging","observability","plugin"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Netcracker.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE-OF-CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-19T13:32:26.000Z","updated_at":"2025-01-29T13:42:30.000Z","dependencies_parsed_at":"2025-03-13T08:47:23.359Z","dependency_job_id":null,"html_url":"https://github.com/Netcracker/qubership-graylog-obfuscation-plugin","commit_stats":null,"previous_names":["netcracker/qubership-graylog-obfuscation-plugin"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Netcracker%2Fqubership-graylog-obfuscation-plugin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Netcracker%2Fqubership-graylog-obfuscation-plugin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Netcracker%2Fqubership-graylog-obfuscation-plugin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Netcracker%2Fqubership-graylog-obfuscation-plugin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Netcracker","download_url":"https://codeload.github.com/Netcracker/qubership-graylog-obfuscation-plugin/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243475364,"owners_count":20296714,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["logging","observability","plugin"],"created_at":"2025-03-13T20:18:12.818Z","updated_at":"2025-12-28T00:36:06.387Z","avatar_url":"https://github.com/Netcracker.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Graylog Obfuscation Plugin\n\nA Graylog plugin for automatic analysis and obfuscation of sensitive data in log messages using configurable regular expressions.\n\n[![Release](https://img.shields.io/github/v/release/Netcracker/qubership-graylog-obfuscation-plugin?style=flat-square)](https://github.com/Netcracker/qubership-graylog-obfuscation-plugin/releases)\n[![Docs](https://img.shields.io/website?url=https%3A//github.com/Netcracker/qubership-graylog-obfuscation-plugin/tree/main/docs\u0026label=docs\u0026style=flat-square)](https://github.com/Netcracker/qubership-graylog-obfuscation-plugin/tree/main/docs)\n[![Super Linter](https://github.com/Netcracker/qubership-graylog-obfuscation-plugin/actions/workflows/super-linter.yaml/badge.svg)](https://github.com/Netcracker/qubership-graylog-obfuscation-plugin/actions/workflows/super-linter.yaml)\n\n\u003c!-- toc --\u003e\n\n## Features\n\n- **Automatic sensitive data detection** using configurable regular expressions\n- **Flexible obfuscation strategies** with pluggable text replacers (e.g., static star replacer)\n- **Stream-based filtering** - apply obfuscation to specific Graylog streams\n- **Field-level granularity** - target specific message fields for obfuscation\n- **White-listing support** - exclude certain patterns from obfuscation\n- **REST API configuration** - manage obfuscation rules via HTTP endpoints\n- **MongoDB integration** - persistent configuration storage\n- **Web interface** - React-based configuration management UI\n- **Real-time processing** - integrates with Graylog's message pipeline\n\n## Requirements\n\n### Runtime Requirements\n- **Graylog Server**: Version 3.3.0 or higher\n- **Java**: OpenJDK 8 or higher\n- **MongoDB**: For configuration storage\n\n### Development Requirements\n- **Java**: OpenJDK 8 or higher\n- **Maven**: 3.6.0 or higher\n- **Node.js**: 12.16.1 or higher\n- **Yarn**: 1.22.0 or higher\n- **Graylog Sources**: Required for building (see development guide)\n\n## Installation\n\n1. **Download the plugin JAR** from the [releases page](https://github.com/Netcracker/qubership-graylog-obfuscation-plugin/releases)\n\n2. **Copy to Graylog plugins directory**:\n   ```bash\n   sudo cp graylog-obfuscation-plugin-*.jar /var/lib/graylog/plugins/\n   ```\n\n3. **Set proper permissions**:\n   ```bash\n   sudo chown graylog:graylog /var/lib/graylog/plugins/graylog-obfuscation-plugin-*.jar\n   ```\n\n4. **Restart Graylog server**:\n   ```bash\n   sudo systemctl restart graylog-server\n   ```\n\n5. **Verify installation** by checking the Graylog web interface under System \u003e Plugins\n\n## Usage\n\n### Basic Configuration\n\n1. **Access the plugin configuration** via Graylog web interface or REST API\n2. **Enable obfuscation** by setting `is-obfuscation-enabled: true`\n3. **Configure streams** to specify which log streams to process\n4. **Define field names** to target specific message fields\n5. **Add sensitive data patterns** using regular expressions\n\n### Example Configuration\n\n```json\n{\n  \"is-obfuscation-enabled\": true,\n  \"text-replacer\": \"Static Star Replacer\",\n  \"stream-titles\": [\"security-logs\", \"application-logs\"],\n  \"field-names\": [\"message\", \"user_data\"],\n  \"sensitive-regular-expressions\": [\n    {\n      \"id\": 1,\n      \"name\": \"Credit Card\",\n      \"pattern\": \"\\\\b\\\\d{4}[\\\\s-]?\\\\d{4}[\\\\s-]?\\\\d{4}[\\\\s-]?\\\\d{4}\\\\b\",\n      \"importance\": 1\n    }\n  ],\n  \"white-regular-expressions\": [\n    {\n      \"id\": 1,\n      \"name\": \"Test Credit Card\",\n      \"pattern\": \"4111-1111-1111-1111\"\n    }\n  ]\n}\n```\n\n## Configuration\n\nThe plugin can be configured through:\n\n### REST API Endpoints\n\n- **GET** `/api/plugins/com.netcracker.graylog2.plugin/obfuscation/configuration` - Get current configuration\n- **POST** `/api/plugins/com.netcracker.graylog2.plugin/obfuscation/configuration` - Update configuration\n- **PUT** `/api/plugins/com.netcracker.graylog2.plugin/obfuscation/configuration/sync` - Sync with default\n- **PUT** `/api/plugins/com.netcracker.graylog2.plugin/obfuscation/configuration/reset` - Reset to default\n- **POST** `/api/plugins/com.netcracker.graylog2.plugin/obfuscation` - Test obfuscation\n\n### Configuration Parameters\n\n- `is-obfuscation-enabled`: Enable/disable obfuscation\n- `text-replacer`: Replacement strategy (currently \"Static Star Replacer\")\n- `stream-titles`: Array of stream names to process\n- `field-names`: Array of message field names to obfuscate\n- `sensitive-regular-expressions`: Patterns for sensitive data detection\n- `white-regular-expressions`: Patterns to exclude from obfuscation\n\nFor detailed configuration options, see the [Configuration Guide](docs/configuration_guide.md).\n\n## API Reference\n\n### OpenAPI Specification\nComplete API documentation is available in [docs/api/openapi.yaml](docs/api/openapi.yaml).\n\n### Postman Collection\nImport the [Postman collection](docs/postman/Graylog-Obfuscation-Plugin.postman_collection.json) for easy API testing.\n\n### Documentation\n- [Installation Guide](docs/installation.md)\n- [Configuration Guide](docs/configuration_guide.md)\n- [Development Guide](docs/development.md)\n- [Sensitive Data Analysis](docs/sensitive-data-analysis.md)\n\n## Architecture\n\n### System Overview\n\n```mermaid\nflowchart LR\n    A([Log Sources]) --\u003e B[Graylog Server]\n    B --\u003e C{Obfuscation\u003cbr/\u003ePlugin}\n    C --\u003e D[Obfuscation\u003cbr/\u003eEngine]\n    D --\u003e E[RegEx\u003cbr/\u003eMatcher]\n    D --\u003e F[Text\u003cbr/\u003eReplacer]\n    C --\u003e G[(MongoDB\u003cbr/\u003eConfig)]\n    C --\u003e H[Web UI]\n    I[REST API] --\u003e C\n    C --\u003e J[Processed\u003cbr/\u003eLogs]\n```\n\nThe plugin integrates with Graylog's message processing pipeline through the MessageProcessor interface. It consists of two main components:\n\n1. **Obfuscation Engine** - Core logic for pattern matching and text replacement\n2. **Graylog Integration** - Plugin interface, REST API, and web UI for configuration management\n\nConfiguration is stored in MongoDB and can be managed through either the web interface or REST API endpoints.\n\n## Testing\n\n### Running Tests\n\n```bash\n# Run all tests\nmvn test\n\n# Run with coverage\nmvn test jacoco:report\n\n# Run specific test class\nmvn test -Dtest=ObfuscationEngineTest\n```\n\n### Development Linting\n\nBefore committing, run the Super Linter locally:\n\n```bash\ndocker run \\\n  -e RUN_LOCAL=true \\\n  -e DEFAULT_BRANCH=$(git rev-parse --abbrev-ref HEAD) \\\n  --env-file .github/super-linter.env \\\n  -v ${PWD}:/tmp/lint \\\n  --rm \\\n  ghcr.io/super-linter/super-linter:slim-$(sed -nE 's#.*uses:\\s+super-linter/super-linter/slim@([^\\s]+).*#\\1#p' .github/workflows/super-linter.yaml)\n```\n\nFor detailed development setup, see the [Development Guide](docs/development.md).\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch: `git checkout -b feature/amazing-feature`\n3. Make your changes following the coding standards\n4. Run tests and linting: `mvn test \u0026\u0026 docker run ...` (see Testing section)\n5. Commit your changes: `git commit -m 'Add amazing feature'`\n6. Push to the branch: `git push origin feature/amazing-feature`\n7. Open a Pull Request\n\nPlease read [CONTRIBUTING.md](CONTRIBUTING.md) for details on our code of conduct and the process for submitting pull requests.\n\n## License\n\nThis project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetcracker%2Fqubership-graylog-obfuscation-plugin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnetcracker%2Fqubership-graylog-obfuscation-plugin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetcracker%2Fqubership-graylog-obfuscation-plugin/lists"}