{"id":29193965,"url":"https://github.com/snakemake/snakemake-interface-logger-plugins","last_synced_at":"2025-07-02T03:37:20.313Z","repository":{"id":260960556,"uuid":"882811846","full_name":"snakemake/snakemake-interface-logger-plugins","owner":"snakemake","description":null,"archived":false,"fork":false,"pushed_at":"2025-06-05T15:38:17.000Z","size":71,"stargazers_count":3,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-05T16:39:06.217Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/snakemake.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-11-03T20:24:23.000Z","updated_at":"2025-06-05T15:37:55.000Z","dependencies_parsed_at":"2025-03-21T14:16:23.330Z","dependency_job_id":null,"html_url":"https://github.com/snakemake/snakemake-interface-logger-plugins","commit_stats":null,"previous_names":["cademirch/snakemake-interface-logger-plugins","snakemake/snakemake-interface-logger-plugins"],"tags_count":12,"template":false,"template_full_name":null,"purl":"pkg:github/snakemake/snakemake-interface-logger-plugins","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snakemake%2Fsnakemake-interface-logger-plugins","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snakemake%2Fsnakemake-interface-logger-plugins/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snakemake%2Fsnakemake-interface-logger-plugins/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snakemake%2Fsnakemake-interface-logger-plugins/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/snakemake","download_url":"https://codeload.github.com/snakemake/snakemake-interface-logger-plugins/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snakemake%2Fsnakemake-interface-logger-plugins/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263071309,"owners_count":23409255,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-02T03:37:19.547Z","updated_at":"2025-07-02T03:37:20.275Z","avatar_url":"https://github.com/snakemake.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Snakemake Logger Plugin Interface\n\nThis package provides a stable interface for interactions between Snakemake and its logger plugins.\n\nPlugins should implement the following skeleton to comply with this interface.\nIt is recommended to use Snakemake's poetry plugin to set up this skeleton (and automated testing) within a python package, see https://github.com/snakemake/poetry-snakemake-plugin.\n\n## Overview\n\n```python\nfrom snakemake_interface_logger_plugins.base import LogHandlerBase\nfrom snakemake_interface_logger_plugins.settings import LogHandlerSettingsBase\n\nfrom dataclasses import dataclass, field\nfrom typing import Optional\n\n\n@dataclass\nclass LogHandlerSettings(LogHandlerSettingsBase):\n    myparam: Optional[int] = field(\n        default=None,\n        metadata={\n            \"help\": \"Some help text\",\n            # Optionally request that setting is also available for specification\n            # via an environment variable. The variable will be named automatically as\n            # SNAKEMAKE_LOGGER_\u003cLOGGER-name\u003e_\u003cparam-name\u003e, all upper case.\n            # This mechanism should ONLY be used for passwords and usernames.\n            # For other items, we rather recommend to let people use a profile\n            # for setting defaults\n            # (https://snakemake.readthedocs.io/en/stable/executing/cli.html#profiles).\n            \"env_var\": False,\n            # Optionally specify a function that parses the value given by the user.\n            # This is useful to create complex types from the user input.\n            \"parse_func\": ...,\n            # If a parse_func is specified, you also have to specify an unparse_func\n            # that converts the parsed value back to a string.\n            \"unparse_func\": ...,\n            # Optionally specify that setting is required when the LOGGER is in use.\n            \"required\": True,\n            # Optionally specify multiple args with \"nargs\": \"+\"\n        },\n    )\n\n\nclass LogHandler(LogHandlerBase):\n    def __post_init__(self) -\u003e None:\n        # initialize additional attributes\n        # Do not overwrite the __init__ method as this is kept in control of the base\n        # class in order to simplify the update process.\n        # See https://github.com/snakemake/snakemake-interface-logger-plugins/blob/main/src/snakemake_interface_logger_plugins/base.py # noqa: E501\n        # for attributes of the base class.\n        # In particular, the settings of above LogHandlerSettings class are accessible via\n        # self.settings.\n        # You also have access to self.common_settings here, which are logging settings supplied by the caller in the form of OutputSettingsLoggerInterface. # noqa: E501\n        # See https://github.com/snakemake/snakemake-interface-logger-plugins/blob/main/src/snakemake_interface_logger_plugins/settings.py for more details # noqa: E501\n        \n        # access settings attributes\n        self.settings \n        self.common_settings\n\n    # Here you can override logging.Handler methods to customize logging behavior.\n    # For example, you can override the emit() method to control how log records\n    # are processed and output. See the Python logging documentation for details:\n    # https://docs.python.org/3/library/logging.html#handler-objects\n\n    # LogRecords from Snakemake carry contextual information in the record's attributes\n    # Of particular interest is the 'event' attribute, which indicates the type of log information contained\n    # See https://github.com/snakemake/snakemake-interface-logger-plugins/blob/2ab84cb31f0b92cf0b7ee3026e15d1209729d197/src/snakemake_interface_logger_plugins/common.py#L33 # noqa: E501\n    # For examples on parsing LogRecords, see https://github.com/cademirch/snakemake-logger-plugin-snkmt/blob/main/src/snakemake_logger_plugin_snkmt/parsers.py # noqa: E501\n\n    @property\n    def writes_to_stream(self) -\u003e bool:\n        # Whether this plugin writes to stderr/stdout.\n        # If your plugin writes to stderr/stdout, return\n        # true so that Snakemake disables its stderr logging.\n        ...\n\n    @property\n    def writes_to_file(self) -\u003e bool:\n        # Whether this plugin writes to a file.\n        # If your plugin writes log output to a file, return\n        # true so that Snakemake can report your logfile path at workflow end.\n        ...\n\n    @property\n    def has_filter(self) -\u003e bool:\n        # Whether this plugin attaches its own filter.\n        # Return true if your plugin provides custom log filtering logic.\n        # If false is returned, Snakemake's DefaultFilter will be attached see: https://github.com/snakemake/snakemake/blob/960f6a89eaa31da6014e810dfcf08f635ac03a6e/src/snakemake/logging.py#L372 # noqa: E501\n        # See https://docs.python.org/3/library/logging.html#filter-objects for info on how to define and attach a Filter\n        ...\n\n    @property\n    def has_formatter(self) -\u003e bool:\n        # Whether this plugin attaches its own formatter.\n        # Return true if your plugin provides custom log formatting logic.\n        # If false is returned, Snakemake's Defaultformatter will be attached see: https://github.com/snakemake/snakemake/blob/960f6a89eaa31da6014e810dfcf08f635ac03a6e/src/snakemake/logging.py#L132 # noqa: E501\n        # See https://docs.python.org/3/library/logging.html#formatter-objects for info on how to define and attach a Formatter\n        ...\n\n    @property\n    def needs_rulegraph(self) -\u003e bool:\n        # Whether this plugin requires the DAG rulegraph.\n        # Return true if your plugin needs access to the workflow's\n        # directed acyclic graph for logging purposes.\n        ...\n\n```\n\n## Migrating from `--log-handler-script`\n\nTo migrate a log handler script to a logger plugin, follow these steps:\n\n### 1. Understand the differences\n\n**Old approach (`--log-handler-script`):**\n- Single function that receives message dictionaries\n- Direct access to message fields like `msg['level']`, `msg['name']`, `msg['output']`\n- Manual file handling and stderr writing\n\n**New approach (Logger Plugin):**\n- Class-based handler inheriting from `LogHandlerBase`\n- Integration with Python's logging framework\n- Access to structured `LogRecord` objects with event context\n\n### 2. Convert your script function to a plugin class\n\n**Example old script:**\n```python\ndef log_handler(msg):\n    if msg['level'] == \"job_error\" and msg['name'] in ['rule1', 'rule2']:\n        logfile = msg['log'][0]\n        sys.stderr.write(f\"Error in {msg['output'][0]}. See {logfile}\\n\")\n        with open(logfile) as f:\n            for line in f:\n                sys.stderr.write(f\"    {line}\")\n```\n\n**Converted to plugin:**\n```python\nfrom snakemake_interface_logger_plugins.base import LogHandlerBase\nfrom snakemake_interface_logger_plugins.common import LogEvent\nfrom rich.console import Console\nimport logging\n\nclass LogHandler(LogHandlerBase):\n    def __post_init__(self) -\u003e None:\n        super().__post_init__()\n        self.console = Console()\n    \n    def emit(self, record):\n        # Access event type from record\n        if hasattr(record, 'event') and record.event == LogEvent.JOB_ERROR:\n            # Access job information from record attributes\n            if hasattr(record, 'name') and record.name in ['rule1', 'rule2']:\n                logfile = record.log[0] if hasattr(record, 'log') else None\n                output = record.output[0] if hasattr(record, 'output') else \"unknown\"\n                \n                # Use rich console for pretty printing\n                self.console.print(f\"[red]Error in {output}. See {logfile}[/red]\")\n                if logfile:\n                    try:\n                        with open(logfile) as f:\n                            for line in f:\n                                self.console.print(f\"    {line.rstrip()}\", style=\"dim\")\n                    except FileNotFoundError:\n                        self.console.print(f\"    Log file {logfile} not found\", style=\"yellow\")\n\n    @property\n    def writes_to_stream(self) -\u003e bool:\n        return True # we're using rich in this plugin to pretty print our logs\n\n    @property\n    def writes_to_file(self) -\u003e bool:\n        return False  # we're not writing to a log file\n\n    @property\n    def has_filter(self) -\u003e bool:\n        return True  # we're doing our own log filtering\n\n    @property\n    def has_formatter(self) -\u003e bool:\n        return True  # we format our own output\n\n    @property\n    def needs_rulegraph(self) -\u003e bool:\n        return False # we're not using the rulegraph\n```\n\n### 3. Key migration points\n\n1. **Message access:** Replace `msg['field']` with `record.field` or `getattr(record, 'field', default)`\n\n2. **Event filtering:** Replace `msg['level'] == \"job_error\"` with `record.event == LogEvent.JOB_ERROR`\n\n3. **Output method:** Replace direct stderr/stdout calls with your plugin's output handling in the `emit()` method\n\n4. **Error handling:** Add proper exception handling for file operations\n\n5. **Property configuration:** Set the abstract properties to inform Snakemake about your handler's behavior\n\n## Available Log Events\n\nThe `LogEvent` enum defines particularly important Snakemake events such as workflow starting, job submission, job failure, etc. Below are the available events and the fields you can typically expect in `LogRecord` objects for each event type. **Note: These field lists are guidelines only and may change between versions. Always use defensive programming practices like `getattr()` with defaults or `hasattr()` checks when accessing fields.**\n\n### Event Types and Typical Available Fields\n\n**`LogEvent.ERROR`**\n- `exception: Optional[str]` - Exception type\n- `location: Optional[str]` - Location where error occurred\n- `rule: Optional[str]` - Rule name associated with error\n- `traceback: Optional[str]` - Full traceback\n- `file: Optional[str]` - File where error occurred\n- `line: Optional[str]` - Line number where error occurred\n\n**`LogEvent.WORKFLOW_STARTED`**\n- `workflow_id: uuid.UUID` - Unique workflow identifier\n- `snakefile: Optional[str]` - Path to the Snakefile\n\n**`LogEvent.JOB_INFO`**\n- `jobid: int` - Job identifier\n- `rule_name: str` - Name of the rule\n- `threads: int` - Number of threads allocated\n- `input: Optional[List[str]]` - Input files\n- `output: Optional[List[str]]` - Output files\n- `log: Optional[List[str]]` - Log files\n- `benchmark: Optional[List[str]]` - Benchmark files\n- `rule_msg: Optional[str]` - Rule message\n- `wildcards: Optional[Dict[str, Any]]` - Wildcard values\n- `reason: Optional[str]` - Reason for job execution\n- `shellcmd: Optional[str]` - Shell command to execute\n- `priority: Optional[int]` - Job priority\n- `resources: Optional[Dict[str, Any]]` - Resource requirements\n\n**`LogEvent.JOB_STARTED`**\n- `job_ids: List[int]` - List of job IDs that started\n\n**`LogEvent.JOB_FINISHED`**\n- `job_id: int` - ID of the finished job\n\n**`LogEvent.SHELLCMD`**\n- `jobid: int` - Job identifier\n- `shellcmd: Optional[str]` - Shell command being executed\n- `rule_name: Optional[str]` - Name of the rule\n\n**`LogEvent.JOB_ERROR`**\n- `jobid: int` - ID of the job that failed\n\n**`LogEvent.GROUP_INFO`**\n- `group_id: int` - Group identifier\n- `jobs: List[Any]` - Jobs in the group\n\n**`LogEvent.GROUP_ERROR`**\n- `groupid: int` - Group identifier\n- `aux_logs: List[Any]` - Auxiliary log information\n- `job_error_info: Dict[str, Any]` - Job error details\n\n**`LogEvent.RESOURCES_INFO`**\n- `nodes: Optional[List[str]]` - Available nodes\n- `cores: Optional[int]` - Available cores\n- `provided_resources: Optional[Dict[str, Any]]` - Provided resources\n\n**`LogEvent.DEBUG_DAG`**\n- `status: Optional[str]` - DAG status\n- `job: Optional[Any]` - Job information\n- `file: Optional[str]` - Related file\n- `exception: Optional[str]` - Exception information\n\n**`LogEvent.PROGRESS`**\n- `done: int` - Number of completed jobs\n- `total: int` - Total number of jobs\n\n**`LogEvent.RULEGRAPH`**\n- `rulegraph: Dict[str, Any]` - Rule graph data structure\n\n**`LogEvent.RUN_INFO`**\n- `per_rule_job_counts: Dict[str, int]` - Job count per rule\n- `total_job_count: int` - Total number of jobs\n\n### Accessing Event Fields\n\nYou can filter for specific events and access their fields in your `emit()` method:\n\n```python\ndef emit(self, record):\n    if hasattr(record, 'event'):\n        if record.event == LogEvent.JOB_ERROR:\n            # Access job error fields\n            jobid = getattr(record, 'jobid', 0)\n            # Handle job errors\n            pass\n        elif record.event == LogEvent.JOB_FINISHED:\n            # Access job completion fields\n            job_id = getattr(record, 'job_id', 0)\n            # Handle job completion\n            pass\n        elif record.event == LogEvent.PROGRESS:\n            # Access progress fields\n            done = getattr(record, 'done', 0)\n            total = getattr(record, 'total', 0)\n            # Handle progress updates\n            pass\n```\n\nAlways use `getattr(record, 'field_name', default_value)` or check with `hasattr(record, 'field_name')` before accessing fields, as not all fields may be present in every record.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnakemake%2Fsnakemake-interface-logger-plugins","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsnakemake%2Fsnakemake-interface-logger-plugins","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnakemake%2Fsnakemake-interface-logger-plugins/lists"}