{"id":20153458,"url":"https://github.com/nextflow-io/nf-weblog","last_synced_at":"2025-10-12T08:39:28.439Z","repository":{"id":182096791,"uuid":"667864656","full_name":"nextflow-io/nf-weblog","owner":"nextflow-io","description":"Implement weblog plugin for Nextflow","archived":false,"fork":false,"pushed_at":"2023-11-22T15:27:08.000Z","size":112,"stargazers_count":9,"open_issues_count":5,"forks_count":9,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-09T21:41:07.707Z","etag":null,"topics":["nextflow"],"latest_commit_sha":null,"homepage":"https://nextflow.io","language":"Groovy","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nextflow-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-18T13:25:39.000Z","updated_at":"2024-12-06T21:33:18.000Z","dependencies_parsed_at":"2023-11-09T00:21:46.218Z","dependency_job_id":"645ae430-6739-4cb1-a04f-cdd7c6eb9122","html_url":"https://github.com/nextflow-io/nf-weblog","commit_stats":null,"previous_names":["nextflow-io/nf-weblog"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/nextflow-io/nf-weblog","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nextflow-io%2Fnf-weblog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nextflow-io%2Fnf-weblog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nextflow-io%2Fnf-weblog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nextflow-io%2Fnf-weblog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nextflow-io","download_url":"https://codeload.github.com/nextflow-io/nf-weblog/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nextflow-io%2Fnf-weblog/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279010790,"owners_count":26084807,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-12T02:00:06.719Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["nextflow"],"created_at":"2024-11-13T23:19:07.436Z","updated_at":"2025-10-12T08:39:28.424Z","avatar_url":"https://github.com/nextflow-io.png","language":"Groovy","funding_links":[],"categories":[],"sub_categories":[],"readme":"# nf-weblog plugin \n   \nThis plugin allows Nextflow to send detailed workflow execution metadata and runtime statistics to a HTTP endpoint. \nTo enable this feature, use the `-with-weblog` as shown below:\n\n```bash\nnextflow run \u003cpipeline name\u003e -with-weblog [url]\n```\n\nOr enable it in the Nextflow configuration:\n\n```groovy\nweblog {\n  enabled = true\n  url = '...'\n}\n```\n\nWorkflow events are sent as HTTP POST requests to the given URL. The message consists of the following JSON structure:\n\n```json\n{\n  \"runName\": \"\u003crun name\u003e\",\n  \"runId\": \"\u003cuuid\u003e\",\n  \"event\": \"\u003cstarted|process_submitted|process_started|process_completed|error|completed\u003e\",\n  \"utcTime\": \"\u003cUTC timestamp\u003e\",\n  \"trace\": {  },\n  \"metadata\": {  }\n}\n```\n\nThe JSON object contains the following attributes:\n\n- `runName`: The workflow execution run name.\n\n- `runId`: The workflow execution unique ID.\n\n- `event`: The workflow execution event. One of `started`, `process_submitted`, `process_started`, `process_completed`, `error`, `completed`.\n\n- `utcTime`: The UTC timestamp in ISO 8601 format.\n\n- `trace`: The task runtime information as described in the [Trace report](https://nextflow.io/docs/latest/tracing.html#trace-report) documentation. The set of included fields is determined by the `trace.fields` setting in the Nextflow configuration file. See the [Trace configuration](https://nextflow.io/docs/latest/config.html#scope-trace) docs to learn more.\n\n  *Included only for the following events: `process_submitted`, `process_started`, `process_completed`, `error`*\n\n- `metadata`: The workflow metadata including the [config manifest](https://nextflow.io/docs/latest/config.html#scope-manifest). For a list of all fields, have a look at the bottom message examples.\n\n  *Included only for the following events: `started`, `completed`*\n\n## Configuration\n\nThe following configuration options are available:\n\n`weblog.enabled`\n\nIf `true` it will send HTTP POST requests to a given url.\n\n`weblog.url`\n\nThe url where to send HTTP POST requests (default: `http://localhost`).\n\n`weblog.basicToken`\n\nA Basic authentication token to append to each HTTP request. It should be the user ID and password joined by a single colon, for example `USER:PASSWORD`.\n\n## Examples\n\n### Example `started` event\n\nWhen a workflow execution is started, a message like the following is posted to the specified end-point. Be aware that the properties in the parameter scope will look different for your workflow. Here is an example output from the `nf-core/hlatyping` pipeline with the weblog feature enabled:\n\n```json\n{\n  \"runName\": \"friendly_pesquet\",\n  \"runId\": \"170aa09c-105f-49d0-99b4-8eb6a146e4a7\",\n  \"event\": \"started\",\n  \"utcTime\": \"2018-10-07T11:42:08Z\",\n  \"metadata\": {\n    \"params\": {\n      \"container\": \"nfcore/hlatyping:1.1.4\",\n      \"help\": false,\n      \"outdir\": \"results\",\n      \"bam\": true,\n      \"singleEnd\": false,\n      \"single-end\": false,\n      \"reads\": \"data/test*{1,2}.fq.gz\",\n      \"seqtype\": \"dna\",\n      \"solver\": \"glpk\",\n      \"igenomes_base\": \"./iGenomes\",\n      \"multiqc_config\": \"/Users/sven1103/.nextflow/assets/nf-core/hlatyping/conf/multiqc_config.yaml\",\n      \"clusterOptions\": false,\n      \"cluster-options\": false,\n      \"enumerations\": 1,\n      \"beta\": 0.009,\n      \"prefix\": \"hla_run\",\n      \"base_index\": \"/Users/sven1103/.nextflow/assets/nf-core/hlatyping/data/indices/yara/hla_reference_\",\n      \"index\": \"/Users/sven1103/.nextflow/assets/nf-core/hlatyping/data/indices/yara/hla_reference_dna\",\n      \"custom_config_version\": \"master\",\n      \"custom_config_base\": \"https://raw.githubusercontent.com/nf-core/configs/master\"\n    },\n    \"workflow\": {\n      \"start\": \"2019-03-25T12:09:52Z\",\n      \"projectDir\": \"/Users/sven1103/.nextflow/assets/nf-core/hlatyping\",\n      \"manifest\": {\n        \"nextflowVersion\": \"\u003e=18.10.1\",\n        \"defaultBranch\": \"master\",\n        \"version\": \"1.1.4\",\n        \"homePage\": \"https://github.com/nf-core/hlatyping\",\n        \"gitmodules\": null,\n        \"description\": \"Precision HLA typing from next-generation sequencing data.\",\n        \"name\": \"nf-core/hlatyping\",\n        \"mainScript\": \"main.nf\",\n        \"author\": null\n      },\n      \"complete\": null,\n      \"profile\": \"docker,test\",\n      \"homeDir\": \"/Users/sven1103\",\n      \"workDir\": \"/Users/sven1103/git/nextflow/work\",\n      \"container\": \"nfcore/hlatyping:1.1.4\",\n      \"commitId\": \"4bcced898ee23600bd8c249ff085f8f88db90e7c\",\n      \"errorMessage\": null,\n      \"repository\": \"https://github.com/nf-core/hlatyping.git\",\n      \"containerEngine\": \"docker\",\n      \"scriptFile\": \"/Users/sven1103/.nextflow/assets/nf-core/hlatyping/main.nf\",\n      \"userName\": \"sven1103\",\n      \"launchDir\": \"/Users/sven1103/git/nextflow\",\n      \"runName\": \"shrivelled_cantor\",\n      \"configFiles\": [\n        \"/Users/sven1103/.nextflow/assets/nf-core/hlatyping/nextflow.config\"\n      ],\n      \"sessionId\": \"7f344978-999c-480d-8439-741bc7520f6a\",\n      \"errorReport\": null,\n      \"scriptId\": \"2902f5aa7f297f2dccd6baebac7730a2\",\n      \"revision\": \"master\",\n      \"exitStatus\": null,\n      \"commandLine\": \"./launch.sh run nf-core/hlatyping -profile docker,test -with-weblog 'http://localhost:4567'\",\n      \"nextflow\": {\n        \"version\": \"19.03.0-edge\",\n        \"build\": 5137,\n        \"timestamp\": \"2019-03-28T14:46:55Z\"\n      }\n    },\n    \"stats\": {\n      \"computeTimeFmt\": \"(a few seconds)\",\n      \"cachedCount\": 0,\n      \"cachedDuration\": 0,\n      \"failedDuration\": 0,\n      \"succeedDuration\": 0,\n      \"failedCount\": 0,\n      \"cachedPct\": 0.0,\n      \"cachedCountFmt\": \"0\",\n      \"succeedCountFmt\": \"0\",\n      \"failedPct\": 0.0,\n      \"failedCountFmt\": \"0\",\n      \"ignoredCountFmt\": \"0\",\n      \"ignoredCount\": 0,\n      \"succeedPct\": 0.0,\n      \"succeedCount\": 0,\n      \"ignoredPct\": 0.0\n    },\n    \"resume\": false,\n    \"success\": false,\n    \"scriptName\": \"main.nf\",\n    \"duration\": null\n  }\n}\n```\n\n### Example `completed` event\n\nWhen a task is completed, a message like the following is posted to the specified end-point:\n\n```json\n{\n  \"runName\": \"friendly_pesquet\",\n  \"runId\": \"170aa09c-105f-49d0-99b4-8eb6a146e4a7\",\n  \"event\": \"process_completed\",\n  \"utcTime\": \"2018-10-07T11:45:30Z\",\n  \"trace\": {\n    \"task_id\": 2,\n    \"status\": \"COMPLETED\",\n    \"hash\": \"a1/0024fd\",\n    \"name\": \"make_ot_config\",\n    \"exit\": 0,\n    \"submit\": 1538912529498,\n    \"start\": 1538912529629,\n    \"process\": \"make_ot_config\",\n    \"tag\": null,\n    \"module\": [\n\n    ],\n    \"container\": \"nfcore/hlatyping:1.1.1\",\n    \"attempt\": 1,\n    \"script\": \"\\n    configbuilder --max-cpus 2 --solver glpk \u003e config.ini\\n    \",\n    \"scratch\": null,\n    \"workdir\": \"/home/sven1103/git/hlatyping-workflow/work/a1/0024fd028375e2b601aaed44d112e3\",\n    \"queue\": null,\n    \"cpus\": 1,\n    \"memory\": 7516192768,\n    \"disk\": null,\n    \"time\": 7200000,\n    \"env\": \"PATH=/home/sven1103/git/hlatyping-workflow/bin:$PATH\\n\",\n    \"error_action\": null,\n    \"complete\": 1538912730599,\n    \"duration\": 201101,\n    \"realtime\": 69,\n    \"%cpu\": 0.0,\n    \"%mem\": 0.1,\n    \"vmem\": 54259712,\n    \"rss\": 10469376,\n    \"peak_vmem\": 20185088,\n    \"peak_rss\": 574972928,\n    \"rchar\": 7597,\n    \"wchar\": 162,\n    \"syscr\": 16,\n    \"syscw\": 4083712,\n    \"read_bytes\": 4096,\n    \"write_bytes\": 0,\n    \"native_id\": 27185\n  }\n}\n```\n\n\n## Testing and debugging\n\nTo build and test the plugin during development, configure a local Nextflow build with the following steps:\n\n1. Clone the Nextflow repository in your computer into a sibling directory:\n    ```bash\n    git clone --depth 1 https://github.com/nextflow-io/nextflow ../nextflow\n    ```\n  \n2. Configure the plugin build to use the local Nextflow code:\n    ```bash\n    echo \"includeBuild('../nextflow')\" \u003e\u003e settings.gradle\n    ```\n  \n   (Make sure to not add it more than once!)\n\n3. Compile the plugin alongside the Nextflow code:\n    ```bash\n    make compile\n    ```\n\n4. Run Nextflow with the plugin, using `./launch.sh` as a drop-in replacement for the `nextflow` command, and adding the option `-plugins nf-weblog` to load the plugin:\n    ```bash\n    ./launch.sh run nextflow-io/hello -plugins nf-weblog\n    ```\n\n### Test endpoint\n\nThe `echo.py` script can be used to deploy a local server that logs POST requests, which is useful for testing the plugin:\n\n```bash\n./echo.py\n```\n\n## Testing without Nextflow build\n\nThe plugin can be tested without using a local Nextflow build using the following steps:\n\n1. Build the plugin: `make buildPlugins`\n2. Copy `build/plugins/\u003cyour-plugin\u003e` to `$HOME/.nextflow/plugins`\n3. Create a pipeline that uses your plugin and run it: `nextflow run ./my-pipeline-script.nf`\n\n## Package, upload, and publish\n\nThe project should be hosted in a GitHub repository whose name matches the name of the plugin, that is the name of the directory in the `plugins` folder (e.g. `nf-weblog`).\n\nFollow these steps to package, upload and publish the plugin:\n\n1. Create a file named `gradle.properties` in the project root containing the following attributes (this file should not be committed to Git):\n\n   * `github_organization`: the GitHub organisation where the plugin repository is hosted.\n   * `github_username`: The GitHub username granting access to the plugin repository.\n   * `github_access_token`: The GitHub access token required to upload and commit changes to the plugin repository.\n   * `github_commit_email`: The email address associated with your GitHub account.\n\n2. Use the following command to package and create a release for your plugin on GitHub:\n    ```bash\n    ./gradlew :plugins:nf-weblog:upload\n    ```\n\n3. Create a pull request against [nextflow-io/plugins](https://github.com/nextflow-io/plugins/blob/main/plugins.json) to make the plugin accessible to Nextflow.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnextflow-io%2Fnf-weblog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnextflow-io%2Fnf-weblog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnextflow-io%2Fnf-weblog/lists"}