{"id":13613821,"url":"https://github.com/commsor/titanoboa-tasklets","last_synced_at":"2025-04-13T18:31:39.176Z","repository":{"id":110789442,"uuid":"168300537","full_name":"commsor/titanoboa-tasklets","owner":"commsor","description":"ready-made workflow steps for titanoboa","archived":false,"fork":false,"pushed_at":"2022-01-05T07:38:27.000Z","size":89,"stargazers_count":15,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-11-07T22:41:41.778Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://titanoboa.io","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"unlicense","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/commsor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-01-30T07:41:05.000Z","updated_at":"2022-03-04T03:31:32.000Z","dependencies_parsed_at":null,"dependency_job_id":"0cc554a0-7824-43ee-9f27-33ef1cf44503","html_url":"https://github.com/commsor/titanoboa-tasklets","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/commsor%2Ftitanoboa-tasklets","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/commsor%2Ftitanoboa-tasklets/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/commsor%2Ftitanoboa-tasklets/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/commsor%2Ftitanoboa-tasklets/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/commsor","download_url":"https://codeload.github.com/commsor/titanoboa-tasklets/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248760380,"owners_count":21157347,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T20:00:53.926Z","updated_at":"2025-04-13T18:31:39.161Z","avatar_url":"https://github.com/commsor.png","language":"Clojure","funding_links":[],"categories":["Clojure"],"sub_categories":[],"readme":"\n\n![alt Logo](https://s3.eu-central-1.amazonaws.com/www.titanoboa.io/tb-logo-dark-nosubtitle.svg)\n# Titanoboa Step Functions\nThis repository contains sample ready-made steps for [titanoboa](https://titanoboa.io) (github repository is [here](https://github.com/mikub/titanoboa) ):\n\n**AWS** \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/aws.svg\"/\u003e\n\n* [AWS EC2](#aws-ec2-) \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/aws-ec2.svg\"/\u003e\n\n* [AWS S3](#aws-s3-) \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/aws-s3.svg\"/\u003e \n\n* [AWS SES](#aws-ses-) \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/aws-ses.svg\"/\u003e \n\n* [AWS SNS](#aws-sns-) \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/aws-sns.svg\"/\u003e \n\n* [AWS SQS](#aws-sqs-) \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/aws-sqs.svg\"/\u003e \n\n🧬 **Bioinformatics** :microscope:\n\n* 🧬 [K-mer Count](#-k-mer-count)\n\n[Http Client](#http-client-) \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/http-client.svg\"/\u003e\n\n[JDBC Client](#jdbc-client-) \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/jdbc.svg\"/\u003e\n\n[Kafka Producer \u0026 Consumer](#kafka-producer--consumer-) \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/kafka.svg\"/\u003e\n\n[PDF Generation](#pdf-) \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/pdf-generation.svg\"/\u003e\n\n[SFTP Client](#ssh-and-sftp-) \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/custom.svg\"/\u003e\n\n[Smtp Client](#smtp-client-) \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/smtp.svg\"/\u003e\n\n[SSH Client](#ssh-and-sftp-) \u003cimg width=\"26\" height=\"26\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/ssh.svg\"/\u003e\n\n---\n\n## AWS EC2 \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/aws-ec2.svg\"/\u003e\n\nProvides functions to list, start and stop EC2 instances. Primarily uses [amazonica](https://github.com/mcohen01/amazonica) library. Refer to the library's documentation for detailed information on the supported properties.\n\n### Installation\n 1. Add following maven coordinates into titanoboa's external dependencies file: [![Clojars Project](https://img.shields.io/clojars/v/io.titanoboa.tasklet/aws-ec2.svg)](https://clojars.org/io.titanoboa.tasklet/aws-ec2)\n 2. Require namespace: `io.titanoboa.tasklet.aws.ec2`\n\n### Usage\n#### List EC2 Instances\n#### :workload-fn\n```clojure\nio.titanoboa.tasklet.aws.ec2/list-instances\n```\n#### Sample Step Definition\n```clojure\n{:type :aws-ec2-list,\n :supertype :tasklet,\n :description \"Lists all EC2 instances for all reservations.\\nReturns :ec2-instances key with list of instances as a value:\\n{:ec2-instances [{instance1 map} {instance2 map} ...]}\",\n :properties {:credentials {:access-key \"\", :secret-key \"\", :endpoint \"eu-central-1\"}},\n :workload-fn #titanoboa.exp.Expression{:value \"io.titanoboa.tasklet.aws.ec2/list-instances\", :type \"clojure\"}}\n ```\n ---\n #### Start EC2 Instances\n#### :workload-fn\n```clojure\nio.titanoboa.tasklet.aws.ec2/start-instances\n```\n#### Sample Step Definition\n```clojure\n{:type :aws-ec2-start,\n :supertype :tasklet,\n :description \"Starts an EC2 instance.\\nReturns :starting-instances key with status value map.\",\n :workload-fn #titanoboa.exp.Expression{:value \"io.titanoboa.tasklet.aws.ec2/start-instances\", :type \"clojure\"}\n :properties {:credentials {:access-key \"\", :secret-key \"\", :endpoint \"eu-central-1\"}, :instance-ids [\"i-0a123a454b678aeb6\"]}\n}\n```\n ---\n #### Stop EC2 Instances\n#### :workload-fn\n```clojure\nio.titanoboa.tasklet.aws.ec2/stop-instances\n```\n#### Sample Step Definition\n```clojure\n{:type :aws-ec2-stop,\n :supertype :tasklet,\n :description \"Stops an EC2 instance.\\nReturns :stopping-instances key with status value map.\",\n :properties {:credentials {:access-key \"\", :secret-key \"\", :endpoint \"eu-central-1\"}, :instance-ids [\"i-0a123a454b678aeb6\"]},\n :workload-fn #titanoboa.exp.Expression{:value \"io.titanoboa.tasklet.aws.ec2/stop-instances\", :type \"clojure\"}\n}\n```\n---\n---\n## AWS S3 \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/aws-s3.svg\"/\u003e\n\nProvides functions to read, download and upload S3 objects. Primarily uses [amazonica](https://github.com/mcohen01/amazonica) library. Refer to the library's documentation for detailed information on the supported properties.\n\n### Installation\n 1. Add following maven coordinates into titanoboa's external dependencies file: [![Clojars Project](https://img.shields.io/clojars/v/io.titanoboa.tasklet/aws-s3.svg)](https://clojars.org/io.titanoboa.tasklet/aws-s3)\n 2. Require namespace: `io.titanoboa.tasklet.aws.s3`\n\n### Usage\n#### Read S3 Object\n#### :workload-fn\n```clojure\nio.titanoboa.tasklet.aws.s3/read\n```\n#### Sample Step Definition\n```clojure\n{:type :aws-s3-read,\n :supertype :tasklet,\n :description \"Reads textual content of a s3 file and returns it as a job property :s3-object\",\n :workload-fn #titanoboa.exp.Expression{:value \"io.titanoboa.tasklet.aws.s3/read\", :type \"clojure\"}\n :properties {:key \"index.html\", :credentials {:access-key \"\", :secret-key \"\", :endpoint \"eu-central-1\"}, :bucket \"\"}}\n ```\n ---\n #### Download S3 Object\n#### :workload-fn\n```clojure\nio.titanoboa.tasklet.aws.s3/download\n```\n#### Sample Step Definition\n```clojure\n{:type :aws-s3-download,\n :supertype :tasklet,\n :description \"Downloads a file from s3 bucket to job directory under the specified name.\",\n :properties {:key \"index.html\", :credentials {:access-key \"\", :secret-key \"\", :endpoint \"eu-central-1\"}, :save-as \"path/to/file\", :bucket \"bucket-name\"},\n :workload-fn #titanoboa.exp.Expression{:value \"io.titanoboa.tasklet.aws.s3/download\", :type \"clojure\"}}\n```\n ---\n #### Upload S3 Object\n#### :workload-fn\n```clojure\nio.titanoboa.tasklet.aws.s3/upload\n```\n#### Sample Step Definition\n```clojure\n{:type        :aws-s3-upload,\n :supertype   :tasklet,\n :description \"Uploads specified file from job directory into the given s3 bucket.\",\n :properties  {:key \"index.bkp\", :credentials {:access-key \"\", :secret-key \"\", :endpoint \"eu-central-1\"}, :file-path \"index.html\", :bucket \"\"},\n :workload-fn #titanoboa.exp.Expression{:value \"io.titanoboa.tasklet.aws.s3/upload\", :type \"clojure\"}}\n```\n---\n---\n\n## AWS SES \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/aws-ses.svg\"/\u003e\n\nProvides functions to send email via AWS SES. Primarily uses [amazonica](https://github.com/mcohen01/amazonica) library. Refer to the library's documentation for detailed information on the supported properties.\n\n### Installation\n 1. Add following maven coordinates into titanoboa's external dependencies file: [![Clojars Project](https://img.shields.io/clojars/v/io.titanoboa.tasklet/aws-ses.svg)](https://clojars.org/io.titanoboa.tasklet/aws-ses)\n 2. Require namespace: `io.titanoboa.tasklet.aws.ses`\n\n### Usage\n#### :workload-fn\n```clojure\nio.titanoboa.tasklet.aws.ses/send-email\n```\n#### Sample Step Definition\n```clojure\n{:type :aws-ses,\n :supertype :tasklet,\n :description \"Sends an email via SES.\\nReturns :message-id key with message id value.\\n\",\n :properties {:credentials {:access-key \"\", :secret-key \"\", :endpoint \"eu-west-1\"}, :from \"info@titanoboa.io\",\n              :message {:body {:html \"testing 1-2-3-4\", :text \"testing 1-2-3-4\"}, :subject \"greetings from titanoboa\"}, :to [\"miro@titanoboa.io\"]},\n :workload-fn #titanoboa.exp.Expression{:value \"io.titanoboa.tasklet.aws.ses/send-email\", :type \"clojure\"}} \n ```\n ---\n ---\n \n ## AWS SNS \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/aws-sns.svg\"/\u003e\n\nProvides functions to send notification via AWS SNS. Primarily uses [amazonica](https://github.com/mcohen01/amazonica) library. Refer to the library's documentation for detailed information on the supported properties.\n\n### Installation\n 1. Add following maven coordinates into titanoboa's external dependencies file: [![Clojars Project](https://img.shields.io/clojars/v/io.titanoboa.tasklet/aws-sns.svg)](https://clojars.org/io.titanoboa.tasklet/aws-sns)\n 2. Require namespace: `io.titanoboa.tasklet.aws.sns`\n\n### Usage\n#### :workload-fn\n```clojure\nio.titanoboa.tasklet.aws.sns/publish\n```\n#### Sample Step Definition\n```clojure\n{:type :aws-sns,\n :supertype :tasklet,\n :description \"Publishes a message into an SNS topic.\",\n :workload-fn #titanoboa.exp/Expression{:value \"io.titanoboa.tasklet.aws.sns/publish\",\n                                        :type \"clojure\"},\n :properties  {:topic-arn \"arn:aws:sns:us-east-1:676820690883:my-topic\",\n               :subject \"test\",\n               :message \"\",\n               :message-attributes {\"attr\" \"value\"}}}\n ```\n ---\n ---\n \n ## AWS SQS \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/aws-sqs.svg\"/\u003e\n\nProvides functions to send message via AWS SQS. Primarily uses [amazonica](https://github.com/mcohen01/amazonica) library. Refer to the library's documentation for detailed information on the supported properties.\n\n### Installation\n 1. Add following maven coordinates into titanoboa's external dependencies file: [![Clojars Project](https://img.shields.io/clojars/v/io.titanoboa.tasklet/aws-sqs.svg)](https://clojars.org/io.titanoboa.tasklet/aws-sqs)\n 2. Require namespace: `io.titanoboa.tasklet.aws.sqs`\n\n### Usage\n#### :workload-fn\n```clojure\nio.titanoboa.tasklet.aws.sqs/send-message\n```\n#### Sample Step Definition\n```clojure\n{:type :aws-sqs,\n :supertype :tasklet,\n :description \"Sends a text message to a queue.\",\n :workload-fn #titanoboa.exp/Expression{:value \"io.titanoboa.tasklet.aws.sqs/send-message\",\n                                        :type \"clojure\"},\n :properties {:credentials {:access-key \"\",\n                            :secret-key \"\",\n                            :endpoint \"eu-central-1\"},\n              :message-attributes {},\n              :message-body \"\",\n              :queue-url \"\"}}\n ```\n \n ---\n ---\n \n## JDBC Client \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/jdbc.svg\"/\u003e\n\nPerforms a JDBC query and returns corresponding data. Note that [code of jdbc tasklet](https://github.com/mikub/titanoboa/blob/master/src/clj/titanoboa/tasklet/jdbc.clj) is part of standard Titanoboa distribution and is not in this repository.\n\n### Installation\n 1. Add whatever jdbc driver you need to use to titanoboa's ./lib folder\n 2. Require namespace: `titanoboa.tasklet.jdbc` in titanoboa's external dependencies file. You may also need to require `titanoboa.system.jdbc` (see point 3.)\n 3. Do not forget to also define and configure corresponding jdbc system for DB connection pooling in your [server configuration](https://github.com/mikub/titanoboa/wiki/Server-Configuration#server-properties) (in this [example](https://github.com/mikub/titanoboa/wiki/Server-Configuration#non-core-systems) there is a connection pool system :test-db that is using [`titanoboa.system.jdbc/jdbc-pool`](https://github.com/mikub/titanoboa/blob/master/src/clj/titanoboa/system/jdbc.clj)\n\n### Usage\n#### :workload-fn\n```clojure\ntitanoboa.tasklet.jdbc/query\n```\n#### Sample Step Definition\n```clojure\n{:type :jdbc\n :supertype :tasklet\n :workload-fn #titanoboa.exp/Expression {:value \"titanoboa.tasklet.jdbc/query\"}\n :properties {:response-property-name :db-data\n              :data-source-ks [:test-db :system :pool]\n              :query {:select [:o.ordernumber :o.TotalAmount :c.FirstName :c.LastName :c.City :c.Country],\n                      :from [[:customers :c]]\n                      :left-join [[:orders :o] [:= :c.id :o.customerid]]\n                      :order-by [[:o.totalamount :desc :nulls-last]]\n                      :limit 50}}}\n              \n ```\n \nExpected step properties are as follows:\n\n- `:query` - either a query string or a map in [honeysql](https://github.com/seancorfield/honeysql) format\n- `:data-source-ks` key set pointing to the JDBC data source object among the running systems, when used with `titanoboa.system.jdbc/jdbc-pool` the format is  `[:\u003c jdbc pool systemu\u003e :system :pool]` so e.g. if the jdbc system is `:test-db` then it is `[:test-db :system :pool]` \n- `:response-property-name` is self-explanatory\n \n ---\n ---\n \n## Http Client \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/http-client.svg\"/\u003e\n\nMakes an http(s) call and returns (parsed) response. Primarily uses [clj-http](https://github.com/dakrone/clj-http) library. Refer to the library's documentation for detailed information on the generation process and all supported properties.\n\n### Installation\n 1. Add following maven coordinates into titanoboa's external dependencies file: [![Clojars Project](https://img.shields.io/clojars/v/io.titanoboa.tasklet/http-client.svg)](https://clojars.org/io.titanoboa.tasklet/http-client)\n 2. Require namespace: `io.titanoboa.tasklet.httpclient`\n\n### Usage\n#### :workload-fn\n```clojure\nio.titanoboa.tasklet.httpclient/request\n```\n#### Sample Step Definition\n```clojure\n{:type :http-client\n :supertype :tasklet\n :workload-fn #titanoboa.exp/Expression{:value \"io.titanoboa.tasklet.httpclient/request\" :type \"clojure\"}\n :properties {:url \"https://jsonplaceholder.typicode.com/posts/1\"\n              :request-method :get\n              :as :json\n              :proxy-host \"127.0.0.1\"\n              :proxy-port 8118\n              :response-property-name :rest-response\n              :body-only? false\n              :connection-pool {:timeout 5 :threads 4 :insecure? false :default-per-route 10}}}\n              \n ```\n \n ---\n ---\n \n ## Smtp Client \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/smtp.svg\"/\u003e\n\nSends email via smtp. Primarily uses [postal](https://github.com/drewr/postal) library. Refer to the library's documentation for detailed information on the generation process and all supported properties.\n\n### Installation\n 1. Add following maven coordinates into titanoboa's external dependencies file: [![Clojars Project](https://img.shields.io/clojars/v/io.titanoboa.tasklet/smtp.svg)](https://clojars.org/io.titanoboa.tasklet/smtp)\n 2. Require namespace: `io.titanoboa.tasklet.smtp`\n\n### Usage\n#### :workload-fn\n```clojure\ntitanoboa.tasklet.smtp/send\n```\n#### Sample Step Definition\n```clojure\n{:type :smtp\n :supertype :tasklet\n :workload-fn #titanoboa.exp/Expression{:value \"titanoboa.tasklet.smtp/send\"}\n :properties {:connection {:host \"localhost\"\n                           :port 25\n                           :user \"\"\n                           :pass \"\"\n                           :ssl false\n                           :tls false}\n              :email {:from \"miro@example.bla\"\n                      :to \"joe@example.com\"\n                      :cc [\"joe@example.com\", \"jim@example.com\", \"jeff@example.com\"]\n                      :bcc \"archive@example.com\"\n                      :subject \"Cat!\"\n                      :date #titanoboa.exp/Expression{:value \"(java.util.Date.)\"}\n                      :message-id \"\"\n                      :user-agent \"\"\n                      :body [{:type \"text/plain\"\n                              :content \"Hey folks,\\n\\nCheck out these pictures of my cat!\"}\n                             {:type :inline\n                              :content #titanoboa.exp/Expression{:value \"(File. \\\"/tmp/lester-flying-photoshop\\\")\"}\n                              :content-type \"image/jpeg\"\n                              :file-name \"lester-flying.jpeg\"}\n                             {:type :attachment\n                              :content #titanoboa.exp/Expression{:value \"(File. \\\"/tmp/lester-upside-down.jpeg\\\")\"}}]}}}              \n ```\n \n ---\n ---\n \n   ## SSH and SFTP \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/ssh.svg\"/\u003e\n\nSSH and SFTP Client. Primarily uses [clj-ssh](https://github.com/hugoduncan/clj-ssh) library. Refer to the library's documentation for detailed information on the generation process and all supported properties.\n\n### Installation\n 1. Add following maven coordinates into titanoboa's external dependencies file: [![Clojars Project](https://img.shields.io/clojars/v/io.titanoboa.tasklet/ssh.svg)](https://clojars.org/io.titanoboa.tasklet/ssh)\n 2. Require namespace: `io.titanoboa.tasklet.ssh`\n\n### Usage\n#### SSH\n#### :workload-fn\n```clojure\nio.titanoboa.tasklet.ssh/ssh\n```\n#### Sample Step Definition\n```clojure\n{:type :ssh,\n :supertype :tasklet,\n :description \"SSH Client\",\n :properties {:ssh-agent-settings {:use-system-ssh-agent false}, \n              :identities {:private-key-path \"/path/to/key.pem\"}, \n               :ssh-cmd-map {:in \"echo hello\"}, \n               :host \"xxx.eu-central-1.compute.amazonaws.com\", \n               :session-options {:username \"ec2-user\", :strict-host-key-checking \"no\", :preferred-authentications \"publickey\"}},\n :workload-fn #titanoboa.exp.Expression{:value \"io.titanoboa.tasklet.ssh/ssh\", :type \"clojure\"}}\n ```\n\n#### SFTP\n#### :workload-fn\n```clojure\nio.titanoboa.tasklet.ssh/sftp\n```\n#### Sample Step Definition\n```clojure\n{:type :sftp,\n :supertype :tasklet,\n :description \"SFTP Client\",\n :properties {:ssh-agent-settings {:use-system-ssh-agent false}, \n               :identities {:private-key-path \"/path/to/key.pem\"}, \n               :sftp-cmds-vec [[:ls \"/home/ec2-user/\"]], \n               :host \"xxx.eu-central-1.compute.amazonaws.com\", \n               :session-options {:username \"ec2-user\", \n                                 :strict-host-key-checking \"no\", \n                                 :preferred-authentications \"publickey\"}},\n :workload-fn #titanoboa.exp.Expression{:value \"io.titanoboa.tasklet.ssh/sftp\", :type \"clojure\"}}\n ```\n\n ---\n ---\n \n## PDF \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/pdf-generation.svg\"/\u003e\n\nGenerates a pdf file based on job properties. Primarily uses [clj-pdf](https://github.com/clj-pdf/clj-pdf) library. Refer to the library's documentation for detailed information on the generation process and all supported properties.\n\n### Installation\n 1. Add following maven coordinates into titanoboa's external dependencies file: [![Clojars Project](https://img.shields.io/clojars/v/io.titanoboa.tasklet/pdf.svg)](https://clojars.org/io.titanoboa.tasklet/pdf)\n 2. Require namespace: `io.titanoboa.tasklet.pdf`\n\n### Usage\n#### :workload-fn\n```clojure\nio.titanoboa.tasklet.pdf/generate-pdf\n```\n#### Sample Properties\n```clojure\n{:pdf-sections [[:list {:roman true}\n          [:chunk {:style :bold} \"a bold item\"]\n          \"another item\"\n          \"yet another item\"]\n   [:phrase \"some text\"]\n   [:phrase \"some more text\"]\n   [:paragraph \"yet more text\"]] \n :file-name \"example.pdf\" \n :pdf-metadata {:bottom-margin 10, :creator \"Jane Doe\", :doc-header [\"inspired by\" \"William Shakespeare\"], :right-margin 50, :left-margin 10, :footer \"page\", :header \"page header\", :size \"a4\", :title \"Test doc\", :author \"John Doe\", :top-margin 20, :subject \"Some subject\"}}\n```\n#### Sample Step Definition\n```clojure\n{:type :pdf-generation\n :supertype :tasklet\n :properties\n {:pdf-sections [[:list {:roman true}\n          [:chunk {:style :bold} \"a bold item\"]\n          \"another item\"\n          \"yet another item\"]\n   [:phrase \"some text\"]\n   [:phrase \"some more text\"]\n   [:paragraph \"yet more text\"]] \n :file-name \"example.pdf\" \n :pdf-metadata {:bottom-margin 10, :creator \"Jane Doe\", :doc-header [\"inspired by\" \"William Shakespeare\"], :right-margin 50, :left-margin 10, :footer \"page\", :header \"page header\", :size \"a4\", :title \"Test doc\", :author \"John Doe\", :top-margin 20, :subject \"Some subject\"}}\n :workload-fn #titanoboa.exp.Expression{:value \"io.titanoboa.tasklet.pdf/generate-pdf\", :type \"clojure\"}}\n```\n\n ---\n ---\n \n   ## Kafka Producer \u0026 Consumer \u003cimg width=\"28\" height=\"28\" align=\"left\" src=\"https://github.com/mikub/titanoboa-tasklets/blob/master/_doc/step-icons/kafka.svg\"/\u003e\n\nA simple Kafka producer and consumer. Primarily uses [dvlopt/kafka](https://github.com/dvlopt/kafka) library. Refer to the library's documentation for detailed information on the generation process and all supported properties.\n\n### Installation\n 1. Add following maven coordinates into titanoboa's external dependencies file: [![Clojars Project](https://img.shields.io/clojars/v/io.titanoboa.tasklet/kafka.svg)](https://clojars.org/io.titanoboa.tasklet/kafka)\n 2. Require namespace: `io.titanoboa.tasklet.kafka`\n\n### Usage\n#### Producer\n#### :workload-fn\n```clojure\nio.titanoboa.tasklet.kafka/produce\n```\n#### Sample Step Definition\n```clojure\n{:type        :kafka-produce,\n :supertype   :tasklet,\n :workload-fn #titanoboa.exp/Expression{:value \"io.titanoboa.tasklet.kafka/produce\",\n                                        :type  \"clojure\"},\n :properties  {:kafka-producer-config {:dvlopt.kafka/nodes             [[\"localhost\"\n                                                                         9092]],\n                                       :dvlopt.kafka/serializer.key    :long,\n                                       :dvlopt.kafka/serializer.value  :string,\n                                       :dvlopt.kafka.out/configuration {\"client.id\"        \"my-producer\",\n                                                                        \"transactional.id\" \"some transaction id\"}},\n               :records               [{:topic \"test-topic\",\n                                        :key   123,\n                                        :value \"Hello World!\"}]}} \n```\n\n#### Consumer\n#### :workload-fn\n```clojure\nio.titanoboa.tasklet.kafka/consume\n```\n#### Sample Step Definition\n```clojure\n{:type        :kafka-consume,\n :supertype   :tasklet,\n :workload-fn #titanoboa.exp/Expression{:value \"io.titanoboa.tasklet.kafka/consume\",\n                                        :type  \"clojure\"},\n :properties  {:kafka-topics          [\"test-topic\"],\n               :poll-options          {:dvlopt.kafka/timeout [1\n                                                              :seconds]},\n               :kafka-consumer-config {:dvlopt.kafka/nodes              [[\"localhost\"\n                                                                          9092]],\n                                       :dvlopt.kafka/deserializer.key   :long,\n                                       :dvlopt.kafka/deserializer.value :string,\n                                       :dvlopt.kafka.in/configuration   {\"auto.offset.reset\"  \"earliest\",\n                                                                         \"enable.auto.commit\" false,\n                                                                         \"max.poll.records\"   \"50\",\n                                                                         \"group.id\"           \"my-group\"}}}}\n ```\n\n\n ---\n ---\n \n   ## 🧬 K-mer count\n\nFew simple functions to help with [K-mer](https://en.wikipedia.org/wiki/K-mer) counting and analysis of [FASTQ](https://en.wikipedia.org/wiki/FASTQ_format) data files. Also contains functions for splitter (map) and agregator (reduce) type of steps to help with parallel processing. \n\nNote that a thought needs to be put into what underlying file system that would be used (e.g. HDFS, EFS etc.) and whether a physical splitting of the file would be performed prior to the counting.\n\n### Installation\n 1. Add following maven coordinates into titanoboa's external dependencies file: [![Clojars Project](https://img.shields.io/clojars/v/io.titanoboa.tasklet/kmer.svg)](https://clojars.org/io.titanoboa.tasklet/kmer)\n 2. Require namespace: `io.titanoboa.tasklet.kmer`\n\n### Usage\n#### K-Mer count\n#### :workload-fn\n```clojure\nio.titanoboa.tasklet.kmer/kmer-count\n```\n#### Sample Job Properties\n```clojure\n{:create-folder? false,\n              :fastq-file \"/path/to/fastq/file\",\n              :start 0,\n              :end 12,\n              :k 3,\n              :top-n 10}\n ```\n\n#### Map/Reduce Steps\n#### Map :workload-fn\n```clojure\nio.titanoboa.tasklet.kmer/split-fastq\n```\n#### Reduce :workload-fn\n```clojure\nio.titanoboa.tasklet.kmer/reduce-kmers\n```\n#### Sample Job Properties\n```clojure\n{:fastq-file \"/path/to/fastq/file\",\n              :k 3,\n              :split-to 12}\n ```\n\n#### Sample Map/Reduce Workflow Definition\n```clojure\n{:first-step \"splitter\",\n :name \"kmer-map-reduce\",\n :revision 4,\n :type nil,\n :properties {:fastq-file \"/mnt/efs/sars2/reclojure.fastq\",\n              :k 3,\n              :split-to 12,\n              :top-n 10},\n :steps [{:id \"splitter\",\n          :type :map,\n          :supertype :map,\n          :next [[\"*\" \"aggregator\"]],\n          :workload-fn #titanoboa.exp/Expression{:value \"io.titanoboa.tasklet.kmer/split-fastq\",\n                                                 :type \"clojure\"},\n          :properties {:jobdef-name \"k-mer-count\",\n                       :sys-key :core,\n                       :standalone-system? false},\n          :revision 1}\n         {:id \"aggregator\",\n          :type :reduce,\n          :supertype :reduce,\n          :workload-fn #titanoboa.exp/Expression{:value \"io.titanoboa.tasklet.kmer/reduce-kmers\",\n                                                 :type \"clojure\"},\n          :next [],\n          :properties {:map-step-id \"splitter\", :commit-interval 100},\n          :revision 1}]}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcommsor%2Ftitanoboa-tasklets","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcommsor%2Ftitanoboa-tasklets","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcommsor%2Ftitanoboa-tasklets/lists"}