{"id":15908097,"url":"https://github.com/ignalina/thund","last_synced_at":"2026-05-16T22:02:38.431Z","repository":{"id":44693170,"uuid":"512941988","full_name":"Ignalina/thund","owner":"Ignalina","description":"GO based DAG processor.","archived":false,"fork":false,"pushed_at":"2022-12-18T23:26:52.000Z","size":311,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-13T01:46:19.802Z","etag":null,"topics":["arrow","arrow2","brokkr","go","rust"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Ignalina.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-07-11T23:58:14.000Z","updated_at":"2022-10-18T21:30:44.000Z","dependencies_parsed_at":"2023-01-29T20:15:20.188Z","dependency_job_id":null,"html_url":"https://github.com/Ignalina/thund","commit_stats":null,"previous_names":[],"tags_count":43,"template":false,"template_full_name":null,"purl":"pkg:github/Ignalina/thund","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ignalina%2Fthund","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ignalina%2Fthund/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ignalina%2Fthund/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ignalina%2Fthund/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Ignalina","download_url":"https://codeload.github.com/Ignalina/thund/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ignalina%2Fthund/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33120450,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-16T18:38:32.183Z","status":"ssl_error","status_checked_at":"2026-05-16T18:38:29.903Z","response_time":115,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arrow","arrow2","brokkr","go","rust"],"created_at":"2024-10-06T14:09:39.194Z","updated_at":"2026-05-16T22:02:38.416Z","avatar_url":"https://github.com/Ignalina.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Thund , DAG processor based on Apache Arrow.\nA modern and performant/robust dag processor for data pipelines allowing processing data in/out from storages like S3 or Iceberg/delta lakes without interruptions.\n\n# Why ??\nWhenever it is not feasible for an Apache Airflow / NIFI / Hadoop flying circus alike. Legacy software could remain operating on your storage/lake data in conjunction with Thund handling In/Out.  For a complete modern stack combine Apache Arrows Balista/Datafusion in combination with Thund.   \n\n\nIf you dont get it , no worries its an early experiment , perhaps \"Grímnismál\" (Year 1300-1325) in the Poetic Edda explains it goal better  \n*__\"Thunda's waters hast'ning fleet,__*  \n*__Touch not Valgom! with thy feet.\"__*\n\n\n# Design goals are\n* Apache Arrow centric\n* Minimalistic / Performant / Robust\n* Conform to  Brokkr's Software critera https://github.com/Ignalina/brokkr\n\n\nGoals below are to be sorted for V1,V2 or V never\n\n### Functional Goals  V0\n* Fix eventhandler/step arguments from simple reader to functions for create reader and writer.\n* Picture of the watcher-\u003eeventhandlers mechanics and tossing of parameters.\n\n### Functional Goals  V1 \n* Alloy component , Could Arrow references be used betwen Golang-Rust ?\n* Support for Arrows filesystem HDFS,\n* Incorporate RCLONE\n* Graph support\n* Add handlers to Arrow-\u003eTantivy/Apache flight/kafka/delta-rs\n* Handlers Deployable/Callable from minifi\n\n### Functional Goals V2\n\n* Steps spread out on multiple Processors\n* Jaeger \n* Metrics\n* Static Deployment via ipmi\n* Deployment via kubernetes, as static as possible.\n\n\n### Thund in the litterature\nTranslations poeems describing Thund [Germanic mythology](http://www.germanicmythology.com/PoeticEdda/GRM21.html)  \nLearn pronounce in Icelandic [ÓÐSMÁL](https://odsmal.org/thund-thund-mythological-river)  \n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fignalina%2Fthund","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fignalina%2Fthund","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fignalina%2Fthund/lists"}