{"id":24635158,"url":"https://github.com/shinexia/executors-go","last_synced_at":"2025-03-20T08:17:52.952Z","repository":{"id":262587107,"uuid":"887696305","full_name":"shinexia/executors-go","owner":"shinexia","description":"Executors implemented in Go","archived":false,"fork":false,"pushed_at":"2024-12-30T09:53:23.000Z","size":26,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-25T09:14:42.535Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shinexia.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-13T05:56:09.000Z","updated_at":"2024-12-30T09:53:27.000Z","dependencies_parsed_at":"2024-11-13T08:22:32.676Z","dependency_job_id":"6ebab543-89f8-46d6-bba0-8a4adc79d7db","html_url":"https://github.com/shinexia/executors-go","commit_stats":null,"previous_names":["shinexia/executors-go"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shinexia%2Fexecutors-go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shinexia%2Fexecutors-go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shinexia%2Fexecutors-go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shinexia%2Fexecutors-go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shinexia","download_url":"https://codeload.github.com/shinexia/executors-go/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244574842,"owners_count":20474823,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-25T09:14:45.605Z","updated_at":"2025-03-20T08:17:52.902Z","avatar_url":"https://github.com/shinexia.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Executors\n\nImplement a stateful task executor, `Executor`, with the following features:\n\n- Events are stateful and retryable, with the function signature: `func(sin any) (sout any, err error)`.\n- Tasks can be orchestrated using `Pipe` (sequential), `Map` (data parallel), and `Parallel` (compute parallel) methods.\n- For a multi-step task, if a step fails, the next retry will resume from the failed step, and previously successful steps will not be re-executed.\n- Task states can be dumped and loaded, and upon reloading, execution resumes from the last failed step.\n- Tasks automatically retry upon failure, but if new events arrive, the retrying tasks can be canceled. For example, when dumping a snapshot, if a new version of the snapshot is available, only the latest snapshot needs to be written, and previously blocked tasks can be canceled.\n\n## Getting started\n\n```go\nimport (\n\t\"fmt\"\n\t\"math/rand\"\n\n\t\"github.com/shinexia/executors-go\"\n)\n\ntask := executors.Pipe(\n    // split\n    func(sin int) (any, error) {\n        if rand.Intn(100) \u003c 50 {\n            // must return the original sin if error\n            return sin, fmt.Errorf(\"split error\")\n        }\n        out := make([]int, sin)\n        for i := range sin {\n            out[i] = i\n        }\n        return out, nil\n    },\n    // map\n    executors.Map(func(sin int) (any, error) {\n        if n := rand.Intn(100); n \u003c 50 {\n            // must return the original sin if error\n            return sin, fmt.Errorf(\"map error\")\n        }\n        return sin * 100, nil\n    }),\n    // reduce\n    func(sin []int) (any, error) {\n        if n := rand.Intn(100); n \u003c 50 {\n            // must return the original sin if error\n            return sin, fmt.Errorf(\"reduce error\")\n        }\n        return executors.Sum(sin), nil\n    },\n)\nvar snapshot []byte = nil\nvar result int\nfor {\n    var sin any = nil\n    if len(snapshot) == 0 {\n        sin = 10\n    } else {\n        // load state from snapshot\n        state := \u0026executors.TaskState{}\n        json.Unmarshal(snapshot, state)\n        sin = state.Stateful\n    }\n    sout, err := executors.Run(\"test\", task, sin, executors.WithCallback(func(state *executors.TaskState, err error) {\n        // dump sout to a file or db\n        snapshot, _ = json.Marshal(state)\n    }))\n    if err != nil {\n        fmt.Printf(\"retring, sout: %v, err: %v\\n\", executors.ToJSON(sout), err)\n        if executors.IsRuntimeError(err) {\n            t.Fatalf(\"runtime error: %+v\", err)\n        }\n        continue\n    }\n    result, _ = executors.Cast[int](sout)\n    fmt.Printf(\"succeed, sout: %v\\n\", sout)\n    break\n}\nif result != 4500 {\n    t.Errorf(\"result: %v, expect: 4500\", result)\n}\n```\n\n## More examples\n\nSee [examples_test.go](examples_test.go)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshinexia%2Fexecutors-go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshinexia%2Fexecutors-go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshinexia%2Fexecutors-go/lists"}