{"id":20812556,"url":"https://github.com/superstreamlabs/memphis-functions.go","last_synced_at":"2025-03-12T04:45:56.307Z","repository":{"id":208770598,"uuid":"722441225","full_name":"superstreamlabs/memphis-functions.go","owner":"superstreamlabs","description":null,"archived":false,"fork":false,"pushed_at":"2024-01-15T14:26:02.000Z","size":37,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-05-07T12:46:25.911Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/superstreamlabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-23T06:45:24.000Z","updated_at":"2024-05-07T12:46:31.826Z","dependencies_parsed_at":"2023-12-20T15:30:29.735Z","dependency_job_id":"e126a7d9-8a6a-4546-903f-84fd37bcc43d","html_url":"https://github.com/superstreamlabs/memphis-functions.go","commit_stats":null,"previous_names":["memphisdev/memphis-functions.go","superstreamlabs/memphis-functions.go"],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superstreamlabs%2Fmemphis-functions.go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superstreamlabs%2Fmemphis-functions.go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superstreamlabs%2Fmemphis-functions.go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superstreamlabs%2Fmemphis-functions.go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/superstreamlabs","download_url":"https://codeload.github.com/superstreamlabs/memphis-functions.go/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243158972,"owners_count":20245669,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-17T20:55:29.594Z","updated_at":"2025-03-12T04:45:56.287Z","avatar_url":"https://github.com/superstreamlabs.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Github (6)](https://github.com/memphisdev/memphis/assets/107035359/bc2feafc-946c-4569-ab8d-836bc0181890)](https://www.functions.memphis.dev/)\n\u003cp align=\"center\"\u003e\n\u003ca href=\"https://memphis.dev/discord\"\u003e\u003cimg src=\"https://img.shields.io/discord/963333392844328961?color=6557ff\u0026label=discord\" alt=\"Discord\"\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/memphisdev/memphis/issues?q=is%3Aissue+is%3Aclosed\"\u003e\u003cimg src=\"https://img.shields.io/github/issues-closed/memphisdev/memphis?color=6557ff\"\u003e\u003c/a\u003e \n  \u003cimg src=\"https://img.shields.io/npm/dw/memphis-dev?color=ffc633\u0026label=installations\"\u003e\n\u003ca href=\"https://github.com/memphisdev/memphis/blob/master/CODE_OF_CONDUCT.md\"\u003e\u003cimg src=\"https://img.shields.io/badge/Code%20of%20Conduct-v1.0-ff69b4.svg?color=ffc633\" alt=\"Code Of Conduct\"\u003e\u003c/a\u003e \n\u003cimg alt=\"GitHub release (latest by date)\" src=\"https://img.shields.io/github/v/release/memphisdev/memphis?color=61dfc6\"\u003e\n\u003cimg src=\"https://img.shields.io/github/last-commit/memphisdev/memphis?color=61dfc6\u0026label=last%20commit\"\u003e\n\u003c/p\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \n\u003cimg width=\"177\" alt=\"cloud_native 2 (5)\" src=\"https://github.com/memphisdev/memphis/assets/107035359/a20ea11c-d509-42bb-a46c-e388c8424101\"\u003e\n  \n\u003c/div\u003e\n \n \u003cb\u003e\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://memphis.dev/pricing/\"\u003eCloud\u003c/a\u003e - \u003ca href=\"https://memphis.dev/docs/\"\u003eDocs\u003c/a\u003e - \u003ca href=\"https://twitter.com/Memphis_Dev\"\u003eX\u003c/a\u003e - \u003ca href=\"https://www.youtube.com/channel/UCVdMDLCSxXOqtgrBaRUHKKg\"\u003eYouTube\u003c/a\u003e\n\u003c/p\u003e\u003c/b\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n  \u003ch4\u003e\n\n**[Memphis.dev](https://memphis.dev)** is more than a broker. It's a new streaming stack.\u003cbr\u003e\nMemphis.dev is a highly scalable event streaming and processing engine.\u003cbr\u003e\n\n  \u003c/h4\u003e\n  \n\u003c/div\u003e\n\n## ![20](https://user-images.githubusercontent.com/70286779/220196529-abb958d2-5c58-4c33-b5e0-40f5446515ad.png) About\n\nBefore Memphis came along, handling ingestion and processing of events on a large scale took months to adopt and was a capability reserved for the top 20% of mega-companies. Now, Memphis opens the door for the other 80% to unleash their event and data streaming superpowers quickly, easily, and with great cost-effectiveness.\n\nThis repository is responsible for the Memphis Functions Golang SDK\n\n# Installation\nAfter installing and running memphis broker,\u003cbr\u003e\nIn your project's directory:\n\n```shell\ngo get github.com/memphisdev/memphis-functions.go\n```\n\n# Importing\n```go\nimport \"github.com/memphisdev/memphis-functions.go/memphis\"\n```\n\n### Creating a Memphis function\nMemphis provides a `CreateFunction` utility for more easily creating Memphis Functions.\n\nThe user will write a function which will act as an event handler and will be called for every event that is to be processed by Functions. \n\nThe user created event handler must fulfill the following function signature:\n```go\ntype HandlerType func(any, map[string]string, map[string]string) (any, map[string]string, error)\n\n```\n\nThe `any` first parameter will always be a pointer to the message payload. \n\nThe event handler will take in a `[]byte` or `*Object` representation of an event. It will also take a `map[string]string` of headers that belong to that event, and `map[string]string` representation of the function inputs.\n\nThe event handler will then return a modified version of these fields. The return must be either a `[]byte` or a type that will be able to be used with `json.Marshal`.\n\nIf the processing fails, the user function should return `nil, nil, err`. If the user wishes to skip over an event and not send it to the station, return `nil, nil, nil` and the event will be skipped. Events that return an error will be sent to the dead letter station. \n\n```go\npackage main\n\nimport (\n\t\"encoding/json\"\n    \"github.com/memphisdev/memphis-functions.go/memphis\"\n)\n\ntype Event struct {\n\tField1 string `json:\"field1\"`\n\tField2 string `json:\"field2\"`\n}\n\nfunc eventHandlerFunc(msgPayload any, msgHeaders map[string]string, inputs map[string]string) (any, map[string]string, error){\n    // Type assert to user type. By default this is a []byte\n    as_bytes, ok := msgPayload.([]byte)\n\tif !ok{\n\t\treturn nil, nil, fmt.Errorf(\"object failed type assertion: %v, %v\", message, reflect.TypeOf(message))\n\t}\n    \n    // Get data from msgPayload\n    var event Event\n    json.Unmarshal(as_bytes, \u0026event)\n    \n    // Modify or do something with the payload\n    event.Field1 = \"modified\"\n    \n    // Return the modified payload\n    return event, msgHeaders, nil\n}\n\nfunc main() {\n\tmemphis.CreateFunction(eventHandlerFunc)\n}\n```\n\nTo use a user specified object, create an empty one in main and pass that to the `memphis.PayloadAsJSON` function.\n\n```go\npackage main\n\nimport (\n\t\"encoding/json\"\n    \"github.com/memphisdev/memphis-functions.go/memphis\"\n)\n\ntype Event struct {\n\tField1 string `json:\"field1\"`\n\tField2 string `json:\"field2\"`\n}\n\nfunc eventHandlerFunc(msgPayload any, msgHeaders map[string]string, inputs map[string]string) (any, map[string]string, error){\n    // Get data from msgPayload\n    typedPayload = msgPayload.(*Event)\n\n    // Modify or do something with the payload\n    typedPayload.Field1 = \"modified\"\n    \n    return typedPayload, msgHeaders, nil\n}\n\nfunc main() {\n    var eventObject Event\n\tmemphis.CreateFunction(eventHandlerFunc, memphis.PayloadAsJSON(\u0026eventObject))\n}\n```\n\n\u003e Note the type assertion is using a pointer to the object, and the address of the object is passed into `memphis.PayloadAsJSON`.\n\nAs mentioned previously, if the user would like to send the message to the dead letter station, simply return an error. The unproccessed payload and headers will be included with the message to the dead letter station.\n\n```go\npackage main\n\nimport (\n\t\"encoding/json\"\n    \"strings\"\n    \"errors\"\n    \"github.com/memphisdev/memphis-functions.go/memphis\"\n)\n\ntype Event struct {\n\tField1 string `json:\"field1\"`\n\tField2 string `json:\"field2\"`\n}\n\nfunc eventHandlerFunc(msgPayload any, msgHeaders map[string]string, inputs map[string]string) ([]byte, map[string]string, error){\n    // Type assert to user type. By default this is a []byte\n    as_bytes, ok := msgPayload.([]byte)\n\tif !ok{\n\t\treturn nil, nil, fmt.Errorf(\"object failed type assertion: %v, %v\", message, reflect.TypeOf(message))\n\t}\n\n    // Get data from msgPayload\n    var event Event\n    json.Unmarshal(as_bytes, \u0026event)\n    \n    // Modify or do something with the payload\n    if strings.Contains(event.Field1, \"Bob\"){\n        return nil, nil, fmt.Errorf(\"String had Bob in it!\")\n    } \n    \n    // Return the modified payload\n    return event, msgHeaders, nil\n}\n\nfunc main() {\n\tmemphis.CreateFunction(eventHandlerFunc)\n}\n```\n\nIf the user would rather this message just be skipped, instead of being sent to the dead letter station, return `nil` for all values:\n\n```go\npackage main\n\nimport (\n\t\"encoding/json\"\n    \"strings\"\n    \"github.com/memphisdev/memphis-functions.go/memphis\"\n)\n\ntype Event struct {\n\tField1 string `json:\"field1\"`\n\tField2 string `json:\"field2\"`\n}\n\nfunc eventHandlerFunc(msgPayload any, msgHeaders map[string]string, inputs map[string]string) (any, map[string]string, error){\n    // Type assert to user type. By default this is a []byte\n    as_bytes, ok := msgPayload.([]byte)\n\tif !ok{\n\t\treturn nil, nil, fmt.Errorf(\"object failed type assertion: %v, %v\", message, reflect.TypeOf(message))\n\t}\n\n    // Get data from msgPayload\n    var event Event\n    json.Unmarshal(as_bytes, \u0026event)\n    \n    // Modify or do something with the payload\n    if strings.Contains(event.Field1, \"Bob\"){\n        return nil, nil, nil\n    } \n    \n    // Return the modified payload\n    return event, msgHeaders, nil\n}\n\nfunc main() {\n\tmemphis.CreateFunction(eventHandlerFunc)\n}\n```\n\nLastly, if the user is using a format like Protocol Buffers, the user may simple decode the msgPayload with their proto function. Assuming that we have a proto definition like: \n```proto\nsyntax = \"proto3\";\npackage protobuf_example;\n\nmessage Message{\n    string data_field = 1;\n}\n```\nThat is saved in the directory `user_message/message.pb.go`, we can use this Message object like so:\n\n```go\npackage main\n\nimport (\n\t\"encoding/json\"\n    \"strings\"\n    \"github.com/memphisdev/memphis-functions.go/memphis\"\n    \"google.golang.org/protobuf/proto\"\n    \"current_directory/user_message\"\n)\n\nfunc eventHandlerFunc(msgPayload any, msgHeaders map[string]string, inputs map[string]string) (any, map[string]string, error){\n    // Type assert to user type. By default this is a []byte\n    as_bytes, ok := msgPayload.([]byte)\n\tif !ok{\n\t\treturn nil, nil, fmt.Errorf(\"object failed type assertion: %v, %v\", message, reflect.TypeOf(message))\n\t}\n\n    // Get data from msgPayload\n    var my_message user_message.Message\n    proto.Unmarshal(as_bytes, \u0026user_message)\n    \n    // Modify or do something with the payload\n    my_message.data_field = \"new_data\"\n    \n    // Return the payload back as []bytes so we don't json.Marshal the message\n    eventBytes, _ := proto.Marshal(\u0026my_message)\n    return eventBytes, msgHeaders, nil\n}\n\nfunc main() {\n\tmemphis.CreateFunction(eventHandlerFunc)\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsuperstreamlabs%2Fmemphis-functions.go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsuperstreamlabs%2Fmemphis-functions.go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsuperstreamlabs%2Fmemphis-functions.go/lists"}