{"id":19585481,"url":"https://github.com/stremovskyy/recorder","last_synced_at":"2026-05-10T23:50:04.790Z","repository":{"id":242587789,"uuid":"809975366","full_name":"stremovskyy/recorder","owner":"stremovskyy","description":"A Go library for recording and retrieving requests, responses, errors, and metrics. This library provides both synchronous and asynchronous methods for recording data to Redis or file storage.","archived":false,"fork":false,"pushed_at":"2024-06-14T09:25:52.000Z","size":14,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-24T10:06:52.115Z","etag":null,"topics":["go","golang","http","library","package","redis"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stremovskyy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-03T20:10:29.000Z","updated_at":"2024-10-26T16:57:50.000Z","dependencies_parsed_at":"2024-06-19T03:22:01.744Z","dependency_job_id":null,"html_url":"https://github.com/stremovskyy/recorder","commit_stats":null,"previous_names":["stremovskyy/recorder"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stremovskyy%2Frecorder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stremovskyy%2Frecorder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stremovskyy%2Frecorder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stremovskyy%2Frecorder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stremovskyy","download_url":"https://codeload.github.com/stremovskyy/recorder/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240858578,"owners_count":19868998,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang","http","library","package","redis"],"created_at":"2024-11-11T07:54:19.063Z","updated_at":"2026-05-10T23:50:04.772Z","avatar_url":"https://github.com/stremovskyy.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Recorder\n\nA Go library for recording and retrieving requests, responses, errors, and metrics. This library provides both synchronous and asynchronous methods for recording data to Redis or file storage.\n\n## Features\n\n- Record and retrieve requests, responses, errors, and metrics.\n- Support for both Redis and file-based storage backends.\n- Asynchronous methods for non-blocking operations.\n- Easy to extend with custom storage backends.\n\n## Installation\n\nTo install the `recorder` library, use `go get`:\n\n```sh\ngo get github.com/stremovskyy/recorder\n```\n\n## Usage\n\n### Interface\n\nThe `recorder` package now splits responsibilities between the high-level `Recorder` API and the pluggable `Storage` abstraction:\n\n```go\n// Storage abstracts persistence. Implement it to target any backend (SQL, NoSQL, files, etc.).\ntype Storage interface {\nSave(ctx context.Context, record Record) error\nLoad(ctx context.Context, recordType RecordType, requestID string) ([]byte, error)\nFindByTag(ctx context.Context, tag string) ([]string, error)\n}\n\n// Recorder is the public interface for the recorder.\ntype Recorder interface {\n\tRecordRequest(ctx context.Context, primaryID *string, requestID string, request []byte, tags map[string]string) error\n\tRecordResponse(ctx context.Context, primaryID *string, requestID string, response []byte, tags map[string]string) error\n\tRecordError(ctx context.Context, id *string, requestID string, err error, tags map[string]string) error\n\tRecordMetrics(ctx context.Context, primaryID *string, requestID string, metrics map[string]string, tags map[string]string) error\n\tGetRequest(ctx context.Context, requestID string) ([]byte, error)\n\tGetResponse(ctx context.Context, requestID string) ([]byte, error)\n\tFindByTag(ctx context.Context, tag string) ([]string, error)\n\tAsync() AsyncRecorder\n}\n\n// AsyncRecorder defines the asynchronous methods for the recorder.\ntype AsyncRecorder interface {\n\tRecordRequest(ctx context.Context, primaryID *string, requestID string, request []byte, tags map[string]string) \u003c-chan error\n\tRecordResponse(ctx context.Context, primaryID *string, requestID string, response []byte, tags map[string]string) \u003c-chan error\n\tRecordError(ctx context.Context, id *string, requestID string, err error, tags map[string]string) \u003c-chan error\n\tRecordMetrics(ctx context.Context, primaryID *string, requestID string, metrics map[string]string, tags map[string]string) \u003c-chan error\n\tGetRequest(ctx context.Context, requestID string) \u003c-chan Result\n\tGetResponse(ctx context.Context, requestID string) \u003c-chan Result\n\tFindByTag(ctx context.Context, tag string) \u003c-chan FindByTagResult\n}\n\n// New wraps a Storage implementation and returns a fully featured Recorder.\nfunc New(storage Storage) Recorder\n```\n\n### Sensitive Data Scrubber\n\nUse the scrubber utilities when you need to strip secrets before persisting payloads.\n\n```go\nscrub := recorder.NewScrubber()\npayload := map[string]any{\n    \"password\": \"super-secret\",\n    \"headers\": map[string][]string{\"Authorization\": []string{\"Bearer token\"}},\n}\nmasked := scrub.Scrub(payload).(map[string]any)\n// masked[\"password\"] == \"[REDACTED]\"\n\nstorage := yourStorage{} // implements recorder.Storage\nrec := recorder.New(storage, recorder.WithScrubber(scrub))\n_ = rec.RecordRequest(ctx, nil, \"req-42\", []byte(`{\"token\":\"abc\"}`), nil)\n```\n\nYou can tailor which fields are scrubbed and how the data is transformed by declaring rules:\n\n```go\nscrub := recorder.NewScrubber(\n    recorder.WithDefaultReplacement(\"\u003chidden\u003e\"),\n    recorder.WithoutDefaultRules(),\n)\nscrub.AddRules(\n    recorder.NewRule(\n        \"mask-token\",\n        recorder.MatchPathInsensitive(\"credentials.token\"),\n        recorder.MaskString('*', 0, 4),\n    ),\n)\n\nrec := recorder.New(\n    storage,\n    recorder.WithScrubber(scrub, recorder.ScrubberFailOnError()),\n)\n```\n\nFor non-JSON payloads or advanced logic, supply your own sanitizers with `recorder.WithPayloadScrubber` or `recorder.WithTagScrubber`.\n\n### Redis Implementation\n#### Usage\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\t\"time\"\n\n\t\"github.com/stremovskyy/recorder/redis_recorder\"\n)\n\nfunc main() {\n\toptions := \u0026redis_recorder.Options{\n\t\tAddr:          \"localhost:6379\",\n\t\tPassword:      \"\",\n\t\tDB:            0,\n\t\tPrefix:        \"myapp\",\n\t\tDefaultTTL:    24 * time.Hour,\n\t\tCompressionLvl: 5,\n\t\tDebug:         true,\n\t}\n\n\trec := redis_recorder.NewRedisRecorder(options)\n\n\t// Record a request\n\terr := rec.RecordRequest(context.Background(), nil, \"req1\", []byte(\"request data\"), nil)\n\tif err != nil {\n\t\tlog.Fatalf(\"Failed to record request: %v\", err)\n\t}\n\n\t// Retrieve a request\n\tdata, err := rec.GetRequest(context.Background(), \"req1\")\n\tif err != nil {\n\t\tlog.Fatalf(\"Failed to get request: %v\", err)\n\t}\n\tfmt.Println(\"Request data:\", string(data))\n}\n```\n\n### File-based Implementation\n\n#### Usage\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\n\t\"github.com/stremovskyy/recorder/file_recorder\"\n)\n\nfunc main() {\n\trec := file_recorder.NewFileRecorder(\"/path/to/store/files\")\n\n\t// Record a request\n\terr := rec.RecordRequest(context.Background(), nil, \"req1\", []byte(\"request data\"), nil)\n\tif err != nil {\n\t\tlog.Fatalf(\"Failed to record request: %v\", err)\n\t}\n\n\t// Retrieve a request\n\tdata, err := rec.GetRequest(context.Background(), \"req1\")\n\tif err != nil {\n\t\tlog.Fatalf(\"Failed to get request: %v\", err)\n\t}\n\tfmt.Println(\"Request data:\", string(data))\n}\n```\n\n### GORM + MySQL Implementation\n\nUse GORM with the MySQL driver and hand the configured *gorm.DB to the factory:\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"fmt\"\n    \"log\"\n\n    \"gorm.io/driver/mysql\"\n    \"gorm.io/gorm\"\n\n    \"github.com/stremovskyy/recorder/gorm_recorder\"\n)\n\nfunc main() {\n    dsn := \"user:password@tcp(localhost:3306)/recorder?parseTime=true\"\n    db, err := gorm.Open(mysql.Open(dsn), \u0026gorm.Config{})\n    if err != nil {\n        log.Fatalf(\"connect: %v\", err)\n    }\n\n    rec, err := gorm_recorder.NewRecorder(db)\n    if err != nil {\n        log.Fatalf(\"build recorder: %v\", err)\n    }\n\n    err = rec.RecordRequest(context.Background(), nil, \"req1\", []byte(\"request data\"), map[string]string{\"env\": \"dev\"})\n    if err != nil {\n        log.Fatalf(\"record: %v\", err)\n    }\n\n    data, err := rec.GetRequest(context.Background(), \"req1\")\n    if err != nil {\n        log.Fatalf(\"get: %v\", err)\n    }\nfmt.Println(\"Request data:\", string(data))\n}\n```\n\nIf you need different table names or extra fields on the persisted models, implement the small `RecordModel` and `TagModel` abstractions and provide them when constructing the recorder:\n\n```go\ntype RequestRecord struct {\n    gorm.Model\n    Kind          string  `gorm:\"column:kind;size:32;not null\"`\n    CorrelationID string  `gorm:\"column:correlation_id;size:255;not null\"`\n    ReferenceID   *string `gorm:\"column:ref_id\"`\n    Body          []byte  `gorm:\"column:body;type:blob;not null\"`\n}\n\nfunc (RequestRecord) TableName() string { return \"custom_records\" }\n\nfunc (r *RequestRecord) GetID() uint            { return r.ID }\nfunc (r *RequestRecord) GetType() string        { return r.Kind }\nfunc (r *RequestRecord) SetType(v string)       { r.Kind = v }\nfunc (r *RequestRecord) GetRequestID() string   { return r.CorrelationID }\nfunc (r *RequestRecord) SetRequestID(v string)  { r.CorrelationID = v }\nfunc (r *RequestRecord) SetPrimaryID(v *string) { if v == nil { r.ReferenceID = nil; return }; tmp := *v; r.ReferenceID = \u0026tmp }\nfunc (r *RequestRecord) SetPayload(data []byte) { r.Body = append(r.Body[:0], data...) }\nfunc (r *RequestRecord) GetPayload() []byte     { return r.Body }\n\ntype RequestTag struct {\n    ID       uint   `gorm:\"primaryKey\"`\n    RecordID uint   `gorm:\"column:record_ref;index\"`\n    Key      string `gorm:\"column:tag_key\"`\n    Value    string `gorm:\"column:tag_value\"`\n}\n\nfunc (RequestTag) TableName() string { return \"custom_tags\" }\n\nfunc (t *RequestTag) SetRecordID(id uint) { t.RecordID = id }\nfunc (t *RequestTag) SetKey(k string)     { t.Key = k }\nfunc (t *RequestTag) SetValue(v string)   { t.Value = v }\n\nopts := gorm_recorder.NewOptions(func() *RequestRecord { return \u0026RequestRecord{} }, func() *RequestTag { return \u0026RequestTag{} }).\n    WithRecordTable(\"custom_records\").\n    WithRecordColumns(\"id\", \"kind\", \"correlation_id\").\n    WithTagTable(\"custom_tags\").\n    WithTagColumns(\"record_ref\", \"tag_key\", \"tag_value\")\n\nrec, err := gorm_recorder.NewRecorderWithModels(db, opts)\nif err != nil {\n    log.Fatalf(\"build recorder: %v\", err)\n}\n```\n\n### Asynchronous Methods\n\nBoth implementations support asynchronous methods via the `Async()` method:\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\n\t\"github.com/stremovskyy/recorder/file_recorder\"\n)\n\nfunc main() {\n\trec := file_recorder.NewFileRecorder(\"/path/to/store/files\").Async()\n\n\t// Record a request asynchronously\n\tresultChan := rec.RecordRequest(context.Background(), nil, \"req1\", []byte(\"request data\"), nil)\n\tif err := \u003c-resultChan; err != nil {\n\t\tlog.Fatalf(\"Failed to record request: %v\", err)\n\t}\n\n\t// Retrieve a request asynchronously\n\tdataChan := rec.GetRequest(context.Background(), \"req1\")\n\tresult := \u003c-dataChan\n\tif result.Err != nil {\n\t\tlog.Fatalf(\"Failed to get request: %v\", result.Err)\n\t}\n\tfmt.Println(\"Request data:\", string(result.Data))\n}\n```\n\n## Extending the Library\n\nImplement the `Storage` interface to back the recorder with your own persistence layer (SQL databases, object storage, message queues, etc.). Once you have a `Storage`, wrap it with\n`recorder.New(storage)` to obtain the high-level API.\n\n```go\npackage sqlrecorder\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"strings\"\n\n\t\"github.com/stremovskyy/recorder\"\n)\n\ntype SQLStorage struct {\n\tdb *sql.DB\n}\n\nfunc NewSQLRecorder(db *sql.DB) recorder.Recorder {\n\treturn recorder.New(\u0026SQLStorage{db: db})\n}\n\nfunc (s *SQLStorage) Save(ctx context.Context, record recorder.Record) error {\n\ttagsJSON, err := json.Marshal(record.Tags)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t_, err = s.db.ExecContext(\n\t\tctx,\n\t\t`INSERT INTO recordings (type, request_id, primary_id, payload, tags)\n         VALUES ($1, $2, $3, $4, $5)\n         ON CONFLICT (type, request_id)\n         DO UPDATE SET payload = EXCLUDED.payload, tags = EXCLUDED.tags`,\n\t\trecord.Type,\n\t\trecord.RequestID,\n\t\toptionalString(record.PrimaryID),\n\t\trecord.Payload,\n\t\ttagsJSON,\n\t)\n\treturn err\n}\n\nfunc (s *SQLStorage) Load(ctx context.Context, recordType recorder.RecordType, requestID string) ([]byte, error) {\n\tvar payload []byte\n\terr := s.db.QueryRowContext(\n\t\tctx,\n\t\t`SELECT payload FROM recordings WHERE type = $1 AND request_id = $2`,\n\t\trecordType,\n\t\trequestID,\n\t).Scan(\u0026payload)\n\treturn payload, err\n}\n\nfunc (s *SQLStorage) FindByTag(ctx context.Context, tag string) ([]string, error) {\n\tparts := strings.SplitN(tag, \":\", 2)\n\tif len(parts) != 2 {\n\t\treturn nil, fmt.Errorf(\"tag must be in key:value format\")\n\t}\n\n\trows, err := s.db.QueryContext(\n\t\tctx,\n\t\t`SELECT request_id FROM recordings WHERE tags @\u003e jsonb_build_object($1, $2)`,\n\t\tparts[0],\n\t\tparts[1],\n\t)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdefer rows.Close()\n\n\tvar ids []string\n\tfor rows.Next() {\n\t\tvar id string\n\t\tif err := rows.Scan(\u0026id); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tids = append(ids, id)\n\t}\n\treturn ids, rows.Err()\n}\n\nfunc optionalString(value *string) sql.NullString {\n\tif value == nil {\n\t\treturn sql.NullString{}\n\t}\n\treturn sql.NullString{String: *value, Valid: true}\n}\n```\n\nThe SQL example above stores payloads as raw bytes and tags as JSON. Adapt the schema, serialization, and tag search logic to match your database of choice.\n\n## Contributing\n\nContributions are welcome! Please follow these steps:\n\n1. Fork the repository.\n2. Create a new branch with a descriptive name.\n3. Make your changes.\n4. Commit your changes with clear commit messages.\n5. Push to your fork and submit a pull request.\n\nPlease ensure your code adheres to the standard Go formatting and includes tests for any new functionality.\n\n## License\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.\n\n## Contact\n\nFor any questions or suggestions, please open an issue on GitHub or contact the repository owner.\n\n## Acknowledgments\n\nSpecial thanks to all contributors who have helped improve this project.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstremovskyy%2Frecorder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstremovskyy%2Frecorder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstremovskyy%2Frecorder/lists"}