{"id":13471850,"url":"https://github.com/cloudfoundry/go-diodes","last_synced_at":"2025-05-14T03:07:45.211Z","repository":{"id":18947832,"uuid":"85625360","full_name":"cloudfoundry/go-diodes","owner":"cloudfoundry","description":"Diodes are ring buffers manipulated via atomics.","archived":false,"fork":false,"pushed_at":"2025-04-03T12:11:53.000Z","size":2805,"stargazers_count":472,"open_issues_count":0,"forks_count":23,"subscribers_count":30,"default_branch":"main","last_synced_at":"2025-04-06T04:01:34.787Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloudfoundry.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-03-20T20:45:16.000Z","updated_at":"2025-04-03T12:11:50.000Z","dependencies_parsed_at":"2023-02-16T13:01:10.152Z","dependency_job_id":"febd2f4b-4dee-420d-af62-90838f23e0d0","html_url":"https://github.com/cloudfoundry/go-diodes","commit_stats":{"total_commits":130,"total_committers":19,"mean_commits":6.842105263157895,"dds":0.4538461538461539,"last_synced_commit":"612558937770fa6bc5310596e516e9525525cde4"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudfoundry%2Fgo-diodes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudfoundry%2Fgo-diodes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudfoundry%2Fgo-diodes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudfoundry%2Fgo-diodes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloudfoundry","download_url":"https://codeload.github.com/cloudfoundry/go-diodes/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248675543,"owners_count":21143793,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T16:00:49.751Z","updated_at":"2025-04-13T07:19:12.662Z","avatar_url":"https://github.com/cloudfoundry.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"![diode][diode-logo]\n\n[![GoDoc][go-doc-badge]][go-doc]\n\nIf you have any questions, or want to get attention for a PR or issue please reach out on the [#logging-and-metrics channel in the cloudfoundry slack](https://cloudfoundry.slack.com/archives/CUW93AF3M)\n\nDiodes are ring buffers manipulated via atomics.\n\nDiodes are optimized for high throughput scenarios where losing data is\nacceptable. Unlike a channel, a diode will overwrite data on writes in lieu\nof blocking. A diode does its best to not \"push back\" on the producer.\nIn other words, invoking `Set()` on a diode never blocks.\n\n### Installation\n\n```bash\ngo get code.cloudfoundry.org/go-diodes\n```\n\n### Example: Basic Use\n\n```go\nd := diodes.NewOneToOne(1024, diodes.AlertFunc(func(missed int) {\n\tlog.Printf(\"Dropped %d messages\", missed)\n}))\n\n// writer\ngo func() {\n\tfor i := 0; i \u003c 2048; i++ {\n\t\t// Warning: Do not use i. By taking the address,\n\t\t// you would not get each value\n\t\tj := i\n\t\td.Set(diodes.GenericDataType(\u0026j))\n\t}\n}()\n\n// reader\npoller := diodes.NewPoller(d)\nfor {\n\ti := poller.Next()\n\tfmt.Println(*(*int)(i))\n}\n```\n\n### Example: Creating a Concrete Shell\n\nDiodes accept and return `diodes.GenericDataType`. It is recommended to not\nuse these generic pointers directly. Rather, it is a much better experience to\nwrap the diode in a concrete shell that accepts the types your program works\nwith and does the type casting for you. Here is an example of how to create a\nconcrete shell for `[]byte`:\n\n```go\ntype OneToOne struct {\n\td *diodes.Poller\n}\n\nfunc NewOneToOne(size int, alerter diodes.Alerter) *OneToOne {\n\treturn \u0026OneToOne{\n\t\td: diodes.NewPoller(diodes.NewOneToOne(size, alerter)),\n\t}\n}\n\nfunc (d *OneToOne) Set(data []byte) {\n\td.d.Set(diodes.GenericDataType(\u0026data))\n}\n\nfunc (d *OneToOne) TryNext() ([]byte, bool) {\n\tdata, ok := d.d.TryNext()\n\tif !ok {\n\t\treturn nil, ok\n\t}\n\n\treturn *(*[]byte)(data), true\n}\n\nfunc (d *OneToOne) Next() []byte {\n\tdata := d.d.Next()\n\treturn *(*[]byte)(data)\n}\n```\n\nCreating a concrete shell gives you the following advantages:\n\n- The compiler will tell you if you use a diode to read or write data of the\n  wrong type.\n- The type casting syntax in go is not common and should be hidden.\n- It prevents the generic pointer type from escaping in to client code.\n\n### Dropping Data\n\nThe diode takes an `Alerter` as an argument to alert the user code to when\nthe read noticed it missed data. It is important to note that the go-routine\nconsuming from the diode is used to signal the alert.\n\nWhen the diode notices it has fallen behind, it will move the read index to\nthe new write index and therefore drop more than a single message.\n\nThere are two things to consider when choosing a diode:\n\n1. Storage layer\n2. Access layer\n\n### Storage Layer\n\n##### OneToOne\n\nThe OneToOne diode is meant to be used by one producing (invoking `Set()`)\ngo-routine and a (different) consuming (invoking `TryNext()`) go-routine. It\nis not thread safe for multiple readers or writers.\n\n##### ManyToOne\n\nThe ManyToOne diode is optimized for many producing (invoking `Set()`)\ngo-routines and a single consuming (invoking `TryNext()`) go-routine. It is\nnot thread safe for multiple readers.\n\nIt is recommended to have a larger diode buffer size if the number of producers\nis high. This is to avoid the diode from having to mitigate write collisions\n(it will call its alert function if this occurs).\n\n### Access Layer\n\n##### Poller\n\nThe Poller uses polling via `time.Sleep(...)` when `Next()` is invoked. While\npolling might seem sub-optimal, it allows the producer to be completely\ndecoupled from the consumer. If you require very minimal push back on the\nproducer, then the Poller is a better choice. However, if you require several\ndiodes (e.g. one per connected client), then having several go-routines\npolling (sleeping) may be hard on the scheduler.\n\n##### Waiter\n\nThe Waiter uses a conditional mutex to manage when the reader is alerted\nof new data. While this method is great for the scheduler, it does have\nextra overhead for the producer. Therefore, it is better suited for situations\nwhere you have several diodes and can afford slightly slower producers.\n\n### Benchmarks\n\nThere are benchmarks that compare the various storage and access layers to\nchannels. To run them:\n\n```\ngo test -bench=. -run=NoTest\n```\n\n### Known Issues\n\nIf a diode was to be written to `18446744073709551615+1` times it would overflow\na `uint64`. This will cause problems if the size of the diode is not a power\nof two (`2^x`). If you write into a diode at the rate of one message every\nnanosecond, without restarting your process, it would take you 584.54 years to\nencounter this issue.\n\n[diode-logo]:   https://raw.githubusercontent.com/cloudfoundry/go-diodes/gh-pages/diode-logo.png\n[go-doc-badge]: https://godoc.org/code.cloudfoundry.org/go-diodes?status.svg\n[go-doc]:       https://godoc.org/code.cloudfoundry.org/go-diodes\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudfoundry%2Fgo-diodes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloudfoundry%2Fgo-diodes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudfoundry%2Fgo-diodes/lists"}