{"id":15394384,"url":"https://github.com/xyproto/swish","last_synced_at":"2025-10-24T05:41:04.635Z","repository":{"id":136551779,"uuid":"190179024","full_name":"xyproto/swish","owner":"xyproto","description":":white_check_mark: Optimized Swish activation function, for neural networks","archived":false,"fork":false,"pushed_at":"2023-02-25T03:16:48.000Z","size":2489,"stargazers_count":5,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-29T03:03:54.277Z","etag":null,"topics":["activation-function","neural-network","optimized","swish"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xyproto.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-06-04T10:21:47.000Z","updated_at":"2021-08-10T02:43:24.000Z","dependencies_parsed_at":null,"dependency_job_id":"2a25d7c0-7b32-432f-8e4b-146966ff255a","html_url":"https://github.com/xyproto/swish","commit_stats":{"total_commits":40,"total_committers":2,"mean_commits":20.0,"dds":"0.050000000000000044","last_synced_commit":"0412bc18544a5ebcd09fbabab8cedc5c05055adc"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyproto%2Fswish","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyproto%2Fswish/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyproto%2Fswish/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyproto%2Fswish/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xyproto","download_url":"https://codeload.github.com/xyproto/swish/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249173061,"owners_count":21224481,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["activation-function","neural-network","optimized","swish"],"created_at":"2024-10-01T15:23:26.354Z","updated_at":"2025-10-24T05:40:59.598Z","avatar_url":"https://github.com/xyproto.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Swish\n\n[![Build Status](https://travis-ci.org/xyproto/swish.svg?branch=master)](https://travis-ci.org/xyproto/swish) [![Go Report Card](https://goreportcard.com/badge/github.com/xyproto/swish)](https://goreportcard.com/report/github.com/xyproto/swish) [![GoDoc](https://godoc.org/github.com/xyproto/swish?status.svg)](https://godoc.org/github.com/xyproto/swish)\n\nAn optimized Swish activation function ([Ramachandran, Zoph and Le, 2017](https://arxiv.org/abs/1710.05941)), for neural networks.\n\n## Screenshots\n\n![](img/swish.png)\n\n![](img/sigmoid.png)\n\nThe graphs above were drawn using the program in `cmd/graph`, which uses [goterm](https://github.com/buger/goterm).\n\n## Benchmark Results\n\n### Using a `Swish` function that uses `math.Exp`\n\nFirst run:\n\n```\ngoos: linux\ngoarch: amd64\npkg: github.com/xyproto/swish\nBenchmarkSwish07-8   \t200000000\t         8.93 ns/op\nBenchmarkSwish03-8   \t200000000\t         8.95 ns/op\nPASS\nok  \tgithub.com/xyproto/swish\t5.391s\n```\n\n### Using the optimized `Swish` function that uses `exp256`\n\n```\ngoos: linux\ngoarch: amd64\npkg: github.com/xyproto/swish\nBenchmarkSwish07-8   \t2000000000\t         0.26 ns/op\nBenchmarkSwish03-8   \t2000000000\t         0.26 ns/op\nPASS\nok  \tgithub.com/xyproto/swish\t1.108s\n```\n\nThe optimized `Swish` function is **34x** faster than the one that uses `math.Exp`, and quite a bit faster than my (apparently bad) attempt at a hand-written assembly version.\n\nThe average error (difference in output value) between the optimized and non-optimized version is `+-0.0013` and the maximum error is `+-0.0024`. This is for `x` in the range `[5,3]`. See the program in `cmd/precision` for how this was calculated.\n\n```\n0.00015\n0.00001\ngoos: linux\ngoarch: amd64\npkg: github.com/xyproto/swish\nBenchmarkSwishAssembly07-8      500000000                3.63 ns/op\nBenchmarkSwishAssembly03-8      500000000                3.65 ns/op\nBenchmarkSwish07-8              2000000000               0.30 ns/op\nBenchmarkSwish03-8              2000000000               0.26 ns/op\nBenchmarkSwishPrecise07-8       200000000                9.07 ns/op\nBenchmarkSwishPrecise03-8       200000000                9.25 ns/op\nPASS\nok      github.com/xyproto/swish        11.100s\n```\n\nI have no idea why the assembly version is so slow, but `0.26 ns/op` isn't bad for a non-hand-optimized version.\n\n## General info\n\n* Version: 1.3.0\n* License: MIT\n* Author: Alexander F. Rødseth \u0026lt;xyproto@archlinux.org\u0026gt;\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxyproto%2Fswish","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxyproto%2Fswish","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxyproto%2Fswish/lists"}