{"id":24483169,"url":"https://github.com/sukiboo/nn_approximation_challenge","last_synced_at":"2025-10-16T17:03:35.070Z","repository":{"id":128451779,"uuid":"468914145","full_name":"sukiboo/nn_approximation_challenge","owner":"sukiboo","description":"Challenge of shallow neural network approximation with one-dimensional input.","archived":false,"fork":false,"pushed_at":"2022-05-31T14:28:53.000Z","size":18,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-14T19:52:27.953Z","etag":null,"topics":["approximation-theory","shallow-neural-network","supervised-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sukiboo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-03-11T22:12:58.000Z","updated_at":"2023-12-10T19:17:28.000Z","dependencies_parsed_at":null,"dependency_job_id":"fcf6085a-27b7-4754-a062-a60a78b1d318","html_url":"https://github.com/sukiboo/nn_approximation_challenge","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sukiboo/nn_approximation_challenge","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sukiboo%2Fnn_approximation_challenge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sukiboo%2Fnn_approximation_challenge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sukiboo%2Fnn_approximation_challenge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sukiboo%2Fnn_approximation_challenge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sukiboo","download_url":"https://codeload.github.com/sukiboo/nn_approximation_challenge/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sukiboo%2Fnn_approximation_challenge/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278903034,"owners_count":26065786,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-08T02:00:06.501Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["approximation-theory","shallow-neural-network","supervised-learning"],"created_at":"2025-01-21T12:31:11.380Z","updated_at":"2025-10-08T06:54:48.876Z","avatar_url":"https://github.com/sukiboo.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Shallow Network Approximation Challenge\n\nIn this challenge your goal is to obtain a shallow neural network (preferably of a reasonable size) that approximates the data provided in `./data.npy`.\n\nTraining and test sets are loaded with ```(x_train, y_train), (x_test, y_test) = np.load('./data.npy', allow_pickle=True)``` and consist of pairs of numbers `(x,y)` that represent the input and the output respectively.\n\nFor clarity's sake, the data is sampled from the function `cos(10*pi*x) * (1 - x**2)` on the interval `[-1,1]`, though it is not immediately relevant to this challenge.\n\nIf you manage to obtain a shallow network with a decent performance, e.g. the mse-loss on the test set of about `0.01`, please let me know how you did it!\n\n\n## Setup\nInstall the dependancies with `pip install -r requirements.txt`, then run with `python main.py`.\n\n## Discussion\nDespite a simple formulation, training a shallow network that approximates the given data is surprisingly difficult.\nThe reasons for the poor performance provided by conventional methods lies in the parameterization of shallow neural networks and the backpropagation-based optimization.\nIf you're interested in the underlying justification, reach out to me and I'll tell you what I think (*though I might be wrong*)!\n\n![This is what a conventional approach typically provides](https://user-images.githubusercontent.com/38059493/158447963-f679f9a4-a061-4ea5-825a-661b201f3f97.png)\n\n## References\nHere are some papers that discuss this phenomenon in more details:\n- [Deep ReLU Networks Have Surprisingly Few Activation Patterns](https://arxiv.org/abs/1906.00904)\n- [The gap between theory and practice in function approximation with deep neural networks](https://arxiv.org/abs/2001.07523)\n- [Training ReLU networks to high uniform accuracy is intractable](https://arxiv.org/abs/2205.13531)\n- [Greedy Shallow Networks: An Approach for Constructing and Training Neural Networks](https://arxiv.org/abs/1905.10409)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsukiboo%2Fnn_approximation_challenge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsukiboo%2Fnn_approximation_challenge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsukiboo%2Fnn_approximation_challenge/lists"}