{"id":15061151,"url":"https://github.com/jewelltaylor/camlgrad","last_synced_at":"2025-04-10T06:36:46.499Z","repository":{"id":231586151,"uuid":"756036631","full_name":"jewelltaylor/camlgrad","owner":"jewelltaylor","description":"Toy autograd engine in OCaml with Apple Accelerate backend","archived":false,"fork":false,"pushed_at":"2024-07-31T13:11:41.000Z","size":1317,"stargazers_count":30,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-24T07:40:33.704Z","etag":null,"topics":["autograd","deep-learning","machine-learning","macos","neural-network","ocaml","tensor"],"latest_commit_sha":null,"homepage":"","language":"OCaml","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jewelltaylor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-11T19:47:43.000Z","updated_at":"2024-09-16T00:58:20.000Z","dependencies_parsed_at":"2024-04-06T03:25:37.313Z","dependency_job_id":"d87b4bf6-8e87-4e95-a223-592ed6e6263b","html_url":"https://github.com/jewelltaylor/camlgrad","commit_stats":null,"previous_names":["jewelltaylor/camlgrad"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jewelltaylor%2Fcamlgrad","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jewelltaylor%2Fcamlgrad/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jewelltaylor%2Fcamlgrad/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jewelltaylor%2Fcamlgrad/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jewelltaylor","download_url":"https://codeload.github.com/jewelltaylor/camlgrad/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248170824,"owners_count":21059260,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autograd","deep-learning","machine-learning","macos","neural-network","ocaml","tensor"],"created_at":"2024-09-24T23:09:57.586Z","updated_at":"2025-04-10T06:36:46.438Z","avatar_url":"https://github.com/jewelltaylor.png","language":"OCaml","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://github.com/VectorInstitute/forecasting-with-dl/assets/34798787/b6e54bca-522d-459d-b498-c5e971acc06d\" width=\"600\" height=\"400\" /\u003e\n\u003c/p\u003e\n\u003ch1 align=\"center\"\u003e 🐫 camlgrad 📉\n\u003c/h1\u003e\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://github.com/jewelltaylor/camlgrad/actions/workflows/unit-test.yml/badge.svg\" /\u003e\n\u003c/p\u003e\n\nInspired by [micrograd](https://github.com/karpathy/micrograd) and [tinygrad](https://github.com/tinygrad/tinygrad), **camlgrad** is a toy autograd engine in OCaml from scratch using an Apple Accelerate backend for vectorized computation. **camlgrad** offers a `Tensor` module with a wide variety of unary operations and binary operations that can be composed in arbitrary ways to define computation graphs that admit forward and backward passes on scalar valued functions. Using the `Tensor` module, several other modules are defined for calculating losses (`Loss`), defining multi-layer perceptrons (`Mlp`) and performing gradient descent (`Optimizer`).\n\n## Installation\nInstall ocaml compiler and package manager (opam) along with some useful platform tools:\n```bash\nbrew install ocaml\nopam install ocaml-lsp-server odoc ocamlformat utop\n```\n\nSubsequently, we can create a switch (an isolated ocaml environment), activate it and install the packages necessary for **camlgrad**:\n```bash\nopam switch create camlgrad 5.1\nopam switch camlgrad\nopam install ocamlgraph ctypes-foreign alcotest\n```\n\n## Usage\n**camlgrad** makes it easy to define an MLP and perform forward and backward passes. As illustrated in the code snippet below, we can simply: \n- Define an MLP containing a single layer with sigmoid activation\n- Define arbitrary input and target\n- Perform a forward pass on the MLP model\n- Calculate binary cross entropy\n- Update parameters of MLP using gradient descent with respect to the loss\n\n```ocaml\nlet mlp = Mlp.get_mlp [|(Tensor.sigmoid, (100, 1))|] in\nlet input = Tensor.random (1, 100) in\nlet target = Tensor.ones (1, 1) in\nlet (pred, _) = Mlp.mlp_forward mlp input in \nlet bce_loss = Loss.binary_cross_entropy pred target in\nOptimizer.gradient_descent mlp bce_loss 0.01;\n```\n\nIn order to visualize the computation graph, we can export a general specification of the graph into a file using: \n```ocaml\nTensor.visualize_computation_graph bce_loss \"graph.dot\";\n```\n\nTo render graph we use the [Graphviz](https://graphviz.org/) CLI:\n```bash\ndot -Tpng graph.dot -o graph.png\n```\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://github.com/VectorInstitute/forecasting-with-dl/assets/34798787/5b3430b4-81a5-4eae-af1d-03e4e8a0ae31\" width=\"600\" height=\"800\" /\u003e\n\u003c/p\u003e\n\n**Note**: Must first install graphviz with `brew install graphviz`\n\n## Apple Accelerate Details \n[Apple Accelerate](https://developer.apple.com/documentation/accelerate) is a set of API's to perform large-scale mathematical computations and image calculations, optimized for high performance and low energy consumption. The 2 APIs I leverage in particular are: \n- [BLAS](https://developer.apple.com/documentation/accelerate/blas): Perform common linear algebra operations with Apple’s implementation of the Basic Linear Algebra Subprograms (BLAS).\n- [VForce](https://developer.apple.com/documentation/accelerate/veclib/vforce): Perform transcendental and trigonometric functions on vectors of any length.\n\nThe BLAS and VForce API are in C which we can easily interface with from OCaml using the [Ctypes library](https://github.com/yallop/ocaml-ctypes). The Ctypes library lets you define the C interface in pure OCaml, and the library then takes care of loading the C symbols and invoking the foreign function call.  \n\nTo represent tensors, the [OCaml Bigarray](https://v2.ocaml.org/api/Bigarray.html) is used. Bigrarray implements multi-dimensional arrays of integers and floating-point numbers. In particular, it allows efficient sharing of large numerical arrays between OCaml code and C or Fortran numerical libraries.\n\n**camlgrad** only requires a few libraries outside the standard library ([alcotest](https://github.com/mirage/alcotest) for testing, [ctypes-foreign](https://github.com/yallop/ocaml-ctypes) for foreign function interface and [ocamlgraph](https://github.com/backtracking/ocamlgraph) for generating viusalization of computation graph). \n\n## Contributing\nIn the unlikely event someone read this far and is interested in contributing, feel free to put up an issue or pull request 😊 \u003cp align=\"center\"\u003e\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjewelltaylor%2Fcamlgrad","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjewelltaylor%2Fcamlgrad","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjewelltaylor%2Fcamlgrad/lists"}