{"id":19288350,"url":"https://github.com/techascent/tech.compute","last_synced_at":"2025-12-12T01:06:23.832Z","repository":{"id":62434875,"uuid":"123461834","full_name":"techascent/tech.compute","owner":"techascent","description":"Compute abstraction for clojure","archived":false,"fork":false,"pushed_at":"2019-11-17T20:08:31.000Z","size":346,"stargazers_count":14,"open_issues_count":0,"forks_count":1,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-07-13T06:06:03.318Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"epl-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/techascent.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-03-01T16:32:37.000Z","updated_at":"2022-07-22T00:05:34.000Z","dependencies_parsed_at":"2022-11-01T20:46:00.196Z","dependency_job_id":null,"html_url":"https://github.com/techascent/tech.compute","commit_stats":null,"previous_names":["tech-ascent/tech.compute"],"tags_count":80,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techascent%2Ftech.compute","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techascent%2Ftech.compute/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techascent%2Ftech.compute/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techascent%2Ftech.compute/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/techascent","download_url":"https://codeload.github.com/techascent/tech.compute/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223888369,"owners_count":17220083,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T22:08:46.875Z","updated_at":"2025-12-12T01:06:18.792Z","avatar_url":"https://github.com/techascent.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"# tech.compute\n\n\n[![Clojars Project](https://clojars.org/techascent/tech.compute/latest-version.svg)](https://clojars.org/techascent/tech.compute)\n\n\nLibrary designed to provide a generic compute abstraction to allow\nsome level of shared implementation between a cpu, cuda, openCL,\nwebworkers, etc. design.  This system is not specific to neural networks\nnor it is specific to doing linear algebra but is rather the basis\nto implement any generic algorithm across a range of environments but\ntargetting environments where there is a distinct transfer step between\nthe main computing system (called the host) and some external compute\nsystem (called the device).  The primitives of this layer are carefully\nchosen to be implementable across a wide range of different 'devices' such\nthat a unified codebase and run unmodified in the various potential runtimes\nlisted above and this *includes* operations like:\n\n* Offsetting device buffers to create new buffers.  Thus allows efficient implementation\nof pooling algorithms or buffer coalescing where the user desires to create 1 large\nbuffer and the create a series of smaller buffers piecemeal later.  Sub-buffers are not\ndistiguishable to the implementations from the original source buffers.\n* Initializing buffers to fixed values.\n* Transfer of data between the host and device.\n* Overlapping transfer with device compute using multiple 'streams' of execution.\n* Synchronizing stream execution with either the host system or with another on-device\n  stream.\n\n\nThere is a generalized underlying compute layer that works across 6 primitive datatypes\nin the JVM - bytes, shorts, ints, longs, floats and doubles - along with 4 datatypes\n(unsigned versions of the integer types above). This compute layer contains the\nprotocols that define an interface to a compute driver, either cpu or cuda at this\ntime. This interface contains ways to move data from the host architecture to the\nspecific device architecture.\n\n\nComponents of the compute abstraction:\n\n### [driver.clj](src/tech/compute/driver.clj)\n\n(ns documentation)\n```clojure\n  \"Base set of protocols required to move information from the host to the device as well as enable some form of computation on a given device.  There is a cpu implementation provided for reference.\n\n  Base datatypes are defined:\n   * Driver: Enables enumeration of devices and creation of host buffers.\n   * Device: Creates streams and device buffers.\n   * Stream: Stream of execution occuring on the device.\n   * Buffer: Lightly typed (java primitive types at this time) heavy on-device buffer.\n```\n\n\nThis layer defines operations you would expect to find on any buffer in a C-based\nlanguage such as:\n\nBuffers allow memcpy, offsetting, upload to device and download from device.\nBuffers can be offset to produce a new (shorter) buffer with a different base address.\n\n\nThere are two concepts used in this file that are not defined:\n\n1. [resource management](https://github.com/tech-ascent/tech.resource)\n2. [datatype](https://github.com/techascent/tech.datatype)\n\nResource management allows stack-based resource algorithms similar to the RAII concept\nin c++.  This is important to ensure that using GPU resources is as pleasant and as\nforgiving as possible.\n\n\nThe datatype library allows identification and efficient copying of data into packed\nsequential buffers, including marshalling copies where we want to say copy a buffer of\nbytes into a buffer of floats.  This library is very carefully written to allow\nefficient upload/download characteristics so client code does not have unnecessary\nbottlenecks around moving data to or from a compute device while still allowing users\ncomplete flexibility with regards to their choice of data used in their systems external\nto the compute system.\n\n\nThis combination of systems (driver, datatype, and resource) are designed to work\ntogether to enable a generic computing abstraction, not specific to linear algebra or\nneural networks.  Should clojure become a language where people are expressing a wide\nrange of algorithms on the GPU, CPU, and perhaps in the web then this would be an\nappropriate substrate to enable efficient data management for those algorithms.\n\n\nThere is one default driver provided for a pure-cpu implementation\n\n1. [cpu](src/tech/compute/cpu/driver.clj)\n\n\n## License\n\nCopyright © 2018 TechAscent, LLC\n\nDistributed under the Eclipse Public License either version 1.0 or (at\nyour option) any later version.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftechascent%2Ftech.compute","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftechascent%2Ftech.compute","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftechascent%2Ftech.compute/lists"}