{"id":17141459,"url":"https://github.com/iskandr/shiver","last_synced_at":"2025-04-13T10:20:40.864Z","repository":{"id":57466835,"uuid":"9858518","full_name":"iskandr/shiver","owner":"iskandr","description":"A multi-threaded work queue for LLVM functions","archived":false,"fork":false,"pushed_at":"2013-07-26T19:29:14.000Z","size":5898,"stargazers_count":14,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-27T01:46:36.759Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iskandr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-05-04T18:43:17.000Z","updated_at":"2018-05-02T08:48:21.000Z","dependencies_parsed_at":"2022-09-10T03:43:01.720Z","dependency_job_id":null,"html_url":"https://github.com/iskandr/shiver","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iskandr%2Fshiver","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iskandr%2Fshiver/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iskandr%2Fshiver/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iskandr%2Fshiver/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iskandr","download_url":"https://codeload.github.com/iskandr/shiver/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248696147,"owners_count":21147077,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-14T20:25:26.955Z","updated_at":"2025-04-13T10:20:40.835Z","avatar_url":"https://github.com/iskandr.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Shiver\n======\n*a shiver of sharks, biting and tearing, until all is consumed*\n\nA multi-threaded work queue for functions compiled with [llvmpy](http://http://www.llvmpy.org/). \nGive *Shiver* a function whose last argument is an index (or multiple indices) and an iteration space\n (i.e. a number of iterations, a tuple of integers, or even slice objects with start/stop/step fields), \nand shiver does all the messy plumbing of running your code in parallel. \n\nExample:\n\n```python\n\n   # we're going to fill this array with the numbers [0...9]\n   x = np.empty(10, dtype=int)\n\n   # compile an LLVM function which takes an array, and an index\n   fn1 = shiver.from_c(\"void fn1(long *x, long i) { x[i] = i;}\")   \n\n   # run fn_one_idx in parallel;\n   # - shiver will supply x's data pointer as a fixed argument to all threads \n   # - each worker thread will also get a subrange of the indices [0..9]\n   # - the numpy array 'x' will be passed in as the underlying pointer x.ctypes.data \n   shiver.parfor(fn1, niters=len(x), fixed_args = [x])\n\n   # if the function you compile returns a value, \n   # then shiver will collect those values into a result array \n   ident = shiver.from_c(\"long identity(long i) { return i; }\")\n   y = shiver.parfor(ident, 10)\n   assert (y==x).all()\n    \n   # let's do the same thing again, but here we'll explicitly convert the \n   # fixed array argument to a type LLVM understands \n   x_gv = GenericValue.pointer(x.ctypes.data)\n   shiver.parfor(fn1, niters=len(x), fixed_args = [x_gv])\n   \n   # Now we'll build a function which takes two indices which range \n   # over all pairs of integers [0..9] and [0..20] and fills x with their products\n   fn2 = shiver.from_c(\"float mult(long i, long j) { return (float) i*j; }\")\n   result_grid = shiver.parfor(fn2, (10,20)\n   assert result_grid.shape == (10,20)\n\n```\n\n\nFAQ\n----\n\n*Can I use this library to run Python code in parallel?* \n    \nSorry, no. Shiver is only useful if you're already compiling [LLVM](http://www.drdobbs.com/architecture-and-design/the-design-of-llvm/240001128) code using [llvmpy](http://www.llvmpy.org/) or if you want to use shiver's *from_c* helper to compile simple C functions. Shiver takes an LLVM function which constitutes the body of a loop (or a nesting of loops) and runs that code in parallel. It uses the Python threading API to split up your work and saves you from having to deal with [pthreads](http://www.cs.fsu.edu/~baker/realtime/restricted/notes/pthreads.html). \n\n\n*You're using Python threads, doesn't that mean you're still stuck behind the [GIL](http://stackoverflow.com/questions/1294382/what-is-a-global-interpreter-lock-gil)?*\n\nWhen Shiver calls into native code it releases the Global Interpreter Lock, allowing its threads to actually utilize all of your processors. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiskandr%2Fshiver","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fiskandr%2Fshiver","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiskandr%2Fshiver/lists"}