{"id":15540341,"url":"https://github.com/rm-hull/cljs-dataview","last_synced_at":"2025-03-29T00:15:10.201Z","repository":{"id":12454704,"uuid":"15116297","full_name":"rm-hull/cljs-dataview","owner":"rm-hull","description":"A ClojureScript library for asynchronously fetching \u0026 dicing remote binary objects","archived":false,"fork":false,"pushed_at":"2014-04-06T15:51:03.000Z","size":517,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-10-03T12:17:59.966Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rm-hull.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-12-11T19:07:26.000Z","updated_at":"2014-04-06T15:51:00.000Z","dependencies_parsed_at":"2022-09-16T04:10:26.578Z","dependency_job_id":null,"html_url":"https://github.com/rm-hull/cljs-dataview","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rm-hull%2Fcljs-dataview","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rm-hull%2Fcljs-dataview/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rm-hull%2Fcljs-dataview/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rm-hull%2Fcljs-dataview/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rm-hull","download_url":"https://codeload.github.com/rm-hull/cljs-dataview/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246117762,"owners_count":20726069,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-02T12:13:34.013Z","updated_at":"2025-03-29T00:15:10.182Z","avatar_url":"https://github.com/rm-hull.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"# cljs-dataview  [![Build Status](https://secure.travis-ci.org/rm-hull/cljs-dataview.png)](http://travis-ci.org/rm-hull/cljs-dataview)\n\nA ClojureScript library for asynchronously fetching \u0026amp; dicing remote binary objects\n\n_PREAMBLE_: TODO\n\n### Pre-requisites\n\nYou will need [Leiningen](https://github.com/technomancy/leiningen) 2.3.4 or above installed.\n\n### Building\n\nTo build and install the library locally, run:\n\n    $ lein clean\n    $ lein cljsbuild once\n    $ lein install\n\n### Testing\n\nTo run the tests in a browser, ensure the generated javascript files are up-to-date,\nand open ```resources/run-tests.html``` in a browser - this executes the tests and\nthe test results are displayed on the page.\n\nAlternatively, to run using PhantomJS, execute:\n\n    $ lein cljsbuild test\n\nThis will only show a tally of failed tests.\n\n### Including in your project\n\nThere is an 'alpha-quality' version hosted at [Clojars](https://clojars.org/rm-hull/cljs-dataview).\nFor leiningen include a dependency:\n\n```clojure\n[rm-hull/cljs-dataview \"0.0.1-SNAPSHOT\"]\n```\n\nFor maven-based projects, add the following to your `pom.xml`:\n\n```xml\n\u003cdependency\u003e\n  \u003cgroupId\u003erm-hull\u003c/groupId\u003e\n  \u003cartifactId\u003ecljs-dataview\u003c/artifactId\u003e\n  \u003cversion\u003e0.0.1-SNAPSHOT\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n## Basic Usage\n\n### Example: Reading binary STL files\n\n[Binary STL](https://en.wikipedia.org/wiki/STL_\\(file_format\\)#Binary_STL) files\ncan be read in and processed with the following code sample (see\n[example/binary_stl.cljs](https://github.com/rm-hull/cljs-dataview/blob/master/example/binary_stl.cljs)\nfor fully working example). Starting with some necessary pre-amble:\n\n```clojure\n(ns binary-stl\n  (:require [cljs.dataview.loader :refer [fetch-blob]]\n            [cljs.dataview.ops :refer [create-reader read-fixed-string read-float32-le\n                                       read-uint16-le read-uint32-le]])\n  (:require-macros [cljs.core.async.macros :refer [go]]))\n\n(def local-url \"http://localhost/~rhu/test/torus.stl\")\n```\n\nThe ```torus.stl``` contains polygons for the classic/ubiquitous 3D torus as per:\n\n![Torus](https://raw.github.com/rm-hull/wireframes/master/doc/gallery/shaded/torus.png)\n\nIn order to read the binary STL data, we must first define some decoders; so\nto create a 3D point, a ```point-spec``` generates an ordered map of _x_, _y_\nand _z_ floating-point components from a reader:\n\n```clojure\n(defn point-spec [reader]\n  (array-map\n    :x (read-float32-le reader)\n    :y (read-float32-le reader)\n    :z (read-float32-le reader)))\n```\nA _reader_ is a stateful implementation of an\n[IReader](https://github.com/rm-hull/cljs-dataview/blob/master/src/cljs/dataview/ops.cljs#L57)\nprotocol -- this has methods that traverse a javascript\n[DataView](https://developer.mozilla.org/en-US/docs/Web/API/DataView?redirectlocale=en-US\u0026redirectslug=Web%2FJavaScript%2FTyped_arrays%2FDataView)\nsequentially as bytes, 16-bit \u0026 32-bit integers, floating-point numbers and\nfixed-width/delimited strings. The reified reader object may also implement\n[IRandomAccess](https://github.com/rm-hull/cljs-dataview/blob/master/src/cljs/dataview/ops.cljs#L67)\nso that _seek_/_rewind_/_tell_ operations (similar to that used with Unix file\ndescriptors) are also available.\n\nSecondly, a triangle is composed of a [surface normal](https://en.wikipedia.org/wiki/Surface_normal),\nfollowed by 3 vertex co-ordinates and some attributes in the form of a 16-bit\nword -- the normal and the vertexes are constructed out of repeated application\nof the ```point-spec``` above. Note that since the ```point-spec``` has side-effects\nit is important to call _doall_ to force evaluation, otherwise _repeatedly_ will\nact lazily.\n\n```clojure\n(defn triangle-spec [reader]\n  (array-map\n    :normal (point-spec reader)\n    :points (doall (repeatedly 3 #(point-spec reader)))\n    :attributes (read-uint16-le reader)))\n```\nFinally, the overall STL spec header consists of 80 padded characters,\nfollowed by a triangle count: notice how this determines how many times the\n```triangle-spec``` is subsequently invoked in the body:\n\n```clojure\n(defn stl-spec [reader]\n  (array-map\n    :header (read-fixed-string reader 80)\n    :triangles (doall (repeatedly\n                 (read-uint32-le reader) ; \u003c== triangle-count\n                 #(triangle-spec reader)))))\n```\nSo in order to fetch the binary data, ```fetch-blob``` below returns a\n_core.async_ channel, from which a javascript DataView is produced. In order\nto then _sort-of_ treat the DataView object as an input stream, it is wrapped in a\nreader (by virtue of the ```create-reader``` function), which is then passed on\nto the ```stl-spec```: hence the binary data is progressively diced\ninto a persistent map structure.\n\n```clojure\n(go\n  (-\u003e\n    (\u003c! (fetch-blob local-url))\n    (create-reader)\n    (stl-spec)\n    (println)))\n```\nThe resulting output (curtailed and slightly formatted):\n\n```clojure\n{:header \"Torus, created with https://github/rm-hull/wireframes [October 16 2013]         \",\n :triangles (\n   {:normal {:x -0.9972646832466125, :y -0.05226442590355873, :z 0.05226442590355873},\n    :points ({:x 4, :y 0, :z 0}\n             {:x 3.9945218563079834, :y 0.10452846437692642, :z 0}\n             {:x 3.972639560699463, :y 0.10452846437692642, :z -0.4175412356853485}),\n    :attributes 0}\n   {:normal {:x -0.9972646832466125, :y -0.05226442590355873, :z 0.05226442590355873},\n    :points ({:x 4, :y 0, :z 0}\n             {:x 3.972639560699463, :y 0.10452846437692642, :z -0.4175412356853485}\n             {:x 3.9780876636505127, :y 0, :z -0.4181138575077057}),\n    :attributes 0}\n   {:normal {:x -0.9863678216934204, :y -0.1562253087759018, :z 0.05169334635138512},\n    :points ({:x 3.9945218563079834, :y 0.10452846437692642, :z 0}\n             {:x 3.978147506713867, :y 0.2079116851091385, :z 0}\n             {:x 3.956354856491089, :y 0.2079116851091385, :z -0.4158296585083008}),\n    :attributes 0}\n\n...\n)}\n```\n\n## TODO\n\n* ~~Implement ```read-string``` with a delimiter~~\n* Implement IWriter protocol\n* ~~Proper EOD handling/testing~~\n* [Gloss](https://github.com/ztellman/gloss)-style codecs\n* Integrate CORS handling with http://www.corsproxy.com/\n* Test framework, ~~travis integration~~ \u0026 unit tests\n* ~~Implement ```fetch-image``` in loader namespace~~\n* ~~Support proper reading UTF-8 strings in ```IReader```~~\n* Boyer-Moore searching in IRandomAccess with a ```find!``` method\n\n## Known Bugs\n\n* ~~Fix bug in binary STL example where repeatedly was not being forced.~~\n* Triangle test causes PhantomJS to segfault.\n\n## References\n\n* http://www.html5rocks.com/en/tutorials/file/xhr2/\n* https://developer.mozilla.org/en-US/docs/Web/API/DataView?redirectlocale=en-US\u0026redirectslug=Web%2FJavaScript%2FTyped_arrays%2FDataView\n* https://developer.mozilla.org/en-US/docs/Web/JavaScript/Typed_arrays?redirectlocale=en-US\u0026redirectslug=JavaScript%2FTyped_arrays\n* http://stackoverflow.com/questions/327685/is-there-a-way-to-read-binary-data-in-javascript\n* https://github.com/tbuser/thingiview.js\n* https://github.com/MarcoPolo/servant\n* http://www.ietf.org/rfc/rfc3629.txt\n* http://phpjs.org/functions/utf8_encode/\n\n## License\n\nThe MIT License (MIT)\n\nCopyright (c) 2013 Richard Hull\n\nPermission is hereby granted, free of charge, to any person obtaining a copy of\nthis software and associated documentation files (the \"Software\"), to deal in\nthe Software without restriction, including without limitation the rights to\nuse, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of\nthe Software, and to permit persons to whom the Software is furnished to do so,\nsubject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS\nFOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR\nCOPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER\nIN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\nCONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frm-hull%2Fcljs-dataview","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frm-hull%2Fcljs-dataview","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frm-hull%2Fcljs-dataview/lists"}