{"id":15447474,"url":"https://github.com/datadavev/isamples_cesium","last_synced_at":"2025-07-15T11:41:37.370Z","repository":{"id":83563466,"uuid":"483327772","full_name":"datadavev/isamples_cesium","owner":"datadavev","description":"Tools for generating point cloud data for isamples Cesium","archived":false,"fork":false,"pushed_at":"2022-12-08T14:25:52.000Z","size":32,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-28T08:44:15.012Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/datadavev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-04-19T16:39:08.000Z","updated_at":"2022-04-19T16:39:35.000Z","dependencies_parsed_at":"2023-07-02T23:32:03.009Z","dependency_job_id":null,"html_url":"https://github.com/datadavev/isamples_cesium","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/datadavev/isamples_cesium","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datadavev%2Fisamples_cesium","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datadavev%2Fisamples_cesium/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datadavev%2Fisamples_cesium/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datadavev%2Fisamples_cesium/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/datadavev","download_url":"https://codeload.github.com/datadavev/isamples_cesium/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datadavev%2Fisamples_cesium/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265432900,"owners_count":23764181,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-01T20:06:05.594Z","updated_at":"2025-07-15T11:41:37.311Z","avatar_url":"https://github.com/datadavev.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Generating Cesium assets from iSamples record\n\nThis repo contains tools that can be used to generate a PointCloud from iSamples records. \n\nThe basic workflow is:\n\n1. Get all points along with source, material type, specimen type, sampled feature\n2. Compute elevations for distinct locations\n3. Assign Z values to each point\n4. Generate `.las` file from points using `pdal`\n5. Upload `.las` to Cesium ion to create point tiles\n\n## Environment Setup\nUsing homebrew, install:\n\n```\nbrew install pdal\n```\n\nNote: for my installation it was necessary to create a symlink `libgdal.29.dylib`:\n\n```\ncd /usr/local/opt/gdal/lib/\nln -s libgdal.30.dylib libgdal.29.dylib\n```\n\nThis is not recommended because of the version difference. It should be reconciled when the maintainers of `pdal` update dependencies.\n\n## Get points from iSamples\n\nUse the streaming API. Requests resulting in more than 500k points requires direct connection to solr, which can be done via ssh port forward.\n\n```\nhttps://hyde.cyverse.org/isamples_central/thing/stream?fl=producedBy_resultTimeRange%20hasContextCategory%20id%20keywords%20hasMaterialCategory%20registrant%20source%20hasSpecimenCategory\u0026q=searchText%3Amoorea\u0026fq=source%3A(%22SESAR%22)\u0026rows=1000\u0026wt=jsonl\n```\n\nTo generate the sqlite database `records.sqlite`:\n\n```\npython loadpoints.py -v INFO get\n```\n\nThis process takes a long time on initial run. Subsequent runs can use a query on index time to limit the records retrieved to just those added to the index since the last download.\n\n## Compute elevations for distinct locations\n\nUse the `elevate` node project. e.g.:\n\n```\n$ export TOKEN=\"eyJhbG...\"\n$ node elevate.js -k $TOKEN -f ../isamples/source/isamples_cesium/records.sqlite\nTotal rows to calculate = 2302\nTotal pages to process = 3\nPoints remaining: 2302\nPage 1 loading...\nPage 1 computing...\nPage 1 saving...\nUpdated 1000 points\n...\nPoints remaining: 0\nDone.\n```\n\n## Create a `.las` file from points and elevation data\n\n```\npython loadpoints.py csv \u003e records.csv\npdal pipeline pipeline.json\n```\n\n### `las` property mapping\n\nThese are the only per-point properties available in Cesium `las` to point cloud conversion:\n\n| las | bytes | Source |\n| --- | --- | ---- |\n| X | long | longitude |\n| Y | long | latitude |\n| Z | long | surface elevation + n*COUNT_SCALE |\n| classification | unsigned char  | data source |\n| intensity | unsigned short |  |\n| red | unsigned short |  |\n| green | unsigned short |  |\n| blue | unsigned short |  |\n\n```\nlong: −2,147,483,647, +2,147,483,647\nunsigned char: 0 - 255\nunsigned short: 0 - 65,535\n```\n\n$intensity = scale(sample_date - epoch)$\n```python\n\u003e\u003e\u003e epoch = datetime.datetime(2029,12,31).toordinal()-65535\n\u003e\u003e\u003e epoch\n675542\n\u003e\u003e\u003e datetime.datetime.fromordinal(epoch)\ndatetime.datetime(1850, 7, 28, 0, 0)\n```\n\nMaterial type has 19 distinct values. Sampled Feature has 18. Specimen Type has 16. 19 needs 5 bits. Using a full 5 bits provides a maximum value of 31. So each of the vocabularies could be expressed in 5 bits, so all three could be provided by a single unsigned short value.\n\n```\nencoded = v0 + (v1 \u003c\u003c 5) + (v2 \u003c\u003c 5)\n\nv0 = encoded \u0026 31\nv1 = encoded \u0026 (31 \u003c\u003c 5)\nv2 = encoded \u0026 (31 \u003c\u003c 10)\n```\n\n## Upload to Cesium and view result\n\n\nMinimal cesium viewer for records, adjust the asset id:\n```javascript\n// Grant CesiumJS access to your ion assets\nCesium.Ion.defaultAccessToken = \"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJqdGkiOiIwNzk3NjkyMy1iNGI1LTRkN2UtODRiMy04OTYwYWE0N2M3ZTkiLCJpZCI6Njk1MTcsImlhdCI6MTYzMzU0MTQ3N30.e70dpNzOCDRLDGxRguQCC-tRzGzA-23Xgno5lNgCeB4\";\n\nconst worldTerrain = Cesium.createWorldTerrain({\n    //requestWaterMask: true,\n    //requestVertexNormals: true,\n});\n\nconst viewer = new Cesium.Viewer(\"cesiumContainer\",\n  {\n    terrainProvider: worldTerrain,\n  }\n);\n\n\nfunction addPointsBySource(assetId) {\n  const tileset = viewer.scene.primitives.add(new Cesium.Cesium3DTileset({\n    url: Cesium.IonResource.fromAssetId(assetId),\n    depthFailMaterial: new Cesium.PolylineOutlineMaterialProperty(\n        {\n          color: Cesium.Color.RED,\n        }\n      ),            \n  }));\n  tileset.style = new Cesium.Cesium3DTileStyle({\n    color: {\n      conditions: [\n          [\"${Classification} === 0\", \"color('purple')\"],\n          [\"${Classification} === 1\", \"color('brown')\"],\n          [\"${Classification} === 2\", \"color('cyan')\"],\n          [\"${Classification} === 3\", \"color('orange')\"],\n          [\"${Classification} === 4\", \"color('green')\"],              \n          [\"true\", \"color('white')\"]\n      ]\n    },\n    pointSize: 5,\n    zIndex: 100,\n  });\n  return tileset;\n}\n\n\nconst tileset = addPointsBySource(897325);\n\n(async () =\u003e {\n  try {\n    await tileset.readyPromise;\n    await viewer.zoomTo(tileset);\n\n    // Apply the default style if it exists\n    var extras = tileset.asset.extras;\n    if (\n      Cesium.defined(extras) \u0026\u0026\n      Cesium.defined(extras.ion) \u0026\u0026\n      Cesium.defined(extras.ion.defaultStyle)\n    ) {\n      tileset.style = new Cesium.Cesium3DTileStyle(extras.ion.defaultStyle);\n    }\n  } catch (error) {\n    console.log(error);\n  }\n})();\n\n```\n\n### Selecting points\n\nYou can't select individual points in a point cloud. However, we can fake it by styling to show points in the vicinity of the cursor:\n\n```\n1. get the point of interest (cursor position in space)\n2. set color / size using distance to POI\n3. Use the coordinates of the cursor in earth coordinates to query the index\n```\n\n## References\n\n* [Discussion on las properties available](https://community.cesium.com/t/available-variables-from-las-file/15884/3)\n* [Example for point cloud styling](https://sandcastle.cesium.com/?src=3D%20Tiles%20Point%20Cloud%20Styling.html\u0026label=3D%20Tiles)\n* [LAS file format](http://www.asprs.org/wp-content/uploads/2019/03/LAS_1_4_r14.pdf)\n* [PDAL LAS writer](https://pdal.io/stages/writers.las.html#writers-las)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatadavev%2Fisamples_cesium","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatadavev%2Fisamples_cesium","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatadavev%2Fisamples_cesium/lists"}