{"id":20515722,"url":"https://github.com/ignf/spark-iqmulus","last_synced_at":"2025-08-07T04:48:10.699Z","repository":{"id":43101197,"uuid":"46111951","full_name":"IGNF/spark-iqmulus","owner":"IGNF","description":"Spark Package to read and write PLY, LAS and XYZ lidar point clouds using Spark SQL.","archived":false,"fork":false,"pushed_at":"2018-12-19T12:00:50.000Z","size":6708,"stargazers_count":23,"open_issues_count":16,"forks_count":8,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-08-07T04:48:08.880Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/IGNF.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-11-13T09:14:44.000Z","updated_at":"2025-02-04T04:07:26.000Z","dependencies_parsed_at":"2022-08-12T10:20:14.839Z","dependency_job_id":null,"html_url":"https://github.com/IGNF/spark-iqmulus","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/IGNF/spark-iqmulus","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IGNF%2Fspark-iqmulus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IGNF%2Fspark-iqmulus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IGNF%2Fspark-iqmulus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IGNF%2Fspark-iqmulus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/IGNF","download_url":"https://codeload.github.com/IGNF/spark-iqmulus/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IGNF%2Fspark-iqmulus/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269201199,"owners_count":24377453,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-07T02:00:09.698Z","response_time":73,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T21:24:14.097Z","updated_at":"2025-08-07T04:48:10.601Z","avatar_url":"https://github.com/IGNF.png","language":"Scala","readme":"# Spark SQL IQmulus Library\n\nA library for reading and writing Lidar point cloud collections in PLY, LAS and XYZ formats from [Spark SQL](http://spark.apache.org/docs/latest/sql-programming-guide.html).\n\n[![Build Status](https://travis-ci.org/IGNF/spark-iqmulus.svg?branch=master)](https://travis-ci.org/IGNF/spark-iqmulus)\n[![codecov.io](http://codecov.io/github/IGNF/spark-iqmulus/coverage.svg?branch=master)](http://codecov.io/github/IGNF/spark-iqmulus?branch=master)\n\n## Requirements\n\nThis Spark package is for Spark 1.6.2\n\n## Usage\n\nThe `spark-iqmulus` library is published as a [Spark package](http://spark-packages.org/package/IGNF/spark-iqmulus\n), which can be added to Spark jobs launched through `spark-shell` or `spark-submit` by using the `--packages` command line option. For example, to include it when starting the spark shell:\n\n```sh\n$ bin/spark-shell --packages IGNF:spark-iqmulus:0.1.0-s_2.10\n```\n\nUnlike using `--jars`, using `--packages` ensures that this library and its dependencies will be added to the classpath. The `--packages` argument can also be used with `bin/spark-submit`. For now this library is only published for Scala 2.10.\n\n## Scala API\n\nExamples below require the creation of an SQLContext and the following imports, depending on which aspect of the library is being used:\n```scala\nval sqlContext = new SQLContext(sc)\n\nimport fr.ign.spark.iqmulus.ply._\nimport fr.ign.spark.iqmulus.las._\nimport fr.ign.spark.iqmulus.xyz._\n```\n\nReading files as Spark dataframes:\n\n```scala\nval df1 = sqlContext.read.ply(\"myfile.ply\")\nval df2 = sqlContext.read.las(\"myfile.las\")\nval df3 = sqlContext.read.xyz(\"myfile.xyz\")\n```\n\nWriting back Spark dataframes as files:\n\n\nsimilarly, for XYZ and LAS :\n```scala\nval df = ...\ndf.write.ply(\"/tmp/ply\")\ndf.write.las(\"/tmp/las\")\ndf.write.xyz(\"/tmp/xyz\")\n```\n\n## Acknowledgments\n\n[\u003cimg alt=\"IGN France\" align=\"right\" src=\"http://ign.fr/sites/all/themes/ign_portail/logo.png\"\u003e](http://www.ign.fr)\n[\u003cimg alt=\"IQmulus\" align=\"right\" src=\"http://iqmulus.eu/img/logo.svg\"\u003e](http://iqmulus.eu)\nThe development of this library is partly supported by:\n* the EU FP7 Project [IQmulus](http://iqmulus.eu), which leverages the information hidden in large heterogeneous geospatial data sets and make them a practical choice to support reliable decision making (N. ICT- 2011-318787).\n* [IGN](http://www.ign.fr), the French National Mapping Agency and its [MATIS](http://recherche.ign.fr/labos/matis) research lab.\n\n## References\nTo our knowledge this library has been used in the following research:\n\n* M. Brédif, B. Vallet, B. Ferrand. [Distributed dimensionality-based rendering of lidar point clouds](http://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XL-3-W3/559/2015/isprsarchives-XL-3-W3-559-2015.html). Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XL-3/W3, 559-564, doi:10.5194/isprsarchives-XL-3-W3-559-2015, 2015.\n[Paper](http://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XL-3-W3/559/2015/isprsarchives-XL-3-W3-559-2015.pdf)\n[Presentation](http://www.isprs-geospatialweek2015.org/workshops/geobigdata/data/p16_presentation.pdf)\n[Poster](http://www.isprs-geospatialweek2015.org/workshops/geobigdata/data/p16_poster.pdf)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fignf%2Fspark-iqmulus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fignf%2Fspark-iqmulus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fignf%2Fspark-iqmulus/lists"}