Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/poogles/l2spark
https://github.com/poogles/l2spark
Last synced: 4 days ago
JSON representation
- Host: GitHub
- URL: https://github.com/poogles/l2spark
- Owner: Poogles
- Created: 2016-11-22T16:38:02.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2016-11-22T16:39:31.000Z (about 8 years ago)
- Last Synced: 2024-12-06T21:56:17.255Z (27 days ago)
- Language: Scala
- Size: 1000 Bytes
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Spark HDFS Interaction.
```
sam.pegler@ietl-1:~/spark-2.0.0-bin-hadoop2.7/bin$ ./spark-submit --class org.infectious.spark.ListHDFS --master local[1] /tmp/listhdfs_2.11-1.0.jar
```
Runs stuff.
```
sam.pegler@ietl-1:~/spark-2.0.0-bin-hadoop2.7/bin$ hdfs dfs -text /tmp/counts.txt/part-00000 | tail
16/11/22 16:36:29 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
(matrimony,,2)
(perseverance.,1)
(companion,12)
(confusion,,4)
(silence,,7)
(at!",1)
(aunt,",2)
(absolutely,17)
(justifying,1)
(young,123)
```
Is the output.