Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/rurumimic/apache-arrow
https://github.com/rurumimic/apache-arrow
apache cuda gpu hadoop parquet
Last synced: about 1 month ago
JSON representation
- Host: GitHub
- URL: https://github.com/rurumimic/apache-arrow
- Owner: rurumimic
- License: mit
- Created: 2024-03-15T14:15:07.000Z (11 months ago)
- Default Branch: master
- Last Pushed: 2024-03-25T05:30:40.000Z (11 months ago)
- Last Synced: 2024-11-09T05:19:12.324Z (3 months ago)
- Topics: apache, cuda, gpu, hadoop, parquet
- Language: Python
- Homepage:
- Size: 32.2 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Apache Arrow
- [arrow.apache.org](https://arrow.apache.org/)
- [github](https://github.com/apache/arrow)
- [docs](https://arrow.apache.org/docs/)
- [install](https://arrow.apache.org/install/)
## Developing inside a Container
- [containers.dev](https://containers.dev/)
- [reference](https://containers.dev/implementors/json_reference/)
- github: [devcontainers](https://github.com/devcontainers)
- [images](https://github.com/devcontainers/images)
- [features](https://github.com/devcontainers/features)
- vscode
- docs: [Developing inside a Container](https://code.visualstudio.com/docs/devcontainers/containers)
### Launch the container
Launch a Apache Arrow development environment:
1. Start VS Code
2. Open **Command Palette** with `F1`
3. Run: **Dev Containers: Open Folder in Container...**
### Hello World
- python: [Hello World](python/helloworld/README.md)
```bash
cd /workspace/python/helloworld
python -m helloworld
```
### Attach to Hadoop Nodes
```bash
docker exec -it apache-arrow_devcontainer-namenode-1 bash
docker exec -it apache-arrow_devcontainer-datanode-1 bash
```
```bash
hdfs dfs -ls -R /
drwxr-xr-x - vscode supergroup 0 2024-03-25 02:59 /user
drwxr-xr-x - vscode supergroup 0 2024-03-25 02:59 /user/vscode
-rw-r--r-- 3 vscode supergroup 1247 2024-03-25 02:59 /user/vscode/birthdays.parquet
drwxr-xr-x - vscode supergroup 0 2024-03-25 02:59 /user/vscode/output
-rw-r--r-- 3 vscode supergroup 1207 2024-03-25 02:59 /user/vscode/output/0.parquet
-rw-r--r-- 3 vscode supergroup 1207 2024-03-25 02:59 /user/vscode/output/1.parquet
-rw-r--r-- 3 vscode supergroup 1207 2024-03-25 02:59 /user/vscode/output/2.parquet
```