Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/spotify/tfreader
TensorFlow TFRecord reader CLI tool
https://github.com/spotify/tfreader
cats cli fs2 graal-native scala tensorflow tfrecords
Last synced: about 2 hours ago
JSON representation
TensorFlow TFRecord reader CLI tool
- Host: GitHub
- URL: https://github.com/spotify/tfreader
- Owner: spotify
- License: apache-2.0
- Created: 2020-01-29T22:08:41.000Z (almost 5 years ago)
- Default Branch: main
- Last Pushed: 2024-04-13T18:02:26.000Z (7 months ago)
- Last Synced: 2024-04-14T05:11:34.779Z (7 months ago)
- Topics: cats, cli, fs2, graal-native, scala, tensorflow, tfrecords
- Language: Scala
- Homepage:
- Size: 1.01 MB
- Stars: 60
- Watchers: 19
- Forks: 16
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# tfreader [](https://github.com/spotify/tfreader/actions/workflows/build.yml)
Simple native CLI tool to read `TensorFlow` `TFRecords`.
## Install
### MacOs
```bash
brew tap spotify/public
brew install tfreader
```
### Linux
Right now we only have binaries available under [releases](https://github.com/spotify/tfreader/releases)
## Usage
```bash
Usage: tfr [options]
TensorFlow TFRecord reader CLI tool
Options:
-c, --check-crc32 Enable checks CRC32 on each record
-f, --flat Output examples as flat JSON objects
-n, --number Number of records to output
-r, --record Record type to be read { example | prediction_log }
-h, --help Show help message
trailing arguments:
files (not required) files? | STDIN
```
## Examples
#### Google Cloud Storage
```bash
tfr -n 1 gs:////part-00000-of-00004.tfrecords | jq .
```
#### Local Filesystem
```bash
tfr -n 1 core/src/test/resources/part-00000-of-00004.tfrecords | jq .
```
#### `stdin`
```bash
cat core/src/test/resources/part-00000-of-00004.tfrecords | tfr -n 1 | jq .
```
### Output
#### Flat
```json
{
"tips": [0],
"trip_seconds": [60],
"payment_type": ["Cash"],
"trip_miles": [0],
"dropoff_longitude": [-87.63785],
"dropoff_latitude": [41.893215],
"pickup_longitude": [-87.63187],
"pickup_latitude": [41.89204],
"trip_start_timestamp": [1402934400],
"trip_start_day": [2],
"trip_start_hour": [16],
"trip_start_month": [6],
"fare": [3.25],
"dropoff_census_tract": ["17031081800"],
"dropoff_community_area": ["8"],
"pickup_community_area": ["8"],
"trip_id": ["8106c1f6-e6f3-426f-9aaf-b4e9703b4f10"]
}
```
#### Default
```json
{
"features": {
"feature": {
"tips": {
"floatList": {
"value": [0]
}
},
"trip_seconds": {
"int64List": {
"value": [60]
}
},
"payment_type": {
"bytesList": {
"value": ["Cash"]
}
},
"trip_miles": {
"floatList": {
"value": [0]
}
},
"dropoff_longitude": {
"floatList": {
"value": [-87.63785]
}
},
"dropoff_latitude": {
"floatList": {
"value": [41.893215]
}
},
"pickup_longitude": {
"floatList": {
"value": [-87.63187]
}
},
"pickup_latitude": {
"floatList": {
"value": [41.89204]
}
},
"trip_start_timestamp": {
"int64List": {
"value": [1402934400]
}
},
"trip_start_day": {
"int64List": {
"value": [2]
}
},
"trip_start_hour": {
"int64List": {
"value": [16]
}
},
"trip_start_month": {
"int64List": {
"value": [6]
}
},
"fare": {
"floatList": {
"value": [3.25]
}
},
"dropoff_census_tract": {
"bytesList": {
"value": ["17031081800"]
}
},
"dropoff_community_area": {
"bytesList": {
"value": ["8"]
}
},
"pickup_community_area": {
"bytesList": {
"value": ["8"]
}
},
"trip_id": {
"bytesList": {
"value": ["8106c1f6-e6f3-426f-9aaf-b4e9703b4f10"]
}
}
}
}
}
```