https://github.com/juliareinforcementlearning/tfrecord.jl
A pure Julia implementation to read/write TFRecord
https://github.com/juliareinforcementlearning/tfrecord.jl
dataset julia machine-learning tensorflow tfrecords
Last synced: 5 months ago
JSON representation
A pure Julia implementation to read/write TFRecord
- Host: GitHub
- URL: https://github.com/juliareinforcementlearning/tfrecord.jl
- Owner: JuliaReinforcementLearning
- License: mit
- Created: 2020-10-14T15:18:25.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2022-10-07T03:07:12.000Z (about 3 years ago)
- Last Synced: 2024-04-13T21:20:21.379Z (over 1 year ago)
- Topics: dataset, julia, machine-learning, tensorflow, tfrecords
- Language: Julia
- Homepage:
- Size: 57.6 KB
- Stars: 5
- Watchers: 2
- Forks: 6
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# TFRecord
## Usage
### Install
```julia
julia> ] add TFRecord
```
### Write TFRecord
```julia
using TFRecord
n = 10
f1 = rand(Bool, n)
f2 = rand(1:5, n)
f3 = rand(("cat", "dog", "chicken", "horse", "goat"), n)
f4 = rand(Float32, n)
TFRecord.write(
"example.tfrecord",
(
Dict(
"feature1" => f1[i],
"feature2" => f2[i],
"feature3" => f3[i],
"feature4" => f4[i],
)
for i in 1:n
)
)
```
Here we write `10` observations into the file `example.tfrecord`. Internally each dictionary is converted into a `TFRecord.Example` first, which is a known prototype by TensorFlow. Note that the type of key must be `AbstractString` and the type of value can be one of the following types:
- `Bool`, `Int64`, `Float32`, `AbstractString`
- `Vector` of the above types
For customized data types, you need to convert it into `TFRecord.Example` first.
### Read TFRecord
```julia
for example in TFRecord.read("example.tfrecord")
println(example)
end
```
Please refer to `test/runtest.jl` to get the content encoded in the `example`.
For more fine-grained control, please read the doc:
```julia
julia> ? TFRecord.read
julia> ? TFRecord.write
```