Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
https://github.com/mettekou/FSharp.Data.Tdms

TDMS 2.0 support for F# and C#
https://github.com/mettekou/FSharp.Data.Tdms
dotnet fsharp tdms typeprovider
Last synced: 4 months ago
JSON representation
TDMS 2.0 support for F# and C#
Host: GitHub
URL: https://github.com/mettekou/FSharp.Data.Tdms
Owner: mettekou
License: mit
Created: 2019-01-09T17:10:40.000Z (over 5 years ago)
Default Branch: master
Last Pushed: 2022-12-26T12:18:41.000Z (over 1 year ago)
Last Synced: 2024-02-27T04:52:53.110Z (4 months ago)
Topics: dotnet, fsharp, tdms, typeprovider
Language: F#
Homepage: https://mettekou.github.io/FSharp.Data.Tdms/
Size: 21.3 MB
Stars: 12
Watchers: 1
Forks: 2
Open Issues: 6
Metadata Files:
- Readme: README.md
- License: LICENSE
Lists

awesome-fsharp - FSharp.Data.Tdms ★ 0 ⧗ 1 - TDMS support for F# [MIT] (Type providers / Performance analysis)
README

        # `FSharp.Data.Tdms` [![](https://buildstats.info/nuget/FSharp.Data.Tdms?includePreReleases=true)](https://www.nuget.org/packages/FSharp.Data.Tdms)

`FSharp.Data.Tdms` provides support for TDMS 2.0 files<<#the-ni-tdms-file-format-entry,^\[1\]>>

from F# and C# on .NET 6 and later. From F# it allows you to access these in a

type-safe manner through a generative [type

provider](https://docs.microsoft.com/en-us/dotnet/fsharp/tutorials/type-providers/),

while plain old functions and methods are available to both F# and C#.

`FSharp.Data.Tdms` reads raw channel data from TDMS 2.0 files as arrays.

## Missing features

- Reading DAQmx raw data

- Defragmenting TDMS files

- Writing TDMS files

## License

`FSharp.Data.Tdms` is licensed under an MIT License. Please refer to

[`LICENSE`](https://github.com/mettekou/FSharp.Data.Tdms/blob/master/LICENSE)

for its full text.

## Installation

### .NET CLI

To add `FSharp.Data.Tdms` to a project using the .NET CLI, run the command:

```sh

dotnet add package FSharp.Data.Tdms --version 1.0.0-alpha.111

```

Be sure to replace `1.0.0-alpha.111` by the version you want to install.

### Interactive

For interactive use, either in C# or F#, add the following directive to your script file or enter it into your REPL (e.g. `dotnet fsi`):

```fsharp

#r "nuget: FSharp.Data.Tdms, 1.0.0-alpha.111"

```

Be sure to replace `1.0.0-alpha.111` by the version you want to use.

## Usage

### From F#

Using the generative type provider, given a TDMS file `Experiment.tdms`

with a channel containing floating-point values named `Channel1` within

a group named `Group1`:

```fsharp

#r "nuget: FSharp.Data.Tdms"

open FSharp.Data

[]

let Path = "Experiment.tdms"

type Experiment = TdmsProvider

let experiment = Experiment(Path)

experiment.Group1.Channel1.Data |> Array.iter (printfn "%f")

```

Alternatively, use the functions in the `FSharp.Data.Tdms` namespace,

mainly those in the `File` module:

```fsharp

#r "nuget: FSharp.Data.Tdms"

open FSharp.Data.Tdms

File.read "Experiment.tdms" true

|> File.tryRawData "Group1" "Channel1"

|> Option.defaultValue [||]

|> Array.iter (printfn "%f")

```

These functions have asynchronous equivalents:

```fsharp

#r "nuget: FSharp.Data.Tdms"

open FSharp.Data.Tdms

task {

    let! file = File.readAsync "Experiment.tdms" true

    let! data = File.tryRawDataAsync "Group1" "Channel1"

    Option.defaultValue data

    |> Array.iter (printfn "%f")

}

|> Async.AwaitTask

|> Async.RunSynchronously

```

### From C#

When using `FSharp.Data.Tdms` from C#, prefer the API for idiomatic C#:

```csharp

using System;

using FSharp.Data.Tdms;

class Program

{

    static void Main(string[] args)

    {

        File.Read("Experiment.tdms", true).TryGetRawData("Group1", "Channel1", out double[] data);

        foreach (double sample in data)

        {

            Console.WriteLine(sample);

        }

    }

}

```

Or asynchronously:

```csharp

using System;

using FSharp.Data.Tdms;

class Program

{

    static async Task Main(string[] args)

    {

        var file = await File.ReadAsync("Experiment.tdms", true);

        var data = await file.TryGetRawDataAsync("Group1", "Channel1");

        foreach (double sample in data)

        {

            Console.WriteLine(sample);

        }

    }

}

```

### Property and raw data types

Most TDMS 2.0 data types directly map to .NET data types. The first

exception is `tdsTypeTimeStamp`, which you can read as either

[`FSharp.Data.Tdms.Timestamp`](FSharp.Data.Tdms/Timestamp.fs) (this data

type corresponds to [the NI LabVIEW

timestamp](https://www.ni.com/nl-be/support/documentation/supplemental/08/labview-timestamp-overview.html)),

`System.DateTime`, `System.DateTimeOffset`, or `System.TimeSpan`. In

case of `System.TimeSpan`, `FSharp.Data.Tdms` returns the time elapsed

since `01/01/1904 00:00:00.00 UTC`, as per [this support document from

NI](https://www.ni.com/nl-be/support/documentation/supplemental/08/labview-timestamp-overview.html).

The second exception is `tdsTypeExtendedFloat`. Since .NET do not

support 80-bit extended precision floating point numbers,

`FSharp.Data.Tdms` reads these as

[`FSharp.Data.Tdms.Extended`](FSharp.Data.Tdms/Extended.fs) values.

Mapping from TDMS 2.0 to .NET data types in

FSharp.Data.Tdms

Name

TDMS 2.0 data type

.NET data type

F# alias

C# alias


Void

tdsTypeVoid

FSharp.Core.Unit

unit

None


8-bit signed integer

tdsTypeI8

System.SByte

int8

sbyte


16-bit signed integer

tdsTypeI16

System.Int16

int16

short


32-bit signed integer

tdsTypeI32

System.Int32

int

int


64-bit signed integer

tdsTypeI64

System.Int64

int64

long


8-bit unsigned integer

tdsTypeU8

System.Byte

uint8

byte


16-bit unsigned integer

tdsTypeU16

System.UInt16

uint16

ushort


32-bit unsigned integer

tdsTypeU32

System.UInt32

uint

uint


64-bit unsigned integer

tdsTypeU64

System.UInt64

uint64

ulong


32-bit single-precision floating

point



tdsTypeSingleFloat

tdsTypeSingleFloatWithUnit



System.Single

float32

float


64-bit double-precision floating

point



tdsTypeDoubleFloat

tdsTypeDoubleFloatWithUnit



System.Double

float

double


80-bit extended-precision floating

point



tdsTypeExtendedFloat

tdsTypeExtendedFloatWithUnit



FSharp.Data.Tdms.Extended

float80

None


Character string

tdsTypeString

System.String

string

string


Boolean

tdsTypeBoolean

System.Boolean

bool

bool


Timestamp

tdsTypeTimeStamp



FSharp.Data.Tdms.Timestamp

System.DateTime

System.DateTimeOffset

System.TimeSpan



None

None


32-bit single-precision floating point

complex

tdsTypeComplexSingleFloat

System.ValueTuple<System.Single, System.Single>

struct (float32 * float32)

(float, float)


64-bit double-precision floating point

complex

tdsTypeComplexDoubleFloat

System.Numerics.Complex

None

None


## Performance

The [BenchmarkDotNet](https://benchmarkdotnet.org) benchmarks in this

section give an idea of the performance of `FSharp.Data.Tdms` when

compared to [`TDMSReader`](https://github.com/mikeobrien/TDMSReader),

the only other TDMS 2.0 implementation which works on .NET 6 and later. Since

`TDMSReader` does not support reading TDMS index files, the benchmark

disables this feature for `FSharp.Data.Tdms` as well, for a fair

comparison. This means that `FSharp.Data.Tdms` may perform better in

practice for TDMS files with many raw data segments.

### Small file

This benchmark reads 30,489 double-precision floating points from a segmented 3.1 MB TDMS 2.0 file.

```ini

BenchmarkDotNet=v0.13.2, OS=opensuse-tumbleweed 20221223

AMD Ryzen 9 5950X, 1 CPU, 32 logical and 16 physical cores

.NET SDK=7.0.101

  [Host]   : .NET 7.0.1 (7.0.122.56804), X64 RyuJIT AVX2 DEBUG

  .NET 7.0 : .NET 7.0.1 (7.0.122.56804), X64 RyuJIT AVX2

Job=.NET 7.0  Runtime=.NET 7.0

```

| Method              |       Mean |    Error |   StdDev | Ratio |

| ------------------- | ---------: | -------: | -------: | ----: |

| TDMSReader          | 1,677.4 μs |  3.55 μs |  2.96 μs |  1.00 |

| FSharpDataTdms      |   741.4 μs |  4.04 μs |  3.78 μs |  0.44 |

| FSharpDataTdmsAsync |   882.0 μs | 13.53 μs | 12.65 μs |  0.52 |

### Medium-sized file

This benchmark reads a channel of 43,200 strings from a segmented 138.1 MB TDMS 2.0 file.

```ini

BenchmarkDotNet=v0.13.2, OS=opensuse-tumbleweed 20221223

AMD Ryzen 9 5950X, 1 CPU, 32 logical and 16 physical cores

.NET SDK=7.0.101

  [Host]   : .NET 7.0.1 (7.0.122.56804), X64 RyuJIT AVX2 DEBUG

  .NET 7.0 : .NET 7.0.1 (7.0.122.56804), X64 RyuJIT AVX2

Job=.NET 7.0  Runtime=.NET 7.0

```

| Method              |    Mean |    Error |   StdDev | Ratio |

| ------------------- | ------: | -------: | -------: | ----: |

| TDMSReader          | 8.570 s | 0.0539 s | 0.0505 s |  1.00 |

| FSharpDataTdms      | 2.581 s | 0.0105 s | 0.0098 s |  0.30 |

| FSharpDataTdmsAsync | 2.691 s | 0.0526 s | 0.0516 s |  0.31 |

### Large file

This benchmark reads a channel of 779,297 double-precision floating points from a segmented 1.54 GB TDMS 2.0 file.

```ini

BenchmarkDotNet=v0.13.2, OS=opensuse-tumbleweed 20221223

AMD Ryzen 9 5950X, 1 CPU, 32 logical and 16 physical cores

.NET SDK=7.0.101

  [Host]   : .NET 7.0.1 (7.0.122.56804), X64 RyuJIT AVX2 DEBUG

  .NET 7.0 : .NET 7.0.1 (7.0.122.56804), X64 RyuJIT AVX2

Job=.NET 7.0  Runtime=.NET 7.0

```

| Method              |     Mean |   Error |  StdDev | Ratio |

| ------------------- | -------: | ------: | ------: | ----: |

| TDMSReader          | 996.0 ms | 8.77 ms | 7.77 ms |  1.00 |

| FSharpDataTdms      | 522.1 ms | 5.81 ms | 5.43 ms |  0.52 |

| FSharpDataTdmsAsync | 654.5 ms | 6.84 ms | 6.07 ms |  0.66 |

## How to contribute

Imposter syndrome disclaimer: I want your help. No really, I do.

There might be a little voice inside that tells you you’re not ready;

that you need to do one more tutorial, or learn another framework, or

write a few more blog posts before you can help me with this project.

I assure you, that’s not the case.

And you don’t just have to write code. You can help out by writing

documentation, tests, or even by giving feedback about this work. (And

yes, that includes giving feedback about the contribution guidelines.)

Thank you for contributing!

## References

<<#the-ni-tdms-file-format,^\[1\]>>

National Instruments. 2019. The NI TDMS File Format. (January 2019).

Retrieved January 12, 2019 from

`http://www.ni.com/white-paper/3727/en/`.