{"id":15066293,"url":"https://github.com/alexeytochin/tf-dataclass","last_synced_at":"2026-01-03T03:43:20.273Z","repository":{"id":57474798,"uuid":"380743781","full_name":"alexeytochin/tf-dataclass","owner":"alexeytochin","description":"Support python dataclass containers as input and output in callable TensorFlow graph for tensorflow version \u003e= 2.0.0.","archived":false,"fork":false,"pushed_at":"2022-08-11T17:24:52.000Z","size":19,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-23T11:45:49.913Z","etag":null,"topics":["autograph","dataclass","dataclasses","deep-learning","nested","nested-objects","nested-structures","python","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alexeytochin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-06-27T13:14:34.000Z","updated_at":"2022-04-07T13:01:22.000Z","dependencies_parsed_at":"2022-09-10T04:05:02.599Z","dependency_job_id":null,"html_url":"https://github.com/alexeytochin/tf-dataclass","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexeytochin%2Ftf-dataclass","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexeytochin%2Ftf-dataclass/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexeytochin%2Ftf-dataclass/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexeytochin%2Ftf-dataclass/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alexeytochin","download_url":"https://codeload.github.com/alexeytochin/tf-dataclass/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243814893,"owners_count":20352038,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autograph","dataclass","dataclasses","deep-learning","nested","nested-objects","nested-structures","python","tensorflow"],"created_at":"2024-09-25T01:05:10.115Z","updated_at":"2026-01-03T03:43:20.242Z","avatar_url":"https://github.com/alexeytochin.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# tf-dataclass\n\nSupport python dataclass containers as input and output in callable TensorFlow 2 graph.\n\n## Install\nMake sure that ```tensorflow\u003e=2.0.0``` or ```tensorflow-gpu\u003e=2.0.0``` is installed.\n```bash\n$ pip install tf-dataclass\n```\n\n## Why\nTensorFlow 2 \n[autograph function](https://www.tensorflow.org/api_docs/python/tf/function)\nsupports only nested structures of python tuples as inputs and output.\n(Outputs can be also python dictionaries.)\nThis is inconvenient once we go beyond small hello world cases,\nbecause we have to work with unstructured armfuls of tensors.\nThis small package is dedicated to fill this gap by letting \n```@tf.function``` \ndecorated functions to take and return pythonic\n```dataclass``` \ninstancies. \n\n## Examples of usage:\n### 1. Sequential features\n\n```python\nimport tensorflow as tf\nimport tf_dataclass\n\n# Batch of sequential features of different length\n@tf_dataclass.dataclass\nclass Sequential:\n    feature: tf.Tensor  # shape = [batch, length, channels],    dtype = tf.float32\n    length: tf.Tensor   # shape = [batch],                      dtype = tf.int32\n\n# Initialize a batch of two sequences of lengths 6 and 4\ninput = Sequential(\n    feature = tf.random.normal(shape=[2, 6, 3]),\n    length = tf.constant([6, 4], dtype=tf.int32),\n)\n    \n# Define a convolution operator with a stride such that length -\u003e length / stride\n@tf_dataclass.function\ndef convolution(input: Sequential, filters: tf.Tensor, stride: int) -\u003e Sequential:\n    return Sequential(\n        feature = tf.nn.conv1d(input.feature, filters, stride),\n        length = tf.math.floordiv(input.length, stride),\n    )\n\n# Output is an instance of Sequential with lengths 3 and 2 due to convolution stride = 2\noutput = convolution(\n    input = input,\n    filters = tf.random.normal(shape=[1, 3, 7]),\n    stride = 2,\n)\nassert isinstance(output, Sequential)\nprint(output.length) # -\u003e tf.Tensor([3 2], shape=(2,), dtype=int32)\n```\n\n### 2. Minibatch as a data transfer object:\n```python\nimport tensorflow as tf\nimport tf_dataclass\n\n@tf_dataclass.dataclass\nclass DataBatch:\n    image: tf.Tensor            # shape = [batch, height, width, channels], dtype = tf.flaot32\n    label: tf.Tensor            # shape = [batch],                          dtype = tf.int32\n    image_file_path: tf.Tensor  # shape = [batch],                          dtype = tf.string\n    dataset_name: tf.Tensor     # shape = [batch],                          dtype = tf.string\n    ...\n    \n@tf_dataclass.function\ndef train_step(input: DataBatch) -\u003e None:\n    ...\n```\n\n### 3. Containerized outputs:\n```python\nimport tensorflow as tf\nimport tf_dataclass\n\n@tf_dataclass.dataclass\nclass ModelOutput:\n    loss_value: tf.Tensor   # shape = [batch],  dtype = tf.flaot32\n    label: tf.Tensor        # shape = [batch],  dtype = tf.int32\n    prediction: tf.Tensor   # shape = [batch],  dtype = tf.int32\n    ...\n    \n    @property\n    def mean_loss(self) -\u003e tf.Tensor: # shape = [batch],  dtype = tf.float32\n        return tf.reduce_mean(self.loss_value)\n    \n    @property\n    def num_true_predictions(self) -\u003e tf.Tensor: # shape = [batch],  dtype = tf.int32\n        return tf.reduce_sum(tf.cast(self.label == self.prediction, dtype=tf.int32))\n\n    @property\n    def num_false_predictions(self) -\u003e tf.Tensor: # shape = [batch],  dtype = tf.int32\n        return tf.reduce_sum(tf.cast(self.label != self.prediction, dtype=tf.int32))\n\n    ...\n\n@tf_dataclass.function\ndef get_loss(...) -\u003e ModelOutput:\n    ...\n```\nSuch containers can be merged along datasets and workers.\n\n### 4. Easy tensorflow shape and dtype runtime verification:\n```python\nimport tensorflow as tf\nimport tf_dataclass\n\n@tf_dataclass.dataclass\nclass Sequential:\n    feature: tf.Tensor  # shape = [batch, length, channels], dtype = tf.flaot32\n    length: tf.Tensor   # shape = [batch]                    dtype = tf.int32\n\n    def __post_init__(self):\n        # Verify feature\n        assert self.feature.dtype == tf.float32\n        assert len(self.feature.shape) == 3\n        \n        # Verify length\n        assert self.length.dtype == tf.int32\n        assert len(self.length.shape) == 1\n        \n        # Verify batch size\n        # Works only in eager mode for better perfomance  \n        assert self.feature.shape[0] == self.length.shape[0]\n\n    @property\n    def batch_size(self) -\u003e tf.Tensor: # shape = [], dtype = tf.int32\n        return tf.shape(self.feature)[0]\n```\n\n## Other features:\n* Support hierarchical composition.\n* Support inheritance including multiple one (for free from original ```dataclass```).\n* Highliting, autocomplete, and refactoring from your IDE.\n\n## Usage\n1. Import ```dataclass``` and ```function``` from ```tf_dataclass```\n```python\nfrom tf_dataclasses import dataclass, function\n```\n2. \u003cstrong\u003eIt is mandatory to use return type hints for the function decorated with ```@function```.\u003c/strong\u003e\nFor example,\n```python\nfrom typing import Tuple\n\n@dataclass\nclass MyDataclass:\n    ...\n\n@function\ndef my_func(...) -\u003e Tuple[tf.Tensor, MyDataclass]:\n    ...\n    return some_tensor, my_dataclass_instance\n```\n3. Type hints for the arguments are optional but recommended.\n\n4. \u003cstrong\u003ePositional arguments are not currently supported\u003c/strong\u003e:\n\nFor example, for\n```python\n@function\ndef my_graph_func(x: ..., y: ...) -\u003e ... :\n    ...\n```\ntype\n```python\nmy_graph_func(x=x, y=y)\n```\nbut not\n```python\nmy_graph_func(x, y)\n```\n\n## Known Problems\n1. IDE autocomplete is currently not well-supported, for example, in PyCharm. \nSolution: use import\n```python\nfrom typing import TYPE_CHECKING\nif TYPE_CHECKING:\n    from dataclasses import dataclass\nelse:\n    from tf_dataclass import dataclass\n```\nin each ```*.py``` file where ```dataclass``` is used.\n\n## Under the roof\nDataclasses and their nested structures are simply converted into nested pythonic tuples and back.\nThis way we wrap given functions such that all inputs and outputs are nested tuples.\nThen \n```@tf.function``` \nis applied. Afterward the graph function is wrapped bach to dataclass form.\nType hints are used in python runtime for the graph creation as temples to pack and unpack dataclass arguments. \n\n## Future plans\n1. Support ```tf.cond```, ```tf.case```, ```tf.switch_case```, ```tf.while_loop```, ```tf.Optional```, and \n```tf.data.Iterator```.\n2. Support positional arguments.\n3. Conversion to ```tf.nest``` structures.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falexeytochin%2Ftf-dataclass","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falexeytochin%2Ftf-dataclass","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falexeytochin%2Ftf-dataclass/lists"}