https://github.com/servicenow/synbols-benchmarks

Benchmarks for the Synbols project. Synbols is a ServiceNow Research project that was started at Element AI.
https://github.com/servicenow/synbols-benchmarks

Last synced: 5 months ago
JSON representation

Benchmarks for the Synbols project. Synbols is a ServiceNow Research project that was started at Element AI.

• Host: GitHub
• URL: https://github.com/servicenow/synbols-benchmarks
• Owner: ServiceNow
• License: apache-2.0
• Created: 2020-07-14T21:19:13.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2022-06-30T12:48:16.000Z (over 3 years ago)
• Last Synced: 2024-03-30T05:22:37.096Z (almost 2 years ago)
• Language: Python
• Homepage:
• Size: 1.24 MB
• Stars: 9
• Watchers: 6
• Forks: 2
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
*ServiceNow completed its acquisition of Element AI on January 8, 2021. All references to Element AI in the materials that are part of this project should refer to ServiceNow.*

![#Synbols](https://github.com/ElementAI/synbols/raw/master/title.png)

# Probing Learning Algorithms with Synthetic Datasets

[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)

![Synbols](https://github.com/ElementAI/synbols-benchmarks/raw/master/cover.png)

## Description

This repository contains the code for reproducing experiments in [1].

To use the Synbols tool for generating new datasets, please visit https://github.com/ElementAI/synbols

## Pytorch Dataset Quick Setup

```python

from pytorch_examples.datasets import Synbols

from torchvision import transforms as tt

dataset_path = "./"

dataset_name = "default_n=100000_2020-Oct-19.h5py"

synbols = Synbols(args.data_path,

                  dataset_name=args.dataset)

train_dataset = synbols.get_split('train', tt.ToTensor())

val_dataset = synbols.get_split('val', tt.ToTensor())

```

For a complete example run `./pytorch_examples/minimal_classification.py` from the root folder of this project:

```bash

python -m pytorch_examples.minimal_classification

```

It should reach >70% accuracy.

## Bibliography

[1] Lacoste, A., Rodríguez, P., Branchaud-Charron, F., Atighehchian, P., Caccia, M., Laradji, I., Drouin, A., Craddock, M., Charlin, L. and Vázquez, D., 2020. [Synbols: Probing Learning Algorithms with Synthetic Datasets.](https://arxiv.org/abs/2009.06415)