https://github.com/reubenjohn/sandblox
Fundamentally, it introduces a representation for marrying static and dynamic graph based computing with procedural and object oriented programming(yup a lil ambitions), design patterns and standards open a world of possibilities for a vibrant community.
https://github.com/reubenjohn/sandblox
deep-learning machine-learning python reinforcement-learning tensorflow
Last synced: 2 months ago
JSON representation
Fundamentally, it introduces a representation for marrying static and dynamic graph based computing with procedural and object oriented programming(yup a lil ambitions), design patterns and standards open a world of possibilities for a vibrant community.
- Host: GitHub
- URL: https://github.com/reubenjohn/sandblox
- Owner: reubenjohn
- License: mit
- Created: 2018-04-02T19:36:11.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2022-12-08T02:53:08.000Z (over 2 years ago)
- Last Synced: 2025-03-15T22:48:02.803Z (3 months ago)
- Topics: deep-learning, machine-learning, python, reinforcement-learning, tensorflow
- Language: Python
- Homepage:
- Size: 150 KB
- Stars: 2
- Watchers: 2
- Forks: 1
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
===
**DISCLAIMER: Many features of this project are now available natively with Tensorflow 2. This project is no longer being maintained.**
The corresponding package has been yanked from pip: https://pypi.org/project/sandblox/ due to security vulnerabilities in dependencies.
**What** is it?
---
A software technology framework, library, platform and movement for Machine Learning.
Fundamentally, it introduces a representation for marrying static and dynamic graph based computing with procedural and object oriented programming(yup a lil ambitions), design patterns and standards open a world of possibilities for a vibrant community.
It borrows ideas from some of the best frameworks, libraries, and communities that exist today, while contributing novelty of it's own.
**Why** create it?
---
1. Modular & Composable
2. Expressive & Concise
3. Open Source
- Package management
- Sandblox Beach
- Reusable & Extensible
- Flexible
4. Scalable
5. Standardization
6. Reproducible
- Benchmarking
- Dataset audit
7. Continuous Integration
8. Rapid Prototyping
9. Interoperable: Marries static & dynamic graph based computing
10. Object Oriented => Modular Inheritance
11. Multiple Backends
- Tensorflow
- PyTorch (work in progress)
12. Out-of-the-box
- Logging
- Saving models
- Hierarchical graph building inferred from code
- Tensorboard support
- Advanced model slicing and dicing, eg: reusing initial layers of another network, transfer-learning, etc
13. More sweg...
So you can focus on the core logic, and not about the boiler-plate
**When** will it complete?
---
Well, as soon as possible, we're looking for all the help we can get. Besides, it's for the community and so it should be by the community!
**Who**'s it for?
---
In no particular order:
Community | Justification
---|---
Research | General Purpose, Expressive, Rapid, Reproducibility, Benchmarking, Backend Invariant
Newbies | Abstract, Reusability, Package Management
Developers | Package Management, Reusability, Object oriented, Open Source
Commercial | Scalable, CI, Maintainable, Standardised, Package Management
**How** does it work?
---
It introduces the concept of a block.
A block is a modular unit of logic that has well defined inputs and well defined outputs.
A block undergoes a well defined life cycle:
- Instantiation & configuration
- Static graph building
- Dynamic computation
Take for instance the creation of a weird block where we concatenate two arrays, add a bias and then pass it through a fully connected layer:
```python
import tensorflow as tf
import sandblox as sx
@sx.tf_static
def foo_block(x, y, bias=tf.constant(.1)):
concat = tf.concat([x, y], axis=0)
biased = concat + bias
return sx.Out.logits(tf.layers.dense(biased, 2)).bias(biased)
```
By adding the `@sx.tf_static` decorator, the function is automagically turned into a block.
This means that calling this function with parameters will create a new instance of the block.
```python
block = foo_block(tf.Variable([[.4, .5]]), tf.placeholder(tf.float32, [None, 2]))
```
And this is where things get interesting.
Sandblox infers what arguments are required to be bound before evaluation.
In this case, the second placeholder argument obviously needs to be provided a value for execution to occur.
Providing this value is as easy as passing it as an argument when running the block as shown below:
```python
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print(block.run([[.3, .3]])) # 2D logits array, and 2D biases array
```
The order of the arguments of the run function matches the order of dynamic binding arguments (in this case only 1 such argument).
Further, the block provides convenient references to it's inputs and outputs. For instance you can evaluate just the outputs like:
```python
print(sess.run(block.o.my_logits, feed_dict={block.i.y: [[.3, .3]]}) # 2D logits array
```
This means you can safely abstract away logic for creating tensors inside blocks and not loose references to them:
```python
def foo_block(x=tf.Variable([[.4, .5]]), input2=tf.placeholder(tf.float32, [None, 2]), bias=tf.constant(.1))
...
```
You can even create blocks in their own name spaces:
```python
block(tf.Variable([[.4, .5]]), ..., props=sx.Props(scope_name='block2'))
```
In case of tensorflow, this allows for better organization of your graph, avoiding collision of tensor names and better visualization in tensorboard.
In case of tensorflow blocks, you can also explicitly specify props for overriding the session (`session=...`) or graph (`graph=...`) in which to perform the operations, or if you want to make the name scope unique by appending an increment (`make_scope_unique=True`).
In cases where you want to separate the instantiation stage from the static graph building stage or if you want to access the props during the static evaluation, you can use class inheritance instead of the decorator:
```python
class FooBlock(sx.TFMold):
def static(self, x, y, bias):
concat = tf.concat([x, y], axis=0)
biased = concat + bias
x = biased
for _ in range(self.props.num_layers)
x = tf.layers.dense(x, 2)
return sx.Out.logits(x).bias(biased)
block = FooBlock(num_layers=4)
...
block(tf.Variable([[.4, .5]]), tf.placeholder(tf.float32, [None, 2]))
...
```
This is especially useful if you want to perform some kind of custom initialization:
```python
class FooBlock(sx.TFMold):
def __init__(self, **props):
...
def static(self, ...):
...
```
Classes also allow you to provide custom dynamic evaluation, allowing you to utilize dynamic compute based libraries such as PyTorch:
```python
class FooBlock(sx.TFMold):
...
def dynamic(*args, **kwargs):
result = super().dynamic(*args, **kwargs)
return result * 2 if result < .5 else resultl
```
Thus by following this well defined design, a lot of things can be provided out-of-the-box, so that you can focus on what's inside the block instead of what's outside block.
Since a block has both a static and dynamic stage, it supports both static and dynamic back-ends.
Once you get the hang of blocks, you start to see the possibilities, for instance:
**Chaining**: The outputs of one block could be provided as inputs to another.
**Hierarchies**: One block could be used within another block, i.e. the output block provides it inputs and uses it's outputs.
Imagine blocks that use hierarchies and chaining together!
For example, a layer that sequentially applies blocks:
```python
result = MySequentialBlock(
Input(tf.placeholder(...)),
ConvBlock(kernel_size=3),
DenseBlock(size=5)
)
result.run(...)
```
Imagine building your entire model in an extremely concise, modular and composable fashion:
```
model = Foo(
Octo(
Cat( # ^---^
With(...), # <{||=- @ @ -=||}>
Dragon(...), # ).V.(
Wings(...), # '/|||\`
), # '|`
LOL(...),
),
Octo(
Cat( # ^---^
Without(...), # ( @ @ )
Dragon(...), # ).V.(
Wings(...) # '/|||\`
) # '|`
),
...
)
```
**Where** does it apply?
---
- Quick and dirty prototype
- Reusable module to share with community
- Large models and lots of data scaling across multiple machines and GPU clusters
- Benchmarking your code against a standard dataset
- Streamlined project lifecycle from model prototyping and deployment
- Opensource version-controlled projects
- R&D and publishing reproducible results