Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/brandonzylstra/essence

🧘🏼‍♂️ Relaxed Rails Modeling & Migrations
https://github.com/brandonzylstra/essence

active-record data database gem hcl modeling rails ruby ruby-on-rails yaml

Last synced: 2 months ago
JSON representation

🧘🏼‍♂️ Relaxed Rails Modeling & Migrations

Awesome Lists containing this project

README 

          
# 🌿 Essence 🫧 - ActiveRecord Modeling #



[![CI](https://github.com/brandonzylstra/essence/workflows/CI/badge.svg)](https://github.com/brandonzylstra/essence/actions)



**Essence** is a powerful tool for rapid database schema iteration in Rails applications. It provides a clean, YAML-based syntax with intelligent defaults and pattern matching that compiles to Atlas HCL format for seamless database migrations.



## 🚀 Why Essence?



- **10x faster schema iteration** - Write schemas in clean YAML instead of verbose Rails migrations or GUI tools

- **Flexible data type syntax** - Use familiar Rails migration types (`:string`, `:integer`) or concise Atlas/SQL types (`string(100)`, `integer`)

- **Smart defaults** - Automatic `id`, `created_at`, `updated_at` columns for every table

- **Pattern-based property inference** - `league_id: ~` automatically becomes a foreign key to `leagues.id`

- **Version control friendly** - Text-based schemas that diff and merge cleanly

- **Rails integration** - Seamless workflow with rake tasks and Atlas backend

- **Template generation** - Quick project setup with sensible defaults



## 🔌 Installation ##



**Note:** This gem is not yet published to RubyGems. Install from source:



### From GitHub (recommended)



Add this line to your application's Gemfile:



```ruby

gem 'essence', git: 'https://github.com/brandonzylstra/essence.git'

```



And then execute:



```shell

$ bundle install

```



### Build and install locally



Clone the repository and build the gem:



```shell

$ git clone https://github.com/brandonzylstra/essence.git

$ cd essence

$ bundle install

$ gem build essence.gemspec

$ gem install essence-0.1.0.gem

```



## 🚀 Quick Start ##



### 1. Generate a schema template ###



```shell

# In your Rails project

rake essence:template

```



This creates `db/schema.yaml` with sensible defaults:



```yaml

# Enhanced Essence Schema with Default Columns and Pattern Matching

schema_name: public



defaults:

  "*":

    columns:

      id: ~

      created_at: ~

      updated_at: ~



column_patterns:

  - "^id$": "primary_key"

  - "_id$": "integer -> {table}.id on_delete=cascade not_null"

  - "_at$": "datetime not_null"

  - ".*": "string"



tables:

  users:

    columns:

      email: string(255) not_null unique

      name: string(100) not_null

      league_id: ~

      last_login_at: ~



  leagues:

    columns:

      name: string(255) not_null unique

      description: text

```



### 2. Compile to HCL format



```shell

rake essence:compile

```



This generates `db/schema.hcl` in Atlas format with foreign keys automatically resolved:



```hcl

schema "public" {

  comment = "Generated by Essence"

}



table "users" {

  schema = schema.public

  column "id" {

    null = false

    type = integer

    auto_increment = true

  }

  column "email" {

    null = false

    type = varchar(255)

  }

  column "name" {

    null = false

    type = varchar(100)

  }

  column "league_id" {

    null = false

    type = integer

  }

  column "last_login_at" {

    null = false

    type = datetime

  }

  column "created_at" {

    null = false

    type = datetime

  }

  column "updated_at" {

    null = false

    type = datetime

  }

  primary_key {

    columns = [column.id]

  }

  foreign_key "FK_USERS_LEAGUE" {

    columns     = [column.league_id]

    ref_columns = [table.leagues.column.id]

    on_delete   = CASCADE

  }

  index "IDX_USERS_EMAIL" {

    columns = [column.email]

    unique  = true

  }

}

```



### 3. Preview and apply changes ###



```shell

# See what would change

rake essence:preview



# Generate Rails migration and apply

rake essence:deploy["add tournament tables"]

```



## 🎯 Core Features ##



### Smart Defaults ###

Every table automatically gets:

- `id` primary key column

- `created_at` and `updated_at` timestamps

- Proper indexing and constraints



### Pattern-Based Intelligence ###

Write `league_id: ~` and—based on the pattern that name matches—automatically get:

- Integer column

- Foreign key to `leagues.id`

- Cascading delete

- Proper indexing



### Flexible Data Type Syntax ###

Choose the syntax that works best for your team:



```yaml

# Atlas/SQL syntax

tables:

  tournaments:

    columns:

      name: string(255) not_null unique

      league_id: ~   # Becomes foreign key automatically

      start_date: ~  # Becomes datetime not_null



# Rails syntax

  users:

    columns:

      name: :string, limit: 255, null: false, unique: true

      league_id: ~   # Pattern matching works with both syntaxes

      created_at: :datetime, null: false



# Mixed syntax

  posts:

    columns:

      title: string(255) not_null           # Atlas/SQL syntax

      content: :text                        # Rails syntax

      user_id: ~                            # Pattern matching

```



### Full Rails Integration ###

```shell

rake essence:template     # Generate schema template

rake essence:convert      # Convert YAML to HCL

rake essence:preview      # Preview changes

rake essence:apply        # Apply to database

rake essence:deploy[name] # Full workflow

```



## 🗃️ Supported Schema Elements ##



Essence provides comprehensive support for database schema elements, with plans to expand coverage over time.



|Status| Group    | Element                  | Explanation                                                        |

|:----:|:--------:|:-------------------------|:-------------------------------------------------------------------|

| ✅   | Core     | **Tables**               | Primary schema containers with columns, constraints, and metadata  |

| ✅   | Core     | **Columns**              | All standard data types with dual syntax support (Atlas/SQL + Rails) |

| ✅   | Core     | **Data Types**           | Support for both `:string` (Rails) and `string(100)` (Atlas/SQL) syntax |

| ✅   | Core     | **Primary Keys**         | Auto-incrementing ID columns with proper constraints               |

| ✅   | Core     | **Foreign Keys**         | Automatic relationship detection with cascade/set null options     |

| ✅   | Core     | **Indexes**              | Single and multi-column indexes with unique constraints            |

| ⏳   | Core     | **Check Constraints**    | Column-level validation rules *(planned for Rails 6.1+ support)*   |

| ⏳   | Advanced | **Views**                | Read-only virtual tables (`scenic` gem integration)                |

| ⏳   | Advanced | **Materialized Views**   | Cached query results (`scenic` gem integration)                    |

| ⏳   | Advanced | **Functions/Stored Procedures**  | Database-level business logic (`scenic`, `fx` gems)        |

| ❓   | Advanced | **Exclusion Constraints**| PostgreSQL-specific constraints                                    |

| ❓   | Advanced | **Enums**                | Type-safe enumerated values (Rails native with PostgreSQL)         |

| ❓   | Advanced | **Partitioned Tables**   | Large table performance optimization (Rails 6.1+ with PostgreSQL)  |

| ❓   | Advanced | **Triggers**             | Event-driven database actions (`hair_trigger` gem integration)     |

| ❓   | Advanced | **Full-text Search Indexes** | Advanced search capabilities (`pg_search` gem)                 |



### Version Compatibility ###



**ActiveRecord Schema Versions:**

- ✅ **8.0** - Fully supported (current)

- 🚧 **7.2** - Possible support

- 🚧 **7.1** - Possible support

- 🚧 **7.0** - Possible support



Priority is given to supporting newer Rails versions as they are released (8.1, 8.2, etc.) before adding backward compatibility.



## 📋 Available Patterns ##



Essence automatically recognizes these column patterns (works with both Atlas/SQL and Rails syntax):



| Pattern | Example | Result |

|---------|---------|--------|

| `*_id` | `user_id: ~` | `integer -> users.id on_delete=cascade not_null` |

| `*_at` | `published_at: ~` | `datetime not_null` |

| `*_on` | `published_on: ~` | `date not_null` |

| `*_date` | `birth_date: ~` | `date not_null` |

| `is_*` | `is_active: ~` | `boolean default=false not_null` |

| `has_*` | `has_premium: ~` | `boolean default=false not_null` |

| `can_*` | `can_edit: ~` | `boolean default=false not_null` |

| `*_flag` | `admin_flag: ~` | `boolean default=false not_null` |

| `*_content` | `body_content: ~` | `text` |

| `*_body` | `email_body: ~` | `text` |

| `*_text` | `description_text: ~` | `text` |

| `*_html` | `content_html: ~` | `text` |

| `*_count` | `view_count: ~` | `integer default=0 not_null` |

| `*_score` | `rating_score: ~` | `decimal(8,2)` |

| `*_amount` | `price_amount: ~` | `decimal(10,2)` |

| `*_price` | `unit_price: ~` | `decimal(10,2)` |

| `*_email` | `contact_email: ~` | `string(255)` |

| `*_url` | `website_url: ~` | `string(500)` |

| `*_code` | `product_code: ~` | `string(50)` |

| `*_slug` | `permalink_slug: ~` | `string(255) unique` |

| `*_status` | `order_status: ~` | `string(50)` |

| `*_state` | `workflow_state: ~` | `string(50)` |



## 🛠️ Advanced Usage ##



### Custom Patterns ###

Define your own patterns in the schema:



```yaml

column_patterns:

  - "_uuid$": "uuid not_null unique"

  - "_json$": "json"

  - "encrypted_": "text not_null"

```



### Table-Specific Defaults ###

Override defaults for specific tables:



```yaml

defaults:

  "*":

    columns:

      id: ~

      created_at: ~

      updated_at: ~

  "audit_*":

    columns:

      id: uuid primary_key

      created_at: ~

      created_by: string(255) not_null

```



### Explicit Overrides ###

Override pattern matching with explicit definitions:



```yaml

tables:

  users:

    columns:

      league_id: string(50) not_null  # Override foreign key pattern

      created_at: timestamp not_null  # Override default

```



## 🔄 Workflow Integration ##



### Complete Development Workflow ###



```shell

# 1. Set up (first time)

rake essence:init           # Import existing schema

# OR

rake essence:template       # Start fresh



# 2. Edit schema

vim db/schema.yaml



# 3. Iterate rapidly

rake essence:compile        # YAML → HCL

rake essence:preview        # See changes

rake essence:generate[name] # Create migration

rake essence:apply          # Apply to DB



# 4. Deploy

rake essence:deploy[name]   # All-in-one

```



### CI/CD Integration ###



```yaml

# .github/workflows/schema.yml

name: Schema Validation

on: [push, pull_request]

jobs:

  validate:

    runs-on: ubuntu-latest

    steps:

      - uses: actions/checkout@v2

      - name: Setup Ruby

        uses: ruby/setup-ruby@v1

        with:

          bundler-cache: true

      - name: Validate schema

        run: rake essence:validate

```



## 📚 Command Reference ##



### CLI Commands ###

```shell

essence template [path]         # Generate template

essence compile [yaml] [hcl]    # Compile formats

essence version                 # Show version

essence help                    # Show help

```



### Rake Tasks ###

```shell

rake essence:template[path]     # Generate schema template

rake essence:compile[yaml,hcl]  # Compile YAML to HCL

rake essence:preview            # Preview changes

rake essence:generate[name]     # Generate Rails migration

rake essence:apply              # Apply schema to database

rake essence:deploy[name]       # Full workflow

rake essence:seed               # Generate seed data

rake essence:init               # Initialize from existing

rake essence:validate           # Validate schema

rake essence:history            # Show migration history

rake essence:help               # Show all commands

```



## 🏗️ Architecture ##



Essence works by:



1. **YAML Definition** - You write clean, pattern-rich schemas

2. **Pattern Resolution** - Automatic foreign keys, constraints, indexes

3. **HCL Generation** - Compiles to Atlas-compatible HCL format

4. **Atlas Integration** - Uses Atlas for migration planning

5. **Rails Compatibility** - Generates standard Rails migrations



```

schema.yaml → [Essence] → schema.hcl → [Atlas] → migration.rb → [Rails] → Database

```



## 💻 Examples ##



### Basic Blog Schema ###

```yaml

schema_name: public

tables:

  users:  # Atlas/SQL syntax

    columns:

      email: string(255) not_null unique

      username: string(50) not_null unique

      first_name: string(100)

      last_name: string(100)

      is_admin: ~

      last_login_at: ~



  posts:  # Rails syntax

    columns:

      title: :string, limit: 255, null: false

      post_slug: ~                    # Pattern matching works with both

      content_html: :text

      user_id: ~                      # Foreign key via pattern

      published_at: :datetime

      is_published: :boolean, default: false

      view_count: :integer, default: 0

```



  comments:

    columns:

      post_id: ~

      user_id: ~

      comment_body: ~

      is_approved: ~

```



### E-commerce Schema ###

```yaml

schema_name: public

tables:

  customers:

    columns:

      email: string(255) not_null unique

      first_name: string(100) not_null

      last_name: string(100) not_null

      phone: string(20)

      birth_date: ~

      is_active: ~



  products:

    columns:

      name: string(255) not_null

      product_code: ~

      description_text: ~

      unit_price: ~

      cost_amount: ~

      stock_count: ~

      is_active: ~



  orders:

    columns:

      customer_id: ~

      order_status: ~

      total_amount: ~

      tax_amount: ~

      shipping_amount: ~

      order_date: ~

      shipped_at: ~

```



## 💡 Tips and Best Practices ##



### 1. Always Preview First ###

```shell

rake essence:preview  # See what will change

rake essence:apply    # Apply changes

```



### 2. Use Descriptive Names ###

```yaml

# Good - descriptive and clear

users:

  columns:

    email: string(255) not_null unique

    contact_email: string(255)



# Avoid - too generic

users:

  columns:

    data1: string

    field2: text

```



### 3. Be Explicit About Constraints ###

```yaml

# Good - explicit constraints

email: string(255) not_null unique

user_id: integer -> users.id on_delete=cascade not_null



# Avoid - relying only on patterns

email: ~

user_id: ~

```



### 4. Group Related Tables ###

Organize your YAML file with related tables together for better readability:



```yaml

# Authentication & Users

users:

  # ...

user_sessions:

  # ...



# Content Management

posts:

  # ...

comments:

  # ...

```



### 5. Use Pattern Matching Effectively ###

```yaml

# Let patterns handle repetitive definitions

published_at: ~     # Becomes: datetime not_null

is_active: ~        # Becomes: boolean default=false not_null

view_count: ~       # Becomes: integer default=0 not_null



# Override patterns when needed

special_id: bigint -> special_table.id on_delete=set_null

```



## 🤝 Contributing ##



1. Fork it (https://github.com/brandonzylstra/essence/fork)

2. Create your feature branch (`git checkout -b my-new-feature`)

3. Commit your changes (`git commit -am 'Add some feature'`)

4. Push to the branch (`git push origin my-new-feature`)

5. Create a new Pull Request



## 📄 License ##



The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).



## 🙏 Acknowledgments ##



- [Atlas](https://atlasgo.io/) for the powerful migration engine

- [Rails](https://rubyonrails.org/) for the amazing framework

- The Ruby community for inspiration and support