Applying Migrations

Overview

Moose automatically manages database migrations by comparing your code with your database schema. Migrations are applied automatically when:

• Development: Changes are detected during moose dev
• Production: Changes are applied during deployment

When Migrations Happen

Development Workflow

Starting the Runtime

Use moose dev to start the Moose runtime with automatic migration detection:

moose dev
 
⡏ Starting local infrastructure
  Successfully started containers
     Validated clickhousedb-1 docker container
     Validated redpanda-1 docker container
  Successfully validated red panda cluster
     Validated temporal docker container
  Successfully ran local infrastructure

Automatic Migration Detection

Moose continuously monitors your code changes and applies migrations automatically. All changes are applied to your local database only.

interface Event {
  id: Key<string>;
  name: string;
  createdAt: Date;
  status: string; // New field - will trigger migration
}
 
export const events = new OlapTable<Event>("events")

class Event(BaseModel):
    id: Key[str]
    name: str
    created_at: datetime
    status: str  # New field - will trigger migration
 
table = OlapTable[Event]("events")

Migration Logs

When you save changes, you’ll see detailed migration logs showing what’s being applied:

⢹ Processing Infrastructure changes from file watcher
             ~ Topic: events - Version: 0.0 - Retention Period: 604800s - Partition Count: 1
             ~ Table events with column changes: [Added(Column { name: "status", data_type: String, required: true, unique: false, primary_key: false, default: None })] and order by changes: OrderByChange { before: [], after: [] }
             ~ Topic to Table Sync Process: events_0_0 -> events
             ~ API Endpoint: events - Version: 0.0 - Path: ingest/events - Method: POST - Format: Some(Json)

Production Workflow

Deployment Process

When you deploy your code to production, Moose follows this migration process:

1. Schema Comparison: Moose compares the final code in your repository with the state of your production database schema
2. Migration Plan: A migration plan is generated showing what changes will be applied
3. Manual Review: Before applying migrations, you will be prompted to review the plan and confirm the changes. You can make changes to the plan before applying it.
4. Application: Changes are applied to the production database during deployment

Migration Types

Adding New Tables or Materialized Views

import { newTable, newMaterializedView } from "./path/to/your/newObjects"
 
// Export the new table and materialized view to apply changes
export {
  newTable, // New table
  newMaterializedView // New materialized view
}

Column Additions

Adding new fields to your data models:

interface AddedColumn {
  id: Key<string>;
  another_column: string;
  some_column: string;
}

interface AddedColumn {
  id: Key<string>;
  another_column: string;
  some_column: string;
  new_column: number; // New field - migration applied
}

Column Removals

Removing fields from your data models:

interface RemovedColumn {
  id: Key<string>;
  another_column: string;
  some_column: string;
  old_column: number;
}

interface RemovedColumn {
  id: Key<string>;
  another_column: string;
  some_column: string;
  // old_column field removed - migration applied
}

Type Changes

Changing data types (use with caution):

interface ChangedType {
  id: Key<string>;
  some_column: string; // String type - migration applied
}

interface ChangedType {
  id: Key<string>;
  some_column: number; // Number type - migration applied
}

Materialized View Changes

Modifying the SELECT statement of a materialized view:

export const userStats = new MaterializedView<UserStats>("user_stats", {
  selectStatement: `
    SELECT 
      userId,
      COUNT(*) as event_count,
      MAX(createdAt) as last_event
    FROM users
    GROUP BY userId
  `
  // other MV config properties...
});

export const userStats = new MaterializedView<UserStats>("user_stats", {
  selectStatement: `
    SELECT 
      userId,
      COUNT(*) as event_count,
      MAX(createdAt) as last_event,
      AVG(value) as avg_value  // New column added
    FROM users
    GROUP BY userId
  `
  // other MV config properties...
});

user_stats = MaterializedView[UserStats]("user_stats", MaterializedViewConfig(
    select_statement="""
    SELECT 
        user_id,
        COUNT(*) as event_count,
        MAX(created_at) as last_event
    FROM users
    GROUP BY user_id
    """
    # other MV config properties...
))

user_stats = MaterializedView[UserStats]("user_stats", MaterializedViewConfig(
    select_statement="""
    SELECT 
        user_id,
        COUNT(*) as event_count,
        MAX(created_at) as last_event,
        AVG(value) as avg_value  # New column added
    FROM users
    GROUP BY user_id
    """
    # other MV config properties...
))

When you modify a materialized view’s SELECT statement, Moose will:

1. Detect the change in your code
2. Drop the existing view in the database
3. Recreate the view with the new SELECT statement
4. Repopulate the data using the new query

Migration Best Practices

Troubleshooting Migrations

Common Issues