Migrations synchronize your code-defined database schema with your production infrastructure. As your application evolves, you'll add/remove fields, change data types, and restructure tables. Moose Migrate handles these schema changes safely and reliably.
Moose Migrate operates by comparing two states:
When these states differ, Moose Migrate generates operations to bring them into alignment. These operations might include:
The full list of operations is available in the Migration Plan Format documentation.
Migrations in Moose revolve around three key decisions:
| Concept | What it is | Where to go |
|---|---|---|
| Lifecycle Management | Controlling what changes are allowed (e.g., preventing data deletion). | Overview • Fully Managed • Deletion Protected • Externally Managed |
| Generating Migrations | Deciding how changes are generated. | Overview • Auto-Inferred • Planned |
| Applying Changes | The workflow for executing migrations: serverless (moose migrate) vs server runtime (moose prod). | Serverless • Server Runtime |
For each table and stream resource defined in your code, you can control what changes are allowed (e.g., preventing data deletion, ignoring schema changes, etc.) with the LifeCycle configuration property:
import { OlapTable, LifeCycle } from "@514labs/moose-lib"; interface Schema { id: string; name: string; age: number;} const table = new OlapTable<Schema>("table_name", { lifeCycle: LifeCycle.FULLY_MANAGED orderByFields: ["id"]});| Option | Behavior | Use When |
|---|---|---|
FULLY_MANAGED (default) | Automatically modifies resources to match your code, including destructive operations. | When you're developing new tables that you want your application to manage and evolve over time. |
DELETION_PROTECTED | Automatically modifies resources to match your code, but blocks destructive operations (drops, deletions). | When you want to protect critical production tables from accidental data loss. |
EXTERNALLY_MANAGED | Does not modify resources. You manage the schema manually directly in your database. | When you have existing tables that you want to manage outside of your application, or if you're using a Managed CDC service like ClickPipes or PeerDB to manage your schema. |
You configure lifecycle modes individually on each OlapTable and Stream object. This allows you to mix fully managed development tables with deletion-protected production tables and externally managed legacy resources in the same application.
Moose Migrate provides two complementary ways to generate migrations. Each is designed for use in different stages of the application lifecycle, and it's best practice to use both in your workflow:
| Option | Behavior | Use Case |
|---|---|---|
| Auto-Inferred | Updates database instantly on file save. Fast iteration, but can be destructive. | Local development, fast prototyping |
| Planned | Generates reviewable plan files. Safe, deterministic, with drift detection. | Production deployment, CI/CD |
You have two options for executing planned migrations against your production database:
| Mode | Behavior | Use Case |
|---|---|---|
Serverless (moose migrate) | You run migrations manually or via CI/CD. | Integrating Moose OLAP into an existing application as a library. |
Server Runtime (moose prod) | Migrations are automatically run within the Moose Runtime on server startup. | Building a dedicated analytics service with the full Moose Runtime. |