Moose
Moose is an open source framework for developing data products & APIs on OLAP/streaming infra using native TypeScript/Python
Core Capabilities
Type-Safe Data Models
High-Performance Data Ingestion
Real-Time Stream Processing
In-Database Transformations
Analytics APIs
Workflow Orchestration
Why Moose Exists
Building Software With Today's Data Stack is Slow
Tool fragmentation
More time spent integrating Kafka, ClickHouse, Postgres, dbt, Airflow, and a dozen other services instead of building your actual application
Schema drift everywhere
Your TypeScript or Python models, database schemas, API validation, and message formats all diverge over time
Painful development workflow
No local testing, long deployment cycles, and constant context switching
SQL-only processing
Having to use SQL for everything when you'd rather use languages you're already comfortable with
The DIY Data Stack Approach
What if you need to add a simple string field to your data model?
Update your TS/Python Code Model
Update your Database Schema
Update your Kafka Topic
Update your Runtime Validation
Update your transformations & queries
Can you see the problem? This process repeats for every change.
You also have to test that everything is working together in a safe, isolated dev environment, which is even more painful with so many moving parts.
What Moose Does
With Moose, your TypeScript and Python code is the single source of truth for both your data application logic AND your data infrastructure:
// Define your model ONCE
import { Key, IngestPipeline } from "@514labs/moose-lib";
interface ExampleModel {
primaryKey: Key<string>;
name: string;
nested: {
isActive: boolean;
value: number;
internalName: string;
}[];
createdAt: Date;
}
// And use it EVERYWHERE - ONE line to wire everything up
export const examplePipeline = new IngestPipeline<ExampleModel>("example", {
ingest: true, // Creates API endpoint with validation
stream: true, // Creates properly structured Redpanda topic
table: true // Creates ClickHouse database table
});You get end-to-end infrastructure for your data pipeline that is:
Defined purely in TypeScript or Python code
Completely type safe and validated
Boilerplate free
Able to catch errors at dev time, not at runtime
No More Context Switching
“Does my Database table use snake_case or camelCase?”
“Did I add the new field to both the model AND the table?”
“Which database field was nullable again?”
Local Dev in Seconds
Local Dev Benefits
One-command startup
Launch your entire data infrastructure locally with a single command
Zero configuration
All components come pre-configured and fully integrated - no setup required
Production parity
Use the same technologies and logic that will run in production
Real-time feedback
See your changes reflected instantly throughout the stack
Moose comes with a pre-configured and fully integrated data stack that runs entirely on your laptop. Spin it up with one command:
moose devAll the infrastructure is automatically spun up:
⡏ Starting local infrastructure
Successfully started containers
Validated clickhousedb-1 docker container
Validated redpanda-1 docker container
Successfully validated red panda cluster
Validated temporal docker container
Successfully ran local infrastructureCommon workflow scenarios
Need to add or change a model?
Moose hot reloads it to your local infrastructure when you save
Added a new field?
It's instantly available in your API, streams, and database
Need to test your pipeline?
Send sample data to your local ingest API and see the data flow through
Hot Reloading Dev Workflow in Action
Make a change to your model:
// Add a field to your model
interface ExampleModel {
primaryKey: Key<string>;
name: string;
nested: {
isActive: boolean;
value: number;
internalName: string;
}[];
createdAt: Date;
status: string; // New field
}Hit save and youll see your changes hot reloaded to your local infrastructure:
⢹ Processing Infrastructure changes from file watcher
~ Topic: orders - Version: 0.0 - Retention Period: 604800s - Partition Count: 1
~ Table orders with column changes: [Added(Column { name: "status", data_type: String, required: true, unique: false, primary_key: false, default: None })] and order by changes: OrderByChange { before: [], after: [] }
~ Topic to Table Sync Process: orders_0_0 -> orders
~ API Endpoint: orders - Version: 0.0 - Path: ingest/orders - Method: POST - Format: Some(Json)That's it! No additional steps needed.
Your API now validates this field
Your database schema is updated
Your streams carry the new field
Modularity
Moose is designed to be modular and configurable. You can pick and choose which components you need and configure them to your liking.
Default Moose Stack:
ClickHouse
OLAP database (always enabled)
Redpanda
Kafka-compatible event streaming platform (can be disabled)
Temporal
Workflow orchestration (can be disabled)
Learn how to enable/disable components in your stack
Review the Architecture Documentation for more details.
Planned Extensions:
Snowflake, Databricks, and BigQuery
Cloud-native data warehouses
Kafka, Kinesis, and Pulsar
Kafka-compatible event streaming platforms
Let us know if you’d like to see support for specific platforms in your stack.
What Can You Build with Moose?
Moose is ideal for a wide range of data-intensive applications, from real-time analytics to complex data pipelines:
User Facing Analytics
Embed leaderboards, charts, metrics, and other real-time features in your web or mobile apps
BI and Data Warehouses
Collect disparate data sources into an analytical database, produce custom reports, and more
Data Migrations
One-time migration of data from legacy systems to a modern data backend
Event Streaming
Real-time processing of events from Kafka, Redpanda, or other event streaming platforms
ETL Workloads
Repeated batch jobs to collect data from different sources and load them into an analytics environment