October 24, 2025

Serverless ClickHouse migrations with moose migrate

Run schema diffs and applies straight against a ClickHouse endpoint—perfect for OLAP-only or CI/CD environments that don’t boot the full Moose runtime.

# Detect changes and persist the migration plan
moose generate migration \
  --clickhouse-url "https://user:pass@ch.serverless.dev/main" \
  --save
 
# Apply the plan directly to ClickHouse
moose migrate --clickhouse-url "https://user:pass@ch.serverless.dev/main"

🐙 PR: (#2872) | 📘 Docs: Serverless ClickHouse migrations guide

Adaptive JSON columns with ClickHouseJson

Model semi-structured payloads while locking in typed paths for the fields you care about.

import { Key, ClickHouseJson, Int64 } from "@514labs/moose-lib";
 
interface UserMetadata {
  userId: string;
  sessionId: string;
}
 
export interface UserActivity {
  id: Key<string>;
  // ClickHouseJson<maxDynamicPaths, maxDynamicTypes, skipPaths, skipRegexes>
  metadata: UserMetadata & ClickHouseJson<256, 16, ["internal.debug"], ["^tmp\\."]>;
}

🐙 PR:(#2887) | 📘 Docs: Data modeling

Moose

• moose migrate --clickhouse-url – Generate and apply migrations directly against hosted ClickHouse, ideal for OLAP-only or CI/CD workflows that run without the full Moose runtime. Docs: Serverless ClickHouse migrations | PRs #2872.
• LLM-friendly docs & endpoints – Framework pages expose TS/Py “LLM view” links and the CLI now serves /llm-ts.txt + /llm-py.txt for assistants that need scoped context. Docs: LLM docs | PRs #2892.
• Flexible JSON columns – ClickHouseJson lets models cap dynamic paths, pin typed paths, or skip keys/regexes so ingestion can accept evolving payloads without breaking typed reads. Docs: Data modeling | PRs #2887.
• Configurable source_dir – moose.config.toml can point at src/ (or any folder) instead of the default app/, simplifying adoption inside existing repos. Docs: Configuration | PRs #2886.
• Array transforms can fan out events – Transform functions that return arrays automatically emit one Kafka message per element, covering explode/normalize patterns without extra producers. Docs: Transform functions | PRs #2882.
• ClickHouse modeling controls – Table DSL now covers TTL per table/column, sampleByExpression, and fully configurable secondary indexes (type, args, granularity) so you can encode retention + performance plans directly in code. Docs: TTL • Docs: Schema optimization • Docs: Secondary indexes | PRs #2845, #2867, #2869.
• get_source MCP tool – AI assistants can resolve a Moose component (tables, APIs, streams) back to its source file for faster code navigation. Docs: MCP get_source tool | PRs #2848.
• Google Analytics v4 connector – Service-account authenticated connector streams GA4 reports and realtime metrics into Moose pipelines so marketing data lands without bespoke ETL. Docs: Connector reference | PRs registry#121.
• Connector registry APIs – Public REST endpoints expose connector metadata, docs, schemas, and versions for catalogs or automation. Docs: Registry API docs | PRs registry#120.
• Onboarding & docs polish – Quickstart, auth, materialized view, and config guides now call out install checkpoints, nullable column behavior, and when to prefer moose.config.toml over Docker overrides. Docs: Quickstart | PRs #2903, #2894, #2893, #2890.
• Integer validation parity – The ingest API enforces every ClickHouse integer range (Int8–UInt256) with clear errors, preventing silent overflows. Docs: Ingest API | PRs #2861.
• MCP watcher stability – The MCP server now waits for file-system changes to settle before responding so IDE bots always read consistent artifacts. Docs: MCP server | PRs #2884.
• Release + schema compiler hardening – Version detection ignores CI-only tags, and ClickHouse parsing handles ORDER BY around PARTITION BY, TTL, SAMPLE BY, and secondary indexes even when optional arguments are omitted. PRs #2902, #2898, #2897, #2889.
• Proxy request fidelity – Consumption APIs now preserve headers/body metadata end-to-end, keeping auth tokens and content negotiation intact. PRs #2881.

Boreal

• Navigation slug fix – Visiting a 404 no longer strips the org ID from subsequent links, so multi-tenant operators stay on the right workspace. (commercial#1014)