MooseStack

Changelog

Moose Changelog

What is this page?

This changelog tracks all meaningful changes to Moose. Each entry includes the PR link and contributor credit, organized by date (newest first). Use this page to stay informed about new features, fixes, and breaking changes that might affect your projects.

How to read this changelog:

Release Highlights

Key features, enhancements, or fixes for each release.

Added

New features and capabilities.

Changed

Updates to existing functionality or improvements.

Deprecated

Features that are no longer recommended for use and may be removed in the future.

Fixed

Bug fixes and reliability improvements.

Breaking Changes

Changes that require user action or may break existing usage.

2025-08-21

Release Highlights

  • Analytics API Standardization — Standardized naming of classes and utilities for analytics APIs
  • Complete S3Queue Engine Support — Full implementation of ClickHouse S3Queue engine with comprehensive parameter support, modular architecture, and generic settings framework.

Added

  • S3Queue Engine Configuration — Added complete support for all ClickHouse S3Queue engine parameters including authentication (aws_access_key_id, aws_secret_access_key), compression, custom headers, and NOSIGN for public buckets. PR #2674
  • Generic Settings Framework — Introduced a flexible settings system that allows any engine to use configuration settings, laying groundwork for future engine implementations.
  • Enhanced Documentation — Added comprehensive documentation for OlapTable S3Queue configuration in both TypeScript and Python SDKs.

Changed

  • Improved Architecture — Moved ClickHouse-specific types from core infrastructure to ClickHouse module for better separation of concerns.
  • Settings Location — Engine-specific settings are now properly encapsulated within their respective engine configurations (e.g., s3QueueEngineConfig.settings for S3Queue).
  • API Consistency — Unified configuration APIs across TypeScript and Python SDKs for S3Queue engine.

Fixed

  • Compilation Issues — Fixed struct patterns to handle new S3Queue parameter structure correctly.
  • Diff Strategy — Enhanced diff strategy to properly handle S3Queue parameter changes.

Breaking Changes

  • ConsumptionApi renamed to Api

  • EgressConfig renamed to ApiConfig

  • ConsumptionUtil renamed to ApiUtil

  • ConsumptionHelpers renamed to ApiHelpers

    #2676 by camelCasedAditya

2025-08-20

Release Highlights

  • Improved IngestPipeline API Clarity — The confusing ingest parameter has been renamed to ingestApi (TypeScript) and ingest_api (Python) for better clarity. The old parameter names are still supported with deprecation warnings.

Changed

  • IngestPipeline Parameter Renamed — The ingest parameter in IngestPipeline configurations has been renamed for clarity:

    • TypeScript: ingest: true → ingestApi: true
    • Python: ingest=True → ingest_api=True

    The old parameter names continue to work with deprecation warnings to ensure backwards compatibility. [Current PR]

Deprecated

  • IngestPipeline ingest parameter — The ingest parameter in IngestPipeline configurations is deprecated:

    • TypeScript: Use ingestApi instead of ingest
    • Python: Use ingest_api instead of ingest

    The old parameter will be removed in a future major version. Please update your code to use the new parameter names. [Current PR]

Breaking Changes

None - Full backwards compatibility maintained

2025-06-12

Release Highlights

  • Enhanced TypeScript Workflow Types — Improved type safety for Tasks with optional input/output parameters, supporting null types for better flexibility.

Fixed

  • TypeScript workflow Task types now properly support optional input/output with null types, enabling more flexible task definitions like Task<Foo, null> and Task<null, Bar>.
    #2442 by DatGuyJonathan

Breaking Changes

None

2025-06-10

Release Highlights

  • OlapTable Direct Insert API — New comprehensive insert API with advanced error handling, typia validation, and multiple failure strategies. Enables direct data insertion into ClickHouse tables with production-ready reliability features.
  • Python Workflows V2 — Replaces static file-based routing with explicit Task and Workflow classes, enabling dynamic task composition and programmatic workflow orchestration. No more reliance on @task decorators or file naming conventions.

Added

  • OlapTable direct insert API with insert() method supporting arrays and Node.js streams. Features comprehensive typia-based validation, three error handling strategies (fail-fast, discard, isolate), configurable error thresholds, memoized ClickHouse connections, and detailed insertion results with failed record tracking.
    #2437 by callicles
  • Enhanced typia validation integration for OlapTable and IngestPipeline with validateRecord(), isValidRecord(), and assertValidRecord() methods providing compile-time type safety and runtime validation.
    #2437 by callicles
  • Python Workflows V2 with Task[InputType, OutputType] and Workflow classes for dynamic workflow orchestration. Replaces the legacy @task decorator approach with explicit task definitions, enabling flexible task composition, type-safe chaining via on_complete, retries, timeouts, and scheduling with cron expressions.
    #2439 by DatGuyJonathan

Breaking Changes

None

2025-06-06

Release Highlights

  • TypeScript Workflows V2 — Replaces static file-based routing with explicit Task and Workflow classes, enabling dynamic task composition and programmatic workflow orchestration. No more reliance on file naming conventions for task execution order.

Added

  • TypeScript Workflows V2 with Task<InputType, OutputType> and Workflow classes for dynamic workflow orchestration. Replaces the legacy file-based routing approach with explicit task definitions, enabling flexible task composition, type-safe chaining via onComplete, configurable retries and timeouts, and flexible scheduling with cron expressions.
    #2421 by DatGuyJonathan

Breaking Changes

None

2025-05-23

Release Highlights

  • TypeScript DeadLetterQueue<T> support — Handle failed streaming function messages with type-safe dead letter queues in TypeScript.
  • Improved Python DeadLetterModel API — Renamed as_t to as_typed for better clarity.

Added

  • TypeScript DeadLetterQueue<T> class with type guards and transform methods for handling failed streaming function messages with full type safety.
    #2356 by phiSgr

Changed

  • Renamed DeadLetterModel.as_t() to DeadLetterModel.as_typed() in Python for better API clarity and consistency.
    #2356 by phiSgr

Breaking Changes

  • DeadLetterModel.as_t() method renamed to as_typed() in Python. Update your code to use the new method name.
    #2356 by phiSgr

2025-05-22

Release Highlights

  • Refactored CLI ‘peek’ command — Now supports peeking into both tables and streams with unified parameters.
  • Simplified CLI experience — Removed unused commands and routines for a cleaner interface.

Changed

  • Updated CLI ‘peek’ command to use a unified ‘name’ parameter and new flags (--table, --stream) to specify resource type. Default is table. Documentation updated to match.
    #2361 by callicles

Deprecated

  • Removed unused CLI commands and routines including Function, Block, Consumption, DataModel, and Import. CLI is now simpler and easier to maintain.
    #2360 by callicles

Breaking Changes

None

2025-05-21

Release Highlights

  • Infrastructure state sync — Auto-syncs DB state before changes, handling manual modifications and failed DDL runs.
  • Fixed nested data type support — Use objects and arrays in your Moose models.

Added

  • State reconciliation for infrastructure planning — Moose now checks and updates its in-memory infra map to match the real database state before planning changes. Makes infra planning robust to manual DB changes and failed runs.
    #2341 by callicles

Fixed

  • Handling of nested data structures in Moose models for correct support of complex objects and arrays.
    #2357 by georgevanderson

Breaking Changes

None

2025-05-27

Release Highlights

  • IPv4 and IPv6 Type Support — Added native support for IP address types in ClickHouse data models, enabling efficient storage and querying of network data.

Added

  • IPv4 and IPv6 data types for ClickHouse integration, supporting native IP address storage and operations.
    #2373 by phiSgr
  • Enhanced type parser to handle IP address types across the Moose ecosystem.
    #2374 by phiSgr

Breaking Changes

None

2025-05-20

Release Highlights

  • ClickHouse Date type support — Store and query native date values in your schemas.

Added

  • ClickHouse Date column support for native date types in Moose schemas and ingestion.
    #2352, #2351 by phiSgr

Breaking Changes

None

2025-05-19

Release Highlights

  • Metadata map propagation — Metadata is now tracked and available in the infra map for both Python and TypeScript. Improves LLM accuracy and reliability when working with Moose objects.

Added

  • Metadata map propagation to infra map for consistent tracking and availability in both Python and TypeScript.
    #2326 by georgevanderson

Breaking Changes

None

2025-05-16

Release Highlights

  • New list[str] support to Python AggregateFunction — Enables more flexible aggregation logic in Materialized Views.
  • Python DeadLetterQueue[T] alpha release — Automatically route exceptions to a dead letter queue in streaming functions.

Changed

  • AggregateFunction in Python now accepts list[str] for more expressive and type-safe aggregations.
    #2321 by phiSgr

Added

  • Python dead letter queues for handling and retrying failed messages in Python streaming functions.
    #2324 by phiSgr

Breaking Changes

None

2025-05-15

Release Highlights

  • **Hotfix — casing fix for JSON columns in TypeScript.

Fixed

  • TypeScript JSON columns to have consistent casing, avoiding confusion and errors in your code.
    #2320 by phiSgr

Breaking Changes

None

2025-05-14

Release Highlights

  • Introduced TypeScript JSON columns — Use Record<string, any> for type-safe JSON fields.
  • Ingestion config simplified — Less config needed for ingestion setup.
  • Python enum support improved — More robust data models.

Added

  • TypeScript ClickHouse JSON columns to use Record<string, any> for type-safe JSON fields. #2317 by phiSgr
  • Pydantic mixin for parsing integer enums by name for more robust Python data models.
    #2316 by phiSgr
  • Better Python enum handling in data models for easier enum usage.
    #2315 by phiSgr

Deprecated

Breaking Changes

None

2025-05-13

Release Highlights

  • New refresh CLI command — Quickly reload data and schema changes from changes applied directly to your database outside of Moose.
  • Python: LowCardinality type support — Better performance for categorical data.

Added

  • refresh command to reload data and schema with a single command.
    #2309 by phiSgr
  • Python support for LowCardinality(T) to improve performance for categorical columns.
    #2313 by phiSgr

Breaking Changes

None

2025-05-10

Release Highlights

  • Dependency-based execution order for Materialized Views — Reduces migration errors and improves reliability.

Changed

  • Order changes for materialized views based on dependency to ensure correct execution order for dependent changes.
    #2294 by callicles

Breaking Changes

None

2025-05-07

Release Highlights

  • Python datetime64 support - Enables more precise datetime handling in Python data models.
  • Type mapping in Python QueryClient - Automatically maps ClickHouse query result rows to the correct Pydantic model types.

Added

  • Row parsing in QueryClient with type mapping for Python.
    #2299 by phiSgr

Fixed

  • datetime64 parsing and row parsing in QueryClient for more reliable data handling in Python.
    #2299 by phiSgr

Breaking Changes

None

2025-05-06

Release Highlights

  • uint type support in TypeScript — Enables type safety for unsigned integer fields in Typescript data models.

Added

  • uint type support in TypeScript for unsigned integers in Moose models.
    #2295 by phiSgr

Breaking Changes

None

2025-05-01

Release Highlights

  • Explicit dependency tracking for materialized views — Improves data lineage, migration reliability, and documentation.

Added

  • Explicit dependency tracking for materialized views to make migrations and data lineage more robust and easier to understand.
    #2282 by callicles

Breaking Changes

  • Required selectTables field in MaterializedView config that must specify an array of OlapTable objects for the source tables.
    #2282 by callicles

2025-04-30

Release Highlights

  • More flexible JSON_ARRAY configuration for IngestApi — Now accepts both arrays and single elements. Default config is now JSON_ARRAY.
  • Python rich ClickHouse type support — Added support for advanced types in Python models:
    • Decimal: clickhouse_decimal(precision, scale)
    • datetime with precision: clickhouse_datetime64(precision)
    • date: date
    • int with size annotations: Annotated[int, 'int8'], Annotated[int, 'int32'], etc.
    • UUID: UUID

Changed

  • JSON_ARRAY to allow both array and single element ingestion for more flexible data handling.
    #2285 by phiSgr

Added

  • Python rich ClickHouse type support with:
    • Decimal: clickhouse_decimal(precision, scale)
    • datetime with precision: clickhouse_datetime64(precision)
    • date: date
    • int with size annotations: Annotated[int, 'int8'], Annotated[int, 'int32'], etc.
    • UUID: UUID for more expressive data modeling.
      #2284 by phiSgr

Breaking Changes

None

TroubleshootingLLM-Optimized Documentation