1. MooseStack
  3. Engines
  5. S3

Direct S3 Access (S3)

Use the S3 engine for direct read/write access to S3 storage without streaming semantics:

S3Table.ts
import { OlapTable, ClickHouseEngines, mooseRuntimeEnv } from '@514labs/moose-lib'; // S3 table with credentials (recommended with mooseRuntimeEnv)export const s3Data = new OlapTable<DataRecord>("s3_data", {  engine: ClickHouseEngines.S3,  path: "s3://my-bucket/data/file.json",  format: "JSONEachRow",  awsAccessKeyId: mooseRuntimeEnv.get("AWS_ACCESS_KEY_ID"),  awsSecretAccessKey: mooseRuntimeEnv.get("AWS_SECRET_ACCESS_KEY"),  compression: "gzip"}); // Public S3 bucket (no authentication)export const publicS3 = new OlapTable<DataRecord>("public_s3", {  engine: ClickHouseEngines.S3,  path: "s3://public-bucket/data/*.parquet",  format: "Parquet",  noSign: true  // Use NOSIGN for public buckets});
S3 vs S3Queue
  • S3: Direct read/write access to S3 files. Use for batch processing or querying static data.
  • S3Queue: Streaming engine that automatically processes new files as they arrive. Use for continuous data ingestion.

Both engines support the same credential management and format options.

FiveonefourFiveonefour
Fiveonefour Docs
MooseStackTemplatesGuides
Release Notes
Source523
  • Overview
Build a New App
  • 5 Minute Quickstart
  • Browse Templates
  • Existing ClickHouse
Add to Existing App
  • Next.js
  • Fastify
Fundamentals
  • Moose Runtime
  • MooseDev MCP
  • Data Modeling
Moose Modules
  • Moose OLAP
  • Moose Streaming
  • Moose Workflows
  • Moose APIs & Web Apps
Deployment & Lifecycle
  • Moose Migrate
  • Moose Deploy
Reference
  • API Reference
  • Data Types
  • Table Engines
  • CLI
  • Configuration
  • Observability Metrics
  • Help
  • Release Notes
Contribution
  • Documentation
  • Framework
S3Table.ts
import { OlapTable, ClickHouseEngines, mooseRuntimeEnv } from '@514labs/moose-lib'; // S3 table with credentials (recommended with mooseRuntimeEnv)export const s3Data = new OlapTable<DataRecord>("s3_data", {  engine: ClickHouseEngines.S3,  path: "s3://my-bucket/data/file.json",  format: "JSONEachRow",  awsAccessKeyId: mooseRuntimeEnv.get("AWS_ACCESS_KEY_ID"),  awsSecretAccessKey: mooseRuntimeEnv.get("AWS_SECRET_ACCESS_KEY"),  compression: "gzip"}); // Public S3 bucket (no authentication)export const publicS3 = new OlapTable<DataRecord>("public_s3", {  engine: ClickHouseEngines.S3,  path: "s3://public-bucket/data/*.parquet",  format: "Parquet",  noSign: true  // Use NOSIGN for public buckets});