API Reference
Viewing:
This is a comprehensive reference for @514labs/moose-lib
moose_lib
, detailing all exported components, types, and utilities.
Core Types
Key<T extends string | number | Date>
A type for marking fields as primary keys in data models.
// Example
interface MyModel {
id: Key<string>; // Marks 'id' as a primary key of type string
}
JWT<T extends object>
A type for working with JSON Web Tokens.
// Example
type UserJWT = JWT<{ userId: string, role: string }>;
ApiUtil
Interface providing utilities for analytics APIs.
interface ApiUtil {
client: MooseClient; // Client for interacting with the database
sql: typeof sql; // SQL template tag function
jwt: JWTPayload | undefined; // Current JWT if available
}
Infrastructure Components
OlapTable<T>
Creates a ClickHouse table with the schema of type T.
// Basic usage
export const myTable = new OlapTable<UserProfile>("user_profiles");
// With configuration
export const myConfiguredTable = new OlapTable<UserProfile>("user_profiles", {
orderByFields: ["id", "timestamp"],
engine: ClickHouseEngines.ReplacingMergeTree,
});
Stream<T>
Creates a Redpanda topic with the schema of type T.
// Basic usage
export const myStream = new Stream<UserEvent>("user_events");
// With configuration
export const myConfiguredStream = new Stream<UserEvent>("user_events", {
parallelism: 3,
retentionPeriod: 86400 // 1 day in seconds
});
// Adding transformations
myConfiguredStream.addTransform(
destinationStream,
(record) => transformFunction(record)
);
IngestApi<T>
Creates an HTTP endpoint for ingesting data of type T.
// Basic usage with destination stream
export const myIngestApi = new IngestApi<UserEvent>("user_events", {
destination: myUserEventStream
});
Api<T, R>
Creates an HTTP endpoint for querying data with request type T and response type R.
// Basic usage
export const myApi = new Api<UserQuery, UserProfile[]>(
"getUserProfiles",
async (params, { client, sql }) => {
const result = await client.query.execute(
sql`SELECT * FROM user_profiles WHERE age > ${params.minAge} LIMIT 10`
);
return result;
}
);
IngestPipeline<T>
Combines ingest API, stream, and table creation in a single component.
// Basic usage
export const pipeline = new IngestPipeline<UserEvent>("user_pipeline", {
ingest: true,
stream: true,
table: true
});
// With advanced configuration
export const advancedPipeline = new IngestPipeline<UserEvent>("advanced_pipeline", {
ingest: true,
stream: { parallelism: 3 },
table: {
orderByFields: ["id", "timestamp"],
engine: ClickHouseEngines.ReplacingMergeTree,
}
});
MaterializedView<T>
Creates a materialized view in ClickHouse.
// Basic usage
export const view = new MaterializedView<UserStatistics>({
selectStatement: "SELECT user_id, COUNT(*) as event_count FROM user_events GROUP BY user_id",
tableName: "user_events",
materializedViewName: "user_statistics",
orderByFields: ["user_id"]
});
SQL Utilities
sql
Template Tag
Template tag for creating type-safe SQL queries with parameters.
// Basic usage
const query = sql`SELECT * FROM users WHERE id = ${userId}`;
// With multiple parameters
const query = sql`
SELECT * FROM users
WHERE age > ${minAge}
AND country = ${country}
LIMIT ${limit}
`;
MooseClient
Client for interacting with ClickHouse and Temporal.
class MooseClient {
query: QueryClient; // For database queries
workflow: WorkflowClient; // For workflow operations
}
ClickHouse Utilities
ClickHouseEngines
Enum for supported ClickHouse table engines.
enum ClickHouseEngines {
MergeTree = "MergeTree",
ReplacingMergeTree = "ReplacingMergeTree",
SummingMergeTree = "SummingMergeTree",
AggregatingMergeTree = "AggregatingMergeTree",
CollapsingMergeTree = "CollapsingMergeTree",
VersionedCollapsingMergeTree = "VersionedCollapsingMergeTree",
GraphiteMergeTree = "GraphiteMergeTree"
}
Task Management
Task<T, R>
A class that represents a single task within a workflow system.
// No input, no output
export const task1 = new Task<null, void>("task1", {
run: async () => {
console.log("No input/output");
},
retries: 3,
timeout: "30s"
});
// With input and output
export const task2 = new Task<InputType, OutputType>("task2", {
run: async (input: InputType) => {
return process(input);
},
onComplete: [nextTask]
});
TaskHandler<T, R>
A function type that handles task execution.
type TaskHandler<T, R> =
T extends null ? () => Promise<R> : (input: T) => Promise<R>;
TaskConfig<T, R>
Configuration options for tasks.
interface TaskConfig<T, R> {
// The main function that executes the task logic
run: TaskHandler<T, R>;
// Optional array of tasks to execute after this task completes
onComplete?: (Task<R extends void ? null : R, any> | Task<R extends void ? null : R, void>)[];
// Optional timeout duration (e.g., "30s", "5m")
timeout?: string;
// Optional number of retry attempts
retries?: number;
}
Workflow
A class that represents a complete workflow composed of interconnected tasks.
const myWorkflow = new Workflow("getData", {
startingTask: callAPI,
schedule: "@every 5s", // Run every 5 seconds
timeout: "1h",
retries: 3
});
WorkflowConfig
Configuration options for defining a workflow.
interface WorkflowConfig {
// The initial task that begins the workflow execution
startingTask: Task<null, any> | Task<null, void> | Task<any, any> | Task<any, void>;
// Optional number of retry attempts
retries?: number;
// Optional timeout duration (e.g., "10m", "1h")
timeout?: string;
// Optional cron-style schedule string
schedule?: string;
}
Ensure your Infrastructure Components is correctly exported from your app/index.ts
file.
Ensure your Infrastructure Components is correctly imported into your main.py
file.
Learn more about export pattern: local development / hosted.
Important: The following components must be exported from your app/index.ts
file for Moose to detect them:
OlapTable
instancesStream
instancesIngestApi
instancesApi
instancesIngestPipeline
instancesMaterializedView
instancesTask
instancesWorkflow
instances
Configuration objects and utilities (like DeadLetterQueue
, Key
, sql
) do not need to be exported as they are used as dependencies of the main components.