Deploying on Amazon ECS

Moose can be deployed to Amazon’s Elastic Container Service (ECS). ECS offers a managed container orchestrator at a fraction of the complexity of managing a Kubernetes cluster.

If you’re relatively new to ECS we recommend the following resources:

• Amazon Elastic Container Service (ECS) with a Load Balancer | AWS Tutorial with New ECS Experience
• Tutorial: Deploy NGINX Containers On ECS Fargate with Load Balancer
• How to configure target groups ports with listeners and tasks

The first step is deciding whether you’ll host your Moose container on Docker Hub or Amazon’s Elastic Container Registry (ECR). Amazon ECR is straightforward and is designed to work out of the box with ECS. Using Docker Hub works if your moose container is publicly available; however, if your container is private, you’ll need to do a bit more work to provide ECS with your Docker credentials.

See: Authenticating with Docker Hub for AWS Container Services

Here is an overview of the steps required:

1. You’ll first need to create or use an existing ECS cluster.
2. Then, you’ll need to create an ECS Task definition. This is where you’ll specify whether you want to use AWS Fargate or AWS EC2 instances. You’ll also have options for selecting your OS and Architecture. Specify Linux/X86-64 or Linux/ARM-64. This is important as you’ll also need to specify a matching moose container image, such as moose-df-deployment-x86_64-unknown-linux-gnu:0.3.175 or moose-df-deployment-aarch64-unknown-linux-gnu:0.3.175
3. As with all AWS services, if you’re using secrets to store credentials, you will need to specify an IAM role with an AmazonECSTaskExecutionRolePolicy and SecretsManagerReadWrite policy.
4. Under the Container section, specify the name of your moose deployment and provide the container image name you’re using.
5. Next, specify the Container Port as 4000.

Configuring container environment variables

While still in the Amazon ECS Task definition section, you’ll need to provide the environment variables on which your Moose application depends. Scroll down to the Environment variables section and fill in each of the following variables.

Note: if you prefer, you can provide the environment variables below via an env file hosted on S3 or using AWS Secrets Manager for sensitive values.

Core Configuration

HTTP Server Configuration

Clickhouse Configuration (Required)

Redis Configuration (Required)

Redpanda Configuration (Optional)

Temporal Configuration (Optional)

Consider using a value of greater than 1000ms (1 second) for the Redpanda message timeout delay if you’re using a hosted Redpanda cloud service.

Review other options on the Task Creation page and press the Create button when ready.

Using AWS Secrets Manager

For sensitive information like passwords and API keys, we recommend using AWS Secrets Manager. To configure a secret:

1. Go to AWS Secrets Manager and create a new secret
2. Choose “Other type of secret” and add key-value pairs for your secrets
3. Name your secret appropriately (e.g., moose/production/credentials)
4. In your ECS task definition, reference the secret:
   • For environment variables, select “ValueFrom” and enter the ARN of your secret with the key name
   • Example: arn:aws:secretsmanager:region:account:secret:moose/production/credentials:MOOSE_CLICKHOUSE_CONFIG__PASSWORD::

Building an ECS Service

Once you’ve completed creating an ECS Task, you’re ready to create an ECS Service. An ECS Service is a definition that allows you to specify how your cluster will be managed. Navigate to your cluster’s Service page and press the Create button to create your new Moose service. The section we’re interested in is the Deployment configuration section. There, you’ll specify the Task Definition you created earlier. You can also specify the name of your service—perhaps something creative like moose-service—and the number of tasks to launch.

The remaining sections on the create service page allow you to specify networking considerations and whether you’ll use a load balancer.

You can press the Create button to launch an instance of your new ECS Moose service.

Setting up health checks

Your generated Moose containers include a health check endpoint at /health that should be configured in your ECS service. We recommend configuring the following health check settings:

Container-level Health Check

In your task definition’s container configuration:

healthCheck:
  command: ["CMD-SHELL", "curl -f http://localhost:4000/health || exit 1"]
  interval: 30
  timeout: 5
  retries: 3
  startPeriod: 60

Load Balancer Health Check

If you’re using an Application Load Balancer:

1. Create a target group for your service
2. Set the health check path to /health
3. Configure appropriate health check settings:
   • Health check protocol: HTTP
   • Health check port: 4000
   • Health check path: /health
   • Healthy threshold: 2
   • Unhealthy threshold: 2
   • Timeout: 5 seconds
   • Interval: 15 seconds
   • Success codes: 200

These health check configurations ensure that your Moose service is properly monitored and that traffic is only routed to healthy containers.