Creating a .NET Core Worker Service and Deploying as a Cron Job in AWS EKS and Kubernetes

Automate Periodic Tasks with .NET Core and AWS Kubernetes

Background tasks are essential for running scheduled operations like database cleanup, data synchronization, or report generation. In this blog, we’ll walk you through creating a .NET Core WorkerService, deploying it as a cron job in AWS Elastic Kubernetes Service (EKS), and scheduling it to run at the 30th minute of every hour.

Overview

We’ll cover:

1. Creating a Worker Service in .NET Core.
2. Writing the task logic in the BackgroundService.
3. Building and pushing the Docker image.
4. Writing a Kubernetes CronJob manifest.
5. Deploying the CronJob to AWS EKS.
6. Verifying the schedule and logs.

📌Explore more at: https://dotnet-fullstack-dev.blogspot.com/
🌟 Clapping would be appreciated! 🚀

Step 1: Create a .NET Core WorkerService

1.1 Initialize a Worker Service Project

Run the following command to create a new Worker Service project:

dotnet new worker -n HourlyTaskService
cd HourlyTaskService

This creates a worker template, which is ideal for running background tasks in .NET Core.

1.2 Write the Background Task Logic

Open the Worker.cs file and implement the logic for your task. For example, this service logs the current timestamp as a placeholder for real tasks.

using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using System;
using System.Threading;
using System.Threading.Tasks;

public class Worker : BackgroundService
{
    private readonly ILogger<Worker> _logger;

    public Worker(ILogger<Worker> logger)
    {
        _logger = logger;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        _logger.LogInformation("Background service started.");

        while (!stoppingToken.IsCancellationRequested)
        {
            _logger.LogInformation($"Task executed at: {DateTime.Now}");
            await Task.Delay(TimeSpan.FromMinutes(1), stoppingToken); // Replace with actual task logic
        }
    }

    public override Task StopAsync(CancellationToken stoppingToken)
    {
        _logger.LogInformation("Background service stopped.");
        return base.StopAsync(stoppingToken);
    }
}

1.3 Add Logging Configuration

Update the appsettings.json file to include logging configuration for better visibility:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Warning"
    }
  }
}

1.4 Test Locally

Run the Worker Service:

dotnet run

Check the console output for logs like:

info: Worker[0]
      Task executed at: 12/14/2024 13:30:00

Step 2: Containerize the .NET Core WorkerService

2.1 Add a Dockerfile

Create a Dockerfile in the project root:

# Build stage
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /app
COPY . .
RUN dotnet publish -c Release -o out

# Runtime stage
FROM mcr.microsoft.com/dotnet/runtime:6.0
WORKDIR /app
COPY --from=build /app/out .
ENTRYPOINT ["dotnet", "HourlyTaskService.dll"]

2.2 Build and Push the Docker Image

Build the Docker image:

docker build -t your-dockerhub-username/hourly-task-service:latest .

Push the image to Docker Hub (or another container registry):

docker push your-dockerhub-username/hourly-task-service:latest

Step 3: Create a Kubernetes CronJob Manifest

3.1 Write the CronJob YAML

Create a file named cronjob.yaml with the following content:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: hourly-task-service
spec:
  schedule: "30 * * * *" # Runs at the 30th minute of every hour
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: hourly-task-service
            image: your-dockerhub-username/hourly-task-service:latest
            imagePullPolicy: Always
            resources:
              requests:
                memory: "128Mi"
                cpu: "250m"
              limits:
                memory: "256Mi"
                cpu: "500m"
          restartPolicy: OnFailure

3.2 Schedule Explanation

The schedule field uses Cron format:

• 30 * * * *: Runs at the 30th minute of every hour.

Step 4: Deploy the CronJob to AWS EKS

4.1 Set Up AWS EKS Cluster

Ensure you have an EKS cluster ready. If not, create one using AWS CLI or AWS Management Console.

Configure kubectl to use the EKS cluster:

aws eks update-kubeconfig --region <region-name> --name <cluster-name>

Verify the cluster connection:

kubectl get nodes

4.2 Deploy the CronJob

Apply the cronjob.yaml file:

kubectl apply -f cronjob.yaml

Verify the deployment:

kubectl get cronjob

Expected output:

NAME                  SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
hourly-task-service   30 * * * *   False     0        <none>           2m

Step 5: Verify the CronJob Execution

Check for job executions:

kubectl get jobs

View logs for a specific pod:

kubectl logs <pod-name>

You should see logs like:

info: Worker[0]
      Task executed at: 12/14/2024 13:30:00

Best Practices

Resource Optimization:

• Configure resources in the CronJob manifest to limit memory and CPU usage.

Monitoring and Alerts:

• Use tools like AWS CloudWatch or Prometheus to monitor CronJob performance.

Error Handling:

• Implement retry mechanisms or error logs for better resilience.

Image Tagging:

• Use versioned tags for Docker images to ensure predictable deployments.

Conclusion

By combining .NET Core Worker Services and Kubernetes CronJobs, you can automate periodic tasks efficiently. This setup, deployed on AWS EKS, ensures scalability, reliability, and ease of management. With the power of Kubernetes, scheduling tasks like running at the 30th minute of every hour becomes seamless.

Let me know if you have questions or need further assistance with your deployment!