Lesson: Compute Decision Framework

Introduction

In modern cloud architecture, "compute" is the engine that powers your applications. Whether you are running a static website, a complex microservices architecture, or a machine learning training pipeline, choosing the right compute service is a critical architectural decision.

A Compute Decision Framework is a systematic approach to evaluating workload requirements against available cloud compute primitives (e.g., Virtual Machines, Containers, Serverless functions). The goal is to balance performance, cost, operational overhead, and scalability. Choosing the wrong compute model can lead to ballooning costs, technical debt, or application performance bottlenecks.

The Decision Framework: A Four-Pillar Approach

When designing compute solutions, evaluate every workload against these four pillars:

1. Control vs. Abstraction

• High Control (IaaS): You manage the OS, runtime, and patches (e.g., Virtual Machines). Use this when you have legacy dependencies or need specific kernel-level tuning.
• High Abstraction (FaaS/Serverless): The provider manages everything except the code (e.g., AWS Lambda, Google Cloud Functions). Use this for event-driven, sporadic workloads.

2. Workload Predictability

• Steady State: Predictable traffic patterns are best served by fixed-size instances or auto-scaled VM groups.
• Burst/Spiky: Highly variable traffic is best suited for Serverless or Container Orchestration (K8s) with aggressive horizontal scaling.

3. Application Architecture

• Monolithic: Often easier to lift-and-shift into Virtual Machines.
• Microservices: Highly suited for Container Orchestration (Kubernetes/ECS).
• Event-Driven: Ideal for Serverless Functions triggered by object storage, database changes, or API requests.

4. Operational Overhead

• Managed Services: If your team is small, prefer PaaS (Platform as a Service) or Serverless to minimize "undifferentiated heavy lifting" like OS patching and scaling logic.

Practical Examples

Scenario A: The Legacy Migration

Requirement: You have a legacy Windows application that requires a specific version of .NET Framework and direct access to a local file system.

• Decision: Virtual Machines (IaaS).
• Why: Containers or Serverless cannot easily accommodate the rigid OS dependencies of legacy monolithic applications.

Scenario B: The Web API

Requirement: A high-traffic REST API that must scale from 10 to 10,000 requests per second in minutes.

• Decision: Containers (Kubernetes/ECS).
• Why: Containers provide the portability and rapid spin-up times required for microservices, while Kubernetes manages the complex networking and scaling logic.

Scenario C: The Data Processor

Requirement: A service that triggers once a day to process log files uploaded to an S3 bucket.

• Decision: Serverless (e.g., AWS Lambda).
• Why: You only pay for the execution time. There is no need to keep a server running 24/7 for a job that takes 5 minutes to run.

Code Snippet: Infrastructure as Code (IaC) Context

When deploying compute, use IaC to ensure consistency. Below is a simplified example of how compute definitions look in Terraform for a Virtual Machine vs. a Lambda Function.

Virtual Machine (Control-heavy):

resource "aws_instance" "web_server" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t3.medium"
  # Requires management of OS patches and updates
}

Serverless Function (Abstraction-heavy):

resource "aws_lambda_function" "processor" {
  function_name = "log_processor"
  runtime       = "python3.9"
  handler       = "index.handler"
  # Provider handles all OS/Runtime patching
}

Best Practices

1. Start Small: Don't over-engineer. Start with a managed service (PaaS/Serverless) unless you have a specific reason to manage the OS.
2. Right-size Constantly: Use cloud-native monitoring (CloudWatch, Prometheus) to check CPU/Memory utilization. If you are consistently below 20% utilization, downsize your instances.
3. Design for Statelessness: Regardless of the compute type, design your application to be stateless. This allows you to terminate and replace instances without losing data.
4. Security First: Use the principle of least privilege for compute identities. If your code doesn't need to write to a database, don't give the compute resource that IAM role.

Common Pitfalls

• "The VM Default": Many architects default to Virtual Machines because they are familiar. This often leads to higher operational costs and slower deployment cycles.
• Ignoring Cold Starts: In Serverless, "cold starts" (latency when a function wakes up) can ruin user experience for latency-sensitive applications.
• Vendor Lock-in: While Serverless is efficient, heavy reliance on provider-specific triggers can make it difficult to migrate your architecture to another cloud provider in the future.
• Scaling Lag: Relying on VM-based auto-scaling for sudden traffic spikes can be dangerous, as VMs take minutes to boot, whereas containers take seconds.

💡 Pro-Tip: The "Container First" Strategy

Many modern organizations adopt a "Container First" policy. By packaging applications in containers, you gain the benefits of portability (run it on any cloud) and consistent deployment environments, while still having the flexibility to move to Kubernetes if the application complexity grows.

Key Takeaways

• Match the tool to the workload: Use VMs for legacy/control, Containers for microservices/portability, and Serverless for event-driven/sporadic tasks.
• Evaluate Total Cost of Ownership (TCO): Consider not just the cost of the compute instance, but the cost of the engineer time required to maintain it.
• Automate Everything: Use IaC to define your compute resources. Manual configuration leads to "snowflake" servers that are impossible to replicate or troubleshoot.
• Embrace Observability: You cannot optimize what you do not measure. Ensure your compute solution integrates with logging and tracing tools from day one.