Deploying Virtual Machines from Images in Microsoft Azure

Introduction: The Foundation of Cloud Infrastructure

When we talk about cloud computing, the Virtual Machine (VM) remains the workhorse of the industry. While modern trends like serverless functions and containers often dominate the conversation, the reality is that the vast majority of enterprise applications still rely on traditional VM architectures. Understanding how to deploy and manage these VMs effectively is a core competency for any cloud engineer. In Azure, the process of deploying a VM is rarely done by starting from a blank slate. Instead, we use "images"—pre-configured templates that contain the operating system, necessary runtimes, and sometimes even pre-installed software.

Deploying VMs from images is fundamentally about consistency and scale. If you were to manually configure every single server in your environment, you would quickly encounter "configuration drift," where two servers that are supposed to be identical start to behave differently due to subtle differences in their setup. By using images, you ensure that every instance of a VM is an exact clone of a known-good configuration. This approach is the bedrock of infrastructure as code (IaC) and automation. Whether you are using the Azure Portal, the Azure CLI, or Terraform, the underlying mechanism of imaging allows you to move from development to production with confidence.

In this lesson, we will explore the lifecycle of Azure VM images, the difference between platform images and custom images, and the best practices for maintaining these assets. By the end of this guide, you will be able to design a strategy for deploying VMs that is repeatable, secure, and scalable.

Understanding Azure VM Images

An Azure VM image is essentially a snapshot of a virtual hard disk (VHD) that has been packaged in a way that allows Azure to instantiate new VMs from it. These images serve as the "blueprint" for your virtual infrastructure. When you request a new VM, Azure takes that blueprint and copies it onto the storage account associated with your new machine.

Platform Images vs. Custom Images

It is helpful to categorize images into two main groups based on who creates and maintains them:

• Platform Images: These are images provided directly by Microsoft or third-party vendors (such as Canonical for Ubuntu or Red Hat for RHEL). They are maintained in the Azure Marketplace. When you use these, you are getting a clean, base-level OS installation that has been verified to work with the Azure platform.
• Custom Images: These are images you create yourself. You might start with a Platform Image, install your specific application, configure security hardening, join it to your domain, and then "generalize" it into a custom image. This is the preferred method for enterprise environments where every server must have a specific set of security agents and monitoring tools installed by default.

Callout: Image Generalization vs. Specialization It is critical to understand the difference between a generalized image and a specialized image. A generalized image has had machine-specific information (like the computer name, user accounts, and unique IDs) stripped out using tools like Sysprep (Windows) or waagent (Linux). A specialized image is an exact copy of a VM that retains all that unique information. You must generalize an image before it can be used to deploy multiple instances; otherwise, you will face severe identity conflicts on your network.

The Azure Compute Gallery (formerly Shared Image Gallery)

If you are working in an organization with more than one or two VMs, managing images through raw VHD files in storage accounts is dangerous and inefficient. You will quickly lose track of which version of your image is the "gold" version. This is where the Azure Compute Gallery comes in.

The Azure Compute Gallery is a service that helps you build structure around your image deployments. It allows you to:

1. Version your images: You can tag an image as "v1.0.0" and later update it to "v1.0.1" without breaking existing deployments.
2. Replicate globally: You can define a policy to automatically copy your images to different Azure regions, ensuring that your deployments are fast regardless of where your data centers are located.
3. Share across subscriptions: If your company uses multiple Azure subscriptions for different business units, the gallery acts as a central repository that can be accessed by all of them.

Step-by-Step: Deploying a VM from a Marketplace Image

Let’s look at the most common scenario: deploying a standard VM from an Azure Marketplace image using the Azure CLI. This is the most efficient way to get started.

Prerequisites

• An active Azure subscription.
• The Azure CLI installed on your local machine.
• An existing Resource Group.

Step 1: Identify the Image

First, you need to find the "URN" (Uniform Resource Name) of the image you want to use. You can list the most popular images using the CLI:

az vm image list --output table

If you know you want an Ubuntu Server, you can filter the search:

az vm image list --publisher Canonical --offer UbuntuServer --all --output table

Step 2: Create the VM

Once you have the URN (e.g., Canonical:0001-com-ubuntu-server-focal:20_04-lts:latest), you can proceed to create the VM.

az vm create \
  --resource-group MyResourceGroup \
  --name MyWebServer \
  --image Canonical:0001-com-ubuntu-server-focal:20_04-lts:latest \
  --admin-username azureuser \
  --generate-ssh-keys \
  --size Standard_DS1_v2

Explanation of the Command

• --resource-group: Defines where the VM metadata will live.
• --name: The unique identifier for your VM.
• --image: The URN we identified earlier.
• --admin-username: The default account created for management.
• --generate-ssh-keys: Automatically creates a public/private key pair to ensure secure access without passwords.
• --size: Determines the CPU and RAM of the machine.

Note: Always choose the smallest VM size that meets your performance requirements. You can easily resize a VM later, but running an oversized instance leads to wasted budget.

Creating and Using Custom Images

While Marketplace images are great for testing, most production environments require custom images. The workflow for creating a custom image follows a specific pattern: create a source VM, customize it, generalize it, and capture it.

The Workflow

1. Provision a Source VM: Deploy a standard Marketplace image.
2. Install Software: Log in to the VM and install your specific application dependencies (e.g., Nginx, Python, logging agents).
3. Generalize the OS:
   • For Linux: Run sudo waagent -deprovision+user. This removes the SSH host keys and the user account you used to configure the machine.
   • For Windows: Run C:\Windows\System32\Sysprep\sysprep.exe /generalize /shutdown /oobe.
4. Capture: Use the Azure CLI or Portal to capture the disk as an Image Version in your Compute Gallery.

CLI Command to Capture an Image

After the source VM is shut down and generalized, you use the following command:

az sig image-version create \
  --resource-group MyGalleryRG \
  --gallery-name MyComputeGallery \
  --gallery-image-definition MyImageDef \
  --gallery-image-version 1.0.0 \
  --managed-image /subscriptions/.../resourceGroups/MyRG/providers/Microsoft.Compute/virtualMachines/MySourceVM

Once the image is captured, you can deploy new VMs using that specific version URN, ensuring that every new machine comes with your pre-installed software and security configurations.

Best Practices for Image Management

Managing images is an operational task that requires discipline. If you treat your images like "pet" servers—manually tweaking them and forgetting what changes were made—you will eventually face a crisis. Here are the industry standards for managing VM images in Azure.

1. Automate Image Creation

Never manually configure your gold image. Instead, use a tool like HashiCorp Packer. Packer allows you to write a configuration file (a template) that defines exactly what software to install. It spins up a temporary VM, applies the configuration, generalizes the OS, and captures the image automatically. This makes your image creation process repeatable and version-controlled.

2. Implement a Versioning Strategy

Use semantic versioning (Major.Minor.Patch) for your images.

• Major: Breaking changes (e.g., upgrading from Ubuntu 20.04 to 22.04).
• Minor: New features added (e.g., adding a new monitoring plugin).
• Patch: Security updates or bug fixes.

3. Lifecycle Management

Images do not stay secure forever. An image created six months ago is likely vulnerable to new security threats. Establish a policy to rotate your images. You should aim to rebuild your gold images at least once a month to include the latest OS patches.

4. Use Managed Disks

Always use Managed Disks for your VM images. They are handled by Azure's underlying storage fabric, which provides better durability and performance than manual storage accounts. Managed Disks allow you to scale to thousands of VMs without worrying about hitting storage account limits.

Warning: The Security Trap Do not store sensitive information like hardcoded API keys, database passwords, or SSH private keys inside your base images. If that image is ever shared or compromised, your secrets are exposed. Always use Azure Key Vault or Managed Identities to handle authentication and secrets at runtime, not at image-build time.

Comparison: Deployment Methods

When deciding how to deploy your images, you have several options. The following table compares the most common approaches.

Choosing the Right Tool

• The Portal is excellent for visual learners and for exploring settings. However, it is not reproducible. If you need to deploy the same environment in a different region, you will have to click through the same menus again, which is prone to human error.
• The CLI is perfect for quick, repeatable tasks. It is ideal for developers who want to spin up a dev environment with a single command.
• Terraform/Bicep (IaC) is the industry standard for production. These tools allow you to describe your entire infrastructure in code. If you delete your resource group, you can simply run your script again to recreate the exact environment.

Common Pitfalls and How to Avoid Them

Even experienced engineers run into trouble when deploying VMs from images. Being aware of these common mistakes can save you hours of debugging.

Pitfall 1: Failing to Generalize the Image

If you capture a VM without running Sysprep (Windows) or waagent (Linux), the resulting image will contain unique identifiers like the computer name and the SSH host keys. When you deploy two VMs from that non-generalized image, they will conflict with each other on the network.

• Solution: Always verify that your source VM has been generalized before running the capture command.

Pitfall 2: Over-provisioning Resources

It is tempting to assign 16GB of RAM to every VM "just in case." In a cloud environment, this leads to massive, unnecessary monthly bills.

• Solution: Monitor your VMs using Azure Monitor. If you see that your average memory usage is under 20%, resize the VM to a smaller SKU.

Pitfall 3: Ignoring Regional Latency

If your users are in Europe but your image is stored in a gallery in the US, the initial deployment of your VMs will be slow as Azure pulls the disk data across the ocean.

• Solution: Use the Azure Compute Gallery's replication feature to push your images to the regions where you actually deploy your VMs.

Pitfall 4: Neglecting OS Patching

Deploying from an image is only the first step. Once the VM is running, it starts to age. If you deploy a VM from a three-month-old image, the first thing that VM will do is spend 30 minutes downloading updates.

• Solution: Use Azure Update Manager to orchestrate patching, or integrate image rebuilding into your CI/CD pipeline so that your "latest" image is always up to date.

Practical Exercise: Your First Image Pipeline

To wrap up this lesson, let's walk through a conceptual workflow for a robust image pipeline. This is how a professional cloud team operates.

1. Code: You write a Packer template that defines the Ubuntu 22.04 base.
2. Build: You trigger a build in a CI/CD tool (like GitHub Actions or Azure DevOps).
3. Test: The build process spins up a temp VM, runs the configuration, and then runs a "smoke test" script to ensure the app is running.
4. Publish: If the test passes, the image is published to the Azure Compute Gallery with a new version number (e.g., 2.1.4).
5. Deploy: Your production Terraform script is updated to point to version 2.1.4, and the infrastructure is updated.

This flow ensures that no human touches the production image. Every change is tracked in version control, and every image is tested before it touches a production VM.

Advanced Configuration: Cloud-Init and Custom Data

Sometimes, you don't need a completely new image for every minor change. You can use Cloud-init (for Linux) or Custom Data (for Windows) to configure a VM at the moment of deployment.

Cloud-init is a script that runs the very first time a Linux VM boots. You can pass this script to the az vm create command using the --custom-data flag.

Example: A simple cloud-init script (cloud-config.yaml)

#cloud-config
package_update: true
packages:
  - nginx
runcmd:
  - systemctl start nginx
  - systemctl enable nginx

Executing with CLI:

az vm create \
  --resource-group MyRG \
  --name MyWebVM \
  --image Canonical:0001-com-ubuntu-server-focal:20_04-lts:latest \
  --custom-data cloud-config.yaml

This approach is powerful because it allows you to use a single "base" image for many different types of servers. You use the base image for the OS, and the cloud-init script to turn that base image into a web server, a database server, or a worker node.

Callout: Image vs. Scripting When should you use a custom image versus using cloud-init? Use a custom image when you have heavy software that takes a long time to install or requires complex system-level configurations. Use cloud-init for lightweight configuration, environment variables, or simple package installations.

Security Considerations for VM Images

Security should be baked into your images, not added after the fact. When you are preparing an image, follow these security principles:

1. Minimalism: Only install what is necessary. Every extra service or package you include in your image is a potential attack vector. If a service isn't required for your application to function, remove it.
2. Hardening: Disable unused ports, remove unnecessary default users, and ensure that root login via SSH is disabled.
3. Encryption: Always enable Azure Disk Encryption or use Customer-Managed Keys (CMK) to encrypt your managed disks. Even if someone gains physical access to the underlying storage hardware, they will not be able to read your data.
4. Scanning: Integrate image scanning tools. Services like Microsoft Defender for Cloud can scan your images for known vulnerabilities (CVEs) before you deploy them. If an image has a critical vulnerability, the scan will alert you, and you can fix it before it ever reaches production.

Troubleshooting Deployment Failures

Despite your best efforts, you will eventually encounter a deployment failure. Here are the most common error types when deploying VMs from images:

• Marketplace Terms: Some images (like SQL Server or specific Linux distributions) require you to "accept the terms" before you can deploy them. If you get a "Marketplace terms not accepted" error, you must accept them via the CLI or Portal once for your subscription.
  • Fix: az vm image terms accept --urn <URN>
• SKU Mismatch: You might try to deploy a VM size that isn't available in the region you selected.
  • Fix: Check the availability of the VM size in your specific region using az vm list-sizes --location <region>.
• Quota Limits: Azure enforces limits on how many vCPUs you can use in a region. If you exceed this, your deployment will fail.
  • Fix: Check your usage in the Azure Portal under "Subscriptions > Usage + quotas" and request an increase if necessary.
• Image Version Not Found: If you are using a shared gallery, ensure the image version is replicated to the region where you are deploying. If the image is in "West US" and you deploy in "East US," the deployment will fail if the image hasn't been replicated.

Summary and Key Takeaways

Deploying VMs from images is a fundamental skill that bridges the gap between manual server administration and automated cloud operations. By leveraging images, you move away from the fragility of manual configuration and toward a model of predictable, scalable, and secure infrastructure.

Key Takeaways

1. Consistency is King: Always use images to ensure that your virtual machine instances are identical, reducing configuration drift and making your environment easier to troubleshoot.
2. Use the Azure Compute Gallery: Stop managing raw VHDs. Use the Gallery to version, replicate, and share your images across your organization safely.
3. Automate with Packer: Never manually configure a gold image. Use tools like HashiCorp Packer to build your images from code, ensuring that your build process is documented and repeatable.
4. Generalize Before Capturing: Always run sysprep or waagent to generalize your source VM before capturing it into an image. This prevents identity and network conflicts when you scale out your fleet.
5. Lifecycle Management: Images are not "set it and forget it." Establish a schedule to rebuild and patch your images regularly to ensure your infrastructure stays secure against the latest threats.
6. Minimalism and Security: Build your images to be as small and lean as possible. Remove unnecessary packages and integrate vulnerability scanning into your pipeline to catch issues before deployment.
7. Know Your Tools: Use the Azure CLI for quick tasks, but adopt IaC tools like Terraform or Bicep for production environments to maintain a "source of truth" for your infrastructure configuration.

By mastering these concepts, you shift your role from someone who "manages servers" to someone who "manages platforms." This transition is essential for building resilient, modern applications in the cloud. Remember that the goal is always to reduce the manual effort required to keep your environment running, allowing you to focus on the applications and services that bring value to your users.