Configuring Diagnostic Settings

Introduction

In any robust cloud environment, understanding the behavior, performance, and security posture of your resources is paramount. Without proper logging and monitoring, you're essentially operating blind, unable to diagnose issues, detect security threats, or optimize performance. Azure Diagnostic Settings are the fundamental mechanism that allows you to collect crucial operational data from nearly every Azure resource.

This lesson will delve into what Diagnostic Settings are, why they are indispensable for a well-governed and monitored Azure environment, and how to configure them effectively. By the end of this lesson, you'll be equipped to design and implement a logging strategy that captures the right data and directs it to the appropriate destinations for analysis and action.

What are Diagnostic Settings?

Azure Diagnostic Settings define where and what operational data (logs and metrics) should be sent from an Azure resource. Almost every Azure service, from Virtual Machines and App Services to SQL Databases and Key Vaults, generates various types of logs and metrics. Diagnostic Settings act as the conduit, allowing you to export this data to external services for storage, analysis, streaming, or integration.

Why are Diagnostic Settings Important?

Configuring Diagnostic Settings is a critical step in establishing a comprehensive logging and monitoring solution for several reasons:

1. Troubleshooting and Debugging: Detailed logs help identify the root cause of application errors, performance bottlenecks, or infrastructure issues.
2. Performance Optimization: Metrics provide insights into resource utilization, allowing you to scale resources appropriately and optimize costs.
3. Security and Compliance: Audit logs track who did what, when, and where, which is vital for security investigations, incident response, and meeting compliance requirements (e.g., GDPR, HIPAA).
4. Operational Insights: Centralized logging enables correlation of events across multiple resources, providing a holistic view of your application's health and behavior.
5. Proactive Alerting: By sending data to services like Log Analytics, you can configure alerts based on specific log patterns or metric thresholds, enabling proactive issue resolution.

Understanding Diagnostic Logs and Metrics

Diagnostic Settings allow you to select specific categories of logs and metrics to export.

• Logs: These are event-driven records of actions, errors, warnings, and informational messages generated by the resource. Examples include:
  • Audit Logs: Record control plane operations (e.g., resource creation, deletion, configuration changes).
  • Application Logs: Output from applications running on services like App Service.
  • Resource Specific Logs: Logs unique to the resource type, such as SQL Database query logs, Key Vault audit logs, Network Security Group flow logs, etc.
• Metrics: These are numerical values that represent the performance or health of a resource over time (e.g., CPU utilization, memory usage, network ingress/egress, database connections). Metrics are typically less detailed than logs but are excellent for trending and performance monitoring.

Destination Options

Diagnostic Settings allow you to send collected logs and metrics to one or more of the following destinations:

Log Analytics Workspace

• Purpose: Centralized log collection, storage, and analysis. Ideal for operational monitoring, querying logs across multiple resources, and creating dashboards and alerts.
• Benefits: Powerful Kusto Query Language (KQL), long-term retention, integration with Azure Monitor features like Workbooks and Alerts.
• Use Case: The recommended destination for most enterprise logging and monitoring strategies.

Azure Storage Account

• Purpose: Archiving logs for long-term retention or compliance.
• Benefits: Cost-effective for massive volumes of data that don't require immediate analysis.
• Use Case: Storing audit logs for several years to meet regulatory requirements, where querying is infrequent.

Azure Event Hubs

• Purpose: Streaming logs to external systems for real-time processing or integration with third-party SIEM (Security Information and Event Management) tools.
• Benefits: Low-latency streaming, highly scalable.
• Use Case: Feeding logs into a custom analytics engine, Splunk, Azure Stream Analytics, or other security tools.

Partner Integrations

• Purpose: Directly integrate with specific third-party monitoring solutions.
• Benefits: Simplified setup for pre-integrated solutions.
• Use Case: Sending logs directly to services like Datadog, Sumo Logic, or Dynatrace if available as a direct integration.

Configuring Diagnostic Settings

You can configure Diagnostic Settings using the Azure Portal, Azure CLI, Azure PowerShell, or Azure Resource Manager (ARM) templates.

Azure Portal (Practical Example: Azure App Service)

Let's configure Diagnostic Settings for an Azure App Service.

1. Navigate to your Resource: In the Azure Portal, search for and select your App Service.
2. Access Diagnostic Settings: In the left-hand menu, under the "Monitoring" section, select "Diagnostic settings".
3. Add Diagnostic Setting: Click "+ Add diagnostic setting".
4. Configure Settings:
   • Diagnostic setting name: Provide a meaningful name (e.g., WebApp-Prod-Logs-to-LogAnalytics).
   • Logs: Under "Logs", select the log categories you want to collect. For an App Service, common choices include:
     • AppServiceHTTPLogs
     • AppServiceConsoleLogs
     • AppServiceAuditLogs
     • AppServicePlatformLogs
   • Metrics: Under "Metrics", select AllMetrics.
   • Destination details: Choose your desired destination(s).
     • Select "Send to Log Analytics workspace".
     • Choose your Subscription and an existing Log Analytics workspace. If you don't have one, create it first.
5. Save: Click "Save".

Once saved, logs and metrics will start flowing to your selected destination within a few minutes.

Azure CLI

The Azure CLI provides a powerful way to automate the configuration of Diagnostic Settings.

First, ensure you have a Log Analytics Workspace ID and your resource ID.

# Get Log Analytics Workspace ID
LA_WORKSPACE_ID=$(az monitor log-analytics workspace show --resource-group <YourResourceGroup> --workspace-name <YourLogAnalyticsWorkspaceName> --query id -o tsv)

# Get the Resource ID for your App Service
APP_SERVICE_ID=$(az webapp show --resource-group <YourResourceGroup> --name <YourAppServiceName> --query id -o tsv)

# Create a diagnostic setting to send all logs and metrics to Log Analytics
az monitor diagnostic-settings create \
  --name "WebApp-Prod-CLI-Logs" \
  --resource $APP_SERVICE_ID \
  --logs '[{"category": "AppServiceHTTPLogs", "enabled": true}, {"category": "AppServiceConsoleLogs", "enabled": true}, {"category": "AppServiceAuditLogs", "enabled": true}, {"category": "AppServicePlatformLogs", "enabled": true}]' \
  --metrics '[{"category": "AllMetrics", "enabled": true}]' \
  --workspace $LA_WORKSPACE_ID

Azure PowerShell

Similar to CLI, PowerShell offers cmdlets for managing Diagnostic Settings.

# Get Log Analytics Workspace Resource ID
$workspaceId = (Get-AzOperationalInsightsWorkspace -ResourceGroupName "<YourResourceGroup>" -Name "<YourLogAnalyticsWorkspaceName>").ResourceId

# Get the App Service Resource ID
$resourceId = (Get-AzWebApp -ResourceGroupName "<YourResourceGroup>" -Name "<YourAppServiceName>").Id

# Define log categories (example: App Service logs)
$logCategories = @(
    @{ Category = "AppServiceHTTPLogs"; Enabled = $true; RetentionPolicy = @{ Enabled = $false; Days = 0 } },
    @{ Category = "AppServiceConsoleLogs"; Enabled = $true; RetentionPolicy = @{ Enabled = $false; Days = 0 } },
    @{ Category = "AppServiceAuditLogs"; Enabled = $true; RetentionPolicy = @{ Enabled = $false; Days = 0 } },
    @{ Category = "AppServicePlatformLogs"; Enabled = $true; RetentionPolicy = @{ Enabled = $false; Days = 0 } }
)

# Define metrics (AllMetrics)
$metricCategories = @(
    @{ Category = "AllMetrics"; Enabled = $true; RetentionPolicy = @{ Enabled = $false; Days = 0 } }
)

# Create/Update diagnostic setting
Set-AzDiagnosticSetting `
    -Name "WebApp-Prod-PS-Logs" `
    -ResourceId $resourceId `
    -Log $logCategories `
    -Metric $metricCategories `
    -WorkspaceId $workspaceId

Azure Resource Manager (ARM) Templates

For infrastructure as code (IaC) deployments, ARM templates are the preferred method to ensure consistent and automated configuration.

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "resourceName": {
      "type": "string",
      "metadata": {
        "description": "Name of the Azure resource (e.g., App Service) to configure diagnostic settings for."
      }
    },
    "resourceType": {
      "type": "string",
      "metadata": {
        "description": "Type of the Azure resource (e.g., Microsoft.Web/sites)."
      }
    },
    "logAnalyticsWorkspaceId": {
      "type": "string",
      "metadata": {
        "description": "Resource ID of the Log Analytics Workspace."
      }
    },
    "diagnosticSettingName": {
      "type": "string",
      "defaultValue": "default-diagnostic-setting",
      "metadata": {
        "description": "Name for the diagnostic setting."
      }
    }
  },
  "resources": [
    {
      "type": "Microsoft.Insights/diagnosticSettings",
      "apiVersion": "2017-05-01-preview",
      "name": "[parameters('diagnosticSettingName')]",
      "scope": "[resourceId(parameters('resourceType'), parameters('resourceName'))]",
      "properties": {
        "workspaceId": "[parameters('logAnalyticsWorkspaceId')]",
        "logs": [
          {
            "category": "AppServiceHTTPLogs",
            "enabled": true,
            "retentionPolicy": {
              "enabled": false,
              "days": 0
            }
          },
          {
            "category": "AppServiceConsoleLogs",
            "enabled": true,
            "retentionPolicy": {
              "enabled": false,
              "days": 0
            }
          },
          {
            "category": "AppServiceAuditLogs",
            "enabled": true,
            "retentionPolicy": {
              "enabled": false,
              "days": 0
            }
          },
          {
            "category": "AppServicePlatformLogs",
            "enabled": true,
            "retentionPolicy": {
              "enabled": false,
              "days": 0
            }
          }
        ],
        "metrics": [
          {
            "category": "AllMetrics",
            "enabled": true,
            "retentionPolicy": {
              "enabled": false,
              "days": 0
            }
          }
        ]
      }
    }
  ]
}

Best Practices for Diagnostic Settings

• Centralize with Log Analytics: For most operational monitoring and analysis, send logs and metrics to a Log Analytics Workspace. This enables cross-resource querying, unified alerting, and powerful visualization tools.
• Automate Deployment: Use ARM templates, Azure CLI, or Azure PowerShell to configure Diagnostic Settings as part of your resource deployment. This ensures consistency and prevents oversight.
• Enable for All Critical Resources: Ensure all production and critical non-production resources have appropriate Diagnostic Settings configured. Consider using Azure Policy to enforce this.
• Select Relevant Log Categories: Don't enable every log category if you don't need it. This reduces ingestion costs and noise. However, err on the side of caution for security-sensitive resources.
• Understand Retention Policies:
  • Log Analytics: Retention is configured at the Log Analytics Workspace level, and for individual tables within the workspace.
  • Storage Accounts: Retention is configured within the Diagnostic Setting itself. Set it according to compliance requirements and cost considerations.
  • Event Hubs: Retention is managed by the consumer of the Event Hub.
• Cost Management: Be mindful of the costs associated with data ingestion and retention in Log Analytics and Storage Accounts. Regularly review your data volume and adjust retention policies as needed.
• Security: Ensure the Log Analytics Workspace and Storage Accounts receiving diagnostic data are properly secured with Azure RBAC and, where applicable, private endpoints.
• Naming Conventions: Use clear and consistent naming conventions for your diagnostic settings (e.g., [ResourceName]-[Environment]-[Destination]-Logs).

Callout: Azure Policy for Governance

You can use Azure Policy to enforce the creation of diagnostic settings for specific resource types, ensuring that new resources automatically comply with your logging standards. This is a powerful governance tool to maintain a consistent monitoring posture across your subscriptions.

Common Pitfalls

• Not Enabling Diagnostic Settings: The most common pitfall is simply forgetting or neglecting to enable diagnostic settings, leaving critical resources unmonitored.
• Sending to the Wrong Destination: Configuring logs to go to a Storage Account when real-time analysis