Understanding the AWS Security Finding Format (ASFF)

Introduction: The Challenge of Security Data Normalization

In modern cloud environments, security professionals are inundated with data. You have logs from VPC Flow Logs, alerts from Amazon GuardDuty, configuration assessments from AWS Security Hub, and custom findings from internal applications. The fundamental problem is that each of these sources speaks a different "language." A finding from an intrusion detection system looks nothing like a finding from a vulnerability scanner, which in turn looks nothing like a configuration compliance report.

When your security operations center (SOC) team tries to investigate an incident, they often spend more time parsing and normalizing these disparate data formats than actually analyzing the threats. This is where the AWS Security Finding Format (ASFF) becomes essential. ASFF is a standardized JSON schema designed by AWS to provide a common language for security findings across all services. By adopting a unified structure, ASFF allows security tools to interoperate, enables automated remediation workflows, and provides a single pane of glass for security visibility. Understanding ASFF is not just about learning a JSON schema; it is about building the foundation for a scalable, automated security architecture.

What is ASFF?

At its core, the AWS Security Finding Format is a JSON-based schema that defines the required and optional fields for a security finding within the AWS ecosystem. Whether you are using AWS Security Hub, Amazon GuardDuty, or a third-party security tool integrated with AWS, the ASFF ensures that every finding contains the same basic information, such as the severity, the resource involved, the timestamp, and the remediation steps.

The format is hierarchical, meaning it allows for complex nesting of data, which is necessary when describing sophisticated threats. It includes fields for identifying the specific AWS account, the region, the type of threat, and the specific resource identifier. Because it is standardized, you can write automation scripts—such as Lambda functions—that can process findings from any source without needing custom logic for each individual tool.

Callout: Why Normalization Matters In the absence of a standard like ASFF, security teams are forced to write "glue code" for every integration. If you have five security tools, you might need five different sets of parsers. ASFF eliminates this technical debt by forcing all participating tools to conform to a single, predictable structure, making your security pipeline modular and maintainable.

The Anatomy of an ASFF JSON Object

To work effectively with ASFF, you must understand its structure. A typical ASFF record is a JSON object that contains a set of top-level fields. Some of these are required for every finding, while others are optional but highly recommended for providing context.

Required Top-Level Fields

• SchemaVersion: The version of the ASFF schema being used. This ensures that your processing logic can handle changes in the format over time.
• Id: A unique identifier for the finding. This is critical for deduplication and tracking the lifecycle of an alert.
• ProductArn: The Amazon Resource Name (ARN) of the product that generated the finding. This tells you which tool or service is responsible for the alert.
• GeneratorId: A unique identifier for the specific detector or rule that triggered the finding within the product.
• AwsAccountId: The ID of the AWS account where the resource resides.
• Types: A list of classification strings that categorize the finding (e.g., "Software and Configuration Checks/Vulnerabilities/CVE").
• FirstObservedAt and LastObservedAt: Timestamps indicating when the issue was first detected and when it was most recently seen.
• CreatedAt and UpdatedAt: Timestamps reflecting when the finding record itself was created or modified in the database.
• Severity: An object containing the label (e.g., INFORMATIONAL, LOW, MEDIUM, HIGH, CRITICAL) and the numeric normalized value (0-100).
• Title and Description: Human-readable text explaining what the finding is and why it matters.

Example ASFF Structure

Consider a simple finding generated by a custom script that detects an S3 bucket with public read access. The JSON would look like this:

{
  "SchemaVersion": "2018-10-08",
  "Id": "arn:aws:securityhub:us-east-1:123456789012:subscription/custom-s3-scanner/finding/12345",
  "ProductArn": "arn:aws:securityhub:us-east-1:123456789012:product/custom-s3-scanner",
  "GeneratorId": "s3-public-read-detector",
  "AwsAccountId": "123456789012",
  "Types": ["Software and Configuration Checks/AWS Security Best Practices"],
  "FirstObservedAt": "2023-10-27T10:00:00Z",
  "LastObservedAt": "2023-10-27T10:00:00Z",
  "CreatedAt": "2023-10-27T10:05:00Z",
  "UpdatedAt": "2023-10-27T10:05:00Z",
  "Severity": {
    "Label": "CRITICAL",
    "Normalized": 90
  },
  "Title": "S3 Bucket Public Read Access",
  "Description": "The S3 bucket 'my-data-bucket' is publicly accessible via read permissions.",
  "Resources": [
    {
      "Type": "AwsS3Bucket",
      "Id": "arn:aws:s3:::my-data-bucket",
      "Partition": "aws",
      "Region": "us-east-1"
    }
  ]
}

Note: The Resources array is arguably the most important part of the ASFF. It links the abstract alert to a concrete asset in your environment. Always ensure that the Id field in the resource object matches the actual ARN of the AWS resource to enable seamless cross-referencing in the AWS Console.

Integrating Custom Tools with ASFF

One of the most powerful aspects of ASFF is that you are not limited to AWS-native tools. You can write your own security scanning tools—perhaps a Python script that checks for expired IAM access keys—and ingest those findings into AWS Security Hub by formatting them as ASFF.

Step-by-Step: Sending Custom Findings to Security Hub

1. Define your Product ARN: You must register your custom tool in Security Hub to obtain a unique Product ARN. This identifies your tool as the source of the findings.
2. Construct the JSON: Use a library like json in Python to build the dictionary object matching the ASFF schema described above.
3. Use the BatchImportFindings API: This is the primary method for pushing findings into Security Hub. You can send a list of findings in a single API call.
4. Handle Errors: Always implement retry logic. If the API returns a 400 or 500 error, your script should log the failure and attempt to resend after a short delay.

Practical Python Example

import boto3
import json
from datetime import datetime

# Initialize the Security Hub client
sh = boto3.client('securityhub', region_name='us-east-1')

def send_finding(finding_data):
    try:
        response = sh.batch_import_findings(Findings=[finding_data])
        if response['FailedCount'] > 0:
            print(f"Failed to import {response['FailedCount']} findings.")
    except Exception as e:
        print(f"Error: {str(e)}")

# Construct a finding
finding = {
    "SchemaVersion": "2018-10-08",
    "Id": "custom-script-101",
    "ProductArn": "arn:aws:securityhub:us-east-1:123456789012:product/my-org/my-scanner",
    "GeneratorId": "iam-key-age-check",
    "AwsAccountId": "123456789012",
    "Types": ["Software and Configuration Checks/Identity and Access Management"],
    "CreatedAt": datetime.utcnow().isoformat() + "Z",
    "UpdatedAt": datetime.utcnow().isoformat() + "Z",
    "Severity": {"Label": "MEDIUM"},
    "Title": "IAM Access Key Older Than 90 Days",
    "Description": "Access key for user 'admin' is 95 days old.",
    "Resources": [{"Type": "AwsIamAccessKey", "Id": "AKIAEXAMPLE123"}]
}

send_finding(finding)

Best Practices for Working with ASFF

As you begin to build your own detection logic using ASFF, there are several industry-standard practices that will save you significant headaches down the road.

Maintain Consistent ID Generation

The Id field is used by Security Hub to perform deduplication. If you generate a new, random ID for every execution of your scanner, Security Hub will treat every scan result as a new finding, leading to alert fatigue. Instead, use a deterministic ID based on the resource ARN and the vulnerability type. For example, f"{resource_arn}-{vulnerability_type}".

Leverage the Types Taxonomy

AWS provides a specific taxonomy for the Types field. While you can technically put any string there, using the official AWS taxonomy allows your findings to be categorized correctly in the Security Hub dashboard. This makes filtering and grouping much more effective for your SOC analysts.

Provide Remediation Context

The Remediation object within the ASFF is often overlooked. It allows you to provide a URL to a documentation page or a set of instructions on how to fix the issue. Including this field turns a passive alert into an actionable task for the engineering team.

"Remediation": {
  "Recommendation": {
    "Text": "Rotate the IAM access key using the IAM console or CLI.",
    "Url": "https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html"
  }
}

Warning: Avoid putting sensitive information (like actual secret keys or PII) into the Description or Title fields. These fields are often indexed by logging services and may be visible to users with read-only access to Security Hub. Always sanitize your findings before submitting them.

Common Pitfalls and How to Avoid Them

Even experienced security engineers fall into common traps when implementing ASFF. Being aware of these will help you maintain a clean and reliable security dashboard.

Pitfall 1: Overloading the Severity Field

Some teams assign "CRITICAL" to every finding to ensure it gets noticed. This is a recipe for disaster. When everything is critical, the team becomes desensitized, and real critical threats are ignored. Use the Normalized severity field (0-100) to allow for granular prioritization, and reserve "CRITICAL" for issues that pose an immediate risk to data integrity or availability.

Pitfall 2: Neglecting the UpdatedAt Field

If your automation script updates a finding (for example, if a vulnerability is still present after a re-scan), ensure that you update the UpdatedAt timestamp. If you don't, your dashboards will show old data, and you will lose the ability to track how long a vulnerability has been present in your environment.

Pitfall 3: Ignoring Resource Relationships

ASFF allows you to define RelatedFindings or even define dependencies between resources. Many developers treat findings as isolated events, but threats are rarely isolated. If you have multiple findings that are linked (e.g., an EC2 instance is compromised, and then an IAM role is assumed), linking these findings in the ASFF structure provides the context needed for root-cause analysis.

Comparison: ASFF vs. Custom Log Formats

Advanced Concepts: Workflow and Automation

Once your findings are normalized into ASFF, you can unlock the full potential of AWS automation. Because Security Hub emits events to Amazon EventBridge for every finding, you can trigger automated responses with minimal effort.

For example, if you detect a security group allowing SSH access (port 22) from the entire internet, you can create an EventBridge rule that filters for that specific GeneratorId and Type. The rule can then trigger a Lambda function that automatically updates the security group to restrict access to a known VPN IP range.

Step-by-Step: Automating Remediation with ASFF

1. Filter with EventBridge: Use an event pattern to match the specific finding.

   {
     "source": ["aws.securityhub"],
     "detail-type": ["Security Hub Findings - Imported"],
     "detail": {
       "findings": {
         "GeneratorId": ["my-custom-sg-detector"],
         "Severity": {"Label": ["CRITICAL"]}
       }
     }
   }
   

2. Trigger Lambda: The EventBridge rule points to your remediation Lambda function.
3. Parse the Event: The Lambda function receives the full ASFF object in the detail field of the event.
4. Execute Remediation: Extract the resource ID from the Resources list and perform the API call to fix the issue.
5. Update Finding (Optional): After the fix, you can update the finding status in Security Hub to "RESOLVED" so that the dashboard reflects the current state of the environment.

Callout: The Power of Event-Driven Security By relying on the ASFF structure in EventBridge, you decouple your detection logic from your remediation logic. You can change your detection rules (the "what") without modifying your remediation scripts (the "how"). This modularity is essential for managing complex cloud environments where policies change frequently.

Practical Scenarios for ASFF Implementation

To truly grasp the value of ASFF, consider a few real-world scenarios where this format provides tangible benefits.

Scenario A: Multi-Account Security Aggregation

In a large organization, you might have hundreds of AWS accounts. Each account generates its own logs and alerts. By using ASFF, you can aggregate all these findings into a central "Security Account." Because every finding is normalized, your security dashboard in the central account can display a unified view of the entire organization's risk profile, regardless of which account the finding originated from.

Scenario B: Integrating Third-Party Vulnerability Scanners

Suppose your company uses a third-party tool like Qualys or Tenable for vulnerability scanning. These tools can be configured to translate their proprietary alert format into ASFF before sending it to Security Hub. This allows your SOC analysts to continue using their preferred tool while ensuring that the data is treated the same way as AWS-native alerts, simplifying training and tool adoption.

Scenario C: Compliance Auditing

During a compliance audit, you need to prove that you are monitoring your environment for specific risks. With ASFF, you can generate reports that query for findings matching specific compliance frameworks (e.g., CIS Benchmarks). Since the Types field in ASFF is standardized, you can easily pull a report of all "Compliance" related findings across your entire infrastructure with a single query.

Troubleshooting ASFF Ingestion

If you find that your findings are not appearing in Security Hub, or they are appearing with missing data, follow this systematic troubleshooting checklist:

1. Check Permissions: The IAM role or user pushing the findings must have the securityhub:BatchImportFindings permission. If the permission is missing, the API call will return a 403 Forbidden error.
2. Validate JSON Schema: Use a JSON validator tool to ensure your finding structure is strictly compliant with the ASFF schema. A single missing comma or a mismatched bracket will cause the API to reject the finding.
3. Check ProductArn: Ensure the ProductArn used in the finding matches a product that is enabled in your Security Hub instance. If the product is not registered, the finding may be silently dropped.
4. Check Region Consistency: Security Hub is a regional service. Ensure that you are sending the findings to the same region where your Security Hub instance is active.
5. Review CloudWatch Logs: If you are using Lambda to push findings, check the CloudWatch logs for that function. Look for error messages returned by the batch_import_findings API.

Future-Proofing Your Security Operations

As cloud environments grow in complexity, the ability to automate security becomes a survival skill. The AWS Security Finding Format is not just a format; it is a protocol for interaction. By standardizing your findings, you enable your team to move away from manual "triage and investigation" and toward "automated detection and response."

Consider the long-term lifecycle of your data. ASFF allows you to export your findings to S3 or Amazon OpenSearch (via Kinesis Firehose) for long-term storage and advanced analytics. Because the format is consistent, your data science team can perform trend analysis—such as identifying which services have the highest frequency of vulnerabilities—without having to write complex data cleaning scripts.

Key Takeaways

• Normalization is Essential: ASFF solves the problem of disparate security data by providing a common JSON schema, allowing for seamless integration across tools and services.
• Structure Matters: The hierarchical nature of ASFF (including Resources, Severity, and Types) provides the context necessary for both human analysts and automated systems to act on findings efficiently.
• Automation-Ready: By utilizing the standardized structure in EventBridge, you can build event-driven remediation workflows that automatically fix common security misconfigurations.
• Best Practices: Always use deterministic IDs for deduplication, sanitize your input to avoid leaking sensitive data, and leverage the official AWS taxonomy for finding types.
• Strategic Advantage: Standardizing on ASFF enables cross-account aggregation, simplifies compliance reporting, and future-proofs your security architecture for advanced analytics and machine learning.
• Troubleshooting: When issues arise, focus on IAM permissions, schema validation, and region alignment to ensure that your security pipeline remains stable and reliable.
• Think Big: Treat ASFF as a foundation for a broader security data lake strategy, where normalized logs and findings enable deep insights into your overall organizational security posture.

By mastering the AWS Security Finding Format, you are doing more than just following a technical specification. You are building a more transparent, efficient, and responsive security posture that can keep pace with the speed of cloud development. Start small—perhaps by converting one custom script to output ASFF—and gradually expand until your entire security ecosystem speaks this common language.