Data Classification Capabilities in Microsoft Purview

In the modern digital landscape, data is often described as the new oil. However, unlike oil, data is not a finite resource; it is expanding at an exponential rate. Organizations today deal with a massive influx of documents, emails, chat messages, and spreadsheets spread across cloud environments, on-premises servers, and endpoint devices. The primary challenge isn't just storing this data, but understanding what it contains. Without knowing what is inside your files, you cannot effectively protect them, satisfy regulatory requirements, or prevent accidental leaks. This is where data classification comes into play.

Data classification is the process of identifying, categorizing, and labeling content based on its sensitivity, value, and the impact its loss would have on the organization. In the Microsoft ecosystem, specifically within Microsoft Purview, data classification serves as the foundational layer for all other compliance and security actions. You cannot apply an encryption policy if you don't know which files are "Confidential." You cannot set a 10-year retention period if you don't know which documents are "Financial Records." This lesson explores the sophisticated tools Microsoft provides to automate and refine this identification process, moving far beyond simple keyword searches into the realms of pattern matching, machine learning, and fingerprinting.

The Foundation: Why Classification Matters

Before we dive into the technical "how," we must understand the "why." Data classification isn't just an IT exercise; it is a business necessity driven by three main factors: risk management, regulatory compliance, and operational efficiency. When data is classified, the organization can prioritize its security spend on the most critical assets rather than trying to protect everything with the same level of intensity, which is often impossible and prohibitively expensive.

From a regulatory standpoint, frameworks like GDPR (General Data Protection Regulation), HIPAA (Health Insurance Portability and Accountability Act), and CCPA (California Consumer Privacy Act) require organizations to know where personally identifiable information (PII) or protected health information (PHI) resides. If a customer requests that their data be deleted, or if an auditor asks for proof of data protection, a robust classification system provides the necessary visibility to respond accurately.

Callout: Classification vs. Labeling It is easy to use these terms interchangeably, but they represent different stages of the data lifecycle.

• Classification is the "detective" work. It is the logic used to determine what a piece of data is (e.g., "This looks like a credit card number").
• Labeling is the "action" or the "stamp." Once the classification logic identifies the data, a label (like "Highly Confidential") is applied to the file metadata to dictate how it should be handled.

Sensitive Information Types (SITs)

The most common way to classify data in Microsoft Purview is through Sensitive Information Types (SITs). A SIT is essentially a set of rules that the system uses to find specific types of data. Microsoft provides over 300 built-in SITs covering common data types like credit card numbers, passport numbers, and bank account details for various countries.

How SITs Work: The Logic of Detection

A SIT doesn't just look for a string of numbers. If it did, it would trigger a "false positive" every time someone typed a long serial number or a formatted date. Instead, SITs use a combination of elements:

1. Primary Element: This is usually a regular expression (Regex) or a specific pattern that the data must follow. For a credit card, this is the 16-digit string.
2. Supporting Evidence: To increase accuracy, the system looks for "keywords" near the primary element. For example, if the system finds a 16-digit number, it looks for words like "Visa," "MasterCard," "CVV," or "Expiry" within a 300-character window.
3. Confidence Level: Based on whether the primary element and supporting evidence are found, the system assigns a confidence level (Low, Medium, or High).
4. Checksums: Many SITs use mathematical formulas (like the Luhn algorithm for credit cards) to verify that the number is valid and not just a random sequence.

Custom Sensitive Information Types

While the built-in SITs are comprehensive, most businesses have unique data types, such as internal employee IDs, project codenames, or specific part numbers. In these cases, you can create Custom SITs using the Purview compliance portal or via XML for more complex requirements.

Example: Custom Employee ID Pattern

Imagine your company uses an Employee ID format that starts with "EMP," followed by a dash, two letters, and four numbers (e.g., EMP-AB1234). You can define a custom SIT to detect this.

<Entity id="f9284235-591a-4754-bf61-6696ef082402" patternsProximity="300" recommendedConfidenceLevel="85">
  <Pattern confidenceLevel="85">
    <IdMatch idRef="Regex_Employee_ID" />
    <Match idRef="Keyword_Employee" />
  </Pattern>
</Entity>

<Regex id="Regex_Employee_ID">EMP-[A-Z]{2}\d{4}</Regex>
<Keyword id="Keyword_Employee">
  <Group expiry="2025-01-01">
    <Term>Employee ID</Term>
    <Term>Staff Number</Term>
    <Term>Payroll ID</Term>
  </Group>
</Keyword>

In this XML snippet, we define the pattern using Regex and provide a list of keywords that must be nearby to confirm the match. This ensures that a random string matching the pattern isn't flagged unless it is actually being used in the context of employee identification.

Exact Data Match (EDM)

Standard SITs are great for finding patterns, but they struggle with "known" data. For example, if you want to find your specific customers' record numbers rather than just any 10-digit number, standard SITs aren't precise enough. This is where Exact Data Match (EDM) comes in.

EDM allows you to upload a schema or a database of actual values you want to protect. Microsoft Purview then creates a "fingerprint" of this data. When the system scans documents or emails, it looks for those exact values.

The EDM Process

The process for EDM is more rigorous than standard SITs because it involves handling actual sensitive data from your databases.

1. Prepare the Data: You export a CSV file from your HR or CRM system containing the data you want to track (e.g., First Name, Last Name, Date of Birth, Account Number).
2. Define the Schema: In the Purview portal, you create a schema that matches the columns in your CSV.
3. Hash and Upload: You use a specialized tool (the EDM Upload Agent) to hash the data on your own local server. Only the "hashes" (mathematical representations) are uploaded to Microsoft, not the actual plain-text data. This preserves privacy.
4. Create the EDM SIT: You link the uploaded hashes to a new Sensitive Information Type.

Note: EDM is particularly powerful for preventing "False Positives." Because it is looking for specific, known values from your database, the likelihood of accidentally flagging a non-sensitive document is extremely low.

Trainable Classifiers

Not all data follows a pattern. How do you identify a "Legal Contract," a "Resume," or "Source Code"? These documents don't have a specific 16-digit number or a predictable Regex pattern. They are defined by their overall structure, language, and context.

Trainable Classifiers use Machine Learning (ML) to solve this problem. Microsoft provides several pre-trained classifiers for common categories:

• Source Code: Identifies various programming languages.
• Agreements: Detects legal contracts and terms of service.
• HR: Finds documents related to performance reviews or hiring.
• Discrimination/Harassment: Used in Communication Compliance to find offensive content.

Creating Your Own Classifier

If the built-in classifiers don't meet your needs—for example, if you need to identify "Project Phoenix Design Specs"—you can train your own.

1. Seed Content: You provide the system with 50-100 examples of the document type (the "positive" samples) and a set of documents that are definitely not that type (the "negative" samples).
2. Initial Training: The ML model analyzes the samples to find common linguistic patterns and structures.
3. Testing and Feedback: You provide the model with a new set of documents it hasn't seen before. You then review the results, telling the system "Yes, this is a match" or "No, this is a mistake."
4. Publishing: Once the model reaches an accuracy level you are happy with (usually 80% or higher), you publish it for use in your labeling policies.

Fingerprinting and Document Matching

Document Fingerprinting is a subset of classification specifically designed for standard forms. If your organization uses a specific "Patent Application Form" or a "Government Tax Form," the structure of the document itself is unique.

When you use Fingerprinting, the system analyzes the static text of the form (the headers, the instructions, the layout) and ignores the variable text (the data entered by the user). This allows it to recognize any completed version of that specific form as sensitive. This is highly effective for organizations that rely heavily on standardized templates for their core business processes.

Comparison of Classification Methods

Sensitivity Labels: Putting Classification to Work

Classification is the "brain," but Sensitivity Labels are the "hands." Once the classification engine identifies a document as containing sensitive data, you need a way to apply protection. Sensitivity labels are tags that stay with the document, regardless of where it travels—whether it is emailed to a partner, uploaded to a personal cloud drive, or saved to a USB stick.

Manual vs. Automatic Labeling

There are three main ways labels are applied based on classification:

1. Manual Labeling: The user chooses the label from a list in Word, Excel, or Outlook. This relies on user awareness and training.
2. Recommended Labeling: The system detects sensitive data (using a SIT) and shows a prompt to the user: "This looks like a Financial Report. Would you like to apply the Confidential label?"
3. Auto-Labeling: The system automatically applies the label without user intervention. This is ideal for ensuring compliance across large volumes of data where you cannot rely on every employee to make the right choice.

Site and Group Level Classification

Classification isn't limited to individual files. You can also classify entire "containers," such as Microsoft Teams, SharePoint Sites, and Microsoft 365 Groups.

When you apply a classification label to a Team, you can enforce high-level policies. For example, if a Team is labeled "Internal Only," the system can automatically block the addition of guest users to that Team and ensure the underlying SharePoint site is set to "Private." This provides a "macro" level of data governance that complements the "micro" level of file classification.

Visibility and Insights: Content Explorer and Activity Explorer

You cannot manage what you cannot see. Microsoft Purview provides two critical tools to help administrators understand how classification is working in their environment.

Content Explorer

Content Explorer gives you a "bird's eye view" of all the sensitive data in your tenant. It shows you exactly which SITs have been detected and where they are located (SharePoint, OneDrive, Exchange).

Warning: Permissions Matter Accessing Content Explorer is a highly privileged action. By default, even Global Admins cannot see the actual text within the files in Content Explorer. You must be assigned the "Content Explorer Content Viewer" role to see the snippets of data that triggered the classification. This ensures that the people monitoring compliance don't accidentally violate privacy themselves.

Activity Explorer

While Content Explorer shows you what you have, Activity Explorer shows you what is happening. It tracks events such as:

• Label changes (e.g., a user downgrading a label from "Secret" to "Public").
• Label removals.
• Auto-labeling applications.
• Files being shared externally that contain sensitive SITs.

This is essential for auditing and for identifying "risky" behavior. If you see a sudden spike in users removing sensitivity labels, it might indicate a need for more training or a change in policy.

Step-by-Step: Implementing an Auto-Labeling Policy

To see how these capabilities come together, let's look at the process of setting up an auto-labeling policy for files stored in SharePoint and OneDrive.

Step 1: Define the Classification Logic

First, decide what you are looking for. In this scenario, we want to find any document that contains a UK National Insurance Number and a Credit Card Number. We will use the built-in SITs for this.

Step 2: Create or Choose a Sensitivity Label

You need a label to apply. We will use a label called "Highly Confidential - Finance." This label is configured to encrypt the file and apply a "Confidential" watermark across the pages.

Step 3: Configure the Auto-Labeling Policy

In the Purview portal, navigate to Information Protection > Auto-labeling.

1. Name the policy: Give it a clear name like "Auto-label Financial Data."
2. Choose Locations: Select SharePoint sites and OneDrive accounts. You can target specific sites or the entire organization.
3. Define Rules: Create a rule where the condition is "Content contains Sensitive Information Types." Add the "U.K. National Insurance Number" and "Credit Card Number" SITs. Set the "Accuracy" to High and the "Instance Count" to "1 to Any."
4. Choose the Label: Select the "Highly Confidential - Finance" label.

Step 4: Simulation Mode

This is a critical best practice. Before the policy goes "live," Microsoft Purview runs it in Simulation Mode. During this time (usually 24-48 hours), the system scans your data and tells you exactly what would have happened if the policy were active.

Review the simulation results in the portal. If the system flagged 10,000 files and you only expected 100, your SIT might be too broad. You can tweak the rules and re-run the simulation until the results are accurate.

Step 5: Turn the Policy On

Once you are confident in the results, you publish the policy. From this point forward, any new or modified file that matches the criteria will be automatically labeled and protected.

Best Practices for Data Classification

Implementing data classification is a journey, not a one-time project. To be successful, consider these industry-standard best practices:

Start Small and Focused

Don't try to classify everything on day one. Start with one or two high-priority data types, such as PII or financial data. Once you have successfully implemented those and refined your SITs, move on to the next category.

Use "Crawl, Walk, Run"

1. Crawl: Use Content Explorer to see what sensitive data you currently have without applying any labels.
2. Walk: Start using "Recommended" labeling where users are prompted to apply a label. This helps educate the workforce.
3. Run: Implement "Auto-labeling" for your most sensitive data to ensure it is protected regardless of user action.

Minimize the Number of Labels

"Label fatigue" is a real problem. If you give users 20 different labels to choose from, they will often get frustrated and choose the wrong one or ignore the system entirely. Aim for 5 or fewer top-level labels (e.g., Public, General, Confidential, Highly Confidential). Use "Sub-labels" if you need more granularity for specific departments.

Involve Business Stakeholders

IT should not decide what is "Confidential." These decisions must come from the business owners—Legal, HR, Finance, and Product Development. They understand the value of the data and the impact of its loss.

Callout: The "False Positive" Balance In data classification, there is always a trade-off between "Recall" (finding all the sensitive data) and "Precision" (not flagging non-sensitive data).

• If your SIT is too strict, you will miss sensitive files (False Negatives).
• If your SIT is too loose, you will annoy users by locking down non-sensitive files (False Positives). Simulation mode is your best tool for finding the "sweet spot" in this balance.

Common Pitfalls and How to Avoid Them

Even with the best tools, classification projects can fail if not managed correctly. Here are the most common mistakes:

Over-Classification

When organizations are afraid of data leaks, they often default to labeling everything as "Confidential." This renders the classification useless. If everything is special, nothing is special. Over-classification leads to users finding "workarounds" to get their jobs done, such as taking screenshots of documents or moving data to personal accounts.

Neglecting the "Legacy" Data

Many organizations focus only on "new" data being created. However, the biggest risk often lies in the "dark data"—the terabytes of old documents sitting in SharePoint folders from five years ago. Ensure your classification strategy includes "Auto-labeling at rest" to scan and protect your historical data.

Ignoring User Education

Technology can only do so much. If users don't understand why they are being asked to label data, or why they can't send a "Highly Confidential" file to their Gmail, they will perceive the compliance system as a barrier to productivity. Clear communication and short, practical training sessions are essential.

Relying Solely on Regular Expressions

Regex is powerful, but it is "dumb." It doesn't understand context. If you only use Regex SITs without supporting keywords or EDM, you will likely suffer from high false-positive rates. Always try to combine patterns with proximity-based keywords to improve accuracy.

Practical Example: Protecting Research Data

Let's look at a real-world scenario. A pharmaceutical company is working on a new drug called "Vaxipro." They need to ensure that any document mentioning the chemical formula or the project name is protected.

1. The Challenge: The project name "Vaxipro" is a unique word, but the chemical formula looks like a standard string of alphanumeric characters that might appear in other technical documents.
2. The Solution:
   • Create a Custom SIT for the keyword "Vaxipro."
   • Create a Trainable Classifier for "Pharmaceutical Research Papers" by feeding it 100 examples of their lab notes.
   • Create a Sensitivity Label called "Project Vaxipro - Internal Only" that applies encryption and restricts access to only the R&D team.
   • Deploy an Auto-labeling policy that triggers if either the SIT is found OR the Trainable Classifier identifies the document as a research paper.
3. The Result: A scientist accidentally uploads a draft of the clinical trial results to a public-facing SharePoint folder. Within minutes, the auto-labeling policy scans the file, identifies it as a research paper mentioning "Vaxipro," applies the label, and encrypts the file. Even though the file is in a public folder, unauthorized people cannot open it.

Key Takeaways

• Classification is the Foundation: You cannot protect or govern data that you haven't identified. Classification provides the necessary metadata for all other Microsoft Purview features.
• Multiple Detection Methods: Microsoft Purview offers a "defense in depth" approach to classification, including Sensitive Information Types (patterns), Exact Data Match (database lookups), and Trainable Classifiers (machine learning).
• SITs are Logic-Based: Effective SITs use a combination of primary patterns (Regex), supporting evidence (keywords), and proximity rules to ensure high accuracy and low false positives.
• EDM for Precision: When dealing with specific, known data like customer lists or employee records, Exact Data Match is the gold standard for accuracy and privacy.
• ML for Unstructured Content: Trainable Classifiers are essential for identifying document types that don't follow a set pattern, such as legal agreements, resumes, or source code.
• Simulation is Mandatory: Always run auto-labeling policies in simulation mode first. This allows you to refine your logic and avoid disrupting business operations with incorrect classifications.
• Visibility Tools Drive Improvement: Use Content Explorer to understand your data landscape and Activity Explorer to monitor how users interact with sensitive information and labels.
• Balance Security and Productivity: The goal of classification is to enable the business to work securely, not to lock data down so tightly that employees cannot perform their duties. Avoid over-classification and prioritize user experience.