Microsoft Purview offers a comprehensive solution for data governance and compliance by classifying sensitive information. This classification system is critical for ensuring that organizations can effectively manage their data, secure sensitive information, and comply with regulatory requirements. In this blog, we explore the system classifications available in Microsoft Purview, provide examples, and delve into how you can extend these capabilities with custom classifications.
Types of Classifications
The Microsoft Purview governance portal supports both system and custom classifications:
System Classifications
- Overview: Microsoft Purview provides over 200 system classifications out of the box. These classifications are designed to identify common patterns and sensitive data formats.
- Icon Representation: System classifications are represented with a thunderbolt icon alongside the classification name. Hovering over a classification provides more details about its type and how it was applied.
- Example: Person’s Name is a system classification.
Custom Classifications
- Overview: Custom classifications are user-defined rules that classify assets based on specific patterns or column names not covered by system classifications.
- Creation Options:
- Based on regular expressions.
- Using a dictionary of keywords or patterns.
- Example: A custom classification for Employee IDs following the
EMPLOYEE{GUID}pattern can be defined with a regular expression:^EMPLOYEE[A-Za-z0-9]{8}-[A-Za-z0-9]{4}-[A-Za-z0-9]{4}-[A-Za-z0-9]{4}-[A-Za-z0-9]{12}$.
System Classifications in Microsoft Purview
Microsoft Purview employs three primary classification methods:
1. Bloom Filter Based Classifications
Bloom filter classifications use keywords and patterns to identify specific types of data. These are effective for classifying structured data like city names or country names.
Examples:
- World Cities:
- Identified by full names or short codes.
- Keywords: “city”, “metropolis”, “municipality”, etc.
- Country Names:
- Identified by names or abbreviations.
- Keywords: “country”, “nation”, “nationality”.
2. Machine Learning Based Classifications
Machine learning models in Microsoft Purview analyze structured data to classify various types of information. These classifiers are pre-trained using global datasets.
Examples:
- Person’s Name:
- Identifies full names in a single column or first and last names in separate columns.
- Person’s Address:
- Detects full addresses that include house numbers, streets, cities, and postal codes.
- Supported Formats: “123 Main St, City, 12345”, “Landmark, City, Zip Code”.
- Person’s Age:
- Detects age values in formats like “30 years”, “30 yrs”, “30 y, 2 m”.
3. Regular Expression (RegEx) Based Classifications
RegEx classifications use defined patterns to identify specific data formats.
Examples:
- ABA Routing Number:
- Pattern: Nine digits starting with specific ranges.
- Australian Business Number (ABN):
- Pattern: 11 digits with optional delimiters.
Minimum Match Threshold
System classifications have a minimum match threshold of 60%. This means that at least 60% of the data in a column must match the pattern for the classification to be applied. This threshold is fixed for system classifications but can be customized for user-defined ones.
Limitations of System Classifications
While Microsoft Purview supports both structured and unstructured data, certain classifications are only applicable to structured formats like CSV, SQL Tables, or JSON. Examples of unsupported classifications for unstructured data include:
- City Names
- Date of Birth
- Email Addresses
Creating Custom Classifications in Microsoft Purview
Custom classifications extend the functionality of Microsoft Purview by allowing you to define your own patterns, formats, or keywords.
Steps to Create Custom Classifications:
- Identify Data Patterns: Determine the unique patterns or keywords in your data.
- Use the Microsoft Purview Portal:
- Navigate to the “Classifications” section.
- Select “Create Classification”.
- Define the Rules:
- Use RegEx or specific keywords to match your data.
- Set a custom match threshold if needed.
- Test and Apply: Run test scans to validate your classification rules before applying them to production data.
Example 1: Custom Classification for Employee IDs
Scenario:
Your organization uses a unique Employee ID format: “EMP-XXXXXX” (where X is a digit).
Steps:
- Define a RegEx:
EMP-\d{6}. - Set the classification name: “Employee ID”.
- Add keywords: “employee”, “ID”, “staff”.
Result:
Employee IDs like “EMP-123456” are automatically classified when scanned.
Example 2: Custom Classification for Invoice Numbers
Scenario:
Your invoices follow the format “INV-YYYYMMDD-XXXX” (date and a four-digit code).
Steps:
- Define a RegEx:
INV-\d{8}-\d{4}. - Set the classification name: “Invoice Number”.
- Add keywords: “invoice”, “billing”.
Result:
Data like “INV-20231218-0012” is identified and tagged.
Best Practices for Custom Classifications
- Be Specific: Use precise patterns and keywords to minimize false positives.
- Test Extensively: Run scans on test data to ensure accurate results.
- Update Regularly: Periodically review and refine classifications as data formats evolve.
Microsoft Purview provides robust out-of-the-box classification capabilities that cater to various data types and formats. By combining system classifications with custom rules, organizations can tailor data governance to their specific needs. Whether you’re protecting sensitive information like credit card numbers or managing internal identifiers like Employee IDs, Purview’s flexibility ensures comprehensive data protection.
Start leveraging these features today to enhance your data governance strategy and meet compliance requirements effectively!
How Anyon Consulting Can Help
Managing a unified data governance strategy can be complex, especially as organizations scale across hybrid cloud environments. This is where Anyon Consulting can assist. Our team of experts is well-versed in implementing Azure Purview and other cloud-based data governance solutions. We can help audit your existing data governance framework, identify areas for improvement, and implement best practices for Azure Purview to ensure seamless operations across your data estate. From streamlining data cataloging and lineage to improving security and compliance, we provide comprehensive solutions that future-proof your organization’s data management strategy. Don’t let data governance challenges hinder your progress—reach out to Anyon Consulting today for a personalized consultation and elevate your data strategy.
