Data type is "unsupported type" of Date and Time after metadata ingestion from Databricks to Purview

Hey Anuj, it looks like you’ve hit a current gap in the Purview Data Quality engine: although Unity Catalog lets you see DATE and TIMESTAMP columns in the asset schema, the Data Quality profiler today only supports a limited set of data types and marks those time-based types as “Unsupported.” That prevents both profiling/rule definition and using them as watermarks for incremental scans.

Here’s how you can work around it today:

1. Create a Unity Catalog view (or table) that converts your DATE/TIMESTAMP columns into a supported type • For example, in Databricks SQL:

   
        CREATE OR REPLACE VIEW catalog.my_schema.my_table_for_purview AS
   
        SELECT
   
          colA,
   
          CAST(my_date_col AS STRING)    AS my_date_col_str,
   
          CAST(my_timestamp_col AS BIGINT) AS my_ts_epoch   -- or AS STRING
   
        FROM catalog.my_schema.my_original_table;
   
   

   • By casting to STRING or a numeric type, Purview DQ will recognize the columns as “supported.”
2. Point your Purview scan at the new view (or register it as a separate asset) • Run a full metadata & data quality scan against that view. • You’ll now see my_date_col_str or my_ts_epoch available for profiling and rule-based checks.
3. Configure incremental scans using the new “watermark” column • In your scan settings, select my_ts_epoch (or the string representation) as the watermark for incremental filters.
4. Keep an eye on the Purview roadmap • Full native support for Unity Catalog DATE/TIMESTAMP in Data Quality is on the backlog. You can upvote or file a feature request via Azure feedback to accelerate it.

Hope that helps unblock you in the meantime!

Reference docs

• Supported data sources & file types in Purview scans

https://docs.microsoft.com/azure/purview/purview-connector-overview

• Configure file-type limitations & complex types

https://docs.microsoft.com/azure/purview/sources-and-scans#file-types-supported-for-scanning