2 weeks ago
Dears
Does anyone have the DDL for _quality_monitoring_summary table?
This is created by the DQ anomaly detector. Since the detector was trying to create a managed table which is not allowed in the environment I work, I am attempting to create this on my own from the documentation https://learn.microsoft.com/en-us/azure/databricks/lakehouse-monitoring/data-quality-monitoring#debu...
But i run into data types mismatches errors.
appreciate if anyone can share this table. Thank you...
2 weeks ago
here is the schema i can see on my workspace. but honestly its not really covering much and fires lot of queries. check it out.
2 weeks ago
Hey, thanks a lot. Can you please share the DDL here...thanks again in advance..
2 weeks ago
@dataoculus_app Can you please share the DDL here...thanks again in advance..
2 weeks ago
Hi,
The _quality_monitoring_summary table is an internal table created by the Data Quality Anomaly Detector in Databricks Lakehouse Monitoring. Unfortunately, the full DDL is not publicly documented in detail, and trying to manually create it can lead to data type mismatches, as you've encountered.
However, you can try the following workaround:
Enable the anomaly detector in a test environment where managed tables are allowed.
Let it create the _quality_monitoring_summary table automatically.
Use the following command to extract the DDL:
SHOW CREATE TABLE <catalog>.<schema>._quality_monitoring_summary;
This will give you the exact DDL with the correct data types.
If setting up a test environment isn't feasible, an alternative is to create an external table and configure the Lakehouse Monitoring to use it by specifying a different target location using Unity Catalog or your desired catalog pathโensuring the system has permissions to write to that location.
Passionate about hosting events and connecting people? Help us grow a vibrant local communityโsign up today to get started!
Sign Up Now