Sue Bayes
January 18, 2022
One of my dataflows failed this morning and gave an incorrect data type as the error. On first glance at the dataflow, all of the columns seemed to be ok and there were no error messages showing in the query pane. However, when you first load the dataset the Column profile only shows the top 1,000 rows. To have the ability to troubleshoot effectively you need to look a the entire dataset.
I would also suggest that you turn on, if it isn’t already, the column distribution view.
Each column is summarised by type of data and gives you an indication of the profile of the column. Are there any errors? How many distinct values are there, of these distinct values, how many of these are unique? Power BI uses the vertipaq engine to compress data at a column based level and understanding the makeup of each column gives you critical information to help model your data. My error, was an age that had been entered within the software as text. Yes, you shouldn’t be able to enter a text entry into a field called Age but unfortunately, that I can’t change. What I can do is replace the errors according to the business decision.
I found the videos below helpful in understanding how the Vertipaq engine compresses data.
Any questions, happy to help,
Sue
January 18, 2022
