Data Cleaning IS Analysis, Not Grunt Work - Randy Au, Counting Things

This resource first appeared in issue #42 on 19 Sep 2020 and has tags Managing A Team: Data Teams

Au’s article can be summed up in one pull quote:

The act of cleaning data is the act of preferentially transforming data so that your chosen analysis algorithm produces interpretable results. That is also the act of data analysis.

Again, professionalism is doing things deliberately. I think we tend to get sloppy about things that are “just” data cleaning or “just” having decent uptime or “just” putting together a script - but these things matter.

Data cleaning in particular requires pretty deep expertise in both the data generation process that lead to the data, and the data analysis steps that will come after it. It may not be glamorous or exciting, it may not use sexy recent algorithms or cool frameworks, but it very much is, in and of itself, data analysis.