Summary

Our journey has begun within this chapter. Leveraging the knowledge gained within previous chapters, we have started facing a challenge that suddenly appeared: discover the origin of a heavy loss our company is suffering.

We received some dirty data to be cleaned, and this was the occasion to learn about data cleaning and tidy data. This was the first set of activities to make our data fit the analyses' needs, and the second a conceptual framework that can be employed to define which structure our data should have to fit those needs. We also learned how to evaluate the respect of the three main rules of tidy data (every row has a record, every column has an attribute, and every table is an entity).

We also learned about data quality and data validation, discovering which metrics define the level of quality of our data and a set of checks that can be employed to assess this quality and spot any needed improvements.

We applied all these concepts to our data, making our data through the...