Comparing model performance to a baseline score

While it is great that we have a high accuracy score from our model of 91.7 percent, it is also important to compare this to a baseline score. We dig deeper into this concept in this section.

How to do it...

This section walks through the steps to calculate the baseline accuracy.

1. Execute the following script to retrieve the mean value from the describe() method:

predictionDF.describe('label').show()

2. Subtract 1- mean value score to calculate baseline accuracy.

How it works...

This section explains the concept behind the baseline accuracy and how we can use it to understand the effectiveness of our model.

1. What if every chat conversation was flagged for do_not_escalate or vice versa. Would we have a baseline accuracy higher than 91.7 percent? The easiest way to figure this out is to run the describe() method on the label column from predictionDF using the following script: predictionDF.describe('label').show()
2. The output of the script can be seen in the...