Was having this conversation recently about what a z-score is.
What is a z-score? Why is it mentioned so often in statistics? You’ve probably heard of it in the context of the Student t-test.
If you are a machine learning guy more than a statistics guy, like me, you’ve probably heard you should “standardize” or “normalize” your variables before putting them into a machine learning model.
For example, if you’re doing linear regression and \( X_1 \) varies from \( 0..1 \) and \( X_2 \) varies from \( 0..1000 \), the weights \( \beta_1 \) and \( \beta_2 \) will give an inaccurate picture as to their importance.
Well, the z-score actually is the standardized variable.
I would avoid this terminology if possible, because in machine learning we usually think of a “score” as the output of a model, i.e. I’m getting the “score” of a set of links because I want to know in what order to rank them.
$$ z = \frac{x – \mu}{\sigma} $$
So instead of thinking of “z-scores”, think of “I need to standardize my variables before using them in my machine learning model so that the effect of any one variable is on the same scale as the others”.
Do you find statistics terminology confusing? “Z score”? “F statistic”? Share your thoughts in the comments!