Logistic Regression & Classification

Despite its name, logistic regression is a classification algorithm. It predicts the probability that an input belongs to a class.

The Sigmoid Function

Logistic regression applies the sigmoid (logistic) function to convert a linear combination into a probability:

σ(z) = 1 / (1 + e^(-z))

σ(z) = 1 / (1 + e^(-z))

P(y=1 | x) = σ(w·x + b)

Predict class 1 if P(y=1) ≥ 0.5
Predict class 0 if P(y=1) < 0.5

P(y=1 | x) = σ(w·x + b)

Predict class 1 if P(y=1) ≥ 0.5
Predict class 0 if P(y=1) < 0.5

L = -[y·log(ŷ) + (1-y)·log(1-ŷ)]

L = -[y·log(ŷ) + (1-y)·log(1-ŷ)]

This penalises confident wrong predictions heavily.

Metric	Formula	When to use
Accuracy	(TP+TN)/(TP+TN+FP+FN)	Balanced classes
Precision	TP/(TP+FP)	Cost of false positives is high
Recall	TP/(TP+FN)	Cost of false negatives is high
F1 Score	2×P×R/(P+R)	Imbalanced classes
AUC-ROC	Area under ROC curve	Overall model quality

              Predicted
              Pos    Neg
Actual  Pos [ TP  |  FN ]
        Neg [ FP  |  TN ]

              Predicted
              Pos    Neg
Actual  Pos [ TP  |  FN ]
        Neg [ FP  |  TN ]

For more than 2 classes: