Why I Switched from Regression to Classification (And Why It's Not That Simple)

The standard advice for predicting flight delays sounds obvious: delays are measured in minutes, so use regression. I started there. After log-transforming the target, engineering dozens of features, and tuning several models, I had a respectable R² of around 0.40 on USDOT data.

Then I reframed the problem as binary classification — delayed (≥15 min) or not — and everything got easier and more useful.

Here's what I learned, and why the honest answer is more nuanced than "regression fails."

The Distribution Problem

The first clue that regression was going to be difficult is the shape of the delay distribution. Raw delay minutes have a skewness above 5. Log-transformation brings it down, but doesn't solve the deeper problem: zero-inflation.

Around 77% of flights depart within 15 minutes of schedule, creating a massive spike at zero that no transformation fully removes. The target isn't really a continuous variable — it's a mixture distribution with a large point mass at "on time" and a long right tail for delayed flights.

Regression Isn't Wrong — It's Solving a Harder Problem

Here's where I want to be precise, because I've seen a lot of posts that oversimplify this.

My regression models achieved R² ≈ 0.40 on USDOT data after log-transformation. That's not nothing. There's real signal. But R² = 0.40 means that 60% of the variance in delay magnitude is unexplained at schedule time.

Why? Because the exact number of minutes a flight is delayed depends on things that aren't knowable when you book a ticket:

• Real-time ATC decisions at the departure airport

• Whether a connecting inbound aircraft actually lands on time that day

• A maintenance issue discovered during the pre-departure check

• A weather system that moved faster or slower than forecast

You can predict whether a route tends to run late. You cannot reliably predict exactly how late.

Classification sidesteps this by asking a fundamentally easier question: will this flight cross the 15-minute threshold? The same behavioral signals — historical delay patterns, time of day, carrier — that are noisy predictors of exact minutes are much cleaner predictors of whether the threshold gets crossed.

The Operational Argument

Even if regression and classification performed identically, classification would be the right choice for how airlines actually use these predictions.

Airlines make binary decisions: rebook this passenger's connection? Reallocate the gate crew? Send the delay notification? A model output of "predicted 23.4 minutes late" is harder to act on than "82% probability of delay ≥15 minutes." The probability score is directly actionable — you can set different thresholds for different decisions:

# High recall: catch most delays, accept false alarms
# Use for crew reallocation (missing a real delay is costly)
crew_action_threshold = 0.35

# High precision: only alert when confident
# Use for passenger notifications (alarm fatigue is a real cost)
passenger_notification_threshold = 0.55

df["crew_flag"]   = df["delay_probability"] >= crew_action_threshold
df["notify_flag"] = df["delay_probability"]

The FAA itself defines delay as ≥15 minutes. The industry standard is already binary. Matching that framing isn't just statistically convenient — it's correct.

The Results

An AUC of 0.845 means the model correctly ranks 84.5% of delayed vs. on-time pairs — good enough to prioritize which flights need proactive intervention.

The Takeaway

Don't reach for regression just because your target looks continuous. Ask: what is the decision this model needs to support?

If the answer is binary, frame it as classification from the start. You'll get a simpler model, more interpretable outputs, and a target distribution that's much better behaved.

Regression is still valuable — I keep it in the project to explain why classification is appropriate. The R² numbers are part of the story. But they're not the model I'd ship.