Feature Engineering Is the Job. Everything Else Is Execution.

How 60 engineered features explained an 11-point performance gap — and why data science is mostly about asking the right questions before writing any code.

The top 10 features by LightGBM split gain include nine rolling or lag features. One carrier-level volume metric. Zero weather features. Zero raw schedule features. Zero aircraft type features.

If you built the model from raw features without engineering, you'd miss everything that actually matters.

The 60-feature build

Rolling features were constructed by aggregating delay history across six time windows (1h, 3h, 6h, 12h, 24h, 7d) by five entity types: origin airport, destination airport, carrier, route, and individual aircraft tail number. That produces 60 features from a single conceptual idea — what's happening in the network right now?

This isn't technically complex. It's a series of grouped time-windowed aggregations. The complexity is conceptual: recognizing that the most predictive signal in flight delays isn't in the flight's own attributes, but in the operational state of the system around it. A flight at O'Hare at 6pm is almost certainly going to be late if O'Hare has been running 45% delayed for the past three hours. That signal is in the network, not in the flight record itself.

Aircraft history features — the previous flight's delay, the average delay across the last five flights for that tail number — capture a different but related dynamic: individual aircraft accumulating delay debt through the day. prev_flight_delay_1 is the single highest-importance feature in the model. It's also one of the simplest.

Why feature engineering comes before model selection

The instinct in most ML workflows is to get data, run a baseline, then tune. Feature engineering often gets treated as a step between baseline and tuning — something you do when the baseline disappoints you.

This project inverted that. The feature engineering decision was made first, informed by domain knowledge about how flight delays actually propagate, before a single model was trained. The result was a 73-feature dataset where the most important features were engineered, not raw.

The Seymour dataset had 27 weather features — genuinely valuable domain information — and still couldn't compete. Weather affects whether delays happen. Rolling operational history tells you whether delays are happening right now. For near-term prediction, the second question matters more.

The principle that transfers

In any prediction problem involving interconnected systems — marketing attribution, demand forecasting, customer engagement — the most predictive features are usually the ones that require you to ask what's the current state of the system, not just what are the attributes of this record. That question requires domain knowledge to answer. It can't be automated. It's the part of data science that looks like thinking before coding.