Introduction
In the NFL, success on offense involves more than just raw talent. Defenses are constantly looking for cues—“pre-snap tells”—that tip them off to whether a play is likely to be a run or pass. In my recent work in the 2025 NFL Big Data Bowl (which uses player tracking data from 2022) l dig into how predictable these offensive cues are at both the player and team level, and how that predictability relates to key performance indicators like Pro Football Focus (PFF) grades and overall offensive efficiency.
In this blog post, I’ll share the high-level concepts, interesting findings, and tactical insights for coaches, analysts, and executives looking to leverage data in strategic decision-making. If you’d like deeper technical details or to see the code used, please check the links at the end.
There are much deeper-divers to be had here, but this is a start on the aspects I found most interesting. To see more of the code behind this work, check out my Kaggle Submission here.
The Core Question
Is there a measurable relationship between an offense’s pre-snap predictability and its on-field success?
From an analytics standpoint, it boils down to:
- Detecting Predictability: Can we accurately classify a play as a run or pass strictly by looking at player locations, orientations, formations, and contextual data (e.g., down/distance, prior tendencies, etc.)?
- Assessing Impact on Performance: Once we understand who and which teams are more predictable, does that correlate with negative (or positive) outcomes in efficiency metrics like offensive line run-blocking (RBLK), quarterback grades, or skill-position performance?
Quick Overview of the Approach
- Data Engineering
- Pulled in the NFL’s Next Gen Stats tracking data for player x-y positions, speeds, orientations, and contextual fields such as down and distance.
- Engineered features to capture pre-snap stance, alignment relative to the quarterback, and possible interplay between formation and the situation.
- Modeling Predictability
- Trained a classification model (run vs. pass) using these engineered features, achieving solid performance (AUC of 0.77+).
- Teams or players whose data made the model’s classification “too easy” were deemed more “predictable.”
- Performance Metrics
- Merged PFF offensive grades (e.g., RBLK for run blocking, OFF for overall offense) to examine how changes in predictability connect to efficiency.
- Performed correlation analyses, difference-in-differences (for players who switched teams), and positional breakdowns to see who is most (and least) impacted by predictability.
Key Findings & Visual Insights
1. Model Performance & Predictive Features
- Formation Matters Most: Features like offenseFormation consistently ranked at the top of the model’s “importance.” This underscores that certain formations telegraph run vs. pass more than we might assume.
- Contextual Clues: Situation-based metrics (like down-and-distance) plus orientation measures often tipped the scale on whether a play would be a run or pass. In other words, defenders can exploit small details such as a guard’s foot angle or how a tight end lines up.
2. Team-Level Predictability vs. Run-Blocking (RBLK)
- Negative Correlation (~ -0.49): Teams with higher “predictive accuracy” in the model tended to have lower run-blocking effectiveness, implying that when it’s easier to guess the run, defenses can gear up and neutralize it. I also purposely left the multiple model metrics in to show the variation of correlation with football performance. There is a deeper and more philosophical dive coming on what this means.
- Eagles & Bears vs. Buccaneers: Teams like the Eagles and Bears performed better at run-blocking but appeared less predictable in our model. Meanwhile, teams such as the Buccaneers graded lower in RBLK while being more predictable.
3. Positional Breakdown
- Quarterbacks & Guards: Higher predictability correlated with worse PFF grades, indicating these positions may be more vulnerable if the defense “knows” what’s coming. Guards, in particular, showed a strong negative correlation between their performance and their team’s predictability (potentially because defenders can key on subtle pulling or angle shifts).
- Skill Positions (RB, FB, WR, TE): Mixed effects. Running backs sometimes benefited from a certain level of predictability (they thrive under “clean” assignments), but wide receivers typically had minimal correlation with predictability—perhaps route running success is multifaceted, and single cues don’t always telegraph a pass attempt.
4. Switchers & Diff-in-Diff
To further isolate whether predictability truly impacts a player’s performance (versus random variation or differing team strengths), I looked at players who switched teams between seasons (a “natural experiment”):
- Change in Team AUC (ΔAUC) vs. Change in Player Grade (Δgrades_offense): Observed a moderate negative correlation (r ~ -0.30). In short, moving to a more predictable team (higher ΔAUC) often led to a decline in that player’s offensive grade, and moving to a less predictable team aligned with improvement.
- Implication: This is more evidence that predictability doesn’t just reflect team style; it actively impacts an individual’s potential success.
Why It Matters
- Coaching & Game Planning
- Beyond Talent: Even highly skilled rosters can be hamstrung by telegraphed play-calling. Implementing formation diversity or subtle “false keys” can keep the defense on its heels.
- Situational Adaptation: Adjusting guard alignments or quarterback cues could mitigate predictable patterns.
- Analytics & Roster Building
- Right Fit: Offensive linemen and QBs sensitive to predictable schemes may flourish if inserted into more varied systems.
- Position-Specific Investment: If your scheme is pass-happy, a highly predictable formation can hamper run-blocking. Balancing your roster to hide or offset these tendencies can pay dividends.
- Future of Sports Analytics
- Machine Learning in Football: Continues to uncover insights where the difference between success and failure can be inches or split-seconds.
- Player Valuation: NFL front offices increasingly weigh data-driven evaluations (like how a guard performs in a predictable vs. unpredictable system).
Guard-Specific Spotlight
One of the most striking details emerged from analyzing guards (G):
- Least Predictable, Higher Grades: Names like Quinn Meinerz and Teven Jenkins came up as “least predictable” in pre-snap orientation. They also posted stronger PFF grades.
- Most Predictable, Lower Grades: Guards who consistently gave away run/pass cues to defenders tended to see lower performance metrics—indicating the line is a critical battleground for capitalizing on unpredictability.
Conclusion & Next Steps
This project confirmed that keeping the defense guessing isn’t just a cliché: it’s backed by quantifiable evidence at both the team and individual levels. From a purely statistical viewpoint:
- Offenses that telegraph run vs. pass risk defenders preemptively blowing up plays—most notably in run-blocking scenarios.
- Players switching from less predictable to more predictable teams saw notable drops in performance, reinforcing that scheme matters as much as skill.
Looking Ahead
- Further Causal Analysis: Incorporating more advanced observational methods (or season-over-season difference-in-differences) can strengthen claims about causation.
- Micro-Level Cues: Getting deeper into alignment orientation, footwork patterns, and the timing of subtle movement for specific positions might uncover new edges.
Let’s Connect
I hope you found these insights helpful! If you’re interested in discussing sports analytics, data engineering in football, or applying machine learning to real-time player data, feel free to reach out:
- LinkedIn: James Young
- My Blog: Foretodata.com
- Further Code: Kaggle Notebook
Thank you for reading, and I look forward to collaborating or hearing your feedback.