Select For Impact

Previously I have mentioned the shorthand concept of "no downside players" as well as the more general principle of making asymmetric bets (small, bounded downside vs high or uncertain upside). The former is a concrete application of the latter.

I wanted to deep dive a bit on this topic a bit and connect it a bit to a tendency I've identified in hockey decision-makers which I've dubbed "check box management" (or check box player evaluation). This one is simple - it's the habit of evaluating a player based on a list of pre-determined / normative factors.

For example: "big guy, hits a lot, plays on the penalty kill, has a few seasons under his belt." Not an uncommon laundry list for, say, a bottom-pairing defender. Check, check, check, check, you got your guy.

What you'll notice, however, is there's no mention of actual impact on shots, chances, goals, goal differential, etc. Assumed Impacts are imputed via the list of personal factors, but they may or may not represent actual on-ice value. This player skated on the PK and is big = must be good on the PK (maybe! Or maybe the other coach had no other options. Or maybe he also made the assumption that big equals good without reference to results).

As noted in the "stop making bad 4L's" post, this tendency presents more as you move down the roster and especially closer to the margins at the bottom of the rotation.

Known high upside/impact players are obvious and valued more or less appropriately. But! As assessing on-ice impact gets fuzzier and more difficult, the tendency is to check boxes and acquire for inputs rather than outputs.

That is, when it’s difficult to tell the difference between players in terms of the true impact, the tendency is to optimize for perceptually obvious factors. So: bigger, meaner, stronger, harder worker, etc. The input becomes a kind of proxy for impact (or at least a signal of usefulness to the coach).

Another hypothetical example: a small prospect isn't clearly a star offensive contributor at the NHL level, but he is very obviously smaller, lighter, and weaker than the typical grinders and vets that populate the bottom six of most clubs. Ergo he is at a disadvantage physically while not yet obviously bringing an advantage offensively. So - the small guy gets waived and the team builds yet another 4L whose only job is to hit people and try to not get scored on.

Check box management and selecting for inputs isn’t “wrong” so much as it is a parallel heuristic to selecting for impact. Meaning, the results are closer to random than consistently bad. Inputs can certainly have a correlation with impact, but it's not always a direct relationship.

(Sometimes it doesn't pay to pick the big, "toolsy" guy who doesn't produce)

The downside of check box management is it can freeze players as one thing or another and assumes their impact based on relatively fixed, ascribed features. It makes player evaluation more static, less probabilistic, and player development less dynamic/fluid. It also makes it far more likely you'll skip over no downside bets owing to the nature of those players as conceived (better than replacement level, but non-obvious upside).

Summary - If a team reflexively defaults to checkbox management lower in the rotation, it is unlikely the org will consciously experiment with no downside (asymmetric) bets. It's how you end up keeping Jason Wiemer and buying out Martin. St. Louis.