Wednesday, January 29, 2014

Is Knowing Better For Everyone, Part 2: The EyeBox

There's a report on the Newsweek site today headlined, "Exclusive: Why The Next Super Bowl Could Be Much Safer." It discusses a new technology patented by a company called Oculogica that uses a sophisticated algorithm to track abnormalities in eye movements that could indicate concussions or trauma resulting from severe head contact.

As writer Kevin Maney states in his story, "There's some excitement building around Oculogica because of a belief that its capability, called EyeBoxCNS, can be built into small, inexpensive devices that every team could have in the locker room. Heck, there's a chance this could end up as a smart phone app, allowing a suburban soccer coach to conduct an accurate sideline concussion scan anytime a couple of players knock heads."

Without bursting anyone's bubble, let me point out again what I noted in my last post, namely that knowing when an athlete suffers a concussion makes athletes only marginally safer, if it makes them safer at all. 

Outside of the Super Bowl, most coaches are already erring on the side of caution, avoiding the side of lawsuits, and holding out any player who shows the slightest sign of a concussion. Certainly there is value in having an accurate diagnosis tool that can make play/no-play decisions more objectively and largely remove coaches from this process. They'll feel better about it, and so will most athletes, parents, and insurance companies. Also, the EyeBox is valuable even if it only keeps one concussed player from re-entering a contest and suffering a second impact. But there is danger in seeing this technology as an advancement in athlete safety.

Think of it this way: If you're climbing a rock wall with a frayed harness and the harness gives way, are you safer because you knew that the harness gave way because it was frayed? There might be something you could do with this knowledge -- grab for the rope, perhaps, because the rope wasn't what failed -- but the only thing that would have made you truly safer would have been a harness that wasn't frayed.

The things that will make athletes safer are changes in rules, changes in culture, changes in coaching, and improvements in equipment. Everything else is after the fact.

One more note on the EyeBox: Like the Checklight, it offers the opportunity to gather data on the number and severity of athletically induced head injuries. This is incredibly valuable data. It's also a huge powderkeg that could blow up in the faces of many large, well-established institutions that have largely been excused from large-scale remediation because of a lack of macro-level data. Widespread adoption of these tools will deliver data and take away that defense.

It's no surprise then that Rich Ellenbogen, a neurosurgeon and co-chairman of the NFL's concussion committee, said that the EyeBox is, in Maney's words, "still more science project than proven technology." Knowing carries with it an obligation to do something with the knowledge. The longer a body can plead ignorance, the more time it buys.

The EyeBox and Checklight are significant steps forward. Now let's see who walks in their footsteps.

Thursday, January 16, 2014

Reebok's Checklight: Is Knowing Better For Everyone?

Part of my day job involves keeping up with the latest technology, and this past week that meant following developments at the massive Consumer Electronics Show in Las Vegas.

Out of all the gadgets on display at CES, from an encapsulated three-wheel-vehicle thing from Toyota to a toothbrush that downloads data to your smartphone, one particularly caught my eye. Reebok’s Checklight Impact Sensor, a skullcap that monitors hits to the head, was first demonstrated at last year’s CES. Now it’s back in the form of a production model with an under-$150 street price that was demonstrated memorably by a human punch-model.

Checklight photo courtesy BusinessWire.
After reading the specs and details on the sensor and thinking about its applications for hockey, I guarantee you two things: The Checklight or something like it will be required or semi-required by virtually every hockey organization in the country within two years, and it will be misunderstood or misused by virtually all of them.

I can start proving that assertion right now. One of the stories on the sensor carries the headline, “Reebok CHECKLIGHT Impact Sensor Cap Helps Keep Athletes Safe.” No; it doesn’t really keep athletes safe. The Checklight detects serious head impacts and lights when the severity of impacts surpass a fixed level. It doesn’t detect or diagnose concussions. It’s simply a monitor of blows to the head, and wearing a monitor of blows to the head won’t lessen the frequency or severity of those blows.

Think of it another way. This is not a sensor built into a pair of jeans that detects shoplifting, or a camera that automatically photographs (and tickets) people who run red lights. This is a check-engine light that comes on when something in its environment sets it off, and knowing that your check-engine light might come on does not change the way you drive.

The second and broader way the Checklight likely will be misused comes under the general heading of “do we really want to know?” Who truly benefits from the impartial, impersonal, machine-generated knowledge that someone has sustained a serious blow to the head, and who might be threatened by that knowledge?

Obviously, the athlete benefits. Not every athlete in the heat of competition is sufficiently attuned to their physical state to realize they have sustained severe head contact. Some are simply too young to grasp the situation. Having this information available to these athletes, their coaches and parents seems immediately and obviously to be a good thing … provided someone takes appropriate action.

Appropriate action: Here’s where things get tricky. What is appropriate action? Does calling a penalty constitute appropriate action? A game misconduct? Suspensions? Or is it something more nuanced?

To answer that, consider the technology behind the Checklight. The Checklight is calibrated to trigger at the same point for everyone who wears it. However, people aren’t Rock-‘Em Sock-‘Em Robots; the concussion threshold is different for every person. This means there are four possibilities when someone wearing a Checklight is hit in the head: the contact was insufficient to light the light or cause a concussion; the contact was sufficient to light the light and cause a concussion; the contact was sufficient to light the light but not cause a concussion; or the contact was not sufficient to light the light but was sufficient to cause a concussion.

The last two are very possible and extremely troublesome, if the Checklight enjoys widespread acceptance and is used as a determining factor in removing a player from a game. If the Checklight indicates severe head contact but the contact did not cause a concussion, and there are organizational mandates that a player be removed from a game when his Checklight lights and it’s a big game … you can see where this is going. Throw in a my-son’s-going-pro parent and you’ve got the recipe for a brouhaha that could easily wind up in a courtroom.

You can certainly argue that mandated removal is erring on the side of caution, but it’s extremely imperfect erring-on-the-side-of-caution. No coach this side of the NFL would play a concussed player knowingly in any game, but seeing as the Checklight isn’t a concussion-diagnosis tool, how many coaches would hold out a player from a crucial game solely on the basis of a lit-up Checklight and no other symptoms – unless they have to?

Coaches and officials are in a pickle with Checklight.  Consider the scenario where the light doesn’t light but the player suffers a concussion. If a coach and officials let a concussed player with an unlit Checklight keep playing, and s/he sustains another head impact and suffers severe injuries from Second Impact Syndrome, who is liable? The coach, who made the decision in ignorance of evidence to the contrary? The officials, who didn’t remove the lit player from the game and didn’t take steps to keep further head contact from occurring? The sanctioning body, which made (or didn’t make) a rule that allowed concussed players to keep playing? Reebok, for making a unit that can’t distinguish between significant but not injury-causing head contact and contact sufficient to cause concussion?

My wife is a former actuary and hockey mom who’s unusually attuned to liability issues. She says she wouldn’t touch the Checklight – not because it doesn’t work but because of liability concerns like these. At some point someone is going to be held liable, and it’s going to be whoever’s caught holding the hot potato.

I come at this from a different angle. At times with my day job I’m a data person. In my line of work, data is isn’t any good if you don’t collect it, and once you collect it, you need to do something with it. I would think and hope that data from the Checklight would not stop with the Checklight. If sharks can send tweets when they’re approaching shore, Checklights should be able to send notification of head injuries to multiple sanctioning bodies – the NCAA, USA Hockey, the local organization, the state organization, academic researchers, medical authorities, whomever – via USB now and wirelessly in the future. There’s potential with the Checklight to collect and aggregate a staggering amount of information on head contact. If every hockey player from the squirt level up wore a Checklight, and if the data from those Checklights was sent to USA Hockey, then USA Hockey would know once and for all the scope of severe head contact in youth hockey.

But maybe, just maybe, USA Hockey (or Pop Warner Football or the Wisconsin Interscholastic Athletic Association or whomever) doesn’t want to know the scope of severe head contact in their sport. Knowing carries with it the obligation to do something about it, and if a sanctioning body is unwilling or unable to do something about it, it might be better to plead ignorance.

If, for instance, Checklight data shows USA Hockey that there were significantly more instances of severe head contact than the organization had previously believed or was willing to admit, what could the organization do with that data? Use it as rationale for banning all head contact? Stiffen penalties for offenders? Based on past experience with similar rule changes, whatever safety steps USA Hockey would propose would meet with fierce opposition, Checklight data or no. And if the organization accedes to rank-and-file pressure and balks at making changes in the face of empirical data that demands changes, then what?

(Just as a case study in the value -- or cost --  of knowing, once the National Football League realized the scope of severe head injuries suffered by players, it settled with a group of former players for whatever it could. In this case it turned out to be between $765 million and $914 million -- and a judge just threw out the settlement, saying the players weren't getting paid enough.)

I know what I’d do, if I were a sanctioning organization. First, this issue is too important not to collect data when the opportunity to collect data is so immediate and obvious. I’d buy Checklights for a sample population and collect the data on head contact. If the data suggested a significant problem and/or additional study, I’d spend the money, escalate the study, and analyze the data from that study.

If the data from that escalated study showed a greater level of head contact than previously thought and a significant correlation between lit Checklights and concussions, I’d mandate Checklight purchase, remove players with lit Checklights from games, and severely penalize opposing teams – but more importantly, I’d gather data on teams and players that delivered the triggering hits and require that data to be reported up along with the data on the hit itself. The only way to lessen the occurrence of head injuries is to remove the culture of headhunting from the sport (any sport, from lacrosse to hockey), and the first step to doing that is to identify the headhunters.

In short, the Checklight is a highly imperfect step forward – but a step forward nonetheless. Kudos to Reebok; the company has met a definite need, and deserves to profit from doing so. In the next several weeks, we hope to have an interview with Reebok and officials from sanctioning bodies discussing the Checklight and its implications. It could be that some of these concerns are groundless. (I hope so.) It could be that we’re thinking ahead of the people whose job it is to think ahead. (I hope not.) One way or another, we’re going to find out.