In-Depth With The Checklight: 'It's Useful, And It Just Can Help'

As we reported two posts ago, one of the hits of this winters Consumer Electronics Show was Reebok’s Checklight, a skullcap with sensors that can detect head impacts, and lights that light to indicate the severity of the impact. The Checklight is an important step forward that deserves more than a single (and occasionally misleading) blog post. In order to get the full story on the Checklight and its development, we interviewed Paul Litchfield, the vice president of Reebok Advanced Concepts, and Bob Rich, the research director at Reebok Advanced Concepts.

Some people would have you believe that one of the worst things about the internet is that it’s killed long-form reporting. Quite the opposite is true: One of the beauties of blogs and the internet is that enables long-form reporting. Paper is expensive; space on a blog isn’t.

Litchfield and Rich’s comments, only slightly edited for clarity, are reproduced below. They’re the first in a series of interviews examining the issue of head impacts and young athletes from as many angles as I can find intelligent people to discuss them.

My bottom line on the Checklight remains unchanged: It’s a valuable product, well worth its purchase price. But as Rich and Litchfield note, it’s also a product that requires education, for athletes and coaches.

So give me the 30-second definition of what the Checklight is.

PAUL: The Checklight is a sport-and-activity head-impact indicator, intended to inform the user or people around the user of the magnitude of impact that just occurred. It is designed to be used in traditional sports, but it is also designed for any activity where the head may be moving quickly and stops abruptly.

What precipitated development of the Checklight?

BOB: The whole process started four years ago, when some members of our team attended a concussion conference. At the conference we learned a lot about concussions, how they affect people’s health and limit their ability to participate in sports and activities, and we learned that a lot of these impacts go unnoticed and undiagnosed. We came back from that conference thinking that identifying these types of impacts would be a good first step toward helping athletes.

PAUL: When Bob and Paul Davis came back from the conference, we didn’t immediately set out to make an electronic assessment product. Actually, we started out by trying to define where we wanted to go, and then we set pathways for that. We did begin to look at a lot of products we could do, from physically assessing impact to actually diminishing impact. We tried a number of various kinds of impact-mitigation processes, thinking that we would try to identify the force of the blow. We examined physical deformation, even something like paint that would shatter and change color at certain forces. We realized it was a big issue to address, but we didn’t immediately start to work on an electronic solution.

However, when we looked electronically we saw there were various ways of assessing impact. Quite frankly, the way your car’s airbag works is very similar to the way the Checklight works. It very quickly gets an indication that an impact has occurred, and the airbag inflates as a response. We also looked at other impact-assessment tools. Riddell has a very complicated system that fits inside football helmets, and there’s a company called BAE that does a lot of work in blast assessment in some of the battle theaters out in the world. So as we found, there certainly are a lot of ways of assessing impact.

How long was the development process?

BOB: We’ve been working on it about four years. We started Phase One development four full years ago, and we launched the product on July, 3, 2013, so that’s a full three and a half years spent developing the electronic head-impact-detection system that evolved into the Checklight.

PAUL: I think it’s interesting that we’ve had lots of projects that have lasted as long as the Checklight, but none have been as consistently intense. For Bob Rich and Paul Davis and the rest of the Checklight team, it’s been a yeoman’s task to get this done. The Checklight project has been running for four years, and there hasn’t been two days running over those four years where Checklight has not been a major portion of their workday.

We realized very early on that this is a very important product, and as we developed this product, we got a sense of all the people out there who the Checklight could potentially help. It’s great to make people gear to work out in, but Reebok is committed to being the home of fitness, and fitness is not just about how many pushups you can do or how many situps you can do. Fitness is part of a holistic approach to being fit and healthy, and where the Checklight fits in is that it allows you to keep on participating in whatever activity you want to. We really wanted to get it right. We wanted to create a product that could keep key people informed, and diminish some of the guesswork surrounding what happens when you hit your head.

How was it tested?

BOB: We tested it through mechanical, biomechanical and sensory-evaluation methods. We have a drop tester in the lab, and we did 15,000 drops using a Hybrid III head form used in crash-test-dummy tests. In addition, we used a variety of helmets and nine different head placements, and tested it on multiple surfaces. We even had Paul Litchfield set up with a headgear and put him in a boxing ring to see what impacts trigger what lights – and there were plenty of people lined up to hit Paul Litchfield in the head! (laughs) But we also did wear testing, seven different rounds in the field with different sports, and athletes ranging from youth athletes to adults. What we wanted was a device that was sensitive enough to pick up significant impacts, but not so sensitive that there would be a line of athletes lining up with the trainer wanting to get back in.

PAUL: Calibration in the lab is one thing. Fifteen thousand hits result in some pretty strong correlations. But when you get into the field, people don’t operate as cleanly as they do in the lab environment. As he said, we did seven rounds of testing, and after each test the evaluation protocol could take a couple of weeks or it could take a couple of months, because some of the tests were more supervised and some were more open-platform, where people just go out and use it and behave more or less typically as they go through their activities. We were happy that Bob and his team were able to correlate how people use it in everyday life with how the Checklight behaves in the lab.

BOB: Once we were we out in the field we learned some things. Certain sports and fitness activities simply have less head contact than others. Field-hockey, soccer, and basketball players pale in comparison to football, hockey and lacrosse players. It’s funny, but we learned over the last four years that many of the innocent hits, hits that don’t look to be as severe, often result in impairment, while the really big hits, the ones you see on Sportscenter, don’t result in impairment. You can’t look at the hit and determine the magnitude of the hit.

Does it work equally well for all ages?

BOB: We had talked about that early in the development cycle, that there would be differences in ages and differences in different sports. What we discovered by looking very closely at an awful lot of research was that football players on the collegiate level and youth football players exhibit the same amount of head acceleration and impact. The younger players show sort of a bobblehead effect; the boys’ head size is very large for their frame, and the neck strength is not developed. So we determined that because the head acceleration is the same for youth and college players, we needed to have only one level of calibration.

PAUL: The Checklight doesn’t measure just peak acceleration and deceleration. It measures the peaks, but it also captures the time of activation of the electronic components. We bundle this information together and look at aggregate signal output. When we do that it shows more of the shape of a curve and not just a straight-line peak. One more thing – and this is the super-important part – another thing we did early on is we focused on making this a very robust system, so not only do we have triangulated accelerometers to measure linear impact, but we have a gyroscope to measure rotational forces. It’s those two in combination that work symbiotically to measure and describe the head impact.

Does it measure the cumulative effects of multiple hits as well as the effect of a single hit?

PAUL: Probably not. One of things we’ve done is go through all the research, and this is one of those things that’s not super-clear throughout the medical industry. When you listen to medical experts there’s a lot of discussion and debate about whether multiple hits result in injury. That’s one of the reasons why we have the three lights – green on and functioning, yellow a moderate hit, red a more severe hit. If in a finite period of time someone suffers a yellow hit and they didn’t reset the device and then they coupled that with some more hits the light could change to red, yes.

BOB: So with a moderate impact the yellow LED lights, and with a more severe impact the yellow changes to red. We scoured medical literature and we couldn’t find any evidence that five yellows equal one red, or anything like that.

PAUL: Now this is super-important: These impacts, this amount of physical contact, is happening anyway as you participate in these physical activities. What the Checklight does is attach information to these impacts. We do have a yellow light that lights when a moderate amount of head contact occurs, and a red light that goes off when you suffer a big-time hit. When either yellow or red LEDs trigger, we are telling athletes, please get checked out.  To Bob’s point, some of the more benign, low-impact hits to the head absorb energy and result in a yellow light. We implore people that when the Checklight triggers yellow or red to get yourself checked out, either formally with trainers or informally with family or coaches. Please get checked out; don’t ignore it.

What applications do you see for the Checklight?

PAUL: I can say very enthusiastically we’ve used it in everything, from boxing and mixed martial arts and hockey – we play hockey – and then all kinds of sports and activities: equestrian activities, horse jumping – we put it on the rider, not the horse [laughs] – downhill skiing. Motocross is extremely excited about it. Traditional team sports are using it. We’re very proud of the skullcap, but we’re also just moments away from announcing a headband version for unhelmeted activities that will still have the same functionality. We can see people using it anytime they participate in an active lifestyle activity where there’s a potential of them hitting their head.

BOB: One thing we haven’t discussed and I think needs to be noted is that the Checklight also counts impacts as well.  There’s been more and more medical and scientific information on cumulative effects of impacts over time. The Sports Legacy Institute has come up with something called the hit count that’s like a pitch count in baseball. It provides guidelines on the number of head impacts that an athlete should receive.  Checklight counts the number of impacts over time and you can determine impact count whenever you want, the end of the day, a week, whatever. Parents can check the number of hits and reset it to zero whenever they want. This is a very important feature. You know, the pitch count was never based on science; it was just a guideline. You see so many coaches with counters in their hand, and when a pitcher gets to a certain number of pitches they’re coming to remove him from the game. Now we just drop off a kid at football practice and say, “Go ahead; hit your head as many times as you want.” Hit count is something we saw coming, and built into the device right from the beginning.

Since a Checklight doesn’t diagnose a concussion, how do you foresee coaches using it?

BOB: If a coach sees a big hit or impaired player and doesn’t respond, whether the player is wearing a Checklight or not, he’s probably not going to respond any time he sees an impaired player or big hit. If doesn’t see and doesn’t respond, he’ll say he didn’t see it. The Checklight is like having an extra set of eyes out there. As you progress down the levels in athletics, fewer and fewer people are watching kids play youth sports. I’m a youth lacrosse coach, and I sometimes I have 20 kids out there. I really need an extra set of eyes to get youth on the pathway to assessment.

PAUL: In the inside of the Checklight box is a card, a simple card about the size of a business card that explains what it is not. It is not a legal obligation, but it describes what the green, yellow, and red light means, and that you should get the participant  athlete checked out. It’s pretty benign, as opposed to being very formal.

BOB: There are so many different concussion-diagnosis tools out there. Whatever individual tool the coach, team, or league is using, and whatever their best practices are for assessing an impaired player, trying to ID those players and get them into their systems, then we want to work with those best practices.

Have you shown this to any leagues, organizations, and/or sanctioning bodies? What has been their reaction?

PAUL: We’ve shown it around to both major youth football leagues, a number of hockey leagues, as well as lacrosse leagues. They’re super-interested and very keen on supporting it. We’re currently working out the logistics of how implementation could occur, whether it could be a mandated part of standard equipment or voluntary. These organizations just need to be enlightened about this, and have a conversation about this. They need to be convinced about another piece of equipment that measures the magnitude of a hit. It’s just completely new.

Do you think the Checklight or something like it should be mandated?

PAUL: Yeah, I do, in the regard that it helps. It’s almost like wearing a mouthguard. It’s a piece of equipment that helps to inform people about what has just happened. You know, we kinda used to have this notion of when someone got their bell rung, you shake off the cobwebs and go back in. Now we know that it’s not probably the most intelligent and coolest thing to do. So I think it should be mandated. It can be very, very helpful. It can’t be helpful if it isn’t used.

BOB: We joined forces with CSMI, which essentially is an electronic-medical-recordkeeping company for the athletic training industry, colleges and high schools. They happen to be located in same town as us, and they have a lot of people from MIT. They’ve been at this for 20 years or so, and they have a database of about 1 million athletes and 2 million injuries. An athletic trainer can use the CSMI database on a mobile device right there on the field to access a player record that contains contact information, medical history, and more. What we've done with CSMI is incorporate CSMI into our software, so we can add epidemiological evidence of a yellow light or a red light into that database, so we can answer the question of when a yellow light or red light goes off, what is the outcome. This is very powerful information, and it’s the sort of information we can get from the Checklight.

What do you foresee for the Checklight five years down the road?

PAUL: Five years down the road I see the Checklight as a device that not only informs but helps instruct. I’ve seen it anecdotally any number of times; I even saw it when we were testing early on in some high-end Boston-area boxing gyms, places where you can get a lot of data quickly when you’re testing on human subjects. People very quickly came to understand that if they lead with their head the device goes off and you could potentially be at risk; at the very least, you need to get assessed. But if you don’t lead with your head and just play the game with the rest of your body parts as you should, you can play equally intensely, and play effectively, and the lights don’t go off.

BOB: There was a 7^th-grade football team in the Boston area where every player got a Checklight and wore it for every practice and game the entire year. The kids knew they would be coming out to be assessed if the Checklight went off, so they modified their blocking and tackling technique so it didn’t go off. Everyone wants to play; no one wants to go off. So five years down the road, I see coaches using this as a biofeedback tool to teach proper techniques as well as a safety device.

It’s not like another pair of sneakers. Everyone knows what they do, how they work,  what body part they go on. This is a new product. This is a completely new arena for us, where we’re introducing this product. This is more instructional than a pair of shoes.

PAUL: There’s been a lot of media coverage; people are very aware of head impacts. It’s not that oblique; they understand the concept of head impact, and they understand the concept of an indicator that shows head impacts. However, the adoption rate has been very slow, and there’s something I can equate it to. We saw with downhill skiing, five years ago only ski racers were wearing ski helmets.  Then, a couple of years later, because of parents’ concerns, most kids were wearing helmets. Now, last year, 85 percent of the people on the slopes were wearing ski helmets. And that’s just one example. The product is there, and as people get more and more aware of it, it becomes more popular. Something like this doesn’t diminish the experience; it just makes you feel less at risk. As more people see it, they’ll  understand it’s not meant to keep people off the field or diminish an active lifestyle. It’s just something to keep yourself informed. It’s similar to what we saw when people started wearing seatbelts or bike helmets. It takes time to adopt to a new way of doing things. Eventually the majority of people do adopt it.

Have you detected any hesitancy on the part of any league or sport official you’ve spoken with to see this data, and understand the scope of severe head contact within their organization?

PAUL: Absolutely, out of some trepidation of what it’s going to do to the quality of the game, and whether it’s going to put us as risk. For those who have listened to the message and those who have tried it, the transition from skeptic to crusader has been awesome to see. People have been skeptical. Honestly, it’s not been the be-all or end-all. It’s a first step, and it’s a really good first step. Overall, it’s been a positive experience, and for some of the coaches we’ve showed this to, their reaction has been really gratifying to see. They see it’s effective, they see it’s useful, and they see this is a device that just can help.

Is Knowing Better For Everyone, Part 2: The EyeBox

There's a report on the Newsweek site today headlined, "Exclusive: Why The Next Super Bowl Could Be Much Safer." It discusses a new technology patented by a company called Oculogica that uses a sophisticated algorithm to track abnormalities in eye movements that could indicate concussions or trauma resulting from severe head contact.

As writer Kevin Maney states in his story, "There's some excitement building around Oculogica because of a belief that its capability, called EyeBoxCNS, can be built into small, inexpensive devices that every team could have in the locker room. Heck, there's a chance this could end up as a smart phone app, allowing a suburban soccer coach to conduct an accurate sideline concussion scan anytime a couple of players knock heads."

Without bursting anyone's bubble, let me point out again what I noted in my last post, namely that knowing when an athlete suffers a concussion makes athletes only marginally safer, if it makes them safer at all. 

Outside of the Super Bowl, most coaches are already erring on the side of caution, avoiding the side of lawsuits, and holding out any player who shows the slightest sign of a concussion. Certainly there is value in having an accurate diagnosis tool that can make play/no-play decisions more objectively and largely remove coaches from this process. They'll feel better about it, and so will most athletes, parents, and insurance companies. Also, the EyeBox is valuable even if it only keeps one concussed player from re-entering a contest and suffering a second impact. But there is danger in seeing this technology as an advancement in athlete safety.

Think of it this way: If you're climbing a rock wall with a frayed harness and the harness gives way, are you safer because you knew that the harness gave way because it was frayed? There might be something you could do with this knowledge -- grab for the rope, perhaps, because the rope wasn't what failed -- but the only thing that would have made you truly safer would have been a harness that wasn't frayed.

The things that will make athletes safer are changes in rules, changes in culture, changes in coaching, and improvements in equipment. Everything else is after the fact.

One more note on the EyeBox: Like the Checklight, it offers the opportunity to gather data on the number and severity of athletically induced head injuries. This is incredibly valuable data. It's also a huge powderkeg that could blow up in the faces of many large, well-established institutions that have largely been excused from large-scale remediation because of a lack of macro-level data. Widespread adoption of these tools will deliver data and take away that defense.

It's no surprise then that Rich Ellenbogen, a neurosurgeon and co-chairman of the NFL's concussion committee, said that the EyeBox is, in Maney's words, "still more science project than proven technology." Knowing carries with it an obligation to do something with the knowledge. The longer a body can plead ignorance, the more time it buys.

The EyeBox and Checklight are significant steps forward. Now let's see who walks in their footsteps.

Reebok's Checklight: Is Knowing Better For Everyone?

Part of my day job involves keeping up with the latest technology, and this past week that meant following developments at the massive Consumer Electronics Show in Las Vegas.

Out of all the gadgets on display at CES, from an encapsulated three-wheel-vehicle thing from Toyota to a toothbrush that downloads data to your smartphone, one particularly caught my eye. Reebok’s Checklight Impact Sensor, a skullcap that monitors hits to the head, was first demonstrated at last year’s CES. Now it’s back in the form of a production model with an under-$150 street price that was demonstrated memorably by a human punch-model.

Checklight photo courtesy BusinessWire.

After reading the specs and details on the sensor and thinking about its applications for hockey, I guarantee you two things: The Checklight or something like it will be required or semi-required by virtually every hockey organization in the country within two years, and it will be misunderstood or misused by virtually all of them.

I can start proving that assertion right now. One of the stories on the sensor carries the headline, “Reebok CHECKLIGHT Impact Sensor Cap Helps Keep Athletes Safe.” No; it doesn’t really keep athletes safe. The Checklight detects serious head impacts and lights when the severity of impacts surpass a fixed level. It doesn’t detect or diagnose concussions. It’s simply a monitor of blows to the head, and wearing a monitor of blows to the head won’t lessen the frequency or severity of those blows.

Think of it another way. This is not a sensor built into a pair of jeans that detects shoplifting, or a camera that automatically photographs (and tickets) people who run red lights. This is a check-engine light that comes on when something in its environment sets it off, and knowing that your check-engine light might come on does not change the way you drive.

The second and broader way the Checklight likely will be misused comes under the general heading of “do we really want to know?” Who truly benefits from the impartial, impersonal, machine-generated knowledge that someone has sustained a serious blow to the head, and who might be threatened by that knowledge?

Obviously, the athlete benefits. Not every athlete in the heat of competition is sufficiently attuned to their physical state to realize they have sustained severe head contact. Some are simply too young to grasp the situation. Having this information available to these athletes, their coaches and parents seems immediately and obviously to be a good thing … provided someone takes appropriate action.

Appropriate action: Here’s where things get tricky. What is appropriate action? Does calling a penalty constitute appropriate action? A game misconduct? Suspensions? Or is it something more nuanced?

To answer that, consider the technology behind the Checklight. The Checklight is calibrated to trigger at the same point for everyone who wears it. However, people aren’t Rock-‘Em Sock-‘Em Robots; the concussion threshold is different for every person. This means there are four possibilities when someone wearing a Checklight is hit in the head: the contact was insufficient to light the light or cause a concussion; the contact was sufficient to light the light and cause a concussion; the contact was sufficient to light the light but not cause a concussion; or the contact was not sufficient to light the light but was sufficient to cause a concussion.

The last two are very possible and extremely troublesome, if the Checklight enjoys widespread acceptance and is used as a determining factor in removing a player from a game. If the Checklight indicates severe head contact but the contact did not cause a concussion, and there are organizational mandates that a player be removed from a game when his Checklight lights and it’s a big game … you can see where this is going. Throw in a my-son’s-going-pro parent and you’ve got the recipe for a brouhaha that could easily wind up in a courtroom.

You can certainly argue that mandated removal is erring on the side of caution, but it’s extremely imperfect erring-on-the-side-of-caution. No coach this side of the NFL would play a concussed player knowingly in any game, but seeing as the Checklight isn’t a concussion-diagnosis tool, how many coaches would hold out a player from a crucial game solely on the basis of a lit-up Checklight and no other symptoms – unless they have to?

Coaches and officials are in a pickle with Checklight.  Consider the scenario where the light doesn’t light but the player suffers a concussion. If a coach and officials let a concussed player with an unlit Checklight keep playing, and s/he sustains another head impact and suffers severe injuries from Second Impact Syndrome, who is liable? The coach, who made the decision in ignorance of evidence to the contrary? The officials, who didn’t remove the lit player from the game and didn’t take steps to keep further head contact from occurring? The sanctioning body, which made (or didn’t make) a rule that allowed concussed players to keep playing? Reebok, for making a unit that can’t distinguish between significant but not injury-causing head contact and contact sufficient to cause concussion?

My wife is a former actuary and hockey mom who’s unusually attuned to liability issues. She says she wouldn’t touch the Checklight – not because it doesn’t work but because of liability concerns like these. At some point someone is going to be held liable, and it’s going to be whoever’s caught holding the hot potato.

I come at this from a different angle. At times with my day job I’m a data person. In my line of work, data is isn’t any good if you don’t collect it, and once you collect it, you need to do something with it. I would think and hope that data from the Checklight would not stop with the Checklight. If sharks can send tweets when they’re approaching shore, Checklights should be able to send notification of head injuries to multiple sanctioning bodies – the NCAA, USA Hockey, the local organization, the state organization, academic researchers, medical authorities, whomever – via USB now and wirelessly in the future. There’s potential with the Checklight to collect and aggregate a staggering amount of information on head contact. If every hockey player from the squirt level up wore a Checklight, and if the data from those Checklights was sent to USA Hockey, then USA Hockey would know once and for all the scope of severe head contact in youth hockey.

But maybe, just maybe, USA Hockey (or Pop Warner Football or the Wisconsin Interscholastic Athletic Association or whomever) doesn’t want to know the scope of severe head contact in their sport. Knowing carries with it the obligation to do something about it, and if a sanctioning body is unwilling or unable to do something about it, it might be better to plead ignorance.

If, for instance, Checklight data shows USA Hockey that there were significantly more instances of severe head contact than the organization had previously believed or was willing to admit, what could the organization do with that data? Use it as rationale for banning all head contact? Stiffen penalties for offenders? Based on past experience with similar rule changes, whatever safety steps USA Hockey would propose would meet with fierce opposition, Checklight data or no. And if the organization accedes to rank-and-file pressure and balks at making changes in the face of empirical data that demands changes, then what?

(Just as a case study in the value -- or cost --  of knowing, once the National Football League realized the scope of severe head injuries suffered by players, it settled with a group of former players for whatever it could. In this case it turned out to be between $765 million and $914 million -- and a judge just threw out the settlement, saying the players weren't getting paid enough.)

I know what I’d do, if I were a sanctioning organization. First, this issue is too important not to collect data when the opportunity to collect data is so immediate and obvious. I’d buy Checklights for a sample population and collect the data on head contact. If the data suggested a significant problem and/or additional study, I’d spend the money, escalate the study, and analyze the data from that study.

If the data from that escalated study showed a greater level of head contact than previously thought and a significant correlation between lit Checklights and concussions, I’d mandate Checklight purchase, remove players with lit Checklights from games, and severely penalize opposing teams – but more importantly, I’d gather data on teams and players that delivered the triggering hits and require that data to be reported up along with the data on the hit itself. The only way to lessen the occurrence of head injuries is to remove the culture of headhunting from the sport (any sport, from lacrosse to hockey), and the first step to doing that is to identify the headhunters.

In short, the Checklight is a highly imperfect step forward – but a step forward nonetheless. Kudos to Reebok; the company has met a definite need, and deserves to profit from doing so. In the next several weeks, we hope to have an interview with Reebok and officials from sanctioning bodies discussing the Checklight and its implications. It could be that some of these concerns are groundless. (I hope so.) It could be that we’re thinking ahead of the people whose job it is to think ahead. (I hope not.) One way or another, we’re going to find out.