No this isn’t the beginning of a game of Catch Phrase or Taboo, these are the tools researchers used to create reproducible seizure forecasting for epilepsy patients(with 2 legs and 4).
The ability to accurately forecast epileptic seizures has been the white rabbit of epilepsy research for years. The problem was a lack of access to long-term data, in open access form, for data scientists to test.
Why is this a game-changer?
There have been several studies that have linked specific pre-seizure “brain states” to epileptic episodes. But have been handcuffed in turning this knowledge into relevant clinical forecasting algorithms or methods for the seizures.
This is where crowdsourcing has swept in and changed the game. Taking the EEG data from eight canines and two humans with the disease. Without getting too technical, the researchers reviewed and classified the data according to different metrics.
All of this data was then made available for download on the site Kaggle.com and the community was challenged to “…to develop algorithms in any computer language and using any features, classification and data processing methods they chose, but classifications were required to come directly from an algorithm—classification by visual review was prohibited. Algorithms were also required to use a uniform data processing method for all subjects, but were permitted to modify data processing methods based on data parameters, such as sampling frequency. Contestants uploaded preictal probability scores (a floating point number between 0 and 1 indicating the probability of each clip being preictal) for the 3935 testing data clips in a comma separated values file, and a real-time public leader board on kaggle.com provided immediate feedback on classification accuracy. Public leader board scores were computed on a randomly sampled 40% subset of the test data clips, but official winners were determined based on the remaining 60% of the testing data”
Using Kaggle.com as a crowdsourcing platform to conduct this contest/study provided the researchers with access to hundreds of data scientists all working on the same data and problem.
It was a huge success with many teams coming up with statistically significant algorithms for forecasting seizures in both epileptic dogs and people. “During the four-month contest, over half of these crowdsourced algorithms performed better than random predictions. The best performing algorithms accurately predicted more than 70 percent of seizures when tested on unseen portions of the canine data.”
That is an amazing achievement for these researchers and data scientists. We hope to see more of this type of research conducted in other fields to bring the power of the internet to the lab!