Results

Fourteen teams submitted results for evaluation prior to the challenge workshop. Below you will find the summary of the results and ranking of the methods. For details about the evaluation measures, please see the Evaluation page. Slides from the presentation given at the workshop by the organizers with a data set description and a summary of the results can be downloaded here

Presentations from some of the teams that were made at the workshop are available for download at the bottom of this page.

Ranking according to the overall F1-score

Ranking according to the overall F1-score. The full team names can be found at the bottom of this page.
Team name Precision Recall F1-Score
IDSIA 0.610 0.612 0.611
DTU 0.427 0.555 0.483
SURREY 0.357 0.332 0.344
ISIK 0.306 0.351 0.327
PANASONIC 0.336 0.310 0.322
CCIPD/MINDLAB 0.353 0.291 0.319
WARWICK 0.171 0.552 0.261
POLYTECH/UCLAN 0.186 0.263 0.218
MINES 0.139 0.490 0.217
SHEFFIELD/SURREY 0.119 0.107 0.113
SEOUL 0.032 0.630 0.061
NTUST 0.011 0.685 0.022
UNI-JENA 0.007 0.077 0.013
NIH 0.002 0.049 0.003

Ranking according to the F1-score computed for each patient separately

Ranking according to the F1-score computed for each patient separately. The full team names can be found at the bottom of this page.
Team name Average F1-score Average rank
IDSIA 0.445 2.364
DTU 0.352 2.818
WARWICK 0.302 4.000
SEOUL 0.231 5.273
ISIK 0.226 5.545
PANASONIC 0.213 5.818
CCIPD/MINDLAB 0.208 6.091
SURREY 0.205 6.273
MINES 0.203 6.273
POLYTECH/UCLAN 0.148 6.909
SHEFFIELD/SURREY 0.099 8.636
NTUST 0.068 9.455
UNI-JENA 0.016 10.455
NIH 0.002 11.273

Submission after the challenge worksop

Team Contact person Date of submission Method Precision Recall F1-score Average F1-score
The Chinese University of Hong Kong Hao Chen 01/28/2014 Deep convolutional neural network (4 convolutional, 2 fully connected layers) 0.690 0.310  0.427 0.271

IDSIA: IDSIA, Dalle Molle Institute for Artificial Intelligence, USI-SUPSI, Lugano, Switzerland [web site]
DTU: Technical University of Denmark 
SURREY: Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, UK
ISIK: Department of Computer Science and Engineering, Isik University, İstanbul, Turkey
PANASONIC: Panasonic Healthcare
CCIPD/MINDLAB: CCIPD at Case Western Reserve University, USA and MindLab at National University of Colombia
WARWICK: University of Warwick, UK
POLYTECH/UCLAN: University Nice - Sophia Antipolis, France and University of Central Lancashire, UK
MINES: Institut Curie, Mines ParisTech, France
SHEFFIELD/SURREY: Sheffield Institute for Translational Neuroscience, Sheffield University, UK and Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, UK
SEOUL: Seoul National University, Korea
NTUST: Graduate Institute of Biomedical Engineering, National Taiwan University of Science and Technology
UNI-JENA: University of Jena, Germany
NIH: National Institutes of Health, USA and Shape Analysis Research Project, National Institute of Standards and Technology, USA