Results

Results prior to the challenge workshop

Fourteen teams submitted results for evaluation prior to the challenge workshop. Below you will find the summary of the results and ranking of the methods. For details about the evaluation measures, please see the Evaluation page. A PDF of the presentation given at the workshop by the challenge organizers with a data set description and a summary of the results can be downloaded from here. A MS Excel file will the full results can be downloaded from here.

Ranking according to the overall F1-score

Ranking according to the overall F1-score. The full team names are given below.
Team name Precision Recall F1-Score
IDSIA 0.610 0.612 0.611
DTU 0.427 0.555 0.483
SURREY 0.357 0.332 0.344
ISIK 0.306 0.351 0.327
PANASONIC 0.336 0.310 0.322
CCIPD/MINDLAB 0.353 0.291 0.319
WARWICK 0.171 0.552 0.261
POLYTECH/UCLAN 0.186 0.263 0.218
MINES 0.139 0.490 0.217
SHEFFIELD/SURREY 0.119 0.107 0.113
SEOUL 0.032 0.630 0.061
NTUST 0.011 0.685 0.022
UNI-JENA 0.007 0.077 0.013
NIH 0.002 0.049 0.003

Ranking according to the F1-score computed for each patient separately

Ranking according to the F1-score computed for each patient separately. The full team names are given below.
Team name Average F1-score Average rank
IDSIA 0.445 2.364
DTU 0.352 2.818
WARWICK 0.302 4.000
SEOUL 0.231 5.273
ISIK 0.226 5.545
PANASONIC 0.213 5.818
CCIPD/MINDLAB 0.208 6.091
SURREY 0.205 6.273
MINES 0.203 6.273
POLYTECH/UCLAN 0.148 6.909
SHEFFIELD/SURREY 0.099 8.636
NTUST 0.068 9.455
UNI-JENA 0.016 10.455
NIH 0.002 11.273

 

Full team names

IDSIA: IDSIA, Dalle Molle Institute for Artificial Intelligence, USI-SUPSI, Lugano, Switzerland [method description: link]
DTU: Technical University of Denmark [presentation slides: pdf]
SURREY: Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, UK [presentation slides: pdf, link]
ISIK: Department of Computer Science and Engineering, Isik University, İstanbul, Turkey
PANASONIC: Panasonic Healthcare
CCIPD/MINDLAB: CCIPD at Case Western Reserve University, USA and MindLab at National University of Colombia
WARWICK: University of Warwick, UK
POLYTECH/UCLAN: University Nice - Sophia Antipolis, France and University of Central Lancashire, UK [presentation slides: link].
MINES: Institut Curie, Mines ParisTech, France
SHEFFIELD/SURREY: Sheffield Institute for Translational Neuroscience, Sheffield University, UK and Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, UK [presentation slides: link]
SEOUL: Seoul National University, Korea
NTUST: Graduate Institute of Biomedical Engineering, National Taiwan University of Science and Technology [presentation slides: pdf]
UNI-JENA: University of Jena, Germany
NIH: National Institutes of Health, USA and Shape Analysis Research Project, National Institute of Standards and Technology, USA

New submissions

Submission date Team Contact person Method Precision Recall F1-score Average F1-score 
28-01-2014 Department of Computer Science and Engineering, The Chinese university of Hong Kong Hao Chen Deep convolutional neural networks. Staining normalization is applied prior to the classification. 0.690 0.310 0.427 0.271