Scoring of challenges

Challenge points

1. judges may assign points at their discretion.
2. any challenger who is NOT challenging appropriately (i.e. unsubstantiated or baseless challenges to game CACAO) will be punished.
3. points are cumulative - if the challenger corrects multiple minor and major problems, the points will be added for each item.

Identification of a Minor Problem

1 point each

• includes format issues
  • PMC used instead of PMID
  • 0005737 used (instead of GO:0005737)
• use of an unnecessary qualifier
• use of an unnecessary/non-applicable with/from
• use of an "illegal" evidence code (i.e. not one of the 7 permitted for CACAO)
• use of an "illegal" GO term (i.e. binding or response to terms)
• incomplete annotations (i.e. needs with/from, but not filled in; missing a required field)
• annotation that doesn't cite a specific figure in the notes field
• reference is a review or not a peer-reviewed article
• annotation is an exact duplicate of an existing annotation

Identification of a Major Problem

2 points each

• wrong protein page
• with/from entry does not have experimental evidence for ISA or ISO
• with/from entry has wrong UniProt accession
• inappropriate GO term (requires suggestion to fix, or will be counted as unsubstantiated challenge)
• evidence code (requires suggestion to fix, or will be counted as unsubstantiated challenge)
• figure(s)/table(s) in notes does not support annotation

Correction of a Minor Problem

1 point each

• correctly suggests right PMID for PMC entry
• correctly suggests fix for simple format problems (i.e. GO:0005737)
• easily identify and suggest the figure(s)/table(s) described in notes
• easily identify and suggest the correct evidence code
• easily identify the correct UniProt accession to go in the with/from field

Correction of a Major Problem

2 or more points each

• correct protein page identified & UniProt accession given
• more appropriate GO term suggested & explained
• more appropriate evidence code suggested & explained
• give the UniProt accession for a protein that could be used in the with/from for ISA or ISO as a "match protein"
• more appropriate figure(s)/table(s) in notes & explained
• new and appropriate article suggested with complete and correct annotation explained

Complete Correction

5 point bonus

• To encourage challenges, any student who successfully challenges and "wins" credit for the annotation will be awarded 5 points in addition to any points awarded for identification and/or corrections

Therefore, if a student successfully fixes the last issue, they will have earned >5 points, not necessarily on the same challenge (if multiple challenges were attempted). Challenges awarded >10 are not uncommon if there were multiple issues in the original annotation and the challenger was persistent and/or thorough.