Abstract
This project developed a protocol for the inaugural Instructional Software Competition of the American Dental Education Association (ADEA). The evaluation instrument was derived from the Guidelines for the Design of Educational Software developed by the ANSI-accredited Standards Committee for Dental Informatics. Eleven judges were calibrated in a conference call and rated a total of 30 submissions using a 66-question instrument. The maximum score was 204 points. The mean score of WWW-based programs was 106.7 points, and of CD-ROM-based programs 109.5 points. The summative review of the judging process identified several potential improvements, such as distinguishing between standalone programs and educational support material; increasing the number of answer choices on rating scales; differential weighting of criteria; and a more discriminative approach to judging formative and summative evaluations. We plan to improve the protocol by supporting the process through a Web-based application; calibrating judges with an online handbook; improving and adapting the rating instrument itself; using at least three judges for each program; and conducting a measurement study.
| Original language | English (US) |
|---|---|
| Pages (from-to) | 603-606 |
| Number of pages | 4 |
| Journal | Proceedings / AMIA ... Annual Symposium. AMIA Symposium |
| State | Published - 2001 |
| Externally published | Yes |
Fingerprint
Cite this
Developing a protocol for an educational software competition. / Schleyer, Titus; Johnson, L.
In: Proceedings / AMIA ... Annual Symposium. AMIA Symposium, 2001, p. 603-606.Research output: Contribution to journal › Article
}
TY - JOUR
T1 - Developing a protocol for an educational software competition.
AU - Schleyer, Titus
AU - Johnson, L.
PY - 2001
Y1 - 2001
N2 - This project developed a protocol for the inaugural Instructional Software Competition of the American Dental Education Association (ADEA). The evaluation instrument was derived from the Guidelines for the Design of Educational Software developed by the ANSI-accredited Standards Committee for Dental Informatics. Eleven judges were calibrated in a conference call and rated a total of 30 submissions using a 66-question instrument. The maximum score was 204 points. The mean score of WWW-based programs was 106.7 points, and of CD-ROM-based programs 109.5 points. The summative review of the judging process identified several potential improvements, such as distinguishing between standalone programs and educational support material; increasing the number of answer choices on rating scales; differential weighting of criteria; and a more discriminative approach to judging formative and summative evaluations. We plan to improve the protocol by supporting the process through a Web-based application; calibrating judges with an online handbook; improving and adapting the rating instrument itself; using at least three judges for each program; and conducting a measurement study.
AB - This project developed a protocol for the inaugural Instructional Software Competition of the American Dental Education Association (ADEA). The evaluation instrument was derived from the Guidelines for the Design of Educational Software developed by the ANSI-accredited Standards Committee for Dental Informatics. Eleven judges were calibrated in a conference call and rated a total of 30 submissions using a 66-question instrument. The maximum score was 204 points. The mean score of WWW-based programs was 106.7 points, and of CD-ROM-based programs 109.5 points. The summative review of the judging process identified several potential improvements, such as distinguishing between standalone programs and educational support material; increasing the number of answer choices on rating scales; differential weighting of criteria; and a more discriminative approach to judging formative and summative evaluations. We plan to improve the protocol by supporting the process through a Web-based application; calibrating judges with an online handbook; improving and adapting the rating instrument itself; using at least three judges for each program; and conducting a measurement study.
UR - http://www.scopus.com/inward/record.url?scp=0035748021&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=0035748021&partnerID=8YFLogxK
M3 - Article
SP - 603
EP - 606
JO - Proceedings / AMIA . Annual Symposium. AMIA Symposium
JF - Proceedings / AMIA . Annual Symposium. AMIA Symposium
SN - 1531-605X
ER -