+ Site Statistics
+ Search Articles
+ PDF Full Text Service
How our service works
Request PDF Full Text
+ Follow Us
Follow on Facebook
Follow on Twitter
Follow on LinkedIn
+ Subscribe to Site Feeds
Most Shared
PDF Full Text
+ Translate
+ Recently Requested

Constructing a validity argument for the Objective Structured Assessment of Technical Skills (OSATS): a systematic review of validity evidence



Constructing a validity argument for the Objective Structured Assessment of Technical Skills (OSATS): a systematic review of validity evidence



Advances in Health Sciences Education 20(5): 1149-1175



In order to construct and evaluate the validity argument for the Objective Structured Assessment of Technical Skills (OSATS), based on Kane's framework, we conducted a systematic review. We searched MEDLINE, EMBASE, CINAHL, PsycINFO, ERIC, Web of Science, Scopus, and selected reference lists through February 2013. Working in duplicate, we selected original research articles in any language evaluating the OSATS as an assessment tool for any health professional. We iteratively and collaboratively extracted validity evidence from included articles to construct and evaluate the validity argument for varied uses of the OSATS. Twenty-nine articles met the inclusion criteria, all focussed on surgical technical skills assessment. We identified three intended uses for the OSATS, namely formative feedback, high-stakes assessment and program evaluation. Following Kane's framework, four inferences in the validity argument were examined (scoring, generalization, extrapolation, decision). For formative feedback and high-stakes assessment, there was reasonable evidence for scoring and extrapolation. However, for high-stakes assessment there was a dearth of evidence for generalization aside from inter-rater reliability data and an absence of evidence linking multi-station OSATS scores to performance in real clinical settings. For program evaluation, the OSATS validity argument was supported by reasonable generalization and extrapolation evidence. There was a complete lack of evidence regarding implications and decisions based on OSATS scores. In general, validity evidence supported the use of the OSATS for formative feedback. Research to provide support for decisions based on OSATS scores is required if the OSATS is to be used for higher-stakes decisions and program evaluation.

Please choose payment method:






(PDF emailed within 0-6 h: $19.90)

Accession: 057496073

Download citation: RISBibTeXText

PMID: 25702196

DOI: 10.1007/s10459-015-9593-1


Related references

Value and face validity of objective structured assessment of technical skills (OSATS) for work based assessment of surgical skills in obstetrics and gynaecology. Medical Teacher 30(2): 212-216, 2008

Assessing the validity evidence of an objective structured assessment tool of technical skills for neonatal lumbar punctures. Academic Emergency Medicine 20(3): 321-324, 2013

Using the Objective Structured Assessment of Technical Skills (OSATS) global rating scale to evaluate the skills of surgical trainees in the operating room. Surgery Today 43(3): 271-275, 2013

Validity and reliability of the robotic objective structured assessment of technical skills. Obstetrics and Gynecology 124(4): 839, 2014

Validity and reliability of the robotic Objective Structured Assessment of Technical Skills. Obstetrics and Gynecology 123(6): 1193-1199, 2014

A new curriculum for hysteroscopy training as demonstrated by an objective structured assessment of technical skills (OSATS). American Journal of Obstetrics and Gynecology 193(5): 1856-1865, 2005

Objective Structured Assessment of Technical Skills (OSATS) evaluation of hysteroscopy training: a prospective study. European Journal of Obstetrics, Gynecology, and Reproductive Biology 178(): 1-5, 2015

Comparing the Effects of Objective Structured Assessment of Technical Skills (OSATS) and Traditional Method on Learning of Students. Nursing and Midwifery Studies 4(2): E27714-E27714, 2015

Adaptation of "Objective Structured Assessment of Technical Skills" for Adenotonsillectomy into Turkish: A Validity and Reliability Study. Turkish Archives of Otorhinolaryngology 57(1): 7, 2019

Construct Validity of a Simple Laparoscopic Ovarian Cystectomy Model Using a Validated Objective Structured Assessment of Technical Skills. Journal of Minimally Invasive Gynecology 24(5): 850-854, 2017

Objective Structured Assessment of Technical Skills (OSATS) evaluation of theoretical versus hands-on training of vaginal breech delivery management: a randomized trial. European Journal of Obstetrics, Gynecology, and Reproductive Biology 171(2): 252-256, 2014

A randomized comparison of video demonstration versus hands-on training of medical students for vacuum delivery using Objective Structured Assessment of Technical Skills (OSATS). Medicine 96(11): E6355-E6355, 2017

Validity evidence for two objective structured clinical examination stations to evaluate core skills of the shoulder and knee assessment. Bmc Medical Education 17(1): 13, 2017

Objective Structured Assessments of Technical Skills (OSATS) Does Not Assess the Quality of the Surgical Result Effectively. Clinical Orthopaedics and Related Research 474(4): 874-881, 2016

Objective structured assessment of technical skill (OSATS) for surgical residents. British Journal of Surgery 84(2): 273-278, 1997