+ Site Statistics
+ Search Articles
+ Subscribe to Site Feeds
Most Shared
PDF Full Text
+ PDF Full Text
Request PDF Full Text
+ Follow Us
Follow on Facebook
Follow on Twitter
Follow on LinkedIn
+ Translate
+ Recently Requested

Monitoring scale scores over time via quality control charts, model-based approaches, and time series techniques

Monitoring scale scores over time via quality control charts, model-based approaches, and time series techniques

Psychometrika 78(3): 557-575

Maintaining a stable score scale over time is critical for all standardized educational assessments. Traditional quality control tools and approaches for assessing scale drift either require special equating designs, or may be too time-consuming to be considered on a regular basis with an operational test that has a short time window between an administration and its score reporting. Thus, the traditional methods are not sufficient to catch unusual testing outcomes in a timely manner. This paper presents a new approach for score monitoring and assessment of scale drift. It involves quality control charts, model-based approaches, and time series techniques to accommodate the following needs of monitoring scale scores: continuous monitoring, adjustment of customary variations, identification of abrupt shifts, and assessment of autocorrelation. Performance of the methodologies is evaluated using manipulated data based on real responses from 71 administrations of a large-scale high-stakes language assessment.

(PDF emailed within 0-6 h: $19.90)

Accession: 054456047

Download citation: RISBibTeXText

PMID: 25106404

DOI: 10.1007/s11336-013-9317-5

Related references

On control charts for monitoring the variance of a time series. Journal of Statistical Planning and Inference 143(9): 1512-1526, 2013

Crop Yield Forecasted Model Based on Time Series Techniques. Journal of Northeast Agricultural University (English Edition) 19(1): 73-77, 2012

Time series techniques for dynamic, real-time control of wood-drying processes. Forest products journal 55(10): 64-71, 2005

Time series prediction by a neural network model based on bi-directional computation style: A study on generalization performance with the computer-generated time series Data Set D. Systems and Computers in Japan 34(10): 64-75, 2003

Detection of changes in time-series of indicators using CUSUM control charts. Aquatic Living Resources 22(2): 187-192, 2009

Time-between control charts for monitoring asthma attacks. Joint Commission Journal on Quality and Safety 30(2): 95-102, 2004

Radiocarbon time scale in the early Holocene and isotope time series based on tree-ring chronologies. Terra Nostra (Bonn) 1-94: 31-33, 1994

Synthetic-type control charts for time-between-events monitoring. Plos One 8(6): E65440, 2015

Quality Control Procedure Based on Partitioning of NMR Time Series. Sensors 18(3), 2018

Research on monitoring mechanical wear state based on oil spectrum multi-dimensional time series model. Guang Pu Xue Yu Guang Pu Fen Xi 30(11): 2902-2905, 2011

Ship's tracking control based on nonlinear time series model. Applied Ocean Research 36: 1-11, 2012

A wavelet-based time-varying autoregressive model for non-stationary and irregular time series. Journal of Applied Statistics 39(11): 2313-2325, 2012

Visualizing the intercity correlation of PM2.5 time series in the Beijing-Tianjin-Hebei region using ground-based air quality monitoring data. Plos One 13(2): E0192614, 2018

Estimation and monitoring system of the cardiovascular dynamics under ventricular assist device pumping based on a time series model. Japanese Journal of Artificial Organs 20(3): 848-857, 1991

Control charts for accident frequency: a motivation for real-time occupational safety monitoring. International Journal of Injury Control and Safety Promotion 21(2): 154-162, 2015