Dynamic Learning Maps
Acknowledgements
1
Introduction
1.1
Impact of COVID-19 on the Administration of DLM Assessments
1.2
Background
1.3
Assessment
1.4
Technical Manual Overview
2
Map Development
3
Item and Test Development
3.1
Items and Testlets
3.1.1
Blueprint Updates
3.1.2
Item Writing
3.2
External Reviews
3.2.1
Review Recruitment, Assignments, and Training
3.2.2
Results of Reviews
3.2.3
Test Development Decisions
3.3
Operational Assessment Items for 2019–2020
3.4
Field Testing
3.4.1
Description of Field Tests
3.4.2
Field Test Results
3.5
Conclusion
4
Test Administration
4.1
Overview of Key Administration Features
4.1.1
Change to Administration Model
4.1.2
The Instruction and Assessment Planner
4.2
Administration Evidence
4.2.1
Assessment Blueprint
4.2.2
Adaptive Delivery
4.2.3
Administration Incidents
4.3
Implementation Evidence
4.3.1
Data Forensics Monitoring
4.3.2
Released Testlets
4.4
Conclusion
5
Modeling
5.1
Overview of the Psychometric Model
5.2
Calibrated Parameters
5.2.1
Probability of Masters Providing Correct Response
5.2.2
Probability of Non-Masters Providing Correct Response
5.2.3
Item Discrimination
5.2.4
Base Rate Probability of Mastery
5.3
Mastery Assignment
5.4
Model Fit
5.5
Conclusion
6
Standard Setting
7
Assessment Results
7.1
Data Files
7.2
Score Reports
7.2.1
Individual Student Score Reports
7.3
Quality Control Procedures for Data Files and Score Reports
7.4
Conclusion
8
Reliability
9
Validity Studies
9.1
Evidence Based on Test Content
9.1.1
Alignment
9.2
Evidence Based on Response Processes
9.2.1
Test Administration Observations
9.2.2
Interrater Agreement of Writing Sample Scoring
9.3
Evidence Based on Internal Structure
9.3.1
Evaluation of Item-Level Bias
9.3.2
Internal Structure Within Linkage Levels
9.4
Conclusion
10
Training and Professional Development
10.1
Required Training for Test Administrators
10.2
Instructional Professional Development
10.2.1
Professional Development Participation and Evaluation
10.3
Conclusion
11
Conclusion and Discussion
11.1
Operational Assessment
11.1.1
Future Research
12
References
Appendix
A
Differential Item Functioning Plots
A.1
Uniform Model
A.2
Combined Model
2019–2020 Technical Manual Update
2019–2020 Technical Manual Update
Year-End Model
December 2020
Placeholder Title
Copyright © 2020 Accessible Teaching, Learning, and Assessment Systems (ATLAS)