Pre- and Post-Conference Workshops
Workshop I, Sunday, October 11, 9:00 a.m.–noon, Kansas room
Development and validation of diagnostic reporting in student monitoring systems (CITO)
Student Monitoring Systems in the Netherlands have been in use for over 25 years in primary education and for over 15 years in secondary education. Recently the system was extended to use in education for children with special needs. CITO also supports the development of student monitoring systems in Kazakhstan, Curaçao, and Switzerland.
A variety of reporting formats have been developed over time, and each time new experiences have been added to adapt the reporting to local use and habits. All systems have in common that they use items response modeling, which makes it possible to equate tests over time. In addition, one of the attractive and convincing applications of item response modeling is the possibility to interpret the operationalized construct through the position of the items on the calibrated scale. This allows interpretation of the ability level required to master the items at specific position on the scale. This interpretation can be utilized in a diagnostic way; however, this use remains under-utilized as the actual use of the systems, as is the case in the Netherlands, has been normative. Although a diagnostic use has been advocated from the very beginning, it has only began gaining attention in more recent years.
Normative and diagnostic uses rely heavily on the expertise of the teacher and the schools to interpret the information of the monitoring system into actions toward the student, in which it is expected that the action results in a learning progression. The type of reporting provided by the system is of paramount importance in this matter. For all kinds of reasons, it is difficult to change the reporting in an existing system, but it is easier to propose new reports, based on experiences from existing systems, when one has the opportunity to develop a new one, like was done in Kazakhstan, Curaçao, and Switzerland. All these experiences have led to the development of various reports, such as ones with level descriptors, profile analysis, and guidelines for action to be taken. The recent developments on validation of the reporting have been inspired by the evidence-centered design approach.
We intend to share our experience in the development and validation of diagnostic reporting with the participants of this workshop on development of diagnostic reporting in student monitoring systems.
Workshop II, Sunday, October 11, 1:00–4:00 p.m., Kansas room
Fundamentals of diagnostic classification modeling (CETE)
Traditional measurement models are aimed at the development of score scales that place examinees along a continuum. Such models serve many purposes well, but do not directly answer some of the most important questions in a learning environment. That is, they do not lead to specific diagnoses of what individual learners do or do not yet know; thus, they do not support precise educational prescription. This workshop, led by Dr. Jonathan Templin, will describe the philosophical principles behind diagnostic classification models, describe several different diagnostic classification models, and demonstrate how these models work. This workshop is aimed at participants who have some formal knowledge of measurement models such as item response theory.
Workshop I, Thursday, October 15, 1:30–4:30 p.m., Kansas room
Test development in a learning maps environment (CETE)
In this workshop, participants will explore how learning maps influence test development. Presenters will share examples from the Dynamic Learning Maps alternate assessments in English language arts, mathematics, and science, which were designed for students with significant cognitive disabilities. An introduction to learning maps will be presented. The process for creating large, fine-grained learning maps will be explained. The influence of learning maps at all phases of test development will be described, with topics ranging from the selection of content and design of test development resources to psychometric approaches for analyzing test content. The benefits of the learning map environment will be explained. No prerequisite knowledge or experience is assumed. This workshop will be led by Dr. Russell Swinburne-Romine, who will be assisted by several key staff from the Dynamic Learning Maps Alternate Assessment Project.
Workshop II, Friday, October 16, 9:00 a.m.–noon, Jayhawk room
Use of learning maps to support formative assessment in mathematics (CETE)
Learning maps can serve as tools for examining in detail the content teachers are responsible to teach, allowing teachers to consider both the content and student learning simultaneously – i.e., content knowledge and knowledge about students, which are two arms of Mathematics Knowledge for Teaching. This workshop will demonstrate how the content and structure of learning maps can support teacher use of good formative assessment practices. No prerequisite knowledge or experience is assumed. This workshop will be led by Dr. Angela Broaddus.