Should student attainment of learning outcomes become the coin of the realm?
By author Ron Carriveau
The answer to this provocative question may lie with the use of the word “outcome” itself. It could be argued that everything in life has an outcome that results from an action that has occurred, or has an expected outcome that is dependent on certain actions taken. If we don’t obey the sign that says “stop” when we are driving, then the outcome may be a traffic ticket or a trip to the courthouse. Could this consequence also have been an expected outcome? In terms of probability, the answer would be yes.
In education, there is an abundance of student learning outcome expectations, often stated as the student “will” know or “do” something or think in a particular way, or “will be able to” meet these expectations. Learning outcome statements read as though 100% attainment is expected, but in reality the expectation is that students will attain the outcomes to some degree. So it is paramount that a valid measure of the attainment of the outcomes can be calculated.
In the education world, outcome statements are created by course instructors and by various types and levels of organizations. Outcomes can come from a program, institution, professional organization, regional accrediting body, and state and federal departments of education, among others. Additionally, large and highly influential educational organizations publish what outcome expectations should be met for anyone who goes through an educational system and receives a document that validates their learning.
To get a measure of the students’ attainment of the outcomes, assessments of various types are used, and the student receives points on the measurement scale used for the assessment. What makes the scores “outcome attainment measures” is that each item used in the various assessments, whether selected response items (like multiple-choice) or constructed response items (like writings, performances, and projects) is directly linked to an outcome statement.
If it is agreed that assessment items should be written to “specific” learning outcome statements, then what should be used to report student attainment of the outcomes? Furthermore, what should be used to evaluate and report the degree of attainment of the outcomes at the course, program, and institutional levels, and beyond to outside organizations?
In Connecting the Dots, I suggest a three-level model to accomplish these tasks:
- Attainment of the specific learning outcomes (sLOs) is derived from the percentage of items correct that are directly linked to the sLO statements and thus is useful to the instructor for making instructional and course design decisions.
- The sLO attainment values aggregate to a broader level, called the General Learning Outcome (GLO) level, useful for reporting at the department and program levels.
- The GLOs aggregate to the broadest level, called GOAL, which is useful for reporting at the institutional levels.
The attainment values can also be assigned to outcome statements from outside organizations when there is an outcome statement match. Thus, we have a structure for calculating attainment of outcomes at all levels and a way to communicate the attainment to all stakeholder levels, including the individual student level.
Should an outcome attainment measure challenge the traditional coin of the realm – grades – for making valid decisions about students, courses, and institutions? The answer is yes, and rightly so.
The validity of grades awarded has been questioned for many years by measurement experts and by educators (Haladyna, 1999). The issue is that with traditional grading, the total points that can be attained for a course includes points for actions that are not directly related to learning outcome statements or the calculation of outcome attainment values, actions such as attendance, attitude, class participation, extra credit, neatness, participation, and violation of deadlines to name a few.
Some educators support giving grade points for some or all of these actions, some do not support giving points, and some say it is open to argument or discussion. Giving points for these actions can cause grade inflation (even when outcome based assessment results are included in the total points earned) or can lower grades that would be awarded if based on outcome-attainment values only, and thus can significantly affect the validity of the grades awarded as measures of student attainment of the outcomes.
Connecting the Dots
Developing Student Learning Outcomes and Outcomes-Based Assessment
Ronald S. Carriveau
This book is designed to help faculty and institutions of higher education meet these demands by obtaining, managing, using, and reporting valid outcome attainment measures at the course level; and mapping outcome attainment from the course level to departmental, degree program, and institutional levels, and beyond. It demonstrates how to communicate clearly what students are supposed to know and be able to do; write assessments that measure the expectations; and produce test scores that are valid for their intended use and interpretation, so that valid inferences can be made about students and programs.
It is a “how-to” manual that is rich with guidelines, model forms, and examples that will lead the reader through the steps to “connect the dots” from outcomes assessment to outcomes-based reporting.
About the Author
Ronald S. Carriveau is the Outcomes, Assessment, and Measurement Specialist, and QEP Assistant Director, At the Center for Learning Enhancement, Assessment, and Redesign, University of North Texas, and is the assistant director for the university’s Quality Enhancement Plan for accreditation.
Dr. Carriveau earned a Ph.D. in Educational Psychology. He has taught in the public school system, held the position of associate professor at the university level, served as director of assessment at a large school district and at the state department of education, and was a test development manager for a national test publisher. He has consulted nationally and internationally.
For more information about Connecting the Dots, sample chapters and book reviews, visit the book page.