Report of Course Assessment

Course: Psy 1061.  Introduction to the Development of the Child and Adolescent

Date Report Released to Assessment Committee: 3/23/11

Instructor:  Dr. Katherine Benson, Psychology

Purpose:  Assessment is an important part of the University of Minnesota, Morris’s accountability as a public institution of higher education. One accountability question frequently raised about college courses is whether the students learned material in the course that they did not know before they took the course. Moreover, a special issue for psychology has to do with the frequently heard characterization that psychology is just “common sense,” and so there is little new learning that takes place in psychology courses. Sometimes this question of how much new learning occurs is referred to as “value-added” learning in discussions of accountability and assessment: how much valuable knowledge was gained in the course that wasn’t there before? The purpose of the present assessment, therefore, was to measure “value-added” learning in my largest course, Psy 1061, Introduction to the Development of the Child and Adolescent.

Rationale: In order to measure the degree to which the Psy 1061 course imparted value-added learning to class members, a pre- and post-test measurement design was used. In other words, the students’ knowledge of the content of one particular module was assessed prior to the introduction of teaching for that module; then, students’ individual pre-test scores were compared to their individual post-test scores on questions from the module included in Exam 1, which exam measured learning in the module as well as on other topics. Improvement in individuals’ scores from the pre- to the post-test demonstrated the degree to which students achieved “value-added” learning in that module of Psy 1061. (Because I believe in the importance of assessment, I try to assess one or two modules in every content course I teach, varying the module from year to year.)

Choice of Content Module: The focus of the present assessment was on the module within the Psy 1061 course on “Genetic Mechanisms and Heredity.”  There were several reasons for the selection of the genetics module. First, many of the students in this course for future teachers have academic majors in one or more of the sciences, and so students might have been introduced to the topic of genetics in one or more science courses prior to this psychology course. Possibly, some students might even have taken an entire course on genetics; these students might be expected to know this material before enrolling in Psy 1061.  Moreover, Introductory Psychology is a pre-requisite for the Psy 1061 course; it also contains substantial content on genetics. Thus, the possibility must be entertained that mastery of the Psy 1061 exam content on genetics might reflect prior learning rather than learning in Psy 1061. 

An additional reason for the choice of genetics is that this is an increasingly important area of developmental psychology; advances in genetics occur every week, and with the near-conquest of infectious diseases, genetic disorders are an increasingly large proportion of causal factors for childhood health and behavioral problems. The future teachers in Psy 1061 will inevitably encounter pupils who have genetic disorders, and a background of the basics of genetics and heredity will be essential to working with these children and their families, if only because knowledge affects attitudes and understanding. For all these reasons, therefore, the purpose of this assessment was to measure the degree to which mastery of the important genetics course material on the Psy 1061 Exam 1 reflected a “value-added” component of Psy 1061 for the students, and was not, to the contrary, an indicator that the students already knew the material before the Psy 1061 module was presented. 

Method:  A 10-point pre-test was given to the students before the topic “Genetic Mechanisms and Heredity” was introduced.  The post-test assessment data were taken from Exam 1 for the course, which exam covered the material from the genetics module along with other material.  For both tests, a sample of the possible knowledge in the unit was used; most of it was basic information about genetics and heredity, e.g., “What are genes made of?” Answer: “DNA.” The machine-scoring, scanner-sheet reader in Behmler Hall’s Computer Center was used to partition the genetics module scores from the exam scores as a whole. More about this technology will be discussed later.

Results:  Before the results are presented, it is important to clarify several things. According to Dean Contant at her first UMM Fall Faculty Retreat, assessment data must be examined individually for each student. However, and this is a very important “however,” the report of these assessment data must be in aggregate. Therefore, I will report the results in what to me is a “cryptic” format—a format I don’t usually use in science writing, so that individual students will not appear even anonymously. In addition, I am not reporting the year that these data were collected for the course, although they are “recent.” I have erred on the side of protecting individuals, since this is the first report I have prepared for the Assessment Committee, and if I went too far to protect student identity, you can tell me. Note: When percentages do not add up to 100%, this is due to rounding.

The results of the Psy 1061 assessment were as follows:

Pre-test Performance:

1. Students, nearly universally, did not know the material on the genetics pre-quiz: 86% of the class failed, meaning they scored lower than 60% correct.  There were 11% of the students who earned a grade of “D-,” which would be a percentage between 60% and 62.9%.  

2. How bad were the failures of those who failed the pre-quiz? 34% answered either no or only one of the 10 questions on the pre-quiz correctly; another 23% answered only 2 or 3 out of 10 pre-quiz questions correctly, and 18% answered 4 of 10 questions correctly.  Only 14% earned a score of 5 of 10 questions correct on the pre-quiz.

3. The 3% of students who had a score of “A-” on the pre-quiz, which was between 90% and 92.9% correct, and which indicated prior learning of the material, had taken a prior genetics module from me; no other students had taken a prior course from me that covered genetics as a topic.) 

Post-test Performance:

1. 95% of the students improved their scores from the pre- to the post-tests for genetics. Only 4% of the class had scores which decreased on the genetics post-test as compared to their pre-test scores, and then by only 1 or 2 points.

2. Of the whole class, 70% earned grades of “B+” or better on the post-test for genetics, which showed good mastery of the material on the genetics portion of Exam 1. 

3. Unfortunately, 14% of the students still failed the module; 16% improved—even substantially—but only achieved a level of “Unsatisfactory,” which would be a grade in the range of the three levels of a  “D” grade.

4. What accounts for the 30% of the class who earned unsatisfactory grades on the post-test? Anecdotally, many of these students were those whose class attendance was irregular, so attendance would be something to emphasize in future classes.

Conclusion:  The students, as a whole, learned a great deal from the module on “Genetic Mechanisms and Heredity” in Psy 1061, with a large majority (70%) mastering the material.  This is important, because genetics is an increasingly important area of knowledge. They did not know this material before they took the Psy 1061 class; (the students who had taken a prior course from me dropped the class before the post-test was administered because they felt it was too similar to the earlier class—and, at least up until that point, it was). The introduction to the material in the pre-requisite course for Psy 1061 was not retained the first time, but still may have helped make the learning easier during the second exposure to it in Psy 1061.

Assistance Offered:  I offer to teach other faculty how to use the technology for scoring and assessing exams in the Behmler Hall Computer Center, if anyone is interested in learning it for assessment. One less-involved way to use the program is quickly to ascertain each student’s performance on each topic/chapter of the course, so that this feedback can be given to individual students. I have found that it encourages struggling students to see that, even if they had great difficulty with parts of the test, they still were able to do well on a particular chapter or topic, etc., and this helps them see that they can, in fact, improve and do well.

Teaching Objectives. My goals for the Psy 1061course, (and all my courses, in fact,), are not only that students will master content knowledge, though that is one important goal. An example of a more experiential assignment in Psy 1061 to help strengthen their judgment as teachers as well as their ability to foresee and avoid problems “on their feet” is through the assigned readings of Case Studies and the writing and discussion based on these. I have not thought of a way to assess this type of learning yet, other than through written assignments, discussion, and feedback, but I am giving it much thought.

Thank you for your attention.