Date: Thu, 05 Jun 2003 00:54:48 -0500
From: Eric Klinger <email@example.com>
Subject: Assessment of the Psychology Discipline; modification of method statement
Cc: firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com
I. Assessment Results
This is our latest report on assessment of the Psychology Discipline, focusing on the Empirical Investigations (EI) courses. These courses constitute a capstone experience in which each student completes an empirical project from beginning to end, culminating in a journal-style report. It entails reviewing and interpreting scientific literature, devising hypotheses and methods for testing these, data collection, data analysis and interpretation, and write-up. Because of its many components, these projects serve for each student a dual function: an intensive educational experience and a competency assessment. Collectively, these projects also reflect on the effectiveness of instruction in the Psychology Major and therefore serve as appropriate data for assessing effectiveness of important aspects of the program. This report conveys our data and conclusions based on our assessment of EI projects completed during academic years 2001-2003.
The procedure was two-fold. First, each Psychology faculty member rated each of his or her own students' projects as to whether the student had met Objectives 2-4 of our discipline's assessment plan. Second, each faculty member also read and rated in the same way two reports (in the case of one reader, three reports) on projects completed by students working with other faculty. The attached Excel table summarizes the results of these ratings.
Inspection of the table indicates that the raters judged the great majority of students to have met program objectives, both when their own instructors made these judgments and when other instructors rated their final papers. The few students who failed to meet some objectives tended to fail at Objective 4, "Competency in quantifying and statistically analyzing behavior."
Therefore, with some softness regarding Objective 4, the group concluded that the program was by and large meeting its objectives.
Psychology Discipline faculty engaged in lengthy discussion of how we might strengthen competency in statistical analysis. One problem that occurred to us is that often years go by between a student's last systematic instruction in statistics and application in an EI. Forgetting occurs. The faculty therefore resolved to build into every feasible course in Psychology some experience in which students actively compute statistics, to serve as a continuing refresher and reinforcer of previous statistical instruction.
The Psychology Faculty decided on a biennial assessment of EI's completed during the previous two years, the assessments henceforth to take place in August so as to provide more time for laggard students to finish their current-year projects. The next such assessment is scheduled for August, 2005.
There are also other kinds of EI-related information that serve as indicators of program effectiveness. Every empirical project in Psychology must meet the approval of either the Institutional Review Board (IRB) or the Institutional Animal Care and Use Committee (IACUC). Both of these committees require investigators to submit applications for the use of human or animal subjects, respectively, in particular projects. Applicants must provide a brief rationale for their projects and describe their methods in detail. The committees consist primarily of faculty investigators from around the University as well as some other members, including some who represent the broader community. The fact that each EI requires IRB or IACUC approval to proceed is a judgment of at least minimal quality of the design stages of our EI's.
In response to a recent survey, we also toted up the numbers of our students who have received authorship credit on manuscripts submitted to professional journals or on conference presentations. We arrived at the following tabulation for the years 1998-2003:
Category of Student Authorship Number of Students
Manuscripts submitted to professional journals 29
Manuscripts submitted to undergraduate journals 0
Papers or posters presented at professional conferences 16
Papers or posters presented at undergraduate conferences 31
Footnote credit in a manuscript for publication. 6
A majority of these authorships arose out of work on EI's, but we have not kept track of the precise number. Undergraduate conferences exclude local presentations such as the UMM Undergraduate Research Symposia. There is some overlap in membership among the categories.
Another result of the discussions arising out of the assessment process was the decision to add to our methods for assessing Learning Objective 1 (Awareness of the range of knowledge (data methods) in psychology). The discipline will investigate the distribution of courses that Psychology majors take at the upper-division level. The purpose is to evaluate the extent to which these courses are well distributed across the principal areas of the field.
Furthermore, in the future we shall seek to keep track of ratings in students' teaching evaluations of how much they learned in their courses.
II. Modification of the Assessment Plan
As a result of our discussions, we realized that we should modify the Assessment Methods section of the Assessment Plan. For Objective 5, it now reads:
Presentation of a paper and other written responses demonstrating satisfactory knowledge of how ethical principles and legal constraints will impact the student's functioning in a postbaccalaureate human-services-related professional position
We now wish to add the following:
completion of a successful application to the Institutional Review Board (IRB) or the Institutional Animal Care and Use Committee (IACUC) for undertaking an empirical investigation, which requires functional knowledge of ethical and legal obligations for conducting research
The entire statement of methods for Objective 5 would then read:
Presentation of a paper and other written responses demonstrating satisfactory knowledge of how ethical principles and legal constraints will impact the student's functioning in a postbaccalaureate human-services-related professional position; completion of a successful application to the Institutional Review Board (IRB) or the Institutional Animal Care and Use Committee (IACUC) for undertaking an empirical investigation, which requires functional knowledge of ethical and legal obligations for conducting research
I don't know how one goes about making this change. Could you inform us about this?
Professor of Psychology
Division of Social Sciences
University of Minnesota, Morris
600 East Fourth Street
Morris, MN 56267 USA
(320) 589-6209 (Morris office), -1023 (home), -6117 (fax) www.morris.umn.edu/academic/psychology/klinger.shtml