December , 1996
The University of Minnesota, Morris (UMM) is an undergraduate, residential, liberal arts campus of the University of Minnesota with about 2000 students and 120 teaching faculty located in a rural community 150 miles west of the Twin Cities. It offers the academic reputation of the University of Minnesota with the special atmosphere of a small college. Its primary institutional mission is to offer undergraduate education in the liberal arts including licensure programs in teacher education. This mission as a single purpose liberal arts college has not changed since the college was founded in 1960 and reflects its historic values, current strengths, and plans for the future.
Evaluation, appraisal, and assessment of the quality of courses, programs, and services has been practiced at the University of Minnesota, Morris for years. With an active institutional research program, it was clear that substantial outcome-oriented evaluation was already taking place including longitudinal profiling of the student body as a whole, student opinion surveys, follow-up studies of graduates and non-returning students, and analysis of graduation and retention rates. Students evaluate every course. A detailed evaluation of the general education program was recently completed and many disciplines appraised the competency of their majors through cap-stone seminars, tutorials, portfolios, and recitals. Some of the current activities related with the assessment of students' learning are given in the appendix.1
The campus is doing a great deal, but these efforts fell short in important ways. For example, the goals of general education and the majors are not directly evaluated through student outcomes, i.e., that students demonstrate that they have acquired the skills, techniques, and knowledge required. Even less frequently are the assessment techniques related directly to well articulated curricular objectives. The results of the assessment efforts that are not tied into institutional planning and resource allocation as directly as they should be. Perhaps most important, the results of assessment are not systematically used to improve student learning in a regular, ongoing way.
To overcome some of these shortcomings in the college's assessment activities, a task force made up of faculty and students representative of the governance committees with a major stake in assessment (and supported by staff familiar with the current institutional research efforts), worked during the spring of 1995 to develop a plan specifically to assess student learning across the campus. The group's principal objective was to develop a conceptual model for the assessment which would be generally applicable at each level where student learning and achievement take place. These levels would include the course; the discipline curriculum and major; the general education program and its components; several support programs such as academic assistance, honors, study abroad; and some of the educational, social, and recreational programs of the extracurriculum. The conceptual model was to unify the assessment process across units, in each case beginning with the institutional mission, moving to unit goals and objectives, then to the assessment and analysis phase leading to appropriate action at the individual, unit, or institutional level as the case may be. The process would be controlled through the governance system by the faculty. In addition, rather than an intrusive chore imposed by an outside agency, the process ought to be compatible with the natural responsibilities of faculty members committed to teaching and research in their respective disciplines. After the receipt of reviewer's reports on the proposed plan which ask for revisions, a second task force has been appointed by UMM Campus assembly in the Fall of 1996. The second task force made up of two faculty representatives from Science and Mathematics, Social Science, and Humanities divisions and one faculty from Education Division who were nominated by the divisions, two students, and the Dean of Academic Affairs. The task force has been charged to review the draft UMM Assessment Plan, bring it back to Assembly with any revisions that seem necessary, and take steps to begin the implementation of the plan, fulfilling the duties of the proposed Assessment of Student Learning Committee until its establishment.
The Assessment of Student Learning is a discipline and program-based process aimed at improving the teaching and learning at UMM. It is intended to determine how well what students are actually learning conforms to the objectives of that academic enterprise. Although the assessment process will produce a body of information which will be useful in the preparation of discipline and institutional self-studies, the primary purpose is program improvement. Further, it is the intention of the both task forces that, as a matter of policy, this assessment process should not bear a connection to the college's faculty evaluations for the purposes of promotion, tenure, and salary determination.
Explicit statements of the institutional mission, goals, and educational objectives are contained in the 1995-97 UMM Bulletin. The formal mission statement, approved by the Campus Assembly in 1993, is as follows:
The mission of the University of Minnesota, Morris as an undergraduate, residential liberal arts college is distinctive within the University of Minnesota. The Morris campus shares the University's statewide mission of teaching, research, and outreach, yet it is a small college where students can shape their own education. The campus serves undergraduate students primarily from Minnesota and its neighboring states, and it is an educational resource and cultural center for citizens of west central Minnesota. Through its instructional excellence, its commitment to research, its numerous extracurricular programs and services, and its strong sense of community, the University of Minnesota, Morris endeavors to achieve its place among the best liberal arts colleges in the region.
The goals of the academic program at the University of Minnesota, Morris are expressed through the requirements for the bachelor of arts degree . The degree requirements consist of three parts, two of which are in general education: Process Requirements and Expanding Perspectives Requirements. The third part is the Major, or field of specialization; its requirements are specified by faculty in each discipline (1995-97 UMM Bulletin, p. 56). The requirements are meant to prescribe student competencies, which are usually demonstrated through the successful completion of qualifying courses but may be met by demonstrating proficiency in other ways.
The first goal of general education is to become familiar with the process of liberal learning--to acquire the intellectual skills, the communication skills, and the framework of knowledge needed for successful advanced work. The second goal is to expand one's intellectual perspectives, gaining enough understanding of the principal areas of human endeavor to be able to continue learning in the future and to have a sense of the limits of one's knowledge. Successful study in a major field, in which one pursues knowledge in depth with the goal of becoming reasonably expert, constitutes the third area required for the B.A. degree.
Process of Liberal Learning
The Process requirements emphasize the development of intellectual skills, the critical and creative thinking skills, and the communication skills needed for future work. The goals of the process requirements are as follows.
The purpose of the major is to ensure that each student pursues a particular field of knowledge in depth, investigates advanced theories and schools of thought, and becomes competent in using the language and methods of inquiry of the field. Through such concentrated study, conducted over an extended period of time, a student begins to master an existing body of knowledge and understands the nature of expertise in the chosen field, including both its power and its limitations. Each of 27 formally approved majors have certain specific goals defined by the faculty of the disciplines involved.
As designed by the task force, the conceptual model consists of unit assessment cycles as well as an institutional assessment cycle. Units include the course, the discipline curriculum, the major, the general education curriculum or its components, and other programs--for example, Study Abroad--where significant student learning has been identified (a detailed list of units is given in appendix)1. The overall institutional cycle aims to provide feedback among units to assist them in identifying overlapping student learning needs, and to integrate the results of individual unit assessments. The conceptual framework not only allows the unit assessment cycles to flow from the institution's published mission and goals, but it also creates channels to identify necessary changes in institutional goals.
The key ingredient of the model is the Unit Assessment Cycle, given in Figure1, which consists of seven elements.
The cycle originates with the unit's goal/mission and returns to this stage as the cycle is completed. Through various actions taken by a unit, the cycle interconnects with cycles from other units and with the institutional assessment cycle as a whole. Since the model cycle will be employed by all the units, the assessment process will be consistent across the campus. In addition, the similarity among assessment cycles will have the beneficial effect of providing multiple measures of effectiveness of the academic program as a whole. Rather than impose a new approach upon units, it is intended that the cycle provides a conceptual framework for assessment which is compatible with appraisals currently being carried out in most units in an informal way.
The cycle begins with a statement of the mission and goals particular to that unit. While they should be compatible with the institutional mission, the unit goals may be quite specific and initially may not take into consideration the mission and goals of other units. The institutional assessment cycle is designed to integrate the goal and mission of all units. Even if it is not true at the beginning, this dynamic process will in time integrate unit and institutional goals.
Learning objectives will flow from the unit's mission and goals and will be detailed enough to cover the different functions of the unit. Based upon the unit's goals, an individual instructor for a course, or the discipline faculty in the case of a major, will identify the specific learning objectives. They may be as specific as those for a particular course (for example, understanding a cost/benefit analysis) or as general as those for the major (for example, provide students with a basic understanding of the nature and functioning of the economic system).
Units must next specify, based upon their learning objectives, a variety of expected outcomes, measurable in qualitative or quantitative terms. Depending upon the unit's goals, the expected outcomes may be stated as cognitive, behavioral, or attitudinal characteristics. The outcomes can be as specific as being able to solve differential equations, being able to integrate trigonometric functions, or being able to interpret the results of a factor analysis, or as broad as being able to explain how the development of mathematics has been part of the evolution of civilizations and is intimately interwoven with their cultural and scientific development. At this stage in the assessment cycle the expected outcomes represent predictions of how student learning will be demonstrated.
Assessment Methods and Tools
Each unit will select or develop its own assessment methods and tools. Assessment methods may be based on observational or experimental data collection processes. These methods will include portions of examinations in key courses, the products of capstone experiences, seminars, recitals, locally-developed examinations, surveys, oral examinations, professional licensure examinations, standardized comprehensive examinations, portfolios, alumni follow-up surveys, and the like (please see the Figure 4).
Different units may select and develop differing assessment methods and tools to measure the same or similar expected outcomes. This will have the advantage of creating multiple assessment measures more likely to capture the complete range of student achievements and promote innovative and "better" assessment techniques as their results are shared during the assessment cycles with other units.
When the assessment method, tools, and techniques have been utilized with the appropriate students, the results will be analyzed by the unit itself and interpreted in terms of the expected outcomes identified earlier during the planning phase. The results will become part of one or more assessment outcome documents. These may include discipline or program self-study reports, annual discipline/division/committee reports, institutional data summaries, accreditation self-study reports.
Neither the process of appraisal nor knowledge of the results automatically leads to constructive change and improvement. The assessment model must include an action stage, providing for an appropriate response to the results of the assessment of student achievement. The most direct action, and that which routinely occurs at the present time, is for the results to be provided to students and used in improving their achievement. Action may occur as the modification by faculty of a course or a discipline curriculum, or by a governance committee of a program or administrative unit. In particular, the assessment process must, as a matter of policy, influence the institution's decision making processes which determine curriculum, pedagogy, and resource allocation. At the action stage, as shown in Figure 2, the cycle provides for sharing recommendations for change based upon documented results of the assessment process.
Impact on the Students' Learning
Once an action has been taken to improve student achievement, its impact will be evaluated to see whether the desired improvement actually occurred. If the proposed action requires a higher unit's involvement that impact should be measured and evaluated as well. Note that this stage might call for the application of the assessment tools developed in a previous stage. It may also lead to a change in the unit's mission and goal.
Back to the Unit Goal/Mission
One of the outcomes of unit assessment will often be a modification of that unit's statement of missions and goals. An Assessment Committee (see below) will both guide the process and act as a clearinghouse for information and recommendations which emerge from the unit assessment cycles. One of its responsibilities will be to provide information to other units, appropriate governance committees, the administration, and the Campus Assembly, which may result in modifications of units or institutional goals.
Administration of the Assessment Process
The assessment of student learning is ultimately a faculty responsibility. The assessment process is to be faculty designed and supervised, with a committee of the Campus Assembly having major responsibility for the program. On November 18, 1996, the UMM Assessment plan and a constitutional By-law amendment proposal establishing the Assessment of Student Learning Committee was presented to the Campus assembly by the Executive Committee for information. The By-law amendment proposal is given below:
By-law Amendment Proposal1
Rationale: In keeping with its effort to fulfill its educational mission, UMM seeks to implement a process whereby assessment of student learning occurs in an ongoing manner at every appropriate level and informs the development of educational policy and practice. A standing committee is necessary to assure that this process operates effectively. This committee will help UMM meet Criteria Three from the NCA Accreditation standards.
Membership: The Assessment of Student Learning Committee consists of ten members, including one faculty member from the Division of Education, and two each from the Divisions of Humanities, Science & Mathematics, and Social Sciences, and two students. The Dean or his designee will serve ex officio.
Powers: The Assessment of Student Learning Committee oversees and provides support to all aspects of the assessment process, receives all data and materials generated by assessment activities, recommends improvements in the assessment program and disseminates reports on the results of assessment and the initiatives based on assessment intended to improve student learning.
Technical restraints within the UMM Constitution prevent the operation of a new standing committee until the next academic year. To allow the process to move forward this academic year, the Executive Committee recommended to the Campus Assembly the creation of a second Task Force on the Assessment of Student Learning. It solicited nominations for membership from the Division chairs and then forwarded a slate to the Campus Assembly. The Assembly approved the recommendation and selected the membership for the Task Force at its meeting of November 18, 1996. The immediate duties of the Task Force are to review the 1995 Assessment Plan in light of the NCA referee critique and other relevant information, to consult with the faculty and appropriate committees in its review of the plan and bring an Assessment Plan to the Campus Assembly for its approval in February, 1997. The Task Force, in carrying the responsibilities of the forthcoming Standing Committee, will also guide the development and implementation of the assessment process, facilitate unit involvement, provide relevant results to the other governance committees (Curriculum, General Education, Campus resources and Planning, and Scholastic are the most germane) and recommend necessary actions to the Campus Assembly. The relevant portions of the agenda of the November 18, 1996 Campus assembly are in the appendix.
Proposed structure of the administration of the assessment process and the place of the Assessment Committee is given in Figure 5.1
At the present time, the Office of the Vice Chancellor for Academic Affairs and Dean is responsible for much of the institutional research, evaluation, and appraisal which occurs on the Morris Campus. This activity will continue, since it plays an essential role in resource allocation and in appraising institutional effectiveness for self studies and accreditation reviews. For this reason, the Office of the Vice Chancellor for Academic Affairs will coordinate and provide support for the Committee for the Assessment of Student Learning. On November, 1996 the Vice Chancellor for Academic Affairs named Engin Sungur, Associate Professor of Mathematics, as Interim Director of Assessment.
Timetable for Implementation
A detailed timeline for the first cycle of the assessment process is
given in the following table. The first cycle includes stages such as
organization, planning, application, dissemination, and overall
assessment of the assessment of students' learning process. The
organization stage consists of activities which will maximize the faculty
involvement and create an atmosphere which will make the assessment a
crucial part of the institutions academic culture. One of the key
activities at this stage is the Assessment of Students' Learning Planning
Exercise and Survey I1 (a copy of the survey is included in the
Appendix). In November the Task Force on Assessment of Students'
Learning designed and start implementing a series of assessment planning
exercises and surveys. The objectives of these exercises and surveys are
The last stage of the first cycle is the overall assessment of the UMM Assessment of Students' Learning Process. The Task Force ( after the approval of the Campus Assembly, the Assessment of Students' Learning Committee) will also work on the unit assessment cycle and develop a goal/mission, learning objectives, expected outcomes, and assessment methods and tools to assess the overall effectiveness of the assessment process. This unit will determine the problems, discuss successes and failures, provide evidence on impact of the process on the students' learning, and most important of all will carry out an academic cost/benefit analysis. A report on the overall assessment of the process will be prepared and distributed to the faculty and all other related units including the NCA.
The following cycles will include planning revision and updating, application, dissemination, and overall assessment of the process stages. The length of each cycle will be proposed by the Committee based on the inputs from the units' experiences on the first cycle.