Spanish Discipline Assessment 2006-2007

Spanish Discipline Assessment 2006-2007

Scope of assessment activities

___⇒__Course-embedded assessment

___⇒___ Pre- and post-testing

______ Outside the classroom

______ Across the discipline

Direct measures of student learning

___⇒__ Capstone experience

______ Portfolio assessment

___⇒__ Standardized tests

______ Performance on national licensure, certification or

preprofessional exams

______ Qualitative internal and external juried review of

of comprehensive senior projects

______ Externally reviewed exhibitions and performances in

the arts

______ External evaluation of performance during internships

Discussion and Description

Discipline goals, direct measures, and improved student learning

1. Spanish discipline objectives. The Spanish curriculum is designed to help students

· develop critical insight into the philosophy and values of another culture

· gain fluency in a second language

· gain sensitivity toward literature that reflects the experience of the Spanish-speaking world.

It accommodates liberal arts students interested in a cross-cultural perspective, language study, secondary school teaching, or preparation for graduate study in the field.

2. Capstone course: Research Symposium

2.1. A new course.

This new course will be required of all Spanish majors entering under Morris Catalog 2007-2009. It was not required of 2007 Spanish majors, since students can graduate under the requirements of any catalog in effect during their residency. However, four seniors chose to take the capstone course.

2.2. Course objectives.

The capstone experience consists of

· an introduction to research methods and critical approaches to literature

· the development of an independent research project and presentation.

2.3. Assessment tool.

Student learning was assessed in the areas of listening, reading, speaking, and writing using American Council for the Teaching of Foreign Languages (ACTFL) guidelines and rating categories.[1] In addition, the instructor produced a written assessment of each research project, noting the critical approach used, and using a quantitative evaluative scale for four categories: originality of ideas; quality of research; persuasiveness of argumentation; and organization.

2.4. Rating student learning.

There are eleven rungs in the ACTFL ladder of ratings: distinguished, superior, and three levels (high, mid, low) each of advanced, intermediate, and novice. No student in any of the four areas of listening, reading, speaking, and writing was evaluated at less than advanced low, and some received a distinguished rating.

2.5. Improving student learning.

The author of the discipline report notes that in “reviewing the four papers we discovered that students did not recognize some situations where a particular theory would have been most appropriate for their discussions. We have decided therefore to introduce literary theory earlier in the curriculum [.]” This will be in a required third year course, Seminar: Origins of the Spanish Character. Students will study seven literary theories and write essays in which they apply them to the literature being read. Their mastery will be gauged on a five-step scale ranging from “does not understand the theories and how to apply them” to “shows exceptional skill in applying the theories.”

3. Course-embedded assessment. Pre-test/post-test

As part of the objective of gaining fluency in a second language, the Spanish discipline has sought to improve listening skills of students in Beginning Spanish II, the second semester of the introductory course.

The assessment tool is the Iowa Placement Exam in Spanish, the listening part consisting of twelve questions. The IPE is used to place first-year students in the proper course, either the first- or second-semester of the introductory course or beyond. The placement constitutes the pre-test. The IPE is again administered at the end of the second semester course, which constitutes the post-test.

The tool to improve listening skills is an eight-week series of listening exercises. Students

· listen to passages on compact discs developed by native speakers who have been UMM students;

· go through the passages with the instructor to resolve any problems of comprehension; and

· listen to the passages again.

Two groups of students were tested, those who had placed directly into Beginning Spanish II, and those who had placed into Beginning Spanish I and were now completing the second semester. On the pre-test, students rarely complete more than half of the twelve questions correctly. On the post-test, the average successful completion rate was 6.5 to 7.0 of 12.

Two issues were identified that each elicited a response from the discipline.

i. The author of the Spanish assessment report notes that there was a “minimal change in listening skill.”[2] Next year the listening exercises will last the entire semester rather than half.

ii. Interpreting the results was problematic because the two groups tested clearly differed in the knowledge and/or skills they possessed on entering college.[3] The spring 2007 assessments now offer a baseline for those in spring 2008 to see if doubling the listening exercises improves student listening skills.

General education categories spanned by the discipline

Spanish courses carry one of the following general education designators: FL, foreign language; Hum, communication, language, literature, and philosophy; IP, international perspectives; or Envt, people and the environment. The only exception is directed study, which carries no general education designator.

Speech Communication Discipline Assessment 2006-2007

Scope of assessment activities

___⇒__Course-embedded assessment

_______ Pre- and post-testing

______ Outside the classroom

___⇒__ Across the discipline

Direct measures of student learning

___⇒__ Capstone experience

___⇒__ Portfolio assessment

______ Standardized tests

______ Performance on national licensure, certification or

preprofessional exams

______ Qualitative internal and external juried review of

of comprehensive senior projects

______ Externally reviewed exhibitions and performances in

the arts

______ External evaluation of performance during internships

Discussion and Description

Discipline goals, direct measures, and improved student learning

1. Speech communication discipline objectives. Students

i. develop a historical and theoretical understanding of the three areas of speech communication: rhetoric, communication studies, and mass media

ii. use a variety of assigned theoretical approaches appropriate to these three areas to describe and evaluate assigned or chosen discourse

iii. participate in a variety of oral communication assignments using informative and persuasive speaking techniques effectively.

The summaries below draw primarily on the report for the 2006-2007 academic year. Discipline objectives have been assessed annually. The corresponding reports, similar to the 2006-2007 report, dating from the 2002-2003 academic year, are at UMM’s discipline assessment web-site.[4]

2. Rhetorical studies

Discipline objectives i) and ii) were assessed for rhetoric. For the first objective, two expected outcomes were identified. Students will

· be able to compare and evaluate various theoretical approaches

· demonstrate a sensitivity to the historical dimensions of theory building.

Data was drawn from student papers, which were evaluated with respect to three criteria. The data set for the first objective was deemed too small to be of value.

For the second objective, an expected outcome was identified.

· The students will be able to choose from a variety of methods to describe and evaluate a specific act or artifact.

Seven papers were assessed according to the same three criteria as for objective one. Students were ranked on a scale of 0-5 on their ability to cite and paraphrase sources, and to analyze discourse. The scores were averaged and recommendations made for program adjustments.[5]

3. Communication studies

Discipline objectives i) and ii) were assessed with the same expected outcomes as for rhetorical studies. Papers from two courses were reviewed but this time with respect to five criteria, the criteria reflecting crucial abilities and skills in communication studies. Once again, the students were rated on a 0-5 scale for each criterion, results averaged, and compared to performances from the previous year. Based on this assessment, recommendations were made for program adjustments.

4. Media studies and technology

Since the professor in this area was on sabbatical, this objective was not assessed in 2006-2007. The following describes the assessment in 2005-2006.[6] Objective number ii) was assessed for electronic mass media. The expected outcome was the same as for the second objective under rhetorical and communication studies. Papers were evaluated with respect to five criteria that once again measured crucial abilities and skills of students of electronic mass media. All relevant papers in the students’ portfolios (vide infra) were reviewed. Once again, each criterion was evaluated on a five-point scale, results averaged, and compared to averages from previous years. The significance of the comparisons was discussed and recommendations made.

5. Personal portfolios

Students create personal portfolios which are evaluated collectively during the senior year.

6. Speech communication senior seminar presentations

Speech communication seminar in 2006-2007 provided the vehicle for assessing learning objective number iii) for the first time. There is an expected outcome.

· The students will be able to design and deliver effective messages through the oral communication channel.

The effectiveness of each of eight student speakers was assessed with respect to ten criteria on a 0-4 numerical scale. The averaged results will be a benchmark against which future assessments can be measured. The results indicate four areas where the student performance was particularly strong and two where improvement is needed. Overall, “the results do indicate that in the aggregate students in Speech Communication meet Learning Objective #3.”[7]

General education categories spanned by the discipline

Almost all speech communication courses carry one of the following general education designators: E/CR, ethical and civic responsibility; Hum, communication, language, literature, and philosophy; IP, international perspective; SS, human behavior, social processes, and institutions; or HDiv, human diversity. Exceptions are directed study, directed experience in teaching speech communication, and speech communication seminar I, which have no general education designator.

Statistics Discipline Assessment 2006-2007

Scope of assessment activities

___⇒__Course-embedded assessment

___⇒___ Pre- and post-testing

___⇒__ Outside the classroom

___⇒__ Across the discipline

Direct measures of student learning

___⇒__ Capstone experience

___⇒__ Portfolio assessment

______ Standardized tests

______ Performance on national licensure, certification or

preprofessional exams

___⇒__ Qualitative internal and external juried review of

of comprehensive senior projects

______ Externally reviewed exhibitions and performances in

the arts

______ External evaluation of performance during internships

Discussion and Description

Discipline goals, direct measures, and improved student learning

1. Structure of assessment activities in the statistics discipline

Assessment of student learning occurs in four areas:

· general education

· the statistics major and minor

· liberal arts statistical support system

· special areas of service learning/civic engagement and technology enhanced learning

The focus in this report will be on the major and general education, and on technology enhanced learning only to the degree that it bears on the two areas of focus.

2. Three phases

The assessment program is divided into three phases:

· setting forth the discipline’s mission, establishing learning objectives, delineating expected outcomes, identifying and organizing assessment methods and tools

· identifying possible uses and actions based on assessment

· improving student learning based on assessment data

3. Discipline learning objectives

· Students will gain the basic knowledge and skills to make statistical contributions to modern society, whether in the form of pure statistics or statistics applied to other disciplines.

· Students will sharpen their statistical intuition and abstract reasoning as well as their reasoning from numeric data.

· Statistics and statistics curriculum will enhance students’ critical thinking in domains involving judgments based on data and stimulate the type of independent thinking requiring research beyond the confines of the textbook.

· The curriculum will prepare students to enter graduate school, and pursue careers in applied statistics.

· The students will be able to see and communicate statistical ideas/results effectively and identify potential pitfalls of any statistical analysis.

Each learning objective is accompanied by expected outcomes.

4. Course-embedded assessment of the general education component

4.1 Learning checks

A learning check is a student’s performance on a statistical topic such as scatterplots or least-squares regression[8]. Thirty checks are used every semester in every section of the two introductory statistics courses, Introduction to Statistics, with a high school algebra prerequisite, and the calculus-based Statistical Methods. This tool was implemented in 1997 when the college curriculum was based on the quarter system. The database has 3,986 points as of spring 2007.

4.2 Retention of student learning study

This study sought to measure the amount of information and types of skills that students retained after they had taken one of the introductory statistics courses. The tool was a new version of the comprehensive final exam previously taken. Students also filled out a comprehensive questionnaire that provided background information to help interpret the results. The results of the exams were converted into a quantitative “relative loss” parameter—how much information and skill had the student lost? On average, the forty-eight students who took the retention exam had taken introductory statistics 2.5 years earlier. The results were analyzed to see which kinds of information and skill were lost or retained, and whether there was a correlation to the instructor, the year the course was taken, the final course grade, and gender.[9]

5. Capstone course and e-portfolios

5.1 Senior seminar

This is a year-long capstone course in which statistics majors demonstrate that they have met the discipline’s learning objectives. There is a three-fold assessment of

· student learning of basic statistical concepts

· [the] student’s ability to carry out research

· [the] student’s ability to communicate findings

The vehicle for this is the presentation of a seminar on a statistical topic, which, besides the expected research by the student, entails weekly meetings with the faculty supervisor, interviews and oral exams. The seminar, with its research and presentation components, is evaluated by statistics faculty, faculty from other disciplines, other senior seminar students, and external individuals related to the project. These evaluations are analyzed statistically.

5.2 E-portfolio

Statistics majors keep a University of Minnesota E-Portfolio, which generates an “individualized student learning profile.”[10] The profile characterizes students before enrollment at UMM, tracks their development as statisticians at UMM, and maintains a record of their professional lives after UMM.

6. Assessment driven actions

6.1 Driven by the need for effective communication

Past assessments “showed our students lacked the ability to communicate their findings correctly and effectively by using simple words that can be understood by non-statisticians,”[11] those in question being both general education students and majors. The discipline’s response was to implement the Media Reports Project[12] in conjunction with UMM’s Center for Small Towns and UMM’s External Relations unit.

6.2 Driven by the capstone course assessment

The discipline has placed greater emphasis on the theory of statistics in higher level courses, started the capstone project earlier, increased coverage of some topics, and redesigned two courses. It is seeking ways to enhance student learning in the areas of critical and independent thinking.

6.3 Driven by the retention of student learning study

The discipline is just completing the statistical analysis of data from this new initiative.

6.4 Driven by the Technology Enhanced Learning survey

“The discipline applied and received a grant to create a vertically and horizontally integrated technology enhanced learning environment...The project aims to respond to diverse ways of learning.”[13]

7. Improving student learning

The most recent assessment identifies nine positive and three negative findings. The discipline has used its accumulated findings to compare earlier and recent statistics majors, 2003 being the dividing year. “It is hypothesized that the second [recent] stage would reflect the changes made based on the findings of the assessment of student learning process.”[14] A classification and regression tree analysis indicates improved student learning especially in communicating statistical ideas effectively. There was no statistically significant change in the critical thinking and independent thinking domains of the learning objectives.

General education categories spanned by the discipline

Statistics courses all bear the M/SR, mathematics/symbolic reasoning, general education designator with the exception of directed study, which bears none.

Studio Art Discipline Assessment 2006-2007

Scope of assessment activities

___⇒__Course-embedded assessment

___⇒___ Pre- and post-testing

___⇒_ Outside the classroom

___⇒_ Across the discipline

Direct measures of student learning

___⇒__ Capstone experience

___⇒__ Portfolio assessment

______ Standardized tests

______ Performance on national licensure, certification or

preprofessional exams

___⇒__ Qualitative internal and external juried review of

of comprehensive senior projects

___⇒__ Externally reviewed exhibitions and performances in

the arts

______ External evaluation of performance during internships

Discussion and Description

Discipline goals, direct measures, and improved student learning

1. Studio art discipline goals

· Students will demonstrate a mastery of fundamental principles, formal strategies and technical skills in a variety of media and approaches to their use, as well as an understanding of relevant contemporary conceptual issues in the visual arts. This includes materials, techniques, the safe use of tools (for example, everything from paint brushes, potters tools, wheels, kilns, carpentry tools, and power tools, to printmaking presses and equipment), and the safe disposal of waste.

· Students will demonstrate a mastery of the skills of critical analysis of works of art and communication skills necessary for activities in the visual arts; this includes the ability to talk clearly, independently and thoughtfully about their own art as well as the art of others.

· Students will demonstrate a mastery of fundamental principles, formal strategies and skills in a variety of drawing, as well as an understanding of relevant traditional and contemporary conceptual issues in the medium.

· Students will demonstrate formal and conceptual competence in at least two disciplines in the studio arts, taking a one- and two-year sequence in two chosen media.

· Students will demonstrate knowledge of the major traditions and the cultural significance of the visual arts, an understanding of the historical and contemporary development of art and their place in it, and the relationship of art to self, culture, and society.

2. Learning objectives and course work

The assessment plan relates learning objectives to the studio art and art history courses where they will be met.

3. Portfolios and course-embedded assessment

The body of work produced by a student in a course is called the portfolio, which may then be graded. Course-embedded assessment relies heavily on critiques by the instructor alone, and by the instructor and class members together, with ongoing critiques being made as the portfolio grows in size from initial work to the entire body of work.

4. Portfolios, assessment across the discipline, the junior and senior reviews

The portfolio for junior and senior reviews is a selection of work from completed and in-progress courses, making its assessment an assessment across the discipline. The student provides an artist statement for use by the review committee, which is comprised of studio art and art history faculty. The committee uses a review sheet to rank the work from 1-10 in nine different categories under the three broad headings of formal concerns, technical concerns, and conceptual and communication skills,[15] and also provides written comments. Results of the review are given to the student, academic advisor, and discipline coordinator. The introduction of a uniform and consistent method of evaluating the junior and senior reviews dates back to a 2003 assessment.

5. Pre- and post-testing

Pre- and post-testing occurs in drawing classes for both majors and non-majors. Comparison of a drawing from the first day of instruction with a final drawing allows the faculty member to assess student improvement. Faculty member and student discuss the drawings.

6. Outside the classroom, outside juror

The studio art discipline has video and digital images dating from 1997 of the senior exhibit and all-student shows. This archived material is useful to students preparing exhibits. Since 2006 the discipline has used an outside juror to select works and write a statement for the annual show.

7. Other course-embedded assessments and learning activities

Quizzes, sketchbook exercises, response papers, class presentations, student-led discussions, group projects, and collaborative activities are used variously to assess the degree to which students have attained the discipline’s learning objectives. Print exchanges with other universities allow the work of UMM printmakers to be compared with that done at the regional and national level.

8. Assessment, improving student learning and the new capstone course

Discipline assessment of learning objectives since 2003 has revealed four areas of concern that are being addressed.

· Students on average do better in the junior review than in the senior.[16] In response, the faculty has made the junior into a second year portfolio review, and integrated the senior review into the new capstone course, the senior art thesis.

· Students need more writing in the arts. The new capstone course has a writing component and there will be more writing in Basic Studio Drawing II.

· Students need more experience with framing and other exhibition skills. These experiences are part of the course description of the new capstone course.

· Students requested a major or minor emphasis in areas such as photography/digital imaging, drawing, and ceramics. These areas were added to the major in the spring of 2006.

General education categories spanned by the discipline

Studio art courses all bear the ArtP, artistic performance, general education designator with the exception of a few courses bearing none (directed study, senior review, senior exhibit, senior thesis project).

Theatre Arts Discipline Assessment 2006-2007

Scope of assessment activities

___⇒__Course-embedded assessment

___⇒___ Pre- and post-testing

___⇒__ Outside the classroom

___⇒__ Across the discipline

Direct measures of student learning

___⇒__ Capstone experience

___⇒__ Portfolio assessment

______ Standardized tests

______ Performance on national licensure, certification or

preprofessional exams

______ Qualitative internal and external juried review of

of comprehensive senior projects

___⇒__ Externally reviewed exhibitions and performances in

the arts

______ External evaluation of performance during internships

Discussion and Description

Discipline goals, direct measures, and improved student learning

1. Theatre Arts discipline learning objectives

· To provide a fundamental knowledge of the art, discipline, techniques and history of the theatre.

· To develop the ability to produce good theatre.

· To develop an appreciation of quality theatre.

2. Capstone course: Senior Project.

In this course, the student demonstrates competence in some area of theatre arts. The project might be completed independently, for example through a research paper or solo acting performance, or as part of a group effort. Acting, scenery, lighting, costume design, playwriting, and theatre history are some of the areas in which the project may be undertaken. All faculty are involved in the assessment of each project.

3. Portfolio assessment.

The portfolio comprises all of the theatre work of significant value that students have done during their career at UMM, including items such as photographs, articles, notes, and performance and design projects. During their third year, students present their portfolios at the weekly meeting of Theatre Arts majors for faculty evaluation. They continue adding to their portfolios and take them out into the world of theatre for interviewing.

4. Course-embedded assessment. Pre-test/post-test.

Stagecraft. The course objective is for students to gain “knowledge and appreciation of the history, theory, tools, materials and techniques employed in the construction, painting, and shifting of stage scenery.”[17] The pre-test is a multiple-choice exam administered at the beginning of the course. From the results the instructor determines which topics require particular emphasis and need particular attention in pre-exam reviews. An unexpected benefit was that students realized at the very beginning of the class what the instructor held to be important. The post-test was a multiple-choice final exam similar but not identical to the pre-test. The pre- and post-test averages were 44.5 and 85.9. No one failed the post-test.

5. External review of performance.

Theatre Arts students have received recognition in externally reviewed performances. Most productions are assessed by outside evaluators. Some productions are evaluated by members of the audience immediately after the performance.

General education categories spanned by the discipline

Almost all Theater Arts courses carry one of the following general education designators: ArtP, artistic performance; FA, fine arts; or Hum, communication, language, literature, and philosophy. Directed study, Backstage on Broadway, London Theater Tour, and Senior Project carry no general education designator.

[1] The author of the discipline report notes that “ratings were remarkably similar among faculty.” The author of this report notes that ratings were generally stronger in listening and reading than in speaking and writing.

[3] The various parameters that may have influenced the outcomes are discussed in the discipline report.

[4] <http://www.morris.umn.edu/committees/asl/results/results.html>

[5] Details are in the speech communication’s discipline report in the appendices.

[8] The full list is on page 18 of the discipline’s report, which is in the appendices.

[9] Pp. 17-18 of the statistics discipline’s assessment report.

[15] The report in the appendices has biannual data for junior and senior reviews beginning in the fall of 2003 through the fall of 2006.

[16] Discussion abridged. See the assessment report in the appendices for more details.

[17] The quote is from the Stagecraft Assessment Report in the appendices.