English Discipline, UMM

Assessment of Student Learning for 2008-09

Introduction:

The English discipline is in the midst of revising its curriculum and developing a comprehensive assessment plan. We have begun implementing this plan by focusing our 2008-09 assessment on the English major’s bookend courses: Introduction to Literature (Engl 1131) and the 4000-level Research Seminar.

Learning Goals for Majors

Note: These goals have been revised since the 2007-08 Assessment Report.

Upon successfully completing their English degree at UMM, English majors should be able to do the following:

(1) Read and discuss in an analytical fashion both primary and secondary texts

(2) Write a coherent argument, both with and without secondary sources

(3) Demonstrate a sophisticated understanding of the English language

(4) Demonstrate basic knowledge of critical approaches to literary study

(5) Demonstrate basic knowledge of literary history

(6) Conduct basic scholarly research and present it in a professional, scholarly setting

(7) Demonstrate a solid foundation for a lifelong appreciation of literature

These goals will be placed on the discipline website [http://www.morris.umn.edu/academic/english/] in order to be readily available to current students, prospective students, and faculty. We will also make sure that they are included in future revisions of the catalogue. Finally, individual faculty members have been encouraged to include pertinent goals (or versions of them) on their syllabi.

Assessment Measures:

The attached Assessment Plan explains the English discipline’s comprehensive plan for assessment (as it has been developed so far) and contains a rubric indicating the multiple ways in which each goal and its associated outcomes are measured within individual courses. This assessment report, however, focuses on programmatic assessment, rather than individual course assessment, since the latter is done by individual faculty members.

For the 2008-09 year, we took a much more multi-pronged approach to assessment of the English discipline than we have done in recent years, combining both direct and indirect assessment measures:

• Checking course coverage of all learning goals (direct measure). With reference to the rubric on the Assessment Plan, we check that all discipline goals are met by the current curriculum. As we make changes to the major curriculum, we update this chart.

• Evaluating sample essays from bookend courses (direct measure). We assess a few components of our program every year but rotate the aspects that we focus on. This year’s focus was the bookend courses: Engl 1131 Introduction to Literature and the 4000-level Research Seminar. To assess these courses, at the end of each semester we collect sample essays from each course that should reflect the students’ best ability to execute the primary skills taught in the course; this means that the sample essays tend to be end-of-term work, though they certainly do not have to be. We collect “average” samples and “above average” samples from each course in order to determine whether the work demonstrates that 1) the goals for these courses are being achieved and 2) the quality of student work is improving from Engl 1131 to the 4000-level seminar. Furthermore, since these courses teach and encourage practice in the skills most essential to the work of English majors, assessment of these courses should demonstrate whether or not most of our learning goals are being successfully achieved. Assessment details based on this measure are as follows:

Introduction to Literature:

• The sample included:
• 6 “average” papers from 2007-2009 from classes taught by 3 faculty members:
• 1 character comparison
• 1 refutation
• 4 analytical papers with secondary criticism
• 6 “above average” papers from 2007-2009 from classes taught by 3 faculty members:
• 1 character comparison
• 1 close analysis without secondary criticism
• 4 analytical papers with secondary criticism

• Based on this sample, students are clearly learning how to do the following:
• state an analytical thesis, even if banal
• analyze specific quotations to support a point
• demonstrate awareness of ongoing critical conversations around texts, even if vague on the details
• cite texts (at the very minimum, they demonstrate awareness that they need to cite texts and that there are conventions for doing so, though the mechanics are not entirely consistent or accurate)

• However, based on this sample, students are not as effectively learning how to do the following:
• carry the argument throughout the entire paper (i.e. develop the thesis)
• foreground their own ideas rather than plot summary (i.e. produce effective topic sentences)
• produce effective analytical paragraphs
• provide close analysis of language, especially of primary texts
• use enough quotes from literary texts (as opposed to secondary texts)
• practice analytical methods unique to literary study (i.e. close reading; analysis through vocabulary, imagery, etc.) (as opposed to responding to and working with secondary texts)

• Assessment of these papers raises a few critical questions for our discipline:
• How much secondary research and/or use of secondary texts is appropriate at this level?
• Should more effective paragraph development be a determining factor in assigning a “C” grade (i.e. inadequate paragraph development means less than a “C”)?
• Is there a consensus among faculty about what this class should accomplish?
• Is there a consensus among faculty about what we are trying to accomplish at each curricular level?

4000-Level Research Seminars:

• The sample included:
• 2 “average” papers from 2007-2009 from classes taught by 2 faculty members
• 4 “above average” papers from 2007-2009 from classes taught by 3 faculty members

• Assessment of these papers suggests that (1) faculty members are in general agreement about what this course should accomplish and (2) our students are achieving the skills taught in this class, including the ability to:
• make a complex and original argument, even if not perfectly executed
• use critical and theoretical terms and concepts or methods in a productive, informed way
• situate an argument in a critical context/conversation rather than “using” or cherry-picking sources
• demonstrate a nuanced understanding of a topic and the critical conversation around a text, group of texts, or movement
• integrate other voices with their own with relative ease

Conclusions Based on Bookend Assessment:

• The term “adequate” would be a more useful measure than “average” for the papers that meet minimum standards.
• We need to collect sample papers from a greater number of professors, ideally from every section of Intro to Lit and the Research Seminars.
• We need a methods class to convey the basic conventions, discourse, and rules of literary study. The question is exactly what form this course should take: Intro to Lit in its current form, Intro to Lit + another course, or some other model.
• We need to decide, as a discipline, exactly what we want covered in Intro to Lit and what general approach we want to take to teaching it. For example, do we want Intro to Lit to convey the basics of literary study by teaching terms in the context of a limited number of texts and expect that students will learn other skills and terms throughout the major? One way to think of this approach is as a “Seminar Jr.” Or do we want Intro to Lit to convey the basics of literary study by teaching as many terms and skills as possible with less depth and expect students will develop comfort and expertise with these skills later in their coursework? Both can work, but there is currently too much disparity in the kinds of assignments students in Intro are being asked to write. This disparity links directly to the larger discussions we are having about the structure of our major. In fact, after we make this decision about Intro to Lit, we will more effectively be able to decide where the surveys fit, among other things.

• Meeting informally with English majors (indirect measure). Early in the spring semester, we had an informal meeting with English majors to discuss their experience of the major and to gauge their interest in some of the discipline changes we have been discussing. We came away from that meeting with the following impressions:
• Students would like more drama courses.
• Students would like a Bible as Literature course.
• Students do not seem to be as concerned as some faculty members are about the mix of lower-level and upper-level students in 1000- and 2000-level courses. On the contrary, many students believe this is a beneficial situation.

• Administering the annual exit survey to graduating seniors (indirect measure). This year we received 17 completed surveys, a much higher return rate than last year’s nine surveys. This year’s surveys suggest the following:
• few of our majors plan to go to graduate school, and even fewer go immediately after graduation from UMM (none of the 17 who turned in surveys)
• few of our majors actually have jobs when they graduate from UMM
• most of our majors look to careers in writing or publishing/editing
• at least six of our majors look to careers in entirely different fields
• all of our majors believe that the English major has given them a solid foundation in the study of literature
• most of our majors believe that the variety of course offerings allowed them to pursue their intellectual interests
• most of our majors believe that the required courses at lower levels effectively prepared them for more advanced coursework
• most of our majors believe that the 4000-level seminar was well-designed as a capstone course
• all of our majors believe that the major challenged them to develop analytical skills and to think critically
• all of our majors believe that the major helped them to communicate more effectively, both orally and in writing
• many of our students would like a greater variety of course offerings, especially at the advanced levels
• only a few of our majors participate in research projects or present research outside of the classroom

• Discussing possibilities for restructuring the major (indirect measure). Continued work on our new discipline assessment plan, assessment findings from the 2007-2008 Report, and an increased concern about how to increase enrollment in our courses at all levels while meeting the needs of both majors and non-majors led to a wide-ranging discussion about revising the English major that will continue at least into next year. To more efficiently manage this discussion, which will have far-reaching consequences for both students and faculty, we created three subgroups to discuss (1) the writing requirement, (2) 3000-level courses, and (3) non-major courses. By the end of the year, the latter two groups had reported back to the whole discipline. Many of our concerns revolve around Engl 1131, which is currently serving two very different functions as the foundational course for English majors and the primary course taken by non-majors to fulfill General Education requirements. There are also many differences among faculty members about exactly what should be taught—and to what extent—in this course. By the end of the year, we had determined that aspects of Engl 1131 that seem to work include the three-genre approach and introducing students to theoretical approaches. However, some faculty members are concerned that too many non-majors enroll in the course and that Engl 1131 should not be the first-choice English course for non-majors (though it should certainly be open to those who are interested). In terms of the research seminar (our other focus for this year’s assessment), all agree that the course is functioning well. However, there is a concern that, in general, English majors do not understand the importance of using correct MLA format, even at the seminar level, and that we could do a better job in the seminar of articulating the role of the final presentation. There was also a brief discussion about whether the seminar as it is currently structured is an appropriate capstone for all of our majors, especially those who do not intend to go to graduate school. Although our discussion about restructuring the major will continue during the 2009-2010 academic year, our conversations have already led to many decisions (detailed below) that will be implemented as soon as possible.

• Collecting reflection papers from students (indirect measure). One faculty member collected reflection papers from students enrolled in Engl 1131 during the Spring 2009 term in order to gauge the students’ impressions of the class. Although this set of papers addresses only one professor’s class, it suggests that the course is fulfilling its major objectives. Students note an improved ability to think critically, analyze texts, write analytical essays, use correct terminology when talking about literary texts, scan poems, and look at texts from a variety of critical perspectives, in addition to an increased respect for writers, especially poets.

Based on all of these measures, we believe that our major is accomplishing our basic goals. That said, we also believe that structural changes in our major may allow us to do an even better job of meeting the needs of our students, both English majors and non-majors, and that coming to firm conclusions about the precise nature of Engl 1131 will allow us to move forward with this restructuring.

Examples of Changes Based on Assessment:

Note: Some of these changes were also mentioned in the 2007-08 Assessment Report submitted earlier this spring, since it included changes up to that point.

• As a result of continued work on our new discipline Assessment Plan, we reduced the number of learning goals for English majors and simplified the goals/outcomes/measures rubric in the Assessment Plan. These changes will enable more manageable assessment while still accurately representing discipline goals for the major.
• In response to a tightening budget, we increased the maximum enrollment in selected 2000-level courses.
• We will recommend a least a minor’s worth of English courses on the UMM campus in order for us to favorably recommend a student for teaching licensure.
• We added courses with a writing focus, including Topics in Writing: Editing and Proofreading (Engl 2121) and Sports Literature and Writing (Engl 2022). These additions help meet the needs of majors, who consistently ask for more writing courses and who often pursue careers in writing after they graduate.
• Given a range of concerns about College Writing (Engl 1011)—including impressions that students are not familiar enough with writing in other disciplines and that many students who test out of College Writing who would be better served by taking the course—we have begun a discussion about revising the writing requirement at UMM, and some faculty have already taken new approaches to College Writing. One example is a gender-focused College Writing course offered by Tisha Turk and Brook Miller during the Fall 2008 semester. This course offers a model for how we might create a variety of College Writing classes (perhaps with different course numbers) that will focus on different topics and/or more effectively foreground issues of disciplinarity.
• In an effort to offer a greater variety of courses and satisfy the curricular interests and needs of our students, we continue to request permission to hire an early American literature specialist, and we believe that we could use a drama specialist (especially given UMM students’ interest in theater and courses in dramatic literature).
• As a result of our year-long (and ongoing) discussion about restructuring the major, we made the following decisions.
• We will change the name of the foundational course for the major, Engl 1131, from “Introduction to Literature” to “Literary Study.” We will file the necessary paperwork for this change early in Fall 2009.
• We will revise the course description for Engl 1131 to emphasize the role of this course as a foundational course for English majors, one that introduces the fundamental skills of literary analysis, entails intensive practice of these skills, and generally prepares majors for the work they will do at more advanced levels of the major. This new description will not only convey more clearly our intentions for this course but also dissuade non-majors who are not interested in this kind of in-depth, skills-based course. Such students will have many other 2000-level options if they want to take a literature course.
• We will reconsider which courses we allow students to use to place out of our Engl 1131, such as College in the Schools versions of “Introduction to Literature.”
• We will require all students enrolled in Engl 1131 to buy a handbook of literary terms, and we will recommend that all students enrolled in the historical surveys buy one.
• We will develop a non-restrictive list of “Best Practices” for Engl 1131. There is tentative agreement that Engl 1131 will introduce students to the following:
• three basic genres
• literary theory
• basic literary terms useful for study of each of the three genres
• close reading
• meter
• plot
• character
• narration
• We will continue to brainstorm about how to create and/or offer more courses with broad coverage to non-majors at the 2000-level, especially given the findings of a follow-up study of UMM graduates (recently shared with faculty by Gary Donovan) that suggests that graduates do not perceive “broadened acquaintance with important literature” as a significant benefit of their UMM education.
• We will continue to discuss increasing the number of credits in the major in order to allow English majors to count more than one 2000-level class toward the major. This is clearly something our majors would like to do, but the ultimate decision will depend on how we restructure the major.
• We will revise the names of 3000-level courses to make them more appealing and—as much as possible—create greater variety in the 3000-level course offerings. One goal with these changes is to increase enrollment in 3000-level classes.
• We will introduce a World literature course into our curriculum, though we have not yet determined whether the course will be a 1000-level or 2000-level course.
• We will work harder to teach our majors the logic behind MLA and to convince them that following MLA conventions is important.
• We will continue to reduce the number of 4000-level seminars offered each year as necessary in order to accommodate lower enrollment numbers and allow faculty to teach other classes with more students. Recent experience suggests that 3 seminars per year (rather than 4) are enough.
• We will use the different course levels to more clearly denote types of classes (as associated skills) as follows:
• 1000-level: College Writing and the current 1131, courses that expose students to literary analysis, literary conventions, and critical approaches.
• 2000-level: Surveys of both literature and writing that provide breadth rather than depth in a historical period, genre, or topic. These shall include all current 2000-level literature and writing courses and possibly what is currently 1131.
• 3000-level: Courses that provide depth in a subject or author (rather than the breadth of 2000-level courses) and fall into one of four categories: Texts in Conversation, Identity, Author-based, and Writing & Language. How much students in these courses will engage in conversation with critics has not yet been determined. But courses should expose students to the conventions of scholarly conversation and provide practice in analyzing those conventions (such as with an annotated bibliography assignment).
• 4000-level: Research Seminars, which require students to practice actually executing the scholarly conversation—finding and reading sources and situating an argument in relation to them.

Recommendations for Improving Assessment Processes:

· Continue to develop the Assessment Plan (see attached document), with particular emphasis on General Education courses offered by the English discipline, including College Writing (Engl 1011).

· Encourage faculty to get in the habit of saving sample papers for each class, especially Engl 1131 and 4000-level seminars. We need more sample papers from more faculty members for the most effective assessment of bookend courses.

· Label sample papers for assessment purposes “adequate” (rather than “average”) and “above average.”

· Continue to encourage faculty to include the pertinent discipline goals that are met by a particular class on course syllabi.

· Designate an assessment meeting for the late spring semester (but not the last meeting of the year) and make sure that no other topics are discussed during this meeting.

· Continue to increase the return rate of senior exit surveys. Providing this survey by email (as we did at the very end of the spring semester) may help, as will asking the student representative to send a reminder email to graduating English majors.