Assessment of Student Learning

Assessment of Student Learning

University of Minnesota, Morris

DATE:          Monday, October 29, 2001

SUBJECT:      Assessment of Student Learning Committee Minutes

PRESENT:     Dian Lopez (Chair), Edith Borchardt, Tom Johnson, Tim O'Keefe, Nancy Mooney, Tim Soderberg, Nick Maxwell

EXCUSED:     Michelle Page

ABSENT:      Rian E. Lenzmeier

Lopez called the meeting to order at 3:00 PM in the Prairie Lounge. She asked if there were any changes or additions to the agenda. Lopez said she would like to add one item, Advising Assessment Tools, to the agenda for discussion. Motion made by Johnson to accept the addition to the agenda, seconded by O'Keefe and unanimously accepted by the committee.

Lopez asked for corrections or approval of the ASL Minutes 9/17/01. The minutes were unanimously approved.

Web Page Design
Lopez praised Mooney and Pederson for their work on the ASL web pages. Mooney displayed the web pages and asked for discussion as the pages were viewed. Mooney noted that the previous web pages had a very busy background; the background has been changed to a pale yellow. She said she had met with Pederson earlier to discuss the original pages and they decided that the pages on the original main page of the ASL web page should either be kept as a current link or be put on a historical link. Mooney noted that some of the links were broken and, therefore, eliminated. Johnson questioned who is expected to access the ASL web site, noting that this would lead to different decisions of where the information should be linked. Lopez said that the committee should decide on the design, and said that students, faculty and staff, administrators, other campuses, and anyone else interested in assessment would probably want to use the site.

Mooney said that the idea to update the ASL web page was implemented because Dean Schwaller had requested changes to make it more appealing and less confusing. As Mooney displayed the web pages, the committee members discussed the links, making some language changes to the links as well as eliminating duplicate links. Johnson suggested that dates should be added, when possible, and that links should be ordered by dates. Mooney asked if the GenEd Assessment site should be linked, noting that there were many negative comments on the survey. Lopez said that we should find out how many comments were negative; a few are normal, many indicate some problem we probably should address. Mooney said she will locate the comments on the survey and an e-mail discussion could follow. O'Keefe said that there is no obligation to make the survey accessible. Mooney asked the committee to authorize her to upload the new web pages after the corrections have been made. O'Keefe said the new pages are superior to the previous web pages and made a motion to replace the old web pages with the new web pages after the corrections are made. The motion was seconded by Maxwell, and unanimously carried.

GenEd Survey (Mooney, O'Keefe)
Mooney reported that they had 82 non-duplicated surveys converted from e-mail to an Excel spreadsheet (approximately 60 pages long). Mooney noted that problems with the survey were encountered; coding problems resulted in missing 1/3 of the responses. Mooney and O'Keefe will analyze the information that was received. Mooney said that Institutional Research and Reporting, Twin Cities, offers institutional web and survey services and asked if the committee had interest in using it. It was agreed that Mooney should check into the cost of this service.

O'Keefe said we should begin now working on the GenEd survey. The committee discussed improvements to the survey. Some suggestions for improving the survey were:

O'Keefe said that the data was pretty messy because of the coding problem. It was suggested that a couple of interesting and informative answers from the qualitative questions should be pulled, and the results presented to the Curriculum Committee, the CRPC, and the Dean. After the results are completed, the UMM community will be able to access the results by a URL that will be provided. O'Keefe said we should consider writing an executive summary of the survey. Lopez said that the survey results would be able to be compared from year to year. Lopez asked O'Keefe and Mooney to analyze the results and bring ideas back to the committee on if and how the survey will be changed.

Unit Assessment (Soderberg, Page, Lopez)
Lopez distributed three documents to committee members which included the draft of General Discussion Points for Disciplines when considering assessment, Assessment of Freshman Advising in Fall semester 2000 and in Fall semester 1999, and a new survey for Advising in the Major for discussion.

The committee first began discussing the document draft pertaining to the General Discussion Points for Disciplines when considering assessment of their research/seminar/capstone course. Lopez said that this is the first draft of the document and changes will need to be made. Lopez said that many disciplines are doing Unit Assessment now and are doing it well. Lopez reported that there were only two disciplines, which had not replied to her memo, and she will speak with their coordinators personally. Members discussed changes to the document. Lopez proposed a memo be sent out asking each Discipline to do assessment on one research/seminar/capstone course. She has recently attended two assessment workshops and it was stressed by NCA reviewers in attendance that institutions should start small. They should do one thing well and build from there. If UMM could assess their research/seminar/capstone courses campus wide, this could give very useful data for assessment. Johnson and O'Keefe suggested taking this idea to the Campus Assembly for input and information purposes. The Campus Assembly meets next Monday, November 5. Lopez agreed to bring this to the Assembly meeting. Lopez said that she will meet with Soderberg and Page to discuss the recommended changes and will bring it back to the committee via email. It will then be ready to send to facpa.

The committee discussed the Freshman Advising Questionnaire developed by an Advising subcommittee of the Scholastic Committee chaired by Karla Klinger. Results are available from surveys given Fall 1999 and Fall 2000. The questionnaire targets freshman and is taken in the First Year Seminar courses. Lopez said that there were some good results showing improvements between the two years. O'Keefe noted (Question #4) that the difference in language could have made a difference in the results.

Lopez then presented a proposed survey for assessment of Advising in the Major. Some questions, suggestions and concerns:

Lopez said she would take these comments back to Karla. Johnson stressed that this questionnaire is valuable for assessment. Our committee should be able to support this survey when it is brought before Assembly.

Web-based course evaluation forms
John Bowers and Dean Schwaller attended the meeting to discuss web-based course evaluation forms. Dean Schwaller began by discussing the Student Opinion of Teaching survey last year. He said he has concerns with the process of how the current survey operates. He said that at present, 1) it takes one person approximately one month to enter all the data; he feels it is not the best way to handle this task because of possible errors and workload issues as well as confidentiality. 2) The complete questionnaire goes back to the faculty, and he said that handwriting could be recognized.

Dean Schwaller said that there are a couple of options to do the student evaluations better. One option is to use scanable forms; this would require the purchase of additional equipment which would cost about $8,000, mechanizing the process, and one staff person would need to be trained for the process. The other option is a web-based form. It was noted that using a web based form:

  1. students can fill out the form on their own time
  2. it could be available on the web at a certain time like the last few weeks of the semester.
  3. Students would sign in using their X500 number, insuring that the student could vote only once.
  4. it is more confidential
  5. it is less likely for faculty to distinguish who did the survey
  6. Individual comments will be associated with the responses.
  7. there would be some expense involved

Bowers projected the electronic UMM Student Opinion of Teaching Form for review and discussion by committee members. He noted that this is the first draft of the web-based form. Some of the following questions were asked:

  1. O'Keefe questioned if the survey is accessible by student ID, is it possible to have a preprinted form of the courses that the students are enrolled in. Bowers replied that the X500 does not know the courses students are signed up for.
  2. Mooney questioned if the numbers could be reversed on the scales, positive numbers first instead of negative.
  3. What would be the response rate doing the survey on-line versus in class?
  4. Is it possible to require the form to be filled out before the student receives their grade? Dean Schwaller didn't think the institution would like to make it mandatory.
  5. What is the time window of the survey?

Dean Schwaller asked the committee if they had endorsement from the committee to move forward with a trial survey. He said that the questions would remain the same, it would be the same printout as before, and that the faculty member would have access to the printout. Johnson said that a procedure paragraph should be added at the beginning of the survey. It was felt that the electronic survey would only improve the comments of students. Students would put more thought into their responses, whereas, in class-time they may rush their answers. Johnson noted that there are a number of faculty teaching classes this fall with about 30 students; he thought these would be ideal classes to try out the electronic survey. These classes are taught each fall and the results could be compared with past years. Johnson noted that there is no reason not to try the survey; he suggested that the survey be administered a couple of times this semester and then they could come back to the committee with a proposal. O'Keefe voiced concerns with the number of surveys that are given to students. Mooney said that people are supposed to come to her with surveys that they would like to administer, but said many do not. O'Keefe said that there should be more control over surveys. Lopez summed the discussion as meaning that the ASL committee would like to proceed with a trial survey and the committee could then discuss the results.

Meeting adjourned at 5:15 PM.

Respectfully submitted,
Linda Pederson