Home » Education » NCATE » Standard 2 » Use of Data For Program Improvement

NCATE/BOT Accreditation Student learning...all students, all places

Use of Data For Program Improvement

The University of Minnesota, Morris, Teacher Education Program has collected data about candidates, courses, programs, and unit. Our goal is to understand our program strengths and growth areas to enhance continued improvement. The results of our multiple assessments indicate that our candidates are well prepared for their work as beginning teachers. As described earlier in Standard One, candidates meet—and often surpass—minimum requirements. Below is a summary of general findings for each of the key assessments.

  • GPA data overall and in content areas provide evidence that UMM candidates know and understand their subject matter. The average unit GPA for the past three years is 3.45 overall, 3.56 in subject matter specialties, and 3.71 in education courses.
  • Praxis I and Praxis II data also provide evidence that UMM candidates have mastered subject matter and professional and pedagogical skills. All but one of our current programs have 100% pass rate for the past four years. A high percentage of candidates receive Recognition of Excellence for their performance on Praxis II exams. We continue to work to improve candidate performance on the Spanish Productive Language exam.
  • Performance-based assessments (including summative evaluations of student teaching, reflective portfolio, and an analysis of student learning assignment) align to Minnesota Standards of Effective Practice and provide evidence of candidate proficiency in ten essential areas. A high percentage of candidates (usually 90-100%) are rated proficient or above by both cooperating teachers and university supervisors. The remaining candidates have met the standards, but at a minimum level. The analysis of student learning has yielded positive results for candidates’ ability to plan and implement an instructional unit with clear focus on assessment data collection, analysis, and use for increased student learning.
  • Follow-up surveys of graduates and employers are aligned to performance-based assessments and program standards and are administered in the first year of the graduate’s teaching experience. The three-year average response rate is 58% for graduates and 63% for employers. Survey results are generally favorable in all areas of preparation. Graduates provide information not only about preparation for teaching but also assess the benefits of specific courses or other program elements.

Formative Assessment

The faculty members of the UMM TEP believe in purposeful, meaningful, and timely evaluation of faculty and candidate performance. We have included a system of regular and intentional formative feedback designed to assist candidates in achieving high standards.

Candidates receive formative feedback as a regular component of program courses and field experiences. Formative assessments include course work, exams, cooperating teacher evaluations, and course evaluations. This assessment data allows faculty to adjust instruction to meet candidate needs. Faculty members can also identify specific growth areas for candidates. Most candidates use the frequent feedback to reach high levels of performance. Because they must master each task within their courses, some candidates also use feedback to improve assignments until they can demonstrate proficiency. One representative sample of this process is shown in the 2007 Program Report and supporting data to the UMM Assessment of Student Learning committee. In field and clinical experiences, frequent observations and formative midterm evaluations provide feedback that candidates can use to improve performance. Candidates are also updated on their performance as monitored in APAS every semester. This too assists faculty and candidates in planning and goal setting.

Professional education faculty and clinical faculty use the assessment data to improve the performance of candidates as well as their own. Based on formative assessments within courses and in the field experience, faculty members alter or extend instruction, reteach important concepts, structure feedback to meet individual needs, and assist candidates in goal setting and attainment. Faculty members use the data not only to work with individual candidates but also to improve instruction and supervision for future candidates.

Attention to Data

Because the UMM TEP is small and personalized, the teacher education faculty is able to maintain continuous discussion and attention to data. Evaluation is ongoing and inclusive. The unit is in fact an assessment committee of the whole. As a unit, we discuss our system, the assessment measures, data results, and the implications of the data. Based on these discussions, we have adapted assignments and rubrics, held reliability sessions prior to scoring shared assignments, created new assessment measures, and made changes to courses. We also consult with the two members of our support staff about ways to improve the collection, storage, and dissemination of the data. The support personnel are critical to the success of our system evaluation.

Formative data serve multiple purposes under the large heading of candidate and program improvement. Summative data, including course grades and evaluations by cooperating teachers, are examined at certain checkpoints in the program to determine candidate advancement. Complete files of all candidates are compiled and retained. Evidence from formative and summative assessments, along with confirming evidence of candidate performance after graduation, is used to examine programmatic success and revisions. Candidate evaluations of courses and post-graduate surveys also are used to revise and improve the program.

Formal attention to summarized data is also a part of our assessment system. Program faculty members analyze data that are unique to their candidates and use it for program improvement. The unit also specifically analyzes summarized data from key assessments, especially the assessments that are shared across programs. These discussions involve changes or improvements that affect elementary and secondary education programs alike.

Data-Driven Decisions

  • Increased Focus on Content

  • The unit has a high pass rate on the required standardized test. Two areas have been the most problematic for our candidates. Though too few students took the test for us to include it in reports, we knew that many of our elementary candidates pursuing middle level social studies were not successfully passing the test. This failure to meet the requirements of the second license meant that the candidates could only receive a three-year temporary license and had to fulfill the requirements to be fully licensed after that period of time. This problem was not exclusive to UMM; the state pass rate was 84% in 2007. We began to work diligently to increase the subject matter experiences and understandings for this group. In doing so, we increased content knowledge preparation in all of the specialty areas. Middle level and preprimary specialties now require 24 credits of course work in the content area—eight additional credits. We also introduced content-specific instruction within the program. For example, in the middle level theory course, all candidates are required to analyze a middle level curriculum in their area and relate it to all of the content standards required by the Minnesota BOT. This allowed candidates to organize their information and to identify areas of weakness that they would need to address with further study. The recent pass rates for elementary education 5-8 social studies have been higher and the four-year pass rate (2005-2008) is 100%. The productive language exam in Spanish remains an area of concern. Our 62% pass rate (2005-2008) is not acceptable for us. We understand that our scores are affected by the small number of test takers. Our instructor for the foreign language methods course along with representatives from the Spanish discipline attended a meeting of state professionals to learn more about the Praxis II exam for Spanish and what might be causing the problem. We have purchased study materials which are readily available for candidate use and are working to explore other ideas as well.
  • For the 5-8 middle level social studies and the K-12 Spanish candidates, we have also worked to give them advanced information about the test, expectations, and ideas to improve their knowledge. We have strongly encouraged them to take extra course work and do extra reading to build their background. The Spanish students are strongly encouraged to study abroad for an extended period of time. We strive for 100% pass rate in all programs.
  • Revised Schedule of Courses in Secondary Education

  • Based on solicited feedback from cooperating teachers and candidates, the secondary education program revised its schedule of instruction to allow candidates to have consecutive days in the practicum to build continuity and coherence to the experience. This has resulted in improved opportunity for the candidates to teach a series of lessons. In addition to improving educational opportunity for the candidates, it has made supervision and planning easier for our school partners.
  • Analysis of Student Learning

  • We have worked diligently to incorporate the analysis of student learning into the clinical experience. The process has been a part of advanced field experiences and course work since 2002, and we knew from observations and discussions that candidates were formally and systematically analyzing assessment data during student teaching. In 2007, we required candidates to complete the analysis during student teaching and report on it in their professional development course, ElEd/SeEd 4901. The scores on the assessment were disappointing. Based on that information, we clarified the assignment and expectations, scheduled it for early in the clinical experience, improved the scoring rubric, and implemented a feedback schedule. Candidates were required to amend or redo parts of the assignment until they met minimum requirements. We were much more satisfied with the results and also surveyed the candidates to gather their feedback on the assignment. Most reported that they valued the assignment (though it was difficult) and many stated that they had learned how to really look at data and use it for improvement. They asked for clearer directions, and we will be addressing that for the 2009 clinical experience.
  • Increased Expectation for Connections to Community

  • Community and family relationships are key elements of the knowledge and disposition goals in our Conceptual Framework. Our candidates have typically received high ratings on our assessments of Minnesota Standards of Effective Practice 9 Reflection and Professional Development and 10 Collaboration, Ethics, and Relationships. In a recent analysis of a sample of the disposition assessment, we learned that though candidates were overwhelmingly successful in demonstrating the desired and required professional dispositions, a problem area for a few candidates was in initiating contact with families and working with other professionals in the school. We believed that this problem was in part related to the structure of the experience and whether or not these opportunities presented themselves. We are working to add specific instruction and related assignments that will assist our candidates in being able to initiate contact and communication.

Data Dissemination

Candidates receive formative feedback regularly as part of all courses and field experiences. Every semester, they receive progress reports to indicate any problems with progress toward graduation or licensure requirements. Candidate status and data is reported to program faculty who also update faculty colleagues with candidate reports during weekly meetings. Unit data is shared annually after it has been collected and summarized. Other stakeholders have received information in informal meetings and at meetings for the teacher education advisory council. The results of the 2007 teacher education survey were shared with school partners at the school faculty meetings. Results of this and other surveys are scheduled to be posted on our website, but this is still a goal for us. We report the results of key assessments to the UMM Assessment of Student Learning committee.





' >