Report of Assessment Results 2002-2003:

Education

 

This summary is divided into two sections: results for general program assessment and results for the Education capstone course (ElEd/SeEd 4901). The capstone is a shared course, wherein elementary education majors and secondary education certification students are mixed together in course groups.

 

I.               General Program Results

 

The goals for Elementary and Secondary Education, as stated in the unit assessment plan, revolve around the ten Standards of Effective Practice set forth by the state of Minnesota. A summary of these goals is as follows: The education programs are designed to help students (future teachers) to

·      acquire the knowledge, skills, and dispositions necessary to being a competent teacher;

·      understand central concepts, tools of inquiry, and structures of disciplines taught in schools;

·      understand children and adolescents and their individual and group behavior;

·      plan and implement instruction adapted to learners of diverse backgrounds and abilities;

·      communicate effectively;

·      encourage critical thinking and problem solving;

·      use formal and informal methods of assessment;

·      collaborate with parents/guardians, families, school colleagues, and the community in an ethical manner.

 

Results pertaining to the goals listed above were obtained through a variety of assessment data. Whether goals were met was determined through analysis of the following data items: scores on the PRAXIS II exams of content and pedagogy; final evaluation scores on field experiences (each goal is evaluated on this assessment); and course grades. In addition, every year the program surveys its graduates in an effort to determine what types of programmatic changes should be made. Data is not yet available from the latest survey and so will not be discussed in this report.

 

PRAXIS II exams of content knowledge and pedagogical knowledge are required by the state and form one data point of external review. Scores for 3 students of 68 were unavailable at the time of this report. All other students have passed all parts of the PRAXIS. In addition, UMM students generally surpass the minimum cutoff scores established by the state (see Table 1). The minimum cutoff scores for the state of Minnesota on PRAXIS I reading, math, and writing sections are 173, 171, and 172, respectively. The average scores of UMM education students are 199.06, 199.94, and 195.16. UMM students also surpassed the minimum cutoff scores for their various content areas by an average of 32 points. In the Pedagogy, Learning and Teaching exam, students surpassed the cutoff by 26.5 points. This set of data revealed to us that UMM teacher candidates do well on state measures and that they are well prepared for this part of the licensure process. However, students of color consistently receive lower scores on the PRAXIS exam than their European American counterparts. This is not surprising to us, as a large body of research has demonstrated that this gap is consistent across test types and age groups. As educators, we are concerned about this and this data will cause us to consider changes to the programs based on further assessment. We will ask: How can we better prepare and support teacher candidates of color? Is there one part of the testing process that proves to be consistently more difficult for candidates of color? How can we be more active in the state governmental process that mandates these tests and lobby for more fair and equitable testing practices?

 

Data from final field experience evaluations reveals that all students met minimum proficiency for licensure. No student received less than an “average” rating on any part of their field experience evaluation from cooperating teachers or university supervisors. We judge this to mean that all students are performing adequately in terms of field experience. One future assessment goal might include analyzing the categories of critique on the final evaluations to determine if there are specific areas that are weaker than others on a consistent basis.

 

Finally, course grades provide valuable data about student learning in courses. Each course is crafted around the ten Standards of Effective Practice and is founded on the idea of mastery learning. This means that students perform tasks to a proficient level and if the student does not successfully master a task, he or she continues to work on this task and repeat an assignment until it is mastered. Therefore, there were no grades of less than B- in courses. All students were given the opportunity to master learning objectives and did so successfully, although at different rates and different levels of proficiency. Some students who struggled self-selected out of the program at various points in the process. Additionally, formal and informal data on course performance did indicate that a few particular students struggled to meet minimum standards of proficiency throughout the program. Therefore, future assessment efforts will explore the following questions: How can we support struggling students’ development? What tools might aid in scaffolding their learning? What avenues are available for helping these students find other professional fields where they might experience more success?

 

II.             Capstone assessment

 

“The Teacher and Professional Development” (ElEd 4901/SeEd 4901) is a course that is collaboratively designed and is taught by all faculty in the Elementary Education and Secondary Education disciplines. The goals of the course are to facilitate professional reflection, to enable students to explore professional issues related to teaching, and to assist students in evaluating the effects of their professional choices and actions on students, parents, other professionals, and the larger learning community.

 

Assessments in this course are designed to assess not only what the student has learned during the course, but also to reflect students’ professional growth since they began the program.

 

The primary assessment of student learning in “The Teacher and Professional Development” is the professional portfolio created by students. Students begin creating this portfolio when they enter the program and continually revise it throughout. The portfolio has ten sections, one section for each of Minnesota’s Standards of Effective Practice—standards in which students must demonstrate competency before being licensed as teachers. For each standard, students write an essay that describes their growth and development in the standard, provides evidence of that growth and sets new goals for deeper understanding. Each faculty member is responsible for evaluating 10-12 professional portfolios. Each year, the faculty engage in a reliability session to ensure fair and reliable grading practices across faculty members. In this session, faculty read and grade sample essays and discuss the reasons for their grades and discuss any discrepancies. The process continues with multiple readings and discussions until the faculty are grading in a manner consistent with one another.

 

Assessments also include a paper on the student’s philosophy of education. This paper then forms the foundation for a senior presentation. The presentation is evaluated on the basis of the student’s professionalism, understanding of various aspects of their content area and of pedagogy, responses to questions, and communication skills. Due to the large number of students (approximately 80 per year), not all faculty attend and evaluate each session. Student attendees also help to informally evaluate the sessions.

 

Table 2 reports and summarizes student scores on individual assignments in the capstone course. The data indicate that the portfolio scores are slightly below other course scores. There are several potential reasons for this: the writing required by the portfolio assignment might not fit with some students’ preferred style or modality of learning; the faculty might grade the portfolio more rigorously than other assignments since it is the linchpin of the course; or, the assignment is more extensive, lasting two weeks in duration and it is difficult for students to maintain the same level of excellence throughout. This minor gap in grading data might provide fodder for reflection as we head into the coming academic year.

 

Table 3 summarizes data pertaining to student portfolio scores. The data show that for many of the ten standards there was little significant change in student performance. Last year we noticed that the scored for Standard 8 on Assessment were much lower than for others of the ten standards. Prompted by this data and the recent emphasis of NCATE on assessment and student learning, we implemented a pilot program during the 2002-2003 academic year. This pilot program focused on enabling our teacher education students to collect assessment data on their own practices and reflect on its implications for their students’ learning. This year’s average score of 2.698 is much higher than last year’s 2.13. While many possibilities exist as to why this is the case, we feel that the experience of collecting data and reflecting on it helped many students to understand assessment more completely and, thus, be able to discuss it more cogently in this section of the portfolio. In addition, the score for Standard 1 on Subject Matter increased from 2.45 to 2.6. This is perhaps due to the use of exemplars as a means to clarify expectations of portfolio performance at the beginning of the course. We also saw significant increases in scores for Standards 9 (Reflection and Professional Development) and 10 (Collaboration and Ethics). This is encouraging, as students often find these topics to be rather sophisticated or have run out of the energy needed to compose effective essays by this point in the course.

 

 

 

TABLE 1

 

2003 ELED & SEED GRAD TEST RESULTS

 

 

PRAXIS

RESULTS

READ

MATH

WRITE

CONTENT

PLT

pts. Above cont.

pts. Above PLT

Student 1

183

183

179

174

192

34

40

2

181

179

173

172

159

32

6

3

180

178

173

164

173

24

21

4

323

322

323

158

182

18

30

5

184

180

177

178

182

33

29

6

334

333

328

181

186

57

33

7

 

 

173

 

162

 

10

8

185

178

178

168

192

23

39

9

183

181

176

179

189

24

36

10

173

181

176

168

177

28

25

11

177

170

173

152

169

12

17

12

186

184

181

187

182

47

30

13

182

184

178

188

177

48

25

14

179

181

173

166

172

26

20

15

183

184

181

187

170

47

18

16

327

325

321

164

178

24

25

17

179

176

178

166

183

26

30

18

184

190

183

169

165

45

12

19

181

184

174

174

184

31

31

20

183

190

185

185

183

45

31

21

183

184

176

 

 

 

 

22

176

182

174

156

156

16

4

23

180

172

176

157

180

9

27

24

186

185

181

161

169

16

16

25

182

177

173

181

175

36

22

26

183

188

178

187

 

47

 

27

176

190

178

156

171

16

18

28

179

181

173

158

187

18

35

29

180

185

177

156

166

11

13

30

184

187

185

165

185

22

32

31

184

187

182

189

170

49

18

32

176

190

176

176

177

36

24

33

176

186

175

160

179

20

27

34

181

175

180

171

182

31

30

35

181

184

179

149

185

6

32

36

184

187

181

181

192

57

39

37

183

189

182

169

190

29

37

38

334

331

325

172

174

32

21

39

176

169

174

168

172

28

20

40

185

182

175

181

183

41

31

41

180

173

177

168

193

28

41

42

184

188

187

190

193

50

41

43

185

183

175

188

190

48

38

44

183

186

183

194

177

54

25

45

185

186

181

186

184

38

31

46

180

181

173

171

185

31

33

47

180

187

179

170

174

27

21

48

335

328

335

200

189

76

36

49

184

179

175

176

182

36

30

50

183

187

177

174

166

34

14

51

173

181

174

162

175

22

23

52

182

184

173

179

176

31

23

53

186

190

179

180

181

35

28

53

184

188

180

172

187

32

35

55

182

176

174

176

175

36

23

56

184

175

178

164

174

24

21

57

180

179

174

179

189

39

37

58

183

187

181

167

183

43

30

59

334

333

331

164

184

24

31

60

180

184

179

170

175

15

22

61

334

330

332

162

175

22

22

62

183

190

182

176

177

36

24

63

330

335

332

170

190

30

37

64

177

172

178

155

181

10

28

65

174

186

173

177

167

22

14

66

175

171

172

 

 

 

 

67

177

180

180

184

183

44

31

68

184

183

174

182

182

37

 

 

 

 

 

 

 

 

 

Average Scores:

199.0597

199.9402

195.16176

172.45

179.03

31.81538462

26.453125

cutoff scores

173

171

172

155 art

152 k-6

 

 

 

 

 

 

140 el ed

153 7-12

 

 

 

 

 

 

148 CAL

 

 

 

 

 

 

 

124 math

 

 

 

 

 

 

 

140 music

 

 

 

 

 

 

 

143 sci

 

 

 

 

 

 

 

145 social

 

 

 

 

TABLE 2

 

 

Portfolio 40%

Philosophy 15%

Book Review 15%

Senior Presentation 30%

Total

 

 

 

MP 1

94

94

96

97

95.2

 

 

 

MP2

98

96

93

98

96.9

 

 

 

MP3

89

94

92

96

92.3

 

 

 

MP4

97

92

96

98

96.4

 

 

 

MP5

86

85

93

98

90.5

 

 

 

MP6

94

96

89

98

94.7

 

 

 

MP7

94

95

89

98

94.6

 

 

 

MP8

98

98

93

98

97.2

 

 

 

MP9

92

97

94

98

94.8

 

 

 

MP10

86

88

89

95

89.4

 

 

 

MP11

95

99

92

98

96.0

 

94.3

INSTRUCTOR AVERAGE

RT1

98

100

100

97

98.7

 

 

 

RT2

83

87

100

97

91.7

 

 

 

RT3

93

93

93

97

94

 

 

 

RT4

93

93

87

93

91.5

 

 

 

RT5

93

93

93

97

94

 

 

 

RT6

85

93

87

87

88

 

 

 

RT7

98

100

87

97

95.5

 

 

 

RT8

100

100

100

97

99.2

 

 

 

RT9

85

87

93

100

91.2

 

 

 

RT10

98

100

100

97

98.7

 

 

 

RT11

98

100

100

97

98.7

 

 

 

RT12

100

100

100

100

100

 

95.1

INSTRUCTOR AVERAGE

RW1

93

98

98

98

96

 

 

 

RW2

94

98

98

98

96.4

 

 

 

RW3

98

98

98

98

98

 

 

 

RW4

93

97

95

98

95.4

 

 

 

RW5

98

98

98

98

98

 

 

 

RW6

93

95

93

93

93.3

 

 

 

RW7

98

98

98

98

98

 

 

 

RW8

98

98

98

98

98

 

 

 

RW9

98

98

98

98

98

 

 

 

RW10

98

98

98

98

98

 

 

 

RW11

98

98

98

98

98

 

 

 

RW12

97

97

98

98

97.4

 

97.0

INSTRUCTOR AVERAGE

CB1

94.6

97

98

98

96.4

 

 

 

CB2

95.6

97

98

98

96.8

 

 

 

CB3

94

98

98

98

96.4

 

 

 

CB4

94.8

98

98

98

96.7

 

 

 

CB5

94

96

97

98

95.9

 

 

 

CB6

91.8

97

98

98

95.3

 

 

 

CB7

94.8

98

98

98

96.7

 

 

 

CB8

94.4

98

98

98

96.5

 

 

 

CB9

94.6

98

98

98

96.6

 

 

 

CB10

94.2

98

98

98

96.4

 

 

 

CB11

94.4

98

98

98

96.5

 

96.4

INSTRUCTOR AVERAGE

CM1

82

100

100

90

89.8

 

 

 

CM2

100

100

100

100

100

 

 

 

CM3

100

100

100

80

94

 

 

 

CM4

70

72

100

90

80.8

 

 

 

CM5

100

100

100

100

100

 

 

 

CM6

97

92

100

88

94

 

 

 

CM7

83

80

100

100

90.2

 

 

 

CM8

100

96

100

100

99.4

 

 

 

CM9

77

76

100

90

84.2

 

92.4

INSTRUCTOR AVERAGE

JK1

98

100

100

100

99

 

 

 

JK2

93

93

93

93

93

 

 

 

JK3

90

93

93

97

93

 

 

 

JK4

90

93

93

83

89

 

 

 

JK5

93

100

100

90

94

 

 

 

JK6

95

100

93

93

95

 

 

 

JK7

88

87

93

93

90

 

 

 

JK8

98

100

100

97

98

 

 

 

JK9

90

87

93

93

91

 

 

 

JK10

83

87

93

97

89

 

 

 

JK11

90

93

93

90

91

 

92.9

INSTRUCTOR AVERAGE

 

 

 

 

 

 

 

 

 

AVERAGES

93.2

95.0

96.1

96.0

94.8

 

 

 

 

 

 

TABLE 3

Name

Standard 1, content

St. 2, learning

St. 3, diverse learners

St. 4, instructional strategies

St. 5, classroom environment

St. 6, Communication

St. 7, Planning

St. 8, Assessment

St. 9, Prof. Dev.

St. 10, Ethics

Student 1

2.5

3

2.75

2.75

2.75

2.75

3

3

3

2.75

Student 2

3

3

3

3

3

3

3

3

3

3

Student 3

2.5

2.75

2.5

2.5

2.5

2.5

2.75

2.5

2.5

2.75

Student 4

3

3

3

2.75

3

3

2.75

2.75

3

3

Student 5

2

1.75

2

2.5

2.5

2

2.5

2.25

2.75

2.75

Student 6

2.5

3

3

2.75

3

3

2.75

3

2.75

2.75

Student 7

2.5

3

2.75

3

2.75

2.75

2.5

2.75

2.75

3

Student 8

3

3

3

3

3

3

3

3

3

3

Student 9

3

2.75

2.5

2.5

2.75

3

2.5

2.75

2.75

2.75

Student 10

2.5

2

2.75

2.25

2

2

1.75

2

2.75

2.5

Student 11

2.75

2.75

3

3

3

3

3

2.75

3

2.75

Student 12

2.5

2.5

3

3

3

3

3

3

3

3

Student 13

2.5

2

2.5

2

2.5

2.5

2.5

2.5

2.5

2.5

Student 14

2.5

2.5

2.5

2.5

3

3

3

3

3

3

Student 15

2.5

2.5

3

3

3

3

2.5

2.5

3

3

Student 16

2.5

2.5

3

3

3

3

2.5

2.5

3

3

Student 17

2

2.5

2.5

2.5

2.5

2.5

2.5

2

2.5

2.5

Student 18

2.5

3

3

3

3

3

3

2.5

3

3

Student 19

3

3

3

3

3

3

3

3

3

3

Student 20

2

2.5

2.5

2.5

2.5

2.5

2

2.5

3

3

Student 21

2.5

2.5

3

3

3

3

3

3

3

3

Student 22

2.5

2.5

3

3

3

3

3

3

3

3

Student 23

3

3

3

3

3

3

3

3

3

3

Student 24

2

3

2.5

3

3

3

2.5

3

3

3

Student 25

3

3

3

3

3

3

3

2.5

3

3

Student 26

3

3

3

3

3

3

3

3

3

3

Student 27

2

3

2.5

3

3

3

3

2.5

3

3

Student 28

3

3

3

3

3

3

3

3

3

3

Student 29

2

3

3

3

3

3

3

2.5

3

3

Student 30

3

3

3

3

3

3

3

3

3

3

Student 31

3

3

3

3

3

3

3

3

3

3

Student 32

3

3

3

3

3

3

3

3

3

3

Student 33

3

3

3

3

3

3

3

3

3

3

Student 34

3

3

3

3

3

3

3

3

3

3

Student 35

3

3

2.5

2.5

3

3

3

3

3

3

Student 36

2

2

3

2

3

2.5

2

2.5

2.5

3

Student 37

3

3

3

3

3

3

3

3

3

3

Student 38

3

3

3

3

3

3

3

3

3

3

Student 39

1.5

2

2

2.5

2

2

3

2

2

2

Student 40

3

3

3

3

3

3

3

3

3

3

Student 41

2

3

3

3

3

3

2.75

3

3

3

Student 42

2.5

2

3

2.5

2

3

2.5

3

2

2.5

Student 43

3

3

3

3

3

3

3

3

3

3

Student 44

2

2

2

2

2

2.75

2.5

2.5

2.75

2.5

Student 45

2.4

2

2.7

3

3

3

3

3

2.8

2.9

Student 46

2.6

2.4

2.9

2.7

2.2

2.9

3

2.7

2.9

2.9

Student 47

2.7

2.9

2.6

3

2.9

2.8

2.8

2.7

2.3

2.3

Student 48

2.8

3

3

3

2.9

3

3

3

2.8

2.9

Student 49

2.2

2.7

2.8

3

2.6

2.9

2.7

2.5

2.7

2.9

Student 50

2.8

2.4

2.3

2.2

2.7

2.4

2.4

2.8

2.9

2.8

Student 51

2.8

3

3

3

2.5

3

3

3

3

3

Student 52

2.8

3

3

2.9

2.9

3

2.9

3

3

3

Student 53

2.7

2.8

3

2.3

2.4

2.9

3

2.6

2.8

3

Student 54

2.5

2.3

2.8

2.3

2

2.5

2.6

2.8

2.8

3

Student 55

2.6

3

3

3

2.6

2.8

2.8

2.9

3

3

 

 

 

 

 

 

 

 

 

 

 

2003 Averages

2.602

2.718

2.8154

2.7890

2.79

2.8536

2.799

2.6979

2.8636

2.8854

 

 

 

 

 

 

 

 

 

 

 

2002Average

2.45

2.71

2.84

2.69

2.86

2.85

2.84

2.13

2.56

2.44