In May 2008, The Teagle Foundation awarded Lawrence University a $94,700 grant to research and develop alternative assessment methods for tutorial courses in liberal arts education.
Overview
The purpose of this study was to research and develop alternative assessment methods for tutorial courses in liberal arts education. Nine faculty members from Lawrence University, Macalester College, The College of Wooster and Williams College developed a Shared Assessment Method (SAM) for assessing three groups of traits: Independent Thinker (takes teacher roles; inquiring mind; self-assessment; learning to argue); Intellectual Maturity (ability to work with complexity/uncertainty; takes intellectual chances); and Creativity (idea generation; curiosity; takes multiple perspectives; connects disparate information and ideas). In 21 tutorial courses involving 40 students the faculty tested rubrics assessing the traits through observations, interviews and rating scales. The results of the study as analyzed from faculty case studies and rubrics suggest that the selected traits were valid and important. The conclusion that the traits are valid for purposes of assessment, and enhanced the pedagogy of tutorials is based on the following findings:
- Faculty and students were able to observe and measure the traits during two cycles of tutorial courses;
- Faculty found methods for encouraging student development of the traits through course plans, assignments and teaching strategies;
- The rubrics discriminated different levels of traits in stronger and weaker students;
- Mean levels of student self-estimated and faculty-estimated levels of all traits rose during the tutorials. Statistically significant improvement in the traits from baseline to final assessment periods were recorded for all the traits of Independent Thinker and Intellectual Maturity and the Creativity traits of idea generation and connectivity;
- Student evaluations indicated that they integrated the traits into their learning goals;
- Several faculty proposed using the rubrics to assess college-wide tutorial programs;
- In some cases, faculty noted that students of equal grade point averages performed unequally on the trait assessments, suggesting that the rubrics captured qualities above and beyond content mastery;
- Trait validity findings stand up across tutorials offered by a wide range of faculty representing disciplines in the Arts, Humanities, and Sciences.
- Nonetheless, several faculty concluded that student curiosity (Creativity) was already at high levels at the start of courses and no important strategies emerged for supporting this trait. Other tutors found too much overlap between connectivity and multiple perspectives (Creativity). We propose reducing the number of traits to eight: Independent Thinker (4); Creativity (2- idea generation, takes multiple perspectives and connects disparate information and ideas); Intellectual Maturity (2). There were no significant proposals to add new traits.
Going forward, we recommend applying the trait rubrics to assessments of tutorials (1-3 students), hybrid courses where tutorials are additions to conventional small courses like seminars (5-10 students), and, perhaps, liberal arts education as a whole. Because The College of Wooster already uses their independent study courses as culminating projects, they essentially serve as assessments for the full undergraduate education. Lawrence University is now introducing the senior experience as a culminating project for students. Both of these examples refer to tutorial-like programs. The rubrics, therefore, would be useful for assessing a student’s liberal arts education. To reduce faculty workload the traits would be used in syllabi to define goals, but only students would fill out rating scales, four or more times per term or semester, and orientation and debriefing class sessions would be retained to teach the criteria and assess effectiveness. The faculty in this study are important resources to train and supervise other faculty on each campus in the extension of use of these assessment/pedagogic rubrics.
The research project draws upon papers delivered at the Lawrence University Workshop on Tutorial Education Assessment March 31-April1, 2007.
Co-Principal Investigators
Lawrence University
- Robert J. Beck, Visiting Professor of Education
- William F. Skinner, Director of Research Administration
Faculty Participants
The College of Wooster
- Nancy Grace, Professor of English
- Pamela Pierce, Associate Professor of Mathematical Sciences
Lawrence University
- Rob Neilson, Associate Professor of Art
- Ron Peck, Assistant Professor of Biology
- Jerald E. Podair, Professor of History
- Claudena Skran, Associate Professor of Government
Macalester College
- Patrick Schmidt, Associate Professor of Political Science
Williams College
- Stefanie Solum, Associate Professor of Art
- Christopher Nugent, Assistant Professor of Chinese
St. John's Annapolis
- Joseph Macfarland, Tutor (Observer)
