Department & Program Assessment

Learning Outcomes

Learning outcomes say what students should be able to do upon completion of the major. While students are expected to grow in many ways and to develop their own interests and talents, every department or program should define a few key outcomes that all students in the major are expected to accomplish and that the department will assess to determine the effectiveness of its academic program.

Learning outcomes are generally written in the form:

Students should be able to <action verb> <object>.

Use just one action verb and one object to make the outcome assessable. State the outcome in language a student is likely to understand.

Example:

A student who completes the major in X should be able to: <list items such as the following>

Produce sound philosophical discourse.

Analyze a literary text.

Translate prose from Latin into English.

Critique a mathematical argument.

Design an experiment to test a hypothesis.

Describe monuments of art in historical context.

Design a performance environment for the production of a play.

Use tools of economic theory to analyze an economic problem.

(etc.)

Components

A useful strategy is to define one or more high-level outcomes that provide overarching goals for the major and then to define specific components that can be directly assessed using student work in courses and capstones. Each component can have its own rubric to rate the quality of the work with respect to one or more outcomes.

Example:

A student who completes the major in Economics should be able to develop an economic analysis. Specifically, the student should be able to:

Formulate a problem for economic analysis;

Find relevant literature that bears on the problem;

Interpret quantitative evidence and regression analyses;

(etc.)

Quality Standards

Quality standards for work should spelled out for students and faculty.  A useful way to do this is to incorporate the standards into rubrics keyed to the desired outcomes (see "How to Assess" below).

---

Tips:

• Define learning outcomes for every major offered by the department or program. There is likely to be overlap, but there should also be differences that distinguish the majors.
• Focus on a few key outcomes that are expected of all students and that will be assessed to determine the effectiveness of the program. Students should go beyond these outcomes in individual ways.
• State outcomes or components with just one action verb and one object to make them assessable.
• State outcomes plainly so that students, faculty, and others can understand what they mean and agree when they have been achieved.
• Develop quality standards for evaluating student work with respect to the outcomes.  Incorporate these into rubrics  to rate work samples from courses or capstones.

Map Major Requirements to Outcomes

Once learning outcomes have been defined, prepare a curriculum map that shows the relationship between major requirements and learning outcomes.  One way to do this is to make a table or spreadsheet with major requirements as rows and outcomes as columns (or vice versa) and to indicate in the cells how requirements address outcomes and at what level.  You may want to do a similar mapping for individual courses.  The curriculum map will help you identify possible gaps in the curriculum and places to gather evidence for assessment.

Click here for a Curriculum Mapping Tool.

Once you have your learning outcomes and curriculum map in place, approach assessment in the following way:

1. Assess work in gateway courses to see if students are developing the desired knowledge and skills.
2. Assess work in capstones (Senior Experiences) to see if the learning outcomes have been achieved.

For both #1 and #2, it's important to discuss work samples or ratings as a department to identify strengths and weaknesses and plan changes to the curriculum. This discussion could be part of a department meeting or annual retreat.  You should keep minutes to document the discussion for program or accreditation reviews.

---

Make a rubric

It can be quite useful to develop rubrics to rate work samples with respect to outcomes.  Rubrics can describe performance at different levels of development so faculty can rate work consistently and students can check their progress.  Decide on a number of levels (from 3 to 5) and a label for each level (such as: 1 = initial, 2 = basic, 3 = proficient, 4 = advanced).  Make it clear which level is the target (3 = proficient).

How to write a rubric

1. Write a description for the "proficient" level first. This should clearly describe achievement of the outcome.
2. Write a description for the "advanced" level next, indicating features that mark expert performance. This gives students a higher level to strive for and assessors a way to identify performance that is a model for others.
3. Write a description for the "basic" level next, indicating features that are emerging but not yet mastered. This helps students identify how they are progressing.
4. Write a description for the "initial" level last, identifying the foundation (prerequisite knowledge or skill) on which the outcome is built. This gives students and faculty a starting point for learning.
5. Tweak the descriptions to clearly distinguish each level from the one just above and below, forming a progressive scale from initial to advanced.
6. Edit descriptions to make them concise, and highlight the words or phrases that distinguish each level. This will make it easier to use the rubric to rate student work.
7. Indicate which level is the target for students in the major.

Click here for a Rubric Template (Word file).

Sample rubrics:

Lawrence's GER rubrics

AAC&U VALUE rubrics

AALHE sample rubrics

---

Tips:

• Give precise characteristics of performance at each level; avoid sweeping statements like "excellent," "average," or "poor."
• State what is evident as well as what is lacking; avoid defining a level solely in terms of deficits ("lacks X, Y, Z... ").
• Limit the criteria at each level; focus on the essential.
• Contrast adjacent levels to make sure they distinguish performance well.
• Avoid recapitulating the GER writing/speaking rubric. You can use that rubric alongside a rubric focused on major outcomes.
• Test the rubric with samples of student work and check for consistency among raters. (Also do this for training before rating student work.)

---

Rate student work

To assess how well students are achieving key outcomes for the major, use your rubrics to rate student work from gateway courses and capstones. Mark the level of achievement on the rating scale and jot comments on strengths and areas for improvement. You can incorporate the rubric into your course grading or use it separately for department assessment.

---

Tips:

• Focus the assessment: Select particular outcomes or components to assess. Use findings from past assessments to guide you.
• Sample widely: Rate a selected assignment from many students to get a cross-section.
• Balance dividing the labor vs. checking reliability: Have different assessors rate different outcomes, or have two or more assessors rate the same outcomes and compare to stimulate discussion.

---

Pool your findings

Gather faculty assessments of student work, both quantitative (ratings) and qualitative (comments). For each outcome, prepare a table, graph, or narrative summarizing the findings.  Describe common strengths and areas for improvement. Discuss these findings as a department and decide what to do next (see below).

Click here for a sample of how to report assessment data (PDF).

Click here for a sample Studio Art Excel tool for computing results (XLS).

---

Tips:

1. Compare findings from your direct measures (assessment of student work) with those from indirect measures (surveys, focus groups, or structured interviews) to try to identify causes for the results you found.
2. Use a chart or table to present the full range of data (not just the average). This will give a better sense of the variability in student performance.
3. Look for patterns and compare to previous years.
4. Relate your findings to your curriculum map as you consider what adjustments might improve student learning.

 

Assessment should stimulate useful conversations among department faculty.  Most important is that you use assessment findings to make improvements, large or small; otherwise, effort will have been expended for little benefit.  Accreditors are mainly interested in how we "close the loop" by using assessment findings for program improvement.

Discuss findings

Gather your ratings and comments and discuss them as a department.  Make this part of an annual retreat.

1. Look for evidence of strengths and weaknesses in the academic program.
2. Examine sample work as a group and discuss.
3. Update your curriculum map and check where knowledge and skills are developed.
4. Compare findings to previous years to identify trends.
5. Document this in meeting minutes for future reports and reviews.

Plan actions

1. Consider possible changes to curriculum, instruction, or assessment.
2. Plan follow-up surveys, focus groups, or student interviews to gather more information on areas of concern.
3. Decide how you will evaluate the impact of any program changes.
4. Document findings and planned actions for your annual assessment update and triennial program report.

---

Tips:

• Tweak requirements, assignments, guides, etc., to close gaps in the academic program or strengthen areas of concern.
• Refine your outcome statements, rubrics, and assessment methods to better align them or make them more useful and reliable (consistent across raters).
• Identify other assessment points to gather developmental data or assess outcomes not addressed by the Senior Experience.
• Shift focus as needed: identify particular areas of concern to probe or follow up in the next assessment cycle.

Academic departments and programs are reviewed internally on a three-year cycle.  Departments or programs scheduled for review submit a triennial report using an online form, while others email an annual update to the Provost's Office.  These are due June 30 each year.  Both the triennial report and annual update include questions about assessment of student learning, findings, and resulting actions. 

More information can be found on the Provost's website under Department Chairs & Program Directors > Program Reports & Updates.

Consult with the Accreditation & Assessment Committee

If you are planning to probe an area of concern or are considering changes to your learning outcomes, curriculum map, or assessment methods, feel free to consult with the Associate Dean of the Faculty for help from the Accreditation & Assessment Committee.

Archive

Be sure to save your assessment materials (learning outcomes, curriculum map, and rubrics), updates and reports, and committee feedback in your department share space so they are available for follow-up studies and future reporting.