Undergraduate Academic Program Review

Frequently Asked Questions:

What is assessment?

For the purposes of undergraduate academic program review, assessment is a form of research that allows for systematic evaluation of the extent to which the program meets its objectives (goals*) and outcomes. At NKU, especially in the academic arena, it is used to improve programs through proactive, well-informed decision-making.

* goals will hereafter be referred to as objectives

What are the purposes of assessment?

1. To indicate the extent to which a program achieves its objectives and outcomes so that the program can use the findings to inform:
• program planning
• decision-making to improve the program
• revisions of program objectives
• resource allocation and budget requests.
2. To report program progress and results to audiences such as faculty, deans, the provost, the Council on Postsecondary Education (CPE), external accrediting bodies, colleagues in other programs and at other institutions, and current and prospective students.

What is the assessment cycle?

The assessment cycle is continuous. It should identify/document strengths, weaknesses, needs, improvements and future plans:

• It begins with the identification of program missions, objectives, and outcomes.
• It continues with the evaluation of those missions, objectives and outcomes.
• It uses the results of the evaluations for the purposes described above (identifying strengths, weaknesses, needs, improvements and future plans)
• Then it begins again....re-examining mission, objectives, and outcomes, evaluating the outcomes, etc.

What are program objectives?

• They are broad, general statements of either
  • what the program wants students to be able to do and to know or
  • what the program will do to ensure what students will be able to do and to know.
• They are evaluated directly or indirectly by measuring specific outcomes related to the objective.
• They are related to the mission and goals of the department and college in which the program resides, and to the mission and goals of the University.

What are program outcomes?

• Outcomes are more detailed and specific statements derived from the objectives.
• Outcomes are used to determine the presence or absence of, amount of, or level of the behavior or knowledge specified by an objective. They may be things the program wants students to know (cognitive), ways students think (affective/attitudinal), or things students should be able to do (behavioral, performance, psychomotor).
• Outcomes are observable, measurable results or evidence of the educational experience.
• Outcomes are detailed and meaningful enough to guide decisions in program planning and improvement and decisions about pedagogy and practice.

What is the relationship between an objective and an outcome?

Objectives and outcomes help translate the very broad goals of university, college, and department mission statements into the curriculum by which you fulfill that mission. They describe the knowledge, attitudes, and skills you want students to have when they finish a part of your program. They can apply to individual assignments, to courses, or to whole programs. For program review, think of what you and your colleagues want students to know or be able to do when they graduate from your program.

One way to think about the relationship between objectives and outcomes is that objectives are broad, general statements of what you want to see and outcomes are specific examples. "Students will be able to analyze writings in our discipline" is a program objective, a general statement of behavior you might want to see in your graduates such as:

"Students can produce a thorough list of references, demonstrating their ability to locate writings relevant to the problem they're working on;"
"Students can discuss those references, distinguishing between well-done and poorly-done research in our field;" and
"Students use the distinction between well-and poorly-done research to evaluate what they read."

These are examples of related outcomes, specific behaviors that tell you whether the students analyze writings in a way that's appropriate for your discipline.

Another way to think about the relationship is that the outcomes are specific enough to suggest assignments, exercises, and experiences from which the students can learn the knowledge, skills, or attitudes related to the objective. Similarly, outcomes are specific enough to suggest ways to evaluate how well the program has accomplished its general objectives for students. If you wish, you could emphasize that last point by the way you phrase your objectives -- e.g., "the program will be structured so that students learn to analyze writings in our discipline." NKU's undergraduate program review process uses the terms "program learning objectives" and "program learning outcomes" for these general and more-specific descriptions.

How detailed should outcomes be? In a typical physics course the set of learning objectives is several pages long because physics is a broad and quantitative subject with many important subtopics. Is it enough to state, for example, that "Students will demonstrate a general understanding of the laws of motion," or must we break out all the topics (kinematics, dynamics, force, momentum, etc.)?

Program outcomes are detailed statements of what students know and/or are able to do. Prior to formulation of program outcomes, it may be helpful to define some specific criteria that clarify faculty expectations of student learning. For example:

• how do students demonstrate their understanding of the laws of motion?
• what would student knows or be able to do in order to demonstrate that understanding?
• what exactly does a "general understanding" mean?

Meaningful outcomes must be measurable and specific enough to guide decisions regarding program planning and improvement. The outcome, "Students will demonstrate a general understanding of the laws of motion," is not specific and, thus, it would be challenging to measure. Here is an example of one way this outcome could be rewritten:

Program graduates are able to explain and apply mathematically the laws of motion to problems involving kinematics, dynamics, force and momentum.This outcome requires graduates to demonstrate skills that are measurable and observable (e.g., explain a problem or phenomenon and apply mathematical formulae to solve problems involving motion). It specifies topics in which this knowledge must be demonstrated. Assessment methods for this outcome would probably involve a collection of student work (homework, exams, reports, etc.).

How should my program's objectives relate to college, university, or professional association objectives? What if some program objectives don't "link upwards?"

Some tools and guidelines for undergraduate program review suggest that program objectives should be examples of, or be associated with, or be clearly related to the goals and objectives of the broader units of which the program is a part. The NKU definition says that program objectives "... are related to the mission and goals of the department and college in which the program resides, and to the mission and goals of the University."

The Guidelines for Undergraduate Academic Program Review say that the program description in your review portfolio should include "...the department's goals for undergraduates, including the relationship of those goals to the department's mission [and] the relationship of that mission to the college and University missions"

If few or none of your program's objectives seem related to your college's (etc.) objectives, it's probably time to reconsider your role in those larger units. But finding that some program objectives don't have clear upward links shouldn't be cause for alarm. Some objectives may be specific to your field of study or to specific educational practices in your program rather than to broader objectives that make sense for a whole college, a whole university, or a whole discipline. You may choose to emphasize something by stating it as an objective even if the larger units of which you are a part don't pull that particular issue out for special emphasis. The important thing is to understand what relationships there are between your program's objectives and those of the larger units, not to make every one of your program objectives reflect larger units' objectives.

What about skills and knowledge that don't lend themselves to measurement? Do they not really count anymore? Are we now supposed to teach only what is measurable?  

The real question is "What does one mean by 'measure'?" Webster's New Collegiate Dictionary gives the following as its fifth definition of the verb measure: "to estimate or appraise by criterion." One does not have to count or summarize by a number in order to make that appraisal. One does have to establish an objective, understand the outcomes that will give evidence of its achievement, observe, record and appraise that student's behavior in order to perceive and present the evidence of achievement, and learn from the evidence either that students are heading in the directions they need to, or that someone needs to modify the course. One does have to discern that certain student works are members of one set, and others are members of another. One does have to establish the criteria that will allow the discernment. All of this can be wonderfully achievable without numbers. Sometimes it can be wonderfully achieved with numbers. But in no way is assessment necessarily dependent on numbers.