SFU

MEMORANDUM

OFFICE OF Till-: VICE-PRESIDENT, ACADEMIC AND PROVOST

University Drive, Burnaby,BC

Canada V5A 1S6

TEL: 778.782.3925

FAX: 778.782.5876

vpacad@sfu.ca

www.sfu.ca/vpacademic

attention

Senate

date

April 25,2013

FROM

Jon Driver, Vice-President, Academic & Provost

pages

nE:

Evaluation of Educational Goals

S.13-40

I wish to propose the following motion to Senate,supported by, and in the csmtcxt of, the discussion that

follows the motions.

That the quality of academic programs at SFU be enhanced through the following practices

and processes:

1. each academic unit will develop a public statement of educational goals for each

academic program prior to its next external review, commencing with reviews in the

spring term of 2014;

2. commencing with reviews scheduled for spring term 2015, each academic unit will

include in

the self-study for the external review an evaluation of the success of the unit

in meeting the educational goals of its program(s), using methods and evidence

selected by the academic unit;

3. resources for the development and evaluation of the statement of educational goals will

be provided by the Vice-President Academic, together with assistance from relevant

support units;

4. SCUTL will oversee these processes, and will report annually to Senate on their

implementation and impact;

5. a thorough evaluation of the costs and benefits of the above practices and processes

will be made for Senate in 2017 after the third year of implementation, and Senate will

determine whether and on what basis they should be continued.

Introduction

Over the last eighteen months the University has been considering a proposal to move towards more

formal definition and assessment of learning outcomes (LOA). The history of this proposal was presented

in a document that Senate discussed in February 2013. At that meeting, Senate also debated at some length

a rationale and proposal for the development and assessment of learning outcomes. Following that meeting

1 prepared a further proposal for Senate's consideration that responded to various concerns and critiques. I

withheld the proposal from the March 2013 Senate agenda at the request of faculty members who wanted

SIMON I'RASER UNIVERSITY

ENGAGING THE WORLD

more time to consider the proposal and its implications. I posted the proposal on a website, requested

comments, and organized three open meetings where I presented the proposal, responded to questions,

and listened to comments.

Havingconsidered the commentsand discussion that have takenplace over the lastyear, I have concluded

that, while some academic units have already begun to implement course-based learning outcomes and

assessment processes, either as a component of programaccreditation or voluntarily, not all academic units

are currently equipped or prepared to do so. However, if we do not implement LOA processes across all

units at this time, I am convinced that we must still do more to clarify the purpose of academic programs

to students, and to demonstrate to the broader community that we are evaluating our ability to deliver

what we promise. Havingconsidered the discussion about the learning outcomes proposal, I suggest that

the University should

take a somewhat different approach to understanding the performance of our

students in relation to the goals ofacademic programs.

Concerns about learning outcomes and assessment proposals

My reasons for suggesting a differentapproach are as follows:

1. Although some units have already developed learning outcomes and assessment processes, and

other units are currentlydoing this, not all unitssee a course-based LOA approach as the right way

to proceed. The term "learning outcomes and assessment" isseen by some as implying a pre

determined method that does not capture the full value of a university education, and would

preventindividual academic units from developing a means of self-assessment that is consistent

with their disciplines and the educational goals of faculty members. The modified proposal

provides much greater autonomy to academic units to develop their own approach.

2. Some faculty members have expressed concern that centralized collection ofinformation about

studentperformance in relation to a set of formal learning outcomes couldresult in a numberof

deleterious consequences, including; teaching to a narrowset of outcomes; preoccupation with

"marketable" skills; emphasis on easily measurable phenomena; development of an unnecessary

bureaucracy; use of student performance data to evaluate individual faculty members; loss of

creativity in teaching; reduction in student responsibility to learn; workload increases for faculty

and support staff; and unknown financial impacts. The modified proposal seeks to address these

concerns.

3. Many faculty members believe that the LOA proposal was drivensolely by the move to become

accredited with NWCCU and they are concerned that the University will be required to conform

to standards imposed externally. The modified proposal addresses these concernsby makingit clear

that academic units willbe free to develop processes and practices that are suited to their

disciplines, programs and educational goals, and that faculty members within theseacademic units

will have control over such

processes, practices and associated information.

A rationale for this change in our practices

I believe that the University should address the following two questions clearly and publicly:

•

What are wc seeking to achieve through delivery

ofour academic programs?

•

How well are we meeting those aspirations?

These questions need to be addressedfor four reasons.

First, there are benefits to students in providing clear, accessible information about the educational goals of

academic programs in a comprehensive university. Such information tells intending students about the

purpose and value of their proposed course of study; and it helps currentstudents map individual course

content onto a more general set of program goals.

Second, a clear statement about the goals ofan academic program provides continuing faculty members

with a focus for discussionabout curriculum content and structure. Such a statement is also helpful for

incoming faculty members or temporary instructors.

Third, assessing performance against the educational goals of an academic program allows us to

demonstrate to external stakeholders that we have well-defined program goals, and that we can link our

performance and the achievements of our students to these goals.

Fourth, there is ample evidence that definition and assessment of post-secondary education goals are

widespread in manyjurisdictions, and are increasingly regarded as the best measure of educational quality

and effectiveness. Examplesinclude: the Bologna Process and Tuning project in Europe; the Quality

Assurance Agency in the UK; the Tuning USA project and Degree Qualifications Profilesponsored by the

Lumina Foundation; USA regional accreditation standards; the Australian Qualifications Framework and

the associated Learning and Teaching Academic Standards project; and the PostsecondaryEducation

Quality Assessment Board in Ontario. Most recently, the

OECD has sponsored a project (Assessment of

Higher Education Learning Outcomes) to determine whether learning outcomes can be evaluated on an

international scale, focusing initially on economics and engineering.

Process

In suggestingthat we need to do something in regard to the questions posed above, I believe that there is

widespread support for the general proposition that we should clearly state the goals ofour academic

programs for the benefit ofstudents and for others who have an interest in the value of post-secondary

education. I am therefore proposing that Senate approve a process for ensuring that each academic unit

develops a statement

ofits educational goals prior to its next external review. "Educational goals" should

be considered broadly as the anticipated benefits to students from participation in an academic program,

and thus may include knowledge, understanding,

skills, competences, experience, attitudes, ethics, and

professional development. Such a statement would provide a framework within which discussions about

curriculum could take place, and would enable the unit to map its courses and curriculum structure against

the general gpals that are identified. I anticipate that some units would also decide to apply this approach to

some or all regularly offered courses, but this issuewould be left to the discretion ofeach unit.

All academic units currently undergo an external review on roughly a seven-year cycle. I am therefore

proposing that Senate approve a modification to these reviews, such that a component

ofthe self-study

document will be an evaluation

of the success of the unit in meeting the goals ofits program(s), using

processes and practices selected by the unit. This proposal would allow a diversity of methods to be

employed, with these methods being evaluated by external reviewers who are colleaguesin the same

disciplines. SFU already has a number of processes in place for assessing various aspects ofits programs, and

many

of these are already incorporated into external reviews. These include: teaching and course

assessments by students; assessment of learning outcomesin some programs; assignment of grades by

instructors; evaluation of course syllabi; curriculum review and revision; internal and external surveys of

students and alumni; surveys of employers; performance ofstudents in externally evaluated competitions;

success ofstudents in a wide range of post-graduate activities. An evaluation of performance in relation to

educational goals would likely draw information from many of these sources, but academic units could

develop other methods if that was seen as appropriate at the unit level.

With this modification in place, the SFU external reviewprocess would ensure that each academic unit has

for each program a well-defined statement of educational

goals that are the subjectof self-assessment and

external review, resulting in useful feedback to the unit that would be incorporated in the action plan that

follows the external review. This in turn would provide evidenceof the qualityof SFU programs to our

students and to external colleagues and stakeholders.

Resources and workload

The process proposed in this document istied to the cycle of external reviews, so thatchanges will be

undertaken by only a few academic units each year. This willspread both the workload for academic units

and the need for support and resources from central sources.

The estimated workload offaculty and staffwithin each academic unit will vary depending on local

decisions. The minimum workload required isset out in the following table, together with resources that

would be available to supplement the work within academic units. It shouldbe noted that some academic

units undergo external assessment or accreditation by their disciplines, often in programs that prepare

students for membership in a

profession. When such accreditation requires that the unit assess learning

outcomes (or some similar equivalent to educational goals), no further work should be necessary for the

SFU external review.

Action

1. Prior to external

review, develop

statement of

educational goals

2. External review

self-study

3. Action plan

resulting from

external review

4. Implementation of

recommendations of

external review

5. Prepare mid-

review report

6. Prepare for next

external review

Academic unit

Consult; write drafts;

ratify

Evaluate performance

against statement

goals. Gather

supporting evidence.

Drafts, consultations

and revisions.

Process already in

place; little additional

workload

Modification to

courses, curriculum or

data gathering

Process already in

place. Some additional

workload in describing

changes made and

gathering data to

assess

effect of changes.

Review statement of

educational goals.

Review previous

recommendations,

action taken, and effect

of changes.

Coordination and Assessment

Central support

Staffsupport from TLC.

Financialsupport from

VPA for temp hires or

faculty buy-out during

development.

Staffsupport from TLC.

Financial support from

VPA for temp hires or

faculty buy-out during

development.

No extra support needed

As this processis already

in place, units would

draw on the same

resources that are

currendy available.

No extra support needed

No extra support needed,

unless major revision of

educational goals is

planned - see point 1 for

resources.

a. h

O-

o- S

Becauseexternal reviews are managed through the office of the Director ofAcademic Planning, we will

modify the external review process to ensure that external reviewersare aware

of the need to comment on

the unit's educational goalsand itssuccess in meeting those goals.

Provision of resources to academic units will be the responsibility ofthe DirectorofUniversity

Curriculum and the Director

of the Teaching and Learning Centre, who will work with academic units to

define resource needs. The VPA will budget for financial support for each academic unit as required at

various stages ofthe cyclical process.

Introduction ofthese initiatives will take place over a number ofyears, so the first few years will be an

opportunity to test and refine implementation

processes. Given the mandate ofthe Senate Committee on

University Teaching and Learning (SCUTL), the VPA will prepare an annual report to SCUTL on the

progress and results ofthis initiative. SCUTL should include this information in its annual report to Senate.

After three external review cycles, an assessment of this initiative will be undertaken under the direction of

SCUTL in order to assist Senate in deciding the future of the initiative.

The VPA will continue to manage external requests for information about the quality ofSFU programs

and SFU'sinternal quality assurance processes. When required, the VPA will prepare reports using

information from self-study documents, external reviews, action plans and mid-review reports to

demonstrate that the University is accountable to its students, to colleagues in other institutions, and to the

general public with regard to the stated educational

goals ofits academic programs.

Timeline

Units that will be reviewed in 2013/14 will be asked to include their statement ofeducational goals in the

self-study (due for completion by December 2013). External reviewers (spring term 2014) will be asked to

comment on the educational goals and to provide advice on appropriate methods for the unit to undertake

a self-assessment of those goals.

Units that will be reviewed in 2014/15, 2015/16 and 2016/17 will be asked to include a statement of

educational goals and a preliminary self-assessment in the self-study (due for completion by December

2014, 2015 or 2016). External reviewers (spring 2015, 2016, 2017) will be asked to comment on the

educational goals and on the results ofthe preliminary self-assessment, and will also be asked to provide

advice on the further development of the methods of self-assessment.

By the end ofspring 2017, three cycles of externalreviews will have examined both the educational goals

and the self-assessment processes. We will therefore have information on the process from approximately

15 academic units. A review of this initiative will begin in spring 2017, with a report and

recommendations to Senate no later than September 2017.