Page 1
Page 2
Page 3
Page 4
Page 5
Page 6
Page 7
Page 8
Page 9

SENATE 

?

From........ ?

.9.AGENDA..AND.. RULES

Subject..P. ?

ASSESSMENT. 

?

Date ...... ......13

1982

The following motion from K. E. Rieckhoff was received by the Senate Committee 

on Agenda and Rules for discussion at Senate. 

MOTION: 

"That the Senate of Simon Fraser University goes on 

record as endorsing the recommendations I to VII 

regarding assessments of teaching effectiveness as 

stated in the memo of April 20th, 1982 by K. E. 

Rieckhoff on the subject of "Student Teaching Assess- 

ments" and requests the administration to implement 

these recommendations as a matter of policy" 

Senator Rieckhoff's memorandum of April 20, 1982 is attached. 

l?"Moi.tINDUM

.10 ?

10 WIiOH IT MAY CONCERN

E30.jr(1

Date....................................................

A recent memo by a department chairman to his faculty arid lab.

Instructors regarding 'STUDENT TEACHING ASSESSMENTS" prompts me

Into action. ?

The use and particularly the misuse of such assess-

merits has been a longstanding concern of mine.

Only the m.iture

and wise attitude regarding such assessments in my own depart-

ment, I.e. to leave It up to iridivldudi instructors to decide on

their use and to take them only "cum grano sails" rather than as

true measures of teaching effectiveness, has prevented me from

going public with my concerns.

I am beginning to realize that as

a responsible member of the Senate and of the Board of Governors

I cannot continue in the luxury of such a parochial attitude.

Elsewhere in the university academic careers are endangered,

promoted, or efforts misdirected on the basis of the widespread

aiicl gross misuse of information of questionable real value to

anyone with the possible exception of the affected instructor.How

seriously such assessments are taken by academic administrators

is exemplified by the following quote from the memo referred to

above:

"This perspective on teaching effectiveness is in impor-

taut one and it is expected that all faculty and instructors

will

ensure that students have an opportunity to assess their

teaching."

1 shall address myself briefly to a number of questions in the

hope of provoking thought, examination arid discussion among my

colleagues, within the Faculty Association, among the senior

academic admiuiistralors, and perhaps even within the Senate

uu(I

the Board and their appropriate committees with a view to corrre

tive action to eliminate this serious threat to our academic

Integrity.

The questions are the following.

Why do I consider the present practice to be deplorable

and a threat to our dCddemiC integrity?

How did the present practice evolve arid why is it widely

tolerated by so many faculty members and academic admin-

1st rato rs?

What measures can and should he tdken to eliminate the

misuse and abuse of student teaching assessments.?

1)

?

Most academic departments at S.F.U. use student teaching 

?

assessments in an attempt to measure the teaching effecilvehess 

of their !nstrucl.LOndl staff. 

?

Often, as exemplified by the memo 

?

quoted above, considerable pressure is put on faculty to cooper- 

te in the practice for "...all 

cIsses, 

large and small." The 

Instruments, (i.e. questiondires) used for this purpose vary lit 

5ophlstication and some have had the benefit of social science 

methodological 

asked to rdte on 

expertise 

a numerical 

in their 

scdle 

design.a 

number 

?

of

Usually 

aspects 

the

of

student 

both 

Is

?

the course Itself and the instructor and/or teaching assistant. 

The questions can rangefrom Integrated judgments on the part of 

the student about the course such as: "Would you recommend tlil 

course... " 

to specifics such as: 

?

"On a scale of 1 to 5 rate the 

?

helpfulness, preparation, approachability, etc. of the instruc- 

tor." Besides such attempts at "objective" ratings, frequently 

general and specific open ended comments are solicited. 

?

Only? 

rarely, if ever, is the student asked to identify his/her own 

base of judgment, i.e. his/her age, acddemic goal, acddemir 

back-

ground and standing etc. 

The return rate on these questiondires while not always 100% 

is usually quite high. The results are made available to the 

liistructors and provide in most departments one of the most 

Important, and often only, pieces of information which are used 

by academic administrators, as well as departmental and univer- 

sity tenure committees, when applicable, to judge the profes- 

sional performance of faculty 

lit

their role as teachers. 

?

Thcs? 

Judgments are used for purposes of renewal, tenure, promotion 

and

merit assessment in salary considerations. 

In the light of the assumption that by and large students 

will be conscientious and honest in filling out these forms (au 

assumption

I am 

prepared to accept

as

reasonable in

the light 

my personal knowledge of and experience with students over

Induly

years) to what may one object? There would

be nothing to 

criticize if these surveys were used by 

instructors as inform-

ation on student opinion. 

?

The objection arises out of the misuse 

made by assuming a unique relationship between student opinion

as

expressed

in such surveys and teaching effectiveness and/or teac- 

her quality. Such

an

assumption

Is

purely based

on

faith and

cannot be backed up by any evidence except 

extreme cases 

poor performance

and downright irresponsibility on 

the part of

the teacher.

1^1

In the first place there are no agreed upon criteria by

which teaching effectiveness at the university level Can be

assessed.

At this time there exists no scholarly consensus

outside a few rather obvious desirable arid a similar number of

obviously undesirable characteristics ofari "effective teacher".

Even on some of those, exceptions to the rule can often be found

except for the one rule that an effective teacher must know arid

understand the subject mdtter that lie/she is teaching.

But ?

ledvilig aside the absence of a consensus on what constitutes

"effective teaching", there is

furiher absence of consensus as

to what extent student surveys are valid tools to measure

anything (even if there were an agreement on what constitutes

teaching effectiveness). ?

Studies have been made and I have read

myself some of the original literature in the field.

All that

can be said at this time is thdt. these studies are controversial

and that no consensus has emerged.

This state of affairs is not really surprising.

In the

first lflStdflCe, a students evaluation will be strongly influenced

by his/her past experience with teachers and by his/her

expectations regarding a particular course. If his/her previous

exposure to teachers has led him/her to expect very little, then

merely competent and conscientious teaching will appear "good"

•

and "very good" to him/her. In the second place, student

populations are not homogeneous and their differences extend to

likes and dislikes and differential responses regarding

particular modes of learning and teaching.

The lmportdnce of

these factors to the student evaluation is exemplified by

evaluations in which a particular teacher in a particular course

was rated by some students as "exceptionally good", by others as

such a dichotomy was further explored In an open-ended

quest ioniaire, where the reason for the assessment was asked for,

and identical reasons were given for these divergent judgments,

i.e. what appeared to some students to be an exceptionally good

aspect of the course was considered by others In a totally

different way.

In some disciplines ideological compatibility

between student and instructor has been shown to be art important

factor inf uencing assessments.

But the problem of validity, reliability, and meaning Of

these opinion surveys is compounded by the fact that the results

are considered, interpreted,

dhd

used - I prefer to say "misused"

- by our socalled "peers", i.e. people who, with soml notable

exceptions, have absolutely noprofessionial knowledge regardinnq

appropriate social science methodology and Its possible 

ramifications. ? 

Thus the "data', which are suspect in the first 

?

place, are interpreted and used in 

WdS

that. in themselves leave 

alot, to be desired. 

?

Specifically we find divergences in iiiler- 

?

pretatlori which depend not only on the methodological sophis-, 

tication of individuals but also on their personal idiosyncracies 

and even prejudices. 

?

As examples let me cite the use of ratings 

?

averaged over a ('lass without much regard to the distribution, 

the

use of one extreme part of a bimodal distribution to justify 

a personal prejudice either In favour or against a colleague, the 

arbitrary weighting and discounting of specific aspects of a 

rating etc. ? 

All In all one can only say that, under the 

?

appearance of objectivity in assessment, arbitrary judgments 

are

in fact made and rationalized. ? 

Such practices should really not 

?

condoned in an institution that purports to value scholarship 

and intellectual Integrity. 

?

That Is probably the reason that 

many years ago the CAUT explicitly cautioned the academic 

community regarding the use of student opinion surveys. ? 

It

specifically recommended against their use for the purposes of 

tenure, promotion, and other career decisions and suggested 

restricting their use to feedback to the instructor. 

2) ? 

in the light of the massive indictment given under 1) one 

may legitimately wonder how, given the validity of this indict- 

ment, the practice evolved and has become so widely accepted. 

?

I ?

believe the answer to be a mutually reinforcing complex of 

historical, political and psychological reasons. 

History placed the founding of S.F.U. into a period of 

legitimate concern about the seriousness of the commitment of 

academics to their teaching responsibilities particularly in the 

U.S.A. academic establishment. 

?

Thus one of the earliest. ? 

announced Intentions of S.F .U. was to lake undergraduate teaching 

most seriously and assure its quality. 

Politically, this period was also the time In which 

"student-power" became an Important concept in universities and 

legitimate as well as merely ideologically oriented demands were 

raised and responded to by increasing student participation in 

all levels of university governance. 

?

The willingness to listen 

to the student arid give consideration to his opinion became a

public "

otherhood" issue.

This led to almost totally uncritical

acceptance of a variety of concepts that tended to abolish, dS

presumably "undemocratic" or "elitist" discrimination on the

basis of differential knowledge.

These trends together with the

absence of'clear alternatives that could be characterized as

"object l

e" teaching assessments led to the acceptance of student

opinion surveys as the dominant tool for such assessments.

The psychological factor abetting such acceptance will he

familiar to everybody who has access to unfiltered student.

Opin ion:

By dnd large students are exceptionally kind arid

generous In their dssessmeflt,s of faculty teaching.

Being aware

of' their limited knowledge and information bdse, they tend to

give the benefit of the doubt to their teachers, rating faculty

of merely average professional competence and normal care and

conscientiousness as "good" or even "very good", acknowledging

normal courtesies shown to them as if they were almost unheard of

kindnesses, and judging even professors who treat them with

disdain arid who are sloppy and unreliable as "o.k." or average.

Of course, there are a fair number of students who never

experienced truly competent teachers and are thus unduly

impressed by what they find at S.F.U..

Their assessments, will

•

err far more frequently in ways that favour dnd flatter us then

the other way around.

Furthermore, there exist a number of

effective ways in which career-conscious instructors can

manipulate students to achieve favourable ratings to the

detriment of true learning:

generosity in giving grades is only

one of the cruder and more obvious ways to do this.

The combination of these factors leads to the general

acquiescence of faculty, since the practice rarely hurts them

dOd

Is frequently to their advantage.

Acddemic admlnistrdtors

naturally love a system about which faculty rarely complain,

which gives them the opportunity to point Out how students

opinion is treasured arid used, and allows them to be seen by the

public in

demonstrable way as guardian of the teaching quality

of the institution.

With a situatio0 in which students, faculty, dnd

administration conspire to fool themselves, each other and the

public by the maintenance of a pleasant set of Illusions (riot

necessarily identical illusions for the various groups) why would

anyone wish a change in this state of affairs? I shall leave it

to you to decide on that question.

Having adressed myself to the above questIons, 1 already

hear the reply:

"student questionaires may have their faults but

they are the best tools we have for teaching assessment arid

lacking better ones we must continue to use them." My answer arid

the recommendations arising from

have two aspects: ?

It would?

be honest to acknowledge the deficiencies

lit

the first place

and

In the second place, given the extent of the deficiencies, it

would

he preferable not to use any teachinq assessment even if

nothing else could be substituted.

But there are ways in which ?

we can avoid intellectual fraud and yet use student input to make

sufficient, albeit admittedly subjective, judgments with respect

to a rough assessment of teaching effectiveness.

Here then are my recommendations:

I) Abandon any attempt to judge teaching effectiveness for

purposes of renewal, promotion, tenure, and salary

review on

d scale that Is finer than the folowing broad

categories. ?

"Exceptionally good", "acceptable", and

"unacceptdbly bad".

II)

Assume in the absence of any specific evidence to the

contrary that any instructor, who demonstrably knows the

subject. matter of the course he/she is teaching, falls

Into the group classifiable as "acceptable".

Ill) Accept as "specific evidence" for other than

"acceptable" performance only the following in

decreasing order of importance:

I) Corroborated testimony from professional colleagues

both within and outside the university who have

personally attended lectures, seminars etc. of the

person to be judged.

ii) A consistent pattern of compliments or complaints by

past and present students who have taken courses

from the person to be judged and whose commenris have

been investigated and weighed according to the

commentators academic standing, experience, arid

general. reliability. ?

(in the case of complaints, ?

the faculty complained about must, of course, have

been given the opportunity to respond and his/her

responses must also be considered).

Iii) ?

Statistical opinion surveys of alumni not enrolled

as students at the time of the survey who have 1kenr

courses from the person to be judned.

—J

1V) Prohibit as a matter of policy the use of opinion 

surveys administered to students still enrolled a& 

the

university for purposes of making judgments about a 

faculty members renewal, promotion, tenure and salary 

review. 

V)

Prohibit as a matter of policy the exertion of any 

pressure expressed or implied on the part of dcddemi(: 

administrators on course instructors to use student 

opinion surveys. 

VI)

Where course instructors for their own purposes and 

benefit wish to use student opinion surveys they are 

of coursee at liberty to do so, but the use of results 

of such surveys should be restricted to the instruclor. 

VII) 

Encourage students dissatisfied or exceptionally 

pleased with the performance of an instructor to make 

their complaints and/or compliments known to the chair 

of a department for appropriate action or (If they are 

too unsure of themselves) have them go to the ombuds- 

person of the Student Society who may then actt, in their 

•

behalf. 

Concluding remarks: 

I am aware that 1 am attacking something that will be per- 

ceived by some as 

sacred cow", which, however, to me appears 

to be merely a myth. I urge the reader to take the time to read 

the above carefully, to consider It as free of emotion as is 

possible, to discuss the merits of my remarks with others, and, 

ultimately, to respond In concrete fashion to my recommendations. 

Your considered opinion will be of interest tome diid should 

of interest to our senior academic administrators. 

?

Let's tear 

from you. 

Sincerely, 

Dr. ICE. Rleckhot'f 

KER/mlb 

Current University policy concerning the evaluation of 

teaching is as follows: 

1.

Faculty 

Appendix II to Policy AC 2 (Renewal, Tenure, and 

Promotion) includes the following statement: 

"2.1 Teachin 

Effectiveness 

Success as a teacher, however measured or assessed, 

is the paramount criterion for evaluation. Generation 

of enthusiasm in students, dedicated involvement within 

one's discipline, openness to innovation and the 

capacity for a broad approach to one's subject matter 

are all important aspects of teaching effectiveness. 

2.

TSSU Bargaining Unit Employees 

A copy of Article XVII of the Collective Agreement 

between the University and TSSU is shown below. 

Article xvi! Employment Evaluation

A. ?

An employee may be evaluated

at least

once during any

semester in which she/he is employed. An employee may

request that an evaluation of her/his performance be

undertaken during a semester and an evaluation will be

made provided such a request is received by the Departmmnt

Chairman at least one month before classes end in that

semester.

evaluation shall be made on the performance of the

duties assigned to the employee.

EvaluatiOn may include assessment of the employee's

performance by the students assigned to the employee.

and t'y the person to whom the employee is responsible

and/or such other person(s) as may be designated by the

Chairman.

A copy of the evaluation shall be forwarded to the

employee and she/he shall be permitted to add relevant

comments on the evaluation to her/his employment file.

r. ? 

uepactmonts tnat do not currently have formal systems

a ?

of evaluation of the work performance of barga1rm$nm

unit employees shall be encouraged by the Ummivecslty to

initiate and maintain such systems.

y . ?

the design, administration and interpretation of such

evaluations

falls within the

area

of management's ri.lt,tC

and reqponslblltt tea.