Report to Senate (September, 2002-July, 2003)

Infor mation

Submitted by

Dr. Tom Grieve, Chair (term ended July/03), Bernard Roitberg (current Chair) &

Dr. David Kaufman, Director, LIDC

For

Membership and Terms of Reference

SCUTL is a standing committee of Senate. Its Membership and Terms of Reference are given in

Appendix I, which also is available at:

Meetings

SCUTL met eleven times on a monthly basis between September, 2002-July, 2003, from 12:30-

2:00pm, with occasional extensions to 2:30pm.

Guests

The following individuals met with SCUTL during this period:

.

.

Visitor

Date

Topic

Laurine Harrison

Oct. 21, 2002

Report on Undergraduate Grading

Ombudsperson, Ombuds

& Grading Practices at SFU

Office

Drew Parker

Dec. 9, 2002

SFUFA Motion on Evaluation of

Associate Professor

Teaching at SFU

Faculty of Business

Administration & President-

SFU Faculty Association

Bill Krane, Associate VP

Feb. 17, 2003

SFU Surrey Long Term Planning

Academic (Chair)

Committee Report

Phil Winne, Professor,

Faculty of Education

Laurie Summers, Director,

Academic Planning

Susan Stevenson

Feb. 17, 2002

SFU Task Force on Academic

Honesty and Integrity

Christine Kurbis,

April 14, 2003

Adjudication of faculty proposals

Coordinator, Teaching

for presentation at SFU Symposium

Enhancement Programs

on Innovative Teaching (To be held

May 12-15, 2003)

Bill Krane, Associate VP

May 26, 2003

Discussion of evaluation of

Academic

teaching at SFU and motion passed

Sue Roppel, Director-

at SFUFA meeting

Academic Relations

Mark Fettes, Allan

June 9, 2003 and

Discuss the design of the workshop

MacKinnon, Faculty of

July 14, 2003

planned for Sept./03 re writing

Education; members of

intensive(W) courses

Curriculum Implementation

Task Force

---

Highlights of Activities

This was a very busy year for SCUTL, providing advice and feedback to a variety of individuals and

groups. These are listed below:

1.

Developed faculty survey questions on teaching at SFU; this forms part of a larger survey to be

administered by Sue Roppel and David Kaufman.

2.

Provided feedback to Laurine Harrison on her report of grading practices at SFU.

3.

Provided feedback to motion circulated by the SFU Faculty Association (SFUFA).

4.

Provided advice to Cheryl Amundsen, Faculty of Education, on her proposal for a faculty

workshop on 'Rethinking Teaching: A Course Design Workshop for Professors'. This week-

long workshop is being offered for the first time at SFU from April 28-May 2, 2003. (It is a

collaborative project among the Faculty of Education, LIDC, and McGill University; it has been

offered successfully at McGill for a dozen years.)

5.

Provided advice regarding the LIDC redesign of the SFU Teaching page.

6.

Provided advice to Charlotte French in Student Services regarding the Student Services'

Student Handbook entitled, 'Making the Grade'.

7.

Provided feedback to the SFU Surrey Long Term Planning Committee.

8.

Provided feedback to the Chair of the SFU Task Force on Academic Honesty and Integrity.

9.

Provided advice to Tom Grieve, Chair of the SFU Writing Requirement Support Group.

10.

Began the process of collecting teaching evaluation forms and procedures from departments and

schools across the university.

11.

Adjudicated the proposals submitted by SFU faculty and others for the SFU Symposium on

Innovative Teaching scheduled for May 12-15, 2003.

12.

Provided advice and assistance to the Director, LIDC.

13.

It should be noted that the Terms of Reference of SCUTL were revised to reflect the

organizational changes that have occurred, i.e., LIDC.

Activities to be Undertaken (2003-04)

Based on its discussions during the past academic year, SCUTL will engage in the following activities

in the 2003-04 academic year.

Use of Web-Based Technology

1.

Write to all departments and schools to encourage them to have as much information as

possible on the web about their courses, and particularly course outlines, and to help them link

these outlines directly to registration information so that students can access this information

directly without searching. We will work with Sue Roppel, Director-Academic Relations to add

this to her 'Best Practices' guide for Chairs and Directors.

2.

Write to Student Services to offer assistance in redesigning the Student Handbook for placing

on the SFU website, accompanied by a shorter and more 'user-friendly' print version prepared

for students. The LIDC can provide design and web assistance.

Policies and Procedures

3.

Write to each department/school to offer assistance in defining clearly the criteria for renewal,

promotion and tenure, in the area of teaching.

0

2.

---

4.

Advocate for the teaching dossier/ portfolio as the primary tool used by faculty to present

evidence of their teaching contributions and performance and by RTP committees to assess

faculty's teaching.

5.

Work with the LIDC to offer resources, e.g., website, printed materials, training, on how to

develop a teaching dossier/portfolio and for RTP committees on how to evaluate teaching

dossiers! portfolios. Model dossiers/portfolios should be developed across various disciplines.

6.

Work with the SFUFA and senior administration to have the motion approved by faculty at the

meeting of the Faculty Association developed as University policy (see Appendix II);

r

3.

---

APPENDIX I

S

r

L

.

Standing (Reporting Category B)

Last updated 5 August 2003

Members

[Conditions

Term

Date

Name

Chair. Elected by

2

May 31,

Bernard

Faculty Member of SCUTL

members of SCUTL

years

2005

Roitberg

Faculty Member (Applied

3

F2004

ay 31,

Martin Laba

Sciences)

years

3

May 31,

Faculty Member (Arts)

Tom Grieve

years

2005

13

ay 31,

Anne

Faculty Member (Business Admin)

Elected by Senate

years

2004

MacDonald

ay31,

Cheryl

Faculty Member (Education)

006

Amundsen

[3ears

F

May 31,

Bernard

Faculty Member (Science)

years

2005

Roitberg

[3

Undergraduate Student

Elected by Senate

1 year May

31,

Gisele Da

2004

Silva

Graduate Student

[May

31,

Jonathon

1 year 2004

Crago

Director, Learning and Instructional

Secretary, Ex-officio

1David

Development Centre

(voting)

Kaufman

Director, Student Academic

Resources

Ex-officio (voting)

-

F

Charlotte

French

Terms of Reference:

1.

To provide advice on matters pertaining to Learning Outcomes and Prior Learning Assessment.

2.

To review periodically, and provide advice on, grading practices and standards in the Faculties.

•

3. To review periodically, and to provide advice and guidance on, the appropriateness of teaching

evaluation instruments in use in the University.

4. To review periodically, and to provide advice on, the instructional development needs of faculty,

sessional instructors and teaching assistants.

---

5.

To review and comment on the annual reports of the Learning and Instructional Development

Centre, the Academic Computing Services, the Student Academic Resources office and the Lohn Lab,

and annual undergraduate surveys.

6.7.

TTo

o

prproviovide

de

aadvidvicce

e

on

on

tthe

he

eafnnuaficacl y

prof

ogrvaarm

ious

of tthe

eacLhieng

arnisng

trataend

gieIs

nsin trruceltaitonaion l

tDo

ecvehalngiopmng ent

teaCcehintng

re.

and

0

learning environments.

8.

To consider such matters, related to teaching and learning, referred to the Committee by Senate and

its committees.

Quorum -

5

members

Reports to Senate annually in May.

0

---

APPENDIX II

Motion Regarding the Evaluation of Teaching

.

Whereas

student ratings of instruction have been used at SFU for about 15 years without a fully grounded and scientific

basis for evaluation, or procedures for their implementation, and appear to have an undue influence on the overall

evaluation of teaching contributions,

and

Whereas questionnaires completed by students regarding the quality of an instructor's teaching are often influenced by

factors extraneous to the quality of teaching (see attachment 1),

and

Whereas the proper administration and interpretation of the results of student questionnaires is widely recognized by

specialists to be of crucial importance in rendering impartial evaluations of teaching performance by Departmental Tenure

Committees (DTC5, soon to be called Tenure and Promotion Committees),

and

Whereas student ratings of instructors comprise only one of several components in the effort to achieve an impartial and

accurate evaluation of teaching ability,

and

Whereas

studies of the value of student questionnaires demonstrate only moderate correlations between the student-

generated evaluations and independently conducted assessments of teaching quality, e.g., correlations between student

ratings of the teaching of their professor and their achievement in the course is typically .4-.5 out of a possible 1.0).

d

an

.

Whereas a joint committee of administration and SFUFA has proposed a new policy for promotion, tenure, and renewal

that includes an expanded set of criteria for the evaluation of teaching (Proposed Policy A11.05, Section

5;

see attachment

2).

Therefore,

the SFUFA urges its executive and SFU administrators to press Senate to strike a subcommittee or taskforce

to develop and recommend for implementation appropriate methods of evaluating faculty contributions to teaching.

Specifically:

The adoption of student questionnaire forms by each department that have been designed by specialists and have been

demonstrated to have a viable degree of reliability and validity across a broad spectrum of teaching styles and fields (see

attachment 3), and that provides opportunity for local adaptation and/or additions of items,

and

The adoption of clear and consistent procedures to be used by DTCs when evaluating and interpreting the results of

student questionnaires. These procedures should require DTCs to make a concerted effort to factor out extraneous

influences and incorporate the faculty member's own narrative,

and

The adoption of the 'teaching dossier/portfolio', which provides a balanced and holistic evaluation of each instructor's

teaching, as the primary document to be submitted by faculty and used by DTCs (see attachment 4),

and

.

The education of DTCs in methods of evaluating teaching dossiers/portfolios, as well as Chairs and Directors of each

academic department and school, to enable them to properly facilitate each DTC's evaluation of teaching/portfolios. Criteria

are specified in Policy All .05, Section

5

(see attachment 2).

Drafted by Adhoc Working Group

of

SFUFA: Bob Hadley,

(0.

Sheila Delaney, Neil Abramcon, David Kaufman (advisor)

---

ATTACHMENT 1

Greenwald, A. Validity Concerns and Usefulness of Student Ratings of Instruction.

A merican Psychologist.

Nov., 1997.

ATTACHMENT

2

SFU

Policy A11.05, Section 5

ATTACHMENT

3

See next two pages

ATTACHMENT

4

See the excellent Teaching Dossier Kit on the University of Victoria website

http://web.uvic .ca/terc/resources/publ

ications/teachi n.htrn

7.,

---

ATTAcEII

V IENT 2

Promotion and Salary Review

PROMOTION

5.1 Each department, school and non-departmentalized faculty or area within

a non-departmentalized Faculty will draw up, and have adopted by the

tenure-track faculty, sets of criteria, standards and methods of assessment

for tenure and for promotion to Professor that will be reviewed and either

reaffirmed or revised no less than every three years. These departmental

criteria must be approved by the Dean, copied to the Vice President,

Academic and must be consistent with the general university requirements

for tenure and promotion contained in this policy.

5.2 When a faculty member is hired into a tenure-track position, he/she must

be given a copy of the most recently approved departmental criteria for

tenure. These will be the applicable standards when that faculty member is

considered for tenure unless he/she opts to be evaluated against the

department's most recently approved criteria for tenure.

5.3 When a faculty member is being considered for promotion to Professor,

his/her performance will be measured against the most recently approved

criteria for promotion to Professor in existence at the time of consideration.

March 1, 2003

---

. ATTACHMENT 3

STUDENT RATINGS OF INSTRUCTION

Standardized Instruments

IDEA - Individual Development Educational Assessment

The IDEA Center, Inc.

211 South Seth Child Road, Manhattan, KS 66502-3089

1.800.255.2757 or 1.785.532.5970 ---- FAX

http://www.idea.ksu.edu/products/Sturatitigs.htrnl

Another website which includes advantages and disadvantages of using IDEA

http://www.si

u.edu/facultvsenate/faccomm/Facult\'Senate/Ad%2OHoc%2OCornm

ittee%2Oon%20Teachi ng%20E

valuation report 021002.htm (See section labelled "Alternative Sb)

http://cea.curtin.edu.au/seeq/about.htrnl

"...SEEQ provides an empirical basis to research teaching and when administered at the end of the semester,

generates a summative profile of teaching performance...

Another website... http://chiron.valdosta.edu/whuitt/flles/tchevai.htnil

Multidimensional

According to Marsh (1984, 1993), there are several reasons why evaluations of teaching should

be considered as multifaceted. First, if effective teaching is multifaceted (e.g., Cruckshank,

1985; Gage and Berliner, 1992; Huitt, 1995), then instruments for student evaluations should

reflect this multi-dimensionality. Second, there is no single criterion of effective teaching (e.g.,

McKeachie, 1990). Therefore, a construct approach to validate student ratings is necessary

whereby the student evaluation ratings are shown to be related to additional indicators of

effective teaching. Finally, different factors of student evaluations will correlate more highly

with different indicators of effective teaching (e.g., McKeachie, 1973). Therefore, student

ratings should not be summarized by one response to a single item or an unweighted average

response to many items.

Throughout the 1980's and 1990's, Marsh (1983, 1984, 1993) studied the multi-dimensionality

of student evaluations of teaching effectiveness by the administration of one million

questionnaires representing 50,000 courses. Marsh's questionnaire, the Students' Evaluations

of Educational Qualit

y

(SEEQ), contains thirty-three evaluation items, divided into nine

subgroups...

ETS - Educational Testing System (Major Field Tests)

Testing Service Educational

Rosedale Road

.

Princeton, NJ 08541 USA

(609) 921-9000

FAX: 609-734-5410

jjf lpj/www

.ets.oriilhea/iii ft/index. html

IT

---

SUSSAI - State University System Student Assessment of Instruction

The State University

System Student Assessment of Instruction (SUSSAI) was created by the

Board of Regents in 1995. It

has

been administered every semester in virtually every course in

all eleven Florida state universities. This section provides information regarding the

administration of the SUSSAI and uses of its results.

http://www.fgcu.edu/planning/Assessment/SUSSA

I/documents/Validation.html

SIR-I! - Student Instructional Report II

College instruction has many dimensions, necessitating evaluation that is multifaceted. The Student Instruction

Report II takes into account that the best way to evaluate an instructors effectiveness is to do so over time and in

a variety of ways. In the SIR II -- which takes 15 minutes or less to complete -- students respond anonymously to

45

questions, plus up to 10 optional questions added by the instructor.

http://www.ets.org/heaIsirl I!

Sample

Report

http://www.spsu.edu/arts/docurnents/Sir2.doc

AZTEQ - Arizona Teacher - Course Evaluation Questionnaire

The AZTEQ was built around four dimensions: instructor's presentation and delivery,

instructor's interaction and feedback, course components and integration, and workload and

difficulty. Clearly the SIR-11, the IDEA, and the AZTEQ cover most of the major components of

the construct of teaching effectiveness....

http://sccai

> I.sunysuffolk.edu/Web/Central/IT/I

nstResearch/strate.htm

Purdue Instructor

and Course Evaluation Service - PICES

http://www.dis.purdue.edu/instructionaldata/Ddf/service.rd

I

Little else could be found about either Cafeteria or PICES. Please see:

http://provost.utsa.edu/documents/Course_Eval

uatio n

Instrument Re

p ort-200 I

0405 .pdf

Another website: (See "Evaluating Teaching" http://onl

ine.sfsu.edu/-perttula/ratin g s/Fi

nk.htrn

CIEQ - Aleamoni Course/Instructor Evaluation Questionnaire

littp:H\N,%vx\;.cedanet.coili/info.litmI

http://provost.utsa.edu/documerns/Course

Evaluation

_Instrument Report-200 I 0405 .pdf

http://vvw.wash

ington.edu/oea/descri

be. hun

Another

website: http://provost.utsa.edu/docurnents/Course

Evaluation Instrument Report-200 10405 .pdf

1^1

/0.