Systematic Program Review - Guide for the Review Team

SPR Home

Guide for the Review Team

Approved by Council, February 1999

Introduction

The primary purpose of the systematic review of programs is to assess the quality of the University's instructional programs¹ including a comparison to similar programs at other universities. The review process is intended to encourage periodic changes to improve programs and to incorporate innovations. It will also satisfy, in part, the need for university accountability for its programs and provide information for decisions on program revisions, program deletions and resource allocations by University Council and academic administrators.

In 1994, University Council agreed that the University of Saskatchewan should initiate a systematic review of its instructional programs on a regular cycle; in 1999, specific procedures for the reviews were approved by Council.

Several of the University's programs are already subject to external accreditation reviews. In cases where the program has recently undergone an accreditation review, the intent of the University's review process is not to duplicate assessment by the accreditation team, but to address issues of concern to the University as a whole. If available, a summary of the accreditation report should have been provided as part of the background material on the program.

The Review Team is expected to assess whether the program is meeting its objectives and the criteria described in A Framework for the Evaluation of Academic Programmes (approved by University Council, April 1996). The primary characteristics and criteria to be addressed in the self-study are the quality of the program (as indicated by the curriculum, faculty, learning environment, infrastructure, outcomes), demand for the program, and the efficiency with which the program uses resources, as well as whether the program is unique or of particular importance to the province. In keeping with the University's Mission Statement and Objectives, the self-study should also comment on the commitment to fair and equitable access to the program, to education equity, to environmental responsibility and to an international perspective.

The Review Team consists of external reviewers from other universities and internal reviewers, the latter being faculty members from the University of Saskatchewan who are from cognate disciplines and/or from a College other than that of the program under review. External reviewers will normally be from the same discipline as the program under review. The role of the external reviewer(s) on the Review Team is to bring a knowledgeable external perspective in assessing the program and comparing it with other similar national, and where appropriate, international programs. One of the internal reviewers will normally act as Chair of the Review Team and co-ordinate preparation of the report.

¹ A program is understood to be a course of study leading to a degree, diploma or certificate.

Information for the Review Team

Three important documents, approved by the Council of the University of Saskatchewan, provide an understanding of the context and criteria for the Systematic Program Review Process -- A Framework for the Evaluation of Academic Programmes, 1996 (Appendix A), A Framework for Planning, 1998 (Appendix B) and Policies and Procedures for Systematic Program Review, 1998 (Appendix C). For specific instructions on review of graduate programs, the Review Team, including the external reviewers, should refer to the policy and program review documents approved by the College of Graduate Studies and Research (CGSR) in February 1998.

In advance of the site visit, members of the Review Team will be provided with the Self-Study Report, along with the guidelines for preparing the report. Additional documents will be available on-site, including data/material related to the self-study and, where applicable, any accreditation review.

During the site visit, meetings will be scheduled with the Vice-President (Academic), the Dean of Graduate Studies & Research, the Dean of the College responsible for the program under review, the program head, faculty providing instruction to students in the program, support staff, students (current and past), and others, as appropriate.

Preparation and Submission of Review Team's Report and the External Reviewer Assessment(s)

Within one month following the site visit, the Review Team should have prepared a report and submitted it to the Vice-President (Academic). The report should, in general, be organized in sections corresponding to the elements of A Framework for the Evaluation of Academic Programmes. The sections should be presented as a narrative providing commentary related to the criteria listed in the Framework. Particular questions which should be addressed are listed in each section.

The external reviewers, as experts in the discipline, have a particular role to play in the assessment of a program. Therefore, the external reviewers, in addition to contributing to the overall report, are asked to prepare their individual assessments of the program for inclusion as attachments to the report.

The Department/College, including the Dean of the College, will have an opportunity to read the report and provide a response to the Vice-President (Academic).

Outline of the Review Team Report

I. Introduction

II. Quality

A. Curriculum

At a minimum, this section should provide an assessment of the strengths and weaknesses of the curriculum and the extent to which the program:

has been designed to meet its objectives;

is 'up-to-date' and reflective of responsiveness to the need for change; and

includes opportunities for synthesis, application and integration of knowledge.

In particular, is the program designed to meet its objectives and are the objectives appropriate in the academic context of the discipline and/or expectations of the profession.

B. Faculty

This section should provide an assessment of whether the number and type of faculty available to deliver and support the program are adequate. It should also include an assessment of faculty commitment to teaching, as well as the quality and quantity of research and scholarly work, professional practice, extension and public service (particularly as they relate to the program under review).

In particular, is there a sufficient number/array of faculty to support all of the specializations offered within the program? Is there evidence that the faculty are committed to teaching and to evaluating and improving the quality of instruction? How and to what extent do faculty research, professional practice, extension and public service activities contribute to the program?

C. Learning Environment

This section should provide an assessment of whether the program is delivered within a learning environment that is supportive and intellectually stimulating, that promotes the integration of scholarship and teaching, that is reflective of education equity goals and that provides an international perspective, where appropriate.

D. Infrastructure

This section should assess the adequacy of the facilities, physical spaces, support staff and other resources necessary for the successful delivery of the program.

In particular, do the facilities and physical spaces provide the essential resources to support the faculty/students adequately in their research, teaching and study?

E. Student Outcomes

This section should provide an assessment of whether the program has met its educational objectives, whether students are satisfied with their program and are successful both in completing the program and in their academic or personal endeavours after graduation.

III. Demand

An assessment should be provided of the level of demand for the program both by students, and where appropriate, by employers of graduates. There should be commentary on the utilization of courses in the discipline by students in other programs in the College or other Colleges.

IV. Efficiency

This section should provide an assessment of whether the program uses resources efficiently in comparison to other similar programs at this University or other similar universities.

V. Unique Features and Relevance to the Province

This section should provide commentary on the unique aspects, if any, of the program and any particular relevance of the program and/or discipline to the Province.

VI. Summary

This section should provide an assessment of quality (curriculum, faculty, learning environment, student outcomes, infrastructure), demand and efficiency of program delivery.

In particular the following questions should be addressed:

Is the program meeting its objectives? If so, how? If not, why not?

What are the strengths and weaknesses of the program(s) within the unit? To what extent can the weaknesses be remedied by non-budgetary means? By budgetary/resource re-allocation?

How would you describe the level and quality of research and scholarly/artistic work associated with units that have a major responsibility for delivery of the program?

Where more than one program (e.g. undergraduate and graduate, more than one undergraduate major or specialization) exists within a unit, how would you characterize the interaction of programs within the unit? i.e., do they enhance or detract from one another?

How could the quality of the program(s) be improved?

How could the efficiency/effectiveness of program delivery and administration be increased?

Would demand for the program and discipline-related courses be characterized as low, moderate or high? Could demand for the program be increased? How?

Could/should the unit accommodate greater numbers of students? How?

Should the unit decrease the number of students admitted? Why?

Based on the Review Team's overall assessment of the program, which of the following categories best describes the program? Why? [Additional information on the assessment categories is contained in the Policy and Procedures for Systematic Program Review (Appendix C)]

Assessment Categories
Category A	The program is supported by evidence of academic vitality in teaching, scholarly/artistic work and extension/public service sustained over a significant period of time. It has achieved at least a national reputation and might be expected to develop as a centre of excellence for the University. Few, if any, fundamental changes are required.
*Category B*	The program is supported by evidence of academic vitality in teaching, scholarly/artistic work and extension/public service sustained over a period of time, but has some weaknesses. Some fundamental changes should be made. Modifications may include program changes (e.g. curriculum revision); internal redirection of faculty resources; additional resource allocations, both faculty and non-faculty; measures to improve student satisfaction/quality of experience in the program.
*Category C*	The program is supported by some evidence of academic vitality in teaching, scholarly/artistic work and extension/public service, but has identifiable deficiencies which should be rectified in a given time frame. It is characterized by sufficient demand and/or may be essential to the service requirements of other units. Several fundamental changes must be made to achieve adequacy and provide a credible program. Deficiencies may be identified in the program (e.g., curricular); faculty/scholarly activity (e.g. quality and quantity of scholarship or breadth of expertise of faculty); and/or non-faculty resources.
*Category D*	The program over a period of years has shown little evidence of academic vitality in teaching, scholarly/artistic work or extension/public service. Such a program may suffer from additional liabilities including a low quality to cost ratio, low student demand, and lack of provision of an important service component to other programs. The academic quality of the program area is unlikely to improve without significant additional resources. Many fundamental changes are required.

VII. Attachments

External Reviewers Assessments -- these assessments should address the following specific questions:

- How would this program rank relative to similar programs across the country (e.g., lower to upper quartile)? Why?

- Which universities would be considered to have programs of similar size and quality?

- Which of the categories above best describes the program?

- Are there any areas of the overall report in which your assessment of the program is significantly different? If so, please provide some explanation for the differences.

SPR Home