Systematic Program Review - Guidelines for the Self-Study Report

SPR Home

Guidelines for the Self-Study Report

Approved by Council, February 1999

Introduction:

As the starting point for systematic review of the University's instructional programs, the academic unit primarily responsible for delivery of the program is responsible for preparing a self-study. The self-study should provide a concise presentation and analysis of data to enable external and internal reviewers to determine whether the program is meeting its objectives and the criteria described in A Framework for the Evaluation of Academic Programmes (approved by University Council, April 1996). The primary characteristics and criteria to be addressed in the self-study are the quality of the program (as indicated by the curriculum, faculty, learning environment, infrastructure, outcomes), demand for the program, and the efficiency with which the program uses resources, as well as whether the program is unique or of particular importance to the province. In keeping with the University's Mission Statement and Objectives, the self-study should also comment on the commitment to fair and equitable access to the program, to education equity, to environmental responsibility and to an international perspective.

Statistics from University Studies Group (USG): 'Statistics' for some portions of the document will be provided by (USG), including information on space allocations and entitlements as provided by the Office of Space Management. These data should be appended to the Self-Study Report, but commented on and interpreted within the appropriate section of the document. Unless otherwise noted, numerical (quantitative) data should be provided for the last five academic years.

College of Graduate Studies and Research Report (CGSR): In the case of graduate programs, the CGSR will prepare a summary report on the program's management procedures and student demographics. The summary will be based on an analysis of the student files and procedural problems identified by the CGSR staff. Also included in the report will be a summary of comments and data from the CGSR Student Exit Survey. This report should be appended to and data commented on in the Self-Study Report.

Report from the University Libraries: the library will prepare an analysis of the current holdings. This report should be appended to and data commented on in the Self-Study Report.

Surveys: any surveys of current students, graduates and/or employers conducted by the College/Department should be summarized (append summary to the Report) and analyzed.

Research and scholarly work: information on awards, research grants and scholarly activities should be compiled by the unit and summarized in the Self-Study Report. The only information which can be provided by USG is the data for Research Revenue as reported annually in the University Statistics Book.

Outline for the Self-Study Report

The Outline suggested for the Self-Study Report is based on the characteristics and criteria in A Framework for Evaluation of Academic Programmes as approved by Council. The Self-Study Report should, in general, be organized in sections corresponding to the elements of that Framework. The sections should be presented as a narrative providing evidence and commentary related to the criteria listed in the Framework. The components of the Framework are not all independent and therefore similar criteria may appear for different components; e.g. both curriculum and learning environment have a criterion related to education equity. Although the Framework provides a basic set of criteria for evaluation of programs, it is important to note that the relative importance of criteria for different types of programmes is expected to vary and that additional criteria specific to disciplines or levels of programs may be appropriate. The Self-Study Report should specify additional criteria thought to be relevant to the program under review.

The table following lists a series of questions to illustrate the type of information and analysis which will be of most interest to the Review Team. All questions may not be relevant to all programs. Tables, figures, charts and/or graphs may be used to present summaries of essential data. Additional data should be included in Appendices.

In cases where departments and colleges offer more than one degree or disciplinary/interdisciplinary major, separate Self-Study Reports should be prepared for each undergraduate degree program or major. Where there are options within a given degree, a single report should be prepared, but should explain any differences between the options that exist (e.g. 3y, 4y, and Honours). Separate Self-Study Reports should be prepared for graduate and undergraduate programs. For specific directions in preparing the Self-Study Report for graduate programs, program heads should refer to the policy and program review documents approved by the CGSR in February 1998. If more than one graduate degree (i.e., a Master's and Doctoral program) is offered within a unit, information for all degrees may by included in one report provided there are separate answers (where pertinent) for each of the degrees. In some cases, the same Appendices of data (e.g. faculty curricula vitae) may serve as background information for Self-Study Reports on all programs within a unit.

Submission of the Self-Study Report

Copies of the completed Self-Study Report should be submitted by the Dean to the Office of the Vice-President (Academic). It is expected that the Dean should have reviewed the Self-Study Document(s) prior to submitting them to the Vice-President Academic. The Self-Study Report will be forwarded to the Dean of the College of Graduate Studies and Research (for graduate program reviews) and to the Review Team.

A specified number of copies should be prepared for the Review Team. The number of copies required will be dependent on the complexity of the programs within a College and the size of the Review Team.

The program review process is not iterative; the Self-Study Report for the program will be evaluated as submitted. It will not be returned to the Department or College for revision.

The Self-Study Report and appended materials will be provided to the Review Team prior to the external reviewers' visit to campus. Other materials (e.g. curricula vitae of faculty) should be available on site.

*Outline for the Self-Study Report*	*Illustrative Questions*
I. Introduction Mission of College (Aims of Department) Programs offered and degrees conferred -- identify the programs and degrees (as well as certificates or diplomas) awarded by the Department or College. Program under review -- provide history and brief description of the program -- identify the degree(s), certificate or diploma awarded on completion of the program; identify majors or areas of specialization available within the program; admission quota (if applicable) and enrollment statistics. Describe how the admission quota is administered to ensure fair and equitable access to the program. Briefly describe how the Department/College is organized to administer the program.
II. Quality
A. Curriculum Objectives of program -- provide a brief statement of the objectives for the program At a minimum, this section should provide an assessment of whether the objectives are appropriate in the academic context of the discipline and/or the expectations of the profession and the extent to which the program has been designed to meet its objectives. It should also include an assessment of the extent to which the curriculum is "up-to-date" and reflective of a responsiveness to the need for change and includes opportunities for synthesis, application and integration of knowledge, as well as an assessment of the strengths and weaknesses of the curriculum. Accredited programs should provide a brief summary of the most recent review, as well as information on the agency responsible for accreditation and the consequences of failing to achieve accreditation status.	have objectives been established for the program? does the program meet its objectives? are the objectives appropriate in the academic context of the discipline and/or the expectations of the profession? how has the program (courses, curriculum, modes of instructional delivery) changed to reflect changes in the discipline/profession? how are students encouraged to apply and integrate knowledge? how has the program addressed issues of education equity? what opportunities are there for students to take courses and pursue interests outside their area of specialization? is the program accredited by a national/international organization?
B. Faculty This section should provide an assessment of whether the number and type of faculty available to deliver and support the program are adequate. This will entail information on the academic/professional qualifications and scholarly work of faculty, as well as their involvement with academic or professional organizations, instructional development, extension and public service activities as appropriate.	do faculty have appropriate academic and/or professional qualifications to support and develop the program? are faculty committed to teaching and improvement of instruction? to what extent are faculty involved in evaluating instruction within the program? how would you characterize the quantity and quality of faculty research and scholarly work? how has the scholarly work of faculty made a significant contribution to the discipline? to what extent have faculty been recognized for their scholarly and/or professional work? to what extent have faculty been successful in obtaining external research support? to what extent are full-time tenure-track faculty involved in the delivery of the program? To what extent are other instructors involved?
C. Learning Environment Evidence should be provided to assess whether the program is delivered within a learning environment that is supportive and intellectually stimulating, that promotes the integration of scholarship and teaching, that is reflective of education equity goals and that provides an international perspective, where appropriate.	to what extent do students participate in the evaluation of instruction and assessment of the program? How? do students perceive that their instruction is effective? are there processes to assess the effectiveness of the program? what opportunities are available for students to be exposed to speakers or experts from outside the University of Saskatchewan? are instructional methods and methods of evaluating students consistent with program/course objectives? are students aware of the objectives of the program? does the program reflect new developments in the field and incorporate practical experiences, where appropriate? how does the scholarly work of faculty enhance the learning environment for students? what efforts have been made to promote education equity? How successful have they been? are policies in place to deal with accommodating students with special needs? do students receive adequate advice regarding program planning and course selection? are appropriate courses available in sufficient number for students to complete their program in the normally expected time?
D. Infrastructure Support staff, space, equipment and supplies should be available in sufficient number, and in the case of equipment, of appropriate currency, In addition the academic unit(s) should be organized for the most effective use of existing resources and for the support of the program.	are there adequate numbers of appropriately trained staff available? are the necessary facilities and equipment (type, number) provided? are computer and other information technology resources adequate? are appropriate library resources available? Are reference and information sources available in the University Libraries supplemented by holdings in the Department or College? are scholarship/bursary funds adequate?
E. Student Outcomes The review is intended to assess whether the program has met its educational objectives, whether students are satisfied with their program and are successful both in completing the program and in their academic or personal endeavours after graduation.	are students satisfied that the program is meeting/has met their personal, intellectual and academic expectations? how successful are students in entering graduate programs (at the U of S? At other universities?) how successful are students in obtaining licensure and/or employment in fields related to their program? what is the average completion time and rate for part-time and full-time students? how does the academic load compare to similar programs at this University and comparable programs elsewhere? how successful are students in external scholarship applications? are subsequent graduate supervisors and/or employers satisfied with the preparation and performance of students?
III. Demand
An assessment should be provided of the demand for the program both by students, and where appropriate, by employers of graduates. There should also be information about the importance of the discipline to other programs in the College or other Colleges.	what is the total enrolment in the program? Of these what proportion are Saskatchewan residents? from other parts of Canada? International students? how many students per year apply to the program? Of those, what proportion are fully qualified and what proportion are admitted? what/where are the primary employment opportunities for graduates of the program? does the program attract highly qualified students? to what extent do students in other programs rely on courses from the discipline(s)?
IV. *Efficiency*
A program which uses resources efficiently would be characterized as delivering the program in a cost-effective manner in relation to other similar programs at this University or other universities. Factored into this assessment is the contribution of the department/College not only to teaching within the program, but also to students outside the program; research productivity; service and extension activities, and administrative contributions within the university community.	what is the ratio of full-time tenure track faculty and total faculty to FTE students (undergraduate, graduate)? what proportion of teaching activity of faculty in the Department/College is directed toward students enrolled in the program? In programs of other Colleges? what is the ratio of support staff to FTE students? what are the average section sizes for courses within the program or delivered by the Department/College? Are upper or lower limits established for enrolment in courses? On what basis are such limits established? what collaborative resources (e.g. other Department or Colleges, non-University organizations) are necessary for or enhance delivery of the program? do commitments to outside organizations and/or administrative work within the university enhance or impinge on program delivery?
V. Unique Features and Relevance to the Province
It is not expected that all programs offered by the University would be characterized as unique or having particular relevance to the Province of Saskatchewan. Certain "core" disciplines would be expected to exist and offer similar programs at many universities.	what aspects of the program, if any, are unique within the Province, Western Canada, Canada? how does the program build on or contribute to the cultural and economic strengths of the Province? what services and expertise are available within the Province because of the faculty and other personnel associated with the program? how do extension and public service activities of faculty contribute to the Province?
VI. Summary This section should provide a summary of evidence attesting to the *quality* of the program (curriculum, faculty, learning environment, outcome, infrastructure), the *demand* for the program and the *efficiency* of program delivery. Also included should be an assessment of the overall strengths and weaknesses of the program and the academic unit primarily responsible for the program. Where more than one program exists within a Department/College, this section should explain the relationship between the programs (e.g. undergraduate and graduate) and analyze to what extent the programs enhance or compete with one another. Any current plans for changes within the Department/College or for revisions to the program(s) should be outlined.

Appendices: containing supporting data and other documents, for example:

Where possible information should be summarized using tables, charts and/or graphs. Normally data for the last 5 years should be presented. Some 'standard' data for the Self- Study will be provided to the unit by the University Studies Group (USG) or the College of Graduate Studies & Research (CGSR)

A. Curriculum

Admission requirements (pre-requisites)

Course requirements -- list course requirements; indicating which are compulsory or elective, type of instruction (LEC, SEM, ..); frequency of course offerings

Methods of evaluation of student performance

Program review and revision

type/date of last program review
type (major or minor)/date of last program revision
brief description of changes made (including changes in modes of instructional delivery) and reasons for changes

B. Faculty

Full-time/part-time faculty -- number, qualifications and scholarly work of (append CVs); numbers of tenured, tenure-track or non-tenure track appointments for each rank.

Other faculty and instructional staff -- numbers of visiting scholars, adjunct faculty, associate members, laboratory demonstrators, graduate teaching assistants, etc. and a description of their role(s) in the program.

Other personnel -- post-doctoral fellows, research associates, support staff, etc.

Teaching assignments (cu/FTE faculty) for students in the program and for students from other programs

Graduate student supervision

Scholarly work

Professional practice commitments

Public service and extension work

C. Students

Admission standards

Academic standards for advancement and graduation

Number of fully-qualified applicants relative to numbers admitted

Student demographics

Degrees awarded

Student retention data -- numbers admitted, numbers graduating within specified time frame, number required to discontinue, attrition rate

Success in external competitions, national examinations

Data on employment, post-graduate study

Enrolment trends and projections

D. Resources

Operating Budget

Other funds -- endowments, scholarship/bursary funds, research revenue, contract revenue

Space allocation -- lecture/seminar rooms, teaching and research laboratories, offices

Equipment

Computer and other technical support

Library and other information resources

Instructional and audiovisual facilities and equipment

E. Procedures for Evaluation and Assessment

Of instruction

Of program

F. Administrative structure and Organization of Department and/or College

G. Supporting documents

1.     Program Syllabus -- attach a one-page summary for each of the courses which comprise the program.

2.     Faculty Curricula vitae

3.     Student and Employer Surveys

4.     University Libraries Report

5.     CGSR Report (where applicable)

6.     Accreditation Summary (where applicable)

SPR Home