1. Home
  2. About
  3. Board & Governance
  4. Policies & Procedures

SP 9-47 – Program Review and Evaluation

Colorado Community College System / System Procedure

SP 9-47

EFFECTIVE: June 1, 1989
REVISED: July 1, 1997
RETITLED: September 14, 2000
RETITLED: August 25, 2001

REFERENCE: C.R.S. 23-1-107 (3); BP 9-47

APPROVED:

/ Dorothy A. Horrell /
S/ Dorothy A. Horrell
System President

APPLICATION

This procedure applies to the state system community colleges.

BACKGROUND

By statute, the Colorado Commission on Higher Education (CCHE) has responsibility for reviewing all existing programs within institutions of higher education. The statute, as revised in 1985, requires:

23-1-107(3) Each governing board of the state ¬sponsored institutions of higher education shall submit to the commission a plan describing the procedures and schedule for periodic program reviews and evaluation of each academic program at each institution. The information to be provided to the Commission shall include, but not be limited to, the procedures for using internal and external evaluators, the sequence of such reviews and the anticipated use of the evaluations.

The corresponding Commission Existing Program Review Policy 1-7 elaborates on the statutory requirements, naming materials to be submitted, and deadlines for submission to CCHE. This plan is intended to meet the requirements of those parts of the statute and CCHE policy that call for development of a program review plan.

Program reviews in state system community colleges are viewed as an investment that can pay significant dividends. Instructional program review self-studies, conducted in accordance with established procedures, will assist colleges in achieving improvements in their programs. To assure that result, the reports and the responses to the reports will be used to develop plans that address opportunities and problems identified through the review.

To assure the integrity of the evaluation process, the prime focus will be on assisting the colleges to improve their programs within the context of the broader college and System missions and roles. The program review must not be viewed as punitive nor as a means to expand program budgets. The overriding objective is a valid, straightforward assessment of the strengths and weaknesses in programs. The results are to be used to continue to develop a coordinated plan for enhancing program strengths and developing those program areas that warrant improvement.

OBJECTIVES

State system community colleges and their advisory councils and governing board recognize the importance and utility of program review in educational planning and management. Program review is assuming added importance given increasing public demand for improved quality and accountability, changes in student demand, and ever-present financial constraints.

The overall purpose of program review is the improvement of community and junior colleges. Program review should provide the information needed to make informed decisions regarding program maintenance, enhancement, or restructuring, as well as the allocation of resources.

INSTITUTIONAL OBJECTIVES

Institutional program review is an essential component of institutional management and governance. The review and evaluation of instructional programs should address the following objectives:

  1. To assess program quality and effectiveness: this includes an examination of the congruence between program outcomes, program objectives, and institutional mission;
  2. To assess program efficiency: this includes an examination of resource adequacy and utilization;
  3. To aid in program planning and accountability: this includes having a direct influence on program and financial decisions such as the realignment of priorities and reallocation of resources;
  4. To assess the continuing need for the program: this includes identifying the institutional (internal) and environmental (external) needs that the program satisfies.

STATE SYSTEM OBJECTIVES

  1. To assess the quality, effectiveness, and efficiency of Colorado state system community college educational offerings;
  2. To determine whether System goals and objectives, as stated in current relevant master plans, are being attained;
  3. To assess whether program objectives are in concordance with the role and mission of state system community colleges;
  4. To periodically clarify and refine the roles and missions of state system community colleges in a manner consistent with the concepts of quality, diversity, access, accountability, and equal educational opportunity.

Schedule: The CCHE Existing Program Review Policy calls for review of each instructional program at least once every seven years. The evaluation requirements of the Carl Perkins Vocational Education Act of 1984 mandate program review and evaluation on a five-year cycle. Since many of the programs at state system colleges are vocational and are thus partially funded from Perkins monies, they must be reviewed on a five-year cycle. To avoid unnecessary duplicative review, all instructional programs at state system community colleges will be reviewed and evaluated on a five-year cycle. Some specifics of the program review schedule are presented below in Section C.

OVERVIEW

The process that is outlined below is consistent with the Board’s responsibility as described in statute (23-1-107(3), C.R.S. 1985) and the objectives of program review as presented in the four System-level objectives stated above. Although the perspective of the process is System-level, based upon state statute and CCHE policy, it is intended to encourage cooperation between central system staff and the state system colleges. In the approach taken, each state system community college reviews its own programs and reports the results to central system staff who, in turn, compile and synthesize the results for subsequent Board review and action and later submission to CCHE. Central system staff will be available for consultation when requested by state system colleges. It is the central system staff’s responsibility, in cooperation and conjunction with institutional staff, to ensure that program review is a regular institutional management activity and that review results are considered in decisions for program expansion, maintenance, contraction, or termination; or for the allocation or reallocation of resources.

MAJOR ELEMENTS

Program review activities must involve the consistent application of high standards and valid methodology. Program review standards and procedures must be acceptable to professional evaluators: the State Board, CCHE, and the general public, especially members of lay boards and the Colorado General Assembly. The major elements of the process are:

  1. Each institutional instructional program will be reviewed at least once every five years. Programs to be reviewed include both certificate and degree programs.
  2. By June 20, 1989, each state system community college will submit a five year master schedule for program review to the System. This plan will encompass academic years 1989-90 through 1993-1994. The schedule should be arranged so as to simultaneously meet the evaluation requirements of the Carl Perkins Vocational Education Act. The rationale and criteria used to develop the program review schedule should be included in this submission.
  3. Each state system community college will submit to the System, no later than June 30th of each year, a list of programs to be reviewed in the academic year immediately ahead, together with a description of the review process to be used. Each state system community college is expected to review some of its programs each year. Exceptions must be negotiated with the System.
  4. On occasion, the System may suggest programs that should be reviewed in the near future. Reasons for such suggestions might include the need for statewide study of a particular type of program or the need for studying a specific program at a particular institution. In any case, negotiations between the System and the college(s) will take place to determine when, and if, such reviews will be conducted.

REVIEW ACTIVITIES

Activities to be included in the review process are listed below. Colleges may wish to supplement this basic set of required program review activities with additional activities to enhance their local processes.

  1. The instructional program faculty and administrators will conduct a self-study addressing the data elements and questions contained in Appendix A which is provided as a guide.
  2. The self-study will be formally reviewed by a group of the institution’s faculty outside of the program and/or by external reviewers. The function of such reviewers, both internal and external, is to assess the validity of the conclusions reached concerning program quality, efficiency, and need.
  3. Institutions will develop an analysis of the program based on, but not limited to, the findings and interpretations of the program self-study and the assessment by those inside and outside the program. This analysis and commentary is the official position of the college, when approved by the college president, and is an important component of the report submitted to the System.
  4. State system community college program review reports will be submitted by the college president to the System president no later than July 1 of each year. For example, a review conducted during academic year 1989-1990 will be submitted by July 1, 1990.
  5. Central System staff will review each program review report for completeness and rigor and may suggest revisions. Central System staff will assess the adequacy of each report in summary form via the attached Program Review Checklist.
  6. Where appropriate, central system staff will incorporate the results of program reviews in recommendations to the Board concerning program development, consolidation, and discontinuation.
  7. Central system staff will annually review with appropriate college administrators the implementation of program plans as described in their program review reports and will report progress to the Board.
  8. Annually, in July or August beginning in 1990, central System staff will submit for Board approval a program review report from each state system community college to be forwarded to CCHE.

The report will succinctly focus on:

  • Board decisions made as a result of the program reviews.
  • Timelines for action resulting from program review recommendations.
  • Date of Board action concerning the outcomes of program review.

CONTENT OF REPORTS

To ensure that the responsibilities described in statute (23-1-107(3), C.R.S. 1995) are fulfilled and that state-level program review objectives are met, each program review summary report submitted to the System will include the following:

  1. Description of the institutional review process
    This section should reference the general process noted in III. 3. of this document, but should also include a list of those who participate A in the review and any unique features of the review, such as the use of outside consultants or the conduct of the review in relation to an accreditation visit.
  2. Program Objectives
    Objectives should be written so that the need they address is clear program outcomes can be assessed, and program clientele are specified. Program objectives are extremely important not only because they guide the activities of the program but also because they provide the context for program assessment and planning.
  3. Analysis and assessment
    This section should be developed from an institution-wide perspective including a discussion of how the program contributes to fulfillment of broader objectives of the institution, as well as judgments concerning the accomplishment of program objectives, program need, program quality, and program efficiency. Conclusions should be supported by the factual information considered in the review, including at least those elements listed in Appendix A.
  4. Program plan
    This section should start with a description of decisions that have been made as a result of the review and of actions that are planned to implement these decisions; for example, expand program, maintain program at current level, terminate program. Plans for the future should be clearly linked and supported by the information and analyses that were articulated in the previous sections and should contain a realistic strategy for implementation. For example, if the program is recommended for expansion and will require additional resources, a plan for the acquisition or allocation of such resources should be included. If the program is recommended for termination: What are the strategies for dealing with personnel matters, with students, with physical resources that will now be free for reallocation to other programs.

This section of the report also should include, where appropriate, a discussion of such items as anticipated changes in program objectives, organizational realignments, faculty turnover and renewal, changes in curriculum, changes in clientele, changes in support, and possible requests for changes of role and mission statements.

Central System staff, in their annual summary of reviews, will rely heavily on what is reported in this section.

ATTACHMENTS:

Appendix A

Data elements and Questions for the Review for Existing Instructional Programs

Appendix B

Instructional Program Review Checklist

APPENDIX A, PAGE 1
System President’s Procedure
Program Review and Evaluation

DATA ELEMENTS AND QUESTIONS
FOR THE REVIEW OF EXISTING INSTRUCTIONAL PROGRAMS

Each of the elements listed below must be considered in each of the program review reports submitted to the System or reason must be given for the omission. It is recognized that not all elements are relevant to each program, some elements may not be part of the institution’s data gathering system and some elements may be too costly to collect. It is also recognized that some of the elements do not have standard statewide definitions in such cases the definition used should be reported.

Elements should be reported and will be interpreted in the context of the program under review and its institution and are not intended for the development of comparisons between institutions when differences in institutional missions, program objectives, and data definitions might render such comparisons misleading.

NEED

A description and assessment of the intrinsic value of this program (or discipline that supports the program).

Centrality to mission

  • Is this program central to the mission of its institution?
  • How are program objectives related to institutional and statewide master plans for higher education?
  • How does the program fit into the current and projected Colorado environment?

Student interest and program potential

  • Service to major–how many credit hours were produced for students who have their home in this department? (for most recent three years)
  • Service to non-major–how many credit hours were produced for non-major? (for most recent three years)
  • Supply and demand analysis for types of jobs filled by graduates of program.

Other program potential

  • Public Service
  • Other

APPENDIX A, PAGE 2
Program Review and Evaluation

QUALITY AND EFFECTIVENESS

History of the program (brief)
Library holdings (comment on adequacy)
Facilities and equipment (comment on adequacy)
Accreditation status
Faculty quality (measures appropriate to institution)

Program outcomes

  • Instructional outcomes
  • Public service outcomes
  • Graduates who continue their education (undergraduate, graduate, or professional school)
  • Analysis of placement data on graduates

Efficiency (for each of the following, report for the most recent three years)

  • Faculty FTE
  • Support staff FTE
  • Credit hours produced
  • Average faculty salary
  • Student/faculty ratio
  • Program cost
  • Average class size
  • Average faculty load
  • Number of courses offered
  • Number of degrees granted
  • Number of majors

APPENDIX B. Page I
System President’s Procedure
Program Review and Evaluation

INSTRUCTIONAL PROGRAM REVIEW CHECKLIST

College:

Date:

Program:

REVIEW PROCESS

Self study
Outside review (internal and external reviewers)
Official position of the college or program

CONTENT OF REPORT

Description of review process
Program objectives
Findings and interpretations (see Data Elements)
Analysis and assessment
Program plans

INFORMATION ELEMENTS

Need

intrinsic value of program
centrality to mission
student demand
opportunities for graduates
demand for other program outcomes
(i.e., public service)

Quality and Effectiveness

history of the program
library holdings
accreditation status
faculty quality

Program Outcomes

instructional outcomes
graduates
analysis of placement data
special feature

Efficiency

faculty FTE
support staff FTE
credit hours produced
average faculty
average faculty ratio
average faculty loan
average class size
program cost
number of courses offered
number of degrees granted
number of salary majors