EDUCATIONAL EFFECTIVENESS REVIEW

Submitted to
Western Association of Schools and Colleges
Accrediting Commission of Senior Colleges and Universities
for Reaffirmation of Accreditation

January 6, 2012

WASC Educational Effectiveness Review Committees

Steering Committee
Chair: Brian E. Klunk, Associate Professor and Chair of Political Science
Members: Eric Boyce, Professor and Associate Dean for Academic Affairs, Thomas J. Long School of Pharmacy and Health Sciences
Robert Brodnick, Associate Vice President, Office of the President
Pat Cavanaugh, Vice President, Business and Finance
Bill Coen, Executive Director, Alumni Relations
Ashton Datcher, ASUOP President
Marilyn Draheim, Associate Professor, Benerd School of Education
Lydia Fox, Associate Professor, College of the Pacific
Susan Giraldez, Associate Professor, College of the Pacific
Elizabeth Griego, Vice President, Student Life
Eric Hammer, Professor, Conservatory of Music
Stacy Holguin, Graduate Student Representative
Chris Johnston, Vice President for Development
Rahim Khoie, Professor, Engineering and Computer Science
John Knight, Professor, Eberhardt School of Business
Tom Krise, Dean, College of the Pacific
Ted Leland, Vice President, External Relations and Athletics
Eileen McFall, Director of Learning and Academic Assessment
Tim Nacarrato, Principal Assistant Dean for Academic & Student Life, Pacific McGeorge School of Law
Nader Nadershahi, Executive Associate Dean for Academic Affairs, Arthur A. Dugoni School of Dentistry
Maria Pallavicini, Provost
Gene Pearson, Professor, College of the Pacific
Malik Rahman, Associate Vice President - Chief Information Officer
Mike Rogers, Director of Institutional Research
Greg Rohlf, Associate Professor, College of the Pacific
Peggy Rosson, Assistant Dean of Students
Joanna Royce-Davis, Dean of Students
Bett Schumacher, Deputy to the President
Louise Stark, Professor, Engineering and Computer Science
Rebeca Stovall, Customer Support Center Manager (SAC Rep)
Cynthia Weick, Professor, Eberhardt School of Business, Director of Powell Scholar Program, Interim Dean, School of International Studies
Alex Vu, Professional Student Representative
Brigid Welch, Dean, Library

EER Core Committee
Chair: Brian E. Klunk, Associate Professor, College of the Pacific
Members: Eric Boyce, Professor and Associate Dean for Academic Affairs, Thomas J. Long School of Pharmacy and Health Sciences
Lydia K. Fox, Associate Professor, College of the Pacific
Rahim Khoie, Professor, School of Engineering and Computer Science
Eileen McFall, Director of Learning and Academic Assessment
Nadine Pappas, Administrative Assistant (Staff)
Mike Rogers, Director of Institutional Research
Joanna Royce-Davis, Dean of Students
Gregory Rohlf, Associate Professor, College of the Pacific

WASC Action Team
Chair: Maria Pallavicini, University Provost (February 2011-Present)
Pamela A. Eibeck, University President (September 2010-January 2011)
Members: Eric Boyce, Professor and Associate Dean for Academic Affairs, Thomas J. Long School of Pharmacy and Health Sciences
Robert Brodnick, Associate Vice President
Pat Cavanaugh, Vice President, Business and Finance
Pat Ferrillo, Dean, Arthur A. Dugoni School of Dentistry (as Interim Provost)
Lydia Fox, Associate Professor, College of the Pacific
Elizabeth Griego, Vice President, Student Life
Rahim Khoie, Professor, Engineering and Computer Science
Brian Klunk, Associate Professor and Chair of Political Science
Tom Krise, Dean, College of the Pacific
Mary Lou Lackey, Vice President and Secretary to the Board of Regents
Eileen McFall, Director of Learning and Academic Assessment
Lou Matz, Associate Dean and Director of General Education, Professor, College of the Pacific
Mike Rogers, Director of Institutional Research
Greg Rohlf, Associate Professor, College of the Pacific
Joanna Royce-Davis, Dean of Students

Data and Assessment Team

Chair: Michael Rogers, Director of Institutional Research
Members: Eileen McFall, Director of Learning and Academic Assessment
Sandy Mahoney, Director of Assessment and Student Development Services, Student Life
Sondra Roeuny, Assistant Vice President of Business Assessment, Business and Finance

Inquiry Team One: Institutional Distinctiveness
Chair: Lydia K. Fox, Associate Professor, College of the Pacific
Members: Carol Brodie, Assistant Dean, Research and Graduate Studies
Eddie Hayashida, Associate Dean for Administration, Arthur A. Dugoni School of Dentistry
Frank Gevurtz, Professor, Pacific McGeorge School of Law
Analiese Richard, Assistant Professor, School of International Studies
Dan Shipp, Associate Vice President, Student Life

Inquiry Team Two: Assessment of Student Learning at Pacific

Chair: Eric Boyce, Professor and Associate Dean for Academic Affairs, Thomas J. Long School of Pharmacy and Health Sciences

Members and
Major
Contributors:

Lynn Beck, Dean, Benerd School of Education
David Chase, Assistant to the Dean, Conservatory of Music
Julie Davies, Associate Dean for Academic Affairs, Pacific McGeorge School of Law
Cynthia Eakin, Eberhardt School of Business
Lorrie Knight, Professor, University Library
Xiaoling Li, Associate Dean for Graduate Education and Research, Thomas J. Long School of Pharmacy and Health Sciences
Sandy Mahoney, Director of Assessment and Student Development Services, Student Life
Eileen McFall, Director of Learning and Academic Assessment
Timothy E. Naccarato, Principal Assistant Dean for Academic and Student Life, Pacific McGeorge School of Law
Nader Nadershahi, Executive Associate Dean for Academic Affairs, Arthur A. Dugoni School of Dentistry
Cathy Peterson, Associate Professor, Thomas J. Long School of Pharmacy and Health Sciences
Michael Rogers, Director, Office of Institutional Research
Jon Schamber, Professor and Director of Educational Effectiveness and Assessment, College of the Pacific
Barbara Shaw, Dean, Center for Professional and Continuing Education
Simalee Smith-Stubblefield, Associate Professor, Thomas J. Long School of Pharmacy and Health Sciences
Louise Stark, Professor and Associate Dean, School of Engineering and Computer Science
Shan Sutton, Associate Dean and Head of Special Collections, University Library

Inquiry Team Three: Student Success

Chair: Joanna Royce-Davis, Dean of Students
Members: Robert Alexander, Associate Provost, Enrollment
Anita Bautista, Executive Director, SUCCESS
Lisa Cooper, Assistant Vice President for Diversity and Community Engagement, Student Life
Michael Elium, Associate Professor, Benerd School of Education
Kaben Kramer, Student
Jennifer Low, Graduate Assistant
Sandy Mahoney, Director of Assessment and Student Development Services, Student Life
Arturo Ocampo, Assistant Provost for Diversity
Mike Rogers, Director of Institutional Research
Peggy Rosson, Assistant Dean of Students
Gabriela Satvaldiyev, Enrollment Services Counselor, Benerd School of Education
Simalee Smith-Stubblefield, Associate Professor, Thomas J. Long School of Pharmacy and Health Sciences
Ray Sylvester, Associate Dean, Professor, Eberhardt School of Business
Tenisha Tevis, Assistant Professor, Benerd School of Education
Mary Lou Tyler, Assistant to the Dean, Director of Student Affairs, School of International Studies
Jim Uchizono, Associate Professor, Thomas J. Long School of Pharmacy and Health Sciences

Inquiry Team Four: Program Review

Chair: Rahim Khoie, Professor, Engineering and Computer Science
Members: Berit Gundersen, Assistant Provost for Curriculum, Administration, and Special Programs, & Interim Director, International Programs and Services
Nicolasa Kuster, Assistant Professor, Conservatory of Music
Diane Farrell, Director, Career Resource Center

Inquiry Team Five: Sustainability of Educational Effectiveness

Chair: Gregory Rohlf, Associate Professor, College of the Pacific
Members: Daniel Bender, Director of Academic Affairs, Arthur A. Dugoni School of Dentistry
Betsy Keithcart, Assessment Coordinator, Benerd School of Education
Eileen McFall, Director of Learning and Academic Assessment
Sandy Mahoney, Director of Assessment and Student Development Services, Student Life
Jon Schamber, Professor and Director of Educational Effectiveness and Assessment, College of the Pacific

University of the Pacific
Educational Effectiveness Review

Table of Contents

1. Preliminary Matters
1.1 Introduction and Institutional Context
1.2 Accreditation History
1.3 Response to the 2010 Team Report and Action Letter
1.4 Development of Pacific’s Educational Effectiveness Review Report
1.5 Pacific’s Hallmark Strengths

2. Student Success

3. Assessment of Student Learning
3.1 Pacific’s Approach to Assessment and Student Learning
3.2 Pacific’s Assessment Systems and Practices

4. Program Review

5. Integrative Essay: What Pacific Has Done and Learned Through the Accreditation Process

Appendices and Materials Cited
Appendix A Required Data Exhibits
Appendix B Documentation of Program Outcomes and Assessment
Appendix C Inquiry Team Reports

1. Preliminary Matters

1.1 Introduction and Institutional Context
This Educational Effectiveness Review (EER) report represents the third phase of University of the Pacific’s accreditation self-review. Section 1 of the report provides an overall context for the Educational Effectiveness Review, including a review of Pacific’s background, a summary of the University’s accreditation process to date, and its responses to recommendations made during the Capacity and Preparatory Review (CPR). Section 1 concludes with a summary of what Pacific has learned during the EER process about its distinctive educational strengths and the challenges of improving their effectiveness. The report then focuses on three key areas of Pacific’s EER self-review. The first topic summarizes what Pacific has learned as it has deepened its study of student success, in particular the relationship between financial aid policies and practices and student success. Next, the report considers Pacific’s overall approach to educational effectiveness and provides information about how the University is developing its assessment practices, especially its use of evidence to guide decisions about curriculum and pedagogy. The report then discusses the record of program review at Pacific. In a concluding reflective essay, the EER report considers what Pacific has learned from undertaking its accreditation self-review.

University of the Pacific is the first chartered institution of higher education in California, currently celebrating its 160th year. In 1851 pioneer Methodist ministers founded the school as California Wesleyan College. The University moved in 1923 from its San Jose location to a new campus in Stockton, bringing private higher education to the San Joaquin Valley. Today, University of the Pacific is an independent institution of 6,710 students (as of fall 2011) on three campuses in Northern California. On its Stockton campus, Pacific features the a college of liberal arts and sciences called  College of the Pacific (the College), the Conservatory of Music, the Gladys L. Benerd School of Education (BSE), the Thomas J. Long School of Pharmacy and Health Sciences (PHS), the School of Engineering and Computer Science (SOECS), the Eberhardt School of Business (ESB), and the School of International Studies. Stockton is also the location of the Graduate School and the Center for Professional and Continuing Education (CPCE), which offers continuing education, extension programs, and the University’s summer sessions. The other academic units of the University are the Arthur A. Dugoni School of Dentistry in San Francisco and the Pacific McGeorge School of Law in Sacramento.

The key to Pacific’s distinctiveness is the rich array of liberal arts and professional programs that the University offers despite its modest size. Having a relatively large number of programs provides many opportunities for partnerships and collaborations within the University. The “small-town” feel of the school makes it easier to pursue these opportunities. On the other hand, the diversity of professional, graduate, and liberal arts programs means that “one-size-fits-all" approaches to its mission will rarely be practical.

University of the Pacific has grown in a variety of ways since its year 2000 accreditation review. The size of the student body has increased, as planned, by more than 1000 students since fall 2000. Most of that growth has been on the Stockton campus. Individuals and programs have won many academic awards. Selected programs and the University as a whole have attained high national rankings. Phi Beta Kappa in 2006 placed a chapter at Pacific, a signal recognition of the University’s academic quality. In pursuit of “inclusive excellence,” Pacific has deepened and strengthened its diversity. International and minority students increased from 42% to 54% over the last ten years. During the same time, minority and international faculty increased from 13% to 26%, with the percentage of female faculty rising from 33% to 41%. (CFR 1.5)

The University's guiding educational mission is “to provide a superior, student-centered learning experience integrating liberal arts and professional education and preparing individuals for lasting achievement and responsible leadership in their careers and communities.” This mission, originally drafted and approved by the Board of Regents in 1995, focuses and sharpens Pacific’s aspiration to be counted among the nation’s best universities. (CFR 1.1)

Pacific pursues this mission in curricular programs like its nationally recognized multidisciplinary Pacific Seminar general education series, which includes three thematically linked first-year and fourth-year core courses. Two first-year courses examine the question “What Is a Good Society?” while the fourth-year course prepares and prompts seniors to engage in ethical reasoning about personal relationships, career and work, and citizenship. Co-curricular programs like the Center for Community Involvement encourage students to reach beyond the gates of the University and become involved serving the communities of Stockton, Sacramento, and San Francisco. The Mountains, Ocean, Valley Experience (MOVE) is a unique first-year experience that has garnered regional and national awards from NASPA, Student Affairs in Higher Education. During the weekend before the start of the fall semester, MOVE welcomes new students to the Pacific community through participation in service projects throughout Northern California. In recognition of these and other efforts, Pacific has been named to the President’s Honor Role for Community Service. USA Today included Pacific on its 2010 list of the top 20 colleges and universities committed to community service. In spring 2011, Pacific was selected as one of President Obama’s lead universities for his Interfaith Dialogue and Community Service Initiative. Pacific’s commitment to engaged learning, whether in community service or various modes of experiential learning, has become an institutional hallmark. The University has created and maintains numerous opportunities for every student to take part in a range of experiential learning opportunities, including internships, undergraduate research, education abroad, and others, supports faculty oversight of such experiences, and encourages higher levels of community engagement.

As evidence of its financial stability, the University has continued over the past four years to manage budget surpluses, even during these challenging economic times. Moody’s upgraded the University’s credit rating to A2 ten years ago and that rating held steady during the economic downturn. Its endowment grew from $131 million in FY01 to $221 million in FY07. Moreover, the endowment has effectively recovered from losses it suffered because of the recent economic downturn. From a low of $165 million in FY09, the endowment has increased to $212 million by FY11. Pacific has been able to make annual increases in faculty and staff salaries, bringing them to and then keeping them at competitive levels relative to AAUP and CUPA ranges for school and discipline. It has recently spent over $200 million in renovation and construction of new academic buildings, residence halls, and other facilities. In November 2011, the University made a $47 million purchase of a building to establish a new San Francisco campus. (CFR 3.5)

President Pamela Eibeck, who began her tenure in 2009, and Provost Maria Pallavicini, who joined the University in 2011, are leading Pacific to address important new challenges. Among the key items on their agenda are to improve Pacific’s capacities, especially in planning and budgeting, and to renew its strategic purpose. During the last two years, Pacific has also embraced the goal becoming a more fully integrated tri-city university. The most notable step to this goal is the November 2011 acquisition of property for a new campus in San Francisco.  In addition to providing the physical foundation for the development of Dental School, the new campus will provide space for greater tri-city collaboration. Additional space has also become available on the Sacramento campus to accommodate joint programs from the other campuses.

Under President Eibeck’s leadership, Pacific has reaffirmed its determination to serve communities at the local, national, and global levels. “Beyond Our Gates,” a series of dialogues with the local community conducted during President Eibeck’s first year at Pacific, has led the University to establish the Tomorrow Project, an intensive, multi-year engagement with the region's underserved students. The primary goal of the Tomorrow Project is to increase the college readiness of K-12 students so they can successfully compete in higher education. A major faculty-led study of internationalization at Pacific was completed in 2011. The study’s results will provide the foundation for expanding global competence as another Pacific hallmark. President Eibeck has also appointed task forces on the University’s budget process and diversity and inclusive excellence. The budget task force is to make a report in late 2012 about how the University can fine-tune its budget processes for greater incentives, consistency, accountability, transparency, and efficiency. The diversity task force will propose a University strategy for inclusive excellence in a report by early 2012 (a draft will be available by the visit).

Over the course of a year, Pacific used national best practices to develop a peer data system. Three types of peers were defined. Standard peers are institutions similar to the home institution based on quantifiable institutional characteristics. Aspirant peers are generally defined as institutions that are dissimilar to the home institution but are worthy of emulating because they exemplify excellence in areas relevant to Pacific’s mission. The selection process is often more subjective. Strategic peers are institutions that have one or more compelling characteristics that the home institution would like to emulate as part of its future vision and strategy. More information about peer list development can be found at http://wasc.pacific.edu/ideav11n2.pdf.

Identifying peer institutions began with data from over 1600 private institutions collected from the IPEDS national databases. Factor analysis of this data resulted in  three factor scores that measured relative similarity, or distance, on each factor, using standardized variables (in parentheses) relative to Pacific: 1) resources (expenditures, endowment, size), 2) similar professional school profile (first professional enrollment, degrees, and similar first professional programs), and 3) undergraduate profile (full time, transfers, Pell eligible, diversity). A ranking of all institutions resulted and the list was reduced to the top 200 similar institutions.

From this point the methodologies to select standard versus aspirant peers diverged. For the selection of the 15 standard peers, the top 50 most similar institutions were selected from the list of 200. The list was reduced by removing schools whose missions were very different from Pacific, for example institutions with very high research activity or specialty schools. Additional factors such as financial strength measured by bond rating and athletics were considered in the final winnowing. To select aspirant peers, the top 200 most similar institutions were selected and the 15 standard peers were removed. The list was further reduced by applying additional criteria that included minimums for bond rating, expenditures per student, endowment per student, and maximum research activity as determined by the Carnegie classification system. The President's Advisory Council endorsed the list of the final 30 institutions, 15 standard and 15 aspirant peers. 

Each academic unit has begun to establish a set peer institutions for benchmarking. The Institutional Research Office will build and maintain the new peer system http://iris.pacific.edu/pages_public/peers.shtm  to collect, analyze, and share peer information. 

Finally, during fall 2011, University of the Pacific began work on a renewed, comprehensive strategic plan, based on Pacific’s mission and its fundamental values and commitments. The University aims to adopt a plan that will provide clear direction with measurable standards of progress to guide decision making and resource allocation in the next decade. (CFRs 4.1, 4.2, 4.3, 4.6)

1.2 Accreditation History
University of the Pacific has been a regionally accredited institution since 1949, when the Western Association of Schools and Colleges first established formal criteria for accreditation. Following a ten-year reaccreditation visit in October 1991, the Commission noted serious problems at Pacific, specifically an extremely vulnerable financial position, a lack of systematic planning, and a breakdown in shared governance. The Commission deferred the reaffirmation of Pacific’s accreditation and placed the institution on Warning status. After an October 1994 visit, the WASC Commission reaffirmed Pacific’s accreditation and removed the Warning status. In 1996 Pacific presented a presidential progress report, and the Commission canceled a planned special visit and scheduled a comprehensive visit for 1998. That visit was later rescheduled for 2000.

 The Team Report in 2000 examined areas where the 1991 Team had made important recommendations and the Commission had raised serious concerns. The 2000 Team determined that Pacific had provided substantial evidence that the University had met or exceeded expectations in responding to each area of concern. The team also offered a number of recommendations for Pacific to improve its institutional capacity and educational effectiveness.  In July 2000, the Commission reaffirmed Pacific’s accreditation and scheduled a Capacity and Preparatory visit in spring 2010 with an Educational Effectiveness visit to follow in fall 2011.

In February 2008 University of the Pacific presented an Institutional Proposal indicating that the University would follow a strategic planning approach (an option under the WASC Handbook of Accreditation at the time). In the CPR phase, Pacific would evaluate its institutional capacities relative to programs and initiatives identified as “distinctive,” “hallmarks,” or “marks of distinction.” Pacific would also examine key capacities for marketing, enrollment and financial management. Finally, the University would develop a set of university learning objectives. (CFR 2.3) For the EER phase, Pacific would examine the effectiveness of its hallmark or distinctive programs and student learning relative to the new institutional learning objectives. WASC accepted Pacific’s Institutional Proposal in April 2008 and scheduled a CPR team visit for spring 2010.

Under the leadership of its chair, President Andrew Benton of Pepperdine University, the visiting team considered Pacific’s CPR report, visited the three campuses, and offered a series of commendations and recommendations. In June 2010, the Commission endorsed the team’s recommendations, specifically highlighting areas of governance and leadership, faculty workload, student success, and program review and assessment. These recommendations and the University’s response to them are detailed below. The Commission scheduled an Educational Effectiveness Review visit for spring 2012. The commission also required the University to submit an interim report on specific governance and leadership issues by March 1, 2011. The University submitted that interim report, which was received by the Commission in June 2011.

1.3 Response to the 2010 CPR Action Letter and Team Report
In a letter dated June 9, 2010, President Eibeck outlined how the University intended to respond to the recommendations included in the visiting team report. The letter indicated both the University official or officials with principal responsibility for responding to each recommendation and the essential approach the University would take to each recommendation. In addition, in September 2010 President Eibeck convened an ad hoc WASC Action Team to oversee the University’s responses to the recommendations. The WASC Action Team comprised the University officials most involved in responding to the recommendations and others directly engaged in preparing the EER self-review. This group met monthly during the 2010-2011 academic year with the president  chairing, followed by the provost upon her arrival at Pacific. The University’s responses to the recommendations found in the CPR and Interim Report Action Letters are described in the following:

Verbatim recommendations from the WASC Action Letters are placed here in italics.

Governance and Leadership: Pacific finds itself with a new president, a recently hired vice president of development and a new dean of the Business School, and with searches underway for a provost and a high-level IT position. With this arrival of new leadership, the Commission encourages Pacific to focus on three important matters.

Pacific has an opportunity to make “a very complicated (even dense) organization pattern” more efficient and accountable. A balance must be sought between the sometimes centralized and sometimes decentralized management of the university. This includes developing a vision and plan for providing oversight and coordination amount the units of the three-campus structure. (CFRs 1.3, 3.8, 4.1)

Pacific recognizes the importance of ensuring organizational structures and services that support academic programs effectively, efficiently, and responsibly across the three campuses of the University. With the arrival of a new Provost, the university is developing a vision and plan for the academic enterprise that will drive the appropriate administrative structure and management systems to serve all academic programs, regardless of location. Pacific’s leadership recognizes the unique opportunities and responsibilities associated with a three-campus university. This geographic diversity, coupled with the breadth of program offerings, suggests that some academic support, student, and business services will be centralized and others will be decentralized. Pacific has made strides in multiple areas important for oversight and coordination between the three campuses and management of the university. (CFRs 1.3, 3.8, 4.1)

Reviews of programs and unit reviews are foundational to assessing how well the academic, co-curricular, and administrative programs and resources vertically align with Pacific’s mission and to ensure their effectiveness, efficiency and accountability. Pacific has developed and is implementing an evidence-based process for decision making to improve institutional effectiveness. The process is based on a self-study, generated by the unit/program members, of the unit’s mission, goals, initiatives, metrics for success, and action plan. The self-study and action plan form the basis for periodic program review by a team that includes members external to the unit and the campus. An Institutional Effectiveness Committee (IEC), a university-wide joint faculty/administrative committee, then considers both the quality and recommendations of reviewed programs and the action plan proposed by the unit/program to address review team recommendations. Furthermore, IEC revisits progress on the action plans of reviewed programs two years after the review. The IEC assessment of these program reviews is presented to and discussed at Cabinet, and the VP accountable for the unit includes budget recommendations for the action plan to a University wide budget committee, the Institutional Priorities Committee (IPC), a university-wide joint faculty/administrative/student committee that reviews budgetary recommendations to the President. These processes were implemented in the fall of 2011 as a 2-year pilot. (CFRs 4.1, 4.2, 4.3)

Pacific’s strategy to assess the efficiency and effectiveness of administrative services and organizations, whether they span one or multiple campuses and whether they are centralized or decentralized, is to conduct simultaneous reviews of similar functionalities or complementary academic programs on each campus. The support programs include business services (such as Human Resources and Physical Plant), Student Life services (such as Judicial Affairs and Disability Services) and Academic Support (such as the Library and Information Technology). In the case of administrative program reviews that span the three campuses, the self-study requires a reflection on the effectiveness of the unit’s management structure, whether decentralized or centralized. It also asks if efficiency and effectiveness could be enhanced with a modified structure. The reflections of the individuals providing the services and their managers, along with the recommendations of an impartial program review committee, will help the university leadership define the optimum management structure for each program and assure that the appropriate services, controls and accountability are in place. (CFR 3.8)

The Provost Office was reorganized to improve alignment with the three-campus university structure, to simplify reporting, clearly align position responsibilities with functional areas, address gaps in service, centralize accountability for university-wide academic programs, and provide university wide-opportunities for faculty to develop skills and knowledge in administrative matters. The Provost Office organization includes a new position of Vice Provost, with oversight of university-wide programs, international programs, faculty affairs and inclusivity and diversity. Faculty members serve as part-time or full time Assistant Provosts (for university-wide programs, international affairs, and faculty affairs) for three-year terms. The re-organization process was open for comment and input from the entire university for comment and input, and the new leadership positions were open to faculty members on all three campuses.

Finally, the current situation in which there is a president emeritus who reports to the chairman of the board, rather than to you as president, is not in keeping with Commission Standards or common practice. This is especially problematic in that the president emeritus has a long and distinguished career at the institution, where certainly many personal relationships were established and remain strong. Moreover, the lack of specific job requirements and responsibilities for the position of president emeritus is especially difficult to understand, and could lead to significant leadership issues in the future if not attended to early on. You and the board must resolve this relationship and authority issue, in order to assure that ultimate authority rests solely with you as the President. (CFRs 1.3, 3.8, 3.10)

With respect to the position of president emeritus generally, the Board of Regents have amended the University’s bylaws to provide the Board with the authority to designate a president who has served with distinction the title of president emeritus, and to clarify that a president emeritus reports to the president, or his or her designee. The post-presidency agreement between the University and the current president emeritus has been revised to specify this reporting relationship. That agreement already set forth the job requirements and responsibilities of the position. More detail on and documentation of these provisions were contained in a confidential addendum to the 2010 Interim Report. The Interim Report was made public to the University’s broad public constituencies, while the addendum remains confidential to the Commission, Commission staff, the visiting team, and the Interim Report Committee because it contains confidential information related to employee contractual matters. (CFRs 1.3, 3.8, 3.10)

Faculty Workload: the CPR team found that the Stockton faculty and staff broadly report that they feel overextended, citing a combination of instructional demands, high enrollments, and “mushrooming of initiatives,” added to the faculty’s commitment to personal interactions with students and the work required for advancement toward promotion and tenure. “Workload fatigue” can lead to problems of morale and to the inability for, or resistance to, the assessment work that is required for an institution to be considered educationally effective. Pacific should address faculty workload in a systematic fashion, considering issues of class size, number of initiatives, time for scholarship, and the standardization and implementation of equitable rank, tenure, and promotion processes and practices. (CFRs 2.8, 2.9, 3.1-3.4, 3.11, 4.1, 4.2, 4.6, 4.7)

Three separate analyses of faculty workload have been conducted since the CPR visit. First, the Academic Council surveyed the full time faculty in spring 2011 and an analysis was conducted by faculty leadership. (CFR 3.11)The faculty survey requested responses to a number of questions focused on how faculty spent their time. Faculty reported spending most of their time in preparation for scheduled classes and in scheduled teaching.

Second, the institution administered the HERI Faculty Survey in spring 2011, which provided important comparative data to similar institutions. Faculty respondents at other four year institutions spent similar effort in various activities. Almost 70% of Pacific faculty spent at least 9 hours/week in teaching preparation compared to 57.3% of faculty at other four-year institutions. Almost half of faculty at Pacific (45.6%) spent more than 9 hours per week in actual teaching compared to 39.8% of faculty at normative institutions. The Pacific faculty work on committees and in meetings was reported as less than that of peers. Only 2.6% of Pacific faculty spent more than 9 hours/week on committees compared to 11.0% of faculty at other institutions. Almost two-thirds of faculty at Pacific reported spending only 1-4 hours per week on committee work.

Finally, the Office of Institutional Research completed a quantitative analysis of faculty instructional activity by program, rank, and department in fall 2011. This analysis revealed that the faculty teaching loads for lectures, labs, and seminars are generally consistent with the stated workload policy for the unit, although there are variances among schools and rank in workload and in the number of courses taught at each rank. With few exceptions, the average class size was below 25. The average number of non-lecture courses, labs, and seminars ranged between 1-10.5, with considerable variation among schools and rank. With the exception of Pharmacy and Health Sciences and Dental Hygiene, class size was relatively small. Deeper analysis of the non-lecture courses revealed that a considerable number of non-lecture courses were considered experiential learning, in which a faculty member supervises an individual student’s internship or undergraduate research project.

Overall, the workload analysis revealed that Pacific faculty members spend the majority of their time teaching or preparing for teaching, and that a relatively small fraction of the faculty is involved in committee work exceeding 9 hours/week. Data also indicate some disparities within some schools in the number of courses taught at different ranks, the size of the courses taught at different ranks, and the variation in the number of experiential courses. Many faculty members receive additional compensation for experiential learning courses. Overload teaching is also compensated, yet overload courses are not yet identified in the Banner database. The University’s Institutional Research office is working to disaggregate some of these data. In addition, the deans are being asked to carefully analyze the data based on rank, school, and discipline to ensure equity and to report an action plan to address inequities to the Provost in spring 2012. While time spent in overload teaching, experiential learning, and summer school teaching provides the faculty additional compensation, it also affects the time available for scholarship, which is important for the teacher-scholar model and for faculty advancement. The Council of Deans will carry out additional analysis of workload policies, in conjunction with the Provost, to develop strategies to ensure success of junior and mid-career faculty and improve equity in instructional workload. As a final note, Pacific will take part in the next Delaware Study of Instructional Costs and Productivity to gain more insight into the relevant data in relation to similar institutions. (CFRs 2.8, 2.9)

Student Success: Defined in terms of persistence, retention, and graduation rates, overall and disaggregated, student success is one measure of the effectiveness of a university. It is also important to an institution’s reputation and marketing success. Pacific’s graduation rates vary among ethnic groups. Pacific should examine its persistence, retention, and graduation rates vis--vis the amount of its financial aid and its financial aid practices, especially as financial aid affects diversity. The team recommended that Pacific have a “serious conversation” about diversity. Distinguishing global diversity from domestic diversity, it is recommended that Pacific develop a strategy for domestic diversity which will also respond to this issue as stated in the 2000 WASC review. Pacific should also respond to the request of commuting and transfer students for places where a sense of community can be developed. (CFRs 1.5, 2.10, 2.11, 2.13, 2.14, 3.7)

In response to the WASC visiting team, Pacific has conducted extensive research to understand how well it achieves its student success goals and evaluate the impact of its strategies to improve student success.  Special attention has been given to differences among ethnic and socioeconomic groups, differences by academic programs, and the impact of financial aid policies. Results suggested that financial aid practices are not correlated with student success by race, ethnicity, or income level. While graduation rates based on gender, ethnicity, and program do vary, the variation is not greater than observed among peer institutions and nationally. (CFRs 1.5, 2.10, 2.13)

The Retention Analysis Network (composed of assistant and associate deans, academic support, institutional research, and enrollment staff, and led by the dean of students) continues its community of practice approach to the study of student persistence and attrition and institutional support programs designed to increase retention. Section 2 below more fully reports on the University's inquiry regarding   Student Success, including plans for further efforts to improve student success. 

Pacific continues its serious conversation about diversity with the work of the Task Force on Diversity and Inclusive Excellence, and the institution has broadened its approach to diversity to include a variety of strategic working groups in addition to the research and analyses mention above. All of this continues Pacific’s history of valuing and pursuing diversity and its recent history of advancing diversity goals, values and learning outcomes. Recent initiatives such as the diversity curriculum requirement; diversity training for hiring practices; Difference, Power, and Discrimination Summer Institutes; diverse faculty mentoring program; cultural programming that includes the community of Stockton; activation of a  bias response team; Visions Dialogue training and programming; a widespread SAFE program; and award-winning programs such as Pacific’s selection as a model program in President Obama’s community service and interfaith dialogue initiative; regional leadership and conferences for the  lesbian, gay, bisexual, and transgender community; and a growing number of gateway programs and support programs for low income and minority youth in the community, all show that Pacific has an authentic commitment to developing and supporting a diverse learning community.

Assessment of Student Learning and Program Review: Assessment of student learning, along with assessment of learning outcomes and student success, is necessary for program review, which in turn is required to demonstrate educational effectiveness under Commission standards. The team found that “progress on assessment of student learning at the program level is insufficiently developed and needs to be addressed…and constitutes a significant challenge for UOP as it approaches its Educational Effectiveness Review.” It is expected that UOP will have student, course, program, and institutional learning outcomes developed by the time of the EER and that the learning outcomes will be measured and implemented. A systematic program review process and schedule is also expected by the time of the EER, as well as completed program reviews available for in-depth review by the EER team. (CFRs 1.2, 2.3, 2.4, 2.7, 4.4, 4.6, 4.7)

Programmatic accreditation and review of individual academic and administrative programs occur on a rolling basis. Program review provides the opportunity for an academic unit to reflect on its progress toward achieving its educational objectives for students, its progress toward its strategic plans, and ways it can improve its operations. In the case of the supporting units, program reviews allow reflection on their contributions toward institutional learning objectives and their strategic goals and continuous improvement opportunities. Each Department, School or College, and each support unit is expected to prepare a self-study and undergo program review according to a schedule maintained by the Provost’s Office and accountable to the President. The institutional-level review of these program reviews and assessment occurs through the Institutional Effectiveness structure led by the Provost. (CFRs 2.11, 4.1, 4.4, 4.6, 4.7, 4.8)

Regarding assessment of student learning, University of the Pacific’s academic leadership has decided to employ a professional development approach, working with faculty and staff at the program level to refine assessment plans, build expertise in student learning assessment, and grow support for an evidence-based approach to improving student learning. The University made that choice based on the understanding, consistent with research on individual and organizational learning, that progress would be most sustainable if it worked closely with faculty to help them develop meaningful outcomes, assessments, and the skill to manage assessment for program improvement over time. (CFRs 4.6, 4.7)

At Provost Gilbertson’s direction, the University formed an Assessment Working Group (AWG) led by the Director of Learning and Academic Assessment, with representation from the schools and the library, the Director of General Education, the Director of Writing in the Disciplines, and the Director of Assessment for Student Life. The main work of the Assessment Working Group since its formation in the spring of 2010 has been to promote communication about assessment progress and practices among schools and units, to broaden and deepen organizational learning about Pacific’s professional-development approach to assessment, to bring specificity and clarity to assessment of university learning objectives and outcomes, and to facilitate revision of General Education outcomes consistent with university learning objectives. Details and results are included in the section below on assessment of student learning and on the sustainability of assessment and educational effectiveness. (CFRs 4.6, 4.7)

Based on an evaluation of needs among the different units, the academic division focused on developing faculty knowledge, skill and assessment practice within the College of the Pacific, Pacific McGeorge School of Law, the Conservatory of Music, the School of International Studies, and the Library. The Conservatory continues to grapple with assessment at the school and program levels, but has made measurable progress in recent months. Assessment of individual student development and achievement continues to be robust in the Conservatory. The College, McGeorge, SIS, and the Library have all developed program outcomes and have implemented assessment, generated actionable evidence, and are using findings for program improvement as well as continuing to develop assessment skills, practices, and further evidence. The Center for Professional and Continuing Education has incorporated outcomes and assessment in new program development. Other schools and units, including SOECS, BSE, PHS, Dentistry, and ESB, as well as the Division of Student Life, all have well-developed objectives and assessment of student learning. (CFRs 2.3, 2.4, 2.6, 4.7)

Similar to the academic division, the division of Student Life has formed a committee called CASE, Committee on Assessment of Student Experiences. The vice president chairs the committee, which meets each fall with each of the 16 department directors to review and provide coaching on annual assessment plans. Each Student Life director serves on another department’s assessment team, in addition to overseeing learning assessment in their home department. Directors are responsible for explaining to their colleagues the learning that they hope to facilitate for students, and cross-department understanding. Collegiality is promoted as departments share and invest in a strong commitment to student learning.

As detailed below, Pacific has identified several aspects of its learning assessment work that should be improved. The University has begun to develop more consistent approaches to including information about learning objectives, assessment, and student learning outcomes in the Catalog, on the public website, and through other media.  Other areas where work is in progress include engaging students with assessment practices and establishing expected levels of mastery for programs and for university level objectives. 

Planning Processes: The interim report panel observed that there are several major planning processes going on simultaneously and noted that it will be a challenge to move them all forward effectively. Observing also that WASC expects that “the institution monitors the effectiveness of its plans and planning processes, and revises them as appropriate,” the panel recommends that Pacific implement its processes for review and realignment in a timely fashion so that the vision and the plan will be achieved within the proposed timeline. The panel also encourages establishing process goals that include reports on the achievement of the sub-goals identified in the timeline, noting that establishing the habit of reporting on sub-goals, timelines, and accountability will be useful in preparing reports for the EER visit. (CFRs 4.1-4.3)

The University has launched a significant strategic planning process to be completed by October 2012. The timing of the varied but closely related processes of strategic planning, accreditation review, budget system remodeling, and institutional effectiveness has been seriously considered and the resulting design carefully detailed to assure success. This planning process will engage a cross-section of the University community, including external stakeholders and friends of Pacific. The institution will develop a new strategic plan that gives Pacific more options and is more resilient to external changes, supported by a corresponding financial plan, school-level strategic plans, and an implementation and assessment framework. (CFRs 4.1-4.3)

Engaging in several simultaneous planning processes offers significant advantages to the University. If a strategic plan is to be valuable to a university, it must address the fundamental relationship of the institution to its rapidly changing environment. Political, economic, competitive, social, demographic, technological, and environmental factors are changing so quickly that a plan approved just five years ago cannot chart an adequate course. However, the University is not simply responding to environmental challenges. An additional compelling reason for new strategic thinking is the opportunities provided by expanding operations in San Francisco, shifts in the Sacramento and Central Valley markets, and Pacific’s growing global interests. By capitalizing on these and other opportunities, leveraging and overcoming existing weaknesses, and responding to external challenges, the resulting strategic plan will provide a position of strength for the University in the coming decade and beyond.

A key outcome of the planning process is alignment both horizontally across the important segments of planning activities and vertically from the program and course level through the colleges and schools, across the three campuses, to the central administration and the Board of Regents. Pacific recognizes this as a significant opportunity to address the administrative challenges the Commission noted in its Action Letter, and the University takes an intentional approach to alignment of these planning activities.. This figure depicts the alignment of planning, strategy, and actions of these important processes.

The strategic planning process was designed in four phases with distinct activities and expected achievements. The first phase consisted of designing the strategic planning process and scanning the environment for key factors influencing the University of the Pacific today and in the future. In the second phase, faculty, staff, and students engaged in envisioning the future of the University and its programs and disciplines in the next ten years and generating a series of working papers that outlined potential strategic opportunities. By the third phase, the strategic planning committee became the central node of the process, culling data from the communities of the three campuses and processing rounds of feedback. The important result of this phase will be the filtering, selection, and articulation of the primary strategies that will form the new plan. The final stage of the planning process will focus on the completion of the planning documents to include the planning timeline, specific goals and sub-goals as appropriate, the associated analytics, and the all-important financial strategic plan, which will outline the resources necessary to achieve the anticipated results. Fifteen highly important planning processes intersect to form an integrated whole. These processes are moving forward effectively, achieving results on time, and show a high degree of coordination.

Budgetary System: As noted in the Interim Report, currently “there are different budgetary allocation models at the University,” and a task force has been formed to study budgetary models and recommend a new budget allocation system. The goal is that “all units of the university will operate under the new budgetary allocation system, regardless of geographic location.” According to the Interim Report “the budget system transition will occur over several years,” and once adopted “two parallel budgetary systems will be in place for the first year.” The panel cautions that running a parallel budget could overtax the university, and recommends that UOP move quickly through the beta process. (CFRs 1.3, 3.8, 4.2, .4.4)

Additionally, the Action Letter directed attention to the budgetary system. The intention to move to an alternative budgetary system, as described by you to the panel, may have significant implications to the institution. Given the significance of the possible change, it will be important for the University to include in its EER report a description of the planned budgetary system and an evaluation of its effectiveness, if implemented at that stage. (CFRs 1.3, 3.8, 4.4)

President Eibeck charged Pacific’s Budget Task Force on January 5, 2011. According to her instructions the task force will first review the University’s current budget model, primarily incremental budgeting for the Stockton campus (with the exception of the Center for Professional & Continuing Education, which uses a hybrid of incremental and responsibility centered management) and responsibility centered budgeting for the Sacramento and San Francisco campuses. They will also study and evaluate alternative budget models, and recommend a new unified budget model that will both incentivize and hold accountable the University’s budget units. The work of the task force will help Pacific meet its goal to become a less complex and more efficient organization while also increasing oversight and coordination across the three campuses. (CFRs 1 .3, 3.8, 4.2, 4.4)

The task force began its work by exploring the strengths, weaknesses, and vulnerabilities of the University’s budget processes. The task force thoroughly examined the current budget, facilitated by a series of presentations from the University’s budget office. In-depth presentations from Law and Dental School budget officers supported these presentations. The task force also received presentations about the University’s cost allocation model (CAM) and conducted a thorough review of its implications. The task force staff has developed a library of materials about various academic budget models and established an online archive of relevant materials.

The task force has reviewed how University resources are generated and used, and it has discussed what incentivizing, accountability, and transparency might mean at Pacific and what mechanisms might be developed to pursue these objectives. Additionally, the task force better understands the University’s cost centers and the responsibility to both care for and incentivize these units.

Fall 2010’s efforts reviewed additional budget models and held discussions with possible consultants. These consultants will receive budgetary, academic, and organizational information about Pacific. The task force anticipates obtaining a written report with recommendations from each consultant by February. At that point the task force will choose a consultant to help develop a revenue formula that supports the University’s objectives and incentivizes units to be innovative, adaptable, and accountable. The task force will then be able to recommend a budget allocation model for the campus to consider by the fall of 2012.

The University will need time to carefully consider a new budgetary system and to prepare for its implementation. Hence, the budget system transition will occur over several years. Following the completion of the strategic plan, the Budget Task Force will recommend a new budgetary model aligned to achieve the strategies set forth in the plan. The following year the University community, including the Board, will vet the plan and develop a detailed implementation and evaluation strategy. Adopted through the formal governance process, the new budgetary system will then be introduced over multiple years with expected completion in 2015. (CFRs 3.5, 4.1, 4.2, 4.4)

A tentative timeline for this process is as follows:

Fall 2012

Task force recommends budget allocation model for campus to consider. Speakers visit campus to discuss budget model in open forums and small groups; campus feedback solicited.

January 2013 Board reviews proposed budgetary model.
Spring 2013 Budget task force refines budget model based on input. Budget Implementation team put in place. Simulation to approximate budget allocation for current (FY14) created for budget managers.
2014 Current budget allocation model remains in place. Training and preparation for new budget allocation model occurs.
2015 New budget model implemented.

1.4 Development of Pacific’s Educational Effectiveness Review


University of the Pacific initially proposed two sets of questions for the Educational Effectiveness Review. These questions were related to the University’s strategic priorities at the time Pacific began its accreditation self-review. As originally proposed these two sets of questions ran parallel to inquiries proposed for Pacific’s Capacity and Preparatory Review.

The University proposed to examine programs or initiatives that the former strategic plan, Pacific Rising, referred to as “distinctive,” “hallmarks,” or “marks of distinction” to determine how effectively these designated programs or initiatives serve to distinguish Pacific from other institutions and to evaluate the contribution of these programs and initiatives to student learning and faculty and staff development. Specifically, Pacific proposed to address the following questions:

An inquiry team led by the chair of the Earth and Environmental Sciences Department was charged with carrying out an inquiry to explore those questions. In the course of their work, the team elected in consultation with the interim provost and the WASC Action Team to focus on select graduate programs, centers, and institutes.  This inquiry team’s report provides the basis for section 1.5, entitled “Pacific’s Unique Strengths.”

The second set of questions addressed the university-wide student learning objectives that Pacific adopted during its CPR self-review. The University proposed to examine how well the new university-wide objectives had been integrated into programs across the University, what had been learned from early efforts to assess student learning relative to these objectives, and what larger impacts could be felt from the adoption of university-wide learning objectives. Pacific proposed the following specific questions:

In its CPR report, the visiting team held that “assessment of student learning at the program level is insufficiently developed and needs to be addressed.” The Commission’s action letter endorsed this view and added that “[i]t is expected that UOP will have student, course, program, and institutional learning outcomes developed by the time of the EER and that the learning outcomes will be measured and implemented.” Accordingly, the inquiry team charged with examining this set of questions in consultation with the interim provost and the WASC Action Team developed a revised set of questions to guide its EER work. The new set of questions directed the team to examine assessment of student learning, not just in terms of the university-wide student learning objectives, but also to evaluate the state of assessment of student learning at the program level. The specific set of revised questions follows:

  1. Has each academic program at the University developed program-specific outcomes (or objectives)?
  2. Are the university-wide objectives incorporated into academic and co-curricular program-specific outcomes and program components (curriculum, courses, activities, etc.)? What curricular mapping of objectives and outcomes has occurred?
  3. What types of rubrics and other measures are used to assess the program-specific outcomes and university-wide objectives within programs?
  4. What are the results of assessments of program-specific outcomes and university-wide objectives?
  5. How have those assessments and results been shared and used in decisions about programs (courses, curricula, activities, etc.), resources, planning, faculty and staff development and training, student support services, etc.? What has been learned from those assessments?
  6. What plans are in place to enhance the assessment of student development and learning?

An inquiry team led by the associate dean for academic affairs in the Thomas J. Long School of Pharmacy and Health Sciences carried out this investigation.  This team’s report and an inventory of assessment practices developed by the team provide the basis for section 3.2, entitled “Pacific’s Assessment Systems and Practices.”

Additional inquiry teams were charged with examining matters that WASC requires institutions to include in the EER report. Specifically, these are student success at Pacific, program review at Pacific, and Pacific’s systems for enhancing educational effectiveness and student learning.

An inquiry team led by the dean of students, working closely with the director of institutional research, undertook the investigation of student success at Pacific. The goal of this team was to deepen Pacific’s analysis of the success of its students compared to the University’s own goals and the performance of benchmark institutions. Following the WASC Handbook of Accreditation, the inquiry team was charged with addressing the following questions:

  1. How has Pacific deepened its analysis and understanding of retention and graduation since the CPR?
  2. Has Pacific acted on plans that arose from the CPR?
  3. What is Pacific’s assessment of its progress in promoting student success?
  4. What does Pacific’s analysis of year-to-year retention and graduation data show?
  5. How do these data compare with other institutions?
  6. What do results from campus climate surveys or other inquiries into Pacific’s educational effectiveness tell us?
  7. What more should Pacific be doing to improve retention and graduation rates, time to graduation, and other indicators?

In addition, the CPR recommendations of the visiting team and the Commission indicated that the institution should give specific attention to the student success dimensions of diversity and the impact of financial aid policies and practices on student success. This team’s report is the basis for section 2, entitled “Student Success.”

A team led by the chair of the University’s Academic Affairs Committee (a professor of electrical and computer engineering) conducted the inquiry into Pacific’s program review policies and practices. The team explored the following questions to determine how Pacific’s program review policies and practices are aligned with WASC’s expectations:

  1. Does the program review process meet the expectations reflected in the WASC Rubric for Assessing the Integration of Student Learning Assessment into Program Reviews?
  2. Are all academic and co-curricular programs subject to program review?
  3. Is program review conducted in a timely manner and in keeping with good practice?
  4. Is program review used to assess program effectiveness and student learning at the program level?
  5. Is it used to improve program effectiveness?
  6.  Is it used to align resources with needs?
  7.  How is program review articulated with the budgeting process?
  8.  Is the program review process itself reviewed on a systematic basis?
  9. Are recent program reviews available to the WASC visiting team?

This inquiry team’s report substantially informs section 4, entitled “Program Review.”

A final inquiry team, led by the chair of the History Department, was charged with evaluating Pacific’s systems for enhancing educational effectiveness and student learning, with a view to determining how sustainable Pacific’s approach to educational effectiveness is. This team examined the following questions:

This team’s report informs much of the discussion throughout the EER report about Pacific’s approach to assessment and educational effectiveness.

The chair of the Political Science Department with support from the Provost’s Office chairs the overall self-review process, including the preparation of the report. The institutional research director leads a Data Support Team. These two individuals, along with the director of learning and academic assessment and the chairs of the five inquiry teams, constitute a Core EER Committee, which coordinated information, effectiveness measures, and progress in meeting self-review deadlines.

A University Steering Committee provided oversight for the self-review process and gave formal University endorsement of the EER report. This steering committee is broadly representative, including faculty, staff, administrators, students, and alumni. All divisions of the University, including Athletics, are represented on the University Steering Committee, as are the three campuses. (CFR 1 .7)

Following the CPR phase of Pacific’s reaccreditation review, President Eibeck convened a WASC Action Team. This group met monthly and, among other functions, received periodic progress reports from and offered advice to the five inquiry teams and the self-review chair.

The self-review chair, working closely with the provost, prepared and circulated a draft of the Educational Effectiveness Review report to the University community in November 2011. Members of the University community were invited to review and comment upon the draft until its finalization in late December. The self-review chair reported on the draft report in meetings with each academic unit and also reported regularly to the Academic Council.  He also reported to other groups, including the Alumni Association Board and the Board of Regents. 

1.5 Pacific’s Hallmark Strengths
As detailed above, Pacific embarked on the educational effectiveness review intending to examine programs and initiatives that might serve as marks of distinction for the University. Over the course of inquiry, those working on the EER report, including two provosts and the president, carried out many hours of intense discussions about which programs and initiatives would most meaningfully be presented as truly distinctive since the initial list of items that Pacific proposed to examine, including all graduate programs and centers and institutes, was too long to allow careful inquiry. Those discussions about distinctiveness enhanced the University's understanding of the characteristics that define its signature approaches to higher education and student learning. Three points are especially salient. One, the unusual configuration of Pacific—a relatively small university, with a relatively broad and rich array of liberal arts and science, graduate, and professional programs distributed across three Northern California cities—provides the faculty and students many opportunities to realize synergies across units, disciplines, and divisions. Various cross-university programs and cross-unit academic programs show how fruitful these synergistic efforts can be. Two, Pacific features programs that reach out and connect to broader communities. This is a reflection of the University’s roots in Northern California, especially its home base in a region that faces many severe challenges; the University’s mission to educate people who will provide responsible leadership in their careers and communities; and President Eibeck’s commitment that the University be an asset for community development. Three, Pacific’s long-standing commitment to integrating liberal and professional education has resulted in first-professional programs with distinctive educational approaches. (CFR 1.1, 2.11, 4.1)

Synergies across Units, Disciplines, and Divisions: Experiential learning, leadership development, and a commitment to intercultural competence and international education are prime examples of university-wide programs and initiatives that build on cross-university synergies. While responsibility for discrete aspects of program development and implementation is distributed across many programs, the scale of the University permits and encourages faculty, staff, and administrators to collaborate with one another and learn from one another. A fall 2011 inventory of faculty-Student Life collaborations provides some evidence of the breadth and depth of shared work and the diverse partnerships that commit to student learning outcomes at Pacific.  Over 140 faculty members or academic administrators and dozens of student life personnel have collaborated on more than 60 projects during 2011.

Experiential learning is a high-impact educational practice, and Pacific’s commitment to experiential learning is implicit in the University’s goal to develop practice-ready graduates. Pacific has set a goal of 100% student participation in experiential learning. Given the variety of programs at the University, the institution has adopted a broad understanding of experiential learning based on program appropriateness. For many programs, this means undergraduate research; for others internships, clinical experience, or education abroad. While the University has not yet reached the goal of 100% participation, levels of participation are encouraging. In any given semester about 25% of juniors and seniors are registered for an experiential learning course. A substantial majority of those earning bachelor’s degrees have participated. The Graduate Success Survey shows that, for the class of 2010, 55% of graduates, up from 43% of the class of 2009, had completed an internship, one form of experiential learning at Pacific. In addition, almost all graduate and professional degrees require some form of experiential learning. (CFRs 2.1, 2.2,)

Graduate and professional programs generally carry out assessment of student learning in experiential opportunities in order to meet the expectations of specialized accreditation. Data collected for  the assessment section of the report confirm that most graduate and professional programs assess experiential learning activities. Assessment of experiential learning in undergraduate programs is less consistent, but is likely to grow as assessment plans in those programs mature. Depending on the level of development of a program’s assessment plan, experiential components may or may not be formally assessed. Under a new organizational plan for the Provost’s Office, Pacific will appoint an Assistant Provost for University-Wide Academic Programs. One of the responsibilities for this new position will be to promote continuous development and assessment of experiential learning programs. (CFRs 2.4, 2.6, 2.7, 4.4, 4.6, 4.7)

Leadership development is a widely shared educational goal at Pacific. “Leadership and Collaboration” is one of the university-wide learning objectives and is a major initiative in Student Life co-curricular and curricular learning programs. Roughly two dozen academic and co-curricular programs reported on outcomes related to leadership and collaboration in their most recent assessment reports. Attention to leadership begins with the Mountain, Ocean, Valley Experience program for entering undergraduate students, including assessment and required reflections. Comparison of Pacific’s 2010 graduating cohorts responses on the Freshmen Year survey with their responses to the Senior Survey reveal that Pacific graduates show a larger increase in rating their leadership ability than students at other private schools and at all colleges and universities. Pacific participated in the 2011 Multi-Institutional Study of Leadership (MISL), which also indicates that Pacific students are more likely to report growth in their leadership characteristics than students at other institutions. The MISL results suggest that active participation in leadership activities is a more powerful learning experience than leadership courses and classes. The inquiry team report on leadership development concludes that a “community of practice” has developed around leadership development courses and programs. Already groups of faculty and staff are collaborating on developing a draft collaboration and leadership rubric that might be adopted across programs. Other examples in the team’s report show faculty and staff leveraging cross-university synergies to improve assessment efforts and student learning about leadership development. For example, faculty from various disciplines associated with the Jacoby Center for Public Service and Civic Leadership are developing a unified approach to assessing each of seven distinct leadership programs. (CFRs 2.3, 2.4, 2.6, 2.7, 2.10, 2.11, 4.5, 4.6, 4.7)

Intercultural and global learning is another university learning objective. This is reflected in the programs of all academic units, including General Education, and across co-curricular programs. Both domestic diversity and global learning are included in this objective. The University has undertaken a number of initiatives and supports a number of programs to promote learning in this area. A recent survey of Pacific’s faculty reveals the breadth and depth of international experience and expertise at the University. The survey also indicated considerable faculty support for enhanced international activity. As referenced earlier in this report, President Eibeck has appointed a Diversity Task Force to prepare for the adoption of a Diversity Strategic Plan and an Internationalization Steering Committee to develop a strategic action plan for the further internationalization of the University.  (CFRs 1.5, 4.6)

Over the years Pacific has taken various steps to promote student learning regarding various dimensions of diversity, including race and ethnicity, sexual orientation and gender identification, first generation and social class status, different physical and emotional abilities, and religion or spirituality. Both the divisions of Academic Affairs and Student Life have diversity officers at the assistant vice president level. A rich array of student organizations encourages learning about differences rooted in these many dimensions. Student Life maintains an Office of Multicultural Affairs and an Inter-Faith Chaplaincy for the University. Like all Student Life departments, both offices conduct regular assessment and undergo periodic program review. In 2009, the faculty adopted a curricular diversity requirement for all undergraduate degrees. Pilot assessments have taken place in courses offered during 2010-2011. Assessment of PSYC 029 (Developmental Psychology) showed increases in students’ understanding of social difference.  Assessment of student work in ETHN 011 (Introduction to Ethnic Studies) indicated that 90% of students had mastered the ability to apply critical race theories.  Assessment of diversity learning was a theme in the October 2011 University Diversity Retreat. (CFRs 1.5, 1.2, 2.3, 2.6, 2.11, 4.4, 4.6, 4.7)

The School of International Studies offers courses in cross-cultural training for students participating in Education Abroad and has used the Intercultural Development Inventory to assess intercultural competence of those students; this evaluation suggests greater inter-cultural learning when education abroad is complemented with pre- and post-experience cross-cultural training. The Pacific Institute for Cross-Cultural Training offers various activities to promote inter-cultural competence for Pacific faculty, staff, and students.

Academic Programs Based on Cross-Unit Synergies: Cross-unit academic programs are visible signs of the collaboration across units and programs that Pacific’s distinctive structure enables. Developing and sustaining such programs is one way the University pursues its mission to integrate liberal arts and professional education. Accelerated pre-professional programs that provide avenues for highly qualified Pacific students to begin professional dental, pharmacy, or legal education after two or three undergraduate years are perhaps the most visible of these. Students who accelerate through the Pre-Dental and Pre-Pharmacy programs have similar success rates in their professional programs compared to students with a more traditional undergraduate experience. The first cohort of accelerated Pacific Legal Scholars is just in its first year of law school, and it is too early to evaluate their success in law school. However, the Pacific Legal Scholars (PLS) program has implemented an assessment plan. Among other conclusions, the PLS assessment shows that students become more adept at briefing cases over the course of PLS courses.  PLS assessment has also led to course-level changes, e.g. in how students are instructed about the elements of a case brief.  Several graduate programs are built on cross-unit collaborations. These include the Pharmaceutical and Chemical Sciences Program (MS/PhD), the MA in Educational Administration and Leadership with a concentration in Student Affairs, a partnership between the Benerd School of Education and the Division of Student Life, a Doctor of Pharmacy/MBA option designed to prepare pharmacy graduates for industry leadership, and the Doctor in Professional Education and Leadership Program carried out by the School of Dentistry and the School of Education. The assessment activities associated with these programs is variable; some have well-developed assessment plans and activities for the integrated program and others relying on assessment of individual components.(CFRs 2.1, 2.3, 2.,4, 2.5, 4.6, 4.7)

Connecting to Broader Communities: Notable among the programs that connect Pacific to larger communities are the Brubeck Institute, the Business Forecasting Center, and the John Muir Center. The Brubeck Institute was established to honor the legacy and build on the work of Pacific Alumni Dave and Iola Brubeck. The Institute’s programs have brought scores of exceptional young jazz musicians to Pacific for intense periods of professional development; these programs have garnered favorable attention on a national basis, including awards from Downbeat magazine. The Business Forecasting Center produces quarterly economic forecasts for California and 10 of the state’s metropolitan areas. The Center has also been very successful in connecting Pacific to larger communities and has received considerable favorable media attention. The challenge for both the Brubeck Institute and the Business Forecasting Center is to integrate their activities more effectively with the academic units to which they report, the Conservatory of Music and the Eberhardt School of Business, respectively. Both units are exploring how this can be done. The University is drafting a new policy that will include evaluative criteria and systematic procedures for program review of centers and institutes.  (CFRs 4.1-4.4)

The John Muir Center was established in 1989 to foster a closer academic relationship between the John Muir Papers Collection at Pacific and the larger community of scholars, students and citizens interested in regional and environmental studies. The Muir Papers Collection includes most of the famous naturalist’s manuscripts, personal papers, and personal library. The Center hosts a biennial Muir conference and many Muir-related events. Its materials are used widely, for example in a series of PBS documentaries on Muir and the National Park system and a 2011 exhibit at the Oakland Museum. The Muir Center also supports learning on campus. Students in various classes use the Muir materials for research projects, and the Center is integrally involved in the MOVE (Mountain, Ocean, Valley Experience) program that involves all new Pacific students in environmental service projects.

Finally, the President’s “Beyond Our Gates” initiative, launched in 2010, is a collaborative effort among University of the Pacific and individuals and organizations from throughout San Joaquin County, seeking ways of working together to improve the social and economic wellbeing of our region. The Tomorrow Project, focused on improving K-12 performance in STEM fields, emerged as a high priority from community-faculty forums and spawned academies in partnership with the San Joaquin School district and the Stockton symphony.

Distinctive Approaches to Professional Education: The Dugoni School of Dentistry and Pacific McGeorge School of Law have both taken distinctive approaches to professional education. The Dental School has long been noted for its humanistic approach to dental education, an approach grounded in the core values of humanism, innovation, leadership, reflection, stewardship, collaboration, and philanthropy. Recently the School has implemented the Pacific Dental Helix Curriculum, which places a strong focus on active learning and critical thinking by integrating concepts across multiple disciplinary areas and using small-group case-based learning as signature pedagogy. The school assesses its approach through a variety of direct and indirect measures.  High licensure pass rates and the low level of professional ethical complaints filed against Dugoni graduates are among the measures of the program’s effectiveness. (CFRs 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 4.6, 4.7, 4.8)

Pacific McGeorge School of Law has long been a pioneer in internationally focused legal education. Since 2004, the School has focused even more intently on globalizing its curriculum. International and comparative law are integrated into most required courses, including Constitutional Law, Corporations, Property, Civil Procedure, Contracts, and Torts. All students encounter international research and foreign legal systems in the two-year Global Lawyering Skills courses. These curricular innovations have resulted in the promotion of the School’s international scholarship and the publication of the Global Issues book series. These casebooks follow McGeorge’s curricular approach in integrating comparative, international, and transnational legal materials into texts for traditionally domestic core courses. McGeorge’s distinctive approach has brought a favorable reputation as the School’s international program is regularly ranked in the top 20 nationally. Like many law schools, McGeorge has been a relative latecomer to assessing student learning. However, faculty and administration at the School have now designed and begun to implement a comprehensive approach to outcomes-based learning assessment. As a result, McGeorge has become a leader among ABA schools in assessing student learning. (2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 4.6, 4.7, 4.8)

Pacific’s inquiry into the effectiveness of its prior focus on distinctive programs has helped clarify its hallmark approaches to higher education. Overall, these hallmark approaches show how individual  and collaborative efforts involving curricular and co-curricular programs can be developed to meet the University's goals and objectives and contribute to student, faculty, or staff development. While some areas of good assessment practice have been applied to the programs and initiatives discussed above, significant work remains so that Pacific can systematically evaluate the long-term effectiveness of initiatives like experiential learning. The University will develop models of assessment for experiential learning that may be adopted at the program level. The University will also continue to examine how to achieve its goal of 100% student participation in experiential learning. Pacific will conclude strategic plans for diversity and internationalization and implement those plans, including assessment of their educational effectiveness. Pacific will also develop an approach to the assessment of accelerated pre-professional programs. Finally, the University will implement evaluative criteria for centers and institutes and include centers and institutes in the program review/institutional effectiveness process discussed in this report. (CFR 4.7)

2. Student Success

Current Assessment of Undergraduate Student Success at Pacific: University of the Pacific takes seriously the importance of understanding the factors that affect student success and intervening within its capacity to improve persistence, graduation rates, and time to degree. The University considers persistence and graduation rates to be Key Performance Indicators.  In her memo regarding Planning & Budgeting for Fiscal Years 2013 and 2014, President Eibeck listed improving student success second among five key goals guiding budget and planning decisions. Several recommendations to improve student retention have been submitted through the annual budget process. (CFRs 2.7, 3.8)

Pacific's goal for the first- to second-year persistence rate is 90%, and the six-year undergraduate graduation rate goal is 75%.  Persistence and six-year graduation rates have remained relatively stable over time. A shown in Figure 1, Pacific’s first- to second-year retention rate since 1997 has averaged 86%. This has fallen short of the university’s goal of a 90% rate. The six-year graduation rate is just above 68%, well below the university’s goal of a 75% graduation rate. Models used by Pacific’s Institutional Research Office predict that the six-year graduation rate will hold at about 66-68% for the currently enrolled cohorts. In general, Pacific’s performance compared to peers is similar to standard peers and less than aspirant peers, which show recent graduation rates of 69.7% and 79.4% respectively. The most recent undergraduate cohort, the 2007 new freshmen, showed an unexpectedly low four-year graduation rate. Unusually high attrition does not seem to be the cause for this phenomenon. An analysis of this group of students found that many students were either still enrolled or had applied to graduate after their senior year. The time to degree for this cohort was slower compared to other groups of new freshmen, which lowered the four-year graduation rate as. The University will continue to track this group of students as they finish their work at Pacific.

The gap between Pacific's student success goals and results is somewhat puzzling According to a 2009 Institutional Research IDEA (Institutional Decision-support Evaluation & Analysis) report on high-impact educational practices, Pacific’s educational practices compare favorably nationally recognized models for student success. Nearly all of Pacific’s undergraduate programs require or offer First-Year Seminars and Experiences, Writing-Intensive Courses, Diversity/Global Learning focus, or Capstone Courses and Projects. Despite broad efforts to use these best practices recommended by experts such as George Kuh, Pacific to date has not met its goals for undergraduate student persistence and graduation rates. (CFRs 2.10, 4.4, 4.5)

Figure 1: Retention and Graduation Rates for Pacific Freshmen Cohorts

Undergraduate Demographics and Student Success: For the 2004 freshmen cohort, six-year graduation rates varied at Pacific among ethnic groups and varied as compared to standard and aspirant peer institutions. At Pacific, White and Asian/Pacific Islander students graduate at rates somewhat above the university average while Black, Latino/a, and international students have lower than median graduation rates as shown in Figure 2.  These trends by ethnicity parallel national trends. (CFRs 1.5, 2.10, 4.4, 4.5)

Figure 2: Pacific Six-Year Graduation Rates, 2000-2004 freshmen cohorts

Academic Preparation and Performance: Using regression analysis the strongest predictor of retention was the student’s Pacific GPA, which is in-turn affected by the student’s high school GPA and academic preparation. Students with a high school GPA above 3.60 exceed Pacific’s goals for retention and graduation. Conversely, students with lower high school grades fall below Pacific’s student success goals.

Figure 3: Student Success by High School GPA

Twenty-eight percent of entering Pacific students are required to take one or more pre-college level classes upon matriculation, an indication of inadequate academic preparation for college work. New Pacific students must demonstrate that they have achieved sufficient levels of skill in reading, writing, and mathematics to succeed in university-level work. Students who do not demonstrate acceptable levels in these skills must then take developmental courses to ameliorate these deficiencies. Enrollment in these developmental courses is disproportionate among ethnic groups, with under-represented students more likely to need basic skills instruction. Students from the 2000-2004 students who enrolled in developmental reading courses upon admission demonstrated a six-year graduation rate 3.3% lower than the overall rate. Students who placed into developmental writing courses had a graduation rate nearly 5% lower than the overall rate, and students enrolled in one or more developmental math courses showed a graduation rate over 8% lower than the overall rate. The impact of low levels of preparation in math may be related to the relative dominance of STEM (Science, Technology, Engineering, and Math) fields in Pacific’s student enrollment. Pacific awards approximately one third of its degrees in STEM fields. Failure rates in “gateway” first-year courses (some are required introductory courses with higher than normal rates of failure or withdrawal) also suggest that poor preparation for university level work in these fields negatively affects student success.

 Figure 4: 2000-2004 New freshmen enrolled in a developmental math course

Pacific is working to refine its interventions with students who require developmental courses or who struggle in gateway courses. One hopeful change in practice at Pacific has been the move to an online diagnostic math skills test given to matriculating students. Previously Pacific administered a pencil and paper assessment; students who correctly answered a sufficient number of problems passed. The new assessment provides a more nuanced assessment of an individual student’s skills, thus allowing a more targeted intervention in helping the student develop the skills needed for success. Other tailored interventions include supplemental instruction in biology and engineering courses and a summer bridge academy to build math skills for entering engineering students. (CFRs 2.13, 2.12)

Differences by Academic Program: Student success in terms of persistence and eventual graduation is not consistent across Pacific's academic programs. Student retention and graduation rates are lower in historically under-enrolled programs with below-average pre-enrollment academic quality indicators. While the relationship is not perfect, programs that on average enroll students with higher high school grade point averages and SAT scores, especially PHS and Natural Science students in the College, tend to have higher persistence and graduation rates. Those with lower entering grades and test scores, especially Humanities and Exploratory (undeclared) students in the College, tend to have lower student success rates. On the other hand, the Benerd School of Education (BSE) has relatively high student success rates even though its entering students present relatively low high school GPAs and test scores. Pacific will evaluate the reasons for that success to determine whether BSE provides a model that could be used to enhance success in other programs. Pacific will also evaluate the success of specific program-level interventions. For example, the School of Engineering and Computer Science, which has had below average success rates, has hired a retention coordinator, hosts a summer math academy for entering students, and has implemented regular program checkpoints for its students. (CFR 2.13) 

Figure 5: Retention by academic program

Financial Impact on Student Success: Given the visiting team’s specific concerns, Pacific has conducted specific inquiries to assess the impact of financial factors on Pacific students’ success. Pacific has maintained a strong commitment to high levels of socioeconomic accessibility, and the freshman discount rate has been approximately 40% - 48%, with the overall undergraduate rate approximately 35%.  Significantly, Pacific has enrolled the highest percentage of Pell Grant eligible students of any 2011 U.S. News & World Report top-100 ranked private school, in recent years over 30% of the freshman cohort. 

Pacific’s Institutional Research Office conducted regression analyses to evaluate the impact of several factors hypothetically related to student success. Controlling for other variables, including ethnicity and financial need, the Pacific GPA is the strongest predictor of persistence and graduation rates. Students’ estimated family contribution (EFC) as determined by federal formula was not a statistically significant predictor of student success. Also, students’ adjusted cost of attendance (total cost minus tuition discounts from all sources of aid) was not a statistically significant predictor of student success. In fact, the students in the lowest EFC group (students with Cal Grant and Pell-Grant eligible profiles, EFC < $15,000) demonstrated above-average student success. Research on Pacific’s Pell-Grant recipient group indicates graduation rates that mirror those of the non-Pell recipients, ranging from 66% to 70% depending on the cohort year. The relative success of the highest need group may be at least partially affected by interventions like Pacific’s SUCCESS TRIO program, which provides intensive support, including tutoring, academic, financial, and personal counseling, and other services, for first generation, low income students. Pacific will continue to evaluate the impact of SUCCESS TRIO and other interventions. Mid-level EFC students ($15,000-$34,999) have historically demonstrated above-average student success, but in the last two years, persistence of the mid-level EFC group has declined. This group may be most impacted by the economic downturn because of diminishing external family options to finance education (e.g. diminished home equity borrowing capacity). The high EFC group (>=$35,000 or no FAFSA filed) has consistently demonstrated below-average persistence. Retention and student finances will be monitored and discussed at the Strategic Enrollment Resource Planning (SERP) group monthly meetings. 

 Figure 6: Difference from overall persistence by EFC category over time

Nevertheless, Pacific students will likely face a number of financial pressures as they and their families work to pay their educational costs. CIRP Your First College Year findings indicate that Pacific students had more concern about financing college after their first year. When asked if they had any concern about their ability to finance their college education, 26.3% of Pacific students reported major concerns compared to 20% at other private four year institutions. With federal and state sources of financial aid likely to  be stagnant at best, Pacific’s institutional aid and the families’ direct contributions will probably constitute increasing proportions of educational expenditures. As Pacific seeks to return its freshmen discount rate to normal levels (after a period of inflated discount rates in recent years), pressure will increase on the low- and mid-level EFC groups. Students in those groups, especially under-represented minority students, express the highest levels of finance-related anxiety on orientation surveys and in university-conducted focus groups. The University will continue to monitor relevant data to assess changes in the predictors of student success across EFC and ethnic groups, as ethnic groups are unevenly distributed across EFC categories. (CFRs 1.5, 2.10, 2.12, 4.4, 4.5)

Figure 7: New freshman ethnicity distribution by EFC groups (5 year average)

Campus Climate: The inquiry team also explored matters related to campus climate and the student experience, and overall findings were encouraging. On the issue of whether Pacific has a racially/ethnically inclusive climate, students across all demographic groups over time report a decrease in tense or somewhat hostile interactions with students from other racial groups. While students of color were more likely than White students to report incidents of racial tension, rates were lower than at other private institutions.

Students from historically under-represented groups do not report roadblocks to participating in campus life. Ironically, while Black students in focus group interviews praised the opportunities to become involved in the co-curricular dimensions of university life as a way of ensuring an on-going sense of connection and place at the university, the same students saw the danger that “over-involvement” could come at the expense of academic success. Still, none of the Black students who voluntarily withdrew from Pacific in 2010-2011 said that they were leaving the university because “Pacific is not a good fit for me.” On the other hand, 14% of Latino/a students withdrawing cited “Pacific is not a good fit for me” as the primary reason for withdrawing. Pacific’s Asian American students (who, as a group, show above average student success at Pacific) express less satisfaction with their experience at Pacific. While concerns about the experiences of students in some demographic groups should be more fully explored, responses to CIRP senior survey questions show improvements over time in Pacific students' perceptions regarding campus climate campus climate, both in the student experience and the institution’s efforts to support students from historically under-represented groups.

Another issue of campus climate that may affect student persistence is campus safety, and Stockton has recently been the focus of national attention related to safety issues. Compared to data from peer institutions, Pacific students are more likely to report anxiety about their safety on the Stockton campus and its surrounding area. These rates are greater than those reported at other private institutions, illustrating the salience of personal security within Pacific’s campus climate. While the relationship between anxiety about safety and student persistence has not been documented, this anxiety may interact with other dynamics influencing student success. Enhancing safety and security on campus and the surrounding community has been a central concern for President Eibeck. During summer 2011, the campus hosted two national security consultants who conducted a thorough campus audit of safety precautions and resources. Their final report, in 2012, will be the basis of a multi-year plan for improving safety on campus and the immediate surrounding area. (CFRs 1.5, 2.10, 2.11, 2.13)

Transfer Student Success: Transfer students at Pacific predominately come from the local region, typically more than 40% transferring to the University from San Joaquin Delta College, two miles away. As is true at other universities, transfer students persist at higher rates than students who complete their entire degree program at Pacific. Though transfer students perform better academically overall at the University, African American transfer student success rates are slightly lower (though the N is small). Transfer students do not perform as well as peers who began at Pacific in specific courses and thus may need as much or more academic support than students who began as freshmen.

Understanding the importance of addressing transfer students’ unique needs, transfer orientation programs now include a number of “Transfer Touch Points” that allow for regular check-in before and during the first year, and focus attention on the social integration that transfer students have described as being more difficult for them. These touch points include pre-enrollment academic advising for Delta College students, summer social gatherings, a streamlined orientation experience focused on developing capacity to use essential university advising resources and course planning tools. Transfer students and first year students who are also commuter students have also expressed their need for a “home base” or dedicated place to store belongings, study in between classes, or socialize. To that end, the university has committed to reassigning a classroom space as a commuter lounge space, beginning in spring 2012. The Division of Student Life and offices of admission and financial aid participate regularly with Delta College leadership in the Delta Pacific Collaborative to provide closer connections between department staffs and to ease transition problems for transferring students. More close collaborations are planned with Delta College and other regional Community Colleges to ensure that transfer students enter Pacific with the strongest academic preparation and probability for success. The Retention Analysis Network will continue to monitor transfer student success to evaluate the effectiveness of these interventions. (CFRs 1.5, 1.7, 2.2, 2.11, 2.13, 2.13, 2.14)

Addressing Student Success: University of the Pacific will continue monitoring trends in student success with particular attention to the diverse socioeconomic and ethnic groups of students enrolled at the institution. Student success has been a recurring topic at monthly Strategic Enrollment and Resource Planning (SERP) meetings, which focus on the alignment of enrollment planning with budget resources, academics, and university-wide planning initiatives. The goal of SERP is to engage in dialog with key administrators and leaders about how Pacific serves its students (current and future), its alignment with Pacific’s mission, goals and capabilities, and a changing marketplace and environment.  Recent meetings have included discussion of the changing American educational environment, shaping the student profile, net operating revenues and expenses, financial aid policies, program demand and capacities, and student retention. Additionally, a new Vice Provost position within the academic division has been announced, with primary responsibility for student success, learning and retention. This new position will coordinate the development, implementation and assessment of programs and services that impact student success. 

The University will continue to refine its implementation of best practices, including the identified gateway courses. Pacific’s approach to developmental math instruction will be assessed for effectiveness, and continued curriculum evaluation and refinements will be made. An opportunity may exist for the introduction of focused, targeted bridge programs for incoming students needing basic math coursework. The relative success of the cohorts within Schools of Education, International Studies, and Conservatory of Music (each of which has a small, tight-knit cohort of students following a similar path during their first year) will be examined for wider applicability. Individual academic program interventions within the Schools of Engineering/Computer Science and College of the Pacific Chemistry Department will be assessed and studied for potential wider application across the University. If the intervention initiatives currently underway are insufficient, adaptations will be made and efforts will continue. A university-wide mid-semester early alert system for lower division students is being considered to provide early warnings to students and instructors about student progress in individual courses.

Pacific will also examine retention efforts that can be implemented beyond the first-to-second year timeframe. Pacific loses 15% of students between first and second year, the most well understood opportunity for targeted interventions already underway, but Pacific also loses another 15% between years two and six. A study of peer institution practices will be undertaken to learn from similar approaches taken at nearby institutions, with similar demographic student profiles. (CFRs 1.5, 2.10, 2.11, 2.12, 2.13, 2.14, 3.8, 3.11, 4.4, 4.5, 4.6, 4.7, 4.8)

3. Assessment of Student Learning

3.1 Pacific’s Approach to Assessment and Student Learning.
As noted above, WASC expressed concerns about the development of Pacific’s learning assessment practices at the program level. (CFRs 4.4-4.8) Pacific’s decision to pursue a professional development approach to student learning assessment is discussed above in the section detailing the University’s responses to the CPR recommendations.  Rather than attempting to develop an administratively driven institutional approach to learning assessment, which would be impractical given the number and variety of the University’s programs, Pacific has opted to support the development of local programmatic assessment and then to coordinate what is learned at the program level to assess student learning relative to university learning objectives. Assessment committees and coordinators for the schools and programs, with help as needed from the Director of Learning and Academic Assessment, the Director of Institutional Research, and the Director of Assessment for Student Life, support faculty and staff as they collect, analyze, interpret, and apply findings from performance data as well as direct evidence of student learning.

Thus, Pacific’s professional development model of assessment, especially in the academic division, features several dimensions:

  1. Working with faculty and staff at the program-level to develop their learning assessment plans.  As noted above, these efforts have been focused on programs that had not already articulated plans or that needed assistance to refine their learning assessment plans.  For example, beginning in spring 2010, every department in the College participated in an internal assessment workshop offered by the College’s and University’s Assessment Directors. These workshops were followed with consultations in fall 2010, with the result that every department developed program outcomes, a curriculum map, and an initial assessment plan. 
  2. Working with faculty and staff at the program level to improve assessment expertise.  Depending on the developmental level of a program this may involve consultations or workshops on matters ranging from writing learning objectives and outcomes statements to designing effective rubrics to assess student work. 
  3. Encouraging programs to develop assessment plans that target questions of immediate faculty concern and interest and building over time to comprehensive assessment of program learning objectives and outcomes.
  4. Engaging programs in processes to clarify the relationship between program-level learning objectives and outcomes.  As Pacific’s Capacity and Preparatory Review report notes, “Schools, programs, and divisions will maintain objectives and outcomes that are either unique to the school, program, or division and thus beyond the scope of the institutional objectives and outcomes, or they may operationalize the institutional objectives or outcomes in specific ways. Further, schools, programs, and divisions of the university do not necessarily bear equal responsibility to promote each of the objectives and outcomes.” (p. 27) For example, critical and creative thinking, one of the university learning objectives, looks very different in the Conservatory’s Bachelor of Music Performance degree than it does in the undergraduate Biology program or the McGeorge School of Law’s J.D. program. Perhaps the most salient finding of an assessment of critical thinking conducted in 2010 in Pacific Seminar 001, the first General Education course taken by all first-year undergraduate students, was that faculty teaching in the course did not share a common understanding of critical thinking. For the most part, critical and creative thinking are context dependent and specific to disciplines and professions, as are most areas of knowledge and skill outlined in the university-wide objectives.
  5. Developing systematic processes for programs, including all-University programs like General Education and the Diversity Requirement, to report assessment results and for the University to integrate these results with one another and with results from indirect assessment such as the CIRP.
  6. Providing a forum for the discussion of institution-wide and program-specific assessment activities through the formation of the Assessment Working Group.
  7. Developing a prototype for the collection and sharing of learning objectives, curricular mapping, assessment plans, methods, results, and reports through an inventory created for the EER inquiry.
  8. Collecting and sharing assessment information with professional programs with well-established comprehensive assessment plans and activities. (CFRs 2.1, 2.3, 2.6, 4.4, 4.6, 4.7, 4.8)

The recent record of Pacific McGeorge School of Law provides an instructive example of the application of Pacific’s professional development model.  Before 2011, McGeorge’s approach to learning assessment was not well developed.  Beginning in spring 2011, the DLAA consulted with a working group of McGeorge faculty to develop a statement of student learning outcomes for the Law School. In February 2011, the Law School faculty approved a statement based on the university learning outcomes adapted to be relevant for the School’s professional context.  The School then conducted five workshops on course mapping. Twenty-seven Faculty members mapped over 100 courses, including all required courses, to program and university learning objectives.  As a result of curriculum mapping, faculty made adjustments to the core Global Lawyering Skills courses to strengthen instruction, practice, and assessment of that outcome. McGeorge also used curriculum mapping as a faculty development exercise to familiarize full- and part-time faculty with J.D. program outcomes and have them reflect on how their courses contribute to student learning and what methods they use to assess student progress and mastery. A faculty committee checked for consistency across sections and followed up with student and alumni focus groups to triangulate the results of curriculum mapping. Results with the first focus group of current students indicate student and faculty perceptions of the curriculum are highly congruent.

 A fall 2011 in-service day built on the faculty curriculum mapping workshops; the in-service day included a report on the results of mapping, as well as sessions to help faculty write course-level outcomes and design course-level assessment.  Finally, professors who teach Property Law have formed a faculty learning community.  Consulting with the DLAA they will develop a set of best practices for teaching, learning, and assessment to share within the Law School and across the University.  While this professional development approach is more time-consuming than one in which one or two people conduct assessment for a school or department, it should facilitate deeper understanding and value for inquiry and evidence and a sustainable shift in institutional culture. (CFRs2.3, 2.4, 4.6, 4.7, 4.8)

Pacific still has work to do in articulating and implementing its localized professional-development approach to student learning assessment.  The University needs a reliable system for programs to report results from their assessment work.  Without such a system Pacific will be hobbled in its ability to support professional development.  Pacific will also have a harder time understanding overall progress on university learning outcomes unless it greatly improves its ability to coordinate assessment information from across the University.  Some of this may be facilitated by formalizing the ad hoc Assessment Working Group and appointing a university Director of Institutional Effectiveness. 

3.2 Pacific’s Student Learning Assessment Systems and Practices
The WASC Commission’s CPR Action Letter endorsed the visiting team’s recommendations about student-learning assessment at Pacific: “It is expected that UOP will have student, course, program, and institutional learning outcomes developed by the time of the EER and that the learning outcomes will be measured and implemented. “

To evaluate how well the University has achieved these goals, Pacific altered its proposed inquiry into the institution’s assessment practices. The University had originally proposed to examine how it was assessing learning relative to the institutional learning objectives adopted in 2009. Given the mandate from the Team and the Commission, however, the University broadened this inquiry to examine the full range of its learning assessment practices. This broader approach to the inquiry is also suggested by Pacific’s approach to assessing its institutional learning objectives discussed above. Across the University, institutional learning objectives are integrated with program-level outcomes. Assessing learning relative to institution-wide objectives necessarily requires surveying the full range of learning assessment practices.

To carry out this inquiry a team prepared an inventory of Pacific’s academic and co-curricular learning assessment practices. This inventory provides the principal source of information for the following discussion.  In short, the inventory shows Pacific’s progress in implementing student learning assessment.  At the time of the CPR visit, eight graduate or first-professional programs and 38 undergraduate programs had not yet articulated program learning outcomes. Since then, all graduate and first-professional programs and 33 undergraduate programs have developed program learning outcomes. Every program that has articulated learning outcomes is also implementing an assessment plan. Curricular and co-curricular programs have mapped the alignment between program level learning outcomes and the courses that contribute to those outcomes. Programs systematically inform students about learning objectives and outcomes, and students can use such information to make their own plans. Although programs with specialized accreditation maintain multiyear assessment plans, 31 of the 33 academic programs with recently developed outcomes have not yet developed multiyear plans. Assessment reports for academic programs and co-curricular programs include a section on how faculty and staff “close the loop” by using the results of assessment to refine pedagogy, curriculum, and their assessment plans. (CFRs 2.3, 2.11)

Comprehensive and Assessable Objectives and Outcomes: Essentially all academic and all co-curricular programs have established learning objectives. Since the CPR visit, the College of the Pacific has made significant progress in learning assessment, starting with the articulation of program learning objectives. Working with the Director of Learning and Academic Assessment and the College’s Director of Educational Effectiveness, every department in the College has developed or redefined statements of learning objectives for its major programs. Some units or programs have articulated general program outcomes that apply to all majors, such as the Eberhardt School of Business with its common undergraduate program outcomes supported by competencies for each area, while others have differentiated among degrees and/or concentrations. Over time, programs are expected to review and potentially revise their learning objectives or outcomes as a result of their assessment experiences. Pacific has also adopted learning objectives for the institution as a whole, for the general education program, and for the undergraduate diversity requirement. The University Library has adopted assessable information competency objectives and outcomes from the Association of College and Research Libraries. Student Life programs have well-developed statements of learning objectives and outcomes. A significant gap in the articulation of learning objectives and outcomes may be the accelerated pre-pharmacy and pre-dental programs. These programs are currently assessed through informal observation and information sharing between the Pharmacy and Dentistry programs and the chairs of the Biological Sciences program. These accelerated pre-professional programs are signature programs for the University’s identity, and they figure prominently in Pacific’s enrollment strategy. The University will find ways to formalize assessment of these programs without creating complicated and reductionist procedures.

Most programs have measurable learning objectives that are based on Bloom’s taxonomy of learning objectives, usually in the application, analysis, or synthesis domains of Bloom’s taxonomy. For example, the BS in Geology statement includes outcomes such as “interpret the composition, origin, and uses of common earth materials,” “initiate and conduct research . . . and present the results.”  The Computer Science major includes among other outcomes that “Students will apply knowledge of computing,” and be able “to analyze the impact of computing on individuals, organizations, and society.”  Program learning objective statements for all academic and co-curricular programs are available in Appendix B.

On the other hand, many programs have not yet set levels of proficiency for their outcomes or specified how to assess students’ levels of proficiency. In response to this finding, the Director of Learning and Academic Assessment is preparing a “Guide to Academic Assessment” that addresses how to set and assess levels of proficiency. This document will help Pacific’s programs continue their development in this area.  (CFRs 1.2, 2.3, 2.11)

Mapping and Alignment: Academic and co-curricular programs have prepared documents to show the alignment between program-level objectives, courses, and other programmatic activities with program-specific and institutional learning objectives. Curriculum mapping documents were developed by departments to ensure that students have ample opportunity to achieve program outcomes, as well as to identify gaps in programs. The scope and depth of these mappings vary from program to program.  Those schools and programs with specialized accreditation requirements mapped their curricula to meet accreditation standards as well as university learning objectives and outcomes. Departments in the College of the Pacific, guided by the College’s Director of Educational Effectiveness, mapped courses to program outcomes so that curriculum maps show where the knowledge and skills involved in each outcome are introduced, developed, and mastered. The vast majority of programs have mapped their program-level objectives and outcomes to institutional objectives; the Conservatory standing temporarily as the lone exception. The Conservatory has mapped its curriculum to program objectives, but has not yet done so to university-wide objectives. Course syllabi typically include course learning objectives relative to program learning objectives. Over time, an increasing number of syllabi show alignment to university learning outcomes. During Fall 2011 the University is developing a web-based database of course syllabi that will be available broadly to the university community. 

Most programs have mapped their offerings to the institutional learning outcomes of field competence, critical and creative thinking, and communication. Many have also shown how their programs are connected to the leadership and collaboration outcome. Relatively few programs, however, have indicated that they promote the institutional outcomes of intercultural and global perspectives, ethical reasoning, and sustainability. The provost has begun conversations with the University’s sustainability director and the chair of the University’s sustainability committee to explore how individual programs can establish connections between their offerings and the sustainability objective. Similar conversations about intercultural and global perspectives and ethical reasoning will ensue. (CFRs 2.3, 2.4, 4.6)

Assessment Plans, Methods, and Reports: Each of the academic and co-curricular programs with defined program learning outcomes also has a plan for assessment. In 16 of the 18 departments in the College, and in all the programs of the Conservatory, assessment plans to date are short-term plans, created each year from analysis of results of the previous year. Programs without specialized accreditation have been encouraged to begin with annual assessment projects. As program faculty and staff enhance their understanding of assessment and build trustworthy processes over time, more multi-year plans will emerge from programs “closing the loop” on their findings about student learning. Assessment plans for programs that have specialized accreditation are comprehensive and systematic in targeting all learning objectives over a multi-year cycle. As Pacific’s program review process (based on annual reports culminating in a periodic major self-review) becomes fully implemented across the University, most programs will find it expedient to develop and implement multi-year assessment plans that would allow them to use the periodic self-review to review student learning across all learning objectives. (CFRs 2.7, 2.11, 4.4, 4.6, 4.7, 4.8)

The Student Experience: Students at Pacific have many opportunities to become informed about learning objectives and assessment. In the first session of new undergraduate student orientation, the vice president for student life discusses the university learning objectives with new students and their families. Course syllabi, University catalogs, program websites, new student orientation sessions and other means are used to make students aware of learning objectives, outcomes, and various assessment activities. A number of programs offer seminars for new students during which learning objectives and outcomes are prominently discussed. While information about learning objectives and assessment is amply available internally, Pacific is striving to more consistently present this information in public media. Thirty-one programs listed learning objectives in the 2011-2012 catalog and 65 listed learning objectives on their websites. The Division of Student Life uses its website to publish assessment reports, including student learning assessment data and improvement plans, as well as program review self-studies and reports. Also, the recently updated University website enables academic programs to clearly feature visible links to information about program goals, mission, learning objectives and outcomes, and assessment plans. Some programs, including all Student Life departments, involve students in reviewing and providing feedback on the program-specific outcomes or mapping courses to the program-specific outcomes. For example, the Eberhardt School of Business surveys its students regarding learning objectives and outcomes, and the Pharmacy program has student groups perform curricular mapping following completion of courses. The History department includes student representatives in an annual meeting to review learning objectives and outcomes.  Students in the Sociology senior capstone course conduct focus groups among second-year sociology students. Several programs use senior exit interviews to gather student views about learning objectives and outcomes. These practices, however, are not sufficiently widespread across the University.  Promoting more effective student participation is an aspect of Pacific’s unfinished business regarding assessment. The Assessment Working Group will address the question of how programs can best engage students in the review of program learning objectives and assessment practices. (CFRs 2.5, 4.8)

“Closing the Loop”: Given the variety of specialized accreditation organizations for which Pacific’s programs must prepare assessment reports, the University does not presently feature a standard format for programs to report the results of their learning assessment programs. Nevertheless, most programs do present regular assessment reports that show how the programs have used the analysis of assessment information to make decisions about pedagogy and curricula. Assessment reports in the College and School of International Studies feature a section on how results are being used or will be used. Assessment reports in Business, Dentistry, Education, Engineering, Law, and PHS, as well as Student Life, include decisions or recommendations based on assessment findings. Faculty members in the Conservatory are in their first year of systematic program assessment and have not yet produced reports of their findings. The table below provides a sample of decisions or recommendations made at the program level based on recent assessment of student learning.

Program

Changes or Recommendations Resulting from Learning Assessment

Chemistry Adopted a new laboratory manual for CHEM 023; also developed a new more generalizable rubric to use across laboratory courses and section.
Education Created a new required course, EDUC 160, focused on guiding initial teaching candidates to create productive learning environments, based on data suggesting that candidates needed additional work in classroom management. 
Communication Decided to track “strong” and “weak” writers in early courses. Strong writers are encouraged to become Writing Mentors in the Writing in the Disciplines program; weak writers are referred to the Writing Center.
Dental School Decided on an integrated rather than sequential presentation of topics in anatomy, physiology, and biochemistry.
French Decided to offer an intermediate-level “bridge” between first- and second-year courses and advanced courses to improve reading comprehension.
Mathematics Decided to track math writing in MATH 049, a gateway course to work in advanced topics.
Pacific Recreation Recommended that staff work early with club sports leadership to establish goals to ensure that activities are well attended, that more individuals are exposed to the opportunity to participate, and that the student leaders are able to provide an environment that is conducive to the development of social and emotional competencies, leadership, and individual accountability.
Philosophy Decided to collect reading prep sheets a day before class meetings to give the instructor time to recognize and address student problems with reading.
School of International Studies Changed sequencing of core courses based on analysis of Intercultural Development Index assessment results.

The information sampled in this table shows that Pacific’s programs analyze assessment data and use that analysis to inform pedagogical and curricular decisions. Pacific’s programs are also learning about the assessment process itself and working carefully to refine their program-level approaches to student learning assessment. The recent record of the Eberhardt School of Business (ESB) is illustrative of this point. Initially the School decided to use the ETS Major Field Test as the assessment instrument for their undergraduate program. Their conclusions and recommendations from the first year using the Major Field Test were primarily about what additional data they needed to assess their program and how their own approach to assessment needed to be refined. As a result, the School is purchasing finer-grained reports from ETS Major Field Tests in order to diagnose specific weaknesses in student performance.  ESB is also building in use of rubrics to assess collaboration and teamwork and making more use of formative assessment to complement the summative Major Field Test assessment. (CFRs 2.4, 2.6, 2.11, 4.6, 4.7, 4.8)

Case Studies.  It would be unwieldy to provide comprehensive details on the assessment work done in each of the University’s programs. On the other hand, merely providing a summary table would not demonstrate the quality of assessment efforts across the University. The following case studies, based on various programs’ 2010-2011 assessment plans, demonstrate the development of learning assessment at Pacific and the use of assessment results to inform curricular and pedagogical improvement. 

Communications Department. The Communications Department offers both undergraduate and graduate degrees. The department teaches an introductory course in Public Speaking, a high-enrollment General Education class. In 2010-2011, assessment in the undergraduate program included oral communication, written communication, and teamwork outcomes. The department maintains a learning outcome of “Research, organize, and deliver speeches and oral presentations effectively” for its majors and non-majors who enroll in their General Education courses. The department’s fall 2010 and spring 2011 assessment plans studied achievement of this outcome in the Public Speaking course.  Assessment throughout the academic year resulted in a large enough sample so that findings could be disaggregated by major. Assessment of oral communication in Public Speaking also resulted in recommendations for improvements within the course and sharing of methods and findings beyond the department. Education, Business, and Engineering faculty have all indicated interest in using the Public Speaking oral communication rubric, collaborating with Communications faculty to improve instruction and achievement related to oral communication, or both, with some follow up to improve instruction and student outcomes. 

The combined assessment data from the Fall 2010 and the Spring 2011 reveal that delivery remains one of the more challenging specific outcomes addressed by the course. Giving a speech or an oral presentation involves the use of eye contact, gestures, movement, and vocal variety, as well as poise and confidence, skills largely developed through practice and experience. The mean score on the final speech for delivery was 3.47, as compared to 3.64 for introduction, 3.63 for organization, 3.51 for supporting material, 3.57 for conclusion, 3.62 for language, and 3.52 for outline.  Based on this finding, the course professor has decided to develop a lesson plan for graduate assistants to use during discussion that reinforces the importance of students’ monitoring their own delivery skills both inside and outside of the public speaking course.

The department also disaggregated data by students’ academic programs. The data reveal that students from all of the units improved their oral communication skills from the first to the final speech.  The data from the final speeches also suggest possible differences between the groups of students as follows:

The Communications department came to other conclusions about teaching and learning oral communication skills, assuming that students’ home programs are interested in further developing the oral communication skills of their students. These are discussed in detail in the department’s spring 2011 report. As students become more familiar with disciplinary expectations for oral communication skills, faculty should be able to provide them with more challenging assignments that emphasize the selection of rhetorical choices at a mastery level. 

Dugoni School of Dentistry. Students of University of the Pacific’s Dugoni School of Dentistry achieve desired outcomes at or above levels set by the school, as demonstrated by the pass rate of 97.8% for Part 1 of the DDS licensing exam for the class of 2011. The first-time pass rate for both the DDS and IDS (International Dental Studies) exams have hovered between 90 and 100% since 2004.

Licensure exam pass rates indicate that students are practice ready upon graduation; however, course assessments indicated to faculty that, “when taught separately, students can’t clearly see the interrelationships that exist between biomedical sciences, let alone between biomedical and clinical sciences.”

Faculty spent a year studying and developing a major change in curriculum and instruction. As described in their proposal to the curriculum committee, “Our guiding assumptions are that greater integration will 1) help our students develop a more robust working knowledge of the human body (structure, function and metabolism) and 2) reduce unnecessary repetition and allow greater use of innovative, case-based and other teaching modalities than we do currently…Integration will be achieved by carefully aligning topics within separate anatomy, physiology and biochemistry courses, and by developing a series of case-based, interdisciplinary seminars.” In addition to aligning topics, the curriculum revision involves introducing students and clinical faculty in the first year of the program as clinical faculty are involved in teaching the case-based approach.

The revised curriculum was approved and implemented. The first students to experience the new curriculum are starting their clinical work this year, so direct assessment results are pending. Student self-report through focus groups and a survey indicate that the curriculum is having the intended effect, that students are able to understand the connections between biomedical sciences and applications to clinical work and patient care. Students also report that upperclassmen tell them that they wish they had been taught using the integrated approach. While summative assessment indicated satisfactory student achievement, in this case formative assessment in courses enabled faculty to make curricular revisions to enhance student learning.

University Library. The University Library is in the second year of a plan to integrate information competency instruction and assessment in both the General Education program and in academic units in support of the university learning outcome, "effectively analyze, integrate, and evaluate information. Regarding the General Education program, Library faculty members play an integral part of the second course in the Pacific Seminar series (PACS 2). One of the course objectives in PACS 2 is to introduce students to the skills involved in information competency. Each of the roughly four dozen sections of the course works with one of the library faculty.

The library has established course-level learning outcomes for PACS 2 library instruction:

  1. Draft and refine an appropriate research topic in order to develop a research question/statement.
  2. Identify keywords and synonyms for researching a topic in order to perform successful database searches.
  3. Identify and use appropriate databases for researching the topic in order to find relevant sources.
  4. Use credible and appropriate sources in order to support the research product.
  5. Correctly and consistently cite sources in the text and bibliography of the research paper in order to demonstrate ethical use of information and avoid plagiarism.

During summer 2011, two members of the library faculty led a session to conduct exploratory assessment of students' information skills as demonstrated in the research papers that are a PACS 2 course requirement. They evaluated the papers using the library’s Pacific Seminar information competency learning outcomes as listed above and an AAC&U Information Literacy VALUE Rubric. Outcomes 1, 4, and 5 were the assessment targets. The evaluation aimed to answer the following questions:

While the primary lessons learned had to do with the need for more clarity and customization of the assessment methods and rubric, the project did lead to findings about student learning as demonstrated in the research papers:

Many papers did not demonstrate analysis or critical thinking but were similar to book reviews or encyclopedia articles. It is difficult to determine how sources were selected, evaluated, or used. A question that arose was the importance of having ‘perfect’ citations versus an understanding of when and why to cite. It is clear that students see research as a linear, rather than iterative process.

Based on the library’s assessment of student work, the faculty convener of Pacific Seminar 2 has led the development of a common rubric for assessment of the required research paper. This project contributes to the development of a robust set of assessment methods for information competency, which also includes the library’s assessment of information competency through the Project SAILS test administered to a sample of PACS 2 and PACS 3 students and further instruction and assessment in other General Education and major courses.

Student Life. Student Life measures its student learning outcomes not only with locally developed instruments, but also with a variety of nationally normed instruments. For example, a study conducted in fall 2010 demonstrated that Pacific’s rich array of co-curricular learning opportunities provide positive learning results to a greater extent than do those opportunities at comparable colleges and universities. Pacific’s participation in NASPA’s Student Voice survey (NSV) on student involvement in organizations and activities revealed results for four important learning outcomes in the study that exceed those for comparable institutions: increased self-confidence, meeting people students would not otherwise have met, meeting and learning from individuals with different interests from their own, and providing identifiable skills and abilities that students will use after college in their careers and communities. Pacific’s  and comparable institutions’ data may be found here: Link and include graphs

These assessment results were used to invest further in student involvement  through the purchase and installation of a social networking software system (OrgSync) to further enhance club/organization management and peer-to-peer club communications. Using OrgSync has deepened and extended Pacific students’ involvement in co-curricular learning activities. As of November 2011, 2,526 Pacific students have joined the OrgSync online community, an increase from 856 in fall of 2010. Even more significant, student attendance at events hosted by student clubs and organizations increased 64% in fall of 2011 over fall 2010, building on what is already a salient learning strength for Pacific.

Taken together these cases demonstrate the widespread use of assessment to improve student learning across the University’s programs. What needs to be added is a university-wide system of collecting, analyzing, and communicating the results of assessment.

University Learning Objectives: Pacific adopted institutional learning objectives in 2009 and first published these objectives in the University General Catalog in 2010-2011. The assessment of student learning relative to these objectives is still in its early stages. The University’s approach to educational effectiveness anticipates that results from program-level assessment work will be integrated with results from university programs such as General Education and the Diversity requirement. Institutional assessments like CIRP and NSSE can supplement direct assessment of student learning and provide useful comparisons with nationally normed data. (CFR 4.5)

Each academic and student life program provided the inquiry team with a report on how they meet the university-wide learning objectives. Those reports provide an overview of the general directions that each unit is taking to address the university-wide outcomes.

Selected reports also provide specific data that document how well students within those programs are meeting the university’s learning objectives. The table below summarizes which programs included data on university learning objectives in their recent assessment reports. Comparison of responses on the CIRP Freshman and Senior surveys is another available source of information about university learning objectives. Pacific McGeorge School of Law faculty recently completed a mapping exercise showing the connection between Law School’s curriculum and program-level objectives and the University’s learning objectives. In the future, the Law School will include information about University learning objectives in its assessment reports.

  Major Field Competence Critical and Creative Thinking Communication Collaboration and Leadership Intercultural and Global Perspectives Ethical Reasoning Sustainability
Alignment to university objectives All academic plus 4 Student Life All academic programs, General Education, Library, 2 Student Life All schools (2/3 of the College programs) and General Education, 12 programs in Student Life All schools (4 programs of the College), 9 programs in Student Life

SIS, Business, Education, Dental, PHS, Law, 4 programs of the College, Diversity Course Requirement, 4 programs in Student Life

General Education, Dental, Engineering PHS, Law, 2 programs in Student Life Dental, PHS, 2 programs of Law, 3 programs of student life

By definition, academic programs first assess student learning in terms of major field competence. A field-specific critical or creative thinking objective is also central to academic programs’ objectives. A number of co-curricular programs have also done assessment related to critical thinking. The University Library has also concentrated its assessment work in the area of critical thinking, specifically in terms of information competency. Four academic programs have assessed collaboration and leadership, three in the College of the Pacific and the Eberhardt School of Business. Six academic programs have assessed intercultural and global Perspectives, three in the College, two in PHS, and SIS, and this outcome has also been assessed in four sections of the Diversity Course Requirement. The curriculum subcommittee of the Diversity Committee is comparing assessment methods to develop a long-term plan for assessing outcomes related to the Diversity Course Requirement. Ethical reasoning is assessed in Pacific Seminar 3 and the first report was submitted after the spring 2011 semester. For each of the outcomes, programs with specialized accreditation requirements conduct assessment according to their multi-year assessment plan.

Few academic programs have so far assessed outcomes related to the sustainability objective, although a number of student life programs have done so. Programs that do not have a direct connection to environmental sciences or environmental studies seem relatively uncertain about how to relate their program-level objectives and curricula to the sustainability objective. To initiate a conversation about how programs could approach the sustainability objective, the provost’s office during fall 2011 surveyed the University’s programs about the meaning of sustainability for their programs and how they could demonstrate such learning. Using the results of this survey, the University’s sustainability director and the chair of the University’s diversity committee prepared an initial report on the assessment of the sustainability objective. According to that report, units and programs vary substantially in their approach to sustainability. Some units "provided a clear, high-level definition of what sustainability means to them that included mention of each of the three spheres of sustainability - economic, environmental, and social. Other units focused on one or two of the spheres or focused on very specific, operational practices rather than a clear, high-level definition." Pacific will continue its discussion about how to interpret, implement, and assess the sustainability learning objective. 

Finally, revisions to the University’s program review processes will require all programs to include information about learning relative to the University’s learning objectives in annual reports and periodic self-studies. As the program review revisions are implemented, assessment information about the University’s learning objectives will accumulate. The director of institutional effectiveness should make it a priority to coordinate such accumulating information. The Assessment Working Group should develop a “report card” or other mechanism to estimate the University’s progress in meeting university learning objectives. (CFRs 2.7, 2.11)

Pacific has a good deal of work to do to capitalize on its initial progress in implementing and assessing university leaning objectives. At the program level, units and departments must refine their assessment plans to provide data and analysis about how programs contribute to achievement of the University’s learning objectives. Currently, some programs and units have provided documentation for meeting institutional learning objectives. Other programs and units appear to have initial processes and data on meeting at least some of the institutional objectives, but have not yet formalized methods to collate, analyze, and report data on all University learning objectives. The University also needs to improve its ability to coordinate these various sources of assessment information, reach conclusions based on that information, and then communicate those conclusions across the University. Finally, Pacific needs to recognize and reward the good work on assessment that is being done by many faculty and individuals in leadership positions. The Provost is working with the Promotions and Tenure Committee to establish guidelines for including assessment in faculty performance. She holds the deans accountable for assessment efforts in their units and similarly, the deans will set expectations of assessment work, which ultimately should be considered in advancement. (CFRs 2.4, 2.6, 2.7, 2.11, 4.4, 4.5, 4.6, 4.7, 4.8)

Pacific had developed a robust system of key performance indicators (KPIs) to define goals and to be able to assess progress towards meeting these goals. Ninety-three separate KPIs across seven important areas include Academic Quality and Effectiveness, External Perceptions, Financial Robustness, Fundraising, Student Success, Demographics, and Student Life. Each of the KPIs shows the current value, most recent value, best value in the last 10 years, and the long range goal for the KPI.  The Board of Regents reviews progress towards the KPIs at each meeting. During the next year, the deans will be working with their faculty to develop KPIs for their units and academic programs. This will facilitate the assessment of the quality of the programs and sustainability of program and unit quality. In the future, continued effort will be made to move away from input measures to better measures of educational process outcomes and effectiveness.

While the description and analysis above indicate that Pacific has done a great deal to develop its approach to assessment and educational effectiveness, the University recognizes that a good deal is still to be done. Pacific will continue implementing its professional-development approach to assessment to bring all programs to a high level of development. Articulating expected levels of mastery at various levels, increasing student engagement with assessment practices, and improving transparency of assessment-related information are areas in which Pacific will improve its performance. The University will also improve its ability to gather, coordinate, and analyze information about student learning across the institution.

4. Program Review

University of the Pacific views program review as a key process for promoting institutional effectiveness, especially educational effectiveness in coordination with planning and decision making for a complex tri-city university. The University has worked progressively to develop and refine program review processes that are grounded in evidence about educational effectiveness and that also inform institutional priorities. Recently implemented changes to the program review process should further enhance evidence-based decision making at Pacific.

Pacific introduced program review in 1996, a time when the University was taking its first steps at financial recovery. During that initial phase of program review, the University reviewed a large number of programs, largely with a view to improving efficiency. While most programs were retained after review, some were discontinued. Pacific then adopted a formal academic program review system in 1998. Under this system, programs carried out periodic self-studies, sometimes with external consultations. Internal panels reviewed these self-studies and made recommendations regarding the programs. Self-studies and panel reports were considered by academic leadership and faculty governance bodies. Ultimately the provost made decisions about the recommendations. Decisions requiring budget changes were submitted to the Institutional Priorities Committee. During the period 2001-2006, the University conducted 38 reviews of academic programs. These reviews resulted in 143 recommendations for program improvement, of which 99 were endorsed by the provost and/or implemented by programs. However, by 2005 faculty and administration had come to agree that it was time to evaluate and revise the program review system. Various problems were noted. Data from learning assessment were not regularly included in self-studies, except for programs with specialized accreditation requirements to do so. Staffing review panels was cumbersome, and the process by which the University made final decisions about review recommendations, especially those with budgetary implications, lacked sufficient transparency. Academic program review was suspended in 2006 as the University crafted new guidelines for program review. These new guidelines were adopted in 2008 as the Program Planning and Self Study Process and program review resumed.

The new guidelines feature significant revisions to the program review process. Programs are required to submit annual reports, including analysis of learning assessment data. In addition, periodic self-studies are now required to include external consultants. Review of academic programs has resumed. At this writing, academic programs including Engineering, Law, Dentistry, and the University Honors program has completed the review process. A number of programs, including Biology, Chemistry, Economics, Theatre Arts, and Sport Sciences, have completed self-studies and are in the midst of the review process. Many administrative and co-curricular programs have completed the process, including the Library and all Student Life departments.
 
Among academic programs, those with specialized accreditation are the most developed in their approach to self-study. The Schools of Engineering, Dentistry, and Education and the Pharmacy and Health Sciences programs most thoroughly integrate student learning assessment results into their self-studies. In most cases, such as Engineering, assessment of student learning is the most important criterion addressed in self-study documents. The Pacific inquiry team that reviewed the University’s program review practices also noted that the Dental School is exemplary in its “culture of constant inquiry.” The team commented in its report, “Although the periodic once-every-five-year program review in these professional schools is in addition to their accreditation review, these schools benefit tremendously from the assessment of their student learning objectives required by their external accreditation review process.”

The Student Life division has a highly developed and participatory system of program review.  All co-curricular programs are now reviewed on a six-year cycle. Each department uses the applicable standards from the Council on Academic Standards to guide the writing of its self-study and each department includes review of its student learning assessment process and outcomes as well as student satisfaction surveys and key performance indicators that are benchmarked with other institutions wherever possible. The self-studies have been reviewed by teams chaired by a tenured faculty member, including faculty, staff, undergraduate and graduate students, outside experts, and other team members as applicable. Team recommendations are responded to, action plans written, and team reports are reviewed by various governance groups and posted on the website. The process also includes a follow-up report in a year or two following the review to report on improvements made. In fall 2011, the division invited the senior student affairs officers from Stanford and the University of California, Berkeley, to conduct an analysis of the program review process, to provide a meta-analysis of the review process itself and to provide suggestions for ways the process can be improved. (CFRs 2.11, 4.6)

Administrative functions, offices, and programs have been reviewed with less consistency and regularity than curricular and co-curricular programs. Still, the inquiry team found positive examples of program review for continuous improvement, evidence of a localized culture of inquiry and evidence and development as a learning organization, especially in Business and Finance. The inquiry team cited the following external reviews conducted for Business and Finance: “2009 ROPA Review” by Sightlines, a review of the physical holdings and condition of the campus; “2009 Go-Green Measurement and Analysis” by Sightlines, a review of greenhouse gas emissions per student by Pacific. The team also cited several internal reviews of aspects of Business and Finance operations: 2010 Debt Policy and Ratio Analysis, 2010 Benchmarking Pacific with its California Peers, 2009 Assessment of Pacific’s Administrative Structures and Functions to Better Integrate Three Campuses, and 2010 Campus Master Plan 4.0, Final Recommendations to Board of Regents, all led by Vice President Patrick Cavanaugh. (CFRs 1.2, 1.3)

The most recent review of the School of Engineering and Computer Science (SOECS) provides an example of Pacific’s program review process in operation.  The School’s self-study concluded that professional standards strongly indicated that SOECS should add a graduate degree to their offerings. The program review provided the basis for the provost’s recommendation to add several faculty lines needed to support a graduate degree.  The Institutional Priorities Committee endorsed this recommendation, which was reflected in budget allocations. 

Since the 2010 CPR visit, the University has continued to refine the program review process as described in the discussion of the University’s responses to WASC’s CPR recommendations. Upon her arrival, Provost Pallavicini conducted a review of Pacific’s program review guidelines to identify ways the process could contribute more powerfully to institutional effectiveness. Working with faculty and administrative leaders, the provost introduced a number of refinements to program review. These include the development of program outcomes and assessment for every academic program, revision and clarification of program review guidelines, and improvements to the program review schedule, the piloting of an Institutional Effectiveness Committee, and strengthening the relationship between program review and budgeting/resource allocations. This latest revision in program review (depicted in the figure below) was implemented in fall 2011 and it will be reviewed comprehensively after two years.  Programs scheduled for review between 2011 and 2013 will be considered according to this revised model. 

With this revised review process, programs at the unit- office- or department level, as appropriate, prepare annual reports based on their missions and goals. Annual reports include, among other matters, data and analysis about student learning relative to program and university objectives. In the sixth year of a program’s cycle, the program conducts a cumulative self-study assessing the program’s success in meeting its mission and goals and concluding in an action plan for program improvement. A review team that includes members external to the program and the University conducts a formal review based on the self-study and action plan.  After the provost or appropriate vice-president responds to the proposed action plan, the Institutional Effectiveness Committee (IEC) considers the quality of the program review and the recommendations included in the review.  After cabinet discussion of the IEC assessment, the provost or responsible vice-president reviews and provides final recommendations on any budget adjustments. The IPC then considers these recommendations before referring them to the president for decision.  Two years after a program’s periodic review, the IEC will also formally review progress on the reviewed program’s action plan.  At the conclusion of this two-year phase, the IEC will make recommendations for any further refinements to the program review process.

This refined approach to program review promises several advantages. A significant aspect of the 2011 refinements is the development of a unified program review schedule that encompasses academic and co-curricular programs as well as administrative offices across the University’s divisions. The unified schedule also anticipates conducting simultaneous reviews of programs with similar functions on the three campuses, e.g. marketing offices. The schedule will be adopted in December 2011, and it will be available at the time of the team visit. Under the revised guidelines, all programs, including administrative programs, will show how their outcomes and assessment align with and contribute to university learning objectives. Finally, this approach creates a stronger integration of program review with budgeting and planning.  Given the well-specified process described above, integration should be transparent.  IEC’s two-year follow up on program review results will play a significant role in promoting this integration and cementing community confidence in the process. (CFRs 1.2, 1.3, 2.7, 2.11, 3.8, 3.11, 4.1, 4.2, 4.3, 4.4, 4.6, 4.8) 

Pacific understands that its revised program review and institutional effectiveness procedures are works in progress. The University will hire, if it has not done so by the team visit, an institutional effectiveness director who will oversee these procedures. The University will also continue the pilot of the revised program review process, including the role of the Institutional Effectiveness Committee. At the conclusion of the pilot, the Institutional Effectiveness Committee will review the process, make recommendations for any needed revisions, and prepare a proposal for the Faculty Handbook to formalize Pacific’s program review procedures.

5. Integrative Essay: What Pacific Has Done and Learned Through The Accreditation Process

Undertaking its accreditation review during the last few years has been timely and meaningful for University of the Pacific. Three related currents now run through the life of the University that represent substantial maturation of its approaches to educational and institutional effectiveness. First, and perhaps most fundamental in importance, the University made a critical commitment in its Institutional Proposal to develop, adopt, and implement University learning objectives. Second, the University made an intentional decision to pursue a professional development approach to learning assessment and educational effectiveness and to enhance assessment practices and support structures at the program, unit, and University levels. Third, Pacific adopted a new set of guidelines for program review and has continued to refine its program review practices. The previous administration of the University had made the initial decisions to move the University along each of these currents. However, President Eibeck and Provost Pallavicini and the whole University leadership have embraced them. Indeed, decisions made during the last two years have brought these currents together in an increasingly integrated approach in which educational and institutional effectiveness cannot be disentangled. Together they provide the infrastructure supporting upcoming decisions about the budgetary policies and strategic direction of the University.

University Learning Objectives: In truth, when Pacific proposed to develop a set of university learning objectives as part of its CPR review, the University did not fully grasp how consequential the adoption of university learning objectives would be. The university learning objectives have taken on a greater role in the life of the University than the Institutional Proposal had anticipated.

As discussed in this report, academic and co-curricular programs have taken a serious approach to articulating the alignment of their programs and curricula with the university learning objectives. Working with academic and student life assessment staff, many programs have made progress in expressing the relationship between their program-level goals and objectives and university objectives. Where the University has observed gaps—as in the case of the sustainability objective—this has engendered new conversations about how to work with programs in developing an understanding of what the objective might mean for their programs. The university learning objectives will be a touchstone for the ongoing development of the University’s approaches to student learning, both in terms of program- and course-level assessment, but also in the design of systems to support assessment and student learning. A key role of the DLAA and Assessment Working Group going forward will be strengthening the support for assessment work that addresses these objectives.

The university learning objectives will also play a key role in program and institutional accountability. As Pacific progressively implements its revised approach to program review and institutional effectiveness, the university learning objectives figure centrally. In annual reports and periodic self-studies, programs across the University, not just academic and co-curricular programs, will account for their effectiveness in promoting learning relative to the university learning objectives.

Finally, the university learning objectives, integrated through Pacific's strategic planning processes, will play an ongoing role in important University decisions. For example, President Eibeck, in her 2011 letter on the University budget process directed the University’s budget units to justify any requests for new spending authority specifically in terms of one of more of the university learning objectives. Notably, Pacific has tended to drop the word “student” when referring to the learning objectives to make the point that the objectives are not just to guide the work of academic and student life programs, but they are objectives for the whole university in all its operations. As such, the University learning objectives will undergo regular review and will be revised as needed to engage all University units in developing and promoting a shared set of institutional learning objectives. (CFRs 1.1, 1.2, 2.3, 2.4, 2.7, 2.11, 4.4, 4.5, 4.6, 4.7, 4.8)

A Professional Development Approach to Assessment: As noted above, the University’s Director of Learning and Academic Assessment (DLAA) began her service at Pacific in February 2010, barely two months before the CPR visit. The University’s expectation in her hiring was that she would emphasize working with faculty and staff members principally at the program level to help them, where necessary, develop assessment plans and especially greater expertise in assessment. Pacific’s judgment was that this approach was most likely to be fruitful, especially given the University’s rich array of academic and co-curricular programs across three campuses.

The first goal of this professional development approach was to help faculty establish program-level assessment plans where these were not already in place. This meant targeting programs in the College, the School of International Studies, the Conservatory of Music, and the Law School. Working with key leaders in these units, the DLAA has emphasized an incremental and pragmatic approach to professional development. First, she has identified individual faculty members in each unit who showed an interest in developing assessment expertise and working with those individuals on whatever aspects of the assessment process appeared most important to them. For some programs this might have been drafting effective statements of learning outcomes and objectives, for others it might have been refining rubrics or other assessment instruments. In working with programs to develop their assessment plans, she has encouraged programs to identify questions about student learning that are most interesting or important within the contexts of those programs.

Working to develop faculty and, in some cases, staff expertise in assessment was the second goal. As individual faculty members develop their assessment skills and implement effective assessment plans, they can become part of a community of practice that has powerful ramifications. Working within their programs, these faculty members can serve both as “peer trainers” but can also assuage concerns that assessment work is nothing more than an administrative mandate. One can see this process unfolding at the Pacific McGeorge School of Law. Work there began with a handful of key academic administrators and faculty leaders. As those people began to see the potential role of assessment in improving student learning, their work began to embrace more and more of the School’s faculty. More than two dozen faculty members participated at a recent workshop to develop the School’s assessment plan. The School of Law has become so invested in the development of assessment effectiveness that they have made office space available for the DLAA’s use; she now spends one day a week on the Pacific McGeorge campus working with faculty on developing their assessment expertise and refining their assessment plans.

Pacific believes that both tactical moves—encouraging programs to target assessment on immediate concerns about student learning and taking an incremental approach to building faculty assessment expertise—provide the most promising approach to building faculty and staff support for continuing assessment work over the long-term, i.e. building a culture of assessment.

Between the 2000 WASC accreditation review and the 2010 hiring of the DLAA, academic programs at Pacific followed three approaches to its assessment systems. One, programs with specialized accreditations tended to develop assessment systems designed to meet the requirements of their accreditors. Two, for the first half of the decade the university level approach was to request annual program assessment reports, sometimes offering commentary in response. Three, for the second half of the decade the emphasis was on developing an online system for reporting assessment plans and results. Because there was little outreach to faculty about how to develop and implement effective assessment, it was natural for many faculty members to take a minimal compliance attitude. The improved quality of assessment work now underway in programs that had previously been indifferent to assessment suggests that the ground-up professional development approach now in use has helped Pacific turn the corner in developing a sustainable approach to assessment. (CFR 4.6)

More needs to be done to ensure that Pacific sustains the momentum it has gained. The inquiry team that examined the sustainability of Pacific’s assessment systems and practices points out a number of actions for the University to consider. For example, the inquiry team recommends that the University and its academic programs look to promotion and tenure guidelines to be sure that these reward high quality assessment work and that they do not provide disincentives for that work.

Finally, the ground-up, professional development approach will provide the mechanism for the progressive implementation and assessment of the university learning objectives. Because Pacific does share a basic commitment to an institutional mission, it was relatively easy to identify a set of shared learning objectives. However, even as the University agreed on the objectives, many still had questions. Would all units and programs be expected to teach to each one of the objectives? Would each unit and program be expected to assess learning on each of the objectives or would the University impose a universal approach to assessing each of the objectives? Given the diversity of the University’s programs and the critical role that specialized accreditation plays for so many of the University’s programs, no out-of-the-box, top-down, one-size-fits-all approach to the university learning objectives will be sustainable. The approach adopted by the University, facilitating a ground-up approach, engaging programs and units in a discussion of what the various university learning objectives mean within the specific program contexts, is the most promising approach to implementing these learning objectives across the University. (CFRs 2.3, 2.4, 2.11, 3.3, 3.4, 4.6, 4.7, 4.8)

Program Review and Pacific’s Institutional Effectiveness Framework: As Pacific was preparing its Institutional Proposal for this accreditation review, the University was also drafting new guidelines for program review. These new guidelines, adopted at the end of the 2008-2009 academic year, put new emphasis on each program establishing meaningful student learning objectives and outcomes and holding programs accountable for student learning. The early record of reviews under the new guidelines was uneven. Programs that already had well-developed assessment practices in place, e.g. the School of Engineering and Computer Science, produced very effective self-studies that enabled the University to make significant decisions about program development and resource allocation. Programs without such well-developed assessment plans could not produce such powerful self-studies. The point here is that assessment practices and program review are interdependent. Clear institutional guidelines encourage programs to develop comprehensive assessment plans and practices. And the more-developed a program’s assessment efforts, the more effective their self-study and program review will be. (CFRs 2.7, 4.2)

As more programs develop their assessment expertise and the quality of their assessment plans and results, the more meaningful will be their program reviews for informing University planning and budgeting. Thus, the ground work described above to develop the assessment process at Pacific will increasingly reinforce and improve the quality of program review. In addition, as Pacific builds its practices under the new institutional effectiveness framework, faculty and staff should see their assessment work as increasingly meaningful for the future of their programs. The university learning objectives also figure here. The approach described above assumes that continued discussion of the relationship between program-level goals and outcomes with university learning objectives will support an increasingly shared view of educational effectiveness at Pacific. To the extent that occurs, internal and external reviewers will be able to provide more relevant reactions to program self-studies. Also, the Institutional Effectiveness Committee will have a reasonably clear set of standards for evaluating the quality of these self-studies and program reviews. In turn, the University will be better positioned to make decisions about budget allocations and other matters within the context of its new strategic plan. (CFRs 1.2, 2.1, 2.3, 2.7, 2.10, 2.11, 3.2, 3.8, 4.3, 4.4, 4.5, 4.8)

Integrated and Aligned Planning, Budgeting, and Institutional Effectiveness: The University has been working hard to create a high degree of alignment in its plans, systems, and processes. By deeply considering its changing environment, Pacific used the period of strategic thinking over the first half of last year’s planning process to settle on its primary intentions in strategy, or strategic intent. As Pacific moves to execute these strategies in its programs, schools, campuses, and administrative processes, the role of the institutional effectiveness (IE) framework will be paramount in determining success. As the two-year pilot of the IE framework adjusts and optimizes the process, the University will be positioned to maximize building capacity and achieving its learning objectives. The simultaneous shift to a resource allocation model that provides greater incentives to units that demonstrative effectiveness at achieving their missions will allow the University as a whole to better serve its communities and ensure institutional effectiveness.

Pacific’s latest accreditation review has served the University well. The institution might have addressed the concerns discussed in this report as separate, compartmentalized matters. In retrospect, the self-review process prompted a dynamic model in which key bodies within the University community simultaneously considered institutional distinctiveness, educational effectiveness, student success, and institutional effectiveness in mutually reinforcing ways. Introducing new leadership into the University as the self-review process was underway made another powerful contribution. Bringing fresh eyes to assessment and program review has helped the University envision them anew as an integrated approach to institutional effectiveness. Pacific wanted to use this accreditation review to position itself strategically for institutional improvement into the next decade. The University is satisfied that the review process has served just that purpose. (CFRs 3.8, 3.11, 4.1, 4.2, 4.3)

Appendices and Materials Cited

Appendix A Required Data Exhibits
Appendix B Documentation of Program Outcomes and Assessment
Appendix C Inquiry Team Reports

Appendix A: Required Data Exhibits

Educational Effectiveness Indicators

Inventory of Educational Effectiveness Indicators

Concurrent Accreditation & Performance Indicators 

Summary Data Form- University Information, Faculty, Budget  

Undergraduate Enrollment
New Freshmen 6-Year Graduation Rates
Transfers 6-Year Graduation Rates
Graduate and Professional Enrollments
Graduate Student 6-Year Graduation Rates
PHARMD 6-Year Graduation Rates
DDS 6-Year Graduation Rates
JD 6-Year Graduation Rates

Appendix B: Documentation of Program Outcomes and Assessment

College of the Pacific
Eberhardt School of Business
Conservatory of Music
Center for Professional and Continuing Education
Arthur A. Dugoni School of Dentistry
Gladys L. Benerd School of Education
School of Engineering and Computer Science
School of International Studies
McGeorge School of Law
Office of Learning and Assessment
Library
Thomas J. Long School of Pharmacy and Health Sciences
Division of Student Life
University Wide

College of the Pacific
Department Outcomes Curriculum Maps Assessment Plans Assessment Methods Assessment Data Assessment Reports
Biological Sciences Outcomes
Graduate Program

Undergraduate Program
Graduate Program

Fall 2010
Spring 2011
Graduate Program

 

 

145 Exam 1, Analysis Document, Fall 2010
145 Exam 2, Analysis Document, Fall 2010
145 Exam 3, Analysis Document, Fall 2010
145 Exam 4, Analysis Document, Fall 2010
145 Final Exam Analysis Document, Fall 2010

BIOL 145, Microbiology, Fall 2010
BIOL 101, Fall 2010
BIOL 101, Genetics, Spring 2011
BIOL 179, Evolution, Spring 2011
BIOL 81, Human Physiology, Spring 2011

Chemistry Outcomes Curriculum Map      

CHEM 23, Fall 2010
CHEM 23, Spring 2011

Communication

Outcomes
Graduate Program

Curriculum Map
Graduate Program

Fall 2010
Spring 2011
Graduate Program

COMM 150, Speech Assignment Rubric
COMM 151, Persuasive Writing Skills Rubric
COMM 151, Teamwork Collaboration Rubric from AACU
COMM 27, Final Speech Rubric
COMM 27, Formal Speech of Introduction Rubric

 

Oral Communication, Fall 2010
Oral Communication, Spring 2011
Teamwork Report, Spring 2011
Graduate Program Seminar, Spring 2011
Teamwork Collaboration, Spring 2011
Writing Skills, Spring 2010

Earth and Environmental Sciences Outcomes   Spring 2011     Spring 2011
Economics Outcomes   Fall 2010     Economics 161, Spring 2011
English Outcomes Curriculum Map

Fall 2010
Spring 2011

    English 141, Spring 2011
Writing a Research Paper, Spring 2011
Geology Outcomes Curriculum Map        
History Outcomes Curriculum Map

Fall 2010
Spring 2011
Fall 2011

Grading Rubric for History Papers Revised
Capstone Rubric
Fall 2011

Fall 2010
Spring 2011
Fall 2011

Legal Scholars Program Outcomes Curriculum Map Assessment Plan     2010-2011
Mathematics Outcomes Curriculum Map

Fall 2010
Spring 2011

 

20-059 MF Total Cohort, Major Field Test
4amf Major Field Test

Fall 2010
Spring 2011

Modern Language and Literature Outcomes Curriculum Map

Fall 2010
Spring 2010

    Fall 2010
Philosophy Outcomes Curriculum Map Fall 2010

Critical Reasoning Assessment Plan and Rubric
Department Reading Assessment

  Fall 2010
Spring 2011
Physics Outcomes Curriculum Map

Fall 2010
Spring 2011

    Fall 2010
Political Science Outcomes Curriculum Map

Fall 2010
Spring 2011

    Spring 2011
Psychology

Outcomes
Graduate Program

Curriculum Map
Graduate Program

Fall 2010
Spring 2011
Graduate Program

    Fall 2010
Graduate Program, Spring 2011
Religious Classical Studies Outcomes Curriculum Map

Fall 2010
Spring 2011

   

Fall 2010
Spring 2011

Sociology Outcomes Curriculum Map Fall 2010     Fall 2010
Spring 2011
Sport Sciences Outcomes Curriculum Map Fall 2010     Spring 2011
Theatre Arts Outcomes Curriculum Map 2010-2011     2010-2011
Visual Arts Outcomes Curriculum Map       Spring 2011
University Outcomes   Curriculum Map       Narrative
Table

 

Eberhardt School of Business
Department Outcomes Curriculum Maps Assessment Plans Assessment Methods Assessment Data Assessment Reports
Business Undergraduate Program
MBA Program
Learning Objectives in Core Courses

Undergraduate

 

 

 

Undergraduate
MBA, Employment

 

Conservatory of Music
Department Outcomes Curriculum Maps Assessment Plans Assessment Methods Assessment Data Assessment Reports
Conservatory Music History
Music Management
Music Therapy
Core Courses

Core Outcomes, MCOM/MHIS
Map of Ensemble Competencies
MCOM
MEDU, Choral
MEDU
MHIS Major
MMGT, BM Major
MTHR Major
Voice

Fall 2011

 

 

 

 

 

 

Center for Professional and Continuing Education
Department Outcomes Curriculum Maps Assessment Plans Assessment Methods Assessment Data Assessment Reports
Organizational Behavior Outcomes
Objectives and Outcomes
     

 

 

 

Arthur A. Dugoni School of Dentistry
Department Outcomes Curriculum Maps Assessment Plans Assessment Methods Assessment Data Assessment Reports
Clinical       Assessment Method
Pre-Clinical Fixed Rubric
   
Dental Competency Statement by Courses
Competency Statement by Courses-II
Support of University Learning Objectives
Curriculum Map Dentistry Outcomes Assessment

Assessment Method
Advanced Restorative Technique

Dental Anatomy Rubrics
Graduation Guidelines
Lit Review
Re-Evaluation Competency
Root Debridement Competency
RS Preclinical Operative 170-176
ICS I-OCP-DP 101&DP 106
  WASC Communications
WASC Ethics
WASC Outcomes, Data Critical Creative Thinking
Dental Integrated Medical Sciences Proposal to Curriculum Committee
Dental Hygiene

Competency Statements

    Patient Assessment    
Endodontic       Pre-Clinical Rubric
Test Cases
   
Oral Surgery       Test Cases    
Periodontal       Test Cases    

Restorative



 

 

Complex Issues in Restorative Dentistry 2010-2011
Pre-Clinical Principles of Restorative Dentistry
Test Cases
 

 

 

Gladys L. Benerd School of Education
Department Outcomes Curriculum Maps Assessment Plans Assessment Methods Assessment Data Assessment Reports
Curriculum and Instruction         CURR 130, Lesson Plan Narrative  
Education Professional Dispositions Expected of All Candidates Pupil Personnel Services
Teacher Education
Goals Aligned with University Goals
Aligned Assessment System, PPS
Aligned Assessment System, TCHR
Teacher Education Assessment System
Overview of Assessment Measures
Aligned Assessment System, PPS
Aligned Assessment System, TCHR
Scoring Guide Related to Diversity
Directed Teaching Rubric, Draft
EDUC 161, ESA4-CAT3 Rubric
EDUC 162, ESA#5 Rubric
Teacher Education Key Transition Point Rubric
Questions for Professional Practice Interviews
PACT Teaching Event Results
EDUC 161, Grades and Scores
Master's Orals 2006-2009
Teacher Candidate Demonstration of Proficiencies Related to Diversity

CTC Biennial Report-Section B Summary
CTC Biennial Report, SLP
CTC Biennial Report, SLP II
CTC Biennial Report, Special Education
CTC Biennial Report, Interim Programs, 2008
Assessment Related to Diversity
EDUC 140 Report
EER Essay
CTC Biennial Report, Interim Programs, 2010
Master's Orals 2006-2009
Overview Changes
Portion of Biennial Report - Submitted October 8, 2008
Teacher Education, 2005-2008 Graduates
Educational Administration and Leadership   Pre-Administrative Services
Aligned Assessment System
Components of Candidate Assessment Systems
Aligned Assessment System
Field, Experience Tools, Draft
Fieldwork Rubric
  CTC Biennial Report
CTC Biennial Report II
Follow-up Studies for 2006-2008 Graduates CTC Biennial Report 2010
Candidate Performance
Education and School Psychology

 

Assessment System

 

Rubrics for Evaluating Portfolios
Rubrics

 

 

CTC Biennial Report
CTC Biennial Report, Tables
CTC Biennial Report, 2010
2004-2006 Graduates
Biennial Report October 7, 2008
Use of Data for Program Changes

 

School of Engineering and Computer Science
Department Outcomes Curriculum Maps Assessment Plans Assessment Methods Assessment Data Assessment Reports
Bio-Engineering Outcomes Curriculum Map Assessment Plan      
Civil Engineering Outcomes Curriculum Map   CIV 180 Project Program Rubric   Assessment Report
Computer Engineering Outcomes Curriculum Map   CE Project Rubric Form, Draft   Assessment Report
Computer Science Outcomes Curriculum Map Assessment Plan Rubric and Data Presentation
Major Field Exam Summary
Rubric and Data

MFT Analysis
Co-Op         Employer Evaluations, Anon  
Electrical Engineering Outcomes Curriculum Map        
Engineering   Science Masters Outcome
WASC Information Request
Masters Outcomes Project Rubric Form
Team Peer Evaluation Rubric
WID Rubric
Writing Rubric
ENGR 30, Tech Writing Summary
ENGR 30,Tech Writing Summary, Summer 2011
 
Engineering Management Outcomes Curriculum Map Assessment Plan      
Mechanical Engineering   Curriculum Map Assessment Plan      
Physical Engineering Outcomes Curriculum Map

Assessment Plan

 

Assessment Outcome, Fall 2010
Assessment Outcome, Spring 2011

 

 

School of International Studies
Department Outcomes Curriculum Maps Assessment Plans Assessment Methods Assessment Data Assessment Reports
International Studies Mission and Learning Outcomes
Revised Mission Vision and Core Learning Outcomes, July 2011

 

Assessment Plan
Summary

Intercultural Development Inventory Instrument Validity Hammer, 2010 Report on Use of IDI

IDI Article
MAIR SIS Learning Assessment Report, June 2011
Assessment Progress Report, Spring 2011

 

McGeorge School of Law
Department Outcomes Curriculum Maps Assessment Plans Assessment Methods Assessment Data Assessment Reports
Law LLM Transitional Business Outcomes
JD Program

Curriculum Map

 

   

Bar Exam Research Team Memo

 

Office of Learning and Assessment
Department Outcomes Curriculum Maps Assessment Plans Assessment Methods Assessment Data Assessment Reports
Learning and Assessment    

 

   

2010-2011 Annual Report

 

Library
Department Outcomes Curriculum Maps Assessment Plans Assessment Methods Assessment Data Assessment Reports
Library Information Competency, PACS2

 

 

   

PACS 2 Research Paper Assessment Project Report
SAILS Executive Summary

 

Thomas J. Long School of Pharmacy and Health Sciences
Department Outcomes Curriculum Maps Assessment Plans Assessment Methods Assessment Data Assessment Reports
Pharmacy and Health Sciences PCSC Outcomes
Performance Objectives
Pharmacy Curriculum Map
PharmD Curriculum Map
PharmD Curriculum Map II
PCSP Assessment Plan, 2010
PCSP Assessment Plan, 2002
Pharmacy Assessment Plan
Bachelor of Arts in Applied Science
PCSC Assessment Tools
PCSP Course Evaluation
Ophthalmic Otic
Pulmonary Devices
Blood Glucose Assessment
Patient Counseling Assessment Form 2010, Smoking Cessation
Rubrics, 2010
Geriatrics IPPE Course Grading Structure
HCO Reflection Form
HCO Semester Activities Form
IPPE Precept Evaluation of Student Final, 2010
Assessment Data 2010
Experiential Education in Pharmacy Report
Curricular Revision Examples, 2010
Curriculum Design and Revision
Overall School and University Wide Learning Objectives
PharmD Program and University Wide Learning Objectives
PharmD Program and University Wide Learning Objectives II
Physical Therapy Outcomes Curriculum Map
Course Objectives Map
Matrix
Outcomes Maps
Assessment Plan
Alumni 1 Year Survey
Alumni 5 Year Survey
Employer Survey
Clinical Performance
  Assessment Results
Assessment Results II
Experiential Education Report
Physical Therapy and University Wide Learning Objectives
Analysis
Assessment Report CO1
Assessment Report CP3
Assessment Report CP4
Assessment Report P4
Assessment Report P4.2
Assessment Report P4.3
Assessment Report P4.4
Speech-Language Pathology Outcomes
MS Graduate Program Outcomes

Curriculum Map

Assessment Plan

SLP 125, Project Grade 2008
SLP 125, Project Grade 2010
Project Grade 2011
Audiology Skills Rubric
Aural Rehabilitation Skills Rubric
Final Grading 2010
Junior Clinician Activity Eval 10/09
Observation instructions, Spring 2011
Visual Phonics Paragraph, Spring 2010
 

CTC Biennial Report

 

Division of Student Life
Department Outcomes Curriculum Maps Assessment Plans Assessment Methods Assessment Data Assessment Reports
Campus Recreation See Student Life See Student Life

2010

  Badminton
Rugby

2011

Career Resource Center (CRC) See Student Life See Student Life 2010-2011 Assessment Method
Assessment Method II
2010-2011

Assessment Report
Assessment Report, 2011

Center for Community Involvement (CCI) See Student Life See Student Life Fall 2010 Assessment Method 2011 Spring 2011
Chaplain See Student Life See Student Life 2010-2011 2010-2011   Fall 2011
Morris Chapel
Community Involvement Program (CIP) See Student Life See Student Life 2010 2011   2010-2011
Counseling Services See Student Life See Student Life Assessment Plan   2011 Assessment Report
Greek Life See Student Life See Student Life Assessment Plan   Spring 2011 Spring 2011
Health Services See Student Life See Student Life Assessment Plan Assessment Method Immunization Survey Assessment Report
Housing and Residential Life See Student Life See Student Life Assessment Plan Safe Trick-or-Treat Survey Assessment Data
Safe Trick-or-Treat Survey
Training Assessment Report
Spring 2011
Judicial Affairs See Student Life See Student Life Assessment Plan Level 1, 30 Day
Level 1, Pre-Test
Level 1, Post-Test
Survey 2011
Survey
Sexual Misconduct Survey
Assessment Report
Assessment Report, Spring 2011
Leadership and Involvement See Student Life See Student Life 2010 Assessment Method Observer Notes, 2011 Spring 2011
MOVE See Student Life See Student Life   Rubric Appendices Assessment Summary
Multicultural Affairs See Student Life See Student Life 2010-2011 Assessment Method
Kwanzaa Celebration 2011
Assessment Data
Martin Luther King Day-Visions Survey
Colors of Rhythm and Rhyme
Visions INC., Staff
African American Recruitment Retention
Orientation See Student Life See Student Life New Student and Family Programs Plan, 2010-2011   Evaluation 2010, FR
Evaluation 2010, TR
New Student and Family Programs
New Student and Family Programs II
Public Safety See Student Life See Student Life 2010-2011   STRIPES
Excerpt, 2010
Spring 2011
Services for Students with Disabilities See Student Life See Student Life 2010-2011 Faculty and Staff Survey, 2011 2011
2011 II
2011
Student Academic Support Services See Student Life See Student Life 2010-2011
Assessment Method At Risk Intervention Leadership Report
2011
2011, II
Student Advising See Student Life See Student Life 2010-2011 Workshop Evaluation Form, 2010
Workshop Evaluation Form, Spring 2010
Assessment Data
Survey, 2010-2011
Assessment Report
Student Life Outcomes Curriculum Map   Ethical Reasoning
Resume Rubric
Teamwork Rubric
Women's Leadership Conference
  Assessment Report
Educational Equity Programs
Executive Summary Report
Information Technology
SUCCESS See Student Life See Student Life 2010 Assessment Method

Graduate Forum
Graduate Forum II

Assessment Report
Women's Resource Center (WRC) See Student Life See Student Life 2010
2011
The Fact of Asian Women
Twilight: The Aftermath
Avatar: The Story You Might Have Missed
Ladies Night
The report is forthcoming

 

University Wide
Department Outcomes Curriculum Maps Assessment Plans Assessment Methods Assessment Data Assessment Reports
College of the Pacific   Courses to University Wide Objectives and Outcomes       University Outcomes Report Narrative
University Outcomes Report Table
Arthur A. Dugoni School of Dentistry Support of University Learning Objectives          
Diversity           Assessment Plan and Preliminary Report
Course Assessment Report, Gesine Gerhard
Course Assessment Report, Xiaojing Zhou
Course Assessment Report, Psych 029
Education   Goals Aligned with University Goals        
General Education   Curriculum Map
Breadth Skills Map
PACS1 2011-2012
Breadth Program Assessment Proposal
PACS1 CT Rubric
PACS1 Writing Rubric
PACS2 Research Paper Rubric, Draft
  PACS1 CT Report
Thomas J. Long School of Pharmacy and Health Sciences   PharmD Program and University Wide Learning Objectives
Speech-Language Pathology Curriculum Map
       
School of International Studies Outcomes          
Student Life   Curriculum Map        
Sustainability  

 

 

  Survey

Survey Memo


Appendix C: Inquiry Team Reports

Team One: Institutional Distinctiveness
Team Two: Assessment of Student Learning at Pacific
Team Three: Student Success
Team Four: Program Review
Team Five: Sustainability of Educational Effectiveness