Legislative Auditor's Conclusion:

The Washington State Opportunity Scholarship (WSOS) complies with statute and provides scholarships to low- and middle-income students pursuing high-demand STEM and health care degrees. WSOS can improve legislative reporting by coordinating with state agencies for financial aid and employment data.

One Page Overview  |  Print Friendly & Accessible Version  |  Video Overview

September 2019

The Opportunity Scholarship Act created the Washington State Opportunity Scholarship (WSOS) in 2011. WSOS provides scholarships to low- and middle-income students who pursue bachelor's degrees in high-demand science, technology, engineering, mathematics (STEM), and health care fields. The legislation also directed a JLARC review (ESHB 2088).

WSOS complies with statute and provides scholarships to students pursuing a high-demand STEM or health care degree

WSOS has provided more than 21,000 scholarships to over 10,000 low- and middle-income Washington residents between academic years 2012-13 and 2018-19. Scholars must pursue a bachelor's degree in a high-demand STEM or health care field. WSOS is funded through private contributions and state matching funds. In total, WSOS has received $175 million in private contributions and state matching funds since fiscal year 2012.

WSOS has spent $64 million on Opportunity Scholarships and $14 million on non-scholarship costs since fiscal year 2013. Per statute, WSOS reports the amount it pays to its nonprofit program administrator each year. This payment amount only reflects a portion of the total non-scholarship expenditures.

Opportunity Scholars have lower out of pocket costs and fewer student loans than their peers

The Legislature created the Opportunity Scholarship to help make college more affordable. There is no single definition of college affordability. Whether college is affordable or unaffordable is ultimately a policy decision.

To assist with that decision, JLARC staff identified several key affordability metrics that can be used to compare Opportunity Scholars to their peers (students who met WSOS eligibility requirements, and did not receive a scholarship). These include:

  • Out of pocket costsCost of attendance not covered by grants and scholarships. (net price of college).
  • Percentage of students borrowing money through loans, and borrowers' annual loan amounts.
  • Unmet need (cost of college not covered by grants, scholarships, and family contributionsExpected family contribution is determined by a federal methodology based on information reported on student financial aid applications, such as income and number of household members attending college.).

In comparison to their peers, Opportunity Scholars have lower out of pocket costs, student loans, and unmet need. For example, Opportunity Scholars pay an average of $11,000 out of pocket each year while their peers pay an average of over $13,000.

Opportunity Scholars are more likely to continue their education than their peers. Early recipients have graduated with degrees in high-demand fields.

One of WSOS's objectives is to increase the number of degrees earned in high-demand STEM and health care fields. While some Opportunity Scholars have graduated, many are still pursuing a degree and more time is needed to identify their outcomes. To date, eighty-four percent of those graduating earned a degree in a high-demand field. Scholars who have not yet graduated are more likely to continue their education than their peers.

WSOS can improve its legislative reporting with student financial aid and employment data collected by state agencies

WSOS issues annual reports to the Legislature that include the information required by statute. These reports do not assess whether WSOS meets the legislative intent to help make college more affordable or help Scholars enter middle-income jobs. The reports provide limited information about Opportunity Scholars' employment and earnings. WSOS does not currently have access to state financial aid and employment information that would improve legislative reporting in the future.

The Opportunity Expansion Program funded three university STEM programs. All have secured funding for the future.

The Opportunity Expansion Program received $6 million through a state Research & Development (R&D) tax credit before the credit expired in 2015. The WSOS Board awarded a total of three Opportunity Expansion Program grants to Central Washington University, Western Washington University, and the University of Washington. These grants focused on capacity building and expansion in STEM-related teacher education, computer science, and engineering. All three grant recipients have secured future funding for these STEM programs. It is too soon to evaluate how these grants impact student outcomes (e.g. graduation, employment, earnings).

Legislative Auditor Recommendations

The Legislative Auditor makes two recommendations to improve legislative reporting.

  1. WSOS should develop and implement a plan to coordinate with state agencies such as the Office of Financial Management's Education Research and Data Center, the Washington Student Achievement Council, and the Employment Security Department to provide additional information in its annual legislative reports. Planning efforts should focus on how to compile, protect confidentiality, and include financial aid and employment data on Opportunity Scholars. WSOS should include updates on its plan implementation and any additional metrics in its annual legislative reports.
  2. WSOS should include its overall non-scholarship costs, including the total cost to administer the Opportunity Scholarship program, in its annual legislative report.

You can find additional information on the Recommendations tab.

This report was updated on October 16, 2019 to clarify the industries of employment and counties of residence for Opportunity Scholars.

Preliminary Report: Opportunity Scholarship and Opportunity Expansion Programs

September 2019

Report Details

1. WSOS funds students in STEM and health care

The Washington State Opportunity Scholarship complies with statute and provides scholarships to students pursuing high-demand STEM or health care degrees

The Legislature created the Washington State Opportunity Scholarship (WSOS) Program in 2011. WSOS provides scholarships to low- and middle-income students pursuing bachelor's degrees in high-demand science, technology, engineering, mathematics (STEM) and health care fields (ESHB 2088).

WSOS is not a state government agency. It is an entity governed by a board appointed by the Governor, and it is staffed by a non-profit administrator. WSOS receives both public and private funding for the scholarship program. The legislation creating WSOS also directed a review by the Joint Legislative Audit and Review Committee (JLARC).

The 2018 Legislature expanded WSOS to include scholarships for students who pursue professional technical certificates or associate degrees in high employer demand fields, or advanced degrees in the health professions. Due to this timing, these new scholarship programs were not part of JLARC's study directive and are outside the scope of this report.

Students must meet residency, income, and education criteria to qualify for scholarships

WSOS began awarding scholarships in academic year 2012-13 based on eligibility requirements set by statuteRCW 28B.145 and the WSOS Board of Directors. Opportunity Scholars must:

  • Be a Washington resident who received their high school diploma or equivalency degree in Washington and have not yet earned a bachelor's degree.
  • File a Free Application for Federal Student Aid (FAFSA) or Washington Application for State Financial Aid (WASFA).
  • Have a family income at or below 125% of the median family income in Washington adjusted for family size.
  • Have a cumulative minimum GPA of 2.75.
  • Pursue an eligible education program defined by the WSOS Board at an eligible Washington college or university (link to list of eligible majors).
  • Have an eligible class standingThis requirement did not apply to the first cohort of Opportunity Scholars. (e.g. high school senior at time of application).

WSOS provides students up to $22,500 in scholarship funds over a maximum of five years. The amount awarded each year depends on how many credits the student has completed. Opportunity Scholars typically receive up to:

  • $2,500 in their first or second year of college.
  • $5,000 in their third year if they have been accepted into an eligible major.
  • $7,500 in their fourth year.
  • $5,000 in their fifth year (if necessary).

WSOS also offers professional development, industry mentoring, and skill building workshops to prepare Opportunity Scholars for the workforce. In addition, WSOS has developed two student support programs that all Scholars must participate in as of academic year 2018-19: the Scholar Lead Program offers peer mentoring to newly selected students, and the Skills that Shine program connects third-year Scholars to industry mentors.

Private donations and state matching dollars have funded more than 10,000 Opportunity Scholars

WSOS is funded through a combination of private contributions and state matching dollars. As statute directs, WSOS established two accounts to operate the program. One provides scholarships, and the second is used to build an endowment for future expenditures. To date, WSOS has received a total of $175 million. This includes:

  • $88 million in private donations. Four donors have contributed 93% of these private funds. Only one private donor has contributed to the endowment account, which had a balance of $28 million at the end of 2018. WSOS reports they also have approximately $12 million in pledged private donations not yet received.
  • $87 million in state matching funds.

The WSOS Board and program administrator have not set annual fundraising goals, as directed by statuteRCW 28B.145.020(6) directs the WSOS Board and the program administrator to solicit funds and set annual fundraising goals.. WSOS reported they have focused on soliciting large, private donations, and have set a $1 million goal for one annual fundraising event. WSOS also indicated that they have received enough funds overall to provide scholarships and awards at the current level for Scholars beginning in academic year 2025 through their graduation.

WSOS provides scholarships to low- and middle-income Washington residents

Between academic year 2012-13 and 2018-19, WSOS spent $64 million on more than 21,000 annual scholarships to over 10,000 studentsA student may receive a scholarship more than once (e.g. in consecutive years).. Opportunity Scholars are low- and middle-income Washington residents pursuing a high-demand STEM or health care major. Nearly 70% of the Scholars have attended a public four-year college or university. Additional Scholar information is available in the Tableau dashboard below.

Screen shot of interactive dashboard on Tableau public
Note: The Tableau dashboard was updated on October 16, 2019 to identify how each county ranks in terms of the percentage of Opportunity Scholars per capita.

WSOS and partners work together to manage and administer the Opportunity Scholarship

Statute and contracts between partners outline the roles and responsibilities for the WSOS Board, the Washington Student Achievement Council (WSAC), and a nonprofit program administrator.

  • WSOS is established in law and managed by an 11-member, governor-appointed Board of Directors (Board). The Board sets WSOS's overall strategic direction, determines eligible education programs, approves selection criteria, and sets the annual scholarship award amount.
  • WSAC is a separate state agency that reviews certain student eligibility requirements. It also confirms the amount of private donations that WSOS receives and distributes the state matching funds equal to the private donations.
  • Washington STEM is currently the private nonprofit program administrator that provides administrative support to WSOS, distributes scholarships, manages WSOS's finances, and coordinates with WSAC to receive state matching funds.

The WSOS Board hired an executive director in 2014 and authorized the director to hire 18 additional staff members in the following years. WSOS staff serve the Board and are employees of Washington STEM. The staff's roles and responsibilities are not defined in statute. Some are involved with fundraising, selecting scholars, and providing Scholar supports.

Exhibit 1.1: WSOS works with its partners to select and distribute Opportunity Scholarships
Flow chart of application process and financial aid package through WSOS partners
Source: JLARC staff analysis.

WSOS has spent $14 million on non-scholarship costs since fiscal year 2013

The WSOS Board approves an annual operating budget for the scholarship program. As of January 2019, WSOS had spent $14 million on non-scholarship costs. This $14 million includes spending on:

  • Salaries, taxes, and benefits for WSOS staff.
  • Consultants and professional services.
  • Fundraising, staff development, and lobbying.
  • General operating costs.
  • Program administration fees.
  • Providing direct Scholar supports.

These expenditures do not directly fund Opportunity Scholarships. WSOS has spent $64 million on scholarships since academic year 2012-13. The program administration fee is just one portion of the total non-scholarship costs (see Tab 4).

Exhibit 1.2: Annual non-scholarship costs have ranged from $1.2 million to $3 million
Bar graph of non-scholarship costs broken out by fiscal year
Source: Washington STEM and College Success Foundation financial data for WSOS program.

Preliminary Report: Opportunity Scholarship and Opportunity Expansion Programs

September 2019

Report Details

2. Opportunity Scholars pay less out of pocket than peers

Compared to their peers, Opportunity Scholars pay lower out of pocket costs and borrow less

The Legislature created the Washington State Opportunity Scholarship (WSOS) to help make college affordable by "mitigat[ing] the impact of tuition increases" for students pursuing a bachelor's degree in a high-demand STEM or health care field (RCW 28B.145.005). The Legislature directed JLARC to evaluate whether WSOS is meeting this policy objective (RCW 28B.145.080).

JLARC staff identified key college affordability metrics

JLARC staff reviewed academic literature, federal and state documents, and reports from policy organizations. There is no consensus on a single definition of college affordability. Whether college is affordable or unaffordable is ultimately a policy decision informed by individual values and preferences.

However, several key metrics exist for comparing whether some students pay less than others to attend college. These metrics include:

  • Out of pocket costs (net price of college).
  • Percentage of students borrowing money through loans.
  • Annual amount of loans for borrowers.
  • Unmet need (cost of college not covered by grants, scholarships, and family contributions).
  • Total grant and scholarship awards.

See Appendix B for more details.

Opportunity Scholars have lower out of pocket costs, student loans, and unmet need than their peers

JLARC staff compared Opportunity Scholars to a group of their peers who met WSOS's eligibility requirements, but did not receive scholarships. JLARC staff used data from the Office of Financial Management's Education Research and Data Center (ERDC) for this analysis. ERDC combined financial aid data from WSAC for the first five years of the Opportunity Scholarship with other data and removed confidential student identifiers.

On average, the data shows that Opportunity Scholars have lower out of pocket costs, student loans, and unmet need than their peers. While our analysis identified differences between these two groups, we did not determine the causes of those differences. Our analysis did not establish that the Opportunity Scholarship caused any differences in college affordability between Scholars and their peers.

Additional information on the data and methodology used to construct the peer group is available in Appendix A.

Compared to their peers, Opportunity Scholars pay lower out of pocket costs and have lower unmet need

Both Opportunity Scholars and their peers are students with family incomes below 125% of the median family income in Washington. Opportunity Scholars, on average, used less of their family income to meet their out of pocket costs.

For example, in academic year 2016-17, an Opportunity Scholar with a family income of approximately $38,000 would have out of pocket costs equal to almost 18% of their family income. Comparable students not receiving the Opportunity Scholarship would have costs equal to 28% of their family income. This means that Scholars, on average, used less of their family income to pay for college than their peers.

Exhibit 2.1: Opportunity Scholars use less of their family income to pay out of pocket costs

Bar chart grouping income levels and comparing opportunity scholars by their peers

Source: JLARC staff analysis of data from the Office of Financial Management's (OFM) Education Research and Data Center (ERDC) for academic year 2016-17.

Overall, the average out of pocket costs (net price of college) for students receiving Opportunity Scholarships were lower than their peers in each of the first five years of the program (Exhibit 2.2).

When compared to their peers, Opportunity Scholars, on average, have lower unmet need when it comes to college costs that are not covered by grants, scholarships, and family contributions (Exhibit 2.2). Students with higher need may need to use savings, work income, loans, or other sources to pay the cost of attendance.

Exhibit 2.2: Opportunity Scholars have lower out of pocket costs and unmet need
One side of graphic shows the out of pocket costs comparison between opportunity scholars and their peers. The other side shows the cost of attendance not covered by grants, scholarships, or family contributions comparison between opportunity scholars and their peers.
Source: JLARC staff analysis of data from the Office of Financial Management’s (OFM) Education Research and Data Center (ERDC) for academic years 2012-13 through 2016-17.

Compared to their peers, Opportunity Scholars take out fewer loans and borrow less each year

As Exhibit 2.3 shows, on average, Opportunity Scholars took out fewer loans and borrowed less than their peers for each of the first five years of the program.

Opportunity Scholars also reported via interviews and surveys that the Scholarship helped to minimize the amount of money they borrowed for college.

Exhibit 2.3: Opportunity Scholars took out student loans at a lower rate and have a lower annual loan amount
One side of graphic shows the percentage of student loan comparison between opportunity scholars and their peers. The other side shows the annual loan amount comparison between opportunity scholars and their peers.

 

Source: JLARC staff analysis of data from the Office of Financial Management’s (OFM) Education Research and Data Center (ERDC) for academic years 2012-13 through 2016-17.

WSOS is only one piece of a student's overall funding package

Opportunity Scholars also receive state, federal, and institutional grants and scholarships (Exhibit 2.4). These are packaged by the students' individual colleges and universities. WSOS does not have access to information about additional financial aid that Scholars may receive (see Tab 4 for more information).

Since WSOS is only one piece of a Scholar's overall funding package, it is difficult to make definitive statements about its affect on college affordability. When Scholars receive more funds from one source, it is possible that they will receive less from another. In addition to receiving grants, scholarships, and loans, approximately 52% of the Scholars worked while in school during the first five years of the program.

Bar graph of state and federal grants and scholarships for academic years 2012 - 13 through 2016 - 17.

 

 

Preliminary Report: Opportunity Scholarship and Opportunity Expansion Programs

September 2019

Report Details

3. Opportunity Scholars graduate in high-demand fields

Opportunity scholars are more likely to continue their education than their peers. Early recipients have graduated with degrees in high-demand fields.

One purpose of the Washington State Opportunity Scholarship (WSOS) is to increase the number of high-demand degrees earned in STEM and health care (RCW 28B.145.005).

While over 1,900 Opportunity Scholars have graduated from the first five cohorts, most Opportunity Scholars are still pursuing their degrees. It is therefore too early to analyze graduation rates for most of the program's scholars. JLARC staff analyzed education records for scholarship awards for academic years 2012-13 through 2016-17. This analysis includes the graduation rates of early scholarship recipients and the rate that first year students returned for their second year of school. Data is not yet available for students who received scholarships between 2017 and 2019.

Opportunity Scholars return to school in their second year at higher rates than their peers

For the first four years that the Opportunity Scholarship was offered, 96% of award recipients returned to school between their first and second academic years. This compares to 89% of their peers (see Appendix A).

Federal and state agencies use the rate in which first year students return for a second year as an early indicator of whether a student will graduate. Although many Opportunity Scholars have not yet graduated, a first year student who returns for their second year of college is more likely to graduate than a student who does not return.

The first group of Scholars graduated at higher rates than their peers. Most graduates earned degrees in STEM or health care.

Students who received Opportunity Scholarships as freshmen in academic years 2012-13 and 2013-14 have graduated within four years at higher rates than their peers.

For example, 49% (227) of the initial group of Scholars have graduated in four years compared to 19% of their peers. This does not include students who graduated after the 2016-17 academic year.

Of all Scholars who graduated, regardless of the year they received a scholarship, 84% earned a degree in a high-demand STEM or health care major. These degrees were concentrated in five main fields of study.

Bar graph of top 5 fields of study for Opportunity Scholar graduates (Biological Sciences at 27%, Engineering at 26%, Healthcare at 15%, Computer and Information Sciences at 13%, and Physical Sciences at 5%).

Too soon for a full evaluation of the employment and earnings of graduates

Using data from the Employment Security Department (ESD), JLARC staff analyzed the first year of employment and earnings for Opportunity Scholars and peer students who graduated between 2013 and 2016. Additional data on earnings and employment is not yet available for these graduates, or for more recent Opportunity Scholar graduates.

Initial results suggest Opportunity Scholars are employed in middle-income jobs

One of WSOS's objectives is to help Scholars enter middle-incomeMiddle-income is defined as earning 200-500% federal poverty line for a family of four. In 2017, this was $49,200 to $123,000. jobs to meet market demands (RCW 28B.145.005). JLARC staff analyzed employment data for scholarship graduates and their peers who have completed at least one year of employment. The Opportunity Scholars are more likely to be employed and earn higher wages than their peers.

  • 83% of Opportunity Scholars (over 1,400) were employed within 1 year of graduation compared to 77% of their peers.
  • Of those employed full-time, Opportunity Scholars earned $66,100 on average compared to an average of $58,200 for their peers.
  • 69% of Opportunity Scholars who were employed full-time earned at least middle-income wages one year after graduation compared to 56% of their peers.

Graduates are concentrated in five industries

Of the graduates who are employed, 66% are concentrated in one of five industries. The ESD tracks data about the type of employers where individuals work, but does not collect information regarding specific job titles or occupations of employees. The graduates may work in a variety of occupations within these five industries.

Bar graph of top 5 industries of employment for Opportunity Scholar graduates (Health Care and Social Assistance at 19%, Educational Services at 15%, Professional, Scientific, and Technical Services at 14%, Construction at 10%, and Administrative & Support and Waste Management & Remediation Services at 8%).

 

Preliminary Report: Opportunity Scholarship and Opportunity Expansion Programs

September 2019

Report Details

4. WSOS could improve legislative reporting

WSOS's annual legislative reports contain reliable, required information. WSOS could improve the information it reports to the Legislature with additional data collected by state agencies.

The Washington State Opportunity Scholarship (WSOS) Board and program administrator are required to submit an annual report to the Legislature. The report must include the following information identified in statute (RCW 28B.145.070):

  • Number of applicants and award recipients disaggregated by race/ethnicity, gender, family income, and county.
  • Number and amount of scholarships awarded.
  • Institution name and graduation rates of the schools where Opportunity Scholars are enrolled.
  • Amount of funds raised through private sources and the state match.
  • Amount spent on program administration fees.

The Legislature also directed WSOS to help make college more affordable and to help its Scholars enter middle-income jobsMiddle-income is defined as 200-500% of the federal poverty level for a family of four. In 2017, this was $49,200 to $123,000. to meet market demands (RCW 28B.145.005). WSOS has not included information in its annual reports on whether it is meeting these goals.

WSOS's annual reports contain reliable information for statutorily required elements

WSOS collects demographic and financial information from its applicants. It supplements its own data with information it purchases from a national non-profit education organization. This supplemental data comes from the specific institutions where Opportunity Scholars are enrolled. It includes Scholars' majors and graduation status. Collectively, WSOS uses this information to confirm scholarship eligibility, distribute awards, and prepare its annual reports.

Program administration fee reported to Legislature is just one portion of the total non-scholarship costs

Per statute, WSOS also collects program financial information and reports the amount it pays to its nonprofit program administrator each year. This reported amount only reflects a portion of the total non-scholarship costs (see Tab 1 for total non-scholarship costs). For example, in fiscal year 2018, the amount paid to the program administrator was 12.5% of the total non-scholarship costs.

JLARC staff was able to replicate the data that WSOS includes in its annual reports. The reports accurately reflect the characteristics of Opportunity Scholars and program finances.

Exhibit 4.1: WSOS complies with statutory requirement to issue annual legislative reports
Images of 2018, 2017, 2016, and 2015 Legislative Report covers
Source: WSOS legislative reports.

WSOS could improve its annual legislative reporting with state financial aid and employment data

Multiple state agencies currently collect financial aid and employment information on students and recent graduates. JLARC staff utilized this data to display specific aggregate information about Opportunity Scholars for this report. This report demonstrates that this data can be informative for exploring whether the Opportunity Scholarship helps make college more affordable and improves the employment and earning outcomes of its graduates.

The Office of Financial Management's Education Research and Data Center (ERDC) combined data and information from multiple sources to assist JLARC staff in its analysis for this report. The data includes:

  • Financial aid data from the Washington Student Achievement Council (WSAC), which describes a student's finances, grants, scholarships, and loans.
  • Public higher education information from the Public Centralized Higher Education Enrollment System (PCHEES) and the State Board for Community and Technical Colleges (SBCTC), which identifies information on courses and degrees completed.
  • Employment and earnings data for Washington workers from the Employment Security Department (ESD).
  • Enrollment information from private colleges and universities such as major and degree information.

WSOS does not have direct access to the data collected by these state agencies, which is confidential under state and federal laws. WSOS has attempted to collaborate with certain state agencies for specific student-level demographic and outcome information. However, WSOS could further engage with state agencies to discuss whether there are methods to collaborate on data analysis and data sharing in order to provide non-confidential aggregate information in their future legislative reports.

  • Financial aid data would allow WSOS to update the Legislature on key affordability metrics for Opportunity Scholars and whether the program continues to help make college more affordable.
  • State employment and earnings data would allow WSOS to update the Legislature on the employment and earnings of Opportunity Scholars who graduate. WSOS has collected limited information on Scholars' employment and earnings through surveys. Its last follow-up survey in 2018 included a random sample of 431 graduates and had a response rate of 51%. This reflects about 6.5% of the nearly 3,400 Scholars who have graduated since the program began.
  • Any efforts to obtain aggregate results for the Legislature would need to be designed to ensure the ongoing protection of personally identifiable information that is currently managed by multiple state agencies.

WSOS could also improve the understanding of its program costs by including the overall non-scholarship costs and the total amount it spends to administer the Scholarship in its annual legislative reports.

The Legislative Auditor makes two recommendations to improve information reported to the Legislature

  1. WSOS should develop and implement a plan to coordinate with state agencies such as ERDC, WSAC, and ESD to provide additional information in its annual legislative reports. Planning efforts should focus on how to compile, protect confidentiality, and include the following types of data on Opportunity Scholars:
    • Financial aid and key affordability metrics such as out of pocket costs (net price of college), unmet need, and student loans.
    • Employment rates and earnings.
    • Whether they are employed in market demand fields or industries.
    WSOS should include updates on its plan implementation and any additional metrics in its annual legislative reports.
  2. WSOS should include its overall non-scholarship costs, including the total cost to administer the Opportunity Scholarship program, in its annual legislative report. As required by statute, WSOS currently includes the amount it pays to its nonprofit administrator in its annual legislative reports. This amount only reflects a portion of the total costs to administer the program.

Preliminary Report: Opportunity Scholarship and Opportunity Expansion Programs

September 2019

Report Details

5. Opportunity Expansion Program funded 3 universities

Three universities received WSOS grants to support STEM programs and each has secured ongoing funding

The Opportunity Scholarship Act (ESHB 2088) that created the Opportunity Scholarship also included the Opportunity Expansion Program (OEP). Through the Expansion Program, WSOS provides funds to college and university programs that are focused on increasing the number of students pursuing bachelor's degrees in STEM and health care fields. The WSOS Board is authorized to select and award opportunity expansion grants to institutions of higher education.

Initial awards went to three public university programs focused on STEM education

In 2016, the Board awarded $6 million in expansion grants to three state universities. The Board formed a working group to develop a process for seeking and evaluating OEP grant proposals from colleges and universities. In the initial round of applications, 20 institutions submitted proposals and five of these schools were asked to provide additional information.

The Board awarded grants to Central Washington University (CWU), Western Washington University (WWU), and the University of Washington (UW).

The working group and Board chose to focus the awards on capacity building and expansion in STEM-related teacher education, computer science, and engineering. The Board's emphasis was to provide one-time initial grant funding for initial development and implementation of programs, but not funding for ongoing program delivery.

The original fund source for the Expansion Program has expired

The Opportunity Expansion Program was designed to be funded by private Washington companies that agreed to donate all or a portion of a research and development business and occupation tax credit. Microsoft was the only company that donated its credits, which totaled $6 million. The tax credit expired in 2015 and the state has not identified a new source of funding for the Expansion Program.

Statute permits the Board to make recommendations to the Legislature for sources of funding (RCW 28B.145.020). The Board and program administrator have not raised funds or promoted the tax credit option to businesses since the program began, and the Board has not made any recommendations to the Legislature on new funding sources for the Expansion Program.

CWU, WWU, and UW have all secured funding for future years

The following three universities received Opportunity Expansion grants in 2016 and have secured future funding for their programs:

  • Central Washington University (CWU) received $2.2 million to adopt the UTeach curriculum developed by the University of Texas to address the shortage of STEM teachers in Washington. CWU's Teach STEM program allows students to earn both their STEM degree and teaching certification in four years. The OEP grant covered the initial start-up costs of the program and funded advisers, faculty, and interns through academic year 2018-19. Deans from the University's College of Education and Professional Studies and College of Sciences have committed to fund 3.5 Teach STEM full-time employees to continue the program after the OEP grant ends. Program staff are also working with the CWU Foundation to create an endowment that will help fund the program in the future.
  • Western Washington University (WWU) received $1.6 million to expand capacity for students seeking computer science degrees as well as develop a new program offering computer science teaching certifications. The grant allowed WWU to hire four computer science faculty to serve more computer science students each year. The University's Provost has committed to permanently fund the faculty positions after the OEP grant expires.
  • The University of Washington (UW) received $2.2 million to expand capacity of the Washington State Academic RedShirt program (STARS). The program enrolls engineering students from low-income families and gives them an additional year of academic prep work to help them succeed in the College of Engineering. The OEP grant covered funding for the STARS program for three years, ending in academic year 2019. The grant allowed the program to hire new advisers and offer more workshops in math and science. During the 2019 legislative session, the Legislature appropriated $500,000 in fiscal year 2020 and another $500,000 in fiscal year 2021 to support the program (ESSHB 2158).

Too soon to determine grant program impact on student outcomes

The three university programs created pathways to high-demand STEM degrees. Initial data suggests that more students will graduate with STEM degrees with the help of these programs. However, it is too soon to determine the total impact the grants have on student outcomes (e.g. graduation, employment, earnings) because many of the students have been enrolled for three years or less. WSOS has required periodic updates from the three grant recipients, but it is not tracking individual students benefiting from these programs.

Preliminary Report: Opportunity Scholarship and Opportunity Expansion Programs

September 2019

Report Details

Appendix A: Peer group methodology

JLARC staff used an analytical approach called machine learning to identify the group of peers who were compared to Opportunity Scholars

The peer group consists of all students who met the WSOS eligibility requirements but did not receive a scholarship

The WSOS eligibility requirements are:

  1. Be a Washington resident who received their high school diploma or equivalency degree in Washington and have not yet earned a bachelor's degree.
  2. File a Free Application for Federal Student Aid (FAFSA) or Washington Application for State Financial Aid (WASFA).
  3. Have a family income at or below 125% of the median family income in Washington adjusted for family size.
  4. Have a cumulative minimum GPA of 2.75.
  5. Have an eligible class standingThis requirement did not apply to the first cohort of Opportunity Scholars..
  6. Pursue an eligible education program defined by the WSOS Board at an eligible Washington college or university (link to list of eligible majors).

Reliable data about intended major was unavailable for many students

The Washington State Office of Financial Management’s Education Research and Data Center (ERDC) provided data for JLARC staff’s analysis.  ERDC compiles data from multiple sources, including the Washington Student Achievement Council (WSAC), the public higher education institutions (PCHEES), the State Board for Community and Technical Colleges (SBCTC), and the Employment Security Department (ESD). ERDC links individuals across the sources, assigns each individual a unique identifier, and removes personal identifying information (e.g. SSN, date of birth). ERDC can also link data from non-state sources, such as private colleges and universities.

Data provided by ERDC contained information about the first four eligibility requirements. However, the data did not reliably indicate a student’s major, or whether a student was planning to pursue a high-demand STEM or health care degree.

JLARC staff developed a predictive model using machine learning to determine which peer students are pursuing a STEM or health care degree

What is machine learning?
Machine learning means programming a computer so it can learn from data. For example, the spam filter on your email is a machine learning program. Using an algorithm, it learns to flag spam emails from samples (e.g., those that you or others have flagged as spam).

To predict whether a student who has not yet graduated is pursuing a high-demand STEM or health care degree, the model compares the student's classes to those taken by others who graduated with a degree in STEM or health care (see list of eligible majors). If the course load is similar, the model predicts that the student is pursuing a STEM or health care degree. 

While simple in principle, the model is complex. The general steps below describe how machine learning was applied in this study:

  1. Identify STEM and health care graduates: flag each graduate as STEM/health care or not based upon their degree at graduation.
  2. Place all classes into four groups (Exhibit A1) based upon the percent of students that took the class and earned a STEM or health care degree.
  3. Build the model: count the number of classes among the four groups for each student. Apply binomial generalized linear model (GLM) on a training set, then apply said model to a testing set of students.
  4.  Test the model’s accuracy: establish a threshold for the model and compare the model’s outputs for students in the testing set to their known degree (STEM/health care or other).

Identify STEM and health care graduates

The graduate data is from the PCHEES system and was provided by ERDC. There were 110,088 total graduates used to develop the model, 35,640 of which graduated with a STEM or health care degree.

Group classes

JLARC staff compiled the course loads for all graduates. Classes were then grouped based on the percentage of students who took the class and graduated with a high-demand STEM or health care degree. This approach identifies the classes that can be used to predict high-demand STEM or health care degrees or predict non-STEM or health care degrees. For example:

  • 93% of graduates that took electromagnetism and 94% of those who took linear analysis (94% STEM) earned STEM degrees. These classes are good indicators of STEM degrees.
  • In contrast, 29% of graduates that took general psychology earned a STEM degree. This means that it is likely not a good indicator of a STEM degree, but may be a good indicator of a non-STEM degree.
Exhibit A1: Groups of all classes at universities
Stacked bar chart with groups and examples of classes in that group. Group 1 (not STEM) Business Writing, group 2 (likely not STEM) Intermediate Algebra, group 3 (likely STEM) Calculus I, group 4 (STEM) Electromagnetism
Source: JLARC staff analysis of data from the Office of Financial Management's (OFM) Education Research and Data Center (ERDC).

Build the model

The algorithm uses class data from a subset of graduates (the “training set”). The algorithm uses a binomial generalized linear model (GLM):

  • JLARC staff input the number of classes, by group, that each student in the testing set took.
  • The model output shows how much the classes in each group contributed to a STEM degree. The range is 0 to 1 (0 means not STEM and 1 means STEM).

As shown in Exhibit A2, Groups 1 and 2 are negatively associated with a student pursuing a STEM degree and Groups 3 and 4 are positively associated.

Exhibit A2: Only classes in Groups 3 and 4 are positively associated with STEM degrees
Independent Variable Estimate Standard Error
Group 1 -0.40 0.005
Group 2 -0.03 0.001
Group 3 0.17 0.003
Group 4 0.41 0.005
Source: JLARC staff analysis of data from the Office of Financial Management's (OFM) Education Research and Data Center (ERDC).

Test the model's accuracy

JLARC staff used data about the graduates who were not in the training set as a testing set. The graduates’ class data was entered into the model and the output was compared to their known degree (STEM/health care or other).

As shown in Exhibit A3, the model accurately showed that students who had higher scores (more classes in Groups 3 and 4) also earned STEM/health care degrees.

Exhibit A3: Binomial general linear model results
Student Group 1 Group 2 Group 3 Group 4 Graduated STEM GLM Output
A 5 14 2 0 0 0.05
B 0 5 4 12 1 0.99
C 0 3 5 1 1 0.60
Source: JLARC staff analysis of data from the Office of Financial Management's (OFM) Education Research and Data Center (ERDC).

The model’s output cannot classify students as STEM or not-STEM without a threshold.

  • Values above the threshold are flagged as STEM.
  • Values below the threshold are flagged as non-STEM.

JLARC staff used the receiver operating characteristic (ROC) curve and the Younden's indexThe threshold value that minimizes type I and II errors while maximizing overall accuracy. to determine a threshold of 0.31. Staff then used cross-validation to determine how many graduates in the testing set were correctly identified. The threshold correctly identified 82% of graduates in the testing set, 82% of the non-STEM graduates in the testing set, and 81% of the STEM graduates in the testing set. The model was also tested on Opportunity Scholars that have graduated. 88% of Opportunity Scholar graduates were correctly identified, 95% of Opportunity Scholar STEM graduates were correctly identified as STEM, and only 51% of Opportunity Scholar non-STEM graduates were correctly identified as non-STEM.

Applying the model to develop the peer group

The model provides an indicator of whether each student intended to pursue a STEM or health care degree. JLARC staff cross-referenced this information with its list of public higher education students that met the first four eligibility requirements to develop the peer group used for the comparisons. A total of 29,848 students met all five requirements and were included in the peer group.

Additional information for metrics reported in Tab 2

JLARC staff used financial aid data collected by the WSACWashington Student Achievement Council to compare Opportunity Scholars to the peer group on key affordability metrics (Appendix B). The ERDCThe Washington State Office of Financial Management's (OFM) Education Research and Data Center compiled the financial aid data with other sources, and removed confidential information identifying individual students.

WSAC requires institutions to submit a report for each student who receives need-based financial aid (e.g. State Need Grant, Federal Pell Grant). The data includes information on federal, state, private, and institutional aid distributed to need-based financial aid recipients. This means the sample of students JLARC staff used for our affordability analysis includes students who received need-based financial aid. It does not include all students attending a Washington college or university.

The financial aid data includes records for each institution a student attended each academic year. Before performing our analysis, JLARC staff aggregated the data so that each student had one record per year. For example, after aggregating the data, a student attending two universities would have a single annual cost of attendance and a total annual amount of aid received from specific sources.

JLARC staff identified and defined Opportunity Scholars as those students in the financial aid data who were recorded as having received an Opportunity Scholarship disbursement greater than zero. Opportunity Scholars attended public and private colleges and universities, and community and technical colleges.

Exhibit A4 and Exhibit A5 provide additional information for the metrics reported in Tab 2.

Exhibit A4: Opportunity Scholars also received federal and state aid
Academic Year College Bound Scholarship Received State Need Grant Received Federal Pell Grant
Number Percent Number Percent Number Percent
2016-17 412 14.1% 1,571 53.9% 1,772 60.8%
2015-16 356 14.6% 1,172 47.9% 1,449 59.3%
2014-15 283 12.6% 1,021 45.5% 1,348 60.1%
2013-14 301 12.2% 1,148 46.3% 1,532 61.8%
2012-13 185 6.6% 1,354 48.3% 1,747 62.4%
Total 1,537 11.9% 6,266 48.6% 7,848 60.9%
Source: JLARC staff analysis of data from the Office of Financial Management's (OFM) Education Research and Data Center (ERDC) for academic years 2012-13 through 2016-17.

 

Exhibit A5: Affordability metrics for Opportunity Scholars and peer group
Affordability Metric Opportunity Scholars Peer Group
Average Annual Amount Number of Records Average Annual Amount Number of Records
Out of Pocket Costs (net price of college) $11,062 12,884 $14,244 58,537
Unmet Need $4,827 12,884 $9,893 58,537
Percentage taking out student loans each year 45.1% 12,884 63% 58,537
Annual student loan amount for those taking loans $7,156 5,806 $9,125 36,902
Annual grants and scholarship amount for those receiving grants and scholarships $16,890 12,884 $11,264 52,197
Note: Each record represents a student during an academic year (AY). A student may appear in multiple years. For example, a student attending college in AY 12-13 and AY 13-14 will be represented in the data twice- once for each year. For Opportunity Scholars, this means the total number of records reported in this Exhibit is greater than the total number of unique Scholars, as a Scholar may have received a Scholarship in multiple years.
Source: JLARC staff analysis of data from the Office of Financial Management's (OFM) Education Research and Data Center (ERDC) for academic years 2012-13 through 2016-17.

Additional information for metrics reported in Tab 3

JLARC staff used public higher education enrollment data (PCHEES) and Employment Security Department (ESD) employment and earnings data to compare Opportunity Scholars' retention, graduation, employment, and earnings to the peer group. Exhibit A6 provides additional information for the metrics reported in Tab 3.

Exhibit A6: Education and employment metrics for Opportunity Scholars and peer group
Metric Opportunity Scholars Peer Group
Number of Graduates 1,962 12,814
Number (percent) of students returning to school from their first academic year to their second academic year (also known as retention rate) 1,041 (95.9%) 8,850 (88.7%)
Number (percent) of graduates after four years in school 227 (48.8%) 1,449 (19.48%)
Number (percent) of graduates earning a degree in a high demand STEM or health care major 1,652 (84.5%) Not Applicable
Number (percent) of graduates employed one year after graduation 1,431 (82.9%) 8,668 (77.3%)
Number (percent) of graduates employed full-time one year after graduation 391 (27.3%) 2,008 (23.2%)
Number (percent) of employed graduates working full-time one year after graduation and earning a middle income 270 (69.1%) 1,125 (56%)
Note: The numbers reported in this exhibit represent unique students.
Source: JLARC staff analysis of data from the Office of Financial Management's (OFM) Education Research and Data Center (ERDC).

Preliminary Report: Opportunity Scholarship and Opportunity Expansion Programs

September 2019

Report Details

Appendix B: Affordability methodology

JLARC staff reviewed literature and interviewed experts to identify key affordability metrics

Legislature did not define college affordability

The Legislature created the Opportunity Scholarship to "help mitigate the impact of tuition increases" (RCW 28B.145.005). JLARC staff interpreted this directive as legislative intent to help make college more affordable for Opportunity Scholars. There is no legislative definition of college affordability.

Two models define college affordability, but no consensus has emerged

JLARC staff identified two models that define whether college is affordable or unaffordable. Neither model has been widely accepted in the federal, state, policy, or academic communities. The models are described below.

Lumina Rule of 10

The Lumina Foundation is private, independent foundation that developed a model of college affordability based on a formula it calls the "Rule of 10." In this model, college is affordable if a student can pay the cost of attendance with a combination of student work, family resources, and student debt. The Rule states that a student should pay no more for college than the funds generated through:

  • The sum of 10 percent of their discretionary incomeDiscretionary income is defined as any income above 200 percent of the federal poverty rate for the student's family size. saved for 10 years.
  • The earnings from 10 hours of work each week while in school.

The formula above also serves as a maximum amount a student should need to borrow to pay for college, if they have not saved, or cannot work. College is unaffordable if the cost of attendance exceeds the amount a student can generate from the Rule of 10.

SHEEO model

The State Higher Education Executive Officers Association (SHEEO) is a national association of state government higher education executives that serves as a liaison between state and federal governments.

The SHEEO model suggests that college is affordable if the cost to attend can be repaid using ten percent of discretionary incomeDiscretionary income is defined as 150% of the federal poverty threshold for a family of three. in the ten years following graduation.

College affordability is ultimately a policy decision

JLARC staff's review of literature, discussions with legislative staff, and interviews with higher education experts found there is no agreement on a common definition for college affordability, nor is there agreement on whether the two models described above are appropriate metrics. Our review did not find federal or state standards, or agreed upon best practices for determining college affordability.

There are multiple issues to consider in order to determine affordability, including whether:

  • Affordability is considered from the perspective of the student or family.
  • Higher education is viewed as a good to be paid for at the time of receipt, or an investment to be paid for over time.
  • The future returns of higher education outweigh the present costs.
  • Student debt is acceptable, and if so, how much.
  • College costs refer to just tuition and fees, or the entire cost of attending college.
These considerations and others make determining if college is affordable or unaffordable a policy decision subject to individual values and preferences.

Key affordability metrics can be used to compare Scholars to peers

JLARC staff reviewed academic literature, state and federal policies and regulations, and interviewed legislative, financial aid, and higher education professionals. Though there is no commonly accepted measurement of optimal affordability, JLARC staff identified common metrics that can help compare whether students face relatively higher or lower college affordability. These metrics allow us to evaluate whether WSOS has helped make college more affordable for Opportunity Scholars compared to other students, even if we cannot conclude whether a particular measurement level equates to being considered "affordable."

Exhibit B.1. Common metrics of college affordability
Affordability Metric Definition Sources
Out of pocket costs (net price of college) Cost of attendance minus total grants and scholarships from federal, state, institution, private, or other sources. U.S. Department of Education's College Scorecard; Washington Student Achievement Council; Education Commission of the States; State Higher Education Executive Officers; academic literature
Unmet Need Cost of attendance minus total grants, scholarships, and family contribution. Washington Student Achievement Council; policy organizations; academic literature
Financial Aid Total amount of financial aid a student receives that does not require repayment. Examples include: grants and scholarships received from federal (e.g. Pell Grant), state (e.g. State Need Grant), institution, and private sources. Work-Study programs are also part of a student's overall financial aid package. Washington Student Achievement Council; Education Commission of the States; Washington State Institute of Public Policy; academic literature
Student Loans Percent of students borrowing money through loans from federal, institution, and private sources each year. U.S. Department of Education; Washington Student Achievement Council; Washington Attorney General; State Higher Education Executive Officers; Western Interstate Commission for Higher Education; Washington Institute for Public Policy; academic literature
Annual Loan Amount Average annual loan amount for students taking out student loans from federal, institution, and private sources each year.
Definitions:
  • The term "cost of attendance" is defined in the federal Higher Education Act and includes: tuition and fees, books and supplies, room and board, transportation and personal expenses. The cost of attendance can vary based on individual circumstances such as dependency status, living situation, and location.
  • The term "family contribution" is defined as the amount a student and student's family are expected to contribute towards the cost of attendance. This is determined by a federal methodology based on information reported on student financial aid applications.
Source: JLARC staff literature review.

Additional affordability analysis for specific sub-groups of Scholars

JLARC staff performed additional analysis for specific sub-groups of Opportunity Scholars and their peers. The data shows that college is likely more affordable for the following sub-groups of Opportunity Scholars than it is for their peers:

  • Scholars who attend one of the six Washington public colleges and universities than their peers attending these schools.
  • Opportunity Scholars at various income levels under 125% of the median family income adjusted for family size than their peers at the same income level.
  • Opportunity Scholars within all reported racial and ethnic groups (e.g. Asian or Hispanic) than their peers in the same racial and ethnic group.

Preliminary Report: Opportunity Scholarship and Opportunity Expansion Programs

September 2019

Recommendations & Responses

Legislative Auditor Recommendations

The Legislative Auditor makes two recommendations to improve legislative reporting

Recommendation #1: WSOS should develop and implement a plan to coordinate with state agencies such as the Office of Financial Management's (OFM) Education Research and Data Center (ERDC), the Washington Student Achievement Council (WSAC), and the Employment Security Department (ESD) to provide additional information in its annual legislative reports.

Planning efforts should focus on how to compile, protect confidentiality, and include the following types of data on Opportunity Scholars:

  • Financial aid and key affordability metrics such as out of pocket costs (net price of college), unmet need, and student loans.
  • Employment rates and earnings.
  • Whether they are employed in market demand fields or industries.

WSOS should include updates on its plan implementation and any additional metrics in its annual legislative reports.

Legislation Required: No
Fiscal Impact: JLARC staff assume plan can be developed within existing resources. Other steps may require additional resources.
Implementation Date: December 2020
Agency Response: To be included with Proposed Final Report

Recommendation #2: WSOS should include its overall non-scholarship costs, including the total cost to administer the Opportunity Scholarship program, in its annual legislative report.

As required by statute, WSOS currently includes the amount it pays to its nonprofit administrator in its annual legislative reports. This amount only reflects a portion of the total costs to administer the program.

Legislation Required: No
Fiscal Impact: JLARC staff assume WSOS can implement within existing resources.
Implementation Date: December 2020
Agency Response: To be included with Proposed Final Report

Preliminary Report: Opportunity Scholarship and Opportunity Expansion Programs

September 2019

Recommendations & Responses

Agency Response

Agency response(s) will be included in the proposed final report, planned for December 2019.

Preliminary Report: Opportunity Scholarship and Opportunity Expansion Programs

September 2019

Recommendations & Responses

Current Recommendation Status

JLARC staff follow up with agencies on Legislative Auditor recommendations for 4 years. Responses from agencies on the latest status of implementing recommendations for this report will be available in 2021.

Preliminary Report: Opportunity Scholarship and Opportunity Expansion Programs

September 2019

More About This Review

Audit Authority

The Joint Legislative Audit and Review Committee (JLARC) works to make state government operations more efficient and effective. The Committee is comprised of an equal number of House members and Senators, Democrats and Republicans.

JLARC's non-partisan staff auditors, under the direction of the Legislative Auditor, conduct performance audits, program evaluations, sunset reviews, and other analyses assigned by the Legislature and the Committee.

The statutory authority for JLARC, established in Chapter 44.28 RCW, requires the Legislative Auditor to ensure that JLARC studies are conducted in accordance with Generally Accepted Government Auditing Standards, as applicable to the scope of the audit. This study was conducted in accordance with those applicable standards. Those standards require auditors to plan and perform audits to obtain sufficient, appropriate evidence to provide a reasonable basis for findings and conclusions based on the audit objectives. The evidence obtained for this JLARC report provides a reasonable basis for the enclosed findings and conclusions, and any exceptions to the application of audit standards have been explicitly disclosed in the body of this report.

Preliminary Report: Opportunity Scholarship and Opportunity Expansion Programs

September 2019

More About This Review

Study Questions

PSQ-01.svg

PSQ-02.svg

Preliminary Report: Opportunity Scholarship and Opportunity Expansion Programs

September 2019

More About This Review

Methodology

The methodology JLARC staff use when conducting analyses is tailored to the scope of each study, but generally includes the following:

  • Interviews with stakeholders, agency representatives, and other relevant organizations or individuals.
  • Site visits to entities that are under review.
  • Document reviews, including applicable laws and regulations, agency policies and procedures pertaining to study objectives, and published reports, audits or studies on relevant topics.
  • Data analysis, which may include data collected by agencies and/or data compiled by JLARC staff. Data collection sometimes involves surveys or focus groups.
  • Consultation with experts when warranted. JLARC staff consult with technical experts when necessary to plan our work, to obtain specialized analysis from experts in the field, and to verify results.

The methods used in this study were conducted in accordance with Generally Accepted Government Auditing Standards.

More details about specific methods related to individual study objectives are described in the body of the report under the report details tab or in technical appendices.

Preliminary Report: Opportunity Scholarship and Opportunity Expansion Programs

September 2019

Contact

JLARC Authors

Casey Radostitz, Research Analyst, 360-786-5176

Jennifer Sulcer, Research Analyst, 360-786-5181

Joshua Karas, Research Analyst, 360-786-5298

Ryan McCord, Research Analyst, 360-786-5186

Valerie Whitener, Audit Coordinator

Keenan Konopaski, Legislative Auditor

Preliminary Report: Opportunity Scholarship and Opportunity Expansion Programs

September 2019

Contact

JLARC Members

Senators

Bob Hasegawa

Mark Mullet, Chair

Rebecca Saldaña

Shelly Short

Dean Takko

Keith Wagoner

Lynda Wilson, Secretary

Representatives

Jake Fey

Noel Frame

Larry Hoff

Christine Kilduff

Vicki Kraft

Ed Orcutt, Vice Chair

Gerry Pollet, Assistant Secretary

Drew Stokesbary