Executive Summary
The 2022 Supplemental Transportation Budget directed JLARC to evaluate Washington State Ferries' design-build contracting approach for hybrid electric ferries. The review must include a comparison to best practices and opportunities for both cost and operational efficiencies. JLARC staff consulted with VARD Marine, Inc. (VARD), a firm with expertise in marine engineering and procurement to assist with the review. VARD's final report is available in Appendix A.
Washington State Ferries is using design-build contract approaches to acquire hybrid electric ferries
Washington State Ferries (WSF) is a division of the Washington State Department of Transportation (WSDOT). WSF's Long Range Plan calls for increasing the state's ferry fleet to 26 vessels and shifting to plug-in hybrid vessels by 2040. Chapter 47.60 RCW requires WSF to use a design-build approach to acquire new ferries. Design-build means that WSF executes a single contract with one shipbuilder to design and construct the vessel. The approach transfers most of the project riskRisks include schedule delays, technical problems, and cost overruns. to the shipbuilder. However, the approach can increase the vessel price as the shipbuilder accounts for the risk.
WSF plans to issue a Request for Proposal (RFP) for the first hybrid electric vessels during the first quarter of 2023. These will be the first vessels acquired under the laws adopted in 2015.
Exhibit: Hybrid electric Olympic class vessels will be the first acquired under the 2015 laws
Source: JLARC staff analysis of WSF documentation.
WSF uses standard cost estimating practices. Current estimates do not convey key assumptions and uncertainty.
Different cost estimation methods are used at different stages of a project. In shipbuilding, it is typical for early cost estimates to vary significantly as more is learned and vessel designs are refined. WSF follows industry practice in starting with top-down cost estimates (e.g., based on previous work) and moving to more detailed bottom-up estimates (e.g., based on actual data). However, WSF does not clearly explain assumptions, data sources, methods, or uncertainty in the estimates that accompany its budget requests. This has reportedly led to confusion for legislators and legislative staff about why estimates change and whether the estimates are comparable.
WSF's cost estimating practices produce single value estimates (e.g., a ferry will cost $220 million). Best practice is to provide a range of possible costs and the likelihood that the actual cost will be at or below a certain amount along the range. This is called a probabilistic approach. WSDOT uses a probabilistic approach for civil engineering projects valued over $100 million. WSF intends to use this process in 2023 for the first hybrid electric ferry acquisition.
Exhibit: A probabilistic approach shows the range of costs. Decision makers can set budgets based on the likelihood that costs will be at or below a certain amount.
Source: JLARC staff depiction based on WSDOT CEVP and notional drawings from the Government Accountability Office and JLARC's project consultant.
Statute does not allow WSF to consider non-price factors when awarding contracts
State law requires WSF to award a vessel contract using a low bid approach that awards the contract to the shipbuilder offering a technically acceptable proposal at the lowest price. A common alternative is a best value approach, in which a bid is selected based on the relative importance of price and other factors. This approach is appropriate for projects like a ferry acquisition where project requirements are less clear, more design work is needed, or risks are high. It can encourage shipbuilders to offer designs that lower life cycle costs or reduce project risks.
Statute does not allow WSF to use faster procurement approaches that still encourage in-state construction
State law also requires WSF to conduct an in-state procurement first, starting with a Request for Proposal (RFP) that is limited to Washington shipbuilders. WSF may only seek out-of-state bids if the fixed prices from all Washington shipbuilders are more than 5% above WSF's engineer's estimates. This approach compares the estimated cost to the shipbuilder's price, which also considers labor, profits, and other proprietary factors. The second RFP process, if necessary, will add time to the procurement and may increase cost.
JLARC staff and its consultant found that at least 12 states offer in-state preferences by applying the preference during the selection process. These approaches allow the state to run a single procurement process and still provide incentives for in-state shipbuilders.
Exhibit: State law requires planning and a three-phase design-build contracting approach
PLANNING | PHASE 1 (RFP) | PHASE 2 (Technical proposals) | PHASE 3 (Bids) |
WSF completes predesign studies. Legislature authorizes acquisition. WSF develops Request for Proposal (RFP).
|
WSF issues RFP. Shipbuilders respond to qualification questions. WSF selects shipbuilders for Phase 2. |
WSF periodically reviews each shipbuilder's design, drawings, and specifications. Each shipbuilder submits its proposal. WSF accepts or rejects each proposal. |
Each shipbuilder submits a firm, fixed price. WSF awards design-build contract based on lowest price. OR, if all responses are more than 5% above engineer's estimates: WSF rejects all bids and issues new RFP nationally. |
Source: JLARC staff depiction of RCWs 47.60.385, 47.60.386, 47.60.814, 47.60.815, 47.60.816, 47.60.818, and 47.60.820.
WSF can address cost concerns by using predesign studies to analyze life cycle costs
Influence over life cycle cost is greatest in early project phasesSource: Adapted from report prepared by consultant to JLARC staff. |
The Government Accountability Office (GAO) estimates that 70% of a vessel's life cycle costs occur after construction. Some of the best opportunities to control costs occur during the predesign phase, before an RFP is issued. Predesign studies explore different ways to address a vessel's requirements. While WSF's predesign studies addressed life cycle costs related to fuel efficiency and preservation, they did not evaluate long-term staffing costs as required by statute. Also, WSF's approach to selecting predesign study topics may unintentionally exclude topics that have a significant effect on life cycle cost.
Although predesign studies can help ensure that requirements are clear and thorough, changes after the contract is signed are typical in any shipbuilding project. Best practice is to ensure that changes are monitored and managed from the time they are raised, through the approval process, and to their resolution. WSF's change management system appears to be aligned with best practices.
WSF should tie payments to measurable project progress
WSF has limited cost and schedule reporting requirements in its contracts and pays shipbuilders through monthly payments. During construction, the payments reflect an estimate of progress since the last billing. While this is administratively simple, it is inconsistent with best practice. Best practice is to link payments to significant project milestones (e.g., final design review, steel cutting, or trials) or measurable project progress. A progress-based approach that is commonly used is Earned Value Management, which considers cost, work completed, and schedule.
WSF does not have documented policies reflecting current laws and best practices for acquisition, cost estimation, or risk management
Although there is room for improvement, WSF has used valid approaches to defining project requirements, managing risk and change, and controlling cost and schedule. WSF's contracting approach for previous vessel acquisitions complied with the statutory requirements in place at the time.
However, WSF has little formal documentation of its policies or practices. Its primary vessel policy document does not reflect statutory changes made in 2015 or best practices (e.g., cost estimation, predesign studies, or lessons learned reviews). Absent documentation, WSF relies heavily on staff expertise and consulting firms.
Legislative Auditor's recommendations
Recommendations to the WSDOT Ferries Division (WSF)
- WSF should ensure that cost estimates provided to OFM, legislative staff, and the Legislature are clear and well documented.
- WSF should apply probabilistic approaches to its cost estimates and clearly indicate the likelihood of the cost falling at or below the funding level requested.
- WSF should ensure predesign studies evaluate long-term costs as directed by statute and that the suite of studies for each acquisition project addresses the most significant potential life cycle cost drivers.
- WSF should tie shipbuilder payments to significant milestones or measurable project progress.
- WSF should update its ferry acquisition policies and procedures to reflect current laws and best practices.
WSDOT concurs with these recommendations.
Recommendations to the Legislature
- The Legislature should consider amending state law to allow WSF to use a best value selection process for ferry acquisition.
- The Legislature should consider amending state law to allow WSF to offer a single nationwide RFP that increases competition, offers in-state preference, and protects against the risk of a longer procurement process.
You can find additional information in Recommendations.
Committee Action to Distribute Report
On May 10, 2023 this report was approved for distribution by the Joint Legislative Audit and Review Committee. Action to distribute this report does not imply the Committee agrees or disagrees with Legislative Auditor recommendations.