New Study on QBS
[by: Thomas C. Leslie]
 Qualifications-Based Selection, QBS, what else can be said about it? For those who know QBS, it's hard to imagine. For those who do not, "What the heck is QBS?"
For the federal government, the State of Georgia, and all but two of the other states, QBS is the settled way design professionals are selected for projects. Competing design firms are evaluated based on their qualifications and ranked in priority order. The owner negotiates a contract (scope, schedule, fee, and business terms) with the top-ranked firm. If an agreement cannot be reached, the owner terminates negotiations with that firm and begins negotiations with the second ranked firm, and so forth. The rationale for this process is that 'good' design leads to lower construction cost and/or higher owner satisfaction and project performance (such as lower operating cost). It has been argued in many settings that QBS has the highest probability of leading to high-performance, cost-effective projects. Typically, QBS is discussed in contrast to selection of a design professional solely on the basis of low cost, where cheap engineering typically leads to higher construction cost and lower project performance.
The easiest and most conclusive argument for QBS would flow from an analysis of a single project, designed and constructed twice in a controlled experiment, once by a firm selected using QBS and secondly by a firm selected based on low bid. The record shows no instance of double design/construction, and it would be foolish to do so. Instead, the 'QBSers' and 'LowBidders' argue their case based on assumptions regarding project development and their own predilections. Worse yet, many inexperienced local government officials select low bid design professionals believing that they must do so, or never considering alternative methodologies.
The case for QBS has been richly enhanced by a recent study conducted for the American Public Works Association and the American Council of Engineering Companies by two University based academics*. With the assistance of the study sponsors, the two authors randomly selected for study 89 projects (located in 29 states) from a pool of 195 (in 37 states) that had been designed and constructed within the past five years. Detailed information was sought for the 89 projects via a Web-based questionnaire and phone/email contact with members of the project team. In the end, detailed information was available for 41 projects (in 23 states). The projects were overwhelmingly public sector (86 percent in the pool of 89 projects and 95 percent in the final study pool of 41 projects--44 percent transportation, 39 percent water/sewer, 15 percent commercial, and two percent industrial). In addition, 90 percent of the projects were conventional design-bid-build, five percent were design-build, and the rest were 'other' delivery methods.
Further information on the 41-project data set:
Design Construction
Procurement Method
QBS 78% 12%
Best Value 10% 17%
Low Bid 5% 59%
Sole Source 7% 2%
Other - - 10%
Costs
Minimum $2.5k $25k
Median $441.5k $4.5m
Maximum $9m $900m
The analysis of these projects suggests several evaluation factors that are more qualitative in nature. This includes a discussion of the 'trust' that exists between an owner and engineer when QBS was utilized in the selection process. Related, is the high degree to which the engineer is 'embedded' in the owners project team due largely to performing work on other projects (repeat business). Also, the QBS-selected engineer is better able to address 'sustainability' and 'flexibility' considerations in project design.
Several other study findings are more quantitative and seem more compelling regarding QBS projects.
Project Costs. Typically, design cost represents about one percent of the total life-cycle cost of a project (which includes construction and operational costs). A frequent indicator of a successful project is the increase in cost during construction due to change orders. It is commonly accepted that the industry average in cost growth is about ten percent of final construction cost. The QBS projects in the study had a cost growth of only three percent. Typically, this suggests that the project design was 'complete,' various design disciplines more fully coordinated, and the project objectives were very clearly defined.
Project Schedule. The national average for construction cost schedule growth is also about ten percent. For the QBS projects in the study, the schedule growth averaged 8.7 percent, although 60 percent of the projects had schedule growth of less than three percent.
Project Performance. Project designers self-reported that all of the QBS projects were of 'very high' or 'high' quality. This assessment was tested by asking owners if the project was 'successful,' where 'success' involved an assessment of quality, how well the design team worked with the owner, and whether the owner would consider using the designer on future projects. For QBS projects, 93 percent of owners rated project success 'very high' or 'high' (very high was 70 percent and high was 23 percent). When the designers were asked about project success, their rating was virtually the same as the owner.
The new study does not make a 'scientific' case for using QBS, but it certainly adds much needed, project-based analysis of the impact of designer selection on project outcomes. It offers new support for the contention that utilizing a QBS selection process for a project designer has a very high probability of leading to a successful project as measured by owner assessment and control of cost and schedule growth. ✪
*An Analysis of Issues Pertaining to Qualifications-Based Selection, Draft, Paul S. Chinowsky, University of Colorado, Boulder, Colorado, and Gordon A. Kingsley, Georgia Institute of Technology, Atlanta, Georgia, APWA and ACEC, 2009.
** For additional information on QBS in Georgia, please visit www.QBSGeorgia.org.
|
