START Lite:
A decision-making aid for SBIR ranking

What enhancements to NASA's system for awarding funding to SBIR applicants might increase the likelihood that the resulting technologies would be employed by NASA missions?

Small Business Innovation Research (SBIR) is a congressionally mandated program in which small businesses compete for funding by proposing research-and-development projects for certain federal agencies, including NASA.

In an environment of restricted budgets and the need for ever more technology development, it becomes increasingly desirable that the technologies NASA selects for development funding are those with the highest probability of infusion -- that is, actually being used in missions. The path to technology infusion is subject to many influencing factors, but it seems reasonable to assume that the probability would be improved by employing a selection process that ranks prospective technologies according to a well-defined and agreed-upon set of attributes targeted to upcoming missions.

The purpose of these studies is to increase the likelihood of funding technologies that will ultimately fly on missions.

NASA's Innovative Partnership Program Office (IPPO), which manages NASA's participation in the SBIR program, has therefore expressed a strong interest in augmenting the existing ranking processes -- which rely heavily on panels of experts who vote on which technologies to fund -- with a transparent, quantitative, decision-support capability based on specific desired criteria. IPPO engaged the START team to help in this effort.

The START team conducted two successful studies for IPPO: one with the Human Research Program (HRP) at Johnson Space Center and the other with the Exploration Technology Development Program (ETDP) at Langley Research Center. We also conducted an interesting analysis with the kind cooperation of NASA's Science Mission Directorate (SMD), which illustrates both START's flexibility and the advantages of deriving technology selections from the desired attributes.

We determined that the full START methodology would not have been appropriate at this point because of insufficient quantitative requirements and metrics to apply to the process. Instead, we developed a simplified approach dubbed "START Lite," a semi-quantitative analog to the START process.

As with the full START methodology, START Lite results are intended to provide decision-makers with an opportunity to conduct a "reality check." If the attributes, scores and weightings used in a given run reflect the true priorities of the decision-makers, the generated rankings should seem right to them. In that case, the system provides a quantifiable and traceable process in support of their decisions.

On the other hand, if the START Lite results differ from a decision-maker's expectations, the process provides an opportunity to identify any differences in the perceived objectives, important attributes that may have been omitted, relative weightings that do not reflect true mission priorities, and any constraints, assumptions, or other underlying factors that may be responsible for the conflicting outcomes. Identifying and explaining these items is a crucial step toward the objective of converging on an optimal solution via a transparent and traceable decision-making process.

HRP study

ETDP study

SMD analysis