HRP study

The START team worked with program managers at the Human Research Program (HRP) at Johnson Space Center to compile a list of five attributes, with a weight for each that reflects its importance relative to the other attributes. The advocates for the proposals (subtopic managers) were asked to give each proposed technology a score of 1, 2, or 3 for each attribute, using the scoring system shown in the table below.

"Time frame" refers to the period in which the proposed technology would be used. Near-term missions include the Orion Crew Exploration Vehicle, the Ares I launch vehicle, and the International Space Station. Mid-term missions include the lunar lander (known as the Lunar Surface Access Module), the Earth-departure stage of the launch vehicle that will send our next group of astronauts to the Moon, and the Ares V heavy-lift launch vehicle. Far-term missions include future outposts on the Moon and beyond.

NASA's Human Research Program conducts research and develops technologies that allow humans to travel safely and productively in the environment of space.

"Criticality of technology to mission" indicates whether the proposed technology would be enabling (essential to a mission because a mission goal could not be achieved without it) or enhancing (not essential, but beneficial).

"Dependence of project upon SBIR" indicates the degree to which HRP relies on SBIR to provide the technology capability under consideration.

"Importance of proposed solution over existing solutions" attempts to capture how much improvement the proposed technology would offer over the current state of the art, and the importance attached to that improvement.

In the peer-review process to which all of the proposals are subjected as part of the traditional decision-making process, a panel of experts discussed the various technology candidates and assigned each one an "evaluation score." We used that score as a "probability of success" indicator representing the degree of certainty that the applicant is capable of delivering the proposed technology. To calculate a final score for each candidate, we multiplied the weighted sum of its attribute scores by its "probability of success" score.

The final scores and the resulting ranking of the technologies were presented to HRP management in the form of an Excel spreadsheet. Sliders on the spreadsheet enabled the managers to see, in real time, the effects of modifying the attribute scores and weights. Using their feedback, we calibrated the scores and weights, and computed a new ranked list of technologies recommended for funding.

HRP management reported that they found the START tool and process very helpful. They modified START's rankings of some technologies because of information that was not captured by the attribute database (e.g., changes in requirements which removed the need for several technologies), but found most of START's recommendations to be close to their own independent rankings. In several cases, they changed their own rankings to match those of START.

One of the key decision-makers informed the START team that "the START tool made my job much easier in that I didn't have to make some subjective selection based on spreading awards evenly across elements (topics)." She went on to say, "When used correctly and validated in person, they really do help us objectively prioritize our first cut at selections."

Back to START Lite main page

ETDP study

SMD analysis