In January 2018, ITEA and DEPS combined two long running events and focused the Joint theme on how test and
evaluation could or should be involved in the DoD trend to focus significant energy on prototyping and
experimentation. This focus was encouraged in an August 2017 DoD "Report to Congress Restructuring the
Department of Defense Acquisition, Technology and Logistics Organization and Chief Management Officer
Organization" that stated the:
"Department (of Defense) must increasingly leverage prototyping, experimentation and other
developmental activities to retire technical risk before either weighing down the research and engineering phase
with costly procurement decisions or weighing down a procurement program with costly technical risk."
In May 2019, the DoD Prototyping Guidebook was updated by the Under Secretary of Defense for Research and
Engineering Emerging Capability and Prototyping Office. The Guidebook shares the following:
"In its report, "Weapon Systems: Prototyping Has Benefited Acquisition Programs, but
More Can Be Done to Support Innovation Initiatives" (GAO-17-309), the Government Accountability Office (GAO)
states that DoD has become increasingly risk averse" and further asserts that risk aversion stifles innovation.
One way of mitigating this risk-averse culture is institutionalizing a new definition of what constitutes
prototyping "success" and "failure." Quite simply, since at its core prototyping is meant to generate a data set
to inform a future decision, a prototyping project "succeeds" if it provides that data set—even if the prototype
itself does not work.
Likewise, a prototyping project that does not generate a data set to inform a future decision "fails."
Perspectives of "success" and "failure" in prototyping should have less to do with the prototype itself and more
to do with the data that the prototyping project generates.
In most cases, the most important step in the prototyping process is evaluating the prototype. In fact,
prototypes are often built specifically for the evaluation activity, and will be discarded after the evaluation.
Evaluations should be designed and conducted in a way that addresses the purpose of the prototyping project.
Evaluations typically come in three forms: demonstrations, experimentation, and red teaming.
Our Joint Conference leverages that DoD vision and the recent significant increases in prototyping and
experimentation efforts to explore the role of T&E in "prototyping, experimentation and other developmental
activities." This Conference seeks to discuss the practical implications of T&E support to getting "game changing"
technologies to the Warfighter.
- How are we incorporating new ideas about prototyping and experimentation "success" and "failure" into
test planning, execution, and analysis?
- How is testing in support of experimentation different than traditional T&E? How is it the same?
- What are the roles of DoD T&E organizations and the Major Range and Test Facility Bases (MRTFBs) in
designing, planning, executing, and supporting these activities?
- Given that each Service has already embarked on significant prototype development and experimentation
campaigns, with execution timelines that span years not decades, how do the Service and DoD T&E communities
respond to rapidly emerging needs for instrumentation, infrastructure, and test methodologies to support these
campaigns?
- How is testing in support of experimentation affected by the use of conceptual, developmental, and
operational prototypes?
Key plenary speakers, panel discussions, and technical sessions are all part of the program designed to identify
challenges, solutions, innovations and a future state; and to move us closer to creation of an
infrastructure and principles conducive to testing and training in a more operationally realistic environment.
Last updated: 27 December 2023
