Test and
evaluation activities can be categorized into two main types.
First are those conducted with prototypes or simulations that
"mockup" a task environment with varying degrees
of correspondence (fidelity) to the real world. Second are
evaluations using prototypes or final products in an actual
real-world setting. In the later case, prototypes would be
indistinguishable from final products - at least from a users
point-of-view.
OACIS
strives to tailor each test and evaluation effort to the
objective and product at hand. To accomplish this, a variety
of methodologies are employed.
Paper
Prototypes. When a product can be graphically depicted
on paper, low-cost, paper-based evaluations can be used to
support quick turnaround design iterations. This type of evaluation
is especially useful for initial usability and "understandability"
evaluations of display- and software-based products.
Simulations.
Simulations can vary from desktop presentations to "man-in-the-loop"
automotive and aircraft environments. The fidelity of the
simulation depends on the question being investigated, the
risk being managed, and the resources that are available.
OACIS has experience at all levels of fidelity and can design
evaluations for a variety of situations from focus groups
to more immersive types of environments.
Real-World.
Real-world or field-based evaluations are useful in two types
of situations. First, when the evaluation can not be effectively
conducted without a final product or actual end-user task
environment. Second, when there are compelling reasons to
rigorously test the product prior to final production. OACIS
has real-world experience ranging from on-the-road automotive
evaluations to hand held computers in industrial settings.
