Situation Awareness Flight Training Evaluator

The Situation Awareness Flight Training Evaluator (SAFTE) is a PC-based, general purpose combat aircraft flight simulator operating on a Pentium microcomputer host. The system consists of two linked CPUs. SAFTE was designed to teach, evaluate, and select for the basic piloting and situation awareness (SA) skills. The existing scenarios are applicable to the performance of reconnaissance and surveillance, air-to-air, and air-to-ground missions under multiple threat environments, and range in time from 3 minute segmented flight training, to 30 minute full testing scenarios.

SAFTE's distinguishing features include a realistic aeromodel that represents the dynamic characteristics of the F-16 fighter aircraft, a 3D HUD-centered flight display designed from F-16 cockpit display specifications, an out-the-window (OTW) display, and piloting performance measures that range from the traditional to the next generation of research techniques.

The millisecond time-stamped measures and are recorded 5 times per second and include: indices of spatial orientation, flight control parameters, aircraft state vector, pilot performance, and schema representation.

The HUD-centered flight display offers an ideal platform for examining the effects of instrument configuration on performance.

Pending implementations of the SAFTE system include two separate components of the Air Force. The USAF Research Laboratory Flight Motion Effects Branch (AFRL/HEPM) has selected SAFTE for human performance measurement under high-G stressor conditions in the centrifuge at Brooks AFB, Texas, and the USAF Research Laboratory Aeromedical Consult Service (USAFSAM/AFC) has selected SAFTE as an integral part of their program that seeks to develop a standardized performance test battery for the certification of aeromedical drugs.

Situation Awareness Measures

SAFTE is a standard, multi-dimensional, theory-based, dynamic SA measurement tool whose metrics can then be used to evaluate such SA as workload, skill level, stress, training, etc., and be validated against the ultimate criterion of mission performance. The SA assessment consists of a set of flight scenarios, during which multiple SA probes are employed. The combination of probe performances on which the subject performed adequately defines his/her level of situation awareness for that type of environment and situation.

There are currently 6 performance measures of SA embedded within the SAFTE system. These measures are summarized below, and are arranged along an information processing "continuum", from the more basic sensory processes to the more complex cognitive processes.

Spatial Orientation / Disorientation

SAFTE examines Spatial Disorientation in two distinct ways. Aircraft performance is measured as a deviation of the flight parameters from the prebriefed performance characteristics in aircraft attitude, engine power, airspeed, altitude, vertical velocity, heading, turn rate, slip/skid rate, angle of attack, acceleration, or flight path (velocity vector), as well as complex orientation indicators, such as discrepancies in initiated maneuvers versus the ideal point of maneuver. Secondly, the aircraft will be presented in a non-stable orientation and the latency and accuracy of the pilot's first stick movement to correct the aircraft's attitude is recorded.

Environmental Status awareness

This probe attempts to explore a person's ability to retrieve sensory input information from short-term memory. Endsley's (1994) blanking probe technique is used to achieve this goal. Specifically, the computer screen would, at defined mission times/events, go blank. A short question requiring a yes/no response appears on the screen, asking for specific information about whether certain conditions were or were not present at the time of the blanking during the mission.

Spatial Awareness

This Spatial Awareness measure also uses the blanking technique proposed by Endsley (1994). The questions asked of the operator during the blanked period do not simply refer to the presence or absence of an environmental condition, but also request distance information about an environmental condition. The operator is queried not only about whether an object exists, but is asked to give an indication of how far away it was when the screen went blank.

Awareness of Information

The probes in this measure are designed to determine whether the person is capable of reporting information which is "derivative" in nature. Probe questions reference possible projections of specific events into the future.

Information Requirements / Processing

This approach will determine the amount of information required to maintain adequate performance. The individual is given control over whether or not the computer screen is displayed. The person is instructed to complete a certain portion of the mission with minimal use of the OTW and instrument panel display while maintaining safety and/or mission accomplishment. The computer will record the time and events occurring during those periods in which the individual sought information.

Schema Acquisition / Demonstration

This measure will examine the response to complex situations as a whole and individual skilled performance. Two levels of schema probes are used, with one exploring the identification of pretrained schemas after minimal exposure to an overlearned situation, and the second exploring even more complex situations in which several "simple" schemas must be integrated into one smooth-flowing response sequence.

Hardware Requirements and Performance Measures

Data in the SAFTE system permits maximum flexibility for investigators, and produces a set of summary statistics which simplifies the majority of subsequent data processing efforts. An integral part of the SAFTE system is the ability to log mission data continuously. With all raw data being stored, researchers maintain the ability to calculate unique or specialized statistics.

The SAFTE system physically consists of two Pentium computers. The two units are connected via their parallel ports. Due to the computationally intense natures, the graphics functions are run on one computer system, and the aircraft and scenario simulation on the other. For effective real time system performance, the graphics system runs on a 120 MHz Pentium (or better) with 16 MB of RAM, a 500 MB HD, and VESA 640x480x256 color graphics support. The simulation system runs on a 133 MHz with 8 MB of RAM and at least 850 Mbytes of hard disk space to allow for adequate storage for data logging during scenario execution. The peripheral equipment required by the system consists of a throttle, rudder, flight stick, and game board (ThrustMaster).

SAFTE provides continuous, time-stamped disk logging of control input data at 5 Hz, aircraft state vector information at 1 Hz, and keyboard and other pertinent data as they occur. To achieve sub millisecond time stamping accuracy in real time, a custom real time kernel (RTK) operating system provides both precise event timing and access to system resources.

The aircraft state vector consists of 29 different quantities including the position, attitude, velocity and angle of attack (AOA) of the aircraft. The control inputs include roll, pitch, rudder and throttle values. Other inputs include keyboard and hands on control entries.

Along with pilot inputs various events are also logged, including SA queries, simulation control events and radar activity.