Test Engineering Fundamentals

Wikipedia has a good definition of a Test Engineer.

A test engineer is a professional who determines how to create a process that would best test a particular product in manufacturing and related disciplines, in order to assure that the product meets applicable specifications. Test engineers are also responsible for determining the best way a test can be performed in order to achieve adequate test coverage.

This definition omits any mention of SQA (software qual). SQA isn’t Test Engineering, unless the software is controlling a machine or other physical system. In my view Engineering is an activity where the ultimate goal is to create physical objects or control physical phenomena — i.e. interact with the physical universe. Software Development that doesn’t interact with physical objects or phenomena isn’t Engineering, any more than Accounting is Engineering.

Test engineers are also often designated as system designers or system integrators. Test Engineers are multi-disciplinary generalists, who posses a breadth of knowledge in industrial machinery, electronics, software programming, and statistical analysis.

To assure that a product meets the applicable specifications the product has to be tested. And testing requires performing measurements and collecting data. The measurements could be static and dynamic properties of the unit under test (UUT) and it’s response to various stimuli.

How is data collected?

But first what is data?

things known or assumed as facts, making the basis of reasoning or calculation.

In engineering data is collected through sensors and instruments. Examples: Thermometer, Clock, Compass, Pressure gauge, Digital Multimeter etc.

There are two types of sensors and instruments: analog and digital. The words analog and digital are understood differently in colloquial and engineering contexts.

In a colloquial context analog refers to a few different types of devices. It could refer to mechanical instruments that have no electrical output like a mercury thermometer, or mechanical scales, and pressure gauges. More broadly any device using mechanical energy is considered ‘analog’. However electrical instruments such as moving-coil voltmeters and ammeters are also analog because they do not contain digital electronics and output continuous signals.

So, colloquially analog usually means non-electronic devices, or electronic instruments without a numeric LCD display. Analog instruments provide mechanical readouts — rotary gauges, floats, liquid levels etc. Digital is the opposite of analog, and is something that uses electricity and has a numeric LCD display.

In engineering terms an analog signal is a continuous signal. A digital signal is a discrete signal. A digital signal or binary signal is a subset of discrete signals where the signal level oscillates between only two values. The concept is best illustrated using a graphic.

The horizontal axis is usually time and the vertical axis is the quantity being measured. The universe is analog, i.e. Light, temperature, pressure and other phenomena are continuous quantities, it is human perception that is limited.

For example a burning candle is emitting light continuously at every instant in time — the candle is emitting light (photons) continuously at every instant. It can also be said that ‘time’ does not exist. Time is a property of human perception. Time can be thought of a counter of intermediate states between two quasi-steady states. When Galileo dropped a cannon ball from the top of the leaning tower of Pisa, the ball went from one steady state (Galileo’s hand at the top of the tower) to another (resting on the earth). As the ball was moving between these states it passed through a number of intermediate states, where the ball was at different heights above the ground. These states can be counted and the value of the counter is time. Without change, time is imperceptible. Time was counted using the change in the seasons, the motion of the stars, the length of shadows, and today through the repeatable motion of mechanical devices, vibration of crystals or decay of atoms. All of these things are counters of change.

Digital electronics (microcontrollers, computers) do not work with continuous signals. They sample the continuous signals at a certain frequency and work with the set of collected discrete data points. This is analogous to a a video camera recording images at 24 or 60 frames per second to produce a motion picture.

In a modern Automated Test Systems analog instruments would be used for providing low-cost visual indicators for diagnostic purposes, or to filter and condition the analog signals for down-stream digital sampling. Digital instruments are used for high accuracy measurements.

Is the data good?

Collecting data is simply the first step. The next step is to determine whether the data is good. While there are various dimensions of data quality the most important ones in test engineering are — accuracy and precision, repeatability, reproducibility, and validity. These data quality metrics are used to answer the following questions.

• Validity: Is the right quantity being measured?
• Accuracy : What is the uncertainty (error) of the measurement?
• Precision/Repeatability: Do repeated measurements yield the same value?
• Reproducibility: Can the study be repeated by different people and in different places?

Accuracy, and precision are easily understood, and are generally functions of the sensor / instrument, and wiring techniques. In a test system where voltage is being measured the accuracy of the measurement is purely a function of the test instrument (Digital Multimeter), if the instrument is connected directly to the UUT. However if the connection is made through long cables and the measuring loop contains terminal blocks and relays the effective accuracy of the test system is degraded.

Similarly in applications involving process control (pressure, temperature, flow etc.) system accuracy will be lower than the accuracy of the sensors. That is to say that installing a really accurate temperature sensor on an oven doesn’t hugely improve the accuracy of the oven, but it may help the oven stabilize quicker.

Reproducibility is a systems level phenomenon. A reproducible measurement system should produce the same result regardless of external variables such as time of day, or person operating the system.

Validity is not quite straightforward to understand and requires a systems level analysis. For example, if a test requires that “ambient temperature” be measured then the placement of the sensor in the test system will determine if the measurements are valid. If the sensor is mounted on a metal plate adjacent to an exhaust fan then the measurements would be invalid since the area being measured contains energy flows. Ambient temperature is the temperature of a static mass of air in a given environment.

After high quality, accurate data is collected, it can be analyzed to determine if the UUT meets the required specifications. Future posts will cover test system design principles, test methods, sensor technology, and best practices and techniques for obtaining high quality measurements.