Fat For Process Control

Commentary on FAT by technology providers
Authored by: Bryan Greenway, Treis Technology
Information provided to The LabMan for the September 2009 blog.

• What is FAT testing and how is it unique in the process of developing an automated system.

What a FAT should be: a formal review/demonstration of how each specification requirement has been addressed. A handy tool for doing this is aVerification Cross Reference Matrix (VCRM). The VCRM is a simple table containing each requirement along with its associated description and method of verification. A sample table heading is given below:
Requirement # Type of Requirement Specification Paragraph # Name Requirement Text Verification Method (Test, demonstration, analysis, design, inspection) Verification Comments

The VCRM is thenused as a tool early in the Spec negotiation process with the vendor, clearly outlines how “success” is defined, and how each requirement will be measured. It also provides the core for the FAT agenda; by simply stepping through each row (requirement) in the table, an organized approach is facilitated. A healthy safeguard for the buyer is to require that the VCRM be used as a reporting tool duringthe project…what requirements are currently met? The customer should require documentation that a complete, mock FAT has been successfully run at least twice before even scheduling that real FAT. The vendor should make sure that the customer is plugged into the development process all the way through. Engage them in regular reviews, and try not to hide anything from them. If this is done properly,the customer becomes well informed, mitigating the chance for late surprises, as well as becoming a calming source of information inside his company. Try to get customer representation to see mock FATs. Also, make sure the vendor understands the expectations and pressures on the customer side.

For vendors and customers: Expect surprises that neither anticipated. Work together to find asolution. Remember, both sides have skin in the game. Also remember that for most of automated systems, it is a uniquely function device…i.e. this is the first time it has been done. Not everything can be anticipated, so there will be discoveries along the way.

What a FAT typically turns in to:
Most FATs that I’ve been involved with uncover surprises. These “surprises” happen for a variety ofreasons:
• A customer turns up with a set of unexpected conditions under which he/she wants to “see what happens”. Many times they’ve done this intentionally to “see how thoroughly the system has been tested”. Make all attempts to discuss (early in the process) the reasonable conditions under which the system will be expected to operate.
• A last minute fire drill where there are first-timeattempts at required functionality during the FAT (this may seem unbelievable, but it is VERY common)
• An abbreviated FAT (abbreviated because too many requirements are failed right away) that turns into a drawn out meeting around “what do we do now?” discussion….do we adjust the requirements we’re having trouble with, give us more time to try to finish it (sometimes asking for more money), ship itanyway and then try to finish it up on site (BAD IDEA!!), or cancel the project all together.
• One side or the other taking a hard, many times emotion-driven, stance, making it very difficult for a collaborative process to move forward. Common sources of disagreement include handling of unexpected or error conditions, error recovery, and means for measuring system performance against aspecification. Be sure to spend time discussing these areas well before the FAT.


• Who should attend a FAT?
For the vendor: have a very small group to lead the FAT, minimally including the sales lead, the project management lead, and the technical lead…but remember to have EVERYONE that was involved on call. You don’t want too many voice around the system during the FAT; it creates a sense of...