A critical challenge facing medical OEMs and suppliers is validating the "analytical" software that determines compliance to Rule 21 CFR Part 11 regarding mechanical drawings for components and assemblies. In general, Part 11 requires medical device manufacturers, biotech companies, biologics developers, and other FDA-regulated industries to implement controls such as audit trails, electronic signatures, and documentation for software and systems that process electronic data according to FDA predicate rules. These rules dictate such things as what records must be maintained and for how long, the content of records, and whether signatures are required.

Medical components can have complex surface geometries and tight feature tolerances, so OEMs mandate the use of high-precision measurement devices and software that ensures traceability to ASME Y14.5M – 1994 for Geometric Dimensioning & Tolerancing (GD&T). The validation of this software is confusing at best. First, a discussion of GD&T should help further understanding.

GD&T is a symbolic design language that defines the nominal (theoretically perfect) geometry of parts and assemblies, allowable variation in form and size of individual features, and allowable variation between features. Its use moves the CAD model from a design to a manufacturing slant and lets software validate steps in manufacturing and inspection, thereby reducing human error.

In the past, major medical OEMs such as Medtronic, Minneapolis, analyzed a broad array of analytical software for determining compliance to Y14.5/Y14.5.1. This resulted in a push to standardize worldwide on analytical software called SmartProfile by Kotem Technologies Inc, in Canada, which successfully passed complex mathematical test cases.

Analytical software limitations

Most companies with advanced metrology needs have purchased a broad range of coordinate measuring machines (CMMs) over the years. Each machine comes with its own analytical software package that requires installation, upgrades, and licenses. Unfortunately, the different packages can evaluate the same data and derive completely different results. The real danger is that all of these results can be repeatable and reproducible — and incorrect. The majority of software does not have the ability to ensure full compliance to ASME Y14.5.1M-1994.

For a worst-case scenario, consider "Analysis of a cylindrical feature," shown above.

It shows the measurement of a cylindrical feature for its mating size and location of the axis of the mating envelope. The majority of software defaults to a least-squares fitting to determine the features size and location of its axis in the X and Y directions. Problem is, using least-squares analysis, commonly referred to as best-fit analysis, generates incorrect results.

In this case, correct results come from analyzing the largest inscribed cylinder or the smallest circumscribed cylinder, depending on whether the feature is internal or external. Most software can apply maximum inscribed and smallest circumscribed algorithms but, in most cases, OEM and supplier metrologists do not use the algorithms.

Page 2 of 2

Unfortunately, in these cases, measurement data can look good — it complies with specs, but parts do not work or fit properly. Sometimes, measurement data can look bad, but parts actually do work. The implications to statistical data analysis on medical components and assemblies are disturbing and place product reliability at risk.

Lowell Inc, Minneapolis, a precision contract manufacturer of orthopedic implants, evaluated their software and determined the same limitations as had Medtronic, Pioneer Surgical Technologies, and other OEMs. Jim Stertz, quality manager at Lowell, says, "SmartProfile and profile tolerancing has changed the way we work. We have reduced inspection time and measurement error while improving accuracy." One of Lowell’s current challenges is informing customers that analytical results are different in some cases because of the algorithms in the various CMM software packages.

Some OEMs consider their software validated if it comes with a certificate of compliance from a national laboratory such as the National Institute of Standards and Technology (NIST) in the US or Physikalisch Technische Bundesanstalt (PTB) in Germany. Many medical device manufacturers are unaware that the critical element of these certificates is that algorithm testing at NIST and PTB are restricted to the basic form elements: straight line, plane, circle, cylinder, cone, and sphere. In addition, reference results are calculated using the "Gaussian method of least squares,” which usually comprises best-fitting algorithms that do not comply with ASME Y14.5.1. These limitations are forcing medical OEMs such as Medtronic to develop their own mathematical data sets to prove compliance to the more challenging requirements of ASME Y14.5.1-1994.

Another problem is the way companies define gage repeatability and reliability (GR&R). Current criteria do not include truncating, or guard banding, the tolerance based on the magnitude of measurement uncertainty as defined in ASME B89.7.3.1-2001 — Considering Measurement Uncertainty in Determining Conformance to Specifications.

A better use of CMM data points

Historically, metrologists have found the measurement of complex surface profiles too challenging due to CMM software limitations. Today, profile tolerancing is considered one of the simplest ways to analyze complex surface geometries as long as users have the correct software, SmartProfile.

SmartProfile has been mathematically validated for its capability to generate data sets for proof of compliance to ASME Y14.5.1M-1994. It quickly communicates compliance or noncompliance to tolerances as well as graphically represents absolute deviations showing the total range of results. This level of information lets manufacturing engineers immediately see root-cause effects resulting from the manufacturing process and provides indications on how to optimize the process for better results.

The software also solves or significantly reduces the software validation effort on every metrology software package. Many companies are not capable of analyzing results to the ASME Y14.5.1 Math Standard. The software allows the use of one package with different CMMs. Companies can use CMMs to simply collect points or a point cloud and then import it into SmartProfile for final analysis. Engineers can simply request the measured point array from the metrologist and analyze the results in minutes rather than rely on confusing inspection reports.


An example of the use of G&T is shown in "Callout profiles" from Pioneer Surgical Technology, (pioneersurgical.com), Marquette, MI. It shows the profile tolerancing of 3D surfaces where callouts contain explicit surface profiles. This kind of information is critical in determining compliance to Part 11. vIn addition, a more complex example comes from "Datum reference frame," which shows a model in which all external 3D surfaces are defined with surface callouts that are in relationship to a datum reference frame. The part is intended to work in a complex assembly and GD&T helps ensure it complies with all requirements.