Reverse Engineering - Acquisition Step
By Bill Macy, Boeing Phantom Works and John M. Switlik, Boeing Define

This article continues the look at Reverse Engineering (RE) as a necessary component of modern modeling. RE is necessary for CAD/CAM/CAE (CoT) and Knowledge Based Engineering (KBE) as described by the following.

• For CoT, RE can be central to answering questions about the as-built reality in relation to the as-designed specification. This quality focus will continue to be important.
• For KBE, RE can provide a basis for knowledge completion via empirical feedback. The general mechanism for verifying computational states will either be a superset of RE or RE will morph to cover a broader context. 

RE has the particular problem that it must deal with data that originates from physical entities and that it must create useful models from this data. Like all forms of modeling, RE face trades related to representation (structure, parameter), methods (in-house/COTS, procedures, algorithms/heuristics, physics), and quality (measurement, fit, continuity, footprint, completeness).

The last article defined the two major steps for geometry-oriented RE, as acquisition and modeling. Acquisition, the subject of this article, involves obtaining data from physical objects. There is a potential for RE to be an enabler of more powerful KBE which will be the subject of a future article.

Background

RE can be an expensive proposition due to the resource mix. RE acquisition approaches need to use specialized hardware to capture the data, customized software to process the data, and a highly trained operator to make it happen. Since RE requirements do vary and any approach cannot fill all requirements, there may be the need for investment to support several approaches. To help meet this need, vendor offerings in the market integrate different approaches into a more powerful combination. Such integration schemes will be very important to RE, and good requirement definition for these will be paramount.

RE acquisition approaches vary over a wide range that is expanding due to progress in research and development. In general, data is either passively or actively sampled from a physical object or item (using singular for ease of discussion) within the scope of the field of view. The data could be recorded manually, but we are looking at computer-based approaches. There are data processes that filter noise and limit via clipping. The end result is data that is within the tolerance specified by the vendor as being attainable by the approach.

Representation

Depending upon the particular item from which data is to be derived, there are many representation options that exist and parameters that need to be controlled. For instance, it does make a difference whether the item is fixed or freestanding. Issues of perspective and point of view must be resolved. When there are multiple views to be joined, a mechanism must exist to relate the views reliably.

Early RE approaches were gross approximations of a location along one axis. Some of the early improvements allowed acquisition along more than one axis and introduced the concept of field of view. The modern techniques allow a wider field of view by using reflectance from light, laser, or sonic emissions.

For all approaches, the controlling software has improved greatly. This allows the acquisition step to provide early modeling support with data filtering and smoothing and meshing of points to support visibility.

The following list shows the range of the known approaches.

Theodolite – Uses methods similar to surveying and provides a point and the associated angles. Some laser methods have helped improve precision for this type of approach. CMM – Allows a point probe to be related to a NC position and provides a point and a vector. The modern variant uses an articulating arm with sophisticated tracking software thereby allowing more flexibility to measure parts outside of NC machines. Photogrammetry – Provides a means to coordinate between multiple photos to get precise measurements of points. Some approaches use this technique to help control scanner quality. The process has been extrapolated to video and can be applied to any capture media. Scanner – Applies a known pattern during emission and interprets the reflective results. Uses white light or laser to project the pattern. Other – There are many types, such as X-ray, Digital X-ray, CAT, MRI, and sonic that offer a wide variety of data acquisition methods and benefits.

Each approach has its own data handling requirements. No item can be captured from one framework and that brings up issues of how different (and potentially disparate) frames of reference will be mapped together. The necessary means for relating the independent views require more attention in the freestanding mode than in the fixed environment. One challenge for RE will be handling large collections of items that are freestanding. Another issue will be capturing data from a moving item.

In the acquisition step, items within a defined view will differ by type. There may be highly complicated relationships in the view necessitating multiple views. Given the many views, an item may be partially represented in several thereby making the identification of an item and its boundaries very difficult.

Representation, in the acquisition step, also deals with how the data is stored. This storage can be in a large range of modes, which ought to support the requirements for the modeling step. One factor might be the amount of data. An open issue is the trade between heavy density of sampled data and less density with the addition of properties, such as normals. Another issue is merging subsets of views in order to extract features that span the views. Because of strict computational limits, all views may not be accessible at once. One area of research is how to separate out sufficient information to be able to identify a feature.

For instance, a limiting box might be applied to each view, assuming that the views can be related correctly. Then the accumulation of all these subsets would be the basis for processing. However even with this partitioning, the density might be too heavy. One required approach might be to approximate the feature identification with an iterative decimation of points.

Representation for acquisition is evolving and will allow smarter processing based on the requirements of particular domains. 

Methods

Methods for acquisition approaches are as varied as we saw with representation. There may be commonality in method between the different RE acquisition approaches; however there are substantial differences that need to be considered. As discussed in a former article, methods relate both to process and its enablers. For either, the process or enabler can be based on operator knowledge or computer expertise. An important step would be to ensure that the acquisition resource has met certification requirements.

Process steps by the operator include setup and capture. Setup involves several tasks, such as calibrating by tuning and focusing. Capture tasks would include keeping track of the acquire data and preparation of the data for the next steps. Computational tasks are directly dependent upon the hardware and the expected data format.

Automation can reduce tedious work by humans and can increase the powers of a process. A case in point is the introduction of the mesh at the acquisition step. This use of modeling techniques early in the process can help improve precision.

The more modern approaches will be digital in nature with fairly sophisticated software handling the data stream and extracting information about the item from the raw data.

• For each acquisition approach, a detailed look at its data handling is of interest. One could rate the approaches by the complexity of the raw data and the amount of steps required to extract the information for an item. This could be considered a power rating in which the Photogrammetry approach might be considered more powerful than the Theodolite approach but is weaker than some other approaches. However, in the case that only a few points are required, the Theodolite approach would be better. Also, there would be less need for post-processing.
• As part of the acquisition step, a plan will define the points of view and their extent. Important factors would be the type of data that will be provided (CMM, laser tracker, white light) and ambient light. Characteristics of the items in the view and the desired product will determine the type of approach.

· Acquisition error handling includes the mathematical steps that resolve points and other properties from the item from the raw data. For several reasons in RE, acquisition is the first step that is followed by modeling. The demarcation line between acquisition and modeling ought to be adjustable by context. Some extraction of features in acquisition would result in a diminishing of the cardinality of the sampled points. This type of partitioning may be potentially smarter than those available later in modeling, since re-sampling in acquisition could help resolve ambiguity.   

Acquisition modeling ought to have access to the definition of the item where it exists in digital form. In Quality (next), there is a brief look at the need to link acquisition and modeling in order to increase the power of an RE approach. This will be the further discussed in the next article.

Quality

In the long list of acquisition approaches, information about performance of the related resources is based upon theoretical limits set by the vendor and by real experience through use. Performance criteria include the tolerances that can be expected, the rate of acquisition, the calibration ease, and others.

Activity prior to RE acquisition would include certification in terms of verifying that the vendor specifications are met in test situations. An issue is the absence of de facto standards on how to certify the acquisition resources.

Calibration will be required at the time of use. This can be problematic since the acquisition process can be subject to many possible sources for noise and other interference. A concern may be the time frame that calibration can be expected to hold. This step requires actual test objects with known properties.

Of interest to RE is the fidelity required by the application that will use the data. Fidelity concerns start with the early handling of the data through the digital signal processing and must consider how well precision expectations are met. Then, given that precision requirements are met, fidelity deals with sufficiency of the data to allow identification of the item and its features. There are other fidelity issues, such as fit and footprint trades. An open issue is how to set the fidelity requirement by context.

That RE has demonstrated success shows that these issues can be resolved for certain types of situations.

In terms of supporting as-built analysis, RE has allowed close mappings between designed and built items, in terms of fit and form measurement. In another demonstration of precision, RE has been used to custom-fit clothes and shoes provided by some manufacturers.

RE will need to support function analysis as well. This requirement is being brought forth by the growing use of KBE. Since, algorithms and heuristics are sensitive to the representation, fidelity is an important aspect. That is, a method will be directly dependent upon the data configurations available to it. An example can be found in the many types of meshes that are possible and in the preference of one over the other given the problem. For instance, one mesh type may be better for flow analysis versus statics analysis, such as stress. RE will need to address these types of concerns when supporting KBE verification.

Conclusion

This article looks at the first step of RE which is acquisition. There are many acquisition approaches and more types of acquisition resource applications. In practice, more than one may be required for a particular RE problem. In all cases, the issues of certification and calibration are on going. As acquisition techniques improve in power, issues related to representation come to fore. This step is vital for downstream determination of items and features in a complex field of view. The next article will discuss these downstream processes that are related to modeling and its trades and how acquisition and modeling ought to work.

For further discussion of the content of this article, please e-mail the authors.

About the authors

John M. Switlik is an Associate Technical Fellow (Boeing, Commercial Airplane, Wichita Define Systems, KBS, john.m.switlik@boeing.com). His interests cover a broad spectrum: computational support for applied science, mathematics, and management; lifecycle and veracity control for system, application, and domain engineering; integration, use, and quality of computational knowledge.

Bill Macy is a Senior Project Engineer (Boeing, Phantom Works, St. Louis, william.d.macy@boeing .com). His interests focus on the need to leverage reverse engineering metrology and software to provide affordable and better quality engineering, fabrication, and support of products.

Definitions:

CoT - CAD/CAM/CAE/etc. plus “Top Gun” talents (see Discussion).

KBE – Knowledge Based Engineering

RE - Reverse Engineering covers all activities related to capturing geometry data about a physical part and to creating a computer model that can be used by CoT. Plus, RE serves as a prototype for a fuller link between computational models and associated physical states.