-
- News
- Books
Featured Books
- smt007 Magazine
Latest Issues
Current IssueBox Build
One trend is to add box build and final assembly to your product offering. In this issue, we explore the opportunities and risks of adding system assembly to your service portfolio.
IPC APEX EXPO 2024 Pre-show
This month’s issue devotes its pages to a comprehensive preview of the IPC APEX EXPO 2024 event. Whether your role is technical or business, if you're new-to-the-industry or seasoned veteran, you'll find value throughout this program.
Boost Your Sales
Every part of your business can be evaluated as a process, including your sales funnel. Optimizing your selling process requires a coordinated effort between marketing and sales. In this issue, industry experts in marketing and sales offer their best advice on how to boost your sales efforts.
- Articles
- Columns
Search Console
- Links
- Events
||| MENU - smt007 Magazine
Solid Shape Modeling — Working Toward 3-D AOI
December 31, 1969 |Estimated reading time: 9 minutes
With ever-increasing complexity and component density on circuit boards, automated inspection of printed circuit boards (PCB) during manufacturing becomes a critical step necessary for high quality and yield.
By Xuemei Zhang and Gareth Bradshaw
Many different inspection methods are used in today's SMT manufacturing lines, including electrical test, as well as X-ray and optical inspection. Optical inspection, while not universally applicable, satisfies most pre- and post-reflow inspection needs, and is still the fastest, lowest-cost solution.
Automated optical inspection (AOI) refers to inspecting a target, such as a portion of a PCB, by imaging it under controlled lighting conditions. Sophisticated machine vision algorithms determine if the target conforms to manufacturing criteria. AOI commonly is used to inspect solder paste deposits, part presence/absence, placement, polarity and type/value checking (OCR/OCV), flux, and solder joint quality. Solder joint inspection is particularly important.
This article provides an overview of various currently used AOI techniques and discusses an emerging technology, solid shape modeling (SSM) and the benefits of this approach.
Traditional 3-D AOI Techniques
Different types of information can be extracted from inspection images. Surface color has been successfully used to check part presence/absence. For solder joint inspection, shape information is more useful than color to accurately determine joint quality. Shape information also can be useful when the part's color is similar to that of the substrate. Therefore, the ability to capture and reconstruct 3-D shapes is highly desirable in an AOI system.
Traditional 3-D imaging techniques that can work at the micron resolution level required for PCB inspection include stereo imaging, laser profiling and confocal microscopy. Some of these techniques, such as confocal scanning, are too time-consuming for in-line inspection, and others, such as time-of-flight methods, do not provide the required resolution for PCB inspection.
Laser profiling is a 3-D imaging technique used successfully in AOI systems. A thin laser line is projected onto the target at an angle to the camera. The displacement of this line in the camera image then is used to determine the height of the illuminated points quickly. Scanning this laser line pattern across a target allows construction of a complete depth profile of the target surface.
The same triangulation approach has been extended to other structured light techniques, where a grid pattern or composite frequency sweep pattern is projected onto the object surface, eliminating the need to scan it.
These techniques, while effective for many types of targets, do not work well with the specular surface of a solder joint, as the light is reflected back in a narrow range of angles, and likely to miss the camera aperture completely, resulting in uncertain height solutions.
In stereo imaging, two images of the target are taken, using two cameras viewing the target from different positions. To obtain 3-D information, a search algorithm is first performed to determine which pixels from the two images correspond to the same spot on the object (pixel correspondence) based on pattern/texture/edge matching. Second, disparity for each pair of corresponding pixels is calculated. With known inter-camera distance and magnification, disparity values can be translated into distance from the camera to the object, yielding a distance map of the visible surfaces.
Practical difficulties exist in applying stereo imaging to AOI. To solve for pixel correspondence, significant overlap of the two cameras' fields of view is required. At magnification levels commonly used for PCB inspection, the lenses must be spaced much closer than most high-quality lens sizes would allow. Thus, either lower-quality compact lenses must be used, the magnification reduced, or the cameras put at an angle to each other, adding the computational cost of perspective correction. For targets with little surface pattern, such as a black component or the smooth metal surface of a solder joint, pixel correspondence is ambiguous, leaving uncertainty in the disparity solution. Because of the ambiguity and complexity found in applying stereo imaging technique to AOI, multiple cameras (some AOI systems use 10 or more) do not automatically produce 3-D information.
A complete depth profile is not the only way to describe object shape. Surface features such as surface slope also provide information about target shape. Some AOI systems are designed to capture such information. In systems with ring lighting, a single camera images the object illuminated from different angles. Images taken with low angle lights show areas with steep slopes, whereas high angle lights show flat regions. Since a single camera is used, information from individual images already is aligned, eliminating the need to solve for pixel correspondence. In fact, AOI systems with a single camera and multiple lights provide a good starting point for developing more sophisticated 3-D imaging systems.
Surface Recovery with a Single Camera
Research has long found that surface shape information (e.g., slope and orientation) can be determined by analyzing how the surface reflects light incident on it from different angles. Various models link illumination and surface properties such as gradient, reflectance and specularity to the level of light reflected into the camera.
These models can be used to compute surface gradient from images taken under different independent light sources. From the surface gradient information, a 3-D surface height map can be computed for a continuous surface by integration.
Research continues to be performed in universities and industrial labs, contributing to understanding of the theoretical basis of these approaches. There also are ongoing explorations into implementations that are practical for use in practical applications.
Single-camera/multiple-light source methods work especially well for surfaces that are mostly diffuse. Impressively detailed and accurate reconstructions of 3-D objects have been demonstrated using advanced variations of these methods, as shown in Figure 1. These techniques can even work for specular surfaces, provided the images have sufficient dynamic range, or a good algorithm is used to separate the diffuse and specular elements of the reflection. Solder joints, if not 100 percent specular, are uniquely suitable for this type of reconstruction due to their general surface smoothness.
Figure 1. Shape reconstruction using a single camera with three independent light sources.
The single-camera methods, however, have their own difficulties in dealing with specular surfaces in practical applications. Since specular reflections tend to be much brighter than diffuse ones, and digital cameras have a limited dynamic range, most images taken for shape recovery tend to have exposure levels that saturate the specular spots on the target to record diffuse reflections. This means the total amount of light reflected is uncertain at the saturated spots, and it is difficult to separate the specular and diffuse elements.
Solid Shape Modeling
Given the constraints of a typical AOI setup, solid shape modeling (SSM) technology was developed to reconstruct the surface shape of an inspection target using a single, high-resolution camera and a configurable light source. SSM technology transforms traditional single-camera/multiple-light source methods into a practical technique that provides speedy reconstruction for highly specular surfaces and dark targets for which the captured images typically are noisy.
With SSM, surface shape of a target is computed in three steps. First, several high-resolution images of the target, illuminated from different angles, are captured. These images then are used to compute the surface slope at each pixel, which are used to compute the surface shape. For a continuous surface, H is the matrix of height values sampled on a regular grid and D(H) is the set of surface slopes at those points. For each light source, LI = (xi, yi, zi) is the direction to the light source from the object and Ci is a matrix of pixel values observed using that light source. Using diffuse and specular reflection models, the image pixel values Ci can be expressed as proportional to the amount of light reflected from the surface into the camera, which is a function of light source position and surface slope:
Kd × P(D(H), L) + Ks × Q(D(H), L) + e = C
where L = (L1, L2, ..., Ln) and C = (C1, C2, ..., Cn). For a set of n images, taken using different light sources, P(x) and Q(x) are the diffuse and specular reflection models and Kd and Ks are the matrices of scalar values modeling diffuse and specular reflectance. The noise matrix e represents the variations in measurement of C. With knowledge of L, P(x), Q(x) and D(x), the measured values for C and an assumed noise distribution for e, the above equation can be solved for the unknown elements, particularly the heights H. This is the general equation underlying the SSM technology.
For each pixel location, intensity values from the n images are processed under the assumption that they represent the same object location. Thus, a rapid sequence of images from a single, high-resolution, high-speed, low-noise camera work much better than a set of images from several cameras of lesser quality, which would require error-prone corresponding pixels calculation.
In a typical AOI setting, the number of images required for SSM can be anywhere between four and nine, depending on camera quality, light source configuration and the level of control over ambient lighting. Improvements to the traditional single-camera technique have been developed. These solve problems that arise from board position variations, camera image noise, lighting angle variations, ambiguity in the type of reflection for each pixel (diffuse or specular) and lighting uniformity. To make a 3-D AOI system work reliably on the factory floor, where ambient illumination and camera and light source positions may not be perfectly controlled, substantial research effort and imaging expertise is needed.
Solid Shape Modeling in SMT
SSM can be used to quickly generate shape reconstructions of SMT components. Figure 2 shows a series of small chip capacitors reconstructed using SSM.* The reconstruction shows the presence and alignment of the parts clearly.
Figure 2. SSM reconstructions of a series of capacitors imaged using an AOI machine.*
Figure 3 shows the SSM reconstruction of the surface of a U.S. dime, imaged and reconstructed using the same AOI machine. This demonstrates SSM's ability to recover detailed shape information for highly specular targets.
Figure 3. SSM reconstruction of a portion of a U.S. dime, imaged using an AOI machine.* Color represents surface orientation.
SSM has advantages over traditional 3-D imaging techniques when applied in AOI, where highly specular targets must be inspected in detail. Shape reconstruction is not dependent on surface pattern, making it possible to recover the shape of objects with a smooth, uniform surface. Using a single, fixed camera to acquire images ensures perfect registration between them, eliminating the need for lens and perspective correction calculations and the costly and error-prone pixel-correspondence calculations required in a multiple-camera system. Use of a single, high-resolution camera allows the system to remain cost-effective, compared to multiple-camera systems.
The Future of SSM
SSM is a promising starting point for full-featured, 3-D AOI solutions. Availability of 3-D shape information opens doors to many new AOI applications, such as solder joint metrology, improved presence/absence checking and better information for the rework station.
Figure 4 shows the surface of a lead and solder joint reconstructed with SSM. Using SSM shape information to parameterize the joint shapes is currently being explored as part of a more sophisticated solder joint inspection algorithm.
Figure 4. Surface wire-frame image of a single solder joint reconstructed with SSM.
SSM also can be used to reconstruct profiles taken across the body of a chip component. These profiles summarize the most important parts of the component's shape, and can be used as an ultra-fast presence/absence check.
* Agilent SJ50 Series II.
References
For a complete list of references, please contact the authors.
Xuemei Zhang, Ph.D., research engineer, Agilent Labs, may be contacted at 3500 Deer Creek Rd., Palo Alto, CA 94304; E-mail: xuemei_zhang@agilent.com. Gareth Bradshaw, Ph.D., R&D software engineer, Agilent Technologies Ireland, may be contacted at Silverstone House, Ballymoss Road, Dublin, Ireland.