-
- News
- Books
Featured Books
- smt007 Magazine
Latest Issues
Current IssueBox Build
One trend is to add box build and final assembly to your product offering. In this issue, we explore the opportunities and risks of adding system assembly to your service portfolio.
IPC APEX EXPO 2024 Pre-show
This month’s issue devotes its pages to a comprehensive preview of the IPC APEX EXPO 2024 event. Whether your role is technical or business, if you're new-to-the-industry or seasoned veteran, you'll find value throughout this program.
Boost Your Sales
Every part of your business can be evaluated as a process, including your sales funnel. Optimizing your selling process requires a coordinated effort between marketing and sales. In this issue, industry experts in marketing and sales offer their best advice on how to boost your sales efforts.
- Articles
- Columns
Search Console
- Links
- Events
||| MENU - smt007 Magazine
Digital Pseudo-Color Inspection Reveals Hidden Soldering Defects
July 20, 2010 |Estimated reading time: 11 minutes
Computational tools that render digitally processed images in pseudo-color can enable inspectors to recognize soldering defects such as lifted leads and cold solder in SMT mounting processes. Soldering defects not detected by the automatic optical inspection (AOI) systems were submitted to the script developed in Matlab programming, where the images from AOI were enhanced with pseudo-color in defect areas. With the images enhanced, comparative evaluations were accomplished between the cases of defects and no defects, and included resources for visual judgement. This method, presented by marcello Goncalves Costa, University of State Amazon, Brazil, University Federal of Amazon, Sony Brazil, enables a semi-automatic system of inspection, where the image is enhanced in pseudo-color to aid visual judgement, presenting prominence in the defect areas and efficiently highlighting low-visability defects. Color has a powerful relevance in human image analyses. The human eye can discern thousands of color shades and intensities, compared to about only 24 shades of gray. It simplifies object identification and extraction from a scene. [1, 2] Color image processing is divided into two major areas: Full-color and pseudo-color processing. In the first category, the images are acquired with a full-color sensor (color TV, camera, scanner). In the second, a color is assigned to a particular monochrome intensity or range of intensities based on a specified criterion. [1] The principal use of pseudo-color is for human visualization and interpretation of gray-scale events in an image or sequence of images. [1] Pseudo-color processing consists of intensity slicing and color coding. Thus, if an image is interpreted as a 3-D function, the method places planes parallel to the coordinate plane of the image; each plane slices the function in the area of intersection (Figure 1). [1]
Figure 1: Slicing technique applied on pseudocolor processing.
The intensity of color assignments are made according to their relation. If a different color is assigned to each side of the plane, any pixel whose intensity level is above the plane will be coded with one color, and any pixel below the plane will be coded with the other color. [1] Thus, regions appearing of constant intensity in the monochrome image are really quite variable, as shown by the various colors in the slicing image. [1]
Simple color coding that assigns different gray-scale levels in an image can powerfully aid differentiation by inspection operators, especially if numerous images are involved. In electronics manufacturing facilities, automatic and miniaturized PCB assemblies require discerning inspection to detect flaws of pick-and-place and paste print/reflow processes on SMT components. Soldering defects are characterized by non-establishment of solder joint between component terminals and the board pad. To ensure defect detection, AOI systems are widely used. These inspection systems can even use CCD high-resolution cameras, set at different angles, to generate 3-D solder joint images in great detail. Visual inspection of lifted component leads of components and malformed solder joints (cold solder) is challenging. [3, 5] Lifted or floating leads (floating) are raised from the board pad post-reflow, normally associated with terminal presenting deformation. The solder joint is not established and there is no, or sporadic, metallic contact between component and board. SMDs with leads, such as ICs (SOP, QFP), connectors, etc., are vulnerable to this hard-to-find defect. [5]
Cold solder is a phenomenon in which the solder joint is only partially formed or not formed at all between solder on the pad and the lead base, although it appears visually that such a solder joint is formed. [5]
When AOI systems cannot detect these soldering defects, operators manually perform a visual inspection, which requires a long time and has low reliability on flaw perception. Digital pseudo-color enhancement can semi-automate visual inspection.
Experiment
The experimental procedure consisted of tests with images of soldering defects not detected by AOI systems. These defects were identified in subsequent production stages through electrical tests on PCBs or functional tests in completely mounted sets. Those images selected presented a sampling of the defects that occur in assembly. The samples were submitted to digital processing, which assigned pseudo colors to distinguish different levels of gray-scale present in the soldered area. In the next step, the images were analyzed visually and flaw areas characterized against acceptable soldered areas. This method was then used in inspection of potential flaws to discover defects and verify inspection capability.
The defect sampling was collected from April to September 2009. As observed in Figure 2, lifted leads represent 35% of defect cases, demonstrating AOI’s low ability to catch this defect before final assembly.
Figure 2: Escape defects occur in accompaniment stage: April-September 2009. (Assembly Defect Ratio Sony Brazil.)
Among the most frequent cases of lifted leads, defects predominantly occur with components with multiples terminals, such as ICs and connectors (65% of the cases). They demonstrate a high susceptibility to those flaws, due to mounting errors or inadequate component handling.
The images of selected defects were obtained through CCD cameras on AOI systems. Each inspection zone image is processed through a programming interface that applies image-recognition to identify mounting and soldering defects. Normally, different luminance programs enhance the board and components surfaces to emphasize solder joint areas via tilted surface enhance. The combination of lighting techniques--tilted (side) and direct (top) surface reflections--allows the AOI system to highlight one solder joint. However, all defect alarms presented by the system are judged visually as red, green, blue (RGB) images without auxiliary enhancement.
Figure 3: Capture image process and the set of lights used in typically AOI systems.
In the analysis, each defect image was captured under the same inspection illumination conditions. Images were obtained in bitmap RGB format.
The image enhancement script applies the pseudo-color to grayscale images to enhance the soldering area, characterizing cases of imperfection on solder joints (lifted leads, cold solder) and simplifying visual defect recognition. It converts the original bitmap RGB image in a gray-scale image (monochrome) for pseudo-color assigning. The image is divided in four slices coding with different colors for each gray-scale band defined. Red is assigned to the higher intensity levels (190~255); intermediate levels correspond to yellow color (127~191) and cyan (64~127); lower levels are assigned blue (0~64). Separating the intensities in the soldering area and leads allows better visual analysis for characterization of defect conditions and non-defect good joints.
The developed script consists of image selection, conversion RGB to grayscale, and subsequent coding colors into four slices of intensities. Finally, the script generates figures of the original image and enhanced it with pseudo-color for visual inspection.
Table 1: Matlab script developed to obtain the pseudo-color through the original image. Command Description I = imread('Image.bmp'); Read the RGB image in BMP format. I2=rgb2gray(I); Convert RGB image to gray-scale (monochrome) image to apply pseudocolor. Y2cores = grayslice(I2,4); Return an indexed image with four illumination ranges coded with four different colors. figure;imshow(I2), title 'Image Original'); Show the original image. figure;imshow(Y2cores,jet(4)),colorbar, title('Image in Pseudocolor'); Show the pseudocolor image with color bar. Results
Of the defect images analyzed, a satisfactory amount of defective areas were detected visually when enhanced via pseudo-color. For visual judgment, enhancing AOI images in pseudocolor simplifies flaw recognition, allowing easy fault perception in comparison with the conditions using grayscale or RGB images.
Figure 4: Script images resulting: (a) original image selected to analyses and (b) pseudo-color image to visual interpretation.
Figures 5-8 present the AOI images in pseudocolor for the analyzed cases to demonstrate enhancement in the defect soldering areas.
Figure 5: IC presenting lifted leads with terminal deformed. Gray-scale image (left); pseudo-color image (center) real image of defect (right).
Case 1. IC SOP 16 terminals. In Figure 5, the flotation in the terminal generates alterations in the illumination levels on the soldering area and the terminal body. Under normal inspection conditions, in gray-scale, high illumination levels are observed at the solder joint, because the solder volume is concentrated near the component termination. The lead-to-pad solder joint formation reflects the light brightly, while lower levels are observed in the rest of board pad. In this case, contours on the terminal create divisions in the illumination levels. The higher levels are observed in plane areas (pad, lead top) and low illumination levels are present where reflection is low (slope area). The floating conditions in the region between terminal top and soldering area show constant levels of illumination. The terminal deformation surface creates direct reflection and brighter intensities. The soldering area presents a volume dispersed in the pad extension with high levels. Applying pseudo-color enhancements to the terminal and soldering areas allows inspectors to characterize the flaw with the color signature (red) at the highest levels of the image. In a normal solder joint, presented in pseudo-color, the red portion would be reduced on the soldering area, representing only the solder joint. The terminals would present different colors.
Figure 6: IC presenting lifted lead. Gray-scale image (left); pseudo-color image (center); real image of defect (right).
Case 2. IC SOP 24 terminals. This case of flotation (Figure 6) doesn't present significant deformities in the terminals, as observed in Case 1. However, the characteristics of the soldering area present the same aspect, because absence of contact with the terminal prevents solder joint formation. A larger soldering volume in the extension of pad reflects higher illumination levels. In the terminal, the illumination evidences a slope area that characterizes the mechanical integrity of the joint. Pseudo-color enhancement detaches the defect presence on soldering area (red), although apparent normality exists in the terminal (yellow in the middle of terminal).
Figure 7: Connector presenting lifted lead. Gray-scale image (left); pseudo-color image (center); real image of defect (right).
Case 3. Connector a12 terminals. The analyzed component has more rigid terminals presenting little deformity level. Therefore, the defect cannot be characterized in the image. The soldering area, however, presents characteristics similar to lifted lead conditions, where there isn’t contact between the terminal and the solder on the pad. Pseudo-color enhancing reinforces the suspicion that solder volume is dispersed along the pad, creating a defect (Figure 7).
Figure 8: IC presenting lifted leads with accentuated deforming. RGB Image (left); pseudo-color image (center); real image of defect (right).
Case 4. IC QFP 96 terminals. As in Case 1, Figure 8 shows high deformation of the terminals. This deformity can be observed in the image enhanced in pseudo-color, where the slope area is absent and the levels of illumination are maintained constant on the lead (red), characterizing the defect. However, the solder pad, in this case, is covered by the lead defect, hidden from AOI.
Visual Inspection
With pseudo-color enhancements applied to the defective images, inspectors were asked to make judgments of possible defects. The images were separated and presented, firstly in gray-scale with defects and non-defects side by side for comparison (Phase 1). Later, the image in pseudo-color was offered under the same comparative conditions (Phase 2). After training, inspectors were presented the images of defects separately (without comparison) in gray-scale and in pseudo-color (Phase 3). The perception results can be observed in Table 2, where the judgments made by three inspectors of each shift are presented. The condition "OK" represents the correct recognition of the defective areas and "NG" represents the wrong judgment.
Table 2: Recognition tests results applied with gray-scale or RGB images in comparison with pseudo-color-enhanced images. PHASE 1: COMPARISON BETWEEN DEFECTS AND NON-DEFECTS IMAGES Inspection GRAY-SCALE OR RGB IMAGE PSEUDO-COLOR IMAGE Shift 1 Shift 2 Shift 3 Shift 1 Shift 2 Shift 3 Inspector 1 OK OK NG OK OK OK Inspector 2 NG OK OK OK OK OK Inspector 3 OK OK OK OK OK OK PHASE 2: ISOLATED IMAGES OF DEFECTS Inspection GRAY-SCALE OR RGB IMAGE PSEUDO-COLOR IMAGE Shift 1 Shift 2 Shift 3 Shift 1 Shift 2 Shift 3 Inspector 1 OK NG NG OK OK OK Inspector 2 NG NG OK OK OK OK Inspector 3 OK OK NG OK OK OK
In agreement with Table 2, in Phase 1, defects were recognized more successfully in pseudo-color (100%), while the images in gray-scale present some wrong judgments even with the aid of comparison conditions of defects and non-defects. For the second phase, the images of terminals were made isolated (without comparison images) to test the efficiency of perception at visual inspection. In this situation, successful defect-identification attempts for the gray-scale images are low (44%), while 100% success was achieved with the image in pseudo-color.
Conclusion
Based on these results, the critical soldering defects associated with lifted leads of SMD components were satisfactorily recognized at visual inspection when submitted to pseudo-color enhancing. Pseudo-color image progressing makes optical inspection an easier task with more automation.
The use of pseudo-color aids in visual judgment of defect conditions; inspectors had a higher level of defect perception when looking at enhanced images in comparison with gray-scale or RGB. In normal SMT production conditions, inspection time is relevant to product quality and line productivity. Usually, defect inspections are made visually and offline posts (ICs, connectors). Applying pseudo-color to images of the components with high potential for lifted leads is a significant aid in the visual inspection on the line, allowing better detection and decreased time spent to judge PCBs. This experimental process can be included in SMD mounting lines, associated to the AOI systems, allowing efficient visual detection of lifted leads when the AOI alerts operators of a potential defect.
Acknowledgements
The work presented in the article was developed with the university’s support to joining knowledge for research and development to SMT processes. Special thanks are extended to Marcia Helena Veleda Moita, Ph.D., industrial engineering, Professor, University of Amazonas and Juliana Ferreguete Sena, Ms.c., electrical engineering, researcher, University of Amazonas.
References:
1. Gonzalez, R.C.; Woods, R.E., Digital Image Processing, New Jersey, USA: PearsonPrentice Hall, 2008. 914 p.2. Jain, A.K. Fundamentals of Digital Image Processing, New Jersey, USA: Prentice Hall Inc., 1989. 560 p.3. Savage, R.M.; PARK. H.S.; FAN, M.S. Automated Inspection Of Solder Joints forSurface Mount Technology. NASA Technical Memorandum, USA, 1993. 32 p.4. Gonzalez, R.C.; Woods, R.E.; Eddins, S.L., Digital Image Processing UsingMatlab, New Jersey, USA: Prentice Hall, 2003. 624 p.5. ASSEMBLY DEFECT RATIO 1HY2009. Módulo de controle estatístico de defeitos de escape. Sony Brasil LTDA, Manaus, 2009.
Marcello Gonçalves Costa, graduated in Electrical Engineering, University of State Amazonas (UEA), Brazil, Specialist in Video Systems, University of Amazonas (UFAM), Brazil and is a process engineer - SMT automount/AOI at Sony Brazil. He may be contacted at marcelloee@hotmail.com.