基于深度学习的电子元件焊点缺陷检测方法*

doi:10.16257/j.cnki.1681-1070.2023.0081

电子与封装 ›› 2023, Vol. 23 ›› Issue (6): 060206 . doi: 10.16257/j.cnki.1681-1070.2023.0081

基于深度学习的电子元件焊点缺陷检测方法^*

刘玉龙^1,2;吕权权^1,2;吴浩^1,2;单建华^1,2

1. 特种重载机器人安徽省重点实验室，安徽马鞍山 243032；2. 安徽工业大学机械工程学院，安徽马鞍山 243032

收稿日期:2022-10-02 出版日期:2023-06-26 发布日期:2023-05-06
作者简介:刘玉龙（1998—），男，安徽合肥人，硕士，主要研究方向为深度学习与缺陷检测。

Solder JointDetection Method for Electronic Components Based on Deep Learning

LIU Yulong^1,2, LYU Quanquan^1,2, WU Hao^1,2, SHAN Jianhua^1,2

1. Anhui Province Key Laboratory of Special Heavy Load Robot, Maanshan 243032, China;2.School of Mechanical Engineering, Anhui University of Technology, Maanshan 243002, China

Received:2022-10-02 Online:2023-06-26 Published:2023-05-06

摘要/Abstract

摘要： 提出一种用于训练掩模区域卷积神经网络（Mask R-CNN）的半自动生成焊点图像掩模的方法。由于传统的通过人工标注获取掩模的方法费时费力，提出了一种简便快捷的基于GrabCut获取图像掩模的方法。该方法由两个阶段组成，第一阶段为基于GrabCut的焊点图像分割，输出像素级分割结果，从而获得所输入图像掩模。第二阶段实现基于Mask R-CNN的焊点表面缺陷检测方法，可以实现对缺陷的定位、分类和分割。实验结果证实了该方法的有效性，在保证Mask R-CNN方法检测精度的前提下，能快速简单地获取训练Mask R-CNN所需的焊点掩模。

关键词: 缺陷检测, 深度学习, 焊点检测, 卷积神经网络

Abstract: A method of semi-automatic of solder joint image Mask generation for training mask region convolutional neural network (Mask R-CNN) is proposed. Because the traditional manual annotation method of obtaining mask is time-consuming and laborious, a simple and fast method of obtaining image mask based on GrabCut is proposed. The method consists of two stages. The first stage is the solder joint image segmentation based on GrabCut. The pixel level segmentation results are output to obtain the input image mask. In the second stage, the detection method of solder joint surface defects based on Mask R-CNN is implemented, which can realize the location, classification and segmentation of defects. Experimental results confirm the effectiveness of the proposed method. Under the premise of ensuring the detection accuracy of Mask R-CNN method, the solder joint Mask required for training Mask R-CNN can be quickly and simply obtained.

Key words: defect detection, deep learning, solder joint inspection, convolutional neural network

中图分类号:

刘玉龙;吕权权;吴浩;单建华. 基于深度学习的电子元件焊点缺陷检测方法^*[J]. 电子与封装, 2023, 23(6): 060206 .

LIU Yulong, LYU Quanquan, WU Hao, SHAN Jianhua. Solder JointDetection Method for Electronic Components Based on Deep Learning[J]. Electronics & Packaging, 2023, 23(6): 060206 .

参考文献

[1] TAO X, ZHANG D, MA W, et al. Automatic metallic surface defect detection and recognition with convolutional neural networks[J]. Applied Sciences, 2018, 8(9): 1575.
[2] TSAI D M, HSIAO B. Automatic surface inspection using wavelet reconstruction[J]. Pattern Recognition, 2001, 34(6): 1285-1305.
[3] TSAI D M, HSIEH C Y. Automated surface inspection for directional textures[J]. Image and Vision Computing, 1999, 18(1): 49-62.
[4] WU H, ZHANG X M, XIE H W, et al. Classification of solder joint using feature selection based on Bayes and support vector machine[J]. IEEE Transactions on Components, Packaging and Manufacturing Technology, 2013, 3(3): 516-522.
[5] TSAI T N. Development of a soldering quality classifier system using a hybrid data mining approach[J]. Expert Systems with Applications, 2012, 39(5): 5727-5738.
[6] ACCIANI G, BRUNETTI G, FORNARELLI G. Application of neural networks in optical inspection and classification of solder joints in surface mount technology[J]. IEEE Transactions on Industrial Informatics, 2006, 2(3): 200-209.
[7] 黄汉成，赵卫东，夏冰.基于深度学习的指针缺陷检测研究[J].安徽工业大学学报（自然科学版），2019，36(4)：375-381.
[8] 郑建聪，谢麒麟，方挺，等. 一种基于特征点的管材表面缺陷视觉检测方法[J]. 安徽工业大学学报(自然科学版)，2022，39(1)：21-24.
[9] CAI N, CEN G D, WU J X, et al. SMT solder joint inspection via a novel cascaded convolutional neural network[J]. IEEE Transactions on Components, Packaging and Manufacturing Technology, 2018, 8(4): 670-677.
[10] REN S, HE K, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.
[11] HE K M, GKIOXARI G, DOLLáR P, et al. Mask R-CNN[C]//2017 IEEE Transactions on Pattern Analysis and Machine Intelligence, Puerto Rico, IEEE, 2017: 2980-2988.
[12] WU H, GAO W B, XU X R. Solder joint recognition using Mask R-CNN method[J]. IEEE Transactions on Components, Packaging and Manufacturing Technology, 2020, 10(3): 525-530.
[13]王联君，王静秋. 基于GrabCut的磨粒图像分割方法研究[J]. 机械制造与自动化，2019，48(2)：127-130，137.
[14] BOYKOV Y Y, JOLLY M P. Interactive graph cuts for optimal boundary and region segmentation of objects in n-d images[C]// 2001 Proceedings, Eighth IEEE International Conference on Computer Vision, Vancouver, British Columbia, Canada, IEEE, 2001: 105-112.
[15] GIRSHICK R. Fast R-CNN[C]// 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 2015: 1440-1448.
[16] 汪鹏宇，瞿栋，黄允，等. 基于Faster R-CNN的PCB缺陷检测研究[J]. 计量与测试技术，2021，48(10)：9-11.
[17] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016: 770-778.
[18] MANNING C D, RAGHAVAN P, SCHüTZE H, Introduction to information retrieval[J]. Natural Language Engineering, 2010, 16(1): 100-103.
[19] OTSU N. A threshold selection method from gray-level histograms[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1979, 9(1): 62-66.
[20] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]// IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(4): 640-651.
[21] SINGH A, SHA J, NARAYAN K S, et al. Bigbird: A large-scale 3D database of object instances[C]// 2014 IEEE international conference on robotics and automation (ICRA), Hong Kong, China, IEEE, 2014: 509-516.
[22] WU H, XU X R. Solder joint inspection using eigensolder features[J]. Soldering and Surface Mount Technology, 2018,30(4): 227-232.
[23] WU H. Solder joint defect classification based on ensemble learning[J]. Soldering and Surface Mount Technology, 2017, 29(3): 164-170.

中国半导体行业协会封装分会会刊

中国电子学会电子制造与封装技术分会会刊

基于深度学习的电子元件焊点缺陷检测方法^*

Solder JointDetection Method for Electronic Components Based on Deep Learning

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 7

编辑推荐

Metrics

本文评价

[1]	鞠虎;高营;田青;周颖. 基于RISC-V的神经网络加速器硬件实现^*[J]. 电子与封装, 2023, 23(2): 20306-.
[2]	董宜平;谢达;钮震;彭湖湾;贾尚杰. 基于FPGA的双源无轨电车的改进型YOLO-V3模型^*[J]. 电子与封装, 2022, 22(8): 80501-.
[3]	胡凯;刘彤;武亚恒;谢达. 基于Vitis-AI架构的语义分割ENET模型实现^*[J]. 电子与封装, 2022, 22(3): 30304-.
[4]	吕璐;程虎;朱鸿泰;代年树. 基于深度学习的目标检测研究与应用综述[J]. 电子与封装, 2022, 22(1): 10307-.
[5]	王彬;高嘉平;司耸涛. 基于卷积神经网络的图像分类及应用[J]. 电子与封装, 2021, 21(5): 50503-.
[6]	谢达，周道逵，季振凯，戴新宇，武睿. 基于异构多核平台的Caffe框架物体分类算法实现与加速[J]. 电子与封装, 2019, 19(5): 16-21.
[7]	魏敬和, 林军. 深度学习算法、硬件技术及其在未来军事上的应用[J]. 电子与封装, 2019, 19(12): 1-6.