9 views

Original Title: 3-D Shape Measurement of Complex Objects by Combining

Uploaded by Ravi Kiran

3-D Shape Measurement of Complex Objects by Combining

nural network

Attribution Non-Commercial (BY-NC)

- Using Artificial Neural Networks & Twitter Sentiment Analysis to Predict Stock Movement
- Implementation of 3D Optical Scanning Technology For
- Soft Computing Unit 1 and 2 Questions
- High-Speed Neural Network Controller for Autonomous Robot Navigation using FPGA
- Backpropagation Lecture
- PNN
- Neural-based Flight Control System
- Tire_a_part
- Neural Networks Backtracking
- 172
- 3D Training 2011 LW Series
- A Java Based tool for diagnosis of Diabetes type 2
- Knowledge Extraction From Trained Neural Networks
- Introduction to Neural Networks, Deep Learning (Deeplearning.ai Course)
- New Microsoft Word Document
- Understanding of FF-ANN
- A pleminary estimate of time and cost in uran and road construction using neural networks.pdf
- neural
- Developing Study Program Choice by Using Neural Network
- backprop.pdf

You are on page 1of 5

3-D Shape Measurement of Complex Objects by Combining Color-Coded Fringe and Neural Networks

QIN Dahui LI Zhongwei

State K e y L a b o r a t o r y of Material Processing and Die and M o u l d Technology, H u a z h o n g University of Science and Technology, W u h a n 430074, China

Abstract:

A new approach for three dimensional (3-D) shape measurement was proposed based on color-

coded fringe and neural networks. By applying the phase-shift technique to fringe projection, point clouds were generated with high spatial resolution and limited accuracy. The picture element correspondence problem was solved by using projected color-coded fringes with different orientations. Once the high accurate corresponding points were decided, high precision dense 3-D points cloud was calculated by the well trained net. High spatial resolution can be obtained by the phase-shift technique and high accuracy 3-D object point coordinates are achieved by the well trained net, which is not dependent on the camera model and will work for any type of camera. Some experiments verify the performance of this method.

Key words:

three dimensional (3-D) shape measurement; color-coded fringe; neural networks; correspon-

dence problem

Introduction

The automatic, non-contact three dimensional (3-D) measurement of object surfaces is one of the most important applications of digital close-range photogrammetry and computer vision, and is becoming more and more important in industrial application such as design, manufacturing, and automatic error detection ' . Many

[1 2]

still slow for 3-D coordinates acquirement. The projection of a laser line or a grid of coded light is more efficient than the projection of laser spot. For line laser range scan and coded light scan measurement, spatial resolution of image information acquired by chargecouple device (CCD) camera is low, moreover, all images need to be thresholded, so it tends to be sensitive to the circumstances. The phase-shifting technique not only can obtain high spatial resolution but also is insensitive to the circumstances. Besides, it is highly robust. So fringe projection based on phase shift is adopted. A fringe projection system consisting of one C C D camera and a projector shown in Fig. l a is based on an active pattern projection because the pattern geometry of the projection unit must be calibrated. A fringe projection system consisting of at least two C C D cameras and a projector shown in F i g . l b is based on a passive pattern projection because calibration and orientation of the projector is not required . The calibration of the

[6]

[3] [4]

and phase-shift

[5]

few years. These sensor systems are based on triangulation and structured light. The scanning of a surface with a light spot is a simple method of surface 3-D measurement, which provides a non-contact and fast measurement compared to conventional coordinate measurement machines, but is

Received: 2008-11-09; revised: 2009-02-23 * Supported by the Eleventh Five-Year Pre-Research Project of China * * To whom correspondence should be addressed. E-mail: qindahui@gmail.com; Tel: 86-13098820919

QIN Dahui

(Jl^-kJ^)

et al:

3-D Shape

Measurement

of Complex

Objects

by Combining

67

projector is very complicated. The accuracy of the calibration is lower than the calibration of a camera. Most of the projectors use non sinusoidal patterns to project the phase shift images. The phase error is unable to be compensated . A passive pattern projec[7]

color-coded fringe projection. A three-layer BP neural network (BPNN) is well trained to establish the mapping relationship between the homologous image point coordinates and corresponding 3-D coordinates of the measured object. Then, correspondence between homologous points can be performed by a special approach for phase determination

[12]

tion adopted in this paper can eliminate systematical error sources like non-sinusoidal higher precision.

Projector Projector

wave-forms

and

tage of this approach is that high spatial resolution can be obtained by the phase-shifting technique using fewer fringe images and that high accurate 3-D object point coordinates are achieved by the well trained net which is not dependent on the camera model and works for any type of camera in 3-D shape measurement for very complex objects.

1

(a) A c t i v e t r i a n g u l a t i o n Fig. 1 (b) P a s s i v e t r i a n g u l a t i o n T w o t y p e s o f s t r u c t u r e d light s y s t e m

Principle

B P neural network (BPNN) is a multilayer feedforward network with backpropagation algorithm. The typical architecture of the three layers B P N N with a hidden layer is shown in Fig. 2.

Input layer Hidden layer Output layer

Camera calibration is a critical step for reconstruction of object surface, and its goal is to establish coordinate mapping between the global 3-D-coordinates of a point and the 2-D-coordinates of the projected ima g e ^ . Classical camera calibration approach based on perspective projection

10

model

involves

full-scale

nonlinear optimization^' \ and requires a lot of good initial guess values. Therefore, it is very computingintensive. Tsai's two-step calibration

[11]

is a representa-

tive for the two-step type calibration techniques which involves a direct solution of some camera parameters and an iterative solution for the other parameters. Twostep approach has significant improvement, but it is still cumbersome with decreased accuracy. Neural networks is a well-known approach to approximate a nonlinear system without an explicit physical model, and it can use two image coordinates (x y\

f

Fig. 2

The neural network has m inputs and neurons in its output layer, these numbers were picked by a particular application. The hidden layer has r neurons. This number was picked by guesswork and experience. The wy ( z - 1 , 2, 3, r; j = 1, 2, 3, m) is the connection (scalar weight) from the No. neuron in the input layer to the No. j neuron in the hidden layer. The wjd (k = 1, 2, 3, n; i = 1, 2, 3, r) is a connection (scalar weight) from the z-th neuron in the hidden layer to the -th neuron in the output layer. The bj is a scalar bias to the y'-th neuron of the input layer and b is a scalar bias

t

x"y")

of

homologous (corresponding) image points and corresponding object 3-D world coordinates point coordinates (XYZ) to train net so as to establish the relationship between the global 3-D-coordiantes of a point and stereo-pair images of homologous image point projection for two C C D cameras. Therefore, it eliminates camera mathematic model and does not calculate explicit values of extrinsic or intrinsic camera parameters. Therefore, the cameras need not to be calibrated actually. In this paper, a method is proposed based on stereovision using back propagation (BP) neural network and

to the z-th neuron of the hidden layer. Neural networks are commonly adjusted or trained, so that a particular input leads to a specific target output. The network is adjusted, based on a comparison of the output and the target, until the network output matches the target. Generally speaking, the least

68

Tsinghua

Science

and Technology,

June 2 0 0 9 , 1 4 ( S 1 ) : 6 6 - 7 0

mean square (LMS) error algorithm is the performance function for supervised training in BP neural network. Assume that there are training samples input/target pairs [X Y ] (i = 1,2,... , N). Here X is an input to neth t l

is a smoothing parameter which can be determined by a cross-validation procedure. A trained three-layer B P neural network with enough sigmoid function neurons in the hidden layer can approximate any function with any precision paper, the image coordinates (x y\

f [14]

work, and Yj is the corresponding target output. As each input is applied to the network, the network output is compared to the target. The error is calculated as the difference between the target output and the network output. The goal is to minimize the average of the sum of these errors. mse =

. In this

x"y") 3-D

of homolocoordinate

gous image points can be put into a one-to-one correspondence with the correspondent (XYZ) of an object and obviously it is a complex

V

N

Y-Y

(1)

of various lens distortions. So stereo-pair images and correspondent 3-D coordinate (XYZ) can be used to train networks. The input vector is the image stereopair point coordinates (x'y , x"y"),

f

where mse is the mean square error. The L M S algorithm adjusts the weights and biases of the B P network so as to minimize this mse. The traditional B P neural network suffers from the over-fitting problem and the convergence to global minimum

[13]

spondent 3-D coordinate (XYZ) is obtained by the output layer. To obtain the 3-D coordinate (XYZ) of an object point, the homologous points of stereo-pair images are needed. So correspondence is crucial for 3-D shape measurement. Because there is no explicit camera model parameters, epipolar condition is unable to be used. Correspondence is solved by color-coded fringe projection based four-step phase shift technology. The whole process for correspondence is shown in Fig. 3. The periods of the phase image in Fig. 3 are only for

is

not

always

achieved. Marzban et a l .

proposed a method to

avoid the over-fitting and local minima problem and improve the generalization ability of neural network models, adding a weight-decay or regularization term in the estimation process. Therefore, a regularization or weight-decay term is often added to Eq. (1) E = K - K + r . m s w

1

N

(2)

Fig. 3

W h o l e p r o c e s s for c o r r e s p o n d e n c e

QIN Dahui

(Jl^-kJ^)

et al:

3-D Shape

Measurement

of Complex

Objects

by Combining

69

expatiating correspondence algorithm, and are not the actual measurement. The first 24-bits true color image projected includes three frequencies in the horizontal direction, in which each pixel values are divided into R, G, component of three primary colors, forming three solid color images with sinusoidal intensity distribution to represent three different frequencies. The period numbers of R, G, solid color images are 59, 64, and 70. The other three 24-bits true color images are shifted 90 degrees in turn. Then, four-step phase shift technology is used to obtain four sawtooth images from sinusoidal images b y E q . (3). () = a r c t a n ^ ^ ()- ()

3

Experiments

A 400 m m 300 m m high precise calibration board with 99 circle marks is used to obtain sample data sets for neural network training. To ensure the precision of the neural network, twenty groups of calibration images are captured from different height. One calibration image is shown in Fig. 4. Then the training sample sets can be extracted from the images by using an automatic circle-center-extraction algorithm. Totally 99x20 sample points are collected, and fifteen groups sample data are picked up as training samples and the other five groups are used for test. Once the neural network is trained, it can be employed for 3-D reconstruction.

(-

2 K

(

L

(3)

are the same

where (),

(),

2

(),

3

and 7 ( )

4

solid color image sinusoidal intensity respectively. Then, the sawtooth image with 5 periods is obtained based on the heterodyne principle using two images with 59 and 64 p e r i o d s

[15]

periods is obtained based the same principle using two images with 64 and 70 periods. For last two sawtooth images, the heterodyne principle is used to solve the problem of phase unwrapping again, and finally the image of phase unambiguous over the field of view in the horizontal direction is obtained. For 24-bits true color images projected with three frequencies in the vertical direction, the image of phase unambiguous over the field of view in the vertical direction is obtained by the same process. So correspondence is solved by a pair of absolute phase values in two directions perpendicular to each other.

Fig. 4 Calibration image

To verify the performance of the method, a complex metal part is measured with the trained network. Eight groups color fringe patterns are captured during the measurement process. One group vertical images are shown in Fig. 5. The measurement result is shown in Fig. 6, and the local enlarging graph shows that the system can reconstruct complex 3-D model with very smooth details.

Fig. 5

70

Tsinghua

Science

and Technology,

June 2 0 0 9 , 1 4 ( S 1 ) : 6 6 - 7 0

Fig. 6

Conclusions

[5]

A new approach for 3-D shape measurement is proposed based on color-coded fringe and neural networks. A three-layer enhanced BP neural network is used to train the stereo vision application system to calculate 3-D coordinates. The training set for neural network consists of a variety of stereo-pair images and corresponding 3-D world coordinates. Two different directions fringe patterns are used to solve the correspondence problem. Once the high accurate homologous points are obtained, high precision dense 3-D points cloud can be calculated by the well trained networks. The obvious advantage of this approach is that high spatial resolution can be obtained using fewer fringe images by the phase-shifting technique and high accuracy 3-D object point coordinates are achieved by the well trained networks. A complex metal part is measured with the trained networks, and the experiment result shows that the system can reconstruct complex 3D model with very smooth details. References

[1] Chen F, B r o w n G W, Song M. Overview of three-

[6]

L u h m a n n T, R o b s o n S, K y l e S, H a r l e y I. C l o s e R a n g e P h o togrammetry: Principles, Thechniques and Applications. N e w Jersey: John Wiley and Sons, 2006.

[7]

[8]

[9]

Photogram-

methods.

[13] M a r z b a n C, W i t t A . A B a y e s i a n n e u r a l n e t w o r k for s e v e r e h a i l size p r e d i c t i o n . Weather 610. [14] T a n g Y, C h e n W, S u X , et al. N e u r a l n e t w o r k a p p l i e d t o r e c o n s t r u c t i o n o f c o m p l e x o b j e c t s b a s e d o n fringe p r o j e c t i o n . Forecast, 2 0 0 1 , 16(5): 600-

[3]

Forest

J,

Salvi

J. A

review

of

laser

scanning

three-

Optics

Communication,

d i m e n s i o n a l digitizers, intelligent r o b o t s a n d s y s t e m s . In: I E E E / R S J International Conference on Intelligent Robots and System. E P F L Lausanne, Switzerland, 2002: 73-78, [4] K r a t t e n t h a l e r W, M a y e r J, D u w e P. 3 D - s u r f a c e m e a s u r e m e n t w i t h c o d e d light a p p r o a c h . In: F o u r t h I n t e r n a t i o n a l

[15] R e i c h C, R i t t e r R, T h e s i n g J. W h i t e light h e t e r o d y n e p r i n ciple for 3 - D m e a s u r e m e n t . Proc. 242. SPIE, 1997, 3100: 236-

- Using Artificial Neural Networks & Twitter Sentiment Analysis to Predict Stock MovementUploaded byRohan Patel
- Implementation of 3D Optical Scanning Technology ForUploaded byDinesh Kumar
- Soft Computing Unit 1 and 2 QuestionsUploaded byHarshali Y. Patil
- High-Speed Neural Network Controller for Autonomous Robot Navigation using FPGAUploaded byIOSRjournal
- Backpropagation LectureUploaded byBoonKhaiYeoh
- PNNUploaded byashut911
- Neural-based Flight Control SystemUploaded byRitesh Singh
- Tire_a_partUploaded byarnaudmariel
- Neural Networks BacktrackingUploaded byRamyasree Bade
- 172Uploaded byWasyhun Asefa
- 3D Training 2011 LW SeriesUploaded byFrank Ferrary
- A Java Based tool for diagnosis of Diabetes type 2Uploaded byPrateek Shukla
- Knowledge Extraction From Trained Neural NetworksUploaded bystephenlim7986
- Introduction to Neural Networks, Deep Learning (Deeplearning.ai Course)Uploaded byJanmejay Pant
- New Microsoft Word DocumentUploaded byAguz Munandar
- Understanding of FF-ANNUploaded byapi-3807547
- A pleminary estimate of time and cost in uran and road construction using neural networks.pdfUploaded bySalut Sa Lipunan
- neuralUploaded byAli Faiz Abotiheen
- Developing Study Program Choice by Using Neural NetworkUploaded byAJER JOURNAL
- backprop.pdfUploaded bylubeck abraham huaman ponce
- 9fcfd50e4b1db3b3f6Uploaded byRia Singh
- Singh 2000Uploaded byVitthal Patnecha
- Minin HandoutUploaded byDimas Bagus Cahyaningrat. W
- 00078592Uploaded byali
- Using-Artificial-Neural-Network-Algorithm-for-Voltage-Stability-Improvement.pdfUploaded byAnonymous zpyb7Bq
- ecml10Uploaded bynaufraghi
- OMEGA Gutierrez2010Uploaded byHaq Kaleem
- ClassificationUploaded byJijeesh Baburajan
- Multi-criteria ABC Analysis Using Artificial-Intelligence Based Classification TechniquesUploaded byFabricio Costa
- v11-187Uploaded bySrinivas Hn

- Network Analysis by Van Valkenburg chap 5 solution manualUploaded byBilal Mobeen
- Preamble Tgt (Languages) Result NotfcnUploaded byRavi Kiran
- Ece III Network Analysis [10es34] NotesUploaded byGautam Sharma
- D-88-12_2Uploaded byRavi Kiran
- 06_SPDF Model VHDL Code GenerationUploaded byRavi Kiran
- 18209_1_Job offer Policy 2.docUploaded byRavi Kiran
- UGC NET DECEMBER 2012Uploaded bynanda_kavala
- D-00-12 (W)Uploaded byshreeshlko
- A Novel High Adaptability Out-Door MobileUploaded byRavi Kiran
- Chapter 18Uploaded byRavi Kiran
- anwerkey_paperI.pdfUploaded byMahi Raperia
- BOOTHUploaded byBharath Vijai
- 1.IJAEST Vol No 7 Issue No 1 FPGA Implementation of 2 D DCT for JPEG Image Compression 001 009Uploaded byhelpdesk9532
- School Forensic Sciences AllahabadUploaded byRavi Kiran
- Sequential Shift Add-MethodUploaded byanjanbs
- Artificial Neural Network Maximum Power Point TrackerUploaded byRavi Kiran
- Approximating Nonlinear Relations Between Susceptibility AndUploaded byRavi Kiran
- 18198_1_List of students 9th 10th Nov.xlsUploaded byRavi Kiran
- 2D-FFTUploaded byAngel Mariano
- Introduction (4)Uploaded byRavi Kiran

- MPMC Mod-1Uploaded bySyeda Afreen
- JtagUploaded byunavilable
- Essentials OneWorld Training Data SheetUploaded byamigokalavera
- 07 SEP674 RET670 Overexcitation ProtectionUploaded byطه محمد
- Vector Webinar AUTOSAR Configuration Process 20130419 EnUploaded bysebasTR13
- MS_rec - Reference Material for Writing Recommendation LettersUploaded bysiddharthbhatla
- Guava_PI_Kit_2Uploaded bySiddharth Dev
- Omni MedSci Patent Suit Targeting Apple WatchUploaded byMikey Campbell
- ba204Uploaded byRaviteja Palanki
- CRM-System for FMCG-Producing Companies 1Uploaded byJan Mohammed
- 0808-af-000736-libreUploaded byDEADMAN
- Hacking for Dummies - CheatsheetUploaded bydjtai87
- ATM SignallingUploaded byPhiip Yim
- TDS-11SH Top Drive D392004689-MKT-001 Rev. 01Uploaded byIsrael Medina
- DSE4120 Installation InstructionsUploaded byRahmat Nur Ilham
- mc9s12hz128val-datasheetz.pdfUploaded byCarlos Geeket' El Halabi
- Daknet PDFUploaded bysukk
- ThesisUploaded bySayantan Sinha
- Flownet Nuclear Architecture, Implementation and Verification & ValidationUploaded bySrashmi
- ProtocolUploaded byharry_balken
- e-STUDIO 305Uploaded byMauricio Romeo Venegas Rivera
- VI-D ReportingFramework Steele MKT2Uploaded byWidyan Irwadi
- Leica ScanStation P50 EnUploaded bysniperone1
- e Wastemanagementrules2016Uploaded bySushant Panda
- Faq s84 Standard for Safety Instrumented SystemsUploaded byrgongorav
- Tivoli CampaignUploaded byAnonymous uac2hBX3yb
- Fan+efficiency+articlesUploaded byRami Ghanem
- Talents Potters Test PaperUploaded byVijay Paidipalli
- 8516Uploaded byDanyal Chaudhary
- Documentation T 710Uploaded byIohannes Johnson