A self-calibrated photo-geometric depth camera

所需积分/C币:5 2018-06-09 11:50:54 10.64MB PDF

In this paper, we present a practical 3D imaging system by combining the near-light photometric stereo and the speckle-based stereo matching method. The system is compact in structure and suitable for multi-albedo targets. The parameters (including position and intensity) of the light sources can be
A Self-Calibrated Photo-Geometric Depth Camera (1)The near point light source model is adopted for making a compact sySten he parameters of the point light sources can be calibrated automati (3) The system can be adapted to the mult i-albedo objects (4)The PS method and the stereo matching method are combined reconstruct surfaces with rich details and high accuracv According to the guideline above. we design a 3D imaging system combin- ing the PS method and the traditional stereo vision, called the photo-geometric depth camera. The systcm consists of two CMOs cameras, a ncar infrared speckle projector, four near infrared LED and a synchronous circuit, as shown in Fig. 1 and Fig. 5. The experimental hardware cost is about $400 The main contributions of this paper are 1)A high-precision 3D imaging system is introduced, which combines the photometric stereo and the binocular stereo. The system is low cost, compact and can bc adapted to multi-albcdo targets in the gencral indoor environment (2)A point, light The tradi tional point light source calibration method usually requires specific calibration spheres). In the distant lighting Illodel 7 to estinate the initial surface albedo Imap. Thel, with the estimated albedo map and the normal vector field fixed the parameters of the near lighting model are optimized. Next, with the optimized lighting model wc use the ncar-light photometric stereo(NLPS) mcthod to rc-computc tho albedo map and use the method 9 to compute higher quality depth map. We repeat the above two steps iteratively until convergence or the iteration times reaching a predefined maximum number The reInlaiIlder of this paper is organized as follows: The details of the pro- posed system are presented in Section 2, and experimental results are provided in Section 3. Finally, Section 4 concludes the paper 2 Photo-Geometric Depth Camera 2.1 Hardware Our photo-geometric depth camera consists of two CMOS cameras. a Kinect- typc ncar infrared (NIR) spccklc projector, four NIR LEDs with a wavelength of 830 nm, and a microcontrollcr-bascd circuit, as shown in Fig. 1. Thc cameras have a frame rate of 60 fps and a resolution of 1280x 960. The cameras are connected to the PC via Giga bit Ethernet interfaces. The narrow-band filters are mounted on the 4 mm focal length lenses to filter out ambient light. The two caMeras and the four LEDs are mounted on a rigid structure to keep the stable relative positional relationship. The cameras, the LEDs, and the projector are synchronized by the trigger signal of the microcontroller-based circuit, which is controlled by the PC via USB 2.0 interface. The NIR projector can emit a large number of random speckles to enhance the surface texture 4 Liang Xie et al ( Fig. 1 The proposed photo-geametric depth camera D Speckle projector OFF LEDI-LEDA OFF LEDI-ED4 OF E Speckle projector OFF tor o Deckle projector OFI LEDI--LED OFF LEDI CN utens OFF LED2 ON, otcS OFF LED? ON, utleIs OFF LED4 oN, theiS OFF Each catera s,lap Eacli canela suay cue fi a e Each caela silap uue fiat E-h camela silap Jue iane Ead call.eia siap oe fiane Fig. 2 The timing diagram of a reconstruction cycle As illustratcd in Fig. 2, in a reconstruction cyclc, the projector is lit firstly and the cameras are triggered simultaneously to capture a pair of images of the speckles. The stereo image pair is used to generate the initial dept h map. Then the four LEDs are lit one by one, and the cameras are triggered to capture iNages under the illumination of each LED. Ii these inages, the images of the left camera are used for photometric computation 2.2 Near-light photometric stereo We use the near point light source assumption in our image formation model [11]. In the j-th image, the light vector lii from the surface point xi to the ght sourcc s, is written as With the near light source assumption, intensity observation O: is computed with accounting the inverse squarc law as E A Self-Calibrated Photo-Geometric Depth Camera 5 < surface Light Fig,3 Near-light photometric stereo model Where, E; is the light source intensity at a unit distance, Pi is surface lbedo, and ni is the surface normal vector(Fig. 3) Once we know the light source parameters, we can estimate the normal vector I: anld the albedo Pi according to Ey.(2) froI at least three observa tiOns In order to balance the efficiency and the quality, we use four point light sources in our setup 2.3 Reconstruction pipeline We use the stereo camera calibration method in [17 to calibrate our stere o cameras. The light source will be sclf-calibrated using the mcthod to bo discussed in section 2 Given the calibrated cameras and lightings, the 3D reconstruction pipeline is illustrated in Fig. 4. Firstly, the stereo Latching Inlethiod is applied to the speckle image pair to generate the initial depth map Then, the photometric computation is applied to the four images under the illumination of the four LEDs rcspcctivcly to gcncratc the surfacc normal vector ficld. Finally, thc ob tained initial depth map and the normal vector field are integrated to generate the higher quality depth map 2.4 Initial dcpth generation The semi-global matching(SGM) method[18] performs an energy minimiza tion using dynamic programming on multiple lD paths. The energy function consists of three terIns: d data terIll for photo-consistency, a sInloothness terI for slanted surfaces that change the disparity slightly(parameter P1), and a smoothness term for depth discontinuities(parameter P2). Due to that SGM has a good balance in efficiency and accuracy we use this method to estimate the initial depth map. 6 Liang Xie et al Compute the initial depth map using stereo match lhod Speckle image pair Optimized depth map Initial depth man Four images under illumination of leds Normal vector field he normal field using Ps method Fig. 4 The pipeline of the reconstructi 2.5 Point light source self-calibration This scction proposcd a new calibration mcthod for point light sourcc includin the geometric parameters and the light intensity. With the self-calibration method, our system does not rely on the fixed calibration such as mirror spheres[14[15[16, which makes the system more flexible and practical To estimate Ei and si, we use the system described in Section l to capture five image pairs of the target according to the timing diagram shown in Fi 2. Our light source calibration firstly makes a distant lighting assumption and estimate a rough albedo map. Then, an iterative manner optimization is applied to estimate the parameters of the near light sources. The calibration algorithm is summarized as follows Algorithm 1: Point light, source calibration (1)Initialization Rough depth map generation. Each stereo pair is rectified to obtain a row-aligned epipolar geometry. The stereo Latching Inethod described iI Section 2.4 o the speckle image pair to generate the initial depth Do of the target. A bilateral filter [19 glied to the to obtain a discontinuity preserved depth map with reduccd noisc Do )No (Do(u)-Do(ql)Do(q)( 3) ∈b(u) Where, No=exp(-t20-), nb(u) denotes the neighborhood of the pixel u,and Wp is a normalizing constant A Self-Calibrated Photo-Geometric Depth Camera Initial position estimation. According to the mounting position of the LEDs relative to the reference camera we use the left calera as the reference camera), we can estimate an initial value Si, o for Sj Initial albedo estimation. Wc follow the automatic calibration mcthod 7 with distant, lighting assumption to estimate the initial al bedo map p We first robust y estimate a rank-3 approximation of the observed brightness matrix using an iterative reweighting method, and then factorize this rank reduced brightness matrix into the corresponding lighting, albedo and surface IIOrInlal coInponents Initial intcnsity estimation. With the depth map dn and the camera parameters, the point cloud xi of the target surface can be generated. Fur thermore, the surface normal vectors ni can be estimated with the point cloud 20. Up to nOw, using Fq.(2), we can estimate the initial value of F, with the linear least square method O E i=i pi; Where, N is the number of the surface point (2) Position and intensity refinement q(2)is a typical non-linear least squares problem. With the estimated initial values, we use the Levenberg-Marquardt(LM)algorithm 21 to op timizc Ei and s, with the albedo map and the normal ficld fixcd. The cost function is defined as: cost(E, s) 4N ∑∑(n1-EA Where,E={E},S={s},j=1,2.3,4. (3 Updating albedo map, surface points and normal With the optimized Fi and s,, we use the near lighting model(Eq(2)) to re-compute the albedo map and the normal field. By combining the roug depth map and the normal field with the method in 9: the higher quality depth map can be acquired. Note that, the normal field used for the followin optimization is obtained from the optimized depth map, rather than the PS mcthod (4)Iterative optimization Jump to Step(2) until convergence or the iteration times reaching the predefined IllaxiInuIl nunber pth normal fusion To estimate the optimal depth by combining the normal vector field by the PS method and the rough depth map by the stereo matching, we can form Liang Xie et al Fig. 5 Prototype of the proposed photo-geometric depth camera a linear system of equations as 9 to refine the quality of the reconstructed surtace AI 入D D (6) Where d is the refined depth Imap, V is a laplacian operator, I is all identity matrix and A is a weighting parameter controlling the contribution of sparse, it can be efficiently solved usIp ox ny. y ie& for each normalneN* pth constraint aN is the stacks of -one-a While it forms a la Lations. because the left matrix ng existing sparse linear solvers(e CHOLMOD 22) 3 Experimental results Fig 5 is the prototype of the proposed photo-geometric depth camera. The baseline length of the stereo system is 176.92 mm. The dimension of the depth camera is 260m.. 76mm x 150mm 3. 1 Qualitativc cvaluation Wc firstly usc thrcc targets including a malc, a fcmalc and a shoc to cvaluatc our depth cameras. The gray images of the three targets are shown in Fig. 7 Fig. 6 shows the convergence curve of the iterative optimization process for the target in Fig 4. The Y-axis is the root Ileal square error defined ill Eq.(4). After 10 iterations, the error converges We compare our method with the distant lighting model [7. Fig. 7(a) shows the estimated albedo maps, and Fig. 7(b)shows the estimated normal vector fields From Fig. 7(a) we can know that the albedo of the eyebrows of A Self-Calibrated Photo-Geometric Depth Camera 8 teration times Fig. 6 Convergence curve of the iterative optinization of the light source paraineters for the dataset shown in Fig. 4. The error is defined in Equation (5) the two persons is relatively low and the albedo of the words on clothes of the Inale is relatively high. Our results correctly reflect these facts. However, the method in 7 cannot show these. The albedo bias of 7 is also severe for the shoc. Furthermore, the estimated surfacc normal vectors in facc regions of tho method in 7 are severely biased. These results show that the auto-calibration method in Subsection 2.5 improves the quality of the estimated albedos and normaIs great ly Fig.8 shows the reconstruction results of three targets. Fig 8(a) shows the results of the Imale shown ill Fig. 7(a), Fig 8(b) shows the reconstructiOn result of a female, and Fig. 8(c)shows the results of a shoe. The left is the result using the speckle images, and the right is the result after combining the initial depth map by the stereo matching and the normal vector field by the PS mcthod. Thesc rcsults show that the reconstruction quality can bc improved remarkably by combining the near-light ps method, in which the para meters of the point light sources are calibrated automatically using our calibration method described in Section 2.5 We also conpare our depth camera witl Kinect, a popular depth camera Fig. 9(a) shows the reconstruction results of a 30 cm tall David sculpture, and Fig 9(b) shows the results of our depth camera. The volume voxels resolution of KincctFusion is sct to 512. Wc can scc that thc rcsult of our dopth camera has higher quality in reconstruction details 3.2 Quantitativc cvaluation Fig. 10 shows the quantitative evaluation results, where the result of a com- mercial phase-shifting system with nominal accuracy of 0.025 mm is treated 10 Liang Xie et al RYI∥ RYT∥ Fig. 7 Comparisons of the estimated albedo naps and the normal vector fields. (a) The gray mages of the three targets and the results of the estimated albedo maps.(b)Results of the estimated normal vector fields. In (a), the first column shows the gray images, the second column shows the results of [7, and the last column shows our results. In (b), the left shows the results of 7), and the right shows ours


关注 私信 TA的资源