Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 7-Day Trial for You or Your Team.

Learn More →

Indoor simultaneous localization and mapping based on fringe projection profilometry

Indoor simultaneous localization and mapping based on fringe projection profilometry Simultaneous Localization and Mapping (SLAM) plays an important role in outdoor and indoor applications ranging from autonomous driving to indoor robotics. Outdoor SLAM has been widely used with the assistance of LiDAR or GPS. For indoor applications, the LiDAR technique does not satisfy the accuracy requirement and the GPS signals will be lost. An accurate and ecient scene sensing technique is required for indoor SLAM. As the most promising 3D sensing technique, the opportunities for indoor SLAM with fringe projection pro lometry (FPP) systems are obvious, but methods to date have not fully leveraged the accuracy and speed of sensing that such systems o er. In this paper, we propose a novel FPP-based indoor SLAM method based on the coordinate transformation relationship of FPP, where the 2D-to-3D descriptor-assisted is used for mapping and localization. The correspondences generated by matching descriptors are used for fast and accurate mapping, and the transform estimation between the 2D and 3D descriptors is used to localize the sensor. The provided experimental results demonstrate that the proposed indoor SLAM can achieve the localization and mapping accuracy around one millimeter. 1. Introduction Simultaneous Localization and Mapping (SLAM) is of great importance in 3D computer vision with many applications in autonomous driving [1, 2], indoor robotics [3, 4], building surveying and mapping [5, 6], etc. In general, SLAM can be divided into two categories of outdoor [7] and indoor SLAM [8]. Outdoor SLAM applications such as autonomous driving and drone cruising work in relatively large scenes, and accordingly only decimeter (dm)- [2] or even meter (m)-level [7] SLAM accuracy is required. In contrast, indoor SLAM applications such as indoor robotics and building surveying [5] usually work in narrow scenes including small scattered objects, and accordingly the higher SLAM accuracy is necessary such as millimeter (mm)-level, which is still unachievable for existing inddor SLAM technologies. 10 A typical SLAM system includes the front-end sensors that perceive surrounding the unknown scene data and the back-end algorithms for building a map of the scene and determining the sensors' location within this map [9]. The performance of SLAM usually relies on the employed sensors, i.e., the data perceived from the Corresponding author Email addresses: [email protected] (Dongliang Zheng), [email protected] (Jing Han) Preprint submitted to arXiv April 26, 2022 arXiv:2204.11020v1 [cs.RO] 23 Apr 2022 environment. Recently, the development of LiDAR and the assistance of GPS signals enables outdoor SLAM applications successfully solved [10, 11]. However, indoor SLAM cannot receive external GPS signals, and its carrier is usually a small robot with low cost [12]. Therefore, it is critical to seek a low-cost, small-volume, and high-accuracy sensor for indoor SLAM. In SLAM, the 3D scenes are perceived by using the LiDAR [13], stereo vision [14] or time of ight (ToF) [15] techniques. The LiDAR has been widely used in outdoor SLAM for autonomous driving due to its advantages of large working distance and high robustness. However, indoor SLAM using LiDAR is a challenging task due 20 to the sparse nature of the obtained 3D data and computational complexities, which is dicult to provide real-time dense maps for indoor service robots [16]. Alternatively, the stereo vision uses two cameras to simulate the human eye according to the principle of bionics and uses images collected from di erent angles to reconstruct 3D data. However, the shortcomings of stereo vision in terms of accuracy, stability, and eld of view limit its application [17, 18]. The ToF technique obtains target depth by probing the round-trip time of light emitted and received by each pixel on the imaging chip, which has been widely used in commercial RGB-D sensors (e.g., Microsoft Kinect) [15]. However, it has defects such as a dicult trade-o between the miniaturization and high pixel resolution of the imaging chip, large measurement errors in the near range, and slow measurement speed. As aforementioned, the exist 3D sensors used in SLAM do not satisfy the requirements of indoor millimeter (mm)-level SLAM, i.e., localizing the sensor and the interaction with the 30 indoor scene. Optical metrology has been widely used for accurate and exible 3D perceiving. Among plenty of state- of-art techniques, fringe projection pro lometry (FPP) can achieve high-accuracy (i.e. micrometer-level) and high-speed (i.e. thousands of fps) indoor scene perceiving because the sub-pixel accurate camera-projector correspondences and binary defocus technology [19, 20]. Compared with other techniques, FPP also has the inherent advantages of simple hardware con guration, exibility in implementation, and dense point clouds acquisition [21]. It is appealing to utilize the FPP sensor to achieve indoor SLAM. However, there are two challenges for the application of FPP sensor to indoor SLAM. First, a fast and robust multi-views registration is required for FPP to sense and map a large scene. Second, estimating camera localization based on FPP coordinate system and subsequent localization optimization is still in the blank. 40 In order to tackle these challenges, in this paper, we propose a FPP-based indoor SLAM method. First, the FPP technique is introduced to perceive the 3D indoor scenes. Based on the perceived data of FPP, the correspond algorithm is proposed for millimeter (mm)-level indoor SLAM, which utilizes the 2D-to-3D descriptor and the coordinate transformation relationship of FPP. The mapping is achieved by solving the transformation matrix based on the correspondences generated by matching descriptors. The localization of sensor is achieved by transformation estimation between the 2D and 3D descriptors. With our method, users or robots can simply obtain accurate indoor maps and immediate feedback during perceiving. We also show both qualitative and quantitative results relating to various aspects of our FPP-based indoor method. The rest of this paper is organized as follows. Section 2 presents the principle of the proposed method. Section 3 provides experiments. Section 4 concludes this paper. 2 50 2. Principle The owchart of the proposed FPP-based indoor SLAM method is provided in Fig. 1, which includes the front-end using FPP technique to perceive the unknown scene, and the correspond back-end algorithm using the 2D-to-3D descriptor and the coordinate transformation relationship of FPP to estimate of the pose of sensors and construct the map of indoor scene. Speci cally, for the back-end, the local mapping module is performed to generate local maps by minimizing errors between the transformed and original point clouds. The corresponding location and the pose of the sensor can be calculated by utilizing the coordinate-map given by FPP. The optimizer then performs localization optimization to eliminate the cumulative errors and revise local mapping. Finally, the multi-view point clouds are aligned in global maps. Figure 1: Overall work ow of the proposed FPP-based SLAM method. 2.1. The front-end: FPP technique 60 The procedure of the FPP technique is provided in Fig. 2. A typical FPP system usually consists of a projector and a camera. The former is used to project coded fringe patterns onto the measured object, and the latter is used to synchronously capture the height-modulated fringe patterns. The projector can be seen as an inverse camera and the patterns help to establish distinct and accurate camera-projector correspondences regardless of whether the surface is textured or not. The desired phase can be calculated from these captured fringe patterns by using the transform-based [22] or phase-shifting algorithms [23]. The calculated phase is always wrapped in a range of (; ] , and a phase unwrapping process is necessary to obtain the absolute phase [20]. The image correspondence between the projector and the camera can be established from the absolute phase [24]. The 3-D shape can be reconstructed from the image correspondence by using the triangulation method, when combining the image correspondence with system parameters calibrated before [25]. 70 As aforementioned, both the system calibration and 3-D reconstruction rely on the retrieved phase, i.e., the wrapped phase and the absolute phase. The phase-shifting algorithm is preferred, and the captured sinusoidal fringe patterns can be described by c c c c c c c c I (u ; v ) = a(u ; v ) + b(u ; v )cos['(u ; v )  ]; n = 1; 2; :::N; (1) n n c c where (u ; v ) denotes the pixel coordinate, N denotes phase steps,  =2(n 1)=N denotes the phase shift 3 Figure 2: Schematic diagram of FPP. amount, a, b and ' denote the background, fringe amplitude and desire phase, respecitively. The desire phase can be calculated by using a least-squares algorithm N c c [I (u ; v )sin( )] n n c c n=1 '(u ; v ) = arctan : (2) c c [I (u ; v )cos( )] n n n=1 To obtain the correct absolute phase, the Gray code-based phase unwrapping method is preferred due to its high robustness [26], and a set of Gray code-based patterns is used to determine the fringe order of each fringe period. The absolute phase can be obtained by [27] c c c c c c (u ; v ) = '(u ; v ) + K (u ; v ) 2; (3) where K denotes the determined fringe order. 80 In this paper, both the projector and the camera can be regarded as a classical hole model and the inverse- camera-based method is applied to reconstruct the 3D points from the absolute phase. The pinhole camera w w w model maps 3D points in the world coordinate frame (x ; y ; z ) to 2D pixel coordinates (u; v) . This gives the equations 2 3 2 3 2 3 w 2 3 w x x c 6 7 c c c c 6 7 u a a a a 6 7 11 12 13 14 6 7 6 7 w 6 7 w 6 y 7 6 y 7 6 7 6 7 c c c 6 7 c c c c 6 7 s = A = ; (4) 6 v 7 6 a a a a 7 6 7 21 22 23 24 6 7 w w 4 5 4 5 6 z 7 6 z 7 c c c c 4 5 4 5 1 a a a a 31 32 33 34 1 1 2 3 2 3 2 3 2 3 x x p p p p p 6 7 6 7 u a a a a 6 7 6 7 11 12 13 14 6 7 6 7 w 6 7 6 7 y y 6 7 6 7 p p p p p p p 6 7 6 7 s 6 7 = A = 6 7 ; (5) v a a a a 6 7 21 22 23 24 6 7 4 5 4 5 6 7 6 7 z z p p p p 4 5 4 5 1 a a a a 31 32 33 34 1 1 where superscript c=p represents camera/projector and s denotes the scaling factor. A of 3 4 matrix is 4 the product of the intrinsic and extrinsic matrix, which is expressed as 2 32 3 c c c c c c f 0 u r r r t x 0 11 12 13 1 6 76 7 6 76 7 c c c c c c c c c c A = I  [R j T ] = ; (6) 6 0 f v 76 r r r t 7 y 0 21 22 23 2 4 54 5 c c c c 0 0 1 r r r t 31 32 33 3 2 32 3 p p p p p f 0 u r r r t x 0 11 12 13 1 6 76 7 6 76 7 p p p p p p p p p A = I  [R jT ] = ; (7) 6 0 f v 76 r r r t 7 y 0 21 22 23 2 4 54 5 p p p p 0 0 1 r r r t 31 32 33 3 I is the 3 3 intrinsic parameter, f and f are the projector focal, and (u ; v ) is the principal coordinate. x y 0 0 [RjT ] is known as the extrinsic matrix. R denotes the 3  3 rotational matric and T 2 R denotes the translation vector that describes the orientation and position of the world frame. When the FPP system is calibrated by capturing patterns on a planar surface at several positions, A 90 and A become the known values. Next, when the corresponding point between the camera and projector is matched, the 3D coordinate can be solved. In the FPP system, the absolute phase can satisfy the matching c c c requirement. For example, the value of the transverse absolute phase of the point p (u ; v ) on the camera is c c c p p (u ; v ) , and u of the according to the point p on the projector can be matched by Eq. (8). c c c (u ; v ) (8) u = ; c c c p where  (u ; v ) represents the transverse absolute phase, and  is the fringe width. If v is required, the p w w w vertical absolute phase can be used. When u is matched, the 3D coordinate (x ; y ; z ) can be obtained by the following equation. 2 3 2 3 2 3 w c c c c c c c c c c c c x a u a a u a a u a u a a 11 31 12 32 13 33 34 14 6 7 6 7 6 7 6 7 6 7 6 7 w c c c c c c c c c c c c (9) = ; 6 y 7 6 a v a a v a a v a 7 6 v a a 7 21 31 22 32 23 33 34 24 4 5 4 5 4 5 w c p c c p c c p c p c c z a u a a u a a u a u a a 31 31 32 31 33 31 34 24 Finally, the 2D texture images are rendered point-to-point onto the point clouds. 2.2. 2.2 The back-end: 2.2.1. Basics of mapping 100 After obtaining high-quality data, a coarse-to- ne pairwise registration algorithm based on the 2D-to- 3D descriptor is proposed for fast and accurate mapping, which is achieved by solving the transformation matrix based on the correspondences generated by matching descriptors. Speci cally, we apply the coordinate transformation to convert the ane transformation between 2D feature points to a 3D transformation to provide an initial prior for the corresponding point clouds registration. Then, in order to improve the eciency of registration, 3D feature points are used instead of 3D point clouds to perform registration. The entire process of mapping method is shown in Fig. 3. Step 1: extract 2D matching points with SURF algorithm and obtain 2D transformation matrix; Step 2: convert the 2D transformation matrix into the 3D transformation matrix 5 according to the coordinate transformation, and use it as the initial prior for registration; Step 3: extract the corresponding 3D points according to the 2D feature points; Step 4: perform ICP on the 3D points to obtain 110 the transformation matrix combined with the initial prior; Step 5: apply the transformation matrix to perform registration on the point clouds; Step 6: return to Step 1 and repeat the above processes. Figure 3: Overall work ow of the proposed FPP-based SLAM method. De ne two adjacent 3D frames as Frame 1 and Frame 2, which have corresponding 2D texture images 1 1 2 2 1 1 1 2 2 2 I : (u ; v ) and I : (u ; v ), and 3D data p1 : (x ; y ; z ) and p2 : (x ; y ; z ) . First, the key points and 1 2 c c c c w w w w w w are obtained and matched by Speeded Up Robust Features (SURF) [28] algorithm. In order to improve the registration eciency, we use paired 3D feature points instead of complete 3D point clouds for registration. By using Eq. 9, we record sets of 3D matching points corresponding to the found 2D ones as and . The corresponding 3D points can be used to calculate the rotation matrix R and initial translation vector T of the coordinate transformation. The calculation can be formulated as X 2 0 0 min P RP T ; (10) 1 2 i=1 The obtained transformation of 2D matching points can be transformed to 3D transformation to initialize 0 0 120 the iteration. After P and RP are roughly registered, they almost have been registered well. To improve 1 2 the registration accuracy, the result is further re ned by using the ICP algorithm [29]. Then, for stitching a sequence of point clouds, the next point cloud data can be as the target data and the target data registered in the previous step can be as the source data. Repeating the above two frame point clouds registration process to achieve a series of point cloud registration. 6 2.2.2. Basics of localization Live camera localization involves estimating the current camera position and pose for each new point data, which is achieved by transformation estimation between the 2D and 3D descriptors. The 3D-2D points matching is performed to estimate the pose of the sensor using the 3D date of the perceived indoor scene. Speci cally, we nd the correspondences between camera and world coordinate points, and calculate the camera 130 pose through optimization. The schematic of the camera localization is shown in Fig. 4, the transformation relationship between the world coordinate system O and the camera coordinate system O can be simply described as w c Figure 4: Overall work ow of the proposed FPP-based SLAM method. c c (11) O = R O + T : c w Thus, the formula can be transformed into Eq. 12. c 1 c 1 c (12) O = (R ) O (R ) T : w c Because the camera localization is for the optical center coordinates, and the optical center coordinates are c T c 1 the origin relative to the camera coordinates. Considering is an orthogonal matrix, so (R ) = (R ) . c 1 c c T c (13) O = (R ) T = (R ) T ; 7 Thus, if we can calculate the rotation matrix and translation matrix of the current captured point data, the current camera position and pose can be easily obtained. w w w For the FPP imaging system, the coordinates of the 3D point cloud in real space are (x ; y ; z ), and w w w the homogeneous coordinates can be expressed as (x ; y ; z ; 1). The corresponding projected points in the c c c c 140 camera coordinate are (u ; v ), and the homogeneous coordinates can be expressed as (u ; v ; 1). The camera's internal parameter I has been obtained through device calibration. By using Eq.8 and Eq.9, we can derive the following form. c c c w c w c w c s u = a x + a y + a z + a 11 12 13 14 c c c w c w c w c ; (14) s v = a x + a y + a z + a 21 22 23 24 c c w c w c w c s = a x + a y + a z + a 31 32 33 34 By eliminating s , Eq. 14 can be simpli ed as c w c w c w c c w c c w c c w c c c a x + a y + a z + a a x u a y u a z u a u = 0 11 12 13 14 31 32 33 34 ; (15) c w c w c w c c w c c w c c w c c c a x + a y + a z + a a x v a y v a z v a v = 0 21 22 23 24 31 32 33 34 According to the above formula, each set of 3D-2D matching points corresponds to two equations, there are a total of 12 unknowns, and at least 6 sets of matching points are required. Assuming there are N sets of matching points, Eq.15 can be rewritten as 2 3 6 7 6 7 6 7 6 7 6 7 6 7 6 7 6 7 2 3 6 7 w w w c w c w c w c 6 7 x y z 1 0 0 0 0 u x u y u z u 1 1 1 1 1 1 1 1 1 1 6 7 6 7 c 6 7 6 7 w w w c w c w c w c 6 7 6 7 0 0 0 0 x y z 1 v x v y v z v 1 1 1 1 1 1 1 1 1 1 6 7 6 7 c 6 7 6 7 6 7 6 7 = 0: (16) ::: ::: 6 7 6 7 c 6 7 6 7 w w w c w c w c w c 6 7 6 7 x y z 1 0 0 0 0 u x u y u z u N N N N N N N N N N 6 7 4 5 c 6 7 w w w c w c w c w c 6 7 0 0 0 0 x y z 1 v x v y v z v N N N N N N N N N N 6 7 6 7 6 7 6 7 6 7 6 7 6 7 6 a 7 4 5 The above formula can be written as ; (17) AF = 1 When N=6, the systems of linear equations can be solved directly. 8 When N6, we can obtain the least-squares solution under the constraint of jFj = 1. And then through 150 SVD decomposition, the last column of the V matrix is the solution. (18) F = UDV : By substituting Eq. 9 into Eq. 18, we can obtain Eq. 19. c c c c (19) F = [I R I T ] : c c Thus, the rotation matrix R and translation matrix T can be calculated as follows. 2 3 c c c a a a 11 12 13 6 7 6 7 c 1 c c c R = I 6 7; (20) a a a 21 22 23 4 5 c c c a a a 31 32 33 2 3 6 7 6 7 c 1 T = I ; (21) 6 7 4 5 After the rotation matrix and translation matrix are calculated, we completed the live camera localization for the current point data. 3. Experiments To investigate the performance of the proposed method, we conducted experiments on metrically consistent and indoor scenes. All experiments were implemented on a PC with AMD Ryzen 5 5600H (3GHz) CPU, and a packaged FPP sensor. The built FPP sensor is shown in Fig. 5, which contains a Basler acA1920-155uc camera (1920  1200 resolution), a LighterCraf ter 4500 (912  1140 resolution) and an interface expansion 160 board. The interface expansion board is used to realize the synchronous triggering of the projector and the camera. The imaging speed of the sensor is 300 fps, the single-view measurement accuracy is 0.067 mm and the global mapping accuracy is around one millimeter. Figure 5: Labeled diagram of FPP sensor. 9 Firstly, a doll was measured to investigate the ability of metrically consistent models from trajectories containing loop closures. As shown in Fig. 6, the FPP system was placed in a xed location and the doll was placed on a turntable. Then, the turntable was spun through a full rotation to capture point clouds from di erent perspectives of the doll for registration, and the trajectory localized by the corresponding camera should also be a circle and return to the origin, which allows us to easily evaluate the quality of camera localization. The main motivation for our experiments is to investigate the convergence of the mapping and localization schemes. As shown in the enlarged image of Fig. 6, the captured single-view point clouds were 170 well registered, which demonstrates the performance of our mapping method. Then, the resulting sensor trajectories are given at the bottom of Fig. 6. Since the FPP sensor is xed, the object is placed on the turntable, which can be equivalent to the sensor rotating around the object. In this experiment, the turntable was rotated 3 times, 13 times for each turn. The loop closures e ect of localization trajectory performs well. The camera forms a perfectly circular trajectory around the doll and can return to the original location after one rotation, which demonstrates the performance of our localization method. Figure 6: Experimental results of the doll. Finally, to investigate the performance of our method for real scenes, we conducted the mapping and localization experiment in the corridor. The system setup of the indoor scene experiment is shown in Fig. 7, which includes a computer, our FPP sensor, a battery and a mobile cart. The FPP sensor is located in the mobile cart to scan the indoor scene. The results of scenes using our system are shown in Fig. 8. To 180 generate this map of the indoor scene, we captured over 100 frames of single-view point clouds. The overlap area of adjacent point clouds accounts for one-third of the total area. As shown at the top of Fig. 8, the 10 generated map point cloud is dense and has high-quality local geometric information. Then, the map with texture information is shown at the top of Fig. 8, which achieves perfect registration of multi-frame point clouds and avoids interference of cumulative errors. This also demonstrates that our global pose alignment strategy scales well to large spatial extents and long sequences. Figure 7: The system setup of indoor scene experiment. Figure 8: Experimental results of the indoor scene. 4. Conclusion The availability of accurate 3-D metrology technologies such as FPP has the potential to revolutionise the elds of indoor localization and mapping, but methods to date have not fully leveraged the accuracy and speed of sensing that such techniques o er. This paper presents a millimeter (mm)-level indoor FPP-based 190 SLAM method, which includes the front-end using FPP technique to perceive the unknown scene, and the correspond back-end algorithm using the 2D-to-3D descriptor and the coordinate transformation relationship of FPP to estimate of the pose of sensors and construct the map of indoor scene. The capabilities of the 11 proposed method have been demonstrated on room sized SLAM. We believe that this paper will open a new door for the application of FPP in indoor SLAM. Data Availability The data that support the ndings of this study are available from the corresponding author upon reason- able request. Declaration of competing interest The authors declare that they have no known competing nancial interests or personal relationships that 200 could have appeared to in uence the work reported in this paper. References [1] C. Cadena, L. Carlone, H. Carrillo, Y. Latif, D. Scaramuzza, J. Neira, I. Reid, J. J. Leonard, Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age, IEEE Transactions on robotics 32 (6) (2016) 1309{1332. [2] G. Bresson, Z. Alsayed, L. Yu, S. Glaser, Simultaneous localization and mapping: A survey of current trends in autonomous driving, IEEE Transactions on Intelligent Vehicles 2 (3) (2017) 194{220. [3] R. Azzam, T. Taha, S. Huang, Y. Zweiri, Feature-based visual simultaneous localization and mapping: A survey, SN Applied Sciences 2 (2) (2020) 1{24. [4] A. Hornung, K. M. Wurm, M. Bennewitz, Humanoid robot localization in complex indoor environments, 210 in: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, 2010, pp. 1690{ [5] M. Zeybek, Indoor mapping and positioning applications of hand-held lidar simultaneous localization and mapping (slam) systems, Turkey Lidar Journal 3 (1) (2021) 7{16. [6] R. Otero, S. Laguela,  I. Garrido, P. Arias, Mobile indoor mapping technologies: A review, Automation in Construction 120 (2020) 103399. [7] P. Newman, D. Cole, K. Ho, Outdoor slam using visual appearance and laser ranging, in: Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006., IEEE, 2006, pp. 1180{1187. [8] S. Zlatanova, G. Sithole, M. Nakagawa, Q. Zhu, Problems in indoor mapping and modelling, Acquisition 220 and Modelling of Indoor and Enclosed Environments 2013, Cape Town, South Africa, 11-13 December 2013, ISPRS Archives Volume XL-4/W4, 2013. 12 [9] H. Durrant-Whyte, T. Bailey, Simultaneous localization and mapping: part i, IEEE robotics & automation magazine 13 (2) (2006) 99{110. [10] D. Chen, G. X. Gao, Probabilistic graphical fusion of lidar, gps, and 3d building maps for urban uav navigation, NAVIGATION, Journal of the Institute of Navigation 66 (1) (2019) 151{168. [11] P. Ellis, B. Griscom, W. Walker, F. Gon calves, T. Cormier, Mapping selective logging impacts in borneo with gps and airborne lidar, Forest Ecology and Management 365 (2016) 184{196. [12] S. Thrun, Learning metric-topological maps for indoor mobile robot navigation, Arti cial Intelligence 99 (1) (1998) 21{71. 230 [13] Z. Ren, L. Wang, L. Bi, Robust gicp-based 3d lidar slam for underground mining environment, Sensors 19 (13) (2019) 2915. [14] R. Mur-Artal, J. D. Tard os, Visual-inertial monocular slam with map reuse, IEEE Robotics and Automa- tion Letters 2 (2) (2017) 796{803. [15] R. A. Newcombe, S. Izadi, O. Hilliges, D. Molyneaux, D. Kim, A. J. Davison, P. Kohi, J. Shotton, S. Hodges, A. Fitzgibbon, Kinectfusion: Real-time dense surface mapping and tracking, in: 2011 10th IEEE international symposium on mixed and augmented reality, IEEE, 2011, pp. 127{136. [16] J. Li, R. Zhong, Q. Hu, M. Ai, Feature-based laser scan matching and its application for indoor mapping, Sensors 16 (8) (2016) 1265. [17] E. Eade, T. Drummond, Scalable monocular slam, in: 2006 IEEE Computer Society Conference on 240 Computer Vision and Pattern Recognition (CVPR'06), Vol. 1, IEEE, 2006, pp. 469{476. [18] Y. M. Mustafah, A. W. Azman, F. Akbar, Indoor uav positioning using stereo vision sensor, Procedia Engineering 41 (2012) 575{579. [19] S. Zhang, Recent progresses on real-time 3d shape measurement using digital fringe projection techniques, Optics and lasers in engineering 48 (2) (2010) 149{158. [20] D. Zheng, Q. Kemao, J. Han, J. Wang, H. Yu, L. Bai, High-speed phase-shifting pro lometry under uorescent light, Optics and Lasers in Engineering 128 (2020) 106033. [21] S. S. Gorthi, P. Rastogi, Fringe projection techniques: whither we are?, Optics and lasers in engineering 48 (ARTICLE) (2010) 133{140. [22] X. Su, W. Chen, Fourier transform pro lometry:: a review, Optics and lasers in Engineering 35 (5) (2001) 250 263{284. [23] L. Huang, Q. Kemao, B. Pan, A. K. Asundi, Comparison of fourier transform, windowed fourier transform, and wavelet transform methods for phase extraction from a single fringe pattern in fringe projection pro lometry, Optics and Lasers in Engineering 48 (2) (2010) 141{148. 13 [24] C. Zuo, L. Huang, M. Zhang, Q. Chen, A. Asundi, Temporal phase unwrapping algorithms for fringe projection pro lometry: A comparative review, Optics and lasers in engineering 85 (2016) 84{103. [25] S. Zhang, P. S. Huang, Novel method for structured light system calibration, Optical Engineering 45 (8) (2006) 083601. [26] S. Zhang, Absolute phase retrieval methods for digital fringe projection pro lometry: A review, Optics and Lasers in Engineering 107 (2018) 28{37. 260 [27] D. Zheng, F. Da, Q. Kemao, H. S. Seah, Phase-shifting pro lometry combined with gray-code patterns projection: unwrapping error removal by an adaptive median lter, Optics express 25 (5) (2017) 4700{ [28] B. Triggs, P. F. McLauchlan, R. I. Hartley, A. W. Fitzgibbon, Bundle adjustment|a modern synthesis, in: International workshop on vision algorithms, Springer, 1999, pp. 298{372. [29] S. Rusinkiewicz, M. Levoy, Ecient variants of the icp algorithm, in: Proceedings third international conference on 3-D digital imaging and modeling, IEEE, 2001, pp. 145{152. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Computing Research Repository arXiv (Cornell University)

Indoor simultaneous localization and mapping based on fringe projection profilometry

Download PDF
14 pages

Loading next page...
 
/lp/arxiv-cornell-university/indoor-simultaneous-localization-and-mapping-based-on-fringe-h0N3N76Ca1

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

eISSN
ARCH-3344
DOI
10.1364/oe.483667
Publisher site
See Article on Publisher Site

Abstract

Simultaneous Localization and Mapping (SLAM) plays an important role in outdoor and indoor applications ranging from autonomous driving to indoor robotics. Outdoor SLAM has been widely used with the assistance of LiDAR or GPS. For indoor applications, the LiDAR technique does not satisfy the accuracy requirement and the GPS signals will be lost. An accurate and ecient scene sensing technique is required for indoor SLAM. As the most promising 3D sensing technique, the opportunities for indoor SLAM with fringe projection pro lometry (FPP) systems are obvious, but methods to date have not fully leveraged the accuracy and speed of sensing that such systems o er. In this paper, we propose a novel FPP-based indoor SLAM method based on the coordinate transformation relationship of FPP, where the 2D-to-3D descriptor-assisted is used for mapping and localization. The correspondences generated by matching descriptors are used for fast and accurate mapping, and the transform estimation between the 2D and 3D descriptors is used to localize the sensor. The provided experimental results demonstrate that the proposed indoor SLAM can achieve the localization and mapping accuracy around one millimeter. 1. Introduction Simultaneous Localization and Mapping (SLAM) is of great importance in 3D computer vision with many applications in autonomous driving [1, 2], indoor robotics [3, 4], building surveying and mapping [5, 6], etc. In general, SLAM can be divided into two categories of outdoor [7] and indoor SLAM [8]. Outdoor SLAM applications such as autonomous driving and drone cruising work in relatively large scenes, and accordingly only decimeter (dm)- [2] or even meter (m)-level [7] SLAM accuracy is required. In contrast, indoor SLAM applications such as indoor robotics and building surveying [5] usually work in narrow scenes including small scattered objects, and accordingly the higher SLAM accuracy is necessary such as millimeter (mm)-level, which is still unachievable for existing inddor SLAM technologies. 10 A typical SLAM system includes the front-end sensors that perceive surrounding the unknown scene data and the back-end algorithms for building a map of the scene and determining the sensors' location within this map [9]. The performance of SLAM usually relies on the employed sensors, i.e., the data perceived from the Corresponding author Email addresses: [email protected] (Dongliang Zheng), [email protected] (Jing Han) Preprint submitted to arXiv April 26, 2022 arXiv:2204.11020v1 [cs.RO] 23 Apr 2022 environment. Recently, the development of LiDAR and the assistance of GPS signals enables outdoor SLAM applications successfully solved [10, 11]. However, indoor SLAM cannot receive external GPS signals, and its carrier is usually a small robot with low cost [12]. Therefore, it is critical to seek a low-cost, small-volume, and high-accuracy sensor for indoor SLAM. In SLAM, the 3D scenes are perceived by using the LiDAR [13], stereo vision [14] or time of ight (ToF) [15] techniques. The LiDAR has been widely used in outdoor SLAM for autonomous driving due to its advantages of large working distance and high robustness. However, indoor SLAM using LiDAR is a challenging task due 20 to the sparse nature of the obtained 3D data and computational complexities, which is dicult to provide real-time dense maps for indoor service robots [16]. Alternatively, the stereo vision uses two cameras to simulate the human eye according to the principle of bionics and uses images collected from di erent angles to reconstruct 3D data. However, the shortcomings of stereo vision in terms of accuracy, stability, and eld of view limit its application [17, 18]. The ToF technique obtains target depth by probing the round-trip time of light emitted and received by each pixel on the imaging chip, which has been widely used in commercial RGB-D sensors (e.g., Microsoft Kinect) [15]. However, it has defects such as a dicult trade-o between the miniaturization and high pixel resolution of the imaging chip, large measurement errors in the near range, and slow measurement speed. As aforementioned, the exist 3D sensors used in SLAM do not satisfy the requirements of indoor millimeter (mm)-level SLAM, i.e., localizing the sensor and the interaction with the 30 indoor scene. Optical metrology has been widely used for accurate and exible 3D perceiving. Among plenty of state- of-art techniques, fringe projection pro lometry (FPP) can achieve high-accuracy (i.e. micrometer-level) and high-speed (i.e. thousands of fps) indoor scene perceiving because the sub-pixel accurate camera-projector correspondences and binary defocus technology [19, 20]. Compared with other techniques, FPP also has the inherent advantages of simple hardware con guration, exibility in implementation, and dense point clouds acquisition [21]. It is appealing to utilize the FPP sensor to achieve indoor SLAM. However, there are two challenges for the application of FPP sensor to indoor SLAM. First, a fast and robust multi-views registration is required for FPP to sense and map a large scene. Second, estimating camera localization based on FPP coordinate system and subsequent localization optimization is still in the blank. 40 In order to tackle these challenges, in this paper, we propose a FPP-based indoor SLAM method. First, the FPP technique is introduced to perceive the 3D indoor scenes. Based on the perceived data of FPP, the correspond algorithm is proposed for millimeter (mm)-level indoor SLAM, which utilizes the 2D-to-3D descriptor and the coordinate transformation relationship of FPP. The mapping is achieved by solving the transformation matrix based on the correspondences generated by matching descriptors. The localization of sensor is achieved by transformation estimation between the 2D and 3D descriptors. With our method, users or robots can simply obtain accurate indoor maps and immediate feedback during perceiving. We also show both qualitative and quantitative results relating to various aspects of our FPP-based indoor method. The rest of this paper is organized as follows. Section 2 presents the principle of the proposed method. Section 3 provides experiments. Section 4 concludes this paper. 2 50 2. Principle The owchart of the proposed FPP-based indoor SLAM method is provided in Fig. 1, which includes the front-end using FPP technique to perceive the unknown scene, and the correspond back-end algorithm using the 2D-to-3D descriptor and the coordinate transformation relationship of FPP to estimate of the pose of sensors and construct the map of indoor scene. Speci cally, for the back-end, the local mapping module is performed to generate local maps by minimizing errors between the transformed and original point clouds. The corresponding location and the pose of the sensor can be calculated by utilizing the coordinate-map given by FPP. The optimizer then performs localization optimization to eliminate the cumulative errors and revise local mapping. Finally, the multi-view point clouds are aligned in global maps. Figure 1: Overall work ow of the proposed FPP-based SLAM method. 2.1. The front-end: FPP technique 60 The procedure of the FPP technique is provided in Fig. 2. A typical FPP system usually consists of a projector and a camera. The former is used to project coded fringe patterns onto the measured object, and the latter is used to synchronously capture the height-modulated fringe patterns. The projector can be seen as an inverse camera and the patterns help to establish distinct and accurate camera-projector correspondences regardless of whether the surface is textured or not. The desired phase can be calculated from these captured fringe patterns by using the transform-based [22] or phase-shifting algorithms [23]. The calculated phase is always wrapped in a range of (; ] , and a phase unwrapping process is necessary to obtain the absolute phase [20]. The image correspondence between the projector and the camera can be established from the absolute phase [24]. The 3-D shape can be reconstructed from the image correspondence by using the triangulation method, when combining the image correspondence with system parameters calibrated before [25]. 70 As aforementioned, both the system calibration and 3-D reconstruction rely on the retrieved phase, i.e., the wrapped phase and the absolute phase. The phase-shifting algorithm is preferred, and the captured sinusoidal fringe patterns can be described by c c c c c c c c I (u ; v ) = a(u ; v ) + b(u ; v )cos['(u ; v )  ]; n = 1; 2; :::N; (1) n n c c where (u ; v ) denotes the pixel coordinate, N denotes phase steps,  =2(n 1)=N denotes the phase shift 3 Figure 2: Schematic diagram of FPP. amount, a, b and ' denote the background, fringe amplitude and desire phase, respecitively. The desire phase can be calculated by using a least-squares algorithm N c c [I (u ; v )sin( )] n n c c n=1 '(u ; v ) = arctan : (2) c c [I (u ; v )cos( )] n n n=1 To obtain the correct absolute phase, the Gray code-based phase unwrapping method is preferred due to its high robustness [26], and a set of Gray code-based patterns is used to determine the fringe order of each fringe period. The absolute phase can be obtained by [27] c c c c c c (u ; v ) = '(u ; v ) + K (u ; v ) 2; (3) where K denotes the determined fringe order. 80 In this paper, both the projector and the camera can be regarded as a classical hole model and the inverse- camera-based method is applied to reconstruct the 3D points from the absolute phase. The pinhole camera w w w model maps 3D points in the world coordinate frame (x ; y ; z ) to 2D pixel coordinates (u; v) . This gives the equations 2 3 2 3 2 3 w 2 3 w x x c 6 7 c c c c 6 7 u a a a a 6 7 11 12 13 14 6 7 6 7 w 6 7 w 6 y 7 6 y 7 6 7 6 7 c c c 6 7 c c c c 6 7 s = A = ; (4) 6 v 7 6 a a a a 7 6 7 21 22 23 24 6 7 w w 4 5 4 5 6 z 7 6 z 7 c c c c 4 5 4 5 1 a a a a 31 32 33 34 1 1 2 3 2 3 2 3 2 3 x x p p p p p 6 7 6 7 u a a a a 6 7 6 7 11 12 13 14 6 7 6 7 w 6 7 6 7 y y 6 7 6 7 p p p p p p p 6 7 6 7 s 6 7 = A = 6 7 ; (5) v a a a a 6 7 21 22 23 24 6 7 4 5 4 5 6 7 6 7 z z p p p p 4 5 4 5 1 a a a a 31 32 33 34 1 1 where superscript c=p represents camera/projector and s denotes the scaling factor. A of 3 4 matrix is 4 the product of the intrinsic and extrinsic matrix, which is expressed as 2 32 3 c c c c c c f 0 u r r r t x 0 11 12 13 1 6 76 7 6 76 7 c c c c c c c c c c A = I  [R j T ] = ; (6) 6 0 f v 76 r r r t 7 y 0 21 22 23 2 4 54 5 c c c c 0 0 1 r r r t 31 32 33 3 2 32 3 p p p p p f 0 u r r r t x 0 11 12 13 1 6 76 7 6 76 7 p p p p p p p p p A = I  [R jT ] = ; (7) 6 0 f v 76 r r r t 7 y 0 21 22 23 2 4 54 5 p p p p 0 0 1 r r r t 31 32 33 3 I is the 3 3 intrinsic parameter, f and f are the projector focal, and (u ; v ) is the principal coordinate. x y 0 0 [RjT ] is known as the extrinsic matrix. R denotes the 3  3 rotational matric and T 2 R denotes the translation vector that describes the orientation and position of the world frame. When the FPP system is calibrated by capturing patterns on a planar surface at several positions, A 90 and A become the known values. Next, when the corresponding point between the camera and projector is matched, the 3D coordinate can be solved. In the FPP system, the absolute phase can satisfy the matching c c c requirement. For example, the value of the transverse absolute phase of the point p (u ; v ) on the camera is c c c p p (u ; v ) , and u of the according to the point p on the projector can be matched by Eq. (8). c c c (u ; v ) (8) u = ; c c c p where  (u ; v ) represents the transverse absolute phase, and  is the fringe width. If v is required, the p w w w vertical absolute phase can be used. When u is matched, the 3D coordinate (x ; y ; z ) can be obtained by the following equation. 2 3 2 3 2 3 w c c c c c c c c c c c c x a u a a u a a u a u a a 11 31 12 32 13 33 34 14 6 7 6 7 6 7 6 7 6 7 6 7 w c c c c c c c c c c c c (9) = ; 6 y 7 6 a v a a v a a v a 7 6 v a a 7 21 31 22 32 23 33 34 24 4 5 4 5 4 5 w c p c c p c c p c p c c z a u a a u a a u a u a a 31 31 32 31 33 31 34 24 Finally, the 2D texture images are rendered point-to-point onto the point clouds. 2.2. 2.2 The back-end: 2.2.1. Basics of mapping 100 After obtaining high-quality data, a coarse-to- ne pairwise registration algorithm based on the 2D-to- 3D descriptor is proposed for fast and accurate mapping, which is achieved by solving the transformation matrix based on the correspondences generated by matching descriptors. Speci cally, we apply the coordinate transformation to convert the ane transformation between 2D feature points to a 3D transformation to provide an initial prior for the corresponding point clouds registration. Then, in order to improve the eciency of registration, 3D feature points are used instead of 3D point clouds to perform registration. The entire process of mapping method is shown in Fig. 3. Step 1: extract 2D matching points with SURF algorithm and obtain 2D transformation matrix; Step 2: convert the 2D transformation matrix into the 3D transformation matrix 5 according to the coordinate transformation, and use it as the initial prior for registration; Step 3: extract the corresponding 3D points according to the 2D feature points; Step 4: perform ICP on the 3D points to obtain 110 the transformation matrix combined with the initial prior; Step 5: apply the transformation matrix to perform registration on the point clouds; Step 6: return to Step 1 and repeat the above processes. Figure 3: Overall work ow of the proposed FPP-based SLAM method. De ne two adjacent 3D frames as Frame 1 and Frame 2, which have corresponding 2D texture images 1 1 2 2 1 1 1 2 2 2 I : (u ; v ) and I : (u ; v ), and 3D data p1 : (x ; y ; z ) and p2 : (x ; y ; z ) . First, the key points and 1 2 c c c c w w w w w w are obtained and matched by Speeded Up Robust Features (SURF) [28] algorithm. In order to improve the registration eciency, we use paired 3D feature points instead of complete 3D point clouds for registration. By using Eq. 9, we record sets of 3D matching points corresponding to the found 2D ones as and . The corresponding 3D points can be used to calculate the rotation matrix R and initial translation vector T of the coordinate transformation. The calculation can be formulated as X 2 0 0 min P RP T ; (10) 1 2 i=1 The obtained transformation of 2D matching points can be transformed to 3D transformation to initialize 0 0 120 the iteration. After P and RP are roughly registered, they almost have been registered well. To improve 1 2 the registration accuracy, the result is further re ned by using the ICP algorithm [29]. Then, for stitching a sequence of point clouds, the next point cloud data can be as the target data and the target data registered in the previous step can be as the source data. Repeating the above two frame point clouds registration process to achieve a series of point cloud registration. 6 2.2.2. Basics of localization Live camera localization involves estimating the current camera position and pose for each new point data, which is achieved by transformation estimation between the 2D and 3D descriptors. The 3D-2D points matching is performed to estimate the pose of the sensor using the 3D date of the perceived indoor scene. Speci cally, we nd the correspondences between camera and world coordinate points, and calculate the camera 130 pose through optimization. The schematic of the camera localization is shown in Fig. 4, the transformation relationship between the world coordinate system O and the camera coordinate system O can be simply described as w c Figure 4: Overall work ow of the proposed FPP-based SLAM method. c c (11) O = R O + T : c w Thus, the formula can be transformed into Eq. 12. c 1 c 1 c (12) O = (R ) O (R ) T : w c Because the camera localization is for the optical center coordinates, and the optical center coordinates are c T c 1 the origin relative to the camera coordinates. Considering is an orthogonal matrix, so (R ) = (R ) . c 1 c c T c (13) O = (R ) T = (R ) T ; 7 Thus, if we can calculate the rotation matrix and translation matrix of the current captured point data, the current camera position and pose can be easily obtained. w w w For the FPP imaging system, the coordinates of the 3D point cloud in real space are (x ; y ; z ), and w w w the homogeneous coordinates can be expressed as (x ; y ; z ; 1). The corresponding projected points in the c c c c 140 camera coordinate are (u ; v ), and the homogeneous coordinates can be expressed as (u ; v ; 1). The camera's internal parameter I has been obtained through device calibration. By using Eq.8 and Eq.9, we can derive the following form. c c c w c w c w c s u = a x + a y + a z + a 11 12 13 14 c c c w c w c w c ; (14) s v = a x + a y + a z + a 21 22 23 24 c c w c w c w c s = a x + a y + a z + a 31 32 33 34 By eliminating s , Eq. 14 can be simpli ed as c w c w c w c c w c c w c c w c c c a x + a y + a z + a a x u a y u a z u a u = 0 11 12 13 14 31 32 33 34 ; (15) c w c w c w c c w c c w c c w c c c a x + a y + a z + a a x v a y v a z v a v = 0 21 22 23 24 31 32 33 34 According to the above formula, each set of 3D-2D matching points corresponds to two equations, there are a total of 12 unknowns, and at least 6 sets of matching points are required. Assuming there are N sets of matching points, Eq.15 can be rewritten as 2 3 6 7 6 7 6 7 6 7 6 7 6 7 6 7 6 7 2 3 6 7 w w w c w c w c w c 6 7 x y z 1 0 0 0 0 u x u y u z u 1 1 1 1 1 1 1 1 1 1 6 7 6 7 c 6 7 6 7 w w w c w c w c w c 6 7 6 7 0 0 0 0 x y z 1 v x v y v z v 1 1 1 1 1 1 1 1 1 1 6 7 6 7 c 6 7 6 7 6 7 6 7 = 0: (16) ::: ::: 6 7 6 7 c 6 7 6 7 w w w c w c w c w c 6 7 6 7 x y z 1 0 0 0 0 u x u y u z u N N N N N N N N N N 6 7 4 5 c 6 7 w w w c w c w c w c 6 7 0 0 0 0 x y z 1 v x v y v z v N N N N N N N N N N 6 7 6 7 6 7 6 7 6 7 6 7 6 7 6 a 7 4 5 The above formula can be written as ; (17) AF = 1 When N=6, the systems of linear equations can be solved directly. 8 When N6, we can obtain the least-squares solution under the constraint of jFj = 1. And then through 150 SVD decomposition, the last column of the V matrix is the solution. (18) F = UDV : By substituting Eq. 9 into Eq. 18, we can obtain Eq. 19. c c c c (19) F = [I R I T ] : c c Thus, the rotation matrix R and translation matrix T can be calculated as follows. 2 3 c c c a a a 11 12 13 6 7 6 7 c 1 c c c R = I 6 7; (20) a a a 21 22 23 4 5 c c c a a a 31 32 33 2 3 6 7 6 7 c 1 T = I ; (21) 6 7 4 5 After the rotation matrix and translation matrix are calculated, we completed the live camera localization for the current point data. 3. Experiments To investigate the performance of the proposed method, we conducted experiments on metrically consistent and indoor scenes. All experiments were implemented on a PC with AMD Ryzen 5 5600H (3GHz) CPU, and a packaged FPP sensor. The built FPP sensor is shown in Fig. 5, which contains a Basler acA1920-155uc camera (1920  1200 resolution), a LighterCraf ter 4500 (912  1140 resolution) and an interface expansion 160 board. The interface expansion board is used to realize the synchronous triggering of the projector and the camera. The imaging speed of the sensor is 300 fps, the single-view measurement accuracy is 0.067 mm and the global mapping accuracy is around one millimeter. Figure 5: Labeled diagram of FPP sensor. 9 Firstly, a doll was measured to investigate the ability of metrically consistent models from trajectories containing loop closures. As shown in Fig. 6, the FPP system was placed in a xed location and the doll was placed on a turntable. Then, the turntable was spun through a full rotation to capture point clouds from di erent perspectives of the doll for registration, and the trajectory localized by the corresponding camera should also be a circle and return to the origin, which allows us to easily evaluate the quality of camera localization. The main motivation for our experiments is to investigate the convergence of the mapping and localization schemes. As shown in the enlarged image of Fig. 6, the captured single-view point clouds were 170 well registered, which demonstrates the performance of our mapping method. Then, the resulting sensor trajectories are given at the bottom of Fig. 6. Since the FPP sensor is xed, the object is placed on the turntable, which can be equivalent to the sensor rotating around the object. In this experiment, the turntable was rotated 3 times, 13 times for each turn. The loop closures e ect of localization trajectory performs well. The camera forms a perfectly circular trajectory around the doll and can return to the original location after one rotation, which demonstrates the performance of our localization method. Figure 6: Experimental results of the doll. Finally, to investigate the performance of our method for real scenes, we conducted the mapping and localization experiment in the corridor. The system setup of the indoor scene experiment is shown in Fig. 7, which includes a computer, our FPP sensor, a battery and a mobile cart. The FPP sensor is located in the mobile cart to scan the indoor scene. The results of scenes using our system are shown in Fig. 8. To 180 generate this map of the indoor scene, we captured over 100 frames of single-view point clouds. The overlap area of adjacent point clouds accounts for one-third of the total area. As shown at the top of Fig. 8, the 10 generated map point cloud is dense and has high-quality local geometric information. Then, the map with texture information is shown at the top of Fig. 8, which achieves perfect registration of multi-frame point clouds and avoids interference of cumulative errors. This also demonstrates that our global pose alignment strategy scales well to large spatial extents and long sequences. Figure 7: The system setup of indoor scene experiment. Figure 8: Experimental results of the indoor scene. 4. Conclusion The availability of accurate 3-D metrology technologies such as FPP has the potential to revolutionise the elds of indoor localization and mapping, but methods to date have not fully leveraged the accuracy and speed of sensing that such techniques o er. This paper presents a millimeter (mm)-level indoor FPP-based 190 SLAM method, which includes the front-end using FPP technique to perceive the unknown scene, and the correspond back-end algorithm using the 2D-to-3D descriptor and the coordinate transformation relationship of FPP to estimate of the pose of sensors and construct the map of indoor scene. The capabilities of the 11 proposed method have been demonstrated on room sized SLAM. We believe that this paper will open a new door for the application of FPP in indoor SLAM. Data Availability The data that support the ndings of this study are available from the corresponding author upon reason- able request. Declaration of competing interest The authors declare that they have no known competing nancial interests or personal relationships that 200 could have appeared to in uence the work reported in this paper. References [1] C. Cadena, L. Carlone, H. Carrillo, Y. Latif, D. Scaramuzza, J. Neira, I. Reid, J. J. Leonard, Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age, IEEE Transactions on robotics 32 (6) (2016) 1309{1332. [2] G. Bresson, Z. Alsayed, L. Yu, S. Glaser, Simultaneous localization and mapping: A survey of current trends in autonomous driving, IEEE Transactions on Intelligent Vehicles 2 (3) (2017) 194{220. [3] R. Azzam, T. Taha, S. Huang, Y. Zweiri, Feature-based visual simultaneous localization and mapping: A survey, SN Applied Sciences 2 (2) (2020) 1{24. [4] A. Hornung, K. M. Wurm, M. Bennewitz, Humanoid robot localization in complex indoor environments, 210 in: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, 2010, pp. 1690{ [5] M. Zeybek, Indoor mapping and positioning applications of hand-held lidar simultaneous localization and mapping (slam) systems, Turkey Lidar Journal 3 (1) (2021) 7{16. [6] R. Otero, S. Laguela,  I. Garrido, P. Arias, Mobile indoor mapping technologies: A review, Automation in Construction 120 (2020) 103399. [7] P. Newman, D. Cole, K. Ho, Outdoor slam using visual appearance and laser ranging, in: Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006., IEEE, 2006, pp. 1180{1187. [8] S. Zlatanova, G. Sithole, M. Nakagawa, Q. Zhu, Problems in indoor mapping and modelling, Acquisition 220 and Modelling of Indoor and Enclosed Environments 2013, Cape Town, South Africa, 11-13 December 2013, ISPRS Archives Volume XL-4/W4, 2013. 12 [9] H. Durrant-Whyte, T. Bailey, Simultaneous localization and mapping: part i, IEEE robotics & automation magazine 13 (2) (2006) 99{110. [10] D. Chen, G. X. Gao, Probabilistic graphical fusion of lidar, gps, and 3d building maps for urban uav navigation, NAVIGATION, Journal of the Institute of Navigation 66 (1) (2019) 151{168. [11] P. Ellis, B. Griscom, W. Walker, F. Gon calves, T. Cormier, Mapping selective logging impacts in borneo with gps and airborne lidar, Forest Ecology and Management 365 (2016) 184{196. [12] S. Thrun, Learning metric-topological maps for indoor mobile robot navigation, Arti cial Intelligence 99 (1) (1998) 21{71. 230 [13] Z. Ren, L. Wang, L. Bi, Robust gicp-based 3d lidar slam for underground mining environment, Sensors 19 (13) (2019) 2915. [14] R. Mur-Artal, J. D. Tard os, Visual-inertial monocular slam with map reuse, IEEE Robotics and Automa- tion Letters 2 (2) (2017) 796{803. [15] R. A. Newcombe, S. Izadi, O. Hilliges, D. Molyneaux, D. Kim, A. J. Davison, P. Kohi, J. Shotton, S. Hodges, A. Fitzgibbon, Kinectfusion: Real-time dense surface mapping and tracking, in: 2011 10th IEEE international symposium on mixed and augmented reality, IEEE, 2011, pp. 127{136. [16] J. Li, R. Zhong, Q. Hu, M. Ai, Feature-based laser scan matching and its application for indoor mapping, Sensors 16 (8) (2016) 1265. [17] E. Eade, T. Drummond, Scalable monocular slam, in: 2006 IEEE Computer Society Conference on 240 Computer Vision and Pattern Recognition (CVPR'06), Vol. 1, IEEE, 2006, pp. 469{476. [18] Y. M. Mustafah, A. W. Azman, F. Akbar, Indoor uav positioning using stereo vision sensor, Procedia Engineering 41 (2012) 575{579. [19] S. Zhang, Recent progresses on real-time 3d shape measurement using digital fringe projection techniques, Optics and lasers in engineering 48 (2) (2010) 149{158. [20] D. Zheng, Q. Kemao, J. Han, J. Wang, H. Yu, L. Bai, High-speed phase-shifting pro lometry under uorescent light, Optics and Lasers in Engineering 128 (2020) 106033. [21] S. S. Gorthi, P. Rastogi, Fringe projection techniques: whither we are?, Optics and lasers in engineering 48 (ARTICLE) (2010) 133{140. [22] X. Su, W. Chen, Fourier transform pro lometry:: a review, Optics and lasers in Engineering 35 (5) (2001) 250 263{284. [23] L. Huang, Q. Kemao, B. Pan, A. K. Asundi, Comparison of fourier transform, windowed fourier transform, and wavelet transform methods for phase extraction from a single fringe pattern in fringe projection pro lometry, Optics and Lasers in Engineering 48 (2) (2010) 141{148. 13 [24] C. Zuo, L. Huang, M. Zhang, Q. Chen, A. Asundi, Temporal phase unwrapping algorithms for fringe projection pro lometry: A comparative review, Optics and lasers in engineering 85 (2016) 84{103. [25] S. Zhang, P. S. Huang, Novel method for structured light system calibration, Optical Engineering 45 (8) (2006) 083601. [26] S. Zhang, Absolute phase retrieval methods for digital fringe projection pro lometry: A review, Optics and Lasers in Engineering 107 (2018) 28{37. 260 [27] D. Zheng, F. Da, Q. Kemao, H. S. Seah, Phase-shifting pro lometry combined with gray-code patterns projection: unwrapping error removal by an adaptive median lter, Optics express 25 (5) (2017) 4700{ [28] B. Triggs, P. F. McLauchlan, R. I. Hartley, A. W. Fitzgibbon, Bundle adjustment|a modern synthesis, in: International workshop on vision algorithms, Springer, 1999, pp. 298{372. [29] S. Rusinkiewicz, M. Levoy, Ecient variants of the icp algorithm, in: Proceedings third international conference on 3-D digital imaging and modeling, IEEE, 2001, pp. 145{152.

Journal

Computing Research Repository – arXiv (Cornell University)

Published: Apr 23, 2022

There are no references for this article.