TY - JOUR
AU - Han,, Soonhung
AB - Abstract This paper proposes a reconstruction method of the as-built model of plant piping, where the port point of the fitting is used as the secondary reference point, which represents a connection point between the fitting and its connected pipeline. The connection point between a fitting and a pipeline is a critical point of reference in the as-built model. In this study, a fitting model is registered to the point clouds first by using the state-of-the-art technique, which is based on an iterative closest point (ICP) algorithm and also considers the alignment of the port direction with the connected pipelines. Then, the port points of the registered fitting are considered as connection points, and the nozzle points of the connected pipelines are adjusted along its axis based on the information. After the registration process, which uses the detailed shape of fitting and the directions of adjacent pipelines at the same time, the port point on the registered fitting becomes a better estimate of the connection point compared with the nozzle point of a pipeline detected. Graphical Abstract Open in new tabDownload slide Graphical Abstract Open in new tabDownload slide 3D reconstruction, automation, piping system, point cloud, port information, segmentation Highlights In the reconstruction of as-built plant piping system, a connection point between pipeline and fitting is important reference point to be checked. In the proposed method of reconstruction, port point of fitting is used as a reference for the connection point. Before using the port point of fitting as a reference, the fitting model is registered to the point cloud using state-of-the-art technique. The nozzle points of the adjacent pipelines are adjusted to the port points of the fitting that is registered to the point cloud. In the experiment, a fitting model was registered to the point cloud and the nozzle point of the adjacent pipeline was adjusted to the correct connection point by using the port point of the fitting as a reference. 1. Introduction In plant configuration management, as-built or as-operated 3D models show high utility in product life cycle management. Since the actual plant site should be reflected in the as-built 3D model, it is not created based on the design drawing but the laser scan data. There is a commercial software (SW) that reverse engineers as-built plant models from 3D point cloud data, which are scan data. Studies have been conducted for the automation of the process. By detecting and removing the point cloud corresponding to the pipelines, which have a relatively simple shape, it is easy to extract the point cloud of the fittings, which exist between the pipelines. For this reason, the existing methods start from the pipeline detection and proceed to fitting recognition as the automation of reverse engineering for an as-built plant 3D model (Pang et al., 2015; Son et al., 2015). The fitting recognition is a task of finding a 3D CAD model that corresponds to a point cloud segment between the pipelines from the model database. For the placement of a recognized model, the orientation and position of the model should be determined. In existing methods, they were inferred from the axis or the nozzle point of the pipeline (Pang et al., 2015; Son et al., 2015). However, the detection of the pipeline with the accurate nozzle point is a challenging task due to the noises and diffractions in the point cloud, especially at the connection point between the pipeline and the connected fitting. The nozzle location on the fitting where the pipeline is connected is called a port in the plant CAD system and the port points are given in the 3D CAD model of the fitting. The port points on the model are equivalent to the mating condition of the mechanical assembly. Besides, the port information of the fitting is a crucial point in terms of the precision of fitting installation at the plant construction site for the supervision requirement. Figure 1 shows the separate names of the meeting points. The port point is the nozzle point at the fitting. The pipe nozzle point is the open-ended point of the pipeline. The connection point is a virtual point where the port point and the pipe nozzle point should meet. For maintenance or decommissioning plan based on a reconstructed as-built 3D model, the accurate location of the connection point between the fitting and the pipeline is required. Those three points should be placed at the same position for the correct alignment. Figure 1: Open in new tabDownload slide Pipe nozzle point, port point, and connection point. Figure 1: Open in new tabDownload slide Pipe nozzle point, port point, and connection point. Although the pipeline should be connected at the port point of the fitting, there can be a mismatch between them due to the various reasons such as noise, diffraction, or occlusion existing in the scanned point cloud. In Fig. 1, the fitting model is not aligned with the point cloud, and the nozzle point of the pipeline ends earlier; therefore, the port point of the fitting and the pipeline nozzle point do not meet at the actual connection point. If the fitting model is correctly registered to the point cloud, the port point of the fitting can be a better reference for the connection point compared with the nozzle point of the detected pipeline. In this study, the fitting model that is recognized is registered to the point cloud, and then the port points of the registered fitting are used as a reference for the connection points. During the registration, the port directions of the fitting are forced to be aligned with the orientations of the connected pipelines, so the misalignment from the false registration is prevented. Finally, the nozzle point of the adjacent pipeline is adjusted to the port point of the fitting but along its pipeline axis, so the orientation of the pipeline can be maintained. 2. Related Works The reconstruction of the 3D model from point clouds was studied in multiple ways. The feature curve, which defines the intersection between developable surfaces, was extracted in the method of Lee and Bo (2016), and the method to obtain a sweep path and cross-sectional 2D shape for the shape reconstruction from the watertight mesh was proposed by Kresslein et al. (2018). Mineo et al. (2019) proposed a method to detect the boundary of the meshed surfaces obtained from 3D point clouds. For a 2D point cloud, Singh et al. (2019) proposed a method to generate unstructured Delaunay mesh out of 2D point cloud without an explicit step of the curve reconstruction process, which made the process only dictated by the geometric complexity of the domain. Studies mentioned above extracted features such as boundary edges, feature curve, and cross-sectional points to reconstruct shape out of point cloud with arbitrary shape, whereas the problem of reconstruction of the as-built plant CAD model from 3D point cloud includes the detection of regular shapes such as pipes, planes, and rectangular structures and the recognition of given shape for the components with complex shapes such as fittings. For the regular-shaped component, the model is generated based on the parameters. For the component with complex shape, the corresponding CAD model is recognized from the model database and placed at a proper position with orientation. Especially for the reconstruction of a piping system, the existing studies have focused on (i) the pipeline detection and modeling, (ii) the fitting recognition, and (iii) the registration of the recognized model. Related researches are classified along the stages of the reconstruction as shown in Fig. 2. This study is focused on the connection point between pipeline and fitting, which was not mentioned in detail in the previous works. Figure 2: Open in new tabDownload slide Reconstruction of as-built plant piping system with related works. Figure 2: Open in new tabDownload slide Reconstruction of as-built plant piping system with related works. The problem of pipeline detection was solved by numerous approaches. It was solved by finding global similarities from the point cloud (Liu et al., 2013; Qiu et al., 2014), by projecting points onto the plane perpendicular to the eigenvector of the point cloud segment to classify whether it corresponds to a pipeline (Kawashima et al., 2014), by computing the curvature of the point cloud segment (Son et al., 2014), or by using Hough transform (Patil el al., 2017; Rabbani et al., 2005). Alternatively, there was a method of finding a cylindrical surface of point cloud having a constant distance from its skeletonized medial axis (Lee et al., 2013). Chida and Masuda (2016) also proposed a method to reconstruct regular shapes such as boxes and polygonal prisms from point clouds. The problem of fitting recognition from the point cloud is similar to the problem of retrieving a model with the shape inside the scanned point cloud. Object recognition from a 3D point cloud was handled in computer vision. In the case of plant CAD system, fitting CAD models in the spec catalog library can be used as a model database for object recognition. Czerniawski et al. (2016) proposed a method for locating and extracting pipe spools in a cluttered point cloud using local data-level curvature estimation, clustering, and bag-of-features. Son et al. (2015) extracted the point cloud segment that corresponds to the fitting and found the corresponding CAD model using the topology of the connection between the pipelines and the fittings and the bounding box dimensions of the segment. Pang and Neumann (2013) used the moving scanning window for the search of the fitting, which exhaustively evaluated the input point cloud, to determine whether the point cloud within the window matches a model in the database. After the fitting recognition from the point cloud, the placement of the model proceeds. For the placement, the position and the orientation of the model in the global coordinate system should be found to register the model to the point cloud. In the method of Pang and Neumann (2013), the rotational options of a model were 90 degrees or 180 degrees in the x, y, and z axes, so the recognized model was within one of the six variations of rotated shapes. In the case of an arbitrary orientation of fitting, the principal directions of the point cloud and the model should be examined in advance using the principal component analysis (PCA). In the case of Son et al. (2015), the axis and nozzle points of adjacent pipelines were used to set the initial posture of the fitting model. However, the pose of a recognized CAD model is not perfect with the approximate position and orientation found by the aforementioned studies. The orientation of the CAD model found by PCA has limited accuracy. If the nozzle points and directions of the adjacent pipelines detected are inaccurate, placement of the model based on the information cannot be accurate. Therefore, a fine registration step should be followed to correct the alignment of the fitting model on the point cloud. There were several methods to register 3D CAD model to point cloud specifically for the reconstruction of as-built 3D model (Bosché 2010; Son et al., 2015), and these studies were based on the application of an iterative closest point (ICP) (Besl & MacKay, 1992) method. In the existing methods, the connection point between the fitting and pipeline was dependent on the nozzle point of the detected pipeline, which contains an error due to the noises and diffractions around the connection point. In this study, the orientation of the detected pipeline is maintained and considered as a primary reference of the reconstructed model, but for the connection point, the port point of the registered fitting is used as a reference instead of the nozzle point of the detected pipeline. To consider the port point as the reference, the fitting needs to be finely registered to the point cloud. Figure 3 compares the proposed method with commercial SW and related studies: In Leica Cyclone (Leica, 2020) SW, pipelines are detected automatically. Afterward, the user manually selects a fitting and its insertion point along the pipeline. Then, the fitting is inserted with its alignment with the pipeline at the selected point. Faro PointSense Plant (FARO Technologies, Inc., 2019) SW automatically detects the pipeline. Since the nozzle point of the detected pipeline can be inaccurate, the user specifies the insertion point of the fitting manually from the panoramic 2D image captured during the scanning. In the method proposed by Pang et al. (2015), pipelines were detected by the global similarities in the scanned point cloud (Qiu et al., 2014). Fittings were recognized and positioned through the method of Pang and Neumann (2013), and they were aligned to the axis of the adjacent pipelines. The nozzle points of the adjacent pipeline that have gaps or overlaps with the fitting were adjusted to the fitting. As a result, the fitting and the pipeline were connected without gaps or overlaps. Still, the position and orientation of fitting can be misaligned with the point cloud without a fine registration step. In the method proposed by Son et al. (2015), point clouds corresponding to pipelines were segmented by the smoothness constraint and curvature computation, and fittings were recognized from the remaining point cloud. The recognized fittings were initially placed by aligning its port direction with adjacent pipelines, and finally by registration while maintaining the alignment. However, after the fittings were registered successfully through this process, the nozzle points of the adjacent pipelines were unchanged. The nozzle point of the detected pipeline can contain error; thus, there can be a mismatch between the nozzle point and the port point of the registered fitting. In this study, the initial placement is done by matching one of the port points of the fitting with the nozzle point of the pipeline that needs to be connected. After the initial placement, the fine registration of the fitting to the point cloud is done referring to the study of Son et al. (2015). Finally, the nozzle points of the adjacent pipelines, which may contain errors during the pipeline detection process, are adjusted to the port points of the fitting that is registered to the point cloud. For the adjustment, the axis of the detected pipeline is maintained, and only the nozzle point is moved along the axis to be closest to the port point of the fitting. So, the connection point between the pipeline and fitting is dependent on the port point of the nearby fitting rather than initially detected nozzle points. Figure 3: Open in new tabDownload slide Comparison with related works. Figure 3: Open in new tabDownload slide Comparison with related works. 3. Reconstruction of Piping System Using Port Points of Fitting When 3D point cloud data is given as an input to the system, the following steps are performed as shown in Fig. 4: Pipelines are detected and segmented. From the segmented point cloud corresponding to the pipeline, the pipeline model is generated. After removing the point cloud corresponding to the pipeline, the remaining point cloud corresponds to fittings with various shapes. By Euclidean clustering, the remaining point clouds can be grouped into point clusters. For each point cluster, the corresponding fitting model in the model database can be recognized. After the fitting CAD model is identified, the initial orientation and position of the model are required to place the model roughly within the point cloud before the registration. The initial placement of the model is done by using PCA and matching one of the fitting port points with the nozzle point of the pipeline that needs to be connected. The fitting model that is initially placed in step (iv) is registered to the point cloud. For the registration, the ICP algorithm is modified by giving a constraint of aligning the port directions on the fitting with the orientations of the adjacent pipelines. The nozzle point of the pipeline is adjusted by moving the point along the pipeline axis and to be closest to the port point of the fitting. Up to (v), steps are similar to the existing methods, and this study proposes additional step (vi), which makes the connection point between pipeline and fitting to be determined by the port points of the fitting that is registered to the point cloud using the state-of-the-art technique. Figure 4: Open in new tabDownload slide Flow diagram of the proposed method. Figure 4: Open in new tabDownload slide Flow diagram of the proposed method. 3.1 Pipe segmentation and modeling by the existing method To classify the points corresponding to a pipeline, and generate the model, automatic or semi-automatic detection of the pipeline from a scanned point cloud can be used. In our study, we developed semi-automatic detection of the pipeline based on two end points selected by the user as in Fig. 5. The method is based on efficient RANSAC proposed by Schnabel et al. (2007). Figure 5: Open in new tabDownload slide Semi-automatic detection and segmentation of pipeline using two end points. Figure 5: Open in new tabDownload slide Semi-automatic detection and segmentation of pipeline using two end points. First, two points |${p_1}$| and |${p_2}$| that are roughly near the nozzle points of a pipeline are selected from the whole set of point cloud P. Then, a point q that is closest to the midpoint of |${p_1}$| and |${p_2}$| is found from P, based on Equation (1). $$\begin{eqnarray} q = \mathop {\arg \,\,\min }\limits_{q \in P} \left( {\left\| {\frac{{{p_1} + {p_2}}}{2} - q} \right\|} \right) \end{eqnarray}$$(1) Since the midpoint between |${p_1}$| and |${p_2}$| is a virtual point that is not necessarily on the surface of the pipeline, the point q is found by (1). The method of Schnabel et al. (2007) is used to fit a cylinder to the neighborhood of q. The neighborhood Q is found by using a radius search using Equation (2). $$\begin{eqnarray} Q = \left\{ {p \in P\left| {\left\| {p - q} \right\|} \right. < {r_{search}}} \right\} \end{eqnarray}$$(2) In (2), |${r_{search}}$| is depending on the density of point cloud and the expected radius range of pipelines. The output parameters of the fitted cylinder include a point |$\rho $|⁠, a direction vector |$\phi $|⁠, and a radius |${r_{cyl}}$|⁠. Then, the fitted cylinder expanded from |${p_1}$| to |${p_2}$| can be generated as a pipeline model. The cylindrical model is generated between a point |${\rho _1}$| and a point |${\rho _2}$|⁠, which are the projected coordinates of |${p_1}$| to |${p_2}$| on the cylindrical axis as determined by (3). $$\begin{eqnarray} {\rho _i} = \,\,\rho + {\phi ^{\rm{T}}}\left( {{p_i} - \rho } \right)\phi \end{eqnarray}$$(3) For the segmentation of the points corresponding to the pipeline, the fitted cylinder expanded from |${p_1}$| to |${p_2}$| is reused but its radius is enlarged by 1.2. The segmented points inside the enlarged cylinder are determined by (4). $$\begin{eqnarray} {P_{pipe}} = \left\{ {p \in P\left| {\begin{array}{@{}*{1}{c}@{}} {\left\| {{{\left( {p - {p_2}} \right)}^{\rm{T}}}\phi } \right\| \le \left\| {{{\left( {{p_1} - {p_2}} \right)}^{\rm{T}}}\phi } \right\|,}\\ {{{\left( {p - {p_2}} \right)}^{\rm{T}}}\phi \times {{\left( {{p_1} - {p_2}} \right)}^{\rm{T}}}\phi \,\, \ge 0,}\\ {\left\| {\left( {p - {p_2}} \right) - {{\left( {p - {p_2}} \right)}^{\rm{T}}}\phi } \right\| \le 1.2 \times {r_{cyl}}} \end{array}} \right.} \right\} \end{eqnarray}$$(4) The first condition ensures that the projected distance of a vector |$p - {p_2}$| in the direction of |$\phi $| is smaller than the projected distance of a vector |${p_1} - {p_2}$|⁠. The second condition ensures that the signs of scalar components of |$p - {p_2}$| and |${p_1} - {p_2}$| in the direction of |$\phi $| are the same. The third condition ensures that a perpendicular distance between p and the axis of the cylinder is less than |$1.2{r_{cyl}}$|⁠. In this method, two points are selected from the scanned point cloud as hints of pipeline nozzle points. The existing methods introduced in Section 2 can be used instead to detect a pipeline without any input during the process. Figure 6 shows an example of the pipeline models detected from the point cloud. Figure 6a is a schematic drawing of a scanned point cloud of two pipelines connected with a fitting in between shown in dotted line and pipeline models shown in solid line. As in Fig. 6a, the pipeline models generated automatically often contain an error in their directions and their nozzle points. The example shown in Fig. 6b also contains the error in the nozzle point of the pipeline and its direction. Figure 6: Open in new tabDownload slide Example of pipeline modeling with errors in direction and nozzle point. Figure 6: Open in new tabDownload slide Example of pipeline modeling with errors in direction and nozzle point. 3.2 Fitting recognition and initial placement After the pipeline detection is achieved in an automatic or semi-automatic way, the result will be similar to Fig. 7a. Afterward, the pipeline point cloud is removed, and the remaining points are grouped into multiple point clusters that represent fittings between the pipelines (Son et al., 2015). The Euclidean clustering is used, which is the region-grouping method using Euclidean distance (Euclidean Cluster Extraction, 2020). It is the region-grouping method that grows the region from a random point until the nearest point nearby the region is further than the threshold value of Euclidean distance (Euclidean Cluster Extraction, 2020). After the Euclidean clustering, each point cluster grouped represents the fitting as shown in Fig. 7c. Figure 7: Open in new tabDownload slide (a) Segmentation of pipeline, (b) the removal of pipeline points, (c) Euclidean clustering, and (d) fitting recognition. Figure 7: Open in new tabDownload slide (a) Segmentation of pipeline, (b) the removal of pipeline points, (c) Euclidean clustering, and (d) fitting recognition. Assuming each cluster of points represents fitting, a fitting model corresponding to each point cluster is recognized from the model library by using a shape similarity measure as in Fig. 7d. To recognize a model using its shape and to register the recognized model to the point cloud, the external shape of the model and the shape captured in the scanned point cloud must be similar. According to the study of Lee (2019), the shape of the fitting CAD model can be expressed in various levels of detail (LoD) depending on its usage, as shown in Fig. 8. For the reconstruction process proposed in this study, a prebuilt library with fitting models in fidelity LoD 3 is required. The fidelity LoD 3 is a level suitable for the handover of the as-built plant CAD model to the owner of the plant, which is reconstructed from the scanned point cloud and has its external shape similar to the actual fitting. The assumption that the models in the prebuilt library are with high fidelity has been made in the previous studies (Huang & You, 2012; Pang & Neumann, 2013). Currently, the library of catalogs managed by EPCs has models with simplified shape that cannot be used for the reconstruction process; therefore, EPCs rely on outsourcing for the manual modeling of fittings with high fidelity based on scanned point cloud. On the other hand, more detailed and realistic shape is required in the handover model due to the requirement of digital twin and the demand from owner specification. As the requirement of as-built model for the handover becomes universal, CAD models with high fidelity will be piled up in the library and should be used for the automation of reconstruction. Figure 8: Open in new tabDownload slide Fidelity LoD for fitting 3D CAD model (Lee, 2019). Figure 8: Open in new tabDownload slide Fidelity LoD for fitting 3D CAD model (Lee, 2019). In our study, the model is recognized by using the shape similarity-based part retrieval system proposed by Kim et al. (2019). The system used the 3D point cloud data acquired by 3D scanner as input and found the 3D CAD model with the most similar shape from the model library. The method compared shape descriptors generated from the CAD models with the shape descriptor generated from the point cluster. For the 3D CAD model, depth images were generated at various viewpoints, and for the point cluster, the depth image was generated from an appropriate viewpoint and image resolution. From each depth image, a descriptor was generated as a set of features. Then, the distance between one descriptor derived from the scanned point cloud and several descriptors derived from the 3D CAD models were calculated. The minimum distance among the values was considered as a similarity between the model and the point cluster. Therefore, the model with the highest similarity to the point cluster was chosen as a recognized model from the library. The method accounted for the actual condition of the scanned point cloud, which has a lot of occlusion and noises. A further explanation of the method can be referred to in the study of Kim et al. (2019). After the recognition, the model should be placed at the proper position with the orientation. The initial placement of the model is needed for the further fine registration step. The initial placement is done by finding rough orientation and attaching one of the port points to the end of the adjacent pipe as shown in Fig. 9. Figure 9: Open in new tabDownload slide Process of initial placement: orientation found by PCA and translation by connecting the fitting port point with the pipe end. Figure 9: Open in new tabDownload slide Process of initial placement: orientation found by PCA and translation by connecting the fitting port point with the pipe end. The approximate orientation of the recognized CAD model is derived through the PCA between the point cluster and the fitting CAD model. The method was also mentioned by Pang and Neumann (2013) for a case when the recognized model has an arbitrary orientation. The principal axes found by PCA can be used to align the model with the point cluster in the same orientation. With the rough orientation set by PCA, the fitting model can be translated, so that one of its port points can be connected to the nozzle point of the adjacent pipeline. This refers to the method of Son et al. (2015), in which the initial placement using the nozzle points of the adjacent pipeline was done before the fine registration step. As shown in Fig. 10a, when the fitting is placed by connecting one of its port points with a pipeline nozzle point, the other port points on the fitting can be mismatched with the nozzle point of the connected pipelines. Additionally, the fitting model that is placed after the inaccurate nozzle point can be misaligned with the scanned point cloud. Figure 10: Open in new tabDownload slide Example of initial placement of the fitting CAD model. Figure 10: Open in new tabDownload slide Example of initial placement of the fitting CAD model. In Fig. 10b, the port point in the left yellow box is mismatched with the pipe end, and the fitting model is overlapped with the pipeline that is already generated, whereas the other port point in the right yellow box is well connected to the nozzle point of the pipeline. As shown in the schematic drawing of Fig. 10a, and the example in Fig. 10b, the recognized fitting model that should be aligned with the scanned point cloud as its shape is misaligned after the initial placement. 3.3 Fitting registration In Fig. 11a, the schematic drawing shows the fitting model that is initially placed in the previous stage is aligned with the point cloud after the registration process. As mentioned in Section 3.2, the result after the initial placement shows the misalignment of the fitting model with the point cloud. Therefore, registration of the model to the point cloud is performed. Figure 11: Open in new tabDownload slide Registration of the fitting model to the point cloud. Figure 11: Open in new tabDownload slide Registration of the fitting model to the point cloud. For the registration, the modification of an ICP (Besl & McKay, 1992) is used, which is similar to the method of Son et al. (2015). The original ICP algorithm finds the transformation matrix that moves one point cloud to be registered to another point cloud when the two point cloud data have a similar shape. The CAD model is converted to the point cloud with the same external shape by using the point sampling method in the study of Kim et al. (2019). The ICP registration process is performed between the converted point cloud and the point cluster that represents fitting. The scanned point cloud of fitting can contain a lot of noises, and the shape of the model from the database is simplified or different from the shape of the point cluster; thus, there can be difficulty in the registration. To overcome this difficulty, the hint that the fitting's port direction should be aligned with the connected pipeline's axis is used. Although the nozzle point of the pipeline can be inaccurate due to noise around the connection point, the direction of the pipeline can be detected with more reliability by making use of global similarity as suggested by Qiu et al. (2014). If the individual pipeline is detected separately instead of using global similarity as in our study, the pipelines that should be in the same orientation can have a little difference in their orientations. Therefore, 10 degrees of accepted tolerance is given for the alignment between the port direction and an adjacent pipeline axis. In Fig. 11a, the vectors going outward from each port, |${v_1}$| and |${v_2}$|⁠, are the direction of each port, and the vectors |${w_1}$| and |${w_2}$| are the directions of the pipelines. During the iterative process of registration, the model's orientation is constrained using the condition of (5). $$\begin{eqnarray} {v_i} \cdot {w_i} > \cos 10^\circ \end{eqnarray}$$(5) With this constraint of aligning the port directions of the fitting with the adjacent pipelines, the model registration becomes a problem of finding a rotation and position along the axis of the pipelines. Figure 11b shows the status before and after the registration of the fitting model to the point cloud through the ICP method. After the model is precisely registered to the point cloud, the port point that was connected with the pipeline in the initial placement is dislocated. 3.4 Pipeline adjustment using fitting port points Figure 12a shows the process of pipeline adjustment. The connection point between the fitting model and the adjacent pipelines is mismatched after the registration. The pipeline model can invade into or have gaps with the fitting model due to the incorrect nozzle point of the pipeline that is initially detected. Therefore, the nozzle points of the pipeline are adjusted to the port points of the registered fitting in this stage. Figure 12: Open in new tabDownload slide Adjustment of the pipe ends to the port points of the fitting. Figure 12: Open in new tabDownload slide Adjustment of the pipe ends to the port points of the fitting. The nozzle point is adjusted by moving the point along the pipeline axis rather than moving it to the fitting's port point directly. In Fig. 12a, |${n_i}$| represents the original coordinate of the nozzle point, and |${w_i}$| represents the direction of the pipeline. The adjustment of the nozzle point |${n_i}$| to the new coordinate |${n_i}^{\prime}$| is carried out by (6). $$\begin{eqnarray} n_i^\prime = {n_i} + w_i^{\rm{T}}\left( {\mathop {\arg \,\,\min }\limits_{{o_j} \in O} \left( {\left\| {{n_i} - {o_j}} \right\|} \right) - {n_i}} \right){w_i} \end{eqnarray}$$(6) In (6), O represents a set of all port points of the fitting. If there is a failure in registration, the direct connection of the pipeline to the fitting port point can disturb the orientation of the pipeline that is initially detected. The orientation of the pipeline can be detected with high reliability especially in the case of using global similarity, so the pipeline orientation is kept as a primary reference over the port point of the fitting that is used as a secondary reference in our study. The nozzle point of the pipeline is moved along the axis of the pipeline and moved to be closest to the port point of the fitting to be connected. First, a coordinate of the port point that is closest to the pipeline nozzle point is projected onto the pipeline axis. Then, the coordinate of the pipeline nozzle point is substituted with the projected coordinate of the port point along the axis. Therefore, the changed nozzle point is still on the pipeline axis and is at the same location as the port point in terms of the projected coordinate on the pipeline axis. Although the modified nozzle point does not meet with the fitting's port point directly, the large discrepancy between the fitting's port point and the pipeline's nozzle point can be relieved visually, and the orientation of the overall pipeline branch is not disturbed at the same time. As shown in Fig. 12a, the directions of the pipelines, |${w_1}$| and |${w_2}$|⁠, are unchanged but the gap or overlap between the fitting model and the pipeline is resolved after the pipeline adjustment process. Figure 12b is the example case that the overlap between the fitting model and the pipeline after the registration in Section 3.3 is resolved visually after the pipeline adjustment process. 4. Implementation and Experimental Results A prototype system for the reconstruction of the as-built 3D CAD model of a plant piping system is implemented to verify the proposed method. The system is implemented in the form of a plug-in to AutoCAD Plant3D 2017. It utilizes the point cloud visualization functions of AutoCAD Plant3D and also the modeling function to generate recognized CAD models from the point cloud. The processing of point cloud data such as normal estimation is done by using the point cloud library (PCL) (Point Cloud Library, 2020). For the segmentation and the detection of a pipeline, an efficient RANSAC function implemented in the Computational Geometry Algorithms Library (CGAL) (The Computational Geometry Algorithms Library, 2020) is used. As mentioned in Section 3.1, two points selected by a user, which are roughly at the pipeline nozzle points, can be inaccurate, and this represents the similar conditions in existing studies that had difficulty in finding accurate nozzle points of the cylindrical shape due to noises around the connection point. The point cloud corresponding to the pipeline is removed, and the remaining point cloud is grouped into point clusters by using the Euclidean clustering function provided in the PCL (Point Cloud Library, 2020). Therefore, the point cluster corresponding to each item is isolated, and the recognition of the CAD model corresponding to each fitting can be applied to each point cluster. A shape similarity-based part retrieval system proposed and developed by Kim et al. (2019) is developed using C++, OpenGL, and the ezSIFT library, as mentioned in their work. For the registration, modification of ICP is implemented based on the function in the PCL (Point Cloud Library, 2020) with the constraint of aligning the port direction to the connected pipeline's orientation as mentioned in Section 3.3. The experiment was done with the prototype system to verify the method of using fitting's port points in the reconstruction of a piping system. For the test data, we used a scanned point cloud shown in Fig. 13 comprising 163 897 points, which is obtained through laser scanning the actual plant piping system. Figure 13: Open in new tabDownload slide Test data: scanned point cloud of the plant piping system. Figure 13: Open in new tabDownload slide Test data: scanned point cloud of the plant piping system. Figure 14 shows the intermediate stages until the initial placement of fittings. The pipeline segmentation was done on the test data and the points classified are shown in pipeline segmentation result in yellow. With the pipeline segmentation, pipeline models were generated. The points classified as pipeline were removed, and fittings were recognized from the remaining points as shown in the figure. The recognized fittings were placed based on the pipelines during the initial placement stage as explained in Section 3.2. Figure 14: Open in new tabDownload slide Reconstruction process until the initial placement of recognized fittings. Figure 14: Open in new tabDownload slide Reconstruction process until the initial placement of recognized fittings. The fitting models were placed with the rough orientation that was determined by PCA. Since the model was placed based on only one of the adjacent pipelines, there was a mismatch with the other adjacent pipeline. Figure 15 shows Fitting A, Fitting B, and Fitting C that were connected with one of the pipelines but had a gap or overlap with another pipeline. Moreover, the model was not perfectly aligned with the scanned point cloud as shown in the case of Fitting D. Figure 15: Open in new tabDownload slide Initial placement results in Fig. 14: gap or overlap between a fitting and the adjacent pipelines, and misalignment between the fitting model and the point cloud. Figure 15: Open in new tabDownload slide Initial placement results in Fig. 14: gap or overlap between a fitting and the adjacent pipelines, and misalignment between the fitting model and the point cloud. The registration process was done to make fitting models to be aligned with the point cloud from the result of initial placement. In Fig. 16, the misalignments between the fitting models and the scanned point cloud are shown before the registration and they are resolved after the process as marked with the yellow boxes. Figure 16: Open in new tabDownload slide Result after the registration: the misalignment of the model to the point cloud resolved. Figure 16: Open in new tabDownload slide Result after the registration: the misalignment of the model to the point cloud resolved. The pipeline adjustment resolved the mismatch at the connection point between the models and adjacent pipeline existed in the result of registration as shown in Fig. 17. The gaps that existed between Fitting A and its adjacent pipelines were resolved by adjusting the nozzle points of the pipelines. The overlaps between the fitting model and the pipelines that were shown in Fitting B, Fitting C, and Fitting D were resolved after the adjustment of the pipelines' nozzle points as shown in the figure. Figure 17: Open in new tabDownload slide Result after the adjustment of the pipeline to the fitting port points: the gap or overlap between the models resolved. Figure 17: Open in new tabDownload slide Result after the adjustment of the pipeline to the fitting port points: the gap or overlap between the models resolved. Figure 18 shows the result after the initial placement. The fitting models were misaligned with the point cloud, and they are shown in detail for Fitting C and Fitting D in Fig. 18a. The nozzle point of the pipeline was mismatched with the fitting port point, as shown in Fig. 18b. Since only one of the fitting port points was connected with the pipeline during the initial placement, the other port points were mismatched with their adjacent pipelines. Figure 18: Open in new tabDownload slide Result after the initial placement of fitting. (a) Rendered with point cloud. (b) Rendered without point cloud. Figure 18: Open in new tabDownload slide Result after the initial placement of fitting. (a) Rendered with point cloud. (b) Rendered without point cloud. Figure 19 shows the result after the registration. As shown in Fig. 19a, the fitting models were aligned with the point cloud, which is different from Fig. 18a. This was done by applying a transformation matrix obtained from the registration process. After the registration, the mismatch between a pipeline nozzle point and a fitting port point occurred on both sides of Fitting C and Fitting B, as shown in Fig. 19b. Figure 19: Open in new tabDownload slide Result after the registration of fitting. (a) Rendered with point cloud. (b) Rendered without point cloud. Figure 19: Open in new tabDownload slide Result after the registration of fitting. (a) Rendered with point cloud. (b) Rendered without point cloud. Figure 20 shows the result after the adjustment of the pipeline nozzle point to the fitting port point. The fittings that were registered to the point cloud in Fig. 19a were unchanged, but the pipeline nozzle points that were mismatched in Fig. 19b were adjusted to the nozzle points of adjacent fittings, as shown in Fig. 20b. The nozzle point of the pipeline was not directly connected to the port point, but its position was adjusted along the axis of the pipeline to resolve the gap or invasion between itself and the fitting port point. Figure 20: Open in new tabDownload slide Result after the registration and pipeline adjustment to the fitting port points. (a) Rendered with point cloud. (b) Rendered without point cloud. Figure 20: Open in new tabDownload slide Result after the registration and pipeline adjustment to the fitting port points. (a) Rendered with point cloud. (b) Rendered without point cloud. 5. Conclusion In this paper, the port point of a fitting is utilized as a secondary reference point, which represents the connection point between a fitting and a pipeline. To use the port point as a reference for the connection point, the fitting is registered to the point cloud by using the state-of-the-art technique. The modified ICP algorithm is used for the registration, which iteratively updates the transformation as maintaining the alignment of fitting's port direction with its connected pipeline's orientation. Then, the nozzle points of the adjacent pipelines are adjusted to resolve the mismatch between the nozzle point and the port point of the fitting. For the adjustment, the nozzle point is moved along the pipeline axis to the projected coordinate of the port point onto the pipeline axis. In this way, the orientation of the detected pipeline is maintained as a primary reference, which represents the overall orientation of the piping system. During the experiment, combining with the recent technique of pipeline detection and fitting recognition, the proposed method improved reconstruction quality at the connection point between pipeline and fitting. The orientation of the piping system obtained from the pipeline axis was preserved, and the registered fitting's port points were only utilized to adjust the inaccurate nozzle points of the detected pipeline. Compared with the use of the nozzle point of a detected pipeline as a reference for the fitting placement and the connection point, the use of registered fitting was more accurate since it used a detailed shape of fitting for finding its position and orientation, and especially the registration was complemented by the use of pipeline orientation during registration. The results of the study are summarized as follows: The proposed method of reconstruction uses port points of the fitting instead of nozzle points of the pipeline for the determination of a connection point between the components. By using the port point of the fitting, it is possible to find a better estimate of the connection point between the fitting and connected pipeline than using the nozzle point of the detected pipeline. Considering port points of registered fitting as a location of the connection point, nozzle points of connected pipelines are adjusted. The prototype system is implemented to verify the method. In the reconstruction result of the piping system, the fittings were correctly aligned with the point clouds, and the pipelines were connected to the fittings without gap or overlap between the models, especially at the connection point. In terms of the limitation, the proposed method is only applicable when a prebuilt library with fitting models in fidelity LoD 3 is prepared. Additionally, the proposed method is dependent on the result of fitting registration. Although the state-of-the-art technique of registration is used in our work, there can be a difficulty due to the difference of shape in the 3D CAD model from the actual point cloud and also noises in the point cloud. In addition, an exact meeting between the pipeline nozzle point and the fitting port point is not achieved yet due to minor errors in orientations of each pipeline. If more than two pipelines are connected to the fitting, the relative orientation and position of each pipeline's axis should be precise to be connected to the fitting exactly. However, the pipelines detected from the scanned point clouds can have minor errors in their orientation since their connection to the fitting is not considered. For future work, this should be solved by the optimization of the pipelines’ axes, which adjusts the axes at minimum, so they can be connected to the fitting port points precisely but do not deviate much from their original axes. References Besl P. J. , McKay N. D. ( 1992 ). Method for registration of 3-D shapes . In Sensor Fusion IV: Control Paradigms and Data Structures (Vol. 1611 , pp. 586 – 606 .). International Society for Optics and Photonics . Google Scholar Crossref Search ADS Google Preview WorldCat COPAC Bosché F. ( 2010 ). Automated recognition of 3D CAD model objects in laser scans and calculation of as-built dimensions for dimensional compliance control in construction . Advanced Engineering Informatics , 24 ( 1 ), 107 – 118 . Google Scholar Crossref Search ADS WorldCat Chida A. , Masuda H. ( 2016 ). Reconstruction of polygonal prisms from point-clouds of engineering facilities . Journal of Computational Design and Engineering , 3 ( 4 ), 322 – 329 . Google Scholar Crossref Search ADS WorldCat Czerniawski T. , Nahangi M., Haas C., Walbridge S. ( 2016 ). Pipe spool recognition in cluttered point clouds using a curvature-based shape descriptor . Automation in Construction , 71 , 346 – 358 . Google Scholar Crossref Search ADS WorldCat Euclidean Cluster Extraction . ( 2020 ). http://pointclouds.org/documentation/tutorials/cluster_extraction.html Accessed 26 July 2020 . FARO Technologies, Inc. PointSense Plant . ( 2019 ). https://knowledge.faro.com/Software/Legacy-Software/Legacy-PointSense_and_CAD_Plugins/PointSense/PointSense_Plant/ Accessed 19 April 2020 . Huang J. , You S. ( 2012 ). Point cloud matching based on 3D self-similarity . In Proceedings of the 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (pp. 41 – 48 .). IEEE . Google Scholar Crossref Search ADS Google Preview WorldCat COPAC Kawashima K. , Kanai S., Date H. ( 2014 ). As-built modeling of piping system from terrestrial laser-scanned point clouds using normal-based region growing . Journal of Computational Design and Engineering , 1 ( 1 ), 13 – 26 . Google Scholar Crossref Search ADS WorldCat Kim H. , Cha M., Kim B. C., Kim T., Mun D. ( 2019 ). Part library-based information retrieval and inspection framework to support part maintenance using 3D printing technology . Rapid Prototyping Journal , 25 , 630 – 644 . Google Scholar Crossref Search ADS WorldCat Kresslein J. , Haghighi P., Park J., Ramnath S., Sutradhar A., Shah J. J. ( 2018 ). Automated cross-sectional shape recovery of 3D branching structures from point cloud . Journal of Computational Design and Engineering , 5 ( 3 ), 368 – 378 . Google Scholar Crossref Search ADS WorldCat Lee H. ( 2019 ). Fidelity levels of 3D point cloud models supporting digital twin . M.Sc. Thesis , Korea Advanced Institute of Science and Technology . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Lee J. , Son H., Kim C., Kim C. ( 2013 ). Skeleton-based 3D reconstruction of as-built pipelines from laser-scan data . Automation in Construction , 35 , 199 – 207 . Google Scholar Crossref Search ADS WorldCat Lee K. W. , Bo P. ( 2016 ). Feature curve extraction from point clouds via developable strip intersection . Journal of Computational Design and Engineering , 3 ( 2 ), 102 – 111 . Google Scholar Crossref Search ADS WorldCat Leica Cyclone 3D Point Cloud Processing Software . ( 2020 ). https://leica-geosystems.com/products/laser-scanners/software/leica-cyclone/ Accessed 19 April 2020 . Liu Y. J. , Zhang J. B., Hou J. C., Ren J. C., Tang W. Q. ( 2013 ). Cylinder detection in large-scale point cloud of pipeline plant . IEEE Transactions on Visualization and Computer Graphics , 19 ( 10 ), 1700 – 1707 . Google Scholar Crossref Search ADS PubMed WorldCat Mineo C. , Pierce S. G., Summan R. ( 2019 ). Novel algorithms for 3D surface point cloud boundary detection and edge reconstruction . Journal of Computational Design and Engineering , 6 ( 1 ), 81 – 91 . Google Scholar Crossref Search ADS WorldCat Pang G. , Neumann U. ( 2013 ). Training-based object recognition in cluttered 3D point clouds . In Proceedings of the 2013 International Conference on 3D Vision-3DV 2013 (pp. 87 – 94 .). IEEE . Google Scholar Crossref Search ADS Google Preview WorldCat COPAC Pang G. , Qiu R., Huang J., You S., Neumann U. ( 2015 ). Automatic 3D industrial point cloud modeling and recognition . In Proceedings of the 2015 14th IAPR International Conference on Machine Vision Applications . (MVA) (pp. 22 – 25 .). IEEE . Google Scholar Crossref Search ADS Google Preview WorldCat COPAC Patil A. K. , Holi P., Lee S. K., Chai Y. H. ( 2017 ). An adaptive approach for the reconstruction and modeling of as-built 3D pipelines from point clouds . Automation in construction , 75 , 65 – 78 . Google Scholar Crossref Search ADS WorldCat Point Cloud Library . ( 2020 ). https://pointclouds.org/ Accessed 19 April 2020 . Qiu R. , Zhou Q. Y., Neumann U. ( 2014 ). Pipe-run extraction and reconstruction from point clouds . In European Conference on Computer Vision (pp. 17 – 30 .). Cham : Springer . Google Scholar Crossref Search ADS Google Preview WorldCat COPAC Rabbani T. , Van Den Heuvel F. ( 2005 ). Efficient hough transform for automatic detection of cylinders in point clouds . ISPRS , Wg III/3, III/4 , 3 , 60 – 65 . Google Scholar OpenURL Placeholder Text WorldCat Schnabel R. , Wahl R., Klein R. ( 2007 ). Efficient RANSAC for point‐cloud shape detection . In Computer Graphics Forum (Vol. 26 , No. 2 , pp. 214 – 226 .). Oxford, UK : Blackwell Publishing Ltd . Google Scholar Crossref Search ADS Google Preview WorldCat COPAC Singh N. , Ray T., Parimi C., Kuchibhotla S. ( 2019 ). Input size independent efficient quality meshing of the interior of 2D point cloud data . Journal of Computational Design and Engineering , 6 ( 3 ), 316 – 326 . Google Scholar Crossref Search ADS WorldCat Son H. , Kim C., Kim C. ( 2014 ). Fully automated as-built 3D pipeline extraction method from laser-scanned data based on curvature computation . Journal of Computing in Civil Engineering , 29 ( 4 ) B4014003 . Google Scholar OpenURL Placeholder Text WorldCat Son H. , Kim C., Kim C. ( 2015 ). 3D reconstruction of as-built industrial instrumentation models from laser-scan data and a 3D CAD database based on prior knowledge . Automation in Construction , 49 , 193 – 200 . Google Scholar Crossref Search ADS WorldCat The Computational Geometry Algorithms Library . ( 2020 ). https://www.cgal.org/ Accessed 19 April 2020 . © The Author(s) 2020. Published by Oxford University Press on behalf of the Society for Computational Design and Engineering. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.
TI - 3D reconstruction of as-built model of plant piping system from point clouds and port information
JO - Journal of Computational Design and Engineering
DO - 10.1093/jcde/qwaa072
DA - 2021-01-25
UR - https://www.deepdyve.com/lp/oxford-university-press/3d-reconstruction-of-as-built-model-of-plant-piping-system-from-point-OTpxUOPtJh
SP - 195
EP - 209
VL - 8
IS - 1
DP - DeepDyve
ER -