Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Automated Classification of Building Roofs for the Updating of 3D Building Models Using Heuristic Methods

Automated Classification of Building Roofs for the Updating of 3D Building Models Using Heuristic... In the Bavarian Surveying Administration, remote sensing methods are applied in the context of nationwide airborne surveys for the acquisition of aerial photographs and airborne laser scanning for the derivation of the digital terrain model (DTM). At the Bavarian Agency for Digitisation, High-Speed Internet and Surveying (LDBV = Landesamt für Digitalisierung, Breitband und Vermessung), image-based digital surface models (bDOM) and digital orthophotos without building lean (trueDOP) are produced using the dense image matching (DIM) method. Buildings and their roofs are displayed in the trueDOP in the correct position to the cadastral ground plan. Based on these data, an expert system was developed for the investigation of construction cases and for the updating of 3D building models, which automatically calculates change notices and makes them available to the Agencies for Digitisation, High-Speed Internet and Surveying (ADBV = Amt für Digitalisierung, Breit- band und Vermessung). The reliable detection of buildings plays a decisive role here. Representative reference classes for the classification of building roofs in the RGB colour space are formed and frequencies are calculated across the boundaries of the photo-flights. The classification is carried out with the aid of a normalized digital surface model (nDOM), which is calculated from the height differences between the bDOM and the DTM, and with heuristically defined threshold values for the colours in the representative RGB colour spaces. The presented method is transferable to all federal states. Keywords LiDAR · LoD2 · Data quality · Classification · Building extraction Zusammenfassung Automatisierte Klassifikation von Gebäudedächern zur Fortführung von 3D-Gebäudemodellen mittels heuristischer Verfahren. In der Bayerischen Vermessungsverwaltung werden Fernerkundungsmethoden im Rahmen der landesweiten Befliegung zur Erfassung von Luftbildern und des Airborne Laserscannings für die Ableitung des Digitalen Geländemodells (DGM) eingesetzt. Im Landesamt für Digitalisierung, Breitband und Vermessung (LDBV) werden bildbasierte Digitale Oberflächenmodelle (bDOM) sowie Digitale Orthophotos ohne Umklappungseffekte von Objekten (trueDOP) mit der Methode des Dense Image Matching (DIM) produziert. Gebäude und deren Dachflächen werden im trueDOP lagerichtig zum Katastergrundriss dargestellt. Auf diesen Datenbeständen wurde ein Expertensystem zur Baufallerkundung sowie zur Fortführung von 3D-Gebäudemodellen entwickelt, das Änderungshinweise automatisiert berechnet und den Ämtern für Digitalisierung, Breitband und Vermessung bereitstellt. Dabei spielt die sichere Erkennung von Gebäuden die maßgebliche Rolle. Es werden über die Grenzen der Bildfluglose hinweg repräsentative Vergleichsklassen zur Klassifikation von Gebäudedächern im RGB-Farbraum gebildet und Häufigkeiten berechnet. Die Klassifikation erfolgt unter Zuhilfenahme eines normalisierten Digitalen Oberflächenmodell (nDOM), das aus den Höhendifferenzen des bDOM zum DGM berechnet wird, * Robert Roschlaub Robert.Roschlaub@ldbv.bayern.de Karin Möst karin.moest@ldbv.bayern.de Thomas Krey thomas.krey@ldbv.bayern.de Bavarian Agency for Digitisation, High-Speed Internet and Surveying, Munich, Germany Vol.:(0123456789) 1 3 86 PFG (2020) 88:85–97 sowie mit heuristisch festgelegter Schwellenwerte für die Farben in den repräsentativen RGB-Farbräumen. Die vorgestellte Methode ist auf alle Bundesländer übertragbar. Schlüsselwörter LiDAR · LoD2 · Datenqualität · Klassifikation · Gebäudeextraktion trueDOP is ideally suited for the nationwide classification 1 Introduction of building roofs. While some publications focus on building recognition According to a resolution of the Working Committee of the including the classification of roof structures from satellite Surveying Authorities of the Laender of the Federal Repub- data or aerial photographs (Alidost 2018; Marmanis et al. lic of Germany (AdV), 3D building models with standard- 2016; Valinger 2015; Shibiao et al. 2018), in Germany this ized roof shapes were recorded throughout Germany on the task can be considered as completed for the LoD2 build- responsibility of the federal states (AdV 2016). The initial ing models. After completion of the initial data capture, the recording of the 3D building models with ten standardized updating of the LoD2 buildings must be ensured (Aringer roof shapes in the so-called Level-of-Detail 2 (LoD2) has and Roschlaub 2013, 2014). The updating of the LoD2 almost been completed nationwide (AdV 2018). The distri- building models is not yet established in all federal states. bution of the LoD2 building models via the Central Author- In Bavaria, for example, the updating is carried out from ity for House Coordinates and House Layouts (ZSHH = Zen- the cadastral two-dimensional ground plans of buildings as trale Stelle Hauskoordinaten und Hausumringe) located in well as the terrestrial three-dimensionally measured special Bavaria could then start in 2020. In some federal states, the building points (ridge and eaves points) and the documented initial capture of LoD2 was carried out by an automated ridge lines. In combination with bDOM or LiDAR data, the derivation, e.g. in North Rhine-Westphalia and Baden-Würt- 3D building models can be updated in a simple way (Hüm- temberg, or by a semi-automatic derivation in which build- mer and Roschlaub 2014). ings are interactively edited, e.g. in Bavaria, Bremen, and In many federal states, there is no obligation to measure Hessen. The classical digital orthophoto (DOP) together or buildings or their modifications. However, to ensure that in combination with point clouds from airborne laser scan- LoD2 continues to be as complete as possible, information ning (LiDAR) or the regular point grid of an image-based on the changes made to the building is required. In the fol- surface model (bDOM) derived with the dense image match- lowing, a procedure is presented that helps federal states ing method serves as the basis for acquisition. According detect changes in buildings from aerial photographs that to a further AdV resolution, by the beginning of 2023 the have not yet been mapped in the cadastral system. classic digital orthophoto (DOP) will be replaced by the true orthophoto (trueDOP) (AdV 2017); see Fig. 1. In those cases where the trueDOP could already be derived from the bDOM by the federal states, the classic DOP was replaced. 2 Colour and Intensity Differences The mapping of buildings and their roofs in a trueDOP at the Borders of the Photo Flight Projects coincides with the cadastral ground plans, although the roof areas may be slightly larger than the building ground plans For the segmentation or classification of objects from aerial in the cadastre depending on the roof overhang. Thus the photographs, imagery that is as unchanged as possible is necessary, because every interference by image processing Fig. 1 a trueDOP with a mapped building floor plan, a ridge line ing) c with LiDAR data (here green points are above the roof of the and ridge and eaves points, b automatically derived LoD2 building garage). The red dots belong to the main building which is not shown displayed together with the grid points of the bDOM (green points, behind it. b, c for visual inspection partly above and partly below the roof of the yellow garage build- 1 3 PFG (2020) 88:85–97 87 Fig. 2 a Colour deviations between the Bavarian photo flight projects. b Section in RGB. c Section in infrared methods leads to a decrease of the initial quality and often to In the mosaics derived from the different epochs of avia- a loss of information. Even in the trueDOP, which is derived tion, unchanged building roofs can appear very die ff rent for from the “Bayernbefliegung” (photo flight of Bavaria) at a the reasons mentioned above. This makes it difficult to classify ground resolution of 20 cm, the underlying images of the objects in the trueDOP mosaic based on colour values and image flight are only very carefully radiometrically pro- requires a representative and comprehensive evaluation of all cessed, so that radiometric differences of varying intensity lots to define a comparison class. can be seen at the lot boundaries of the image flights and flight strips can be seen in the mosaic of the trueDOP (see Fig. 2). Influencing factors are for example: 3 Classification Reference in RGB Colour Cube • Atmospheric light absorption and scattering of light caused by air molecules and particles (aerosols) suspended in the For an automated classification of objects in raster images, air: the greater the flying height, the bigger is the air pack - training areas with the most representative properties of age the light has to pass through. the objects to be classified are always necessary. The nec- • The shadow direction (sun azimuth) that changes during essary training areas for the classification of building roofs the day and the increasing haze that can be counteracted by in a trueDOP mosaic can be easily obtained by intersecting yellow filters in panchromatic images—but not in colour the building ground plans from the official digital cadastral images. map (DFK) with the trueDOP mosaic. The RGB colour • The shade length, which is determined by the elevation of values of all pixels of the trueDOP mosaic lying within the sun. the building ground plans serve as a reference for the auto- • The position of objects in relation to the flight axis, where mated classification of the buildings across the boundaries objects appear darker (backlighting) on the sun-facing of the photo flight projects. image side and brighter (backlighting) on the far side. The pixels of the training areas are now processed as The seasonally different vegetation, which can lead to vio- follows. The number of possible combinations of the grey let undertone for damp soils in spring, while the rather dry values for R, G and B is calculated by multiplying the soils at the photo flight borders at the end of summer often number of grey levels of the individual channels, i.e. at 16 lead to brown-green transitions. Summer sun at noon leads bits 65,536 = 281,474,976,710,656. This number is too to dark drop shadows and harsh contrasts, while a high, large for meaningful further processing. thin cloud layer leads to soft, diminished contrasts with Therefore, the original bandwidth of each individual brightened shadows. colour channel is reduced from 16 bits (0–65,535) to 8 • Motion blur due to different flight speeds. bits (0–255). The number of possible combinations is 1 3 88 PFG (2020) 88:85–97 256 = 16,777,216. Then each pixel of the training areas the significance of the threshold value. For illustration is assigned to one of the combinations and the number per purposes, the RGB colour values of the characteristic roof combination is added up. After the summation, the fre- pixels applied in the three-dimensional RGB colour cube quency of each RGB combination is known. The mask of can be projected into the three colour planes: blue–green, the building floor plans now cuts out not only pixels from blue–red and green–red. In the projected colour planes, too, roofs, but also from overhanging vegetation, especially roofs that occur several times in the RGB colour cube with from trees. Therefore, a method is sought that separates almost identical RGB colour values will appear at the same the roofs from the vegetation. frequency in the three colour planes. As a previous investigation has shown, simple statistical The following images of Fig.  4 show the correspond- tests on the confidence intervals in the colour values of roofs ing point orders of a test tile in the RGB colour cube and hardly allow a separation of vegetation from roofs, since the its projections into the three colour planes in the columns: confidence intervals describe only a cuboid in the RGB col - blue–green, blue–red and green–red. In the left column, the our cube and therefore reflect the characteristics of objects calculations were performed on the pixels of the entire test such as roofs without a sufficient differentiation (Geßler tile; in the middle column, only the roof pixels were exam- et al. 2019; Roschlaub 1992). A better method is based on ined; and in the right column, the pixels resulting from the a representative point cloud of roof pixels, the description difference between the two images were examined. The same of which is not limited to a cuboid, but rather describes the investigations were carried out in a second test tile in a rural envelope of this point cloud. All pixels that lie within this area without visualizing the results here. Figure 4 describes envelope would then represent the colour values characteris- a heavily built-up area of commercial and residential build- tic for roofs and all deviating colour values would represent ings with the following peculiarities: any other object. In this model, however, all cavities in the point cloud enclosed by the envelope are also classified as The point clouds in the RGB cubes are very compact; roofs. However, this must be avoided. there are no outliers outside the point clouds, as the aerial However, previous investigations have shown that such images contain only natural and no synthetic colours. cavities rarely occur, as aerial photographs show quite simi- The point clouds projected into the three colour planes lar colours. This is shown in the illustrations in Figs. 3, 4, 5 have a diagonal characteristic; they differ essentially only and 6, which show the RGB combinations found in the aerial in their basic values with regard to rural and built-up photographs. The value ranges are limited in all cases to a areas. rather compact body in the middle of the value ranges. The The point orders of the buildings (middle column) show a illustrations show one dot per colour combination independ- strong similarity with the point orders of the other object ent of the frequency. pixels (right column), so that a classification of roofs Both methods, the cuboid calculated from confidence from an aerial photograph exclusively on the basis of intervals and the envelope figure enclosing the point cloud, RGB values will be very difficult. do not allow a reliable separation of roof and vegetation. Finally, a method that leads to the goal uses the frequency The more roofs are in a test tile, the more often the of the individual RGB combinations and their distribu- characteristic colour properties of the roofs are ref lected, tion, and separates the roof pixels from the vegetation by due to the standardization to 255 gradations per colour an empirically determined threshold value. Further on, it channel. If, for example, all DFK building layouts for the eliminates outliers. Section 4.1 will particularly focus on whole of Bavaria were blended with the trueDOP mosaic, Fig. 3 a Starting point cloud; b cross-sectional cuboid of the confidence intervals; c triangular meshes of the envelope 1 3 PFG (2020) 88:85–97 89 Fig. 4 Test tile of a built-up area and its column by column consideration of the point orders and frequencies per colour plane 1 3 90 PFG (2020) 88:85–97 Fig. 5 a Distribution in the RGB colour cube for the com- parison classes: roofs, b roofs with a reduced margin 3 m wide In addition to the images to be examined, the original images of the buildings used to define the training areas are also classified to check the quality of the classifica- tion procedure. 4.1 Classification of the trueDOP For further investigations, only the buildings lying in a test tile are used as training areas for a representative reference to RGB colour values of roof pixels. Depending on the selected frequency threshold value, the classification of building objects of a test tile leads to different results, as the follow - Fig. 6 Distribution in the RGB colour cube for vegetation ing pictures of Fig. 7 show: then the colour and intensity differences of the photo When classifying the entire original image and using a threshold value of 1 and 2 (the occurrence of the RGB flight projects would also be taken into account in the training areas of the resulting RGB colour values of the combination), significantly more pixels are classified as supposed buildings than there are actually buildings in roof pixels. The calculation of such a reference data set in the RGB colour cube is very computational intensive, the test tile. This is particularly true for roads that have similar colour values to roofs. but allows the determination of stable and representative reference classes—for example for roofs or vegetation. The other way round, the higher the threshold value, the fewer building pixels are recognized as buildings in the To speed up the classification process, a reduction in the ground resolution of the trueDOP from 20 to 40 cm has original image of the test tile. Thus, at a threshold value of 30, all large factory buildings with a homogeneous proven itself. colour structure are detected, but hardly any residential buildings. Due to the very different roof coverings and the age of the buildings, they have a much more varied colour structure. They are therefore distributed over sev- 4 Classification of the Original Images Based on a Threshold Value for the RGB eral RGB combinations close to each other. Reference Colour Cube An arbitrarily chosen section of the test tile from Fig. 7 illustrates this in Fig. 8. In the left column, only the pixels For the development of a reference class for the auto- mated classification of buildings, only the RGB colour classified within the cut building ground plans are plotted and placed over the initial image, which result from the use values of the roof pixels and the resulting frequencies in the RGB colour cube are considered first. The frequency of the respective threshold value. It becomes clear that the higher the selected threshold value, the less of the pixels of each of the RGB combinations represents a fourth dimension besides the values for R, G and B, and there- originally located within the building perimeters are reclas- sified as building pixels. In the right column, the classifica- fore cannot be visualized. By defining a threshold value, it is possible to classify building objects in each image. tion procedure was applied not only to the buildings, but also 1 3 PFG (2020) 88:85–97 91 Fig. 7 Classified initial image (trueDOP) with 1-, 2-, 8- and 30-fold frequency as black pixels and with the building ground plans of the DFK superimposed in blue to the entire initial image. In the right column, the section from the difference between the bDOM and the DTM. Mis- shows that the classification procedure distinguishes veg - classifications of the road space in a trueDOP can be easily etation very well from buildings and that there are hardly avoided by using the nDOM, in which only those pixels are any misclassifications between vegetation and buildings. At considered which lie above a minimum height of e.g. 2.30 m the same time, as already mentioned, misinterpretations of (see Fig. 9). This would only classify bridges as buildings pixels occur outside the building ground plans, especially in the street space that are also of importance as structure. in the street space. At the same time, the use of an nDOM accelerates the clas- sification of roofs in the trueDOP to a very considerable 4.2 Calculation of an nDOM Mask for trueDOP degree, because only those image sections which are above Classification the selected minimum height in the nDOM are considered to be classified. This means that a significantly smaller part The normalized digital surface model (nDOM) forms the of the original image is subjected to classification and the basis for the calculation of a mask for the classification of scope of classification is reduced accordingly. the trueDOP. It contains height values that are calculated 1 3 92 PFG (2020) 88:85–97 Fig. 8 Classification of a section of the initial image (trueDOP) of Fig.  7 with 1, 4 and 15 times the frequency: applied exclusively to the build- ings (left column) as well as to the entire corresponding section (right column) Fig. 9 a Section: nDOM mask with superimposed DFK with the background of the trueDOP, in which vegetation is still occasionally contained; classification of trueDOP covered by the nDOM mask with a frequency threshold of 2 b and 4 c; roads are no longer recognized as buildings The use of data in binary format is essential for the pro- To accelerate the classification of the trueDOP, the cessing routines of large amounts of data. For example, nDOM is placed as a mask over the trueDOP. The true- LAStools (rapidlasso GmbH) are available to the LDBV. DOP is then cut accordingly—for example with the FME LAStools are suitable for extremely high-performance software (Safe Software Inc.)—and only the pixels of the processing of point clouds. For large parts of Bavaria, the trueDOP superimposed by the nDOM are considered for nDOM can be calculated very efficiently with them. After further processing. calculating the nDOM with LAStools, other softwares must be used for the image interpretation. 1 3 PFG (2020) 88:85–97 93 channel, produces significantly higher frequencies for the 5 Transfer to a Larger Test Area colour values of the roofs compared to the previous investi- gations, which were limited to one test tile. The frequencies To generalize the previous tests, the RGB colour cube is of identical RGB colour values for the 16 million (exactly recalculated for all roofs in an extended test area. The test 16,777,216) different RGB value combinations are now on area covers 21,000 km , which corresponds to approxi- average 350, minimum 1 and maximum 350,000. mately a third of the Bavarian state area. For further inves- If the classification for a test tile with the RGB colour tigations, three reference classes are calculated—the first cube calculated for a third of Bavaria and a threshold value two for the classification of the roofs and the other for of 250 is applied to the trueDOP covered by the nDOM the determination of the vegetation. In the following, the mask, trees and bridges are still classified as roofs in the calculation of the comparison classes is explained and the built-up area (see Fig.  10). A threshold value of 1000 achieved classification results are presented. reduces the misclassification of vegetation as roofs. With an even higher threshold value of 3000, building detection 5.1 T hreshold Value for Building Classification is also significantly reduced. Using the RGB Reference Colour Cube for a Third The pictures in Fig. 10 show that the determination of of Bavaria a suitable threshold value has a considerable influence on the quality of the classification and is one of the challenges To classify the roofs, the reference class “roofs” is deter- in this classification procedure. A threshold value that is mined from the superposition of the DFK with the trueDOP. too high leads to a reduction in the number of roofs to be On the other hand, a comparative class “roofs with a reduced identified; a threshold value that is too low leads to higher margin” of 3 m width is calculated to minimize the influence misinterpretations. of overhanging trees (see Fig. 5). The misclassification of trees as roofs is mainly due to Due to the significantly larger data volume, the RGB the fact that the building roofs considered for determining colour cube, which is limited to 255 grey levels per colour the RGB colour cube in trueDOP contain many RGB colour Fig. 10 a nDOM mask with two new buildings in the upper right cor- the vegetation is only slightly reduced. c At a threshold value of 1000, ner, whose buildings are not measured in the DFK and whose RGB new buildings are not detected, but the misclassification of vegetation values are not in the RGB colour cube. b Classification of the RGB decreases significantly. d At a threshold value of 3000, many roofs values of the roofs in the trueDOP taking into account the nDOM are no longer detected mask with a threshold value of 250. New buildings are detected, but 1 3 94 PFG (2020) 88:85–97 Fig. 11 Differences in the classification of roofs with and without margin. a The colour pixels of the roofs classified without margin in trueDOP are yellow; c the colour pixels of roofs calculated with a negative margin are green. b The overlay of both images values of the vegetation, as of trees that rise above the roofs. In Fig. 12b only the vegetation is obtained as a result, with These vegetation components distort the reference for build- the exception of a few new buildings which still remain in ing roofs in the RGB colour cube, so that in the subsequent the clipped nDOM mask. classification of the trueDOP mask covering the nDOM It is not to be expected that many unmeasured new build- vegetation components are wrongly classified as building ings will appear over a large data set, which would signifi- roofs. Even if an inner margin of 3 m is applied to the ground cantly distort the reference of the RGB colour cube for the plans and only the RGB values of the “inner” roofs are used vegetation. However, due to the selected height threshold to determine the RGB colour cube, there are no significant of ± 2.30 m, the bridges in the nDOM mask remain when differences (see Fig.  11). Especially in the shadow areas, the determining the reference class for the vegetation and are colour values of the vegetation and the dark roofs are similar. included in the reference of the RGB colour cube as a source of error. The average number of identical RGB colour values 5.2 Indirect Building Classification Using an RGB for the vegetation class of the 2.7 million different RGB Reference Colour Cube for Vegetation values is 13,453. The maximum value is 2.3 million. This occurs in the shadow area with almost black colour. To avoid misinterpretations, a conceptual change of the clas- Once the reference of the RGB colour cube for the veg- sification procedure takes place, in which the RGB values for etation has been calculated, the entire trueDOP is classi- the vegetation are determined instead of the building roofs. fied according to a predetermined threshold value. Then However, this is much more computationally intensive for all vegetation pixels lying on the nDOM mask are sub- the creation of the RGB reference colour cube, since the tracted. The remaining pixels of the nDOM mask repre- aerial photographs contain considerably more vegetation sent the searched roofs (see Fig. 13). As a generalization, areas than roof areas. The nDOM mask of Fig. 10 continues by subtracting the vegetation from the nDOM, all those to serve as the basis for calculation. The building ground objects are obtained that have the same colour as building plans of the DFK are cut out of the nDOM mask. Due to the roofs (Figs. 14, 15). fact that the roof overhangs were not measured by cadastral Roofs, for example of garages completely covered by survey, the outlines of the building were extended to the trees, cannot be identified as roofs by the removal of veg- outside with a margin of 3 m. Thus, it is almost guaranteed etation in the nDOM mask. However, elsewhere something that hardly any roof pixels are still contained in the database. of the actual vegetation is filtered out of the nDOM mask. Fig. 12 a nDOM vegetation mask clipped with the building ground plans; b nDOM vegetation mask clipped with a margin of 3 m 1 3 PFG (2020) 88:85–97 95 Fig. 13 With respect to nDOM mask Fig.  10a, classified trueDOP from the nDOM mask, the remaining building roofs or reduced veg- with a threshold of a 1000, b 5000 and c 30,000 with underlying etation (right column), respectively DOM mask (left column); after deduction of all vegetation pixels Fig. 14 a An unclassified trueDOP; b trueDOP classified with a as a vegetation point and was erroneously removed from the mask in threshold of 7000; c trueDOP classified with a threshold of 7000, the middle image. In the right picture the mask has not deteriorated at with white values calculated out. The white solar roof was detected this point In addition, the lower the threshold value selected, the actually existing vegetation. Conversely, the higher the larger the range that is classified in trueDOP, so that the threshold value, the less are the changes of the nDOM nDOM mask can filter out a corresponding amount of the mask. 1 3 96 PFG (2020) 88:85–97 Fig. 15 a Found vegetation points without optimization; b found vegetation points without white values (middle); c and found vegetation values whose sum is R + G + B < 700 (right) To further optimize the vegetation classification, very – shorter calculation times, if a classification only bright values can be subtracted from the vegetation colour takes place on the nDOM mask; cube. White, almost white and grey colour values often orig- – a significantly improved recognition rate of the roofs inate from modifications of the Earth’s surface such as gravel of new buildings; pits and the like. Bridges with their grey values are also – a reduction of vegetation with a suitable threshold largely contained in the nDOM. These colour values occur value; and very frequently and would be detected as vegetation points – no holes in the roofs of the nDOM mask. and thus removed from the mask. This must be prevented to • Potential to avoid misinterpretations exists if the bridges find white and grey roof surfaces. in the nDOM mask could be removed. For this purpose, Technically, this is ensured by not only considering the the bridge objects from the digital landscape model of white pixels with the RGB values (255,255,255), but also the Authoritative Topographic-Cartographic Information all pixels whose sum is R + G + B > 700 as vegetation points. System (ATKIS) would have to be available geometri- The disadvantage of this optimization is that gravel pits cally exact as polygons. remain in the mask. However, they can easily be recognized • What remains unresolved is the heuristic definition of as misinterpretation. threshold values, which cannot be standardized and which probably has to be determined individually for each photo flight project. 6 Conclusion and Outlook The presented classification procedure is transferable to The investigations with the extended RGB colour cubes for all federal states and can be used nationwide as soon as the a third of Bavaria show the following results: federal states have processed the trueDOP. In Bavaria, the classified building roofs of the new buildings are transmit- • The classification of the roofs using the two RGB roof ted to ten Agencies for Digitisation, High-Speed Internet colour cubes “with and without them”; results in and Surveying in order to determine buildings that have not – short calculation times when creating the reference been surveyed in cadastral terms and used to update the real classes, since the evaluation only takes place within estate cadastre and thus also to update the LoD2 building the DFK floor plans; models. Further investigations using AI methods will follow – shorter calculation times, if a classification only to further develop and optimize the results and processes. takes place on the nDOM mask; Acknowledgements Open Access funding provided by Projekt DEAL. – no significant differences in the recognition of build- ing roofs in the evaluation with and without them; Open Access This article is licensed under a Creative Commons Attri- – no significant reduction of vegetation in the nDOM bution 4.0 International License, which permits use, sharing, adapta- mask; and tion, distribution and reproduction in any medium or format, as long – holes in the roofs, by an incomplete classification of as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes the roofs in the trueDOP. were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated • With the indirect classification of the roofs via the RGB otherwise in a credit line to the material. If material is not included in vegetation colour cube, on the other hand, the following the article’s Creative Commons licence and your intended use is not results are obtained: permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativ ecommons .or g/licenses/b y/4.0/. – very high calculation times when creating the refer- ence classes; 1 3 PFG (2020) 88:85–97 97 information. In: Iskidag U (ed) Lecture notes in geoinformation References and cartography, innovations in 3D geo-information sciences. Springer International Publishing, Switzerland, pp 143–157 AdV (2016) Plenumsbeschluss 128/6, 128. Tagung des Plenums der Geßler S, Krey T, Möst K, Roschlaub R (2019) Mit Datenfusionierung Arbeitsgemeinschaft der Vermessungsverwaltungen der Länder Mehrwerte schaffen—Ein Expertensystem zur Baufallerkundung. der Bundesrepublik Deutschland, Sep 2016 in Bad Neuenahr- DVW-Mitteilungen 2(2019):159–187 Ahrweiler, Germany Hümmer F, Roschlaub R (2014) Die Zukunft ist dreidimen- AdV (2017) AK GT Beschluss 30/2, zur Überführung des ATKIS- sional—3D-Gebäudemodelle in Bayern. DVW-Mitteilungen DOP20 in die Qualitätsstufe TrueDOP, Apr 2017 in Saarlouis, 2(2014):165–176 Germany Roschlaub R (1992) Berechnung von Quantilen verschiedener AdV (2018) Produktstandardblätter mit Stand vom 31.12.2018: statistischer Verteilungen. Zeitschrift für Vermessungswesen https://www .adv-online.de/A dV-Produkte/S tandar ds-und-Produkt 117(6):323–335 bla etter /Produ ktbla etter /. Accessed 23 Jul 2019 Shibiao X, Xingjia P, Er L, Baoyuan W, Shuhui B, Weiming D, Shim- Alidoost F, Arefi H (2018) A CNN-based approach for automatic build- ing X, Xiaopeng Z (2018) Automatic building rooftop extraction ing detection and recognition of roof types using a single arial from aerial images via hierarchical RGB-D priors. IEEE Trans image. PFG J Photogramm Remote Sens Geoinf Sci 86:235–248 Geosci Remote Sens 56(12):7369–7387 Aringer K, Roschlaub R (2013) Calculation and update of a 3D build- Valinger J (2015) Automatic rooftop segment extraction using point ing model of Bavaria using Lidar, image matching and cadaster clouds generated from aerial high resolution photography, MSc information. In: ISPRS Annals of the photogrammetry, remote thesis, Umeå University, Faculty of Science and Technology, sensing and spatial information sciences, Vol II-2/W1, ISPRS 8th Department of Computing Science. https ://pdf s.seman ticsc 3DGeoInfo Conference & WG II/2 Workshop, 27–29 Nov 2013, holar .org/4a08/d12b5 49f46 85d85 9c643 2ae26 db930 b5c79 6.pdf. Istanbul, Turkey, pp. 7–12 Accessed 23 Jul 2019 Aringer K, Roschlaub R (2014) Bavarian 3D building model and update concept based on LiDAR, image matching and cadaster 1 3 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png PFG – Journal of Photogrammetry, Remote Sensing and Geoinformation Science Springer Journals

Automated Classification of Building Roofs for the Updating of 3D Building Models Using Heuristic Methods

Loading next page...
 
/lp/springer-journals/automated-classification-of-building-roofs-for-the-updating-of-3d-xA9lUbqWUJ
Publisher
Springer Journals
Copyright
Copyright © The Author(s) 2020
ISSN
2512-2789
eISSN
2512-2819
DOI
10.1007/s41064-020-00099-9
Publisher site
See Article on Publisher Site

Abstract

In the Bavarian Surveying Administration, remote sensing methods are applied in the context of nationwide airborne surveys for the acquisition of aerial photographs and airborne laser scanning for the derivation of the digital terrain model (DTM). At the Bavarian Agency for Digitisation, High-Speed Internet and Surveying (LDBV = Landesamt für Digitalisierung, Breitband und Vermessung), image-based digital surface models (bDOM) and digital orthophotos without building lean (trueDOP) are produced using the dense image matching (DIM) method. Buildings and their roofs are displayed in the trueDOP in the correct position to the cadastral ground plan. Based on these data, an expert system was developed for the investigation of construction cases and for the updating of 3D building models, which automatically calculates change notices and makes them available to the Agencies for Digitisation, High-Speed Internet and Surveying (ADBV = Amt für Digitalisierung, Breit- band und Vermessung). The reliable detection of buildings plays a decisive role here. Representative reference classes for the classification of building roofs in the RGB colour space are formed and frequencies are calculated across the boundaries of the photo-flights. The classification is carried out with the aid of a normalized digital surface model (nDOM), which is calculated from the height differences between the bDOM and the DTM, and with heuristically defined threshold values for the colours in the representative RGB colour spaces. The presented method is transferable to all federal states. Keywords LiDAR · LoD2 · Data quality · Classification · Building extraction Zusammenfassung Automatisierte Klassifikation von Gebäudedächern zur Fortführung von 3D-Gebäudemodellen mittels heuristischer Verfahren. In der Bayerischen Vermessungsverwaltung werden Fernerkundungsmethoden im Rahmen der landesweiten Befliegung zur Erfassung von Luftbildern und des Airborne Laserscannings für die Ableitung des Digitalen Geländemodells (DGM) eingesetzt. Im Landesamt für Digitalisierung, Breitband und Vermessung (LDBV) werden bildbasierte Digitale Oberflächenmodelle (bDOM) sowie Digitale Orthophotos ohne Umklappungseffekte von Objekten (trueDOP) mit der Methode des Dense Image Matching (DIM) produziert. Gebäude und deren Dachflächen werden im trueDOP lagerichtig zum Katastergrundriss dargestellt. Auf diesen Datenbeständen wurde ein Expertensystem zur Baufallerkundung sowie zur Fortführung von 3D-Gebäudemodellen entwickelt, das Änderungshinweise automatisiert berechnet und den Ämtern für Digitalisierung, Breitband und Vermessung bereitstellt. Dabei spielt die sichere Erkennung von Gebäuden die maßgebliche Rolle. Es werden über die Grenzen der Bildfluglose hinweg repräsentative Vergleichsklassen zur Klassifikation von Gebäudedächern im RGB-Farbraum gebildet und Häufigkeiten berechnet. Die Klassifikation erfolgt unter Zuhilfenahme eines normalisierten Digitalen Oberflächenmodell (nDOM), das aus den Höhendifferenzen des bDOM zum DGM berechnet wird, * Robert Roschlaub Robert.Roschlaub@ldbv.bayern.de Karin Möst karin.moest@ldbv.bayern.de Thomas Krey thomas.krey@ldbv.bayern.de Bavarian Agency for Digitisation, High-Speed Internet and Surveying, Munich, Germany Vol.:(0123456789) 1 3 86 PFG (2020) 88:85–97 sowie mit heuristisch festgelegter Schwellenwerte für die Farben in den repräsentativen RGB-Farbräumen. Die vorgestellte Methode ist auf alle Bundesländer übertragbar. Schlüsselwörter LiDAR · LoD2 · Datenqualität · Klassifikation · Gebäudeextraktion trueDOP is ideally suited for the nationwide classification 1 Introduction of building roofs. While some publications focus on building recognition According to a resolution of the Working Committee of the including the classification of roof structures from satellite Surveying Authorities of the Laender of the Federal Repub- data or aerial photographs (Alidost 2018; Marmanis et al. lic of Germany (AdV), 3D building models with standard- 2016; Valinger 2015; Shibiao et al. 2018), in Germany this ized roof shapes were recorded throughout Germany on the task can be considered as completed for the LoD2 build- responsibility of the federal states (AdV 2016). The initial ing models. After completion of the initial data capture, the recording of the 3D building models with ten standardized updating of the LoD2 buildings must be ensured (Aringer roof shapes in the so-called Level-of-Detail 2 (LoD2) has and Roschlaub 2013, 2014). The updating of the LoD2 almost been completed nationwide (AdV 2018). The distri- building models is not yet established in all federal states. bution of the LoD2 building models via the Central Author- In Bavaria, for example, the updating is carried out from ity for House Coordinates and House Layouts (ZSHH = Zen- the cadastral two-dimensional ground plans of buildings as trale Stelle Hauskoordinaten und Hausumringe) located in well as the terrestrial three-dimensionally measured special Bavaria could then start in 2020. In some federal states, the building points (ridge and eaves points) and the documented initial capture of LoD2 was carried out by an automated ridge lines. In combination with bDOM or LiDAR data, the derivation, e.g. in North Rhine-Westphalia and Baden-Würt- 3D building models can be updated in a simple way (Hüm- temberg, or by a semi-automatic derivation in which build- mer and Roschlaub 2014). ings are interactively edited, e.g. in Bavaria, Bremen, and In many federal states, there is no obligation to measure Hessen. The classical digital orthophoto (DOP) together or buildings or their modifications. However, to ensure that in combination with point clouds from airborne laser scan- LoD2 continues to be as complete as possible, information ning (LiDAR) or the regular point grid of an image-based on the changes made to the building is required. In the fol- surface model (bDOM) derived with the dense image match- lowing, a procedure is presented that helps federal states ing method serves as the basis for acquisition. According detect changes in buildings from aerial photographs that to a further AdV resolution, by the beginning of 2023 the have not yet been mapped in the cadastral system. classic digital orthophoto (DOP) will be replaced by the true orthophoto (trueDOP) (AdV 2017); see Fig. 1. In those cases where the trueDOP could already be derived from the bDOM by the federal states, the classic DOP was replaced. 2 Colour and Intensity Differences The mapping of buildings and their roofs in a trueDOP at the Borders of the Photo Flight Projects coincides with the cadastral ground plans, although the roof areas may be slightly larger than the building ground plans For the segmentation or classification of objects from aerial in the cadastre depending on the roof overhang. Thus the photographs, imagery that is as unchanged as possible is necessary, because every interference by image processing Fig. 1 a trueDOP with a mapped building floor plan, a ridge line ing) c with LiDAR data (here green points are above the roof of the and ridge and eaves points, b automatically derived LoD2 building garage). The red dots belong to the main building which is not shown displayed together with the grid points of the bDOM (green points, behind it. b, c for visual inspection partly above and partly below the roof of the yellow garage build- 1 3 PFG (2020) 88:85–97 87 Fig. 2 a Colour deviations between the Bavarian photo flight projects. b Section in RGB. c Section in infrared methods leads to a decrease of the initial quality and often to In the mosaics derived from the different epochs of avia- a loss of information. Even in the trueDOP, which is derived tion, unchanged building roofs can appear very die ff rent for from the “Bayernbefliegung” (photo flight of Bavaria) at a the reasons mentioned above. This makes it difficult to classify ground resolution of 20 cm, the underlying images of the objects in the trueDOP mosaic based on colour values and image flight are only very carefully radiometrically pro- requires a representative and comprehensive evaluation of all cessed, so that radiometric differences of varying intensity lots to define a comparison class. can be seen at the lot boundaries of the image flights and flight strips can be seen in the mosaic of the trueDOP (see Fig. 2). Influencing factors are for example: 3 Classification Reference in RGB Colour Cube • Atmospheric light absorption and scattering of light caused by air molecules and particles (aerosols) suspended in the For an automated classification of objects in raster images, air: the greater the flying height, the bigger is the air pack - training areas with the most representative properties of age the light has to pass through. the objects to be classified are always necessary. The nec- • The shadow direction (sun azimuth) that changes during essary training areas for the classification of building roofs the day and the increasing haze that can be counteracted by in a trueDOP mosaic can be easily obtained by intersecting yellow filters in panchromatic images—but not in colour the building ground plans from the official digital cadastral images. map (DFK) with the trueDOP mosaic. The RGB colour • The shade length, which is determined by the elevation of values of all pixels of the trueDOP mosaic lying within the sun. the building ground plans serve as a reference for the auto- • The position of objects in relation to the flight axis, where mated classification of the buildings across the boundaries objects appear darker (backlighting) on the sun-facing of the photo flight projects. image side and brighter (backlighting) on the far side. The pixels of the training areas are now processed as The seasonally different vegetation, which can lead to vio- follows. The number of possible combinations of the grey let undertone for damp soils in spring, while the rather dry values for R, G and B is calculated by multiplying the soils at the photo flight borders at the end of summer often number of grey levels of the individual channels, i.e. at 16 lead to brown-green transitions. Summer sun at noon leads bits 65,536 = 281,474,976,710,656. This number is too to dark drop shadows and harsh contrasts, while a high, large for meaningful further processing. thin cloud layer leads to soft, diminished contrasts with Therefore, the original bandwidth of each individual brightened shadows. colour channel is reduced from 16 bits (0–65,535) to 8 • Motion blur due to different flight speeds. bits (0–255). The number of possible combinations is 1 3 88 PFG (2020) 88:85–97 256 = 16,777,216. Then each pixel of the training areas the significance of the threshold value. For illustration is assigned to one of the combinations and the number per purposes, the RGB colour values of the characteristic roof combination is added up. After the summation, the fre- pixels applied in the three-dimensional RGB colour cube quency of each RGB combination is known. The mask of can be projected into the three colour planes: blue–green, the building floor plans now cuts out not only pixels from blue–red and green–red. In the projected colour planes, too, roofs, but also from overhanging vegetation, especially roofs that occur several times in the RGB colour cube with from trees. Therefore, a method is sought that separates almost identical RGB colour values will appear at the same the roofs from the vegetation. frequency in the three colour planes. As a previous investigation has shown, simple statistical The following images of Fig.  4 show the correspond- tests on the confidence intervals in the colour values of roofs ing point orders of a test tile in the RGB colour cube and hardly allow a separation of vegetation from roofs, since the its projections into the three colour planes in the columns: confidence intervals describe only a cuboid in the RGB col - blue–green, blue–red and green–red. In the left column, the our cube and therefore reflect the characteristics of objects calculations were performed on the pixels of the entire test such as roofs without a sufficient differentiation (Geßler tile; in the middle column, only the roof pixels were exam- et al. 2019; Roschlaub 1992). A better method is based on ined; and in the right column, the pixels resulting from the a representative point cloud of roof pixels, the description difference between the two images were examined. The same of which is not limited to a cuboid, but rather describes the investigations were carried out in a second test tile in a rural envelope of this point cloud. All pixels that lie within this area without visualizing the results here. Figure 4 describes envelope would then represent the colour values characteris- a heavily built-up area of commercial and residential build- tic for roofs and all deviating colour values would represent ings with the following peculiarities: any other object. In this model, however, all cavities in the point cloud enclosed by the envelope are also classified as The point clouds in the RGB cubes are very compact; roofs. However, this must be avoided. there are no outliers outside the point clouds, as the aerial However, previous investigations have shown that such images contain only natural and no synthetic colours. cavities rarely occur, as aerial photographs show quite simi- The point clouds projected into the three colour planes lar colours. This is shown in the illustrations in Figs. 3, 4, 5 have a diagonal characteristic; they differ essentially only and 6, which show the RGB combinations found in the aerial in their basic values with regard to rural and built-up photographs. The value ranges are limited in all cases to a areas. rather compact body in the middle of the value ranges. The The point orders of the buildings (middle column) show a illustrations show one dot per colour combination independ- strong similarity with the point orders of the other object ent of the frequency. pixels (right column), so that a classification of roofs Both methods, the cuboid calculated from confidence from an aerial photograph exclusively on the basis of intervals and the envelope figure enclosing the point cloud, RGB values will be very difficult. do not allow a reliable separation of roof and vegetation. Finally, a method that leads to the goal uses the frequency The more roofs are in a test tile, the more often the of the individual RGB combinations and their distribu- characteristic colour properties of the roofs are ref lected, tion, and separates the roof pixels from the vegetation by due to the standardization to 255 gradations per colour an empirically determined threshold value. Further on, it channel. If, for example, all DFK building layouts for the eliminates outliers. Section 4.1 will particularly focus on whole of Bavaria were blended with the trueDOP mosaic, Fig. 3 a Starting point cloud; b cross-sectional cuboid of the confidence intervals; c triangular meshes of the envelope 1 3 PFG (2020) 88:85–97 89 Fig. 4 Test tile of a built-up area and its column by column consideration of the point orders and frequencies per colour plane 1 3 90 PFG (2020) 88:85–97 Fig. 5 a Distribution in the RGB colour cube for the com- parison classes: roofs, b roofs with a reduced margin 3 m wide In addition to the images to be examined, the original images of the buildings used to define the training areas are also classified to check the quality of the classifica- tion procedure. 4.1 Classification of the trueDOP For further investigations, only the buildings lying in a test tile are used as training areas for a representative reference to RGB colour values of roof pixels. Depending on the selected frequency threshold value, the classification of building objects of a test tile leads to different results, as the follow - Fig. 6 Distribution in the RGB colour cube for vegetation ing pictures of Fig. 7 show: then the colour and intensity differences of the photo When classifying the entire original image and using a threshold value of 1 and 2 (the occurrence of the RGB flight projects would also be taken into account in the training areas of the resulting RGB colour values of the combination), significantly more pixels are classified as supposed buildings than there are actually buildings in roof pixels. The calculation of such a reference data set in the RGB colour cube is very computational intensive, the test tile. This is particularly true for roads that have similar colour values to roofs. but allows the determination of stable and representative reference classes—for example for roofs or vegetation. The other way round, the higher the threshold value, the fewer building pixels are recognized as buildings in the To speed up the classification process, a reduction in the ground resolution of the trueDOP from 20 to 40 cm has original image of the test tile. Thus, at a threshold value of 30, all large factory buildings with a homogeneous proven itself. colour structure are detected, but hardly any residential buildings. Due to the very different roof coverings and the age of the buildings, they have a much more varied colour structure. They are therefore distributed over sev- 4 Classification of the Original Images Based on a Threshold Value for the RGB eral RGB combinations close to each other. Reference Colour Cube An arbitrarily chosen section of the test tile from Fig. 7 illustrates this in Fig. 8. In the left column, only the pixels For the development of a reference class for the auto- mated classification of buildings, only the RGB colour classified within the cut building ground plans are plotted and placed over the initial image, which result from the use values of the roof pixels and the resulting frequencies in the RGB colour cube are considered first. The frequency of the respective threshold value. It becomes clear that the higher the selected threshold value, the less of the pixels of each of the RGB combinations represents a fourth dimension besides the values for R, G and B, and there- originally located within the building perimeters are reclas- sified as building pixels. In the right column, the classifica- fore cannot be visualized. By defining a threshold value, it is possible to classify building objects in each image. tion procedure was applied not only to the buildings, but also 1 3 PFG (2020) 88:85–97 91 Fig. 7 Classified initial image (trueDOP) with 1-, 2-, 8- and 30-fold frequency as black pixels and with the building ground plans of the DFK superimposed in blue to the entire initial image. In the right column, the section from the difference between the bDOM and the DTM. Mis- shows that the classification procedure distinguishes veg - classifications of the road space in a trueDOP can be easily etation very well from buildings and that there are hardly avoided by using the nDOM, in which only those pixels are any misclassifications between vegetation and buildings. At considered which lie above a minimum height of e.g. 2.30 m the same time, as already mentioned, misinterpretations of (see Fig. 9). This would only classify bridges as buildings pixels occur outside the building ground plans, especially in the street space that are also of importance as structure. in the street space. At the same time, the use of an nDOM accelerates the clas- sification of roofs in the trueDOP to a very considerable 4.2 Calculation of an nDOM Mask for trueDOP degree, because only those image sections which are above Classification the selected minimum height in the nDOM are considered to be classified. This means that a significantly smaller part The normalized digital surface model (nDOM) forms the of the original image is subjected to classification and the basis for the calculation of a mask for the classification of scope of classification is reduced accordingly. the trueDOP. It contains height values that are calculated 1 3 92 PFG (2020) 88:85–97 Fig. 8 Classification of a section of the initial image (trueDOP) of Fig.  7 with 1, 4 and 15 times the frequency: applied exclusively to the build- ings (left column) as well as to the entire corresponding section (right column) Fig. 9 a Section: nDOM mask with superimposed DFK with the background of the trueDOP, in which vegetation is still occasionally contained; classification of trueDOP covered by the nDOM mask with a frequency threshold of 2 b and 4 c; roads are no longer recognized as buildings The use of data in binary format is essential for the pro- To accelerate the classification of the trueDOP, the cessing routines of large amounts of data. For example, nDOM is placed as a mask over the trueDOP. The true- LAStools (rapidlasso GmbH) are available to the LDBV. DOP is then cut accordingly—for example with the FME LAStools are suitable for extremely high-performance software (Safe Software Inc.)—and only the pixels of the processing of point clouds. For large parts of Bavaria, the trueDOP superimposed by the nDOM are considered for nDOM can be calculated very efficiently with them. After further processing. calculating the nDOM with LAStools, other softwares must be used for the image interpretation. 1 3 PFG (2020) 88:85–97 93 channel, produces significantly higher frequencies for the 5 Transfer to a Larger Test Area colour values of the roofs compared to the previous investi- gations, which were limited to one test tile. The frequencies To generalize the previous tests, the RGB colour cube is of identical RGB colour values for the 16 million (exactly recalculated for all roofs in an extended test area. The test 16,777,216) different RGB value combinations are now on area covers 21,000 km , which corresponds to approxi- average 350, minimum 1 and maximum 350,000. mately a third of the Bavarian state area. For further inves- If the classification for a test tile with the RGB colour tigations, three reference classes are calculated—the first cube calculated for a third of Bavaria and a threshold value two for the classification of the roofs and the other for of 250 is applied to the trueDOP covered by the nDOM the determination of the vegetation. In the following, the mask, trees and bridges are still classified as roofs in the calculation of the comparison classes is explained and the built-up area (see Fig.  10). A threshold value of 1000 achieved classification results are presented. reduces the misclassification of vegetation as roofs. With an even higher threshold value of 3000, building detection 5.1 T hreshold Value for Building Classification is also significantly reduced. Using the RGB Reference Colour Cube for a Third The pictures in Fig. 10 show that the determination of of Bavaria a suitable threshold value has a considerable influence on the quality of the classification and is one of the challenges To classify the roofs, the reference class “roofs” is deter- in this classification procedure. A threshold value that is mined from the superposition of the DFK with the trueDOP. too high leads to a reduction in the number of roofs to be On the other hand, a comparative class “roofs with a reduced identified; a threshold value that is too low leads to higher margin” of 3 m width is calculated to minimize the influence misinterpretations. of overhanging trees (see Fig. 5). The misclassification of trees as roofs is mainly due to Due to the significantly larger data volume, the RGB the fact that the building roofs considered for determining colour cube, which is limited to 255 grey levels per colour the RGB colour cube in trueDOP contain many RGB colour Fig. 10 a nDOM mask with two new buildings in the upper right cor- the vegetation is only slightly reduced. c At a threshold value of 1000, ner, whose buildings are not measured in the DFK and whose RGB new buildings are not detected, but the misclassification of vegetation values are not in the RGB colour cube. b Classification of the RGB decreases significantly. d At a threshold value of 3000, many roofs values of the roofs in the trueDOP taking into account the nDOM are no longer detected mask with a threshold value of 250. New buildings are detected, but 1 3 94 PFG (2020) 88:85–97 Fig. 11 Differences in the classification of roofs with and without margin. a The colour pixels of the roofs classified without margin in trueDOP are yellow; c the colour pixels of roofs calculated with a negative margin are green. b The overlay of both images values of the vegetation, as of trees that rise above the roofs. In Fig. 12b only the vegetation is obtained as a result, with These vegetation components distort the reference for build- the exception of a few new buildings which still remain in ing roofs in the RGB colour cube, so that in the subsequent the clipped nDOM mask. classification of the trueDOP mask covering the nDOM It is not to be expected that many unmeasured new build- vegetation components are wrongly classified as building ings will appear over a large data set, which would signifi- roofs. Even if an inner margin of 3 m is applied to the ground cantly distort the reference of the RGB colour cube for the plans and only the RGB values of the “inner” roofs are used vegetation. However, due to the selected height threshold to determine the RGB colour cube, there are no significant of ± 2.30 m, the bridges in the nDOM mask remain when differences (see Fig.  11). Especially in the shadow areas, the determining the reference class for the vegetation and are colour values of the vegetation and the dark roofs are similar. included in the reference of the RGB colour cube as a source of error. The average number of identical RGB colour values 5.2 Indirect Building Classification Using an RGB for the vegetation class of the 2.7 million different RGB Reference Colour Cube for Vegetation values is 13,453. The maximum value is 2.3 million. This occurs in the shadow area with almost black colour. To avoid misinterpretations, a conceptual change of the clas- Once the reference of the RGB colour cube for the veg- sification procedure takes place, in which the RGB values for etation has been calculated, the entire trueDOP is classi- the vegetation are determined instead of the building roofs. fied according to a predetermined threshold value. Then However, this is much more computationally intensive for all vegetation pixels lying on the nDOM mask are sub- the creation of the RGB reference colour cube, since the tracted. The remaining pixels of the nDOM mask repre- aerial photographs contain considerably more vegetation sent the searched roofs (see Fig. 13). As a generalization, areas than roof areas. The nDOM mask of Fig. 10 continues by subtracting the vegetation from the nDOM, all those to serve as the basis for calculation. The building ground objects are obtained that have the same colour as building plans of the DFK are cut out of the nDOM mask. Due to the roofs (Figs. 14, 15). fact that the roof overhangs were not measured by cadastral Roofs, for example of garages completely covered by survey, the outlines of the building were extended to the trees, cannot be identified as roofs by the removal of veg- outside with a margin of 3 m. Thus, it is almost guaranteed etation in the nDOM mask. However, elsewhere something that hardly any roof pixels are still contained in the database. of the actual vegetation is filtered out of the nDOM mask. Fig. 12 a nDOM vegetation mask clipped with the building ground plans; b nDOM vegetation mask clipped with a margin of 3 m 1 3 PFG (2020) 88:85–97 95 Fig. 13 With respect to nDOM mask Fig.  10a, classified trueDOP from the nDOM mask, the remaining building roofs or reduced veg- with a threshold of a 1000, b 5000 and c 30,000 with underlying etation (right column), respectively DOM mask (left column); after deduction of all vegetation pixels Fig. 14 a An unclassified trueDOP; b trueDOP classified with a as a vegetation point and was erroneously removed from the mask in threshold of 7000; c trueDOP classified with a threshold of 7000, the middle image. In the right picture the mask has not deteriorated at with white values calculated out. The white solar roof was detected this point In addition, the lower the threshold value selected, the actually existing vegetation. Conversely, the higher the larger the range that is classified in trueDOP, so that the threshold value, the less are the changes of the nDOM nDOM mask can filter out a corresponding amount of the mask. 1 3 96 PFG (2020) 88:85–97 Fig. 15 a Found vegetation points without optimization; b found vegetation points without white values (middle); c and found vegetation values whose sum is R + G + B < 700 (right) To further optimize the vegetation classification, very – shorter calculation times, if a classification only bright values can be subtracted from the vegetation colour takes place on the nDOM mask; cube. White, almost white and grey colour values often orig- – a significantly improved recognition rate of the roofs inate from modifications of the Earth’s surface such as gravel of new buildings; pits and the like. Bridges with their grey values are also – a reduction of vegetation with a suitable threshold largely contained in the nDOM. These colour values occur value; and very frequently and would be detected as vegetation points – no holes in the roofs of the nDOM mask. and thus removed from the mask. This must be prevented to • Potential to avoid misinterpretations exists if the bridges find white and grey roof surfaces. in the nDOM mask could be removed. For this purpose, Technically, this is ensured by not only considering the the bridge objects from the digital landscape model of white pixels with the RGB values (255,255,255), but also the Authoritative Topographic-Cartographic Information all pixels whose sum is R + G + B > 700 as vegetation points. System (ATKIS) would have to be available geometri- The disadvantage of this optimization is that gravel pits cally exact as polygons. remain in the mask. However, they can easily be recognized • What remains unresolved is the heuristic definition of as misinterpretation. threshold values, which cannot be standardized and which probably has to be determined individually for each photo flight project. 6 Conclusion and Outlook The presented classification procedure is transferable to The investigations with the extended RGB colour cubes for all federal states and can be used nationwide as soon as the a third of Bavaria show the following results: federal states have processed the trueDOP. In Bavaria, the classified building roofs of the new buildings are transmit- • The classification of the roofs using the two RGB roof ted to ten Agencies for Digitisation, High-Speed Internet colour cubes “with and without them”; results in and Surveying in order to determine buildings that have not – short calculation times when creating the reference been surveyed in cadastral terms and used to update the real classes, since the evaluation only takes place within estate cadastre and thus also to update the LoD2 building the DFK floor plans; models. Further investigations using AI methods will follow – shorter calculation times, if a classification only to further develop and optimize the results and processes. takes place on the nDOM mask; Acknowledgements Open Access funding provided by Projekt DEAL. – no significant differences in the recognition of build- ing roofs in the evaluation with and without them; Open Access This article is licensed under a Creative Commons Attri- – no significant reduction of vegetation in the nDOM bution 4.0 International License, which permits use, sharing, adapta- mask; and tion, distribution and reproduction in any medium or format, as long – holes in the roofs, by an incomplete classification of as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes the roofs in the trueDOP. were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated • With the indirect classification of the roofs via the RGB otherwise in a credit line to the material. If material is not included in vegetation colour cube, on the other hand, the following the article’s Creative Commons licence and your intended use is not results are obtained: permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativ ecommons .or g/licenses/b y/4.0/. – very high calculation times when creating the refer- ence classes; 1 3 PFG (2020) 88:85–97 97 information. In: Iskidag U (ed) Lecture notes in geoinformation References and cartography, innovations in 3D geo-information sciences. Springer International Publishing, Switzerland, pp 143–157 AdV (2016) Plenumsbeschluss 128/6, 128. Tagung des Plenums der Geßler S, Krey T, Möst K, Roschlaub R (2019) Mit Datenfusionierung Arbeitsgemeinschaft der Vermessungsverwaltungen der Länder Mehrwerte schaffen—Ein Expertensystem zur Baufallerkundung. der Bundesrepublik Deutschland, Sep 2016 in Bad Neuenahr- DVW-Mitteilungen 2(2019):159–187 Ahrweiler, Germany Hümmer F, Roschlaub R (2014) Die Zukunft ist dreidimen- AdV (2017) AK GT Beschluss 30/2, zur Überführung des ATKIS- sional—3D-Gebäudemodelle in Bayern. DVW-Mitteilungen DOP20 in die Qualitätsstufe TrueDOP, Apr 2017 in Saarlouis, 2(2014):165–176 Germany Roschlaub R (1992) Berechnung von Quantilen verschiedener AdV (2018) Produktstandardblätter mit Stand vom 31.12.2018: statistischer Verteilungen. Zeitschrift für Vermessungswesen https://www .adv-online.de/A dV-Produkte/S tandar ds-und-Produkt 117(6):323–335 bla etter /Produ ktbla etter /. Accessed 23 Jul 2019 Shibiao X, Xingjia P, Er L, Baoyuan W, Shuhui B, Weiming D, Shim- Alidoost F, Arefi H (2018) A CNN-based approach for automatic build- ing X, Xiaopeng Z (2018) Automatic building rooftop extraction ing detection and recognition of roof types using a single arial from aerial images via hierarchical RGB-D priors. IEEE Trans image. PFG J Photogramm Remote Sens Geoinf Sci 86:235–248 Geosci Remote Sens 56(12):7369–7387 Aringer K, Roschlaub R (2013) Calculation and update of a 3D build- Valinger J (2015) Automatic rooftop segment extraction using point ing model of Bavaria using Lidar, image matching and cadaster clouds generated from aerial high resolution photography, MSc information. In: ISPRS Annals of the photogrammetry, remote thesis, Umeå University, Faculty of Science and Technology, sensing and spatial information sciences, Vol II-2/W1, ISPRS 8th Department of Computing Science. https ://pdf s.seman ticsc 3DGeoInfo Conference & WG II/2 Workshop, 27–29 Nov 2013, holar .org/4a08/d12b5 49f46 85d85 9c643 2ae26 db930 b5c79 6.pdf. Istanbul, Turkey, pp. 7–12 Accessed 23 Jul 2019 Aringer K, Roschlaub R (2014) Bavarian 3D building model and update concept based on LiDAR, image matching and cadaster 1 3

Journal

PFG – Journal of Photogrammetry, Remote Sensing and Geoinformation ScienceSpringer Journals

Published: Mar 1, 2020

There are no references for this article.