Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Architectural sampling: three possible preconditions for machine learning architectural forms

Architectural sampling: three possible preconditions for machine learning architectural forms Koh Architectural Intelligence (2023) 2:7 Architectural Intelligence https://doi.org/10.1007/s44223-023-00024-1 Open Access COMMENTAR Y Architectural sampling: three possible preconditions for machine learning architectural forms Immanuel Koh back to 2012, when artist and writer James Bridle first Keywords Sampling, Flattened, Resolutional, Probabilistic, began to visualize ‘invisible’ technology via the notion Perception, Forms, Figure-ground, Part-whole, Shape of glitch. A new movement called the “New Aesthetic” grammars, Machine learning, Deep learning, Artificial was born, coined by Bridle in London, and endorsed by intelligence the internationally recognised cyberspace theorist Bruce Sterling. This new visuality concerns the increasingly pervasive vision of machines, ranging from satellite views “We shape our tools, and thereafter our tools shape us” that stitch maps from distributed sources to surveillance “We need to learn how to see a parallel universe cameras that do facial recognition of pedestrians on the composed of activations, keypoints, eigenfaces, fea- streets and to bots that watch over us on the internet. ture transforms, classifiers and training sets” Essentially a machine vision between that which is physi- cally real and digitally unreal. One of Bridle’s most rec- What does it mean to see architectural forms through ognizable works is the series on Rainbow Planes (Fig. 1). the sampling lens of the machines? A new way of seeing? It is an image of a plane that has moved fast enough to The first quote by John Culkin (a contemporary of Mar - disrupt the normal “panchromatic sharpening’ pro- shall McLuhan) in 1967 is a reminder of the reciprocal cess when merging pixels from different satellites’ sen - relationship between human and technology. The sec - sors, thus resulting in the successive RGB rainbow-like ond quote, just half a century later, comes from the arti- imprints. cle “Invisible Images” by Trevor Paglen for the February This awareness of machine vision has further intensi - 2019 issue of Architectural Design (AD) titled “Machine fied with the rise of deep learning – from the close read - Landscapes: Architectures of the Post-Anthropocene”. ing of glitch samples in the New Aesthetic to the more Paglen is not alone in acknowledging this new way of recent deep neural network layers’ visualization of hal- seeing—a new visuality characterized by today’s AI tools lucination samples in the Neural Aesthetics. Both Deep- of deep neural networks. In fact, this notion of machine 8 9 Dream (Fig.  2) and Neural Style Transfer demonstrate vision and its ramification on aesthetics can be traced (Culkin 1967) p. 50. (Paglen 2019) p. 27. (Sterling 2012). (Young and Ed., 2019). See Rainbow Planes Flickr album for more examples: https:// www. flickr. com/ photos/ stml/ sets/ 72157 63793 80610 15/ with/ 10994 081436/ *Correspondence: (Bridle 2014). Immanuel Koh (Hertzmann 1903). immanuel_koh@sutd.edu.sg (Mordvintsev et al., 2015). Singapore University of Technology and Design, Singapore, Singapore (Gatys et al., 1508). © The Author(s) 2023. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/. Koh Architectural Intelligence (2023) 2:7 Page 2 of 21 Fig. 1 James Bridle, Rainbow Planes. 2013. Coordinates: Chicago: 41.785420°, -87.578890°, 1/7/2010. The New Aesthetic situates machine seeing as an assemblage of different data sources as fragments Fig. 2 Google, DeepDream, 2015 (Left) original input image. (Right) generated output. By zooming in, one could see various neural hallucinations as a result of the ImageNet dataset used for training the deep learning model (GoogLeNet). The Neural Aesthetic situates machine seeing as an assemblage of different data samples as features that by manipulating the layers of a convolutional neu- disposed within the discipline, for it to partake in what ral network, one could alter and extract aspects of its Terry Sejnowski refers to as the deep learning revolution perception to generate new forms. In short, with new and what Andrej Karpathy alludes to as Software 2.0 ? In perceptions come new conception of forms. This gen - my theory of Architectural Sampling, I proposed to first erative potential, especially with the Generative Adver- debunk the three ingrained ways of thinking about archi- sarial Network (GANs), has now even been taken up tectural form, in order to lay the foundational precondi- by world-renowned artist Pierre Huyghe in his UUmwelt tions for a machine-learnable architecture. exhibition at the Serpentine Gallery, as well as being auc- The first of the three preconditions, “No Figure, No tioned off at Christie’s. Ground”, looks at the most basic representation of archi- Our visual culture’s notion of perception has become tecture— the two-dimensional black and white figure- increasingly machinic, it is time to reconsider architec- ground. A dualistic representation of mass and void (or of ture in the light of this phenomenon. What does it mean form and space). This inherently layered representation to see architecture through the sampling lens of machine of architecture will be flattened, such that there is neither vision and deep learning? What are the longstanding figure nor ground. Flatness is the new field. The second conceptual and perceptual assumptions that need to be of the three preconditions, “No Parts, No Whole”, looks at the parts-to-whole and whole-to-parts relationship of 10 12 (Goodfellow, et al., 1406). (Terrence 2018). 11 13 See article “Is artificial intelligence set to become art’s next medium?” See article “Software 2.0” https:// medium. com/@ karpa thy/ softw are-2- 0- https:// www. chris ties. com/ featu res/A- colla borat ion- betwe en- two- artis ts- a6415 2b37c 35 one- human- one-a- machi ne- 9332-1. aspx (Koh 2019a) This text is largely taken from the chapter “Perception through Sampling” of my PhD thesis, but revised significantly to fit with the publica - tion constraints of this journal. Koh Architectural Intelligence (2023) 2:7 Page 3 of 21 architecture in three-dimensions and argues that there is of 1997 and to Patrik Schumacher’s “parametric figura - in fact neither parts nor whole after all, but only resolu- tion” of 2011, many have attempted to propose alterna- tions. Resolution is the new reality. The last of the three tive theories by replacing the ground of the figure-ground preconditions, “No Shapes, No Grammars”, looks at one with field to debunk the figure-ground in architecture. of architecture’s most respected computational paradigm Interestingly, just a year after the publication of Allen’s of shape grammars and claims that it is the edges, not the Field Conditions, outside of architecture, deep learning shapes, that matters at the primordial level; and a proba- pioneer Yann LeCun proposed the first Convolutional bilistic model is a more realistic model than deterministic Neural Network known as LeNet-5 in 1998. The con - grammars. Making decisions is probably better with sta- cept of convolution alone is potent enough to, once and tistics than rules. for all, dispense with the figure-ground (or field) dilemma Each precondition will be discussed and accompanied faced by those architects mentioned. Unfortunately, such by its successive pictorials (i.e., annotated standalone a radical architectural concept remains unknown, even figures). Similarly, I will conclude the text with the pro - after the year when the paper “ImageNet Classifica - ject “Discrete-Mies” to more vividly illustrate how the tion with Deep Convolutional Neural Networks” by Alex proposed three preconditions of architectural sampling Krizhevsky, Ilya Sutskever, and Geoffrey Hinton was pub - could be put in practice to machine-learn Mies van der lished and made headlines when it won the 2012 ILSVRC Rohe’s 1929 Barcelona Pavilion and to generate new Bar- (ImageNet Large-Scale Visual Recognition Challenge) by celona Pavilions of infinite variations. a large margin. The deep neural network model known as AlexNet, itself a direct descendent of LeNet-5, has since 1 No figure, no ground given rise to the deep learning revolution of today. One Figure: might ask now: In what way does such a breakthrough in AI and computer vision has an impact on architec- “The form of anything as determined by the out - tural thought, and specifically, the architectural percep - line.” tion and conception of the figure-ground? To answer this question, we will have to take a step back and recon- Ground: sider: what is the fundamental formal (thus perceptual) “The bottom; the lowest part or downward limit of assumptions of the traditional figure-ground, and in what anything.” way has the new visuality challenged those assumptions? (Figs. 3, 4, 5 and 6). Schumacher uses the term “parametric figuration” to overcome the limitation posed by the classical dual 1.1 No figure concept of g fi ure-ground. Accordingly, these new figu - The training of an architect, may it be during the pre-dig - rations emerged and submerged within a field which is ital or the post-digital times, is closely related to the tools governed by an underlying set of rule-based parameters. of design representation used. One of the most consistent In contrast to modern composition, parametric space is features of these representation tools has been the ability characterised by collective figurations over singular fig - to work in layers. This concept of layering can be found ures, producing global “biases, drifts, gradients”. Schu- in the analogue layering of tracing paper, as well as the macher is not the only who has sought to abandon the digital layer manager embedded in CAD softwares (e.g., figure-ground concept that so permeate architectural Autodesk’s AutoCAD or McNeel’s Rhinoceros) or other thought and perception. From Sanford Kwinter’s Ein- 20 21 graphics editors (e.g., Adobe’s Photoshop or Illustrator). steinian Field of 1986, to Stan Allen’s Field Conditions To represent a design as a figure-ground is essentially to 15 graphically place a figure on a ground. There may be many (Koh 2019b). 16 figures on many grounds, like in Photoshop or AutoCAD, See the video recording of the “AI & Architectural Design” panel discus- sion where Patrik Schumacher expressed that the project has convincingly separated and stacked one after the other. The removal of machine-learned the formal and spatial features of the Barcelona Pavilion. one figure from layer A will not affect those other figures h t t p s :// w w w . yout u b e. c om/ w a t c h? a pp= de skt op&v= u- 8KO_ 7ycgE & list= on layer B in any way. But once the layers are flattened, PLjgJ TkiYX dLcxh B3WTt Roxl7 f52vf dNcQ& index= 19 the pixels’ value can no longer be swapped among lay- See Oxford English Dictionary.https:// www. oed. com/ view/ Entry/ 70079? rskey= Gpov5 F& result= 1# eid ers, and to remove or move a figure would imply a cut - Ibid. https:// www. oed. com/ view/ Entry/ 81805? rskey= Ig9VL q& result= 4# ting (or Photoshop’s ‘lasso’) and pasting of the figure. It is eid (Schumacher 2011). p. 424. (Schumacher 2011). (Kwinter 1986). (Allen 1997), (Allen 1999). (Lecun et al.,  1998). (Krizhevsky et al., 2012). Koh Architectural Intelligence (2023) 2:7 Page 4 of 21 neural network (CNN) will take the input as a tensor of size 256 × 256 × 3, where the third dimension represents the vector of 3 numbers from the image’s RGB channels. u Th s, there is no concept of an explicit differentiation of a figure and ground when AlexNet looks at an image, it simply takes those tensors of numbers. The same princi - ple applies to the SpaceNet, which is similar to the Ima- geNet, except that the dataset is a corpus of high-quality commercial satellite imagery instead of objects. In a way, one could say that it is all ground and no figures. 1.2 No ground From a computational and generative design perspective, the abandonment of the layered figure-ground for a flat - tened machine vision can also be traced back to the com- puter art scene of the 1960s, specifically in the pioneering works of philosopher-programmer Hiroshi Kawano, as contrast with those of mathematician-computer scientist Frieder Nake and engineer Michael Noll. Unlike many of his contemporaries, Kawano’s early work was based on data-driven techniques rather than the explicit program- ming of rules. In his introductory overview of Kawano’s work, Gristwood writes that Kawano was influenced by his reading of Max Bense’s work on Information Aesthet- ics and Claude Shannon’s work on Information Theory and Communication. A striking difference can be found Fig. 3 Excerpt of the many generic figure-ground field diagrams when studying Nake’s Hommage à Paul Klee 13/9/65 used by Stan Allen in his ‘Field Condition’. The bottom-left diagram Nr.2 of 1965, Noll’s Computer Composition with Lines called ‘field vectors’ has been used for the demonstration in Figs. 4 & of 1964 and Kawano’s Artificial Mondrian of 1969. Nake 6. Image as published in the original paper (Allen 1997) first abstracted Klee’s figural lines as explicit rules before plotting them on an empty background of continuous paper space. Kawano, on the other hand, neither differ - this habitual and convenient perception of a floating layer entiates the figure and the ground as separate entities nor called figure over a base layer called ground, that needs attempts to write a Mondrian shape grammar. He simply to be first questioned and eventually replaced by a ‘flat - decomposed Mondrian’s painting into a grid of equally tened’ perception. sized colour blocks for the computer to statistically learn Digitally speaking, the architect’s figure-ground can be their probability distribution. He then sampled from that understood as a binary image where each pixel value is a same distribution to infer new Mondrians. Nake used the 1-bit number (0 as black or 1 as white). In the case of a Zuse Graphomat Z 64 (1960) by Konrad Zuse to plot fig - grayscale image, each pixel value would be an 8-bit inte- ures on ground, while Kawano used the earliest IBM line ger (0 to 255); and for coloured RGB images, each pixel printer to print lines of characters, where each character value would be a vector of 3 numbers. Returning to the represents a colour block. original discussion on machine vision, unlike the ways Even though both Kawano and Noll had used Mondri- in which human would see a figure by separating it from an’s work as their starting point, their aims and processes the ground, the AlexNet did not achieve its 84.7% object were radically different. For Noll, it is about imitating classification accuracy in a similar manner. Instead, it a human vision as he imagines Piet Mondrian himself takes in a ‘flattened’ image input in their numerical repre - holding the paint brush and painting black figures on the sentation as tensors, systematically scanning, convolving white canvas/ground. This intended imitation is evident and pooling (downsampling) them, and outputs a proba- bility distribution of the 1000 object classes with which it was trained on, to make a prediction. For example, given SpaceNet. https:// space netch allen ge. github. io/ an image of size 256 × 256 pixels, the deep convolutional ImageNet. http:// image- net. org/ index (Gristwood Jan. 2018). 25 29 (Krizhevsky et al., 2012). (Noll  1966). Koh Architectural Intelligence (2023) 2:7 Page 5 of 21 Fig. 4 Immanuel Koh, Flattened Field Conditions ( With Figure / With Ground), 2018. One of the many generic figure-ground field diagrams used by Stan Allen is here perceived as 28 unique black figures. It is a traditional and typical reading of the 1990s field conditions where the perceptual distinction of figures and ground is clearly intended Fig. 5 LeNet-5 architecture reflecting the hierarchical perceptual structure of the human visual cortex. The ‘flattened’ input image (leftmost layer) is perceived as a tensor of size 32 × 32x1 and convolved with a smaller scanning window/kernel of 5 × 5x1, without any differentiation of figure or ground. Image as published in the original paper (Lecun et al., 1998). from his article “Human or Machine: A Subjective Com- learning a machine vision as he imagines the computer parison of Piet Mondrian’s ‘Composition with Lines’ (1917) scanning and learning the statistical distribution of col- and a Computer-generated Picture”, where he reported our blocks represented with symbols, without seeing that only 28% of the 100 human subjects were able to any figures and ground. In fact, Kawano’s discrete col - identify the computer-generated picture. Likewise, Noll’s our blocks are what Sobchack might have referred to as conception of a figure-ground process is evident from ‘discrete pixels’. Interestingly, or even prophetically, this his use of a General Dynamic SC-4020 Microfilm Plotter machinic pattern-recognition process of ‘statistical seeing’ to move the beam of a cathode ray tube in drawing the is not too dissimilar to how AlexNet (as well as all vari- figures before photographing it. For Kawano, it is about ants of the convolutional neural networks) is trained from 30 31 (Noll  1966). (Sobchack 1990). Koh Architectural Intelligence (2023) 2:7 Page 6 of 21 Fig. 6 Immanuel Koh, Flattened Field Conditions ( Without Figure / Without Ground), 2018. One of the many generic figure-ground field diagrams used by Stan Allen is here discretized, convolved, and then encoded as a combinatorial instance from 28 unique part-figures/part-grounds patterns. It is an alternative reading of the 1990s field conditions where the distinction of figure and ground is completely abandoned big datasets of images to detect and recognise all sorts of 2 No parts, no whole objects. These training sets of images are similarly simply Whole: ‘discrete fields’ or mathematical matrices of numbers rep - “Complete, undivided, total.” resenting colours. Therefore, not only are there no figures, there is effectively no ground as well. Oxford English Dictionary. https:// www. oed. com/ view/ Entry/ 228723? rskey= j7yrc q& result= 1& isAdv anced= false# eid Koh Architectural Intelligence (2023) 2:7 Page 7 of 21 Part: ontologically unstable. A floor is a floor and nothing else. How true is the preceding statement? Koolhaas himself “A piece or section of something which together with has fought hard throughout his career to falsify the state- another or others makes up the whole (whether actu- ment, as testified by the Jussieu library project which has ally separate from the rest or not)” since triggered a whole series of subsequent built projects In the 2600-page book “Elements of Architecture”, based on the notion of connected floors forming a single Rem Koolhaas organises its content according to the continuous spatial trajectory. Meaning, a floor is also a semantic parts of a building—floor, ceiling, roof, door, ramp or stair or corridor or anything in between (Figs. 7, wall, stair, toilet, window, facade, balcony, corridor, fire - 8, 9 and 10). place, ramp, escalator and elevator. The book is in fact a compilation and expansion of an earlier exhibition at the 2.1 No whole Venice Biennale 2014 and impressively traces the global Similar endeavours in the history of architecture to defy history of each of these architectural elements. Accord- such established architectural semantics have helped ing to such an understanding, architecture is then a whole reinvent architecture both conceptually and formally. made up of different constituent parts , each performing One of them is the seminal theory of ‘anti-object’ by a specific function in relation to its spatial allocation. Kengo Kuma, specifically his notion of ‘particlisation’. The objective for an architecture is then to configure Kuma’s particlised architecture has more recently been that whole, with the belief that Aristotle’s “The whole is adopted by Mario Carpo in his inauguration of the digital greater than the sum of its parts” holds true. Others have discrete. Since the proposition of ‘No Parts, No Whole’ disagreed with such a prioritization of the whole over is closely related to Kuma’s theory and Carpo’s extension, the parts. For one, is the philosopher of object-oriented it is worthwhile taking a moment to introduce them here. ontology, Timothy Morton, who writes “The whole is “No particular skill or effort is required to turn always less than the sum of its parts” to remind us that something into an object. Preventing a thing from a whole is outsized by its parts since there are potentially becoming an object is a far more difficult task.” infinite regress of parts within it. Or French philosopher of the actor-network theory (ANT) Bruno Latour who A quote above from Kengo Kuma in Brett Steele’s writes “The whole is always smaller than its parts” to introduction to the book “Anti-Object: The Dissolution emphasize the connections among entities which could and Disintegration of Architecture”, which exposes the be humans or non-humans. Or like Gestalt psycholo- ease and tendency of thinking and designing architecture gist Kurt Koffka’s “The whole is other than the sum of its as object. In the book’s preface, Kuma defines an object parts” which avoids any measurable comparison between as “a form of material existence distinct from its immedi- the whole and the parts by stating that they are simply ate environment.” By first defining what is an object, he different. It is beyond the scope and purpose of this text proceeds to define his so-called “anti-object” via a gen - to discuss these aforementioned philosophical debates. eral strategy of negation in the rest of the book. Some of Instead, the focus here is to see the conceptual benefits these statements of negation include: “That is why par - engendered by a different perception of the whole and ticlised architecture is the polar opposite of photographic the parts, and in our case, the abandonment of both. architecture. The silhouette is ambiguous.” and “Archi- Returning to Rem Koolhaas’ categorical assignment of tecture is another name for the aggregation of matter (i.e., architectural elements, this text is thus suggesting oth- the creation of object), and ‘particlisation’ is the reversal erwise. Meaning, the precondition for architectural sam- of the aggregation.” Most relevant to our discussion here pling is the denial of the meaning of each part. That is to is his last chapter “Breaking Down into Particles”, where say, instead of Koolhaas’ 15 specific and ontological sta - he elaborates on the concept of “particlisation” in archi- ble parts of a building, there are in fact an indeterminate tecture. Using the impressionist painter Georges Seurat’s number of possible generic parts that encodes a build- pointillism and gothic architecture style as his departure ing’s form, according to the discretization and quantiza- tion process of the proposed sampling. The argument is (Kuma 2012). based on the reality that architectural form is inherently 38 (Carpo 2019). 39 st (Kuma 2012) p.2. & 51. The 1 print was in 2008 and the original Japa- nese version is in 2000 by Chikuma Shobo (Tokyo). Ibid. Ibid. https:// www. oed. com/ view/ Entry/ 138188? rskey= fGtZC w& result= 1& isAdv anced= false# eid Ibid., See Preface. 34 42 (Koolhaas et al., 2018). Ibid., p.106. 35 43 (Morton 2018) pp. 92–93. Ibid., p.119. 36 44 (Latour et al., 2012). Ibid., pp.98–210. Koh Architectural Intelligence (2023) 2:7 Page 8 of 21 Fig. 7 The decomposition of the Parthenon into its semantic parts has become the data structure of architecture’s digital representation and subsequently used as the underlying ontology of today’s BIM (Building Information Modelling). Image as published in the book (Mitchell 1990). points, Kuma illustrates how “particlisation’ is at work decomposed abstraction, in order to attain an anti-object in both. The former concerns the decomposition of pic - configuration. torial objects with the multiplicity of uniformly sized In referencing modern architecture, Kuma points out paint dabs, while the latter concerns the decomposition that Walter Gropius’s Dessau Bauhaus and Le Corbusier’s of building form with a formal articulation based on the Maison Dom-ino are in fact key modernist moments in multiplicity of small elements. Both adopted a strategy of the “particlisation” of architectural form, most notably Koh Architectural Intelligence (2023) 2:7 Page 9 of 21 Fig. 8 (Left) Photograph of a 1765 Neoclassical armchair from the V&A Museum in London. A Classical Relationship of Whole-to-Part: Each part is designed to serve a specific function with a specific meaning to produce a complete and perfect whole. (Right) Photograph of the original 1923 Berlin chair by Rietveld. A Modernist Relationship of Part-to-Whole: Each part is designed with interchangeable functions and semantics, where parts are assembled to form a possible whole Fig. 9 VoxelNet deep neural network architecture showing the decomposition of a point cloud into equally sized generic voxels. Points in each voxel are grouped and transform as a vector representation for learning the features to detect objects of different probable semantics. This suggests a Machinic Sampling Relationship of No-Parts-No-Whole. Image as published in the original paper (Zhou and Tuzel 2017). through their use of dry construction in the context of reinforced by the medium of dissemination, in the form prefabrication. However, he writes that Le Corbusier of two-dimensional photographs, which requires an almost immediate shift to an architecture of objects was unambiguous depiction of the building’s silhouette for a result of the shift in his approach to painting – from effective communication. The object architecture of both cubism to suprematism and purism. This was further Le Corbusier’s Villa Savoye and Mies van der Rohe’s Bar- celona pavilion are reinforced by their respective use of the pilotis and plinth, as a formal means to isolate their Ibid., p.104. Koh Architectural Intelligence (2023) 2:7 Page 10 of 21 Fig. 10 Immanuel Koh, Resolutional Berlin Chair, 2019. An illustration of a relationship of No-Parts-No-Whole. (Left) Digitally remodelled Rietveld’s Berlin chair. (Right Top) Exploded rendered view of the decomposed units at different resolution (i.e., 60, 90 & 180). (Right Bottom) Automatically generated coloured voxel semantic maps at different resolution (i.e., 60, 90 & 180). Each unique semantic voxel is coloured (thus labelled) according to their formal and configurational similarity architecture from the ground, especially when viewed as Upon closer reading, the underlying argument of Carpo images. In fact, Kuma even extrapolated his “particlisa- has always been an attack and upending of the first digital tion” theory to cities, calling it “particle-based urban the- style—the smoothness of the spline, with the second digi- 46 48 ory”. It is a critique of modern city planning’s failure of tal style of the discrete. Even without Carpo’s rhetorical regulating city based on the means of objects and enclo- allusion of a brute-force approach with big data replacing sures. Objects are manifested in the form of monumental the elegant mathematical function of small data, with- buildings and enclosures in the form of zoning or theme out an architecture conceptualized as a whole ‘object’, parks. smoothness cannot be perceived readily. For example, For Kuma, the preconditions of particlising archi- the 90  s Embryological House of Greg Lynn would not tectuire is first the dissolution of the object and sec - have looked smooth if its deformation is not applied on ond the homogenization of the parts. In other words, a single whole object. Or conversely, the 2010s Serpen- it is the decomposition of a whole into a multiplicity of tine Pavilion by Sou Fujimoto would not have looked dis- generic(non-semantic) parts, which in turn, leads to crete if its aggregation is not with a multiplicity of generic an inevitable formal consequence of no whole, no parts parts. Therefore, to oppose smoothness is consequently as proposed here. However, if the whole and parts are to oppose wholeness. To lose the whole, is then to shift all lost, in what way does it contribute to the computa- the focus to its parts. And to lose the parts, is to make tional formulation of sampling architectural forms with them generic without semantics. Finally, once we lose the machine learning. This is the moment where Carpo’s specificity of their semantics, it suddenly becomes pos - pairing of Kuma’s theory and the computational discrete sible to normalize and sample them simply as numerical becomes critical. In his article titled “Particlised: Compu- representations with machine learning and deep learning tational Discretism, or The Rise of the Digital Discrete”, apparatuses. Mario Carpo draws a convincing parallel between the work of Kengo Kuma and those of today’s young avant-2.2 No parts garde of the “second digital style” or “computational Unfortunately, the semantic parts are the most diffi - discretism”, by highlighting a set of shared formal vocab- cult to be forsaken. The longstanding presumption of ulary characterized by terms such as, “disjointed”, “dis- defining a building according to its semantic constitu - connected”, “fragmentary”, “voxellised”, filamentous” and ents has remained extremely ingrained within the dis- “chunky”. Referring to Kuma’s own definition of “parti - cipline. In the “The Logic of Architecture”, Mitchell clisation”, Carpo reminds us of Kuma’s motivation for his shows how the Parthenon could be digitally decom- current “non-figural, aggregational, atomised” style, as a posed with its entablature components encoded in a deliberate attack at the material continuity of modernism relational lattice and their orientations mapped with a (specifically the use of wet construction in concrete). (Carpo 2017). 46 49 Ibid., p.114. (Carpo 2014). 47 50 (Carpo 2019). (Mitchell 1990). Koh Architectural Intelligence (2023) 2:7 Page 11 of 21 function accordingly. Such a prescriptive approach has neural network layers. In short, the heterogenous geo- now become a standard with BIM (Building Informa- metric parts are replaced by homogenous voxels, while tion Modelling). For example, Autodesk’s Revit Archi- the hierarchy of semantics is replaced by the hierarchy of tecture, a BIM software, is built in such a way that any abstract features. model element has to be first assigned as either a host (walls, floors, roofs or ceiling) or a model component 3 No shapes, no grammars (stairs, windows, doors or furniture). This ‘host and “External form or contour; that quality of a mate- hosted’ relationship, such as a window being hosted by rial object (or geometrical figure) which depends on a wall, reflects the exact parallel with Mitchell’s schema constant relations of position and proportionate dis- of a frieze hosted by an entablature. To maintain such an tance among all the points composing its outline or explicit relationship, a BIM model requires a literal tax- its external surface.” onomy of a building’s constituent parts as objects with specific semantics to be stored in its database. Grammar: One might ask: How then could a building be repre- “That department of the study of a language which sented if not with their semantic constituent parts? A deals with its inflectional forms or other means of look at 3D scanning of buildings might provide an alter- indicating the relations of words in the sentence, native mode of perception and representation of archi- and with the rules for employing these in accordance tectural forms. Instead of modelling solids or surfaces with established usage” directly, the scanned point cloud is a collection of 3D points with x, y, z coordinates; and unlike voxels, can Numerous researchers have devoted their entire be unevenly spaced, sparse and massive in data points. careers in the continual development (and establishment) From photogrammetry that outputs 3d coordinates of shape grammar since its inauguration in 1971. Shape with 2D photographs, to LIDAR (Light Detection and grammar is a set of rules of transformation applied recur- Ranging) that outputs 3d coordinates by laser scanning sively to an initial shape, generating new shapes. The in 3D directly, the representation of raw point clouds is rules have the general form of A → B, where A and B are never semantic. It is only through their post-processing shapes and the sign ‘→ ’ denotes a replacement step. For with manual or machine-learning based semantic seg- example, given an initial shape S, if shape A is be found mentation that their functions are defined. Returning in it, subtract A from S and replace it with B to generate a to Kuma’s theory of particlisation, the point cloud rep- new shape S’. This generative process can be simply stated resentation of a building is literally a particlised archi- as ‘seeing and doing’. The research in shape grammars tecture. In this particlised conception of architecture, has encompassed several areas of design, including urban to refer to its parts, is to refer to either each point in design, architecture, product design and mechanical the point cloud or a variably sized cluster of points or design, and is now being technically incorporated in par- sub-cloud. In other words, there are no real ‘parts’ any- ametric software and theoretically extended for mate- more, just numerical values of x, y, z and optionally, that rial-based performative making. of colours and normals too. The hierarchical definition In view of all these accomplishments within the shape of architecture in BIM, whether implicitly or explicitly grammar community, it would be an impossible feat indebted to Christopher Alexander, is now supplanted in attempting to overthrow the entire enterprise and to by a different hierarchy, not of parts but of features. This propose a better theoretical alternative. However, this new hierarchy of formal features is to be found in the is not my objective here. Rather, it is a provocation to post-processing of these generic point clouds. More con- reconsider the shapes and grammars of shape grammar. cretely, in the layers of deep neural networks, such as the This reconsideration would then lead us to expose the VoxelNet. Essentially, the VoxelNet first takes a point underlying formal and perceptual limits of this design cloud as its input and subdivide it into evenly spaced 3D paradigm—a stumbling block in the formulation of a for- voxels. Points that are contained in each voxel is grouped mal basis for machine learning architecture. The shape and transform into a feature representation through its of shape grammar is the cause of its perceptual (‘seeing’) See Oxford English Dictionary. See Revit Getting Started. https:// knowl edge. autod esk. com/ suppo rt/ revit- Ibid. produ cts/ getti ng- start ed/ caas/ Cloud Help/ cloud help/ 2018/ ENU/ Revit- GetSt arted/ files/ GUID- 5BFA4 99A- 5ACA- 4069- 852C- 9B60C 9DE67 08- htm. html (Stiny and Gips 1971). 52 58 (Armeni, et al., 2016), (Qi and pointnet: PointNet, 2017). (Stiny 2006). 53 59 (Alexander 1964). (Stouffs 2019). 54 60 (Zhou and Tuzel 2017). (Knight 2018). Koh Architectural Intelligence (2023) 2:7 Page 12 of 21 Fig. 11 Superstudio, Histogram of Architecture, 1969–1971. It is a catalogue consisting of 33 three-dimensional abstract forms to generate designs across scales. Unlike in shape grammars, instead of defining a set of rules, the Histogram of Architecture is conceptually a ‘training set’ which represents a mental latent distribution of probable forms of the same architectural language. The word histogram in the title and the grid surfaces of the forms clearly allude to a machinic form of statistical seeing Fig. 12 Classification by a Convolutional Neural Network showing texture as the primordial feature over shape. Image as published in the original paper (Geirhos et al., 1811).(Geirhos et al., 2018) limit, while its grammar, the cause of its formal (‘doing’) so is its underlying rewriting schema. Thus, it would be limit (Figs. 11, 12, 13 and 14). most fruitful to at least briefly address Chomsky’s gram- mar directly in the language domain before returning to 3.1 No grammars the grammar in the architecture domain. In order to do To disagree with the grammar of Stiny’s shape grammar so with a higher probability of success, the arguments put means to first disagree with Chomsky himself, specifically forward by one of Chomsky’s strongest critics, computer his Generative Grammar, of which Stiny has appropri- scientist Peter Norvig, will be presented first. ated for his architectural formulation. The formal appro - Peter Norvig, director of research at Google and co- priation is so direct that even in the 1971 paper, shape author of the leading textbook “Artificial Intelligence: A grammar is defined exactly in accordance to the standard Modern Approach”, wrote a 2012 article titled “Col- definition of phrase structure grammar of Chomsky, ourless green ideas learn furiously: Chomsky and the two cultures of statistical learning”. In his article, (Chomsky 1957). (Stiny and Gips 1971). (Russell et al., 2016). 63 65 (Chomsky 1957). (Norvig Aug. 2012). Koh Architectural Intelligence (2023) 2:7 Page 13 of 21 Fig. 13 Immanuel Koh, Probabilistic Histogram of Architecture, 2019. Reinterpretation of the original Histogram of Architecture in Fig. 11 (i.e., last 3 columns of its top row) where the histograms are re-perceived and re-represented as shapeless and grammarless training set of forms. Presented in the same manner as the original catalogue with sets of 3 diagrams laid out in a vertically order, each diagram is also accompanied by its corresponding matrix representation used as machine learnable numerical representation Koh Architectural Intelligence (2023) 2:7 Page 14 of 21 Fig. 14 Immanuel Koh, Probabilistic Histogram of Architecture, 2019. Reinterpretation of the original Histogram of Architecture in Fig. 11 (i.e., all 6 columns of its bottom row) Norvig begins by referring to Chomsky’s remark at the expressed by P(*IE), he convincingly proves the syntactic Brains, Minds, and Machines symposium held at MIT model of Chomsky lacking in accuracy and expressiv- in 2011, that reinstates Chomsky’s well-known disap- ity. Referencing statistician Leo Breiman, he points out proval of statistical models in their usefulness for lan- the two statistical cultures—data modelling culture and guage tasks. First and foremost, what is a statistical algorithmic modelling culture, with which Breiman and model and a probabilistic model? Accordingly, a sta- himself object the former and Chomsky objects the latter. tistical model is “a mathematical model which is modi- The reason for Breiman’s objection is that it is a futile task fied or trained by the input of data points”, while a to uniquely model the truth of nature (or language) and probabilistic model “specifies a probability distribution seeking to make known all the underlying parameters in over possible values of random variables” Essentially, an explainable form; instead, one should be satisfied with probabilistic model does not state an exact determin- a messy function that could sufficiently approximate the istic functional relationship. Using Claude Shannon’s observed data. Accordingly, Chomsky’s favouring of the probabilistic language(communication) model based former is based on the latter’s lack of non-messy and on Markov chains as an example, Norvig reminds us that explanatory principles (or rules). language could escape from the syntactical insistence of Returning to our original discussion of the term gram- Chomsky. Norvig then highlights the disadvantages of mar in shape grammar, the preceding text aims to high- Chomsky’s logical (or categorical) model and the advan- light a similar formal dilemma in architecture. Shape tages of models that are statistical and probabilistic. The grammar shares the same explanatory principles of former is based on logical rules and expresses a boolean Chomsky’s generative grammars with its elegant formal- distinction, whereas the latter models a probability dis- ism of “seeing and doing”, but similarly, it also lacks the tribution. Demonstrating with a simple example of Eng- expressivity of Norvig’s statistical and probabilistic mod- lish spelling, a non-statistical model expressed by the els. If one is to agree with Chomsky that a linguist’s job rule “I before E except after C” and a probabilistic model is to define rules that could distinguish between what is grammatical and ungrammatical, then it would not be Ibid. Ibid. (Breiman  2001). 68 70 (Shannon 1948). (Stiny 2006). Koh Architectural Intelligence (2023) 2:7 Page 15 of 21 Fig. 15 The original floor plan of Mies van der Rohe’s 1929 Barcelona Pavilion redrawn by the author. It is the basis for the digital reconstruction of the 3D model, which is in turn, used for the sampling experiments Fig. 16 Immanuel Koh, Discrete-Mies, 2018. Sets of unique cells convolved at different sampling resolution: (top-left) 2.0 m X 2.0 m, (bottom-left) 1.0 m X 1.0 m (top-right), 0.5 m X 0.5 m and (bottom-right) 0.25 m X 0.25 m. Each cell can be observed in the input architectural model of Barcelona Pavilion. (Middle-column) Each red square represents the relative size of cells according to their sampling resolution, thus providing a way to compare at the same scale too incorrect to say the same has happened to architec- been continually made to develop more and more gram- ture researchers in defining rules to distinguish different mars for a variety of architectural languages. Many of architectural styles or languages. Reflecting on the past these grammars are painstaking defined with basic shape fifty years of research in shape grammars, efforts have primitives and several dozens of rules. For example, the Koh Architectural Intelligence (2023) 2:7 Page 16 of 21 Fig. 17 Immanuel Koh, Discrete-Mies, 2018. Mies van der Rohe’s 1929 Barcelona Pavilion (with roof removed to better show the spatial configuration) as the original input architectural model for the sampling procedures as grammatical (i.e., a deterministic rule-based model) will hinder the formulation of a theoretical and technical framework for machine learning architecture. 3.2 No shapes It is worth quoting Stiny and Gips in full here: “A formal- ism for the complete, generative specification of a class of non-representational, geometric paintings or sculptures is defined, which has shape grammars as its primary struc - tural component. Paintings are material representations of two-dimensional shapes generated by shape grammars, sculptures of three-dimensional shapes.” The word ‘geo- metric’ is the most crucial element for our purposes. It Fig. 18 Immanuel Koh, Discrete-Mies, 2018. Generated using is not surprising to see their idea of painting as a ‘non- a uniform probability distribution (i.e., every cell has the same probability of being generated), without sampling the spatial representational’ composition of geometric shapes as a relations among different cells result of Sol LeWitt’s influence during the thriving peri - ods of conceptual art scene of the 1960s. Consequentially, even the specification of their painting rules is geometri - Palladian villas shape grammar itself contains as many cally conceptualized as “a schema for painting the areas as 69 rules. Although efforts have been made to even contained in a shape.”. With this clear delineation of extend shape grammars to making grammars, in order shape boundaries, shape could then be similarly con- to include aspect of making and materiality, the entire ceived as Venn diagrams from set theory and their col- process of defining these making rules remains labori- ouring rules as follows, “The effect of the painting rules ously hard-coded. In taking the same position as Nor- in the example is to count set overlaps. Areas with three vig and Breiman, it is equally futile to assume that there overlaps are painted lightest, two overlaps second lightest, exists simple and explanatory set of rules that could be one overlap second darkest, and zero overlaps darkest.” syntactically formulated for every possible architecture. Counting can only be possible if each shape is an indi- Given the difficulties in creating/modifying/maintain - vidual figure positioned in a ‘limiting shape’. This limiting ing shape grammars, and its consequential non-data shape is defined as “the size and shape of the canvas on driven, non-generalizable and small-scale production, its which a shape is painted”, and is necessarily a ground, adoption within architectural practices continued to be minimal or non-existent, despite the ferventness among academics. More crucially, the conception of architecture (Stiny and Gips 1971). See the reprint version (Petrocelli 1971) p 125. Ibid., p. 132. (Stiny and Mitchell  1980). Ibid., p. 133. 72 76 (Knight and Stiny  2015). Ibid., p. 133. Koh Architectural Intelligence (2023) 2:7 Page 17 of 21 which in turn necessitate a shape as a figural object. The perceptual premise of shape grammar’s formalism is thus the seeing and counting of geometric shapes. The ques - tion then is: what if there is no shape? Or to put it in a more colloquial way: what if one is looking at a Monet and not a Sol LeWitt? Or in yet another way: what if it is a raster image made of pixels in Adobe Photoshop and not a vector graphic from Adobe Illustrator? Shape grammars are fundamentally defined by geometric trans - formations (i.e., translation, rotation, reflection and scal - ing) and boolean operations (i.e., union, intersection and subtraction), which are not possible with a Monet. To Fig. 19 Immanuel Koh, Discrete-Mies, 2018. Generated using a perceive, and thus to conceive, architecture from a shape probability distribution based solely on the frequency counts of each grammarist’s perspective is to say (using their same state- cell in the training set, but without sampling the spatial relations ment for paintings), “Architectures are material represen- among them tations of three-dimensional shapes generated by shape grammars”. In other words, architecture is the result of a Fig. 20 Immanuel Koh, Discrete-Mies, 2018. Generated using the learnt probability distribution based on local neighbourhood sampling but biased to only infer in the horizontal direction. The isometric view render shows the generated 3D configuration, the plan view render shows the inference arrows and tile indices, and the small diagram shows the contributing neighbouring tile (in black bold outline) used to generate the next tile (in red outline). All three preconditions are met here – a flattened set of tiles where each tile has the same size/resolution and its own neighbourhood transition matrix/probability Koh Architectural Intelligence (2023) 2:7 Page 18 of 21 Fig. 21 Immanuel Koh, Discrete-Mies, 2018. Generated using the learnt probability distribution based on local neighbourhood sampling but biased to only infer in the vertical direction. The isometric view render shows the generated 3D configuration, the plan view render shows the inference arrows and tile indices, and the small diagram shows the contributing neighbouring tile (in black bold outline) used to generate the next tile (in red outline). All three preconditions are met here – a flattened set of tiles where each tile has the same size/resolution and its own neighbourhood transition matrix/probability rule-based composition of solid volumetric objects trans- in full view but textured with the skin of an elephant, the formed and booleaned in space—architecture as object in machine would see an elephant and not a cat. This is a space. far cry from the human perception where it is the shape A recent article titled “Where We See Shapes, AI Sees that intuitively get prioritized. Yet, these CNNs are origi- Textures”, discusses a presentation at the International nally designed to imitate the hierarchical perceptual Conference on Learning Representations 2019 (ICLR) structure of the human visual cortex. In fact, the concept about the difference in perceptions between humans of CNN can be traced back to the 1950s work of David and today’s powerful deep learning models. Convolu- Hubel and Torsten Wiesel who discovered the neural tional neural networks (CNNs) being the state-of-the- basis of visual perception, that then became incorporated art deep learning models in vision tasks, are found to be in the design of CNN by Yann LeCun in the 1990s. In more biased towards texture than shape in recognising the experiments, it is found that visual signals are com- objects. In other words, given an image showing a cat municated in a hierarchical manner, with the first layer (Cepelewicz 2019). (Hubel and Wiesel  1959). 78 80 (Viola and Jones, 2001). (Lecun et al.,  1998). Koh Architectural Intelligence (2023) 2:7 Page 19 of 21 Fig. 22 Immanuel Koh, Discrete-Mies, 2018. Snapshot from an infinite set of generated Barcelona Pavilions, arranged based on their perceived formal fidelities to the original Barcelona Pavilion, itself placed in the middle of the array shown (or primary visual cortex) receiving signals from the eyes state-the-art machine learning approaches, the prospect directly. These signals are simply lower level visual repre - of sampling architecture is promising. sentations in the form of lines in varying orientations that have triggered the activations of the neurons. The recog -4 Sampling architecture nition of shapes does not occur on the first, but only on Leading theorist of digital culture Lev Manovich uses the those higher layers up in the hierarchy. In other words, term cultural sampling in his book “Cultural Analytics” shape entities are not the basic or primitive building to articulate the crucial difference when sampling cultures blocks of human perception. Building on this hierarchical using cultural datasets. Cultural analytics is therefore concept, the first real-time face detection algorithm by not simply data analytics. It refers to the computational Viola & Jones, makes use of a cascade of classifiers con - analysis of patterns in contemporary digital culture via sisting of simple binary rectangular pixel filters to iden - different digital media. In a similar vein, I came up with tify features of the human face, without having to define the term architectural sampling in 2019 as a way to inves- the shape of a face. In other words, shape entities are not tigate whether the sampling of architecture engenders a the basic or primitive building blocks of machine per- different set of perceptual concepts and computational ception as well. More critically, for the discussion here, processes, and if architectural sampling is not simply data all these examples show that human and machine visions sampling. Almost inevitably, it has led to a different way collectively reject shape as the primordial form and basis of seeing architectural forms through the lens of machine of ‘seeing’ and ‘learning’. If architecture can be perceived, learning. Thus, sampling architecture becomes a new way described and conceived beyond fully defined geomet - of seeing, learning and eventually generating architec- ric shapes, such as those well-defined shape primitives ture. Manovich’s ideal dataset is one that is complete (or found in shape grammars, but as abstract hierarchies at least, as complete as possible). A complete dataset, in of non-shapes/textures that are in resonance with the architectural terms, can only realistically exist in the form 81 82 (Viola and Jones 2001). (Manovich 2020). Koh Architectural Intelligence (2023) 2:7 Page 20 of 21 Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learn- of a single building, rather than all the buildings in the ing applied to document recognition. Proceedings of the IEEE, 86(11), world, or all the buildings designed by the same architect, 2278–2324. https:// doi. org/ 10. 1109/5. 726791 or all the buildings of a specific style, typology, region, Gristwood, S. (2018). Hiroshi Kawano (1925–2012): Japan’s Pioneer of Computer Arts. Leonardo, 52(1), 75–80. https:// doi. org/ 10. 1162/ leon_a_ 01605 material and so on and so forth. In this concluding sec- Noll, A. M. (1966). Human or Machine: A Subjective Comparison of Piet tion, I will use the complete digital model of Mies van der Mondrian’s “Composition with Lines” (1917) and a Computer-Generated Rohe’s 1929 Barcelona Pavilion as an example, to more Picture. Psychological Record, 16(1), 1–10. https:// doi. org/ 10. 1007/ BF033 vividly, illustrate the three preconditions of architectural Morton, T. (2018). Being ecological. Penguin Books Limited (Pelican Books). sampling in the project Discrete-Mies (2018), shown in the https:// www. pengu in. co. uk/ books/ 296515/ being- ecolo gical- by- morton- manner of an annotated pictorial series (Figs.  15, 16, 17, timot hy/ 97802 41274 231 Latour, B., Jensen, P., Venturini, T., Grauwin, S., & Boullier, D. (2012). “ The 18, 19, 20, 21 and 22). whole is always smaller than its parts” - a digital test of Gabriel Tardes’ monads: “ The whole is always smaller than its parts.” The British Journal Acknowledgements of Sociology, 63(4), 590–615. https:// doi. org/ 10. 1111/j. 1468- 4446. 2012. N/A 01428.x Carpo, M. (2019). Particlised: Computational Discretism, or The Rise of the Digital Authors’ contributions Discrete. Architectural Design, 89(2), 86–93. https:// doi. org/ 10. 1002/ ad. 2416 Written by Immanuel Koh. The author read and approved the final manuscript. Carpo, M. (2017). The second digital turn: Design beyond intelligence. The MIT Press. Authors’ information Alexander, C. (1964). Notes on the synthesis of form, 1st print. Harvard Univer- Immanuel Koh is an Assistant Professor in both the Architecture & Sustain- sity Press. able Design (ASD) and Design & Artificial Intelligence (DAI) programmes at Stiny, G. (2006). Shape: Talking about seeing and doing. The MIT Press. https:// the Singapore University of Technology and Design (SUTD) where he directs mitpr ess. mit. edu/ 97802 62195 317/ shape/ Artificial-Architecture — an AI & Architecture research laboratory. He was Russell, S., & Norvig, P. (2016). Artificial intelligence: A modern approach, appointed as the Hokkien Foundation Career Chair Professor in 2021 for his Global edition. Pearson Education. https:// books. google. com. sg/ cross-disciplinary research. Prior to obtaining his PhD from the School of books? id=_ BV6DA AAQBAJ Computer Sciences and Institute of Architecture at the École polytechnique Norvig, P. (2012). Colorless green ideas learn furiously: Chomsky and the fédérale de Lausanne (EPFL), he trained at the Architectural Association (AA) two cultures of statistical learning. Significance, 9(4), 30–33. https:// doi. and Zaha Hadid Architects in London. Immanuel is also the author of the book org/ 10. 1111/j. 1740- 9713. 2012. 00590.x ‘Artificial & Architectural Intelligence in Design’ published in 2020. Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal, 27(3), 379–423. https:// doi. org/ 10. 1002/j. Funding 1538- 7305. 1948. tb013 38.x N/A. Breiman, L. (2001). Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author). Statist. Sci., 16(3), 199–231. https:// doi. Availability of data and materials org/ 10. 1214/ ss/ 10092 13726 The datasets generated and/or analysed during the current study are available Stiny, G., & Mitchell, W. J. (1980). The Grammar of Paradise: On the Genera- from the corresponding author on reasonable request. tion of Mughul Gardens. Environment and Planning. B, Planning & Design, 7(2), 209–226. https:// doi. org/ 10. 1068/ b0702 09 Declarations Knight, T., & Stiny, G. (2015). Making grammars: From computing with shapes to computing with things. Design Studies, 41, 8–28. https:// doi. Ethics approval and consent to participate org/ 10. 1016/j. destud. 2015. 08. 006 N/A Hubel, D. H., & Wiesel, T. N. (1959). Receptive fields of single neurones in the cat’s striate cortex. Journal of Physiology, 148(3), 574–591. Consent for publication Manovich, L. (2020). Cultural analytics. Cambridge: The MIT Press. https:// N/A. mitpr ess. mit. edu/ 97802 62037 105/ cultu ral- analy tics/ J. M. Culkin (1967), ‘A Schoolman’s Guide to Marshall McLuhan’, The Saturday Competing interests Review,  51–53. The author is a member of the Editorial Board for Architectural Intelligence L. Young, (2019), ‘Machine landscapes: architectures of the post-anthropo- but was not involved in the journal’s review, or any decisions, related to this cene’, Architectural Design, 89, 1. submission. B. Sterling (2012), ‘An Essay on the New Aesthetic’, https:// www. wired. com/ 2012/ 04/ an- essay- on- the- new- aesth etic/. Accessed 11 July 2022. J. Bridle (2014), ‘On the Rainbow Plane | booktwo.org’, https:// bookt wo. org/ Received: 11 July 2022 Accepted: 15 February 2023 noteb ook/ rainb ow- plane/(accessed 9 Aug 2019). A. Hertzmann (2019), ‘Aesthetics of Neural Network Art’, arXiv: 1903. 05696 [cs],  Accessed: 19 Aug 2019. [Online]. Available: http:// arxiv. org/ abs/ 1903. 05696 A. Mordvintsev, C. Olah, and M. Tyka (2015). ‘Inceptionism: Going Deeper References into Neural Networks’, Research Blog,  http:// googl erese arch. Paglen, T. (2019). Invisible Images: Your Pictures Are Looking at You. Architec-blogs pot. com/ 2015/ 06/ incep tioni sm- going- deeper- into- neural. html tural Design, 89(1), 22–27. https:// doi. org/ 10. 1002/ ad. 2383 (accessed 11 Apr 2016). Terrence, J. (2018). Sejnowski, The deep learning revolution. The MIT Press. L. A. Gatys, A. S. Ecker, and M. Bethge (2015) ‘A Neural Algorithm of Artistic Schumacher, P. (2011). The autopoiesis of architecture: A new framework for Style’, arXiv: 1508. 06576 [cs, q-bio],  Accessed: 10 Apr 2016. architecture. Wiley. [Online]. Available: http:// arxiv. org/ abs/ 1508. 06576 Allen, S. (1997). From object to field + Architecture and urbanism. Architectural I. J. Goodfellow et al., ‘Generative Adversarial Networks’, arXiv: 1406. 2661 [cs, Design, 127, 24–31. stat], Jun. 2014, Accessed: Jan. 10, 2018. [Online]. Available: http:// arxiv. Allen, S. (1999). Points + lines: Diagrams and projects for the city. Princeton org/ abs/ 1406. 2661 Architectural Press. I. Koh (2019a). ‘Architectural Sampling: A Formal Basis for Machine-Learnable Architecture’,  doi: https:// doi. org/ 10. 5075/ epfl- thesis- 7815. Koh Architectural Intelligence (2023) 2:7 Page 21 of 21 I. Koh (2019b). ‘Discrete Sampling: There is No Object or Field … Just Statistical Digital Patterns’, Architectural Design (Special Issue: Discrete: Reappraising the Digital in Architecture), vol. 89, no. 2, pp. 102–109. S. Kwinter, ‘La Citta Nuova: Modernity and Continuity’, Zone, vol. 1, no. 2, 1986. A. Krizhevsky, I. Sutskever, and G. E. Hinton (2012), ‘ImageNet Classification with Deep Convolutional Neural Networks’, in Advances in Neural Informa- tion Processing Systems 25, F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, Eds. Curran Associates, Inc.,  pp. 1097–1105. Accessed: 18 Aug 2019. [Online]. Available: http:// papers. nips. cc/ paper/ 4824- image net- class ifica tion- with- deep- convo lutio nal- neural- netwo rks. pdf V. Sobchack (1990), ‘Towards a phenomenology of cinematic and electronic presence : The scene of the screen.’, Post script : essays in film and the humanities, 10(1), 50–59. R. Koolhaas, J. Westcott, and S. Petermann (2018), Elements of architecture. Köln: Taschen. W. J. Mitchell (1990), The logic of architecture: design, computation and cognition. Y. Zhou and O. Tuzel (2017), ‘VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection’, http:// arxiv. org/ abs/ 1711. 06396, Accessed: 28 Jan 2018. [Online]. Available: K. Kuma (2012), Anti-object: the dissolution and disintegration of architectures, Reprint., vol. 2. London: Architectural Association Publications. M. Carpo (2014), ‘Breaking the curve: Big data and design’, https:// www. artfo rum. com/ print/ 201402/ break ing- the- curve- big- data- and- design- 45013 (accessed 16 Aug 2019). I. Armeni et al. (2016), ‘3D Semantic Parsing of Large-Scale Indoor Spaces’, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, pp. 1534–1543. doi: https:// doi. org/ 10. 1109/ CVPR. 2016. 170. C. R. Qi (2017), pointnet: PointNet: Deep Learning on Point Sets for 3D Classifica- tion and Segmentation.  [Online]. Available: https:// github. com/ charl esq34/ point net G. Stiny and J. Gips (1971), ‘Shape Grammars and the Generative Specification of Painting and Sculpture’, in IFIP Congress (2),  pp. 1460–1465. R. Stouffs (2019), ‘Shape Rule Types and Spatial Search’, in Computer-Aided Architectural Design. ‘Hello, Culture’,  pp. 474–488. T. Knight (2018)  ‘Craft, Performance, and Grammars’, in Computational Studies on Cultural Variation and Heredity, J.-H. Lee, Ed. Singapore: Springer Singapore,  pp. 205–224. doi: https:// doi. org/ 10. 1007/ 978- 981- 10- 8189-7_ 16. R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, and W. Brendel (2018), ‘ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness’, arXiv: 1811. 12231 [cs, q-bio, stat],  Accessed: 13 Aug 2019. [Online]. Available: http:// arxiv. org/ abs/ 1811. 12231 N. Chomsky (1957), Syntactic structure. Mouton. Petrocelli, O. R., & Publishers, A. (1972). The best computer papers of 1971. Auerbach Publishers. https:// books. google. com. sg/ books? id= f6ezA AAAIA AJ J. Cepelewicz (2019), ‘Where We See Shapes, AI Sees Textures’, Quanta Maga- zine, Accessed: 13 Aug 2019. [Online]. Available: https:// www. quant amaga zine. org/ where- we- see- shapes- ai- sees- textu res- 20190 701/ P. Viola and M. Jones (2001), Rapid object detection using a boosted cascade of simple features. . Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in pub- lished maps and institutional affiliations. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Architectural Intelligence Springer Journals

Architectural sampling: three possible preconditions for machine learning architectural forms

Koh, Immanuel
Architectural Intelligence , Volume 2 (1) – Mar 21, 2023
Download PDF
21 pages

Loading next page...
 
/lp/springer-journals/architectural-sampling-three-possible-preconditions-for-machine-kv9vZEdZAQ

References (57)

Publisher
Springer Journals
Copyright
Copyright © The Author(s) 2023
eISSN
2731-6726
DOI
10.1007/s44223-023-00024-1
Publisher site
See Article on Publisher Site

Abstract

Koh Architectural Intelligence (2023) 2:7 Architectural Intelligence https://doi.org/10.1007/s44223-023-00024-1 Open Access COMMENTAR Y Architectural sampling: three possible preconditions for machine learning architectural forms Immanuel Koh back to 2012, when artist and writer James Bridle first Keywords Sampling, Flattened, Resolutional, Probabilistic, began to visualize ‘invisible’ technology via the notion Perception, Forms, Figure-ground, Part-whole, Shape of glitch. A new movement called the “New Aesthetic” grammars, Machine learning, Deep learning, Artificial was born, coined by Bridle in London, and endorsed by intelligence the internationally recognised cyberspace theorist Bruce Sterling. This new visuality concerns the increasingly pervasive vision of machines, ranging from satellite views “We shape our tools, and thereafter our tools shape us” that stitch maps from distributed sources to surveillance “We need to learn how to see a parallel universe cameras that do facial recognition of pedestrians on the composed of activations, keypoints, eigenfaces, fea- streets and to bots that watch over us on the internet. ture transforms, classifiers and training sets” Essentially a machine vision between that which is physi- cally real and digitally unreal. One of Bridle’s most rec- What does it mean to see architectural forms through ognizable works is the series on Rainbow Planes (Fig. 1). the sampling lens of the machines? A new way of seeing? It is an image of a plane that has moved fast enough to The first quote by John Culkin (a contemporary of Mar - disrupt the normal “panchromatic sharpening’ pro- shall McLuhan) in 1967 is a reminder of the reciprocal cess when merging pixels from different satellites’ sen - relationship between human and technology. The sec - sors, thus resulting in the successive RGB rainbow-like ond quote, just half a century later, comes from the arti- imprints. cle “Invisible Images” by Trevor Paglen for the February This awareness of machine vision has further intensi - 2019 issue of Architectural Design (AD) titled “Machine fied with the rise of deep learning – from the close read - Landscapes: Architectures of the Post-Anthropocene”. ing of glitch samples in the New Aesthetic to the more Paglen is not alone in acknowledging this new way of recent deep neural network layers’ visualization of hal- seeing—a new visuality characterized by today’s AI tools lucination samples in the Neural Aesthetics. Both Deep- of deep neural networks. In fact, this notion of machine 8 9 Dream (Fig.  2) and Neural Style Transfer demonstrate vision and its ramification on aesthetics can be traced (Culkin 1967) p. 50. (Paglen 2019) p. 27. (Sterling 2012). (Young and Ed., 2019). See Rainbow Planes Flickr album for more examples: https:// www. flickr. com/ photos/ stml/ sets/ 72157 63793 80610 15/ with/ 10994 081436/ *Correspondence: (Bridle 2014). Immanuel Koh (Hertzmann 1903). immanuel_koh@sutd.edu.sg (Mordvintsev et al., 2015). Singapore University of Technology and Design, Singapore, Singapore (Gatys et al., 1508). © The Author(s) 2023. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/. Koh Architectural Intelligence (2023) 2:7 Page 2 of 21 Fig. 1 James Bridle, Rainbow Planes. 2013. Coordinates: Chicago: 41.785420°, -87.578890°, 1/7/2010. The New Aesthetic situates machine seeing as an assemblage of different data sources as fragments Fig. 2 Google, DeepDream, 2015 (Left) original input image. (Right) generated output. By zooming in, one could see various neural hallucinations as a result of the ImageNet dataset used for training the deep learning model (GoogLeNet). The Neural Aesthetic situates machine seeing as an assemblage of different data samples as features that by manipulating the layers of a convolutional neu- disposed within the discipline, for it to partake in what ral network, one could alter and extract aspects of its Terry Sejnowski refers to as the deep learning revolution perception to generate new forms. In short, with new and what Andrej Karpathy alludes to as Software 2.0 ? In perceptions come new conception of forms. This gen - my theory of Architectural Sampling, I proposed to first erative potential, especially with the Generative Adver- debunk the three ingrained ways of thinking about archi- sarial Network (GANs), has now even been taken up tectural form, in order to lay the foundational precondi- by world-renowned artist Pierre Huyghe in his UUmwelt tions for a machine-learnable architecture. exhibition at the Serpentine Gallery, as well as being auc- The first of the three preconditions, “No Figure, No tioned off at Christie’s. Ground”, looks at the most basic representation of archi- Our visual culture’s notion of perception has become tecture— the two-dimensional black and white figure- increasingly machinic, it is time to reconsider architec- ground. A dualistic representation of mass and void (or of ture in the light of this phenomenon. What does it mean form and space). This inherently layered representation to see architecture through the sampling lens of machine of architecture will be flattened, such that there is neither vision and deep learning? What are the longstanding figure nor ground. Flatness is the new field. The second conceptual and perceptual assumptions that need to be of the three preconditions, “No Parts, No Whole”, looks at the parts-to-whole and whole-to-parts relationship of 10 12 (Goodfellow, et al., 1406). (Terrence 2018). 11 13 See article “Is artificial intelligence set to become art’s next medium?” See article “Software 2.0” https:// medium. com/@ karpa thy/ softw are-2- 0- https:// www. chris ties. com/ featu res/A- colla borat ion- betwe en- two- artis ts- a6415 2b37c 35 one- human- one-a- machi ne- 9332-1. aspx (Koh 2019a) This text is largely taken from the chapter “Perception through Sampling” of my PhD thesis, but revised significantly to fit with the publica - tion constraints of this journal. Koh Architectural Intelligence (2023) 2:7 Page 3 of 21 architecture in three-dimensions and argues that there is of 1997 and to Patrik Schumacher’s “parametric figura - in fact neither parts nor whole after all, but only resolu- tion” of 2011, many have attempted to propose alterna- tions. Resolution is the new reality. The last of the three tive theories by replacing the ground of the figure-ground preconditions, “No Shapes, No Grammars”, looks at one with field to debunk the figure-ground in architecture. of architecture’s most respected computational paradigm Interestingly, just a year after the publication of Allen’s of shape grammars and claims that it is the edges, not the Field Conditions, outside of architecture, deep learning shapes, that matters at the primordial level; and a proba- pioneer Yann LeCun proposed the first Convolutional bilistic model is a more realistic model than deterministic Neural Network known as LeNet-5 in 1998. The con - grammars. Making decisions is probably better with sta- cept of convolution alone is potent enough to, once and tistics than rules. for all, dispense with the figure-ground (or field) dilemma Each precondition will be discussed and accompanied faced by those architects mentioned. Unfortunately, such by its successive pictorials (i.e., annotated standalone a radical architectural concept remains unknown, even figures). Similarly, I will conclude the text with the pro - after the year when the paper “ImageNet Classifica - ject “Discrete-Mies” to more vividly illustrate how the tion with Deep Convolutional Neural Networks” by Alex proposed three preconditions of architectural sampling Krizhevsky, Ilya Sutskever, and Geoffrey Hinton was pub - could be put in practice to machine-learn Mies van der lished and made headlines when it won the 2012 ILSVRC Rohe’s 1929 Barcelona Pavilion and to generate new Bar- (ImageNet Large-Scale Visual Recognition Challenge) by celona Pavilions of infinite variations. a large margin. The deep neural network model known as AlexNet, itself a direct descendent of LeNet-5, has since 1 No figure, no ground given rise to the deep learning revolution of today. One Figure: might ask now: In what way does such a breakthrough in AI and computer vision has an impact on architec- “The form of anything as determined by the out - tural thought, and specifically, the architectural percep - line.” tion and conception of the figure-ground? To answer this question, we will have to take a step back and recon- Ground: sider: what is the fundamental formal (thus perceptual) “The bottom; the lowest part or downward limit of assumptions of the traditional figure-ground, and in what anything.” way has the new visuality challenged those assumptions? (Figs. 3, 4, 5 and 6). Schumacher uses the term “parametric figuration” to overcome the limitation posed by the classical dual 1.1 No figure concept of g fi ure-ground. Accordingly, these new figu - The training of an architect, may it be during the pre-dig - rations emerged and submerged within a field which is ital or the post-digital times, is closely related to the tools governed by an underlying set of rule-based parameters. of design representation used. One of the most consistent In contrast to modern composition, parametric space is features of these representation tools has been the ability characterised by collective figurations over singular fig - to work in layers. This concept of layering can be found ures, producing global “biases, drifts, gradients”. Schu- in the analogue layering of tracing paper, as well as the macher is not the only who has sought to abandon the digital layer manager embedded in CAD softwares (e.g., figure-ground concept that so permeate architectural Autodesk’s AutoCAD or McNeel’s Rhinoceros) or other thought and perception. From Sanford Kwinter’s Ein- 20 21 graphics editors (e.g., Adobe’s Photoshop or Illustrator). steinian Field of 1986, to Stan Allen’s Field Conditions To represent a design as a figure-ground is essentially to 15 graphically place a figure on a ground. There may be many (Koh 2019b). 16 figures on many grounds, like in Photoshop or AutoCAD, See the video recording of the “AI & Architectural Design” panel discus- sion where Patrik Schumacher expressed that the project has convincingly separated and stacked one after the other. The removal of machine-learned the formal and spatial features of the Barcelona Pavilion. one figure from layer A will not affect those other figures h t t p s :// w w w . yout u b e. c om/ w a t c h? a pp= de skt op&v= u- 8KO_ 7ycgE & list= on layer B in any way. But once the layers are flattened, PLjgJ TkiYX dLcxh B3WTt Roxl7 f52vf dNcQ& index= 19 the pixels’ value can no longer be swapped among lay- See Oxford English Dictionary.https:// www. oed. com/ view/ Entry/ 70079? rskey= Gpov5 F& result= 1# eid ers, and to remove or move a figure would imply a cut - Ibid. https:// www. oed. com/ view/ Entry/ 81805? rskey= Ig9VL q& result= 4# ting (or Photoshop’s ‘lasso’) and pasting of the figure. It is eid (Schumacher 2011). p. 424. (Schumacher 2011). (Kwinter 1986). (Allen 1997), (Allen 1999). (Lecun et al.,  1998). (Krizhevsky et al., 2012). Koh Architectural Intelligence (2023) 2:7 Page 4 of 21 neural network (CNN) will take the input as a tensor of size 256 × 256 × 3, where the third dimension represents the vector of 3 numbers from the image’s RGB channels. u Th s, there is no concept of an explicit differentiation of a figure and ground when AlexNet looks at an image, it simply takes those tensors of numbers. The same princi - ple applies to the SpaceNet, which is similar to the Ima- geNet, except that the dataset is a corpus of high-quality commercial satellite imagery instead of objects. In a way, one could say that it is all ground and no figures. 1.2 No ground From a computational and generative design perspective, the abandonment of the layered figure-ground for a flat - tened machine vision can also be traced back to the com- puter art scene of the 1960s, specifically in the pioneering works of philosopher-programmer Hiroshi Kawano, as contrast with those of mathematician-computer scientist Frieder Nake and engineer Michael Noll. Unlike many of his contemporaries, Kawano’s early work was based on data-driven techniques rather than the explicit program- ming of rules. In his introductory overview of Kawano’s work, Gristwood writes that Kawano was influenced by his reading of Max Bense’s work on Information Aesthet- ics and Claude Shannon’s work on Information Theory and Communication. A striking difference can be found Fig. 3 Excerpt of the many generic figure-ground field diagrams when studying Nake’s Hommage à Paul Klee 13/9/65 used by Stan Allen in his ‘Field Condition’. The bottom-left diagram Nr.2 of 1965, Noll’s Computer Composition with Lines called ‘field vectors’ has been used for the demonstration in Figs. 4 & of 1964 and Kawano’s Artificial Mondrian of 1969. Nake 6. Image as published in the original paper (Allen 1997) first abstracted Klee’s figural lines as explicit rules before plotting them on an empty background of continuous paper space. Kawano, on the other hand, neither differ - this habitual and convenient perception of a floating layer entiates the figure and the ground as separate entities nor called figure over a base layer called ground, that needs attempts to write a Mondrian shape grammar. He simply to be first questioned and eventually replaced by a ‘flat - decomposed Mondrian’s painting into a grid of equally tened’ perception. sized colour blocks for the computer to statistically learn Digitally speaking, the architect’s figure-ground can be their probability distribution. He then sampled from that understood as a binary image where each pixel value is a same distribution to infer new Mondrians. Nake used the 1-bit number (0 as black or 1 as white). In the case of a Zuse Graphomat Z 64 (1960) by Konrad Zuse to plot fig - grayscale image, each pixel value would be an 8-bit inte- ures on ground, while Kawano used the earliest IBM line ger (0 to 255); and for coloured RGB images, each pixel printer to print lines of characters, where each character value would be a vector of 3 numbers. Returning to the represents a colour block. original discussion on machine vision, unlike the ways Even though both Kawano and Noll had used Mondri- in which human would see a figure by separating it from an’s work as their starting point, their aims and processes the ground, the AlexNet did not achieve its 84.7% object were radically different. For Noll, it is about imitating classification accuracy in a similar manner. Instead, it a human vision as he imagines Piet Mondrian himself takes in a ‘flattened’ image input in their numerical repre - holding the paint brush and painting black figures on the sentation as tensors, systematically scanning, convolving white canvas/ground. This intended imitation is evident and pooling (downsampling) them, and outputs a proba- bility distribution of the 1000 object classes with which it was trained on, to make a prediction. For example, given SpaceNet. https:// space netch allen ge. github. io/ an image of size 256 × 256 pixels, the deep convolutional ImageNet. http:// image- net. org/ index (Gristwood Jan. 2018). 25 29 (Krizhevsky et al., 2012). (Noll  1966). Koh Architectural Intelligence (2023) 2:7 Page 5 of 21 Fig. 4 Immanuel Koh, Flattened Field Conditions ( With Figure / With Ground), 2018. One of the many generic figure-ground field diagrams used by Stan Allen is here perceived as 28 unique black figures. It is a traditional and typical reading of the 1990s field conditions where the perceptual distinction of figures and ground is clearly intended Fig. 5 LeNet-5 architecture reflecting the hierarchical perceptual structure of the human visual cortex. The ‘flattened’ input image (leftmost layer) is perceived as a tensor of size 32 × 32x1 and convolved with a smaller scanning window/kernel of 5 × 5x1, without any differentiation of figure or ground. Image as published in the original paper (Lecun et al., 1998). from his article “Human or Machine: A Subjective Com- learning a machine vision as he imagines the computer parison of Piet Mondrian’s ‘Composition with Lines’ (1917) scanning and learning the statistical distribution of col- and a Computer-generated Picture”, where he reported our blocks represented with symbols, without seeing that only 28% of the 100 human subjects were able to any figures and ground. In fact, Kawano’s discrete col - identify the computer-generated picture. Likewise, Noll’s our blocks are what Sobchack might have referred to as conception of a figure-ground process is evident from ‘discrete pixels’. Interestingly, or even prophetically, this his use of a General Dynamic SC-4020 Microfilm Plotter machinic pattern-recognition process of ‘statistical seeing’ to move the beam of a cathode ray tube in drawing the is not too dissimilar to how AlexNet (as well as all vari- figures before photographing it. For Kawano, it is about ants of the convolutional neural networks) is trained from 30 31 (Noll  1966). (Sobchack 1990). Koh Architectural Intelligence (2023) 2:7 Page 6 of 21 Fig. 6 Immanuel Koh, Flattened Field Conditions ( Without Figure / Without Ground), 2018. One of the many generic figure-ground field diagrams used by Stan Allen is here discretized, convolved, and then encoded as a combinatorial instance from 28 unique part-figures/part-grounds patterns. It is an alternative reading of the 1990s field conditions where the distinction of figure and ground is completely abandoned big datasets of images to detect and recognise all sorts of 2 No parts, no whole objects. These training sets of images are similarly simply Whole: ‘discrete fields’ or mathematical matrices of numbers rep - “Complete, undivided, total.” resenting colours. Therefore, not only are there no figures, there is effectively no ground as well. Oxford English Dictionary. https:// www. oed. com/ view/ Entry/ 228723? rskey= j7yrc q& result= 1& isAdv anced= false# eid Koh Architectural Intelligence (2023) 2:7 Page 7 of 21 Part: ontologically unstable. A floor is a floor and nothing else. How true is the preceding statement? Koolhaas himself “A piece or section of something which together with has fought hard throughout his career to falsify the state- another or others makes up the whole (whether actu- ment, as testified by the Jussieu library project which has ally separate from the rest or not)” since triggered a whole series of subsequent built projects In the 2600-page book “Elements of Architecture”, based on the notion of connected floors forming a single Rem Koolhaas organises its content according to the continuous spatial trajectory. Meaning, a floor is also a semantic parts of a building—floor, ceiling, roof, door, ramp or stair or corridor or anything in between (Figs. 7, wall, stair, toilet, window, facade, balcony, corridor, fire - 8, 9 and 10). place, ramp, escalator and elevator. The book is in fact a compilation and expansion of an earlier exhibition at the 2.1 No whole Venice Biennale 2014 and impressively traces the global Similar endeavours in the history of architecture to defy history of each of these architectural elements. Accord- such established architectural semantics have helped ing to such an understanding, architecture is then a whole reinvent architecture both conceptually and formally. made up of different constituent parts , each performing One of them is the seminal theory of ‘anti-object’ by a specific function in relation to its spatial allocation. Kengo Kuma, specifically his notion of ‘particlisation’. The objective for an architecture is then to configure Kuma’s particlised architecture has more recently been that whole, with the belief that Aristotle’s “The whole is adopted by Mario Carpo in his inauguration of the digital greater than the sum of its parts” holds true. Others have discrete. Since the proposition of ‘No Parts, No Whole’ disagreed with such a prioritization of the whole over is closely related to Kuma’s theory and Carpo’s extension, the parts. For one, is the philosopher of object-oriented it is worthwhile taking a moment to introduce them here. ontology, Timothy Morton, who writes “The whole is “No particular skill or effort is required to turn always less than the sum of its parts” to remind us that something into an object. Preventing a thing from a whole is outsized by its parts since there are potentially becoming an object is a far more difficult task.” infinite regress of parts within it. Or French philosopher of the actor-network theory (ANT) Bruno Latour who A quote above from Kengo Kuma in Brett Steele’s writes “The whole is always smaller than its parts” to introduction to the book “Anti-Object: The Dissolution emphasize the connections among entities which could and Disintegration of Architecture”, which exposes the be humans or non-humans. Or like Gestalt psycholo- ease and tendency of thinking and designing architecture gist Kurt Koffka’s “The whole is other than the sum of its as object. In the book’s preface, Kuma defines an object parts” which avoids any measurable comparison between as “a form of material existence distinct from its immedi- the whole and the parts by stating that they are simply ate environment.” By first defining what is an object, he different. It is beyond the scope and purpose of this text proceeds to define his so-called “anti-object” via a gen - to discuss these aforementioned philosophical debates. eral strategy of negation in the rest of the book. Some of Instead, the focus here is to see the conceptual benefits these statements of negation include: “That is why par - engendered by a different perception of the whole and ticlised architecture is the polar opposite of photographic the parts, and in our case, the abandonment of both. architecture. The silhouette is ambiguous.” and “Archi- Returning to Rem Koolhaas’ categorical assignment of tecture is another name for the aggregation of matter (i.e., architectural elements, this text is thus suggesting oth- the creation of object), and ‘particlisation’ is the reversal erwise. Meaning, the precondition for architectural sam- of the aggregation.” Most relevant to our discussion here pling is the denial of the meaning of each part. That is to is his last chapter “Breaking Down into Particles”, where say, instead of Koolhaas’ 15 specific and ontological sta - he elaborates on the concept of “particlisation” in archi- ble parts of a building, there are in fact an indeterminate tecture. Using the impressionist painter Georges Seurat’s number of possible generic parts that encodes a build- pointillism and gothic architecture style as his departure ing’s form, according to the discretization and quantiza- tion process of the proposed sampling. The argument is (Kuma 2012). based on the reality that architectural form is inherently 38 (Carpo 2019). 39 st (Kuma 2012) p.2. & 51. The 1 print was in 2008 and the original Japa- nese version is in 2000 by Chikuma Shobo (Tokyo). Ibid. Ibid. https:// www. oed. com/ view/ Entry/ 138188? rskey= fGtZC w& result= 1& isAdv anced= false# eid Ibid., See Preface. 34 42 (Koolhaas et al., 2018). Ibid., p.106. 35 43 (Morton 2018) pp. 92–93. Ibid., p.119. 36 44 (Latour et al., 2012). Ibid., pp.98–210. Koh Architectural Intelligence (2023) 2:7 Page 8 of 21 Fig. 7 The decomposition of the Parthenon into its semantic parts has become the data structure of architecture’s digital representation and subsequently used as the underlying ontology of today’s BIM (Building Information Modelling). Image as published in the book (Mitchell 1990). points, Kuma illustrates how “particlisation’ is at work decomposed abstraction, in order to attain an anti-object in both. The former concerns the decomposition of pic - configuration. torial objects with the multiplicity of uniformly sized In referencing modern architecture, Kuma points out paint dabs, while the latter concerns the decomposition that Walter Gropius’s Dessau Bauhaus and Le Corbusier’s of building form with a formal articulation based on the Maison Dom-ino are in fact key modernist moments in multiplicity of small elements. Both adopted a strategy of the “particlisation” of architectural form, most notably Koh Architectural Intelligence (2023) 2:7 Page 9 of 21 Fig. 8 (Left) Photograph of a 1765 Neoclassical armchair from the V&A Museum in London. A Classical Relationship of Whole-to-Part: Each part is designed to serve a specific function with a specific meaning to produce a complete and perfect whole. (Right) Photograph of the original 1923 Berlin chair by Rietveld. A Modernist Relationship of Part-to-Whole: Each part is designed with interchangeable functions and semantics, where parts are assembled to form a possible whole Fig. 9 VoxelNet deep neural network architecture showing the decomposition of a point cloud into equally sized generic voxels. Points in each voxel are grouped and transform as a vector representation for learning the features to detect objects of different probable semantics. This suggests a Machinic Sampling Relationship of No-Parts-No-Whole. Image as published in the original paper (Zhou and Tuzel 2017). through their use of dry construction in the context of reinforced by the medium of dissemination, in the form prefabrication. However, he writes that Le Corbusier of two-dimensional photographs, which requires an almost immediate shift to an architecture of objects was unambiguous depiction of the building’s silhouette for a result of the shift in his approach to painting – from effective communication. The object architecture of both cubism to suprematism and purism. This was further Le Corbusier’s Villa Savoye and Mies van der Rohe’s Bar- celona pavilion are reinforced by their respective use of the pilotis and plinth, as a formal means to isolate their Ibid., p.104. Koh Architectural Intelligence (2023) 2:7 Page 10 of 21 Fig. 10 Immanuel Koh, Resolutional Berlin Chair, 2019. An illustration of a relationship of No-Parts-No-Whole. (Left) Digitally remodelled Rietveld’s Berlin chair. (Right Top) Exploded rendered view of the decomposed units at different resolution (i.e., 60, 90 & 180). (Right Bottom) Automatically generated coloured voxel semantic maps at different resolution (i.e., 60, 90 & 180). Each unique semantic voxel is coloured (thus labelled) according to their formal and configurational similarity architecture from the ground, especially when viewed as Upon closer reading, the underlying argument of Carpo images. In fact, Kuma even extrapolated his “particlisa- has always been an attack and upending of the first digital tion” theory to cities, calling it “particle-based urban the- style—the smoothness of the spline, with the second digi- 46 48 ory”. It is a critique of modern city planning’s failure of tal style of the discrete. Even without Carpo’s rhetorical regulating city based on the means of objects and enclo- allusion of a brute-force approach with big data replacing sures. Objects are manifested in the form of monumental the elegant mathematical function of small data, with- buildings and enclosures in the form of zoning or theme out an architecture conceptualized as a whole ‘object’, parks. smoothness cannot be perceived readily. For example, For Kuma, the preconditions of particlising archi- the 90  s Embryological House of Greg Lynn would not tectuire is first the dissolution of the object and sec - have looked smooth if its deformation is not applied on ond the homogenization of the parts. In other words, a single whole object. Or conversely, the 2010s Serpen- it is the decomposition of a whole into a multiplicity of tine Pavilion by Sou Fujimoto would not have looked dis- generic(non-semantic) parts, which in turn, leads to crete if its aggregation is not with a multiplicity of generic an inevitable formal consequence of no whole, no parts parts. Therefore, to oppose smoothness is consequently as proposed here. However, if the whole and parts are to oppose wholeness. To lose the whole, is then to shift all lost, in what way does it contribute to the computa- the focus to its parts. And to lose the parts, is to make tional formulation of sampling architectural forms with them generic without semantics. Finally, once we lose the machine learning. This is the moment where Carpo’s specificity of their semantics, it suddenly becomes pos - pairing of Kuma’s theory and the computational discrete sible to normalize and sample them simply as numerical becomes critical. In his article titled “Particlised: Compu- representations with machine learning and deep learning tational Discretism, or The Rise of the Digital Discrete”, apparatuses. Mario Carpo draws a convincing parallel between the work of Kengo Kuma and those of today’s young avant-2.2 No parts garde of the “second digital style” or “computational Unfortunately, the semantic parts are the most diffi - discretism”, by highlighting a set of shared formal vocab- cult to be forsaken. The longstanding presumption of ulary characterized by terms such as, “disjointed”, “dis- defining a building according to its semantic constitu - connected”, “fragmentary”, “voxellised”, filamentous” and ents has remained extremely ingrained within the dis- “chunky”. Referring to Kuma’s own definition of “parti - cipline. In the “The Logic of Architecture”, Mitchell clisation”, Carpo reminds us of Kuma’s motivation for his shows how the Parthenon could be digitally decom- current “non-figural, aggregational, atomised” style, as a posed with its entablature components encoded in a deliberate attack at the material continuity of modernism relational lattice and their orientations mapped with a (specifically the use of wet construction in concrete). (Carpo 2017). 46 49 Ibid., p.114. (Carpo 2014). 47 50 (Carpo 2019). (Mitchell 1990). Koh Architectural Intelligence (2023) 2:7 Page 11 of 21 function accordingly. Such a prescriptive approach has neural network layers. In short, the heterogenous geo- now become a standard with BIM (Building Informa- metric parts are replaced by homogenous voxels, while tion Modelling). For example, Autodesk’s Revit Archi- the hierarchy of semantics is replaced by the hierarchy of tecture, a BIM software, is built in such a way that any abstract features. model element has to be first assigned as either a host (walls, floors, roofs or ceiling) or a model component 3 No shapes, no grammars (stairs, windows, doors or furniture). This ‘host and “External form or contour; that quality of a mate- hosted’ relationship, such as a window being hosted by rial object (or geometrical figure) which depends on a wall, reflects the exact parallel with Mitchell’s schema constant relations of position and proportionate dis- of a frieze hosted by an entablature. To maintain such an tance among all the points composing its outline or explicit relationship, a BIM model requires a literal tax- its external surface.” onomy of a building’s constituent parts as objects with specific semantics to be stored in its database. Grammar: One might ask: How then could a building be repre- “That department of the study of a language which sented if not with their semantic constituent parts? A deals with its inflectional forms or other means of look at 3D scanning of buildings might provide an alter- indicating the relations of words in the sentence, native mode of perception and representation of archi- and with the rules for employing these in accordance tectural forms. Instead of modelling solids or surfaces with established usage” directly, the scanned point cloud is a collection of 3D points with x, y, z coordinates; and unlike voxels, can Numerous researchers have devoted their entire be unevenly spaced, sparse and massive in data points. careers in the continual development (and establishment) From photogrammetry that outputs 3d coordinates of shape grammar since its inauguration in 1971. Shape with 2D photographs, to LIDAR (Light Detection and grammar is a set of rules of transformation applied recur- Ranging) that outputs 3d coordinates by laser scanning sively to an initial shape, generating new shapes. The in 3D directly, the representation of raw point clouds is rules have the general form of A → B, where A and B are never semantic. It is only through their post-processing shapes and the sign ‘→ ’ denotes a replacement step. For with manual or machine-learning based semantic seg- example, given an initial shape S, if shape A is be found mentation that their functions are defined. Returning in it, subtract A from S and replace it with B to generate a to Kuma’s theory of particlisation, the point cloud rep- new shape S’. This generative process can be simply stated resentation of a building is literally a particlised archi- as ‘seeing and doing’. The research in shape grammars tecture. In this particlised conception of architecture, has encompassed several areas of design, including urban to refer to its parts, is to refer to either each point in design, architecture, product design and mechanical the point cloud or a variably sized cluster of points or design, and is now being technically incorporated in par- sub-cloud. In other words, there are no real ‘parts’ any- ametric software and theoretically extended for mate- more, just numerical values of x, y, z and optionally, that rial-based performative making. of colours and normals too. The hierarchical definition In view of all these accomplishments within the shape of architecture in BIM, whether implicitly or explicitly grammar community, it would be an impossible feat indebted to Christopher Alexander, is now supplanted in attempting to overthrow the entire enterprise and to by a different hierarchy, not of parts but of features. This propose a better theoretical alternative. However, this new hierarchy of formal features is to be found in the is not my objective here. Rather, it is a provocation to post-processing of these generic point clouds. More con- reconsider the shapes and grammars of shape grammar. cretely, in the layers of deep neural networks, such as the This reconsideration would then lead us to expose the VoxelNet. Essentially, the VoxelNet first takes a point underlying formal and perceptual limits of this design cloud as its input and subdivide it into evenly spaced 3D paradigm—a stumbling block in the formulation of a for- voxels. Points that are contained in each voxel is grouped mal basis for machine learning architecture. The shape and transform into a feature representation through its of shape grammar is the cause of its perceptual (‘seeing’) See Oxford English Dictionary. See Revit Getting Started. https:// knowl edge. autod esk. com/ suppo rt/ revit- Ibid. produ cts/ getti ng- start ed/ caas/ Cloud Help/ cloud help/ 2018/ ENU/ Revit- GetSt arted/ files/ GUID- 5BFA4 99A- 5ACA- 4069- 852C- 9B60C 9DE67 08- htm. html (Stiny and Gips 1971). 52 58 (Armeni, et al., 2016), (Qi and pointnet: PointNet, 2017). (Stiny 2006). 53 59 (Alexander 1964). (Stouffs 2019). 54 60 (Zhou and Tuzel 2017). (Knight 2018). Koh Architectural Intelligence (2023) 2:7 Page 12 of 21 Fig. 11 Superstudio, Histogram of Architecture, 1969–1971. It is a catalogue consisting of 33 three-dimensional abstract forms to generate designs across scales. Unlike in shape grammars, instead of defining a set of rules, the Histogram of Architecture is conceptually a ‘training set’ which represents a mental latent distribution of probable forms of the same architectural language. The word histogram in the title and the grid surfaces of the forms clearly allude to a machinic form of statistical seeing Fig. 12 Classification by a Convolutional Neural Network showing texture as the primordial feature over shape. Image as published in the original paper (Geirhos et al., 1811).(Geirhos et al., 2018) limit, while its grammar, the cause of its formal (‘doing’) so is its underlying rewriting schema. Thus, it would be limit (Figs. 11, 12, 13 and 14). most fruitful to at least briefly address Chomsky’s gram- mar directly in the language domain before returning to 3.1 No grammars the grammar in the architecture domain. In order to do To disagree with the grammar of Stiny’s shape grammar so with a higher probability of success, the arguments put means to first disagree with Chomsky himself, specifically forward by one of Chomsky’s strongest critics, computer his Generative Grammar, of which Stiny has appropri- scientist Peter Norvig, will be presented first. ated for his architectural formulation. The formal appro - Peter Norvig, director of research at Google and co- priation is so direct that even in the 1971 paper, shape author of the leading textbook “Artificial Intelligence: A grammar is defined exactly in accordance to the standard Modern Approach”, wrote a 2012 article titled “Col- definition of phrase structure grammar of Chomsky, ourless green ideas learn furiously: Chomsky and the two cultures of statistical learning”. In his article, (Chomsky 1957). (Stiny and Gips 1971). (Russell et al., 2016). 63 65 (Chomsky 1957). (Norvig Aug. 2012). Koh Architectural Intelligence (2023) 2:7 Page 13 of 21 Fig. 13 Immanuel Koh, Probabilistic Histogram of Architecture, 2019. Reinterpretation of the original Histogram of Architecture in Fig. 11 (i.e., last 3 columns of its top row) where the histograms are re-perceived and re-represented as shapeless and grammarless training set of forms. Presented in the same manner as the original catalogue with sets of 3 diagrams laid out in a vertically order, each diagram is also accompanied by its corresponding matrix representation used as machine learnable numerical representation Koh Architectural Intelligence (2023) 2:7 Page 14 of 21 Fig. 14 Immanuel Koh, Probabilistic Histogram of Architecture, 2019. Reinterpretation of the original Histogram of Architecture in Fig. 11 (i.e., all 6 columns of its bottom row) Norvig begins by referring to Chomsky’s remark at the expressed by P(*IE), he convincingly proves the syntactic Brains, Minds, and Machines symposium held at MIT model of Chomsky lacking in accuracy and expressiv- in 2011, that reinstates Chomsky’s well-known disap- ity. Referencing statistician Leo Breiman, he points out proval of statistical models in their usefulness for lan- the two statistical cultures—data modelling culture and guage tasks. First and foremost, what is a statistical algorithmic modelling culture, with which Breiman and model and a probabilistic model? Accordingly, a sta- himself object the former and Chomsky objects the latter. tistical model is “a mathematical model which is modi- The reason for Breiman’s objection is that it is a futile task fied or trained by the input of data points”, while a to uniquely model the truth of nature (or language) and probabilistic model “specifies a probability distribution seeking to make known all the underlying parameters in over possible values of random variables” Essentially, an explainable form; instead, one should be satisfied with probabilistic model does not state an exact determin- a messy function that could sufficiently approximate the istic functional relationship. Using Claude Shannon’s observed data. Accordingly, Chomsky’s favouring of the probabilistic language(communication) model based former is based on the latter’s lack of non-messy and on Markov chains as an example, Norvig reminds us that explanatory principles (or rules). language could escape from the syntactical insistence of Returning to our original discussion of the term gram- Chomsky. Norvig then highlights the disadvantages of mar in shape grammar, the preceding text aims to high- Chomsky’s logical (or categorical) model and the advan- light a similar formal dilemma in architecture. Shape tages of models that are statistical and probabilistic. The grammar shares the same explanatory principles of former is based on logical rules and expresses a boolean Chomsky’s generative grammars with its elegant formal- distinction, whereas the latter models a probability dis- ism of “seeing and doing”, but similarly, it also lacks the tribution. Demonstrating with a simple example of Eng- expressivity of Norvig’s statistical and probabilistic mod- lish spelling, a non-statistical model expressed by the els. If one is to agree with Chomsky that a linguist’s job rule “I before E except after C” and a probabilistic model is to define rules that could distinguish between what is grammatical and ungrammatical, then it would not be Ibid. Ibid. (Breiman  2001). 68 70 (Shannon 1948). (Stiny 2006). Koh Architectural Intelligence (2023) 2:7 Page 15 of 21 Fig. 15 The original floor plan of Mies van der Rohe’s 1929 Barcelona Pavilion redrawn by the author. It is the basis for the digital reconstruction of the 3D model, which is in turn, used for the sampling experiments Fig. 16 Immanuel Koh, Discrete-Mies, 2018. Sets of unique cells convolved at different sampling resolution: (top-left) 2.0 m X 2.0 m, (bottom-left) 1.0 m X 1.0 m (top-right), 0.5 m X 0.5 m and (bottom-right) 0.25 m X 0.25 m. Each cell can be observed in the input architectural model of Barcelona Pavilion. (Middle-column) Each red square represents the relative size of cells according to their sampling resolution, thus providing a way to compare at the same scale too incorrect to say the same has happened to architec- been continually made to develop more and more gram- ture researchers in defining rules to distinguish different mars for a variety of architectural languages. Many of architectural styles or languages. Reflecting on the past these grammars are painstaking defined with basic shape fifty years of research in shape grammars, efforts have primitives and several dozens of rules. For example, the Koh Architectural Intelligence (2023) 2:7 Page 16 of 21 Fig. 17 Immanuel Koh, Discrete-Mies, 2018. Mies van der Rohe’s 1929 Barcelona Pavilion (with roof removed to better show the spatial configuration) as the original input architectural model for the sampling procedures as grammatical (i.e., a deterministic rule-based model) will hinder the formulation of a theoretical and technical framework for machine learning architecture. 3.2 No shapes It is worth quoting Stiny and Gips in full here: “A formal- ism for the complete, generative specification of a class of non-representational, geometric paintings or sculptures is defined, which has shape grammars as its primary struc - tural component. Paintings are material representations of two-dimensional shapes generated by shape grammars, sculptures of three-dimensional shapes.” The word ‘geo- metric’ is the most crucial element for our purposes. It Fig. 18 Immanuel Koh, Discrete-Mies, 2018. Generated using is not surprising to see their idea of painting as a ‘non- a uniform probability distribution (i.e., every cell has the same probability of being generated), without sampling the spatial representational’ composition of geometric shapes as a relations among different cells result of Sol LeWitt’s influence during the thriving peri - ods of conceptual art scene of the 1960s. Consequentially, even the specification of their painting rules is geometri - Palladian villas shape grammar itself contains as many cally conceptualized as “a schema for painting the areas as 69 rules. Although efforts have been made to even contained in a shape.”. With this clear delineation of extend shape grammars to making grammars, in order shape boundaries, shape could then be similarly con- to include aspect of making and materiality, the entire ceived as Venn diagrams from set theory and their col- process of defining these making rules remains labori- ouring rules as follows, “The effect of the painting rules ously hard-coded. In taking the same position as Nor- in the example is to count set overlaps. Areas with three vig and Breiman, it is equally futile to assume that there overlaps are painted lightest, two overlaps second lightest, exists simple and explanatory set of rules that could be one overlap second darkest, and zero overlaps darkest.” syntactically formulated for every possible architecture. Counting can only be possible if each shape is an indi- Given the difficulties in creating/modifying/maintain - vidual figure positioned in a ‘limiting shape’. This limiting ing shape grammars, and its consequential non-data shape is defined as “the size and shape of the canvas on driven, non-generalizable and small-scale production, its which a shape is painted”, and is necessarily a ground, adoption within architectural practices continued to be minimal or non-existent, despite the ferventness among academics. More crucially, the conception of architecture (Stiny and Gips 1971). See the reprint version (Petrocelli 1971) p 125. Ibid., p. 132. (Stiny and Mitchell  1980). Ibid., p. 133. 72 76 (Knight and Stiny  2015). Ibid., p. 133. Koh Architectural Intelligence (2023) 2:7 Page 17 of 21 which in turn necessitate a shape as a figural object. The perceptual premise of shape grammar’s formalism is thus the seeing and counting of geometric shapes. The ques - tion then is: what if there is no shape? Or to put it in a more colloquial way: what if one is looking at a Monet and not a Sol LeWitt? Or in yet another way: what if it is a raster image made of pixels in Adobe Photoshop and not a vector graphic from Adobe Illustrator? Shape grammars are fundamentally defined by geometric trans - formations (i.e., translation, rotation, reflection and scal - ing) and boolean operations (i.e., union, intersection and subtraction), which are not possible with a Monet. To Fig. 19 Immanuel Koh, Discrete-Mies, 2018. Generated using a perceive, and thus to conceive, architecture from a shape probability distribution based solely on the frequency counts of each grammarist’s perspective is to say (using their same state- cell in the training set, but without sampling the spatial relations ment for paintings), “Architectures are material represen- among them tations of three-dimensional shapes generated by shape grammars”. In other words, architecture is the result of a Fig. 20 Immanuel Koh, Discrete-Mies, 2018. Generated using the learnt probability distribution based on local neighbourhood sampling but biased to only infer in the horizontal direction. The isometric view render shows the generated 3D configuration, the plan view render shows the inference arrows and tile indices, and the small diagram shows the contributing neighbouring tile (in black bold outline) used to generate the next tile (in red outline). All three preconditions are met here – a flattened set of tiles where each tile has the same size/resolution and its own neighbourhood transition matrix/probability Koh Architectural Intelligence (2023) 2:7 Page 18 of 21 Fig. 21 Immanuel Koh, Discrete-Mies, 2018. Generated using the learnt probability distribution based on local neighbourhood sampling but biased to only infer in the vertical direction. The isometric view render shows the generated 3D configuration, the plan view render shows the inference arrows and tile indices, and the small diagram shows the contributing neighbouring tile (in black bold outline) used to generate the next tile (in red outline). All three preconditions are met here – a flattened set of tiles where each tile has the same size/resolution and its own neighbourhood transition matrix/probability rule-based composition of solid volumetric objects trans- in full view but textured with the skin of an elephant, the formed and booleaned in space—architecture as object in machine would see an elephant and not a cat. This is a space. far cry from the human perception where it is the shape A recent article titled “Where We See Shapes, AI Sees that intuitively get prioritized. Yet, these CNNs are origi- Textures”, discusses a presentation at the International nally designed to imitate the hierarchical perceptual Conference on Learning Representations 2019 (ICLR) structure of the human visual cortex. In fact, the concept about the difference in perceptions between humans of CNN can be traced back to the 1950s work of David and today’s powerful deep learning models. Convolu- Hubel and Torsten Wiesel who discovered the neural tional neural networks (CNNs) being the state-of-the- basis of visual perception, that then became incorporated art deep learning models in vision tasks, are found to be in the design of CNN by Yann LeCun in the 1990s. In more biased towards texture than shape in recognising the experiments, it is found that visual signals are com- objects. In other words, given an image showing a cat municated in a hierarchical manner, with the first layer (Cepelewicz 2019). (Hubel and Wiesel  1959). 78 80 (Viola and Jones, 2001). (Lecun et al.,  1998). Koh Architectural Intelligence (2023) 2:7 Page 19 of 21 Fig. 22 Immanuel Koh, Discrete-Mies, 2018. Snapshot from an infinite set of generated Barcelona Pavilions, arranged based on their perceived formal fidelities to the original Barcelona Pavilion, itself placed in the middle of the array shown (or primary visual cortex) receiving signals from the eyes state-the-art machine learning approaches, the prospect directly. These signals are simply lower level visual repre - of sampling architecture is promising. sentations in the form of lines in varying orientations that have triggered the activations of the neurons. The recog -4 Sampling architecture nition of shapes does not occur on the first, but only on Leading theorist of digital culture Lev Manovich uses the those higher layers up in the hierarchy. In other words, term cultural sampling in his book “Cultural Analytics” shape entities are not the basic or primitive building to articulate the crucial difference when sampling cultures blocks of human perception. Building on this hierarchical using cultural datasets. Cultural analytics is therefore concept, the first real-time face detection algorithm by not simply data analytics. It refers to the computational Viola & Jones, makes use of a cascade of classifiers con - analysis of patterns in contemporary digital culture via sisting of simple binary rectangular pixel filters to iden - different digital media. In a similar vein, I came up with tify features of the human face, without having to define the term architectural sampling in 2019 as a way to inves- the shape of a face. In other words, shape entities are not tigate whether the sampling of architecture engenders a the basic or primitive building blocks of machine per- different set of perceptual concepts and computational ception as well. More critically, for the discussion here, processes, and if architectural sampling is not simply data all these examples show that human and machine visions sampling. Almost inevitably, it has led to a different way collectively reject shape as the primordial form and basis of seeing architectural forms through the lens of machine of ‘seeing’ and ‘learning’. If architecture can be perceived, learning. Thus, sampling architecture becomes a new way described and conceived beyond fully defined geomet - of seeing, learning and eventually generating architec- ric shapes, such as those well-defined shape primitives ture. Manovich’s ideal dataset is one that is complete (or found in shape grammars, but as abstract hierarchies at least, as complete as possible). A complete dataset, in of non-shapes/textures that are in resonance with the architectural terms, can only realistically exist in the form 81 82 (Viola and Jones 2001). (Manovich 2020). Koh Architectural Intelligence (2023) 2:7 Page 20 of 21 Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learn- of a single building, rather than all the buildings in the ing applied to document recognition. Proceedings of the IEEE, 86(11), world, or all the buildings designed by the same architect, 2278–2324. https:// doi. org/ 10. 1109/5. 726791 or all the buildings of a specific style, typology, region, Gristwood, S. (2018). Hiroshi Kawano (1925–2012): Japan’s Pioneer of Computer Arts. Leonardo, 52(1), 75–80. https:// doi. org/ 10. 1162/ leon_a_ 01605 material and so on and so forth. In this concluding sec- Noll, A. M. (1966). Human or Machine: A Subjective Comparison of Piet tion, I will use the complete digital model of Mies van der Mondrian’s “Composition with Lines” (1917) and a Computer-Generated Rohe’s 1929 Barcelona Pavilion as an example, to more Picture. Psychological Record, 16(1), 1–10. https:// doi. org/ 10. 1007/ BF033 vividly, illustrate the three preconditions of architectural Morton, T. (2018). Being ecological. Penguin Books Limited (Pelican Books). sampling in the project Discrete-Mies (2018), shown in the https:// www. pengu in. co. uk/ books/ 296515/ being- ecolo gical- by- morton- manner of an annotated pictorial series (Figs.  15, 16, 17, timot hy/ 97802 41274 231 Latour, B., Jensen, P., Venturini, T., Grauwin, S., & Boullier, D. (2012). “ The 18, 19, 20, 21 and 22). whole is always smaller than its parts” - a digital test of Gabriel Tardes’ monads: “ The whole is always smaller than its parts.” The British Journal Acknowledgements of Sociology, 63(4), 590–615. https:// doi. org/ 10. 1111/j. 1468- 4446. 2012. N/A 01428.x Carpo, M. (2019). Particlised: Computational Discretism, or The Rise of the Digital Authors’ contributions Discrete. Architectural Design, 89(2), 86–93. https:// doi. org/ 10. 1002/ ad. 2416 Written by Immanuel Koh. The author read and approved the final manuscript. Carpo, M. (2017). The second digital turn: Design beyond intelligence. The MIT Press. Authors’ information Alexander, C. (1964). Notes on the synthesis of form, 1st print. Harvard Univer- Immanuel Koh is an Assistant Professor in both the Architecture & Sustain- sity Press. able Design (ASD) and Design & Artificial Intelligence (DAI) programmes at Stiny, G. (2006). Shape: Talking about seeing and doing. The MIT Press. https:// the Singapore University of Technology and Design (SUTD) where he directs mitpr ess. mit. edu/ 97802 62195 317/ shape/ Artificial-Architecture — an AI & Architecture research laboratory. He was Russell, S., & Norvig, P. (2016). Artificial intelligence: A modern approach, appointed as the Hokkien Foundation Career Chair Professor in 2021 for his Global edition. Pearson Education. https:// books. google. com. sg/ cross-disciplinary research. Prior to obtaining his PhD from the School of books? id=_ BV6DA AAQBAJ Computer Sciences and Institute of Architecture at the École polytechnique Norvig, P. (2012). Colorless green ideas learn furiously: Chomsky and the fédérale de Lausanne (EPFL), he trained at the Architectural Association (AA) two cultures of statistical learning. Significance, 9(4), 30–33. https:// doi. and Zaha Hadid Architects in London. Immanuel is also the author of the book org/ 10. 1111/j. 1740- 9713. 2012. 00590.x ‘Artificial & Architectural Intelligence in Design’ published in 2020. Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal, 27(3), 379–423. https:// doi. org/ 10. 1002/j. Funding 1538- 7305. 1948. tb013 38.x N/A. Breiman, L. (2001). Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author). Statist. Sci., 16(3), 199–231. https:// doi. Availability of data and materials org/ 10. 1214/ ss/ 10092 13726 The datasets generated and/or analysed during the current study are available Stiny, G., & Mitchell, W. J. (1980). The Grammar of Paradise: On the Genera- from the corresponding author on reasonable request. tion of Mughul Gardens. Environment and Planning. B, Planning & Design, 7(2), 209–226. https:// doi. org/ 10. 1068/ b0702 09 Declarations Knight, T., & Stiny, G. (2015). Making grammars: From computing with shapes to computing with things. Design Studies, 41, 8–28. https:// doi. Ethics approval and consent to participate org/ 10. 1016/j. destud. 2015. 08. 006 N/A Hubel, D. H., & Wiesel, T. N. (1959). Receptive fields of single neurones in the cat’s striate cortex. Journal of Physiology, 148(3), 574–591. Consent for publication Manovich, L. (2020). Cultural analytics. Cambridge: The MIT Press. https:// N/A. mitpr ess. mit. edu/ 97802 62037 105/ cultu ral- analy tics/ J. M. Culkin (1967), ‘A Schoolman’s Guide to Marshall McLuhan’, The Saturday Competing interests Review,  51–53. The author is a member of the Editorial Board for Architectural Intelligence L. Young, (2019), ‘Machine landscapes: architectures of the post-anthropo- but was not involved in the journal’s review, or any decisions, related to this cene’, Architectural Design, 89, 1. submission. B. Sterling (2012), ‘An Essay on the New Aesthetic’, https:// www. wired. com/ 2012/ 04/ an- essay- on- the- new- aesth etic/. Accessed 11 July 2022. J. Bridle (2014), ‘On the Rainbow Plane | booktwo.org’, https:// bookt wo. org/ Received: 11 July 2022 Accepted: 15 February 2023 noteb ook/ rainb ow- plane/(accessed 9 Aug 2019). A. Hertzmann (2019), ‘Aesthetics of Neural Network Art’, arXiv: 1903. 05696 [cs],  Accessed: 19 Aug 2019. [Online]. Available: http:// arxiv. org/ abs/ 1903. 05696 A. Mordvintsev, C. Olah, and M. Tyka (2015). ‘Inceptionism: Going Deeper References into Neural Networks’, Research Blog,  http:// googl erese arch. Paglen, T. (2019). Invisible Images: Your Pictures Are Looking at You. Architec-blogs pot. com/ 2015/ 06/ incep tioni sm- going- deeper- into- neural. html tural Design, 89(1), 22–27. https:// doi. org/ 10. 1002/ ad. 2383 (accessed 11 Apr 2016). Terrence, J. (2018). Sejnowski, The deep learning revolution. The MIT Press. L. A. Gatys, A. S. Ecker, and M. Bethge (2015) ‘A Neural Algorithm of Artistic Schumacher, P. (2011). The autopoiesis of architecture: A new framework for Style’, arXiv: 1508. 06576 [cs, q-bio],  Accessed: 10 Apr 2016. architecture. Wiley. [Online]. Available: http:// arxiv. org/ abs/ 1508. 06576 Allen, S. (1997). From object to field + Architecture and urbanism. Architectural I. J. Goodfellow et al., ‘Generative Adversarial Networks’, arXiv: 1406. 2661 [cs, Design, 127, 24–31. stat], Jun. 2014, Accessed: Jan. 10, 2018. [Online]. Available: http:// arxiv. Allen, S. (1999). Points + lines: Diagrams and projects for the city. Princeton org/ abs/ 1406. 2661 Architectural Press. I. Koh (2019a). ‘Architectural Sampling: A Formal Basis for Machine-Learnable Architecture’,  doi: https:// doi. org/ 10. 5075/ epfl- thesis- 7815. Koh Architectural Intelligence (2023) 2:7 Page 21 of 21 I. Koh (2019b). ‘Discrete Sampling: There is No Object or Field … Just Statistical Digital Patterns’, Architectural Design (Special Issue: Discrete: Reappraising the Digital in Architecture), vol. 89, no. 2, pp. 102–109. S. Kwinter, ‘La Citta Nuova: Modernity and Continuity’, Zone, vol. 1, no. 2, 1986. A. Krizhevsky, I. Sutskever, and G. E. Hinton (2012), ‘ImageNet Classification with Deep Convolutional Neural Networks’, in Advances in Neural Informa- tion Processing Systems 25, F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, Eds. Curran Associates, Inc.,  pp. 1097–1105. Accessed: 18 Aug 2019. [Online]. Available: http:// papers. nips. cc/ paper/ 4824- image net- class ifica tion- with- deep- convo lutio nal- neural- netwo rks. pdf V. Sobchack (1990), ‘Towards a phenomenology of cinematic and electronic presence : The scene of the screen.’, Post script : essays in film and the humanities, 10(1), 50–59. R. Koolhaas, J. Westcott, and S. Petermann (2018), Elements of architecture. Köln: Taschen. W. J. Mitchell (1990), The logic of architecture: design, computation and cognition. Y. Zhou and O. Tuzel (2017), ‘VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection’, http:// arxiv. org/ abs/ 1711. 06396, Accessed: 28 Jan 2018. [Online]. Available: K. Kuma (2012), Anti-object: the dissolution and disintegration of architectures, Reprint., vol. 2. London: Architectural Association Publications. M. Carpo (2014), ‘Breaking the curve: Big data and design’, https:// www. artfo rum. com/ print/ 201402/ break ing- the- curve- big- data- and- design- 45013 (accessed 16 Aug 2019). I. Armeni et al. (2016), ‘3D Semantic Parsing of Large-Scale Indoor Spaces’, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, pp. 1534–1543. doi: https:// doi. org/ 10. 1109/ CVPR. 2016. 170. C. R. Qi (2017), pointnet: PointNet: Deep Learning on Point Sets for 3D Classifica- tion and Segmentation.  [Online]. Available: https:// github. com/ charl esq34/ point net G. Stiny and J. Gips (1971), ‘Shape Grammars and the Generative Specification of Painting and Sculpture’, in IFIP Congress (2),  pp. 1460–1465. R. Stouffs (2019), ‘Shape Rule Types and Spatial Search’, in Computer-Aided Architectural Design. ‘Hello, Culture’,  pp. 474–488. T. Knight (2018)  ‘Craft, Performance, and Grammars’, in Computational Studies on Cultural Variation and Heredity, J.-H. Lee, Ed. Singapore: Springer Singapore,  pp. 205–224. doi: https:// doi. org/ 10. 1007/ 978- 981- 10- 8189-7_ 16. R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, and W. Brendel (2018), ‘ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness’, arXiv: 1811. 12231 [cs, q-bio, stat],  Accessed: 13 Aug 2019. [Online]. Available: http:// arxiv. org/ abs/ 1811. 12231 N. Chomsky (1957), Syntactic structure. Mouton. Petrocelli, O. R., & Publishers, A. (1972). The best computer papers of 1971. Auerbach Publishers. https:// books. google. com. sg/ books? id= f6ezA AAAIA AJ J. Cepelewicz (2019), ‘Where We See Shapes, AI Sees Textures’, Quanta Maga- zine, Accessed: 13 Aug 2019. [Online]. Available: https:// www. quant amaga zine. org/ where- we- see- shapes- ai- sees- textu res- 20190 701/ P. Viola and M. Jones (2001), Rapid object detection using a boosted cascade of simple features. . Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in pub- lished maps and institutional affiliations.

Journal

Architectural Intelligence – Springer Journals

Published: Mar 21, 2023

Keywords: Sampling; Flattened; Resolutional; Probabilistic; Perception; Forms; Figure-ground; Part-whole; Shape grammars; Machine learning; Deep learning; Artificial intelligence

There are no references for this article.