Like to watch? Here’s a link to our very first video episode, complete with smiling faces, demos and presentations, and a lot of cats, dogs and turtles: https://www.youtube.com/watch?v=F60vG7cZ1nk

In this episode, we discuss the impact of generative AI on the spatial design processes, particularly in architecture and construction. We explore how AI could assist in generating design proposals, optimizing layouts, creating 3D models and visualizations, and automating a lot of mundane tasks. We talk through challenges and limitations, such as the need for human oversight, integration with expert systems, and adherence to building codes and regulations.

Mirek shows us his Augmented Reality (AR) project on parametric architecture, 3D printing in construction, instructional models for assembly, and the role of AI in different stages of the design process.

Our consensus is that while generative AI holds significant potential to enhance and streamline spatial design workflows, it is unlikely to fully replace human expertise and decision-making in the near future – but in almost every scenario, AI can elevate a user’s apparent skill/expertise up… a notch. How big that uplift is depends a lot of the user, the task at hand – and how long it’s been since April 2024 – when we recorded this episode!

Chapters

02:05 – Generative AI in Spatial Design

Violet presents various examples of how generative AI is already being used in spatial design processes. These include text-to-3D rendering, AI-generated assets for video games, and programming functionality in games using natural language prompts. She also discusses her background in generative design in architecture and how AI could potentially impact this field.

33:14 – Instructional Models and Expert Systems

William shares his perspective from his experience as a structures professor and his work on integrating AI into spatial design education. He discusses the potential of using large language models and expert systems to generate design proposals, interpret building codes, and automate certain tasks. The conversation explores the challenges of integrating AI into professional architectural practice and the need for rigorous testing and adherence to regulations.

51:05 – Parametric Architecture and Construction Techniques

Mirek shares his work on parametric architecture and a software tool he developed for generating 3D printable shapes. He discusses the potential of using such techniques for constructing structures and sculptures, as well as the challenges and limitations. The conversation then shifts to the feasibility of 3D printing in architecture and the trade-offs between traditional construction methods and emerging technologies.

58:41 – Future Directions and Challenges

We discuss the potential future directions for generative AI in spatial design, including the integration of expert systems for various domains (e.g., building codes, structural analysis, material selection), the role of AI in different stages of the design process, and the challenges of trust and risk management. Finally, we explore the potential impact of AI on labor and construction processes, as well as the need for human oversight and decision-making.

Transcript and Links

AB
Welcome to SPAITIAL. This is Episode 16 – coming to you in early April, just after an Easter break. Some of us did another thousand k’s on motorbikes, some others by the sound of it are getting ready for spring break over in the northern part of the world.

Welcome anyway, this is a longer episode, this one is dedicated to just one topic. We’ve got most of the team here: we’ve got Mirek, we have Violet, and we’ve got William, and there’s a running chance – but not a guaranteed chance – that you might not just be able to listen to us, but actually watch us.

We are experimenting with another recording medium, audio is good, but video may also be coming. So yes, most of us today are here today. Mirek, wow, you are on point. Well done. And if you need to see on points, you probably need find us on the YouTube or wherever that’s being posted.

For today’s episode though, we’re gonna be talking about generative spatial AI. How we can use spatial to make the design proposals, basically going into the building space. So this is home turf for Violet and William.

I just joked there before we started, this is your origin story. I’d like to hear not only how you got into this field, but whether this was the thing that propelled you into, I guess, thinking about spatial and AI together.

We will pass it over to both Violet and William to kick off the conversation, but look, this a secret dive, this longer topic, so let’s get started. This one should be a meaty one. Violet, over here.

Violet 02:05
All right. All right, well, I’m gonna share my screen because I want to give a more visual taste of what we’re talking about today and I’ll try and also use words in case we don’t have.

AB
For those people with ears only, we shall describe it, but those who can watch along, try it.

Link to the Google Sheets presentation: https://docs.google.com/presentation/d/1R3MCObauhqA5Mw_49mpmaabBiByRBALKFhYMcXSGsHs/edit?usp=sharing

Violet
That’s right. All ready. Are you seeing full screen? Yes. All, right! Okay, so today we’re going to talk about generative AI, and specifically how it’s going to change the spatial design process. So I wanted to just kind of look at the design process as it is changing now, like some of the things that I’ve seen.

I came from an architecture background for a while, as well as William, So we’ve seen a lot of stuff emerging there. Here in the upper left corner, you’re seeing text to 3D rendering. This used to be a really painful process where you have to render buildings, and it would take hours, sometimes a whole evening to render a design.

You have pick all the materials. It’s like it used be so painful, and it’s hilarious now that you can just say, I want a building to look like this. And it will take a model you already have and render it out.

So that’s interesting. Here in the middle, we have an AI -generated assets for video games, so another kind of text -to -something model. Here, you type some text, like, I want to unicorn that prances around and is this color.

And it generates the 3D of those game assets. And one thing I’ll show on the next slide is also we’re starting to see not just 3D models here. We’re actually seeing animated characters based on text prompt.

Violet 04:14
And then also artists have been using this for quite a while. This is, again, not some of the more current generative AI. But I think it’s interesting just visually how it’s changing our processes. So, Roblox is a company that’s starting to use generative AI so that people can actually program full games and functionality.

So here you see someone typing like turn the lights on every time this happens, and it actually changes the functionality in the game so it really interesting to have a programming actually changing spatial design.

And before all this came out, we had before all this generative AI stuff, I came more from the realm of generative design and architecture. All that means is using computers to generate lots of designs so you can find designs.

Delve, this is the product I was on on the right, but in the architecture space, you will use just a bunch of parameters for a computer model and then generate hundreds or thousands of designs and then you evaluate them for all kinds of like goals.

Oh this one’s more walkable, this has better daylight and it’s much better to do that than designing each design one off. So I think we’re gonna start to see generative AI also start impact things like generative design.

Violet 05:40
Last example I’ll show here just because I thing it is really interesting is that this I came across, also before the realm of generative AI, is someone designing a hospital based on the schedule and sequence of appointments.

So I think it’s interesting to think about spatial design changing. Doesn’t just mean like physical elements of the design. It could also mean how you choreograph something like appointments so that things don’t overlap a certain way.

It’s still a very spatial problem. How do you sequence those events?

AB 06:20
This is a floor plan with different iterations of walking distance from A to B. And I guess if this was 2024, you’d actually have active agents in a 3D space and queuing up and like a farm simulator kind of game.

But this is obviously a few years ago – is maybe this is an expert system? Is that perhaps what this kind of tool was? The beginnings of logic, but not necessarily machine learning?

Violet
Yeah, I don’t even know if this was just, this probably just like some brute force thing.

AB
Gotcha. Where someone’s three/four days of tinkering, thinking and coming up with the classic way of human task goal.

Violet
Yeah. I think so.

AB
A perfect example of if you could try every permutation, which one-on-ones would be perfect.

Violet
I can totally imagine this now with like NVIDIA’s Omniverse, like you’re saying, these like 3D simulations of sequence. And I’m sure it’s going to impact things like, yeah, fabrication and factories.

Okay, so I’ll stop there. But I basically wanted to kick us off with this premise kind of big, how will generative AI change the design process? And maybe I’d be curious just to hear from you all. Let me stop sharing.

AB
I love the concept of it’s just a grid or it a plethora of ideas. A lot of people think that AI is the holy grail. You do it once and it does the perfect job. That is a distant future. it is really more of that generative assistant that can guide a few options.

I’ve seen a lot of tools (in recent days actually) of 3D object makers by generative AI that give you a grid of models – say you’re after a treasure box for a game, you don’t hand-craft 3D model, the AI does a 50 by 50 grid of them for you. In the end there are some two thousand objects on a page and the human then comes along and says, oh, I like that one, that one and that one – iterate again based on those design specs. and that speed of ideas is what you could have done if you were doing it by hand within in ten minutes each

but to badly quote three amigos, it gives you many plethora of ideas and that smorgasbord is just enough for you to now swipe left to some of them – and swipe right to the precious few you want to keep.

AB

Violet
I’m curious, so we’ve seen, one thing that’s interesting to me is that we have seen a lot of examples of generative AI and kind of like creative brainstorming capacity and it tends to be like this very aesthetic thing.

But I think we can also see examples with being really beyond just aesthetic, it can be functional, and also beyond the brainstorming phase. Can it help in the later stages and execution phase? So I’m curious, from Ural’s background, do you think that this is something that’s going to have a more impact on the earlier later stages?

Do you thing it’s can have more impacts on that the loosey-goosey creative process, or is it going to have more impact on those kind of more fundamental things?

AB
Yeah, is a more of the low value smorgasbord, or is that up in the medium value help me refine one or two designs, or we’ve settled on this one last design, can we iterate that and finalize it and sign off?

I definitely would think it’s in the lower third, moving upward slowly. I dare say the human in in the loop is the top of that pyramid who does the final sign off, but yeah, by all means, it’s definitely able to assist in, you know, the multiple options on small things.

And if you can give it the right goals of minimize foot traffic, maximize something, if can find a goal to aim it towards, you could do the classic AI training with a reward and give a few hours or days of cycles and tell it to come back when it’s found the best answer, top answers, yeah.

Mirek
Probably depends on what you’re looking for, right? Because I keep following, I can show you parametric architecture, which is where I wanted to, oh, I’m going to get there. But it seems like the idea state of a project like that might be quite different than the actual execution, because you have what what you like and what you feel like might be what you’re looking for could be simply impossible to build or too expensive or just very impractical, right?

Like there’s this like maybe big step or gap between this sort of like application of AI that helps you dream up an idea and then the actual execution which is very technical and needs to somehow translate to the construction techniques we have, or we can do, and into the economy of what you’re building, probably, right?

Violet
Yeah, I’m also almost kind of curious to hear from William, because William was head of product at this company called Florida that very much like had to bake in all these really rigorous rules, it wasn’t like, It was almost the opposite of loosey -goosey in some ways.

Yeah, so kind of curious what you think, William?

William 12:18
Yeah. We had a kind of interesting niche in some way. We were actually more like a 3D visualization company. We have an engineering team who were really, really good at 3d graphics at the time and who are still really great at it.

This was 10 years ago, and so having WebGL in your browser was a new thing. We had a suite of products from an Oculus Rift installation for the retail space at and yards development in New York City to a very kind of rich editing environment for 3D experiences that were intended for a broker to send to their clients so that they could see a space outfitted with furniture and their brand colors and whatnot in a video game like experience.

So you could, in your browser, just kind of walk around like Doom, but it was like, Doom or some other first -person game. But it would be very high -quality rendering. It would have actual Herman Miller chairs, like things like that.

And we had 3D modeling artists who were tuning the models of those furniture for that particular rendering engine. The generative aspect is when we shipped a product for the same audience for office brokers in particular where they could take an iPad or a laptop and modify a floor plan.

If you wanted to change conference rooms to desk benching, you could just swipe your finger and tap bench and it would calculate the optimal layout given a certain set of preferences for for those benching desks and the the innovation there I think was the fact that if we could take that sort of out of the hands of the designers and put it into the hand of the brokers but the designer’s logic was part of the kind of parametric constraints that Merrick is referring to and we We could ship that product relatively quickly because the constraints around offices and the ideal design layouts were pretty straightforward.

William 14:54
We had a very skilled interior designer, interior architect, Lauren Popish, who wrote all of the rules for those particular spaces. She had tons of experience working at Gensler, which is one of the largest architecture firms in the world and one that you would go to for very, very large office deployments.

And so, it was a really curious thing to say, hey, like, instead of calling up your architect, you just have it as an app. So, it’s kind of an example of how we were displacing some of that architectural labor.

but in some ways, there’s a lot that goes on in architecture as a kind of spatial design field that is similar in pragmatically similar, I would argue, to the more mundane physical tasks that we’ve always seen being replaced by, through the industrial revolution and now in the robotics revolution.

And so there’s a lot more than renderings that goes on in building design. It’s often behind the scenes, particularly on large projects that folks really don’t really know unless they’ve really interacted with an architect.

And things like writing the architectural program for a large building, that could be hundreds or thousands of pages of written text on the requirements of the building. Interpreting building codes, particularly overlaid building code because at the national and local levels, there are different requirements.

And then writing the specifications, like the drawing to sort of get all the glory in the architectural design process. But there were thousands and thousands of pages written about that specify materials, the specify finishes, the way that things are supposed to be installed and so forth.

And that hasn’t even gotten really to the contractor yet, who often have to replicate those same instructions in more detail. In my mind, those are some tasks that are right up for grabs for generative AI as we know it, particularly textual large language models.

And so it’s really fascinating for me, from that perspective, to think about can sort of lower -level projects where maybe an owner of a smaller building is tempted to hire an architect, but we might have displaced a bit of that with some generative AI tools who can interpret the building code for them and generate some renderings and then go from sort of rendering to 2D drawings that are required for filing in the buildings department.

William
I think that there’s some projects there that definitely could be some prototypes for something like that.

AB 18:05
So it’s starting with low-level but then by the standard but you were getting to a quite a mature level of sophistication for final output of that choose your own layout tool. What sort of level final human touch was required to massage it from output from the system to final document form, final drawing form?

Was it a half way house? Was 80%? I mean the numbers are arbitrary, but was it most of the way or was that just a good a good slab of away then the humans got involved and I dare say the billing got involved to actually do the final bit.

William
Well at Flord, that product that we called Protofit at the time was just for test fits and just for offices. And the way that it was built, or the way the process worked, was we would consume drawings from the owner of the building or from the broker, whoever had them.

And then we have to convert them manually into the format that worked for that particular product, for Protafit behind the scenes. And once it was built into that kind of specialized semantic form, then we could render it as a floor plan and make it interactive, and we can also render in 3D so that you could see the 3d changes.

And so not only did we have to copy the floorplan into this sort of very specialized, semantic model, we also designed four layouts for them. And there was a lot of human work paired with all of the engineering work and the product work, there was an internal platform and then a customer facing product.

So by the time it got to the broker, it was already laid out for them and they could make changes. And the reason was, we discovered for the user experience, the reasons for that was we discover that brokers, brokers didn’t have time to design.

William 20:05
Like they weren’t really designers per se, that sort of generative aspect of creating and observing a hypothetical world that designers are very, very good at. Like that wasn’t their business. Like, that’s why they hired architects to do that for them.

But they were exceptionally good at identifying what didn’t work in an existing floor plan. They would say, this conference room is in the wrong place. This one’s too small. You want to put it next to the entrance so that you can take your high value customers and go directly from the reception desk into the main conference room, like they were really, really sharp at that.

They didn’t know what they wanted. They wanted to give them that experience.

They knew what the didn’t want. Yeah, that used to be called a client type A in my world.

AB

William
Yeah for sure. And it was a really great, it really was great to watch that happen once we got the product in their hands.

Violet 21:00
I think that what’s interesting about this is what William is bringing up and what you’re bringing it up Andrew, is where the human comes in to smooth everything out. And so if humans are gonna be part of like smoothing out what the AI can’t do yet, what are the first things that AI should pick up in the design process?

Is it gonna to be things like, you know, maybe it’s helpful to more specific here and in terms of what type of design process so you know for William and I were probably thinking a lot more about the design of buildings um but maybe you can all kind of illustrate for us Merrick and Andrew I’d love to hear you illustrate the design problems you’re working on in what domain and and maybe I’ll just start by saying walk us through the current design process you have and where you see generative AI coming in first.

Violet 22:07
Like, what is it gonna do before humans are able to do that? And if you don’t mind, Merrick, can I ask you?

Mirek 22:17
Can I show you what I’ve been working on first? It’s quite different than the model. It is very rough on the surface and I’d never published it, but I think it’s quiet entertaining. Let me turn on. We like entertaining,

So this is parametric architecture on Instagram. I just wanted to set the stage with what we consider innovative and beautiful these days. So it’s these organic shapes, crazy geometries. This might be generated by AI easily, right?

Either as a 2D image or a 3D model. But the the other thing is how do you actually make a shape like that? so I was thinking that you might actually you know work with geometry and come up with the construction technique that is fully under your control and then I Spent some time doing that and I ended up With something that it’s actually a piece of software that runs in the cloud and you feed it geometry you can modify the geometry and produces these sort of 3D printable shapes.

(Mirek’s full video is private, but you can get a preview of parts of it by watching this Episode on YouTube)

This is the interface that I came up with. So you feed it some data, you can notify the Geometry as you go and it calculates what sort of material you need and generates the unique pieces on the fly.

So all those joints that you saw there are unique and labeled and come out of your printer with like little label on it so you know where it goes and there’s AR element to it. So you can actually, you know, see the thing in the one -to -one scale and you can modify everything live.

So I’m doing this and on the background in the cloud or something doing the calculation. And then I get all sorts of pieces and I built this, this little mockup. It’s actually a lot of work. It took surprisingly a lot of effort.

And I was wondering if it scales up, if you can use this kind of thing to build something bigger. I spent quite a lot, I’m just going to fast forward in this video.

AB 24:24
Let me describe that for 10 seconds for our people listening along. This is like those climbing gyms you’d play on as a kid, the half domes, but what you’ve got here are, you got 3D printed spokes, which you then output from this and then using dowels or broomsticks as the medium of rods to join things together.

You can set your own metric imperial measurement to all I take it. So this is what you were then able to do in augmented reality design, pull up nodes, join it how you need to. Is it also calculating things like stresses, strains, and required number of supports to make it happen?

Mirek 25:03
It’s not doing that, it’s optimizing the 3D printed parts for strength and it makes sure that they’re oriented in the most efficient way. It is labeling them and then it gives you a list of materials.

So initially you set it up by saying I’m working with wooden dowels, this thick, this long, and it will tell you that, oh, here, as you’re modifying the geometry, you’ve actually hit a place where you don’t have as much stock material and you would run into trouble.

And this happens both in the web browser, right? This is a web interface for that. And it happens in AR interface. I just finished building this little bit. You see that it’s also kind of taking on the role of the guide and it’s telling you, okay, I’ve got this part, where do I put it?

So there’s this model of what you’re building and you can mark there your progress. So you know that this is built, I’m not gonna have to worry about that, then you just pick the next piece, look it up and I will tell you where it goes.

So it like, it over -engineered for what it is and given how much time I spend on it, I would like to find out some use case for it. But the other thing I built besides this is this little tensor grid, this thing wasn’t the initial vendors also, so this like floating structure.

Mirek 26:27
It’s a plant stand, it can only hold plants, but it’s something that actually came from the very same process. I don’t know if this isn’t useful. I think it is perfectly possible to build something this way and maybe if you’re going for structure.

You don’t want to deal with wooden, fragile plastic. But the same principle could be applied to other things. I just don´t know if this is for sculptures or if this for buildings. This is just a mock -up I was trying to show that maybe there could be Stanford bunnies at Stanford.

Life size, a modern life size. Or they could be more overseeing the San Francisco Bay. But who needs this, right? This is art and if you’re an artist Maybe you want to get your hands dirty and do it other way.

This will pretty much I will like Like what I what? I’ve built in this in. In this case is this It’s not AI it’s algorithmic, but it does all the number crunching but pretty started with this construction technique, you know, I can perfectly describe in a computer and and then as I would get further, I will definitely do some structural analysis on it and some load spreading voodoo to make sure that actually, yeah, it’s gonna do what you want.

Violet 27:48
Mirek – I’m gonna die if I don’t jump in here. Because this is such a great topic for us to go through because William was a structures professor and has a project that’s specifically using a model like this to change the structure.

I don’t know if he’s going to be able to bring it up. And when I was in school, I did this project, that was much more manual and painful, putting together a structure kind of like this, but instead of 3D printing the joints we use silicone so that they could stretch to the I have to share,,,

Mirek 28:37
I just have to show you one more thing. So it’s related to this. I’m gonna jump back to my screen. So this is a company startup from I know the founder and he’s taking a slightly different approach.

https://www.quadrobee.com/how-it-works

So you see that you’re in Oculus, you put on the headset and then you can design structures from these aluminum extrusions that are pretty standardized, you get parts that click on these and it’s pretty much well understood what you can do with this kind of material.

This project takes a very different approach, which in principle is the same. But the constraint is, the building technique, which, in this case, is right angles, everything clicks in 90 degree, too much more simple.

Your material is defined by what you can get. And then as you model something this way, they can sell you the parts as a kit and as the business model. Gotcha. So that makes sense. In my case I wanted to start with whatever I want and then just have something tell me how to work the material to actually make something practical.

I think this approach is much smarter. I don’t know how it works as a business because I have some concerns about that. If you work with this kind of material, you probably have it at hand. It’s not as hard for you to prototype this or just wing it, you know, as you go.

So it’s on the other side. On the side of the spectrum, it is too easy maybe to add more value with some sort of smart tech, I dunno. That’s just my opinion.

Violet 30:17
Yeah, that William and I have talked a lot about, is there ways for us to take something like McMaster Car, if you’re not familiar with this. It’s like this website that just has, it’s a website and a bunch of products for hardware from like springs to screws to bolts to actuators.

it just has like everything under the Sun in terms of hardware and it’s so painful if you’ve ever prototyped something to figure out what bolt goes with what screw connects into this thing and I wonder you know to what degree that example you just are we gonna see more sophisticated…

Mirek
That whole McMaster catalog is integrated into Autodesk’s Fusion 360. And you can just pull parts from there and of course it makes sense as a business because that’s where you get the parts afterwards, right?

But it doesn’t tell you how to put them together, what the logic is there and it does not help you decide. I mean, things maybe click together as you would expect, but that what our work is joins and there’s some basic mechanic to it.

Violet 31:39
Okay, so I’m going to put on hold the structure -like project for a second, and I want to come back to it because I am really interested in this McMaster car premise. And I wonder, you know, William, in your class, you’ve been talking a lot about like getting your students to design with higher levels of intention.

So maybe you can, would you be willing to just describe that process, maybe even show an example, and if you cant bring it up, no worries, about this exact thing, Like, how do you get students to rather than talk about a design in terms of this McMaster car bolt, talk the higher intention of the design, and can you have a system go fetch you parts that would do said thing?

William 32:48
Sure. The premise of the course starts from defining space and then continues to interrogate what is AI. I think it’s interesting for us all to think about, and this is partly why this podcast exists, is when you combine those two things, what do you get?

that. And so the course itself is starting with some of the latest technology and latest developments in AI that are relatively popular and that are new, but that also accessible to spatial designers.

And the variety of students I have go from classical architecture students who are intending to become architectural designers in firms or maybe even professional architects themselves, all the way to urban planners and policymakers and so forth.

So the, and finding a place where someone, where I have 26 students in the course, there are three who are professional software engineers or were in past and three who have never seen a line of code in their lives.

And then how do I make artificial intelligence accessible? to them. And so the large language models are really great at that. And the way that I frame the course, at least in this first instance, is saying, well, let’s talk about forms of spatial reasoning that we can engage through text.

And if I can describe a design at a very high level. So, I’m interested in, say, the social aspects of a particular public space, or I am interested in certain types of experience. Can I find a way to go from those ideas and break down my own semantic model all the way down to the individual operations I need to perform geometrically to propose a design?

William 35:08
And so that’s what we’ve been doing. We’ve be simulating 3D environments with just a simple game engine, really, and then poking at the large language models to say, okay, can you, like yeah, the first exercise was kind of fun, at least it was fun for me, like there’s a ton of balls being dropped from the top of this virtual space, and the challenge is to contain as many of them as possible.

And so you can ask it to position boxes and spheres in this Virtual Space, but can you get the model to not only respond to individual instructions, like put a box here, but instead put wall here. Oh, and walls are modeled as boxes that are long and thin and tall.

What then can you say wrap this or contain this space and have it understand that in order to contain a space it needs to put up walls and walls are then boxes and then it has the ability to add boxes to this virtual environment.

They do surprisingly well actually. The GPT models are really good at this when you put enough effort into the prompts and to the system context. So that’s the premise and the hope and the dream is that we’ll be able to describe whole design propositions.

So could you take mid -century modernism and this sort of white box international style and those notion of like ribbon and punch windows and the different types of experience that Le Corbusier and Mies van der Rohe were investigating at that time.

William 36:59
And can you actually create a functional design language in natural language? And the pioneer William Mitchell involved at the MIT Media Lab, his work, The Logic of Architecture, is very much in line with this.

But it took a formal language approach where his worked looked at Palladian villas and made an argument for how you could break sort of spatial propositions down into this formal language that looks almost like symbolic logic to a certain degree and has this treatise around how programming languages work.

It’s inspired a lot of my work throughout the past few years. Now with these language models, particularly with the function calling capabilities of LLMs, It’s just, I think we’re only just starting to see the capabilities here.

So that’s the primary parts of the course. And then finally, we are starting integrate some of more traditional AI, like spatial AI technologies that we’ve been talking about as well. So things like either the vision models or facial recognition or trying to understand depth estimation and how do you interrogate the metrics of a particular space.

And so now the final project is to set up a set of kind of imaginary sensors or real sensors, like webcams, and then scrub them through multiple layers of models, and then a reason about those inputs with an LLM.

I think this is the way to make these highly, highly technical AI methods accessible to these particular spatial designers.

AB
I’m seeing a thread across all of us here with our examples of equally making spatial tool generation that the general trend I’m sort of saying is that we’re not trying to get to 100% and perfect but I think almost every level of someone’s prowess in a field hand wave and a big statement here but can be elevated slightly by these kinds of tools.

There are there are ways to bring you from being a novice to a junior from a Junior to mid -weight from you know a creator to a hobbyist, those sort of things. I dare say the top of that pyramid to come all the way back to the first sort topics was that probably you can’t elevate yourself from a, I’m looking at two here, adjunct professor to full -blown professor.

There’s probably some gaps there that are that last gap is, you know, the nine nines and may not be hit today, I hope. but in the mid -weight levels, there are tools and ways to speed up some thought processes, take advantage of the previous learnings from previous expert systems who have got more experience and hasten your arrival at some more elevated levels.

This probably means there’s fraught with danger. I’m probably got a project manager in my head right now saying risk, risk. Oh my God, you know, to Merrick’s equally structures, they are phenomenal and same with the aluminium orthogonal structures.

AB 40:27
In my engineering realms – for the record I did do engineering back in my undergrad because it was the closest thing I could find to robotics. I was ahead of my time, hello, welcome to the world. They got me trying to do robots for car manufacturing plants which in Australia now don’t exist.

One of the biggest life lessons from my engineering working experience was I was given a small task at a summer job and told to design a platform for something to hold a certain weight and of course I went back to math and did the structure analysis and the shear force and nothing and I decided to need a M14 bolt and i reported that back as my supervisors who said sounds good if you’re not sure just make it a bigger bolt and that was the engineering heuristics, the rule of thumb for everything.

All that condensed logic would be, yep, I reckon that needs a 20 mil volt rather than a 15 mil bolt, you know. So math was good, but those rules of thumb were fantastic. I’m just gonna ask around yourselves now, do you think that level of expert system logic that has been baked into these large language models by drawing the world’s internet for text, we’re now seeing that emerge slowly as general world models what we’re starting to talk about spatially aware models with vision depth 3D.

Do you think that those logics of your Flord expertise of floor plan layout, are those going to trickle down fast enough and be good enough to avert risk or is this simply a guide with an asterisk and a caveat saying use at own risk consult professional don’t use for you know don t take it onto the road at what level do you think we’re gonna be seeing that transition from guide hobbyist creator tools to no, you can take these to the bank.

You can use this forever for real life. Love to get your point of view of how much knowledge trickling down to what becomes solid.

Violet 42:37
I really think that what we are gonna see is we gonna have very rigorously tested rules. So take for example, the igloo that we just saw or like those, I love the premise that we can create like all these bizarre shapes now.

Well, we could have a structural model that is not Gen AI, that tests whatever layout you’re creating, but the layout itself can be from higher level in time. You could say like, oh, could I make a bookcase of this?

Oh, can I turn this into a Pergola? You know, like I think that’s where Gen A .I. can shine. And I think that there’s, I don’t know when the trust is going to come from a generative model, especially for something that’s structural.

Mirek 43:24
I think the Trust and even real world utility value will come form expert systems. So you can imagine a language model or something like that that is expert in your local building code. And then, depending on what you’re building with, you can consult your design with an expert mason AI or expert woodworker AI and through design edit and ask it, is this something we can build?

And you get an answer from this field of expertise and you run it through the building code AI to give you approval if you know whatever we came up here or you might have a structural assistant who understands the physics of what you’re building and we’ll take a peek and give you really quick about educated guess whether this would work.

And as you’re moving on with this design, with these process, maybe you can spend more time, compute time on each task and then come up with the whole thing bit by bit. So I think these expert systems will start popping up in every field.

And so it’s a question of making them talk together, I think efficiently exchange data in a meaningful way and then have them do their bits because I think like our human knowledge is structured around you know topics and and jobs so maybe that’s how we’re gonna come up with with artificial experts for that and maybe you know bringing them together like this would work much better than than my approach which is like okay I need to design everything because otherwise I don’t know how to how to control the design.

So if you have this crazy designer architect AI that’s just hallucinating whatever might be fancy and whatever pleases the consumer, the human, then you can run it through some more rational AI’s to critique it and then try to figure out how do we actually build this thing.

AB 45:33
I’m just going to go and register that domain now before everyone else does. Sensible .ai. No, you’re quite right. If we’ve gone from language model AI, image cat -dog detectors, and now we’re moving into general world models spatial reasoning, it makes perfect sense.

We’re going to go deeper into niches of domain expertise. But I think you are even more correct, Merrick, totally that the interaction and the interfacing between those bespoke sensible AIs is going be a world of pain for someone to solve – and that someone will probably be very popular if they can solve that problem.

William, your sort of point of view of that expert system floating down, users using it in your class and others, getting more advanced by using that knowledge, where do you think that might meet in the middle? Where’s that sort transition point from? I’ve used all the tools I can today, now I do need to consult. Is that rising or is that still gonna stay in middle state ground of the humans have to leave here?

William 46:39
Yeah, I tend to agree. I think there’s the tolerances for whether an architect will be allowed to use these tools from a kind of professional perspective. That’s going to take some time. Even with as much computation as we use in the building industry, at least in the West today, it’s still, in the U .S.

it still comes down to an architect putting their stamp on those drawings. So that is accepting a certain level of liability for ten years, at least after the the project is done. And I should make a caveat as well though that a lot of the comments that that I’m making at the least are from the perspective of Western architecture, from the Western Architecture market from the US and most of Europe and probably Australia as well.

But East Asia is far ahead of where we are in terms of adopting automated and AI systems for the building industry. It’s sort of known that as Shanghai was being built, was Excuse me, the architectural tools there were quite a bit far beyond what we were using in the U .S.

That there are code checking systems, generative systems that were expert systems precisely, to your point. They were all being used for those massive skyscrapers that are so beautiful now in sort of nighttime photos of Shanghai.

AI, and even just recently, August last year, there was an article published in The Guardian about a company in China called Xcool that is using generative AI for the building industry, and it is things like master planning that they’re using it for, all sorts of capabilities that are beyond just the kind of visuals that we see being experimented with in schools and with mid -journey.

So daylighting requirements, base standards, local planning regulations, generating interiors and construction details. They’re very ambitious. I don’t think their tool has been released yet in the U .S., at least it wasn’t at the time.

William 49:14
But we can put this link to this article in the show notes. But this is really, so this notion of generative AI tools combined with these advanced expert systems where like it’s far more reliable to just say the logic since the logic is there.

And in literally in code, maybe it is written in natural language, but it is a building code. I definitely see where that is going to, where the rubber hits the road. Like, it’s that risk management profile that really governs the building industry to a large degree.

And so making sure that the contractor, the architect, all the subcontractors, and even the sort of squishier requirements of public relations around major projects in particular are all handled. Like that’s, like the expert systems I think are going to be really valuable there.

AB 50:14
Trust is definitely the bar. I think that it’s not the glass ceiling. It just definitely is the point where, you know, we can say, let’s actually go ahead, or it a tool, it not a toy, but it is a guide.

But that bar is raising, almost certainly. I can’t say every year, it raising faster than that. I mean, every couple of months, we get sort of advances that push it forward. Violet, can pass back to you to give us a final summary and conclusion.

Mirek 50:53
Before you wrap up, I’m actually thinking about one thing that’s related to this, and I wondering where you guys are on it. What do you think about 3D printing in architecture? Large scale, concrete or not?

Is that the answer? Because I have opinions, but I think I’d rather let you speak.

Violet
I think it’s going to depend on labor costs, like that’s two by four construction is so inexpensive. Maybe it also depends on things like unions. think the most difficult thing is probably things like changing building codes to abide by a completely new construction process and like William is saying around like all these regulations, we’re so set in a world that it is like, I almost feel like 2×4 stick construction is like what the QWERTY keyboard is to the computer.

We’re like stuck with it because it was baked in early on, so I kind of feel we’re going to be stuck to 2 by 4 construction because all the codes and processes.

IKEA hasn’t got a section for curved furniture, sadly.

AB

Mirek
Like the keyboard, it actually works and it’s really efficient, right?

Violet
Yeah. it’s like it it got its problems like we could probably do better but like you know no one wants to like overhaul whole system or it is really expensive to change all these processes that have already been set up for it so I I’m hopeful that there’s some niche where it can exist but I don’t know where that niche is I think it”s like a salmon trying to swim well that’s a bad analogy because Sammons do swim upstream.

It’s like something trying to swim upstream that cannot.

AB 52:39
Merik, you think it’s a price problem or a concept problem?

Mirek
Actually, I have something I would like to touch on because I actually spent considerable time looking into stick framing and how houses are built in North America. Just looking for something you can optimize with a robot or calculate better.

But the thing is that over the centuries, this process is so well optimized, there’s very little you can make better. And if you just throw two guys on something, you’ll have, structure of a house in two weeks or month.

And as opposed to needing really specialized gear and bunch of engineers coming on site and setting up a gigantic printer to print something that’s kind of limiting in what it can be. Maybe even like, we can argue if it’s practical, if if it is aesthetically pleasing, that is one element to it, the other is practicality and how remodeling a 3D printed house or just like deciding that you want power socket somewhere, you know, two months after it has been printed, whether it is feasible to modify a structure like that and where it actually leaves you.

So there’s all sorts of approaches and like views of this. I don’t think the whole, some years ago these companies like ICON and others started by claiming that this is super sustainable, super green, and then they turn out printing with concrete.

So how green is that? And I dunno, I just don´t feel like it justifies itself on this planet. But on this planet is kind of important because as we want to build, you know on the moon and Mars That works like a charm and you just use local materials and crush them into whatever Make little Mars concrete and and here you go and it’s it the opposite.

It’s the simplest possible Approach there, but here I don’t see the cost of it I do not see how it is going to combat homelessness or do anything, like you often hear these companies claim It’s not very ecological and in terms of cost of labour, it actually requires a lot of specialised personnel to build something that maybe you don’t even need in the first place and it’s going to be cheaper and maybe it won’t feel like a house, a spaceship and people want to live in a houses.

So in summary: good idea, wrong planet?

AB

Violet 55:22
I also think that there’s potentially something to optimizing the instructions. So you know, William’s talked a lot with this company, Mosaic, that’s really thinking around how do we automatically create the instruction for assembling the building.

And I know, you know this is an area that you’re looking on a lot, thinking a little bit about, how do you actually give instructions to a robot or a person to put something together? So I think that maybe is the area where a lots of optimization can happen, like coordinating different subcontractors that come in.

And if you have a model that comes from something that’s like NVIDIA Omniverse and you can have something on the site that checks, has that been built yet? Should this happen next? and then can generate sets of instructions for what goes in next.

And use something like Mirek developed where you actually can AR or whatever, see where this two by four should go. Perhaps that helps.

Mirek 56:35
I think you don’t need to go maybe as slow as your two -by -four. What I actually learned is that there is lot of improvement of efficiency but it usually comes down to pre -building parts of your house in the warehouse somewhere with big robots, oh I’m just trying, with robotic arms with tools that can reach wherever you need them they don’t need to move you know it’s not as dynamic environment as the job site would be and then you take these parts put it on the truck move it to where you need and that seems to be the most efficient way and you definitely have some there’s this expert systems that these companies use exactly like like William Mentioned that and you have some creative freedom.

You can modify the house if you’re kind of You know, we’re talking right angles here But you can have the door a little bit to the right a bit too the left There’s there’s some size some some kind customizability baked into these products and I think it’s It’s the most innovative that I’ve seen.

And besides that, it’s just a bunch of guys doing the woodwork, and they know what they’re doing, so it gonna be fast, it not gonna cost fortune. You’re building with wood, so you’re storing carbon. It good for the planet if you source your wood from well -managed forest.

And all these things kind of come together.

Violet 58:05
Well, I think with that, maybe the most important thing that we took away today is that generative AI is going to change design, perhaps most on Mars, but it’s definitely going to changed the processes here, but there’s a lot to overhaul with the way that our existing world is already set up for a lot of things and maybe aren’t ready to change or like partially set in stone.

AB
No, I hear you. There’s just the same with the building story. If you can hire two more people at x dollars per hour, that solves so many problems. A consultant mind trick is to set the future at the right level of future so that it’s obviously just can’t be solved by adding one more person today.

So oftentimes in my field I don’t want to say let’s think about 20 years in the Future when we’ve all got matrix style neural jacks that’s silly you don’ get any traction but you also don”t get traction if you paint a picture of the feature tomorrow with 5% better or 10% better because you can just solve that by editing one more thing.

But I think we’re all here because as we can see that the future in a middle future in one or two years, although one of two years will almost always never come in this field. It’s coming at a great rate of knots, but these tools are here without fail.

We’ll put all the links in the show notes. The question is whether the tools can give you 5% boost in efficiency, creativity, or 95% boost answers somewhere in the middle and your mileage will vary. But there’s ways to get a lift, to to get ahead to be inspired to if maybe have the, was it sensible .ai, check you one day.

And the things that we do whimsically can be fact checked and reference checked and building code checked and everything checked. I think we’re definitely painting a picture of humans can have a bit more fun in the future.

And then we’ll let the AI do the boring stuff. That’s plan A anyway. I do note John Stewart just did his lovely Daily Show monologue on AI taking jobs, all good. That’s topic a different day, but right now it’s about creativity playing and yeah, jumping up a half a level in your maturity of, what can I grab from the expert systems and learnings from tools higher up?

And how can use that to advance my own, you know, fast track my pace of playing around for 2D, the space optimization, 3D for building, phenomenal.
Mirek, when’s the igloo – the Phantom Cybernetics – igloo creation software going to be released?

Mirek 01:01:04
I really don’t know what to do with it. It’s too much work. I was almost about to announce a product that you can do to, to to that, you could use to exactly what I did. But there’s too much to do to make it like any more useful.

So if anybody has any crazy ideas for what this could be used on all ears. Hold it right in.

Please – hit us up on LinkedIn and say, Mirek: we want more igloos or want more tensegrity plant holders or I want the Stanford bunny in Stanford – tomorrow. I can make many of those. Excellent. The tool sure is phenomenal but no I hear you that is it a product or not that’s all right we’ll look forward to the release of that.

AB 01:01:53
Well thanks team we might leave it there that a long episode we will cut it up for audio right away for your listening pleasure and then if you’re watching this, then it’s good to see our faces – and yes our robot backgrounds need to be more on point we need to get more robots in.

You’ve got living animals. I’ve got living animals. William’s got the many plethora of everything and then Mirek is winning on the robot front. So yeah, we have to up our robot game shortly.

Mirek
Cold medal.

AB
Cold hard medal, nice one. Well, thanks everyone. That’s it for this episode of SPAITIAL. We’ll catch you on the next one for now – bye-bye!

HOSTS

AB – Andrew Ballard
Spatial AI Specialist at Leidos.
Robotics & AI defence research.
Creator of SPAITIAL

Helena Merschdorf
Geospatial Marketing/Branding at Tales Consulting.
Undertaking her PhD in Geoinformatics & GIScience.
 

Mirek Burkon
CEO at Phantom Cybernetics.
Creator of Augmented Robotality AR-OS.
 

Violet Whitney
Adj. Prof. at U.Mich
Spatial AI insights on Medium.
Co-founder of Spatial Pixel.

William Martin
Director of AI at Consensys
Adj. Prof. at Columbia.
Co-founder of Spatial Pixel.

_{To absent friends.}