HOSTS
AB – Andrew Ballard
Spatial AI Specialist at Leidos.
Robotics & AI defence research.
Creator of SPAITIAL
Helena Merschdorf
Geospatial Marketing/Branding at Tales Consulting.
Undertaking her PhD in Geoinformatics & GIScience.
Mirek Burkon
CEO at Phantom Cybernetics.
Creator of Augmented Robotality AR-OS.
Violet Whitney
Adj. Prof. at U.Mich
Spatial AI insights on Medium.
Co-founder of Spatial Pixel.
William Martin
Director of AI at Consensys
Adj. Prof. at Columbia.
Co-founder of Spatial Pixel.
Welcome to SPAITIAL. This episode is a deep dive all on its own. This week we’re covering the rather gnarly but important topic of how do you get started with Spatial AI?
The field is changing so rapidly. If you’re already on the leading edge, then congratulations, The burning question of today is how do you get there? How do get close? Close to – I won’t say leading-edge – close to watching the edge, but then a deeper dive. How does someone fast follow? How do you catch up? Open question! Who’d like to go first? I can see a lot of people with a lot experience here in this chat.
AB 01:10
Violet: could you give us a bit of a brain dump of your experience of watching this field and then go around again for catching up? Go for it.
Violet 01:19
Sounds great. I kind of stumbled into this field because I started in architecture which has a lot of 3D stuff. It has a lotta modeling and simulation. But then ended up getting really interested in the way we interact with physical spaces.
So I wanted to get out of the digital simulation world. I had been at Google on this product for generative design. And so that was all simulation-based. But I was really interested in sensors and the tangible world and physical computing.
So I chased after that really, Creating a new class for that so that I could start to learn more about it. And so I guess my biggest advice for like making the jump into something like Spatial AI is I think you need to find what you’re really interested in and carve out that niche for yourself by just starting to learn more about it, work on the tools.
Like for me, when I started getting into this, I don’t think I had much upper hand other than the fact that I had been working in the simulation space, but I didn’t really understand sensors or how to merge the physical world really other than any experience I have from fabricating things in architecture.
So, I think the best way to ramp up is just to start playing with the tools. So for us, it’s spending a lot of time having ideas about specific projects. “Oh, could we start to use something like OpenAI to have more natural interactions?”
“Oh let me play with this gesture library and see if I can combine that with speech.” And so I think you have to allow yourself a little bit of freedom to be a novice. There are so many people being novices trying to learn how these things behave.
And we have really just poke and prod at the tools and what talks to what and what does and what emergent properties does it have. So I think you just kind of have to chase what you want to do and get muddy.
So I think you just kind of have to chase what you want to do and get muddy.
Violet
AB 03:45
Get muddy indeed. Helena you’re definitely possibly in that muddy field you’ve been playing in the geospatial world. Not only have you been able to go ahead and do some deep research but you also then take your clients through that kind of journey.
How do you sort of get them pushed up that hill without – you know – dragging them up the hill. How do you do it politely?
Helena 04:08
So it’s a bit interesting, because I’m actually quite the opposite approach of what Violet’s just outlined with playing with the tools and kind of taking that technology-first approach as a kind of a front to what my personal philosophy is always a problem-first approach.
So, for me, it’s less about the technology, it is less the tools, It’s more about what problem are we trying to solve.
Helena
So, for me, it’s less about the technology, it is less the tools, It’s more about what problem are we trying to solve. And that’s always what’s at the heart of the conversation I have with clients when they come to me and they say, like, “Hey, I’ve got this really cool idea.”
And I got to say 9 out of 10 client conversations that I had these days are people or startups – like individuals, smaller companies, or larger companies who are kind of pivoting or adding a sector who are going into AI.
I’ve only really had one conversation lately where it’s been about branding that hasn’t been somehow related to AI, so people are seeing the opportunities everywhere. People from various sectors, like AI in education, AI and street lighting, AI this, that, and the other thing are just from this week alone conversations I have had.
And, you know, it’s always really nice and easy to get hung up in all the cool stuff that you can do. But the way I tend to handle it is to pull it back and say like, okay, I know there’s so much cool you stuff you could do, and there is so many potential in there about what problem are we addressing, what problems are solving, because otherwise we fall into this, in the geospatial world.
It’s been described as like the geospecial product trap where you create an amazing product but no one really needs it or wants it. And I suppose some innovations come about that way. And it definitely has its playground. But when you want to make money with it, when want build a career off it when you wanna build the brand off of it then you need to be super clear a problem you’re actually addressing. So that’s where I start.
So we really drill down into what’s the need and how can you address the needs and how could AI come in to help you address that need more efficiently. And most of these business ideas or kind of products are based around the AI component of then addressing a need and more efficient.
And that’s what we then drill down into and say, okay, what do we need to consider? You know, what are the hesitations in the audience? What’s the objections we to overcome? What is it that people are scared of? What are people’s preconceptions where they think they are an AI could never replace this or that certain thing, or you know being worried that it’s going to take their job or on the other hand being like wanting it to do too much.
Helena 06:58
I’ve also had that conversation this week where someone said like then you the AI people who he’s gotten on board to help create his product idea, they want to integrate far too many AI stuff and actually completely take away from the experts who are also involved and so it is really then about how do you have those conversations with the different stakeholders on the team to ensure a balance of kind of the human component, the AI component.
What problem are you addressing? How can you make sure that AI is staying in its lane and the expert is saying in their lane, and bringing it together to create something that’s going to actually drive humanity forward? Does that even make sense? I don’t know.
AB 07:42
Oh, it does. I’m hearing the same theme, even though different ends of the stick potentially I’m hearing balance and awareness and play and by the sound of it ‘knowledge breeds knowledge’ and perhaps both of those are in that same bucket.
Mirek for you, how have you sort of gone to getting to where you are now with background skills but also how would you encourage someone to fast follow if they wanted to, yes: catch up/overtake – what are the the tools and techniques that you’ve found the most useful to you?
Mirek 08:24
I don’t know if anybody would find it interesting, but I agree with both Helena and Violet that you need to look for a problem, what also look at the type and sort of try to merge these things together.
My work is on the intersection of augmented reality and the and human -mission interaction and robotics. There’s quite a lot in all these fields to sort of learn that you need to understand, or you to have a buddy or a teammate who understands that kind of thing deeply.
There’s quite a lot in all these fields to sort of learn that you need to understand, or you to have a buddy or a teammate who understands that kind of thing deeply.
Mirek
And so I started in AR before it even existed. You couldn’t get HoloLens even. It was this thing that’s coming next year or the year after. I still wanted to get into the field, So I just took whatever was available and I hacked it and modified it, and as long as I needed to have something I could work with, it’s a painful process, it is not fun, most of it you don’t see results immediately, it can take a really long time to actually modify something, or I have a platform that you’ve created, especially when you’re approaching it by knowing nothing and, you know, having to learn everything as you go.
AR or VR, that’s no longer the case, you can just grab Oculus and your golden, you can just, you know get that for a few hundred bucks, and Unity, and then you can start prototyping and you can learn from others and see what library is available.
Your mileage may vary. It depends on what you doing and whether it’s already, whether the components are already out there. If they’re not, you have to get your hands dirty in any case. Or you might end up with something that has been done hundreds of times already and nobody cares.
If you just take whatever is available and then mesh it with other things, you’re most likely not creating anything unique. I think for that you really need that deep technical knowledge to be able to actually both understand the system and able to sort of imagine what you can do with that.
Mirek
In robotics, the learning curve is crazy steep. You have to have some general understanding of electronics and how these things works. If you want to get into that, get an Arduino, that’s the fastest path to like get some really basic knowledge and then build it from there.
If want be playing with these AI systems, I reckon, and I didn’t get that far yet, I still have to sort of, you know, catch up on that. Everything is ROS based in this industry, that’s my understanding.
Mirek 11:18
And ROS is this (it stands for robotic operating system) – it’s essentially a messaging system, so you have multiple nodes running on multiple pieces of hardware, talking to each other. one drives your motor, one reads your sensory data and something else and meshes it together and does the decision -making.
If you want AI plugged into that, you’d better understand this environment really well and that may take a long time really. It’s the roast in particular is like after a week you you kind of get the idea how it works like you know the overall principles but then six months later you’re kind like still there and you feel like what you are doing but just be really it takes a lot of patience and a bit of determination to sort of understand these concepts to be able to then innovate on them because that I assume is what do you want to do, you want build something that isn’t out there.
And if you just take things that kind of exist and put them together, maybe it’s not as unique as you’d like to. So my recommendation is just equip yourself with a lot of patience and just set out a time to learn new things because you need that.
AB 12:37
Yeah, I’m hearing a thread emerging. I’ll wrap it up in a tick, but I am definitely hearing thread. I will put that in the parking lot, put it into any other business metaphor you want. William, to you though, your journey and then your way forward? Can you sort of chart as a here to there kind of pathway?
William 13:00
Oh, my journey – it’s been an interesting one. Perhaps I’ll say serendipitous. I’ve been tinkering with electronics and writing codes since I was probably six years old. And when I went to college and grad school, I wanted to do something that was simultaneously technical, but also creative and also tangible.
And so that led me to architecture. And you quickly realize there isn’t actually that much technical in it, at least when when went school. But we saw the emergence of that technology and architecture is always a bit behind the curve on adopting particular types of technology, which wasn’t the case in the early days of what we know as the modern profession, say the late 19th century and other periods where architects were actually inventing new means of drawing, new needs of communicating and representing space and translating that into the real world, things like projected, prospective projection and descriptive geometry, things that.
But I did not like the profession itself. I was more enamored with things like design techniques that were emerging from other fields, and that we’re finally integrated into architecture in the early 2000s, things Like evolutionary design.
William 14:32
And so my thesis involved writing a genetic algorithm to optimize acoustic reflectors, things like that, and there was a small cohort of us who really love this idea of writing code to generate form and design buildings, so how could you start with something that was relatively primitive or random and then evolve it into something meaningful?
Then outside of school, after I graduated, I ended up as an academic for some time and so taught structural engineering and also tried to incorporate some of these newer techniques emerging like what we call parametric design, so tools that take a form and essentially extract the numbers out of it and then you can play with the number and see what that results in.
And so using things like finite element method to analyze structures that you might generate, and you kind of tweak the numbers again, you get a different shape and then you analyze it again and iteratively do that.
But then I ended up sort of going back and forth between something related to space and something related to AI or technology in general. I ended up at a friend’s startup that was effectively a machine learning and data science startup where we were building a platform for a unique programming language that would allow a data scientist to write small chunks of code but execute them at scale on the cloud.
Then I was a product lead at a company called Florida, the head of product there, where we engaged with VR. And so that was like, marketing experiences in the browser for real estate brokers, Where we had an Oculus and where we, had an experience there.
William 16:40
This is before my time, but they built an experience with the Oculus that would allow the largest development in North America at the time which was the Hudson Yards in New York City. They allowed the developer to lease the retail space by having an Oculus headset and having them walk around the mall and essentially pick which retail spot slots that they wanted.
So that was rather remarkable, I think, at the time. One of the products I worked on and helped to build was a generative floor layout product that also leveraged the 3D technology we had at the time. And then I went into cloud computing, and then went in to sustainability tech. And now here I am again recombining space in AI. Now where AI is actually these advanced capabilities of artificial intelligence are far more popular and far accessible.
Just two years ago, things like BERT and GPT-3 were really niche developments and you had to be in the circle to know about them and really appreciate them.
William
AB
And you could run them yourself.
William
Yes, totally. Now that these large language models can demonstrate this reasoning capability, I think it was just the right moment again to take my fascination with space and design and combine it with AI and just ask the question, and like, can we do this?
Can we can do something meaningful and what’s the potential of it? And so the half of me that’s the product manager from cloud computing and from real estate tech is saying, yeah, we need the right problem to solve.
Don’t just throw a chat on it. But then the researcher designer technologist side of me is like okay, but we we need to incubate this in some way and provide some wisdom so we can explore. And finally, the one piece of advice I would have is like you everyone needs a project that they’re passionate about to lead them into something like this which is really complex and really technically grounded and but also has a lot of potential when I was in architecture school there was no there was very little time you had payoff with the work you were doing you would you would work on a design for weeks and weeks on end only to be eviscerated in your jury But with programming, every few minutes you could have a win.
Something worked, and it was magical, and you can see it happening, and so that journey, I think, is really important when you’re trying to deal with something new.
It’s like you have to enjoy the work itself, and that has to motivate you, and the big payoff at the end of you sold to the client, you shipped the product, or whatever, is part of it, but you need the project so that when you get up in the morning, you really look forward to sitting down and struggling through the hard parts.
William
AB 19:44
I hear you big time. If I can summarize a couple of those things and we might wrap in five, is that spatial AI is definitely a niche field within the fields that we generally play in. Spatial computing is obviously 3D and has all the different complications of the 3D modeling world and environments and deeper, denser data in more dimensions, literally.
So spatial computing versus regular computing: data science is rough enough and a great learning curve. It can be tables, it can 2D data, but then it gets to 3D and then time-based, well, 4D, whatever you wish.
It makes it rougher, so that is a steeper learning curve or you’re combining fields there to even start with. The geospatial world, if it’s 2D – it is great: there’s a lot of learning and there’s a, you know, it’s its own field and has been for hundreds of, well, for 100 years.
But to go into third dimension, suddenly you’re playing with 3D tiles and elevations and trying to do data science on, slopes and falls and things like that. It does make it harder to go in to a third-dimension, but that’s probably the defining factor that all of us can either visualize or contend with or our brains are kind of okay to it.
I’d probably suggest that those who have played games since they were little kids, probably are very familiar with navigating 3D worlds at pace. That’s probably been a helpful tool rather than a hindrance. So there you go parents, let your kids play games.
The thing that I’m getting, amongst all of us, is that probably being self-aware is probably the number one thing to sort of get ahead the fastest.
AB
We’ve talked about playing and iterating. We’re talking about as a customer or as person trying to find ways to how can I do things and to honor Mr. Rumsfeld who did the classic known unknowns quadrant thing. You don’t know what you know.
If you can’t ask for things that you don’t know about, so you’ve got to go in baby steps to ask questions, then that leads to more questions. Your vocabulary gets larger as you go down the path. So just starting is probably the number one lesson we could probably say.
AB
I’m also hearing that having a buddy, having someone to bounce ideas off is pretty wild.
Either your code gives you the feedback that it worked is good, that’s a good buddy. Having a real buddy to push you along is fantastic. Having someone to bounce ideas off is quite brilliant. Mirek, I’m quite fascinated by a subtopic that we’ll probably come back to is, are you your own customer or do you see a need and then the build it and they will come and try to be your customer. Is that good enough in 2024?
Mirek 22:23
I think you have to start somewhere and William actually put it put it nicely. Serendipity is quite important in this process. So you’re digging into into something that you find interesting. Engineers like to engineer and you know, understand things and thinker and build stuff.
Sometimes you cross a whole new field that you had no idea existed before you started. And it’s gonna take you ages to become an expert on that. So it helps to have somebody find somebody who can help you with, you know, solid block of things to do that need to be done, or you just need to teach yourself how to do it.
Mirek
I tend to do it the later way, I like to understand things deeply, that takes a lot of time. them. So I start by, you know, usually the process is like, here’s a crazy idea. I need some peer approval for that.
Just tell me I’m not going crazy, because then I am going to spend a year actually building that thing. So it helps to bounce ideas of people. And sometimes you really hit a wall and you need to talk to people who’ve been there before, in this particular field and to help you with This is not how we do it, you just take that and this is how you do it.
You had it all backwards, that sort of thing, because oftentimes you’re stuck in your own head and digging in the wrong direction and it helps to talk to people. Also, the process leads to, in my case, I just build a product by just digging into ROS and robotics and I discovered something that’s really difficult to do.
And I built a thing, tool for that, that I’m using myself. I am going to be releasing that as an open source. That’s something I didn’t plan for at all. It just happened because I, there’s a robot behind me that have been using.
I bought that one, learn about, you know, robotics and all this, but now I modifying it. And there is a bit of hardware. I’ll break you on that. Our listeners definitely won’t appreciate. This is like that allows it to go up and down.
Yeah, but this is where you need to get because that robot is now too small and limiting who I needed to do. I need to some physical modifications of that and that takes you to whole, you know, other places but it’s a slight diversion but serendipity is an interesting property of this of this process and you’re just digging in various different directions.
Sometimes you come across something that ties all these things together. It’s useful. It solves a problem. And you had no idea the problem was there when you, when you started.
AB 25:15
So I’m using the phrase self-aware in a positive sense. I’ve not trying to be negative in the slightest. We probably – all of us, and this is the, probably one of my, uh, life lessons – is that it’s good to know how you like to learn and don’t try and fight that.
If you need to be tactile or you William Violet, can we all have help with the… how do you pronounce it? Icky Guy? (Ikigai: https://lauchlanmackinnon.com/rethinking-ikigai-how-to-find-work-you-love-and-make-a-difference/ )No? Yes? Kind of.
It’s the classic Venn diagram which you’ll see in meme format on everywhere, but it’s basically a VENN diagram of, yes, three or four things that you love, things you’re good at, things the pay well and things I guess the world needs as the new addition to the VENN diagram.
AB 26:21
And there’s no peril wherever you want to sit with your passion, your playing, your learning. But some of those places that you sit are great and that’s an open source project or a weekend project. Others are this is where you want to head to.
So for me being ‘self-aware of’ I can sit here and say “weekend project – this is just a throwaway”. But these ones here over in this hand wavy area? that’s where I kind of want to be. And if you can ever find things that pay well you love you know the old adage of you’ll never work a day in your life.
That’s the hope for us all. We might leave it there, everyone. We’ll catch you next time on SPAITIAL, where we’ll come to you with News of the Week, then on a separate episode, we have the next topic of The Week.
AB
Cheers all, thanks for your time. Bye. Bye
Violet
Bye Bye
Key Takeaways:
- Get hands-on experience with tools and systems
- Collaborate with others in the field
- Take an iterative approach to build skills over time
- Allow yourself to be a novice and experiment
- Start simple (e.g. Arduino) and slowly increase complexity
- Identify a passion project with quick wins to motivate you
To absent friends.
