In this episode, Audrow Nash interviews Erik Schluntz, co-founder and CTO of Cobalt Robotics, which makes a security guard robot. Erik speaks about how their robot handles elevators, how they have humans-in-the-loop to help their robot make decisions, robot body language, and gives advice for entrepreneurs.

Start

Having their robot take elevators

How the sensors are used (aka, fulfilling IoT promises)

Training the robot as a security guard

Adding a taser on your robot? + What it means to be a security guard

On being a technology and security company

Stopping a thief with their robot

How fleets of security robots are controlled

Understanding the human operators

Tech stack

Experience with ROS

Extending ROS packages to work with elevators

On open source software

Growing the team

Manufacturing the Cobalt robot + battle-testing their robots

Sourcing components

Making people feel safe around the robot

Robot body language

“Dog-fooding” to improve robot body language

Working with the research community

Having the Cobalt robot behave like a receptionist during the day

Impact on security guard jobs

Human-in-the-loop with robotics applications + levels of autonomy

How are you using machine learning?

Erik’s background in startups

Advice on picking a problem

Cobalt’s funding

Advice for founders looking for investors

Advice on avoiding bad investment

Advice for a younger you

Getting in touch

Hi, everyone. Welcome to the Sense Think Act Podcast. I'm the host Audrow Nash. And in this episode, I'm talking to Erik Schluntz, co founder and CTO of Cobalt Robotics, which makes indoor security guard robots. Erik and I talked about having the Cobalt robot take elevators, how Cobalt has humans help their robots make decisions, robot body language, advice for entrepreneurs and several other things. It was great to talk to Erik, I enjoyed learning about how cobalt uses humans and robots in their systems, which each play to their strengths. It was also a great seeing how cobalt robotics has navigated getting investors and funding. As always a big thank you to our founding sponsor, Open Robotics, and I hope you enjoy. All right. Hi, Eric, would you introduce yourself?

Hello, I'm Eric Schluntz. I'm the co founder and CTO of Cobalt Robotics.

Absolutely. So Cobalt builds indoor security guard robots. So these are robots that will patrol around an office building, looking for anything out of the ordinary. We work with human operators to help support the robots. So if the robot comes across anything, it doesn't know how to respond to it switches over controls are humans. And then our humans can escalate exactly like a human guard on site would, whether that's calling the customer or calling police, or having a friendly video chat, instead of asking how someone's doing just like a security guard.

Nice. And so for the people listening? Would you describe what your robot looks like? And how do we refer to it? What do you just Is it the cobalt robot

or just a cobalt robot, we actually we decided we didn't want a separate product name from company name. That can be hard to to brand both of us. So it's just a cobalt robot. And I'll describe this actually, while playing a video for people that are watching. Sure, we'll get this going here. So the cobalt robot is about five feet tall. And it'll control around it has wheels, two main drive wheels. And then for supporting casters, it's designed to navigate around through an interoffice space, looking for anything ordinary, that can include doors or windows that have been left open, the body is actually coated in fabric. So it's a little bit soft and squishy. And this was actually a big part of our industrial design was that we wanted the robot to be friendly. The job of a security guard is to keep people and employees feeling safe and comfortable. And so if the robot is scary, you know, it really hasn't done its job, right. There's also a touchscreen on the front of the robot that we use for being able to interact with anyone around the robot, where one of our human operators can video call and talk to a person on site. We also have a lot of sensors, cameras that are 360 degrees around so no one can sneak up on the robot. And then a ton of sensors that you find in a self driving car like LIDAR and depth cameras, and I am Yes.

So how I'm thinking of it. It's like a telepresence robot that has a lot of sensors that makes it more sophisticated. And then you have the whole like person in the loop thing, which makes it a good bit more sophisticated. But is that a like a squishy, a basically a telepresence robot with a kind of cylindrical fabric body?

Yeah, yeah. And I'd say that's what our design is basically that it's telepresence five or 10% of the time. And I'll tell you this 90% of the time.

Okay. Now, and the one behind you has a little arm.

Yes, yeah. So this is one of our add ons for some of our customers. For our security patrols, we want to be able to patrol through multiple different floors. And so to do that, we need to ride elevators. Now in the past, we've done elevator integrations that are wireless. Those take a long time, elevator companies are often very slow. And so actually what we do now at a lot of customer sites is just physically press the button. So we added a one degree of freedom actuator, swing up and press elevator buttons. And we use the cameras and computer vision to detect the buttons know where we should press. And we actually have this rolled out at a bunch of customer sites now. And it is a lot faster to roll out in a wireless integration and it works pretty well. So yeah, I'll show a video of that for people at all kind of narrate that. So the robot first starts and presses the button on the outside of an elevator. waits for the door to open, navigates inside. elevators are very technically challenging thing to do. We find tons of things like elevator thresholds that are actually out of spec that would cause a lot of problems for people in wheelchairs.

What do you mean? Is that like a big bump getting into it?

Exactly, you know, we find elevators where it's a two inch gap, you know, between the elevator car and the floor or you know, one inch vertical, but once the robot gets inside, it uses computer vision to see the paddle and know which button it should press, flex the correct button drives forward, depress it. And as I mentioned earlier, this arm that we add is only a single degree of freedom. So we actually use the robots drive train to steer the robot left and right. And to move forward and backwards, the arm only moves up and down. Yep. And we use the robot for the rest of the degrees of freedom.

You may I think you mentioned so it has to the cobalt robot has differential drive or something so you can turn it's not Omni drive, so it doesn't move side to side.

It's it's differential drives, we have two main drive wheels and four support casters.

Okay, is that tricky to get lined up with the button?

Yes, so we definitely put a lot of work into sort of that motion control and planning to be able to nicely line up and press a button. Especially because elevators often have very tight space, we don't have a lot of a lot of room just sort of back up by 10 feet and approach it nicely. It's, yeah, we have to kind of operate in these very tight spaces.

Gotcha. I wonder, do you have ever the situation where the elevator is so tight that you have to raise the arm and then kind of swing into it?

We haven't hit that yet. And luckily, there are rules around elevators, for people with wheelchairs. So we can actually just follow a lot of those same design specs to know what the robot hardware needs to be capable of gotcha.

And is it difficult when you go into a new elevator? To find the buttons? Like I imagine that this is fairly custom, depending on the use case, depending on how you're detecting things?

Yeah, so we have a sort of a proprietary system that we've set up where one of our sales engineers can go on to new customer site with the robot, and very quickly calibrate the robot for that site. So we don't need a ton of training data or anything new like that. We can kind of just bring the robot through the elevator once, and then after that, it's able to use the elevator by itself.

That's awesome. You kind of... Can I guess that what it is?

So do you use the depth camera at all in this for figuring out how far away you are. And

Yes, we mainly use the LIDAR for that. One of the other interesting challenges is that elevators are very reflective surfaces, a lot of times you have sort of a polished metal. And time of flight depth, the cameras can do very poorly with that, because

they're shooting out infrared and trying to see that but then it bounces off. So you don't get any good depth information, maybe very noisy information,

you get very noisy, but the LIDAR at the bottom of the robot is pretty good. And we know that in an elevator, it's a fairly structured environment where we see the walls is usually in line. Why is LIDAR

more reliable? intuitively, that doesn't make much sense to me. So for the depth cameras, are they structured lighting, where they're projecting something out? And then trying to observe what that is, with the pattern and the deformation in it? Or?

That's a great question. So the one we use is not structured light, it's time of flight. Oh, and I don't think there's any inherent reason that a time of flight depth camera is is worse than a LIDAR, which typically, what we see is that LIDAR is are a lot more powerful, and just put a lot more into those be able to get a much better signal response.

Gotcha. And the LIDAR of this robot is at the it's like, on the base, right?

Yes, at the very bottom of the robot, and you can see around 270 degrees around the robot. So that gives us kind of a good sort of footprint of what the the room that said, Yeah, and then we use the optical camera on the front of the robot, and can get very good sort of optical readings, write up the buttons and see where they are.

Yep. And then how do you control the press for this, I'm imagining like, you draw, you basically raise that to the correct height that you imagined the thing is, and then you just drive forward until you hit some sort of drive

forward. And we have a set Oh, very tip here to know whether we've made contact cool little forest, what are the eggs? Initially, we had a force sensor, and then for reliability, we just switched it to a, you know, a switch. That's, that's calibrated. Yeah, one of the interesting things is sort of, you can we can tell that we've hit something, but you don't necessarily know whether you engage a button or the or that you know, the rim of the button and maybe didn't engage it. So there's some interesting things we can do to you know, to sense whether we'd hit that one of the first things we thought of is oh, you know, we can tell whether the light, the light turned on in the button, but you'll actually find plenty of elevators that don't light up. And so actually what we do now, is we have an accelerometer of the acceleration of the elevator.

Exactly that's kind of you know, the sort of that's like the the ultimate end goal is we feel like the elevators not moving. We know that we've correctly told the elevator what to do.

Now is there ever the case where the robot is in the elevator pushes the button, but Miss This so that it didn't actually click the button and then someone calls the elevator from a different floor, and it sees it.

Absolutely. And so I think what you brought up is that one of the really interesting challenges of this is that when we are in an elevator, in the worst case, we're inside a Faraday cage. And so this has to be often this has to be 100%. I'll

speak about it in a cage for just a little bit. Yeah,

absolutely. So a Faraday cage is a closed metal structure that no radio waves or electromagnetic waves can get in or out of. And so that means that for some elevators, when we're in them, we don't have Wi Fi or cellular. And that means that we can't rely on our human operators that generally we use for ourselves.

That's hard, because it's like one of the more difficult situations. Exactly, exactly. They're in a Faraday box. That's crazy.

Yeah, so it's very funny but And generally, what we say cobalt is that one of our sort of really fun advantages for our engineering team over say, working at a self driving car company, is that we just have a lot of inherent safety and stability that we move slowly through indoor spaces. If anything happens, we can just stop and call a human operator, you can't do that on a freeway. And so that means that we can basically solve for the 99% case, and we don't need to worry about solving the last percent bring anything in the elevator, except in the elevator. That's where we need to get 100% You mean safety

instability? For this yet safety and stability and stability?

Yeah, you know, we're not trying to balance on two wheels, it is always a safe situation for the robot to stop. And so we can take a lot of advantage of that, and basically, handle covering a lot of edge cases with that safety behavior of I stopped the robot and wait for a human operator, except in the elevator, then they realize

it. It's two interviews that this has occurred. So I have to figure out how to turn that off.

Okay, yeah, I'll need to be careful not to save the G word over here, but my phone. Yeah, so And basically, we need to make sure that because we're in this Faraday cage, where we can't call for help from an operator, that this robot is going to get itself out of the elevator come hell or high water. And there's all sorts of really funny situations you can get into. Like you mentioned that, you know, perfect timing, you can miss the button, but someone else calls the elevator at the same time. So it thinks that it succeeded. And we have really robust state machines that we've built around these behaviors. So the robot knows what state it's in and knows how to retry from pretty much any state and knows that these kinds of things happen. That even though it's it's felt the motion, it knows there's a chance that maybe this is just a coincidence. And if it doesn't go to the right floor, hey, I might need to retry.

Ah, okay. Yeah, cuz what I was getting at with that is I'm curious about the mapping problem. So how you split up different maps, I suppose it like maybe Each floor has its own map. And then you try to once you enter a new floor, try to figure out where exactly you are in it. So that which Am I in for one or three? This kind of thing?

Yeah, so there's a I think two interesting things that one is sort of knowing what floor we're on. And the other is, is knowing where we are sort of inside the floors. For the know, in the foreground, we actually use the accelerometer to the

robot. So you need to know the accelerations to try to we do what did you? And I suppose you decided not to use like a barometer or something like that. Was it too noisy, or I mean, not accurate enough.

So we explored a bunch of different sensors, including including barometers. One of the things that we found there is that indoors, pressures terminal lot more by the H vac system, oh, that's an actual altitude. So you can go on one floor and the pressure can actually be lower, even though it's higher, because the air conditioning settings are different. So it wasn't a really good sensor for the Tirpitz. Wild. where we are. Yeah, we did a lot of really interesting sort of looking at different sensory modalities for this. Now the accelerometer, you're absolutely right, we're double integrating Yep, and double integrating is a scary thing. If you do it forever, you're going to either end up going to the earth's core or going to the moon. And so one of the things there is that we can actually apply a lot of structure and constraints that we know about how elevator systems work, in order to basically constrain that and add a lot of these checks that keeps the robot from going to the moon, or the floor. So an interesting one there is that you know, that basically, floors are basically integer divisions. So if you stop, and you're kind of Oh, I think I'm at floor 1.1. You can actually snap that down. Yeah, for what, an elevator never stops at sort of one and a half floors. And similarly,

exactly what's working. And similarly, there's a lot of these other constraints that we found out well researching and testing on elevator systems that they follow very consistent acceleration profiles and speed profiles. And we can use a lot of those things to basically add these constraints. So it doesn't become an unbounded double integration, it becomes sort of a much more constrained problem that we're fitting to. Yeah. And we didn't tell what floor we're on pretty accurate.

Gotcha. So you integrate your acceleration results, and I guess, probably, so integrate, get velocity integrate, again, get position. And then you look at the position of the robot, how it's changed, because it's moving. And then you get some sort of rough approximation of how much you've moved. And then you use that. So it's like weighted information. So you say, Oh, it's probably about this.

Exactly. And we can use this extra information of, you know, when we know that we've stopped, we can basically clamp it to zero until we see more than a certain amount of acceleration. Because we know that when the doors are open when we stopped, you know, we might be reading a little bit of acceleration, but we know that elevators don't actually jiggle around. You know, if it's mostly stopped, it's a terrifying elevator. Yes. it lessens the terrifying Okay, exactly. So basically, we found a lot of these hidden constraints that let us do this very accurately, and filter it no before.

Interesting. What if you have to go up 20 floors or something like this? Like it's a skyscraper, say 50? floors?

There's, there's definitely a point at which breaks down. I assume I'm actually not sure where that is. But we have not yet hit a we've not yet hit a building where it does not work. trouble. Yeah. It. We've been in some high rises in the San Francisco area.

And it still works fine. That's awesome. Yeah. So if you want to go around that, yeah. If you want to go to like the 32nd floor, or what's the how large are these high rises?

We've done jumps of 20 floors with good accuracy. And I've never tested above

That's awesome, though. Hell yeah. I mean, I'm impressed. It works to that level, which is awesome.

Us as well, we were initially much more optimistic about the barometer, than the accelerometer, and we were actually very pleasantly surprised that it works out. And again, you can do nice things like right before you get on the accelerometer, right? Before you get on the elevator, you can pause for five seconds and zero out your IMU calibration, because you know, hey, I'm, unless there's an earthquake going on right now. I know that I'm not accelerating. Let me save that as gravity. And then, you know, for for a five minute elevator trip. You know, that stays pretty good in that over that short period of time.

And then how do you? How do you store your maps? Is it like, so I'm wondering if it's a so say you have two floors? Is it one map that includes both floors? Or is it two separate maps? One for each floor? Or each each? Like? I don't know, Consett? consecutive space are contiguous. I don't know, continuous space contiguous space.

So that's another really interesting question that we had at the beginning while working on this. And it's also an interesting question to ties into our hdri of how our operators and how our customers interact with a robot. Because this isn't just a robot operating in a vacuum. This is the operators and customers want to see where it is on the map, they want to see the alerts coming from the robot. And actually, what we found what they liked, most was not needing to switch between maps and only being able to see one floor at a time. But seeing all their floors laid out side by side, just like you would on a floor plan. And so actually, what we do is we just have them on one giant map in different locations. And so a robot switching floors is implemented as just teleporting to kind of a new island in this map that work and go between.

Interesting. And the elevator is like the portal between all of these different floors. Exactly, yeah. Okay, now. So getting into the robot a little bit more.

So it has a bunch of sensors. Would you just talk a bit more about the sensors that are on it?

Yeah, absolutely. So there's, I'd say two categories here. The bottom half of the robot is like an indoor self driving car. And basically, its job is mobility and safety, to bring the robot around through a customer space and get the robot anywhere it needs to be. And the top half of the robot is all about providing the security service that we provide to our customers and providing that value. And basically, it's a mobile sensor package that can be brought anywhere. And there's a couple of high level goals there. The primary thing that we do for our customers is detecting people. You know, if we're trolling around the warehouse at three in the morning, you want to know if there's a person, you know, they're in the warehouse. Yeah. And a lot of times, you know, they are supposed to be there, but we get an alert to our operators, and they video call in and say, Hey, excuse me, can I see your badge? Why are you here after hours, just like a security guard on site. saying, you know that they can authenticate that and make sure they're allowed to be there. The other big category of things that we do is facilities inspection, going and taking pictures of sensitive things, whether that's an expensive piece of equipment, either using our optical cameras or a thermal camera, or checking doors, sort of sensitive things like that. And also environmental checks. So we have temperature sensors, humidity sensors, carbon dioxide, and carbon monoxide sensors in the robot. And what's really great is being able to move these sensors, this sensor package around at a tire building, and not just get sort of a single point measurement like you would from an air quality sensor. But being able to generate a map of air quality, and being able to say, hey, not just what's the carbon dioxide in my office, but get that map and say, Wow, the carbon dioxide is all coming from this corner here, we must have a broken bet. And be basically taking that promise of what like IoT sensors trying to do, and bring in 1000 sensors bring it everywhere, and to localize that data on a map. And so we've helped customers catch really interesting things. Whether that's broken h vac systems that could possibly pose danger, we actually found h back issues where the with temperature, where we show that one half of the building was like five or 10 degrees colder than another. And they said, Oh, man, we've had employees complaining about this. For months, we didn't believe them. But now that we see the data from the robot, and so really what the robot is, is, we want to bring sort of superhuman sensing to what the guard could do. And the things that a guard could notice, while looking around and that that can be security or facilities.

Did you Do you guys ever so you have this arm now. And I feel like that's a relatively new thing to push elevator buttons. You ever, like try doors to see if it's a hinge door to see if it's locked or interact in the environment, interact with the environment in any way with it.

So we have some interesting things coming out for doors in the next year. I can't talk too much about Okay,

So, and I will say with this, we can press handicap up and Stoker doors.

Ah, that's awesome. Let's see. So with these sensors, now I noticed is with all the air quality ones, I basically how did you select the air quality sensors? So you have like carbon monoxide, carbon dioxide smoke? I don't know what else? How did you select the sensor packages that you had humidity and temperature? How did you select these ones? For as opposed to anything else?

Yep, exactly. So at the beginning, sort of when we we did a lot of customer discovery asked our existing customers, what are the kinds of things you're worried about in your hair? What are the kinds of things you you want to know about your space that you don't have right now, and sort of took a lot of that input and then looked at what was feasible, we actually have a radiation sensor on one of our robots from one of our customers. So that's not a standard option. But you know, we could add other sensors as needed for special cases.

Gotcha. I was gonna ask about customization next. So you are customizing a little bit I imagine. So with the body of the robot being kind of a big cylinder covered by fabric, it seems like you can probably have pretty large sensor payloads, and you can swap them out fairly easily. So you mentioned the radiation. One, but do you do Is there a good amount of customization that occurs or

only for very large customer deals, we really try to stay away from as much customization as possible, the sensor package is one thing that's a little bit easier for us to do. But it's sort of right now it's tied to sort of large new deals that we do with customers, I would say that the majority of what we do that's custom, I would consider more configuration than customization. So we can configure the robot with actually different color fabrics to match the customers. And a lot of the robots of behaviors actually are very different between customer sites. And one of the ways we think about this is that security is basically a big human configuration of if this than that, if you see a person that doesn't have the badge, call this person, if you see a door left open, do this. And it's these are sort of the instructions that are given to human guards and sort of human security teams, but how they should respond to all these different situations. And so our robots are very highly configurable for different customers. So for one customer, you know, seeing a person might be just oh noted in the daily security report. And for another customer it might be immediately called the police. And so you know, there's these very, very divergent actions based on what the robots attacks. Yeah. And we build systems that let us very easily configure that, so that the robot can be to basically do exactly What your guard was doing?

Yeah? How so how does that work? So for you to customize the robot's behavior? How, like, are you using our Do you have users use like a block programming language? Or is it just YouTube and a bunch of values? How does this work? Or use maybe select from a drop down? If this occurs, then do this? How do you set this up? Yeah.

So we'd like to make things as easy as possible for our customers and our customers. They just want security that works in they don't want to think about the robot or programming anything. Yeah, so typically, one of our account managers who's an expert insecurity, and also now an expert in our robot system, they will work with a customer and look at their existing instructions for their guards, and basically translate it into our configuration for the robot. So the customer just has to give us their instructions as if we're a security guard, which they love it. And then basically, we turn it in, on our side, this configuration for what the robot does.

So it's a fairly one off thing done by an expert, is what I am hearing for this. And so

Exactly, yeah. And this is sort of pretty standard for how the security industry works, that every new every sort of new building, or company, when they get security guards, they have an account manager that basically helps set what the what are the instructions? What should they be doing on the site?

how detailed is it? Like, how many things to configure? Is it? Is it like a huge list? Or is it a few? Simple If This Then That statement? I guess it may vary? Yeah,

it varies, I'd say there's some customers that sort of are very new, and, you know, young customers that are expanding have their first few buildings. And we actually usually help them define sort of what their procedure should be. Because we've seen a lot of much more secure much more mature security programs, and help them sort of know, hey, well, what happens if this person doesn't answer the phone, like, you know, you don't want you don't want something to just stop. And so if it's a bigger company, we'll know, here's the whole call tree of how you escalate things. If someone isn't available, and for younger companies, sort of we help them define a lot of that same security program with the same rigor, so that we can help them get

to that level. Gotcha. That's cool. That's interesting. You guys are becoming security experts through this problem.

Exactly. And sort of, for us, we have a big robotics team. And we also have a security operations team. And they're the people that are interacting with customers and using the robots and using our control interfaces, to configure them and to respond to alerts and help keep the customer safe. Yeah.

So have you guys ever had kind of a silly question? But have you ever had a like, where a security guard would like, maybe try to physically stop? If physically stop a person from doing something so you there's like a thief or something? And they would like tase the person? Like, is there any like, would you put a taser on the robot or like on the little arm that comes up?

So there's, there's no taser on the robot. And I'm actually really happy. You asked that. One of the little known facts in security is that a security guard will never touch a person, it's a huge liability issue. All they do is call the police, you'll actually see a lot of security guards that either have fake guns or unloaded guns through that it's entirely for deterrence. Whoa. Because imagine sort of the the liability if a guard misjudges. And a person was actually an employee, they thought they were you know, up to no good and tackle someone that hurts them, you know, be this huge liability issue. And so it's very funny when we started off, this was one of our big assumptions as well. And this actually kind of gets to how we started cobalt of when me and my co founder, Travis style started cobalt, we actually didn't know what we were going to work on. We had both started companies in the past, we both been through Y Combinator. And that's where we met each other. And we've seen a lot of people come up with a startup idea, and then really get overly attached to it, despite it not really being needed by anyone. And so we decided that we were going to take the opposite approach. We were going to find a real customer and a real problem, and then work backwards to the idea. And so you know, he quit Google, I turned down a return offer to SpaceX with no idea what we were going to do. Both of our families thought we were pretty crazy, but we're like, No, no, no, no. startup idea is crazy. This is better. And so we spent the first few months interviewing people from different industries, anyone that we didn't really know what their job actually entailed. We wanted to talk to them and just asked, hey, what are your problems day to day like what what would you pay money to solve? And luckily, one of these people we talked to was a director of security and he started complaining about his guards, sort of how unreliable nightshift you know work can be why his cameras weren't enough. Yeah, and we kind of the first thing we said is can we build these smart security cameras. And he actually said, No, like, I already have smart security cameras. But someone, someone comes in wearing a hat, and you can't see the face. It doesn't matter how smart the cameras, you either can do nothing, or you can call the police. And that's why people still have guards, they still have guards to, to follow up and investigate as something's happening is happening, and understand to collect more information. And that's when we realized, Hey, this is the perfect job for a mobile robot. It's kind of that blend between a guard and a smart camera where it's not just about detecting a person, it's about once you detect them being able to follow up and gather more information, go up to the person and say, hey, how can I help you? And then you can make the call of, you know, what do I need to ask? Or not? Exactly, it's kind of like this, you need this third option of following up and getting more information before making a final decision. And so actually, now that's a big direction that our product is going is that everyone now has tons of smart cameras and smart sensors. And all of these smart devices, for enterprise security, create work, they create alerts, they create alarms. Yeah. And actually, a lot of our customers that we've talked to have had to hire more security guards, because they have all these things, that's fun as they need to go investigate them, they need to go deal with false positives. And our strategy is that all this other security infrastructure is creating alarms. And our robots are actually reducing work. Yeah. And the robot automatically gets dispatched to go investigate these things, and say, Hey, this door for this door force alarm was actually a false positive, the door looks fine. And then it gets this batch to the next alarm. And so basically, there's kind of these two sides of security, there's the detection, and then there's the resolving. And we actually want to be much more on the resolving side. So the robot is not just a moving security camera. It's much more than that. It's the ability to follow up and resolve things, which is what a guard does, not what a camera does.

Interesting. I feel like you've said a lot of very interesting things just now.

No, thanks. Yeah, it's a, it's a really fascinating space, I think it's very cool seeing how more people and enterprise security are building sort of these, they're trying to build these much more autonomous systems. And he's much more sort of systems for handling physical security without needing a lot of human input. But also, you know, one of our customers quoted general patents us and said, fixed infrastructure is a monument to man's stupidity. It's a quote from World War Two. And this is sort of why no one builds castles and fortresses anymore, is that anything for infrastructure, you have anything that static, is at some point, either going to be broken and not replaced, or someone's going to find a way around it. And that's the difference between cameras and sort of fixed sensors, and robots and guards is that we're infinitely adaptable. A customer can call one day and say, Hey, our left door alarm broke, can you change the robots patrol to stay in that area? And we're like, yep, we can do that. Just like a guard, you know, you can just call us. And we'll immediately responds to whatever that is. And that's why we don't want the customer to need to go through some interface. We don't want us. We don't want cobalt to be a static system. It was a flexible and responsive system. Yeah, that's why they have

I'm saying, so the way that I'm thinking of cobalt now, is you guys are almost like, but you guys are a security company. And your personnel is your robots. Exactly. Which is very interesting. It's an interesting model. It's almost like I don't know you've you've taken all it's just it's interesting, because you're not selling the robot, which is what I first thought it was. You're selling the service around the robot.

Exactly. Yeah. And I think that's, you know, as we scale, we're exploring, sort of splitting that off. So cobalt can stay more of a technology company and work with guard partners with the with the operators is

work with an outside partner for sort of those security to operators to be the security part, the cobalt can stay more of a technology. I see. But I think in the beginning, it's a really, really good strategy. Very, very,

I see seems to trigger Siri, when I say see God's sorry, what did you say?

That basically, you know, there's this balance of us being a technology company and us being a security company. And at the beginning, it's really powerful that we can be both of those things. Yes, you can do that. So we get vertical, we get vertical integration. And that means that you know, our engineers are sitting alongside our security operators and seeing what the deficiencies of the robot are and seeing what the pain points are for the customers and They're talking to security end users. And if we were just selling the robots, and someone else was implementing security with them, we wouldn't get any of those learnings. And so we want to be a place where we are a technology company. But that's very, very integrated with the security and very product focused, and really sort of end user focused

gadget that is very interesting. Yeah, initially, all the questions about configuration and things, I was assuming that you just sell your robot, they buy it, the customer buys it, and then they figure out how to use it in their space. When really you're saying, okay, our robot is going to be like an employee. And every time you want to update it, you call us we make the changes. And then you present them do so you present their security manager or something like this with the data, or whoever's relevant with the data that the robot captures this kind of thing. Yeah,

exactly. And that's sort of the way that we provide the end service, you know, the service isn't the robot, it's the data that we collect and provide to them. And there's a couple things a couple of ways we do that. One is, you know, if there's ever any urgent incident, we just call them Yep. And then the customers have a interface on a web dashboard, or they can live see what the robot is saying. And they can dispatch the robot to go see something. So we might call them and say, hey, there's a break in, in progress at this office, you know, pull up the dashboard, and you can watch while we resolve it. And we've actually, we've stopped a few break ins, oh, congratulations. Thank you, it's, it's actually much more boring than you would think, you know, modern Moto, you're just going

up to them. And you're like, stop, and they're like, Oh, no, and then they run out.

Modern criminals are not in ski masks, you know, breaking into Windows, they are social engineering delays, and office buildings. And so

what do you mean social engineering there? What is that

they're there pretending to be an employee that forgot their badge, but they're actually not, they're well dressed. They're, they're trying to blend in, they're trying to blend in, around other people, and sort of. So we had a customer where one of their door locks, stopped working at a door swung open to the streets of San Francisco. And fairly soon after that, our robot detected a person in the site. We approached them because our operators didn't recognize the person. And we video chat and said, Hey, excuse me, can I see your badge? And the person was the video like, Oh, I'm lost? Like, I didn't know I wasn't supposed to be here. It's funny. I was like, I have to in the office building? Sure, yeah, you're just so you know, here. You know, we call it the customer and call the police, and then escorted the person out the door. And then the robot was able to stay century at that door that was open for the rest of the night. And so that's the kind of flexibility and responsiveness that we want is that as an emergency happens, we don't want to just be on a fixed rail, like a camera, yeah, we're going to be doing what a garden would be doing. And that's, you know, making value calls as we go and deciding what is the best way we can provide value to this customer?

So how do you code in all that behavior and the flexibility I'm imagining like, something like behavior, trees, or whatever they do in video games, where it's like, you can have fairly complex layered behavior where I go, I try this, and that doesn't work. So I'm going to try this other thing. And you can keep moving from limb to limb of the tree. Is that how do you encode your behavior? Basically?

Yeah, so that's a really good question. We use a hierarchical state machines for most of our behaviors. But actually, I'd say a layer above that. And I think that it would be misleading to say that our state machines resolve that case, we keep our high level decision making done by our human operators. And so they are able to on the fly, say, Hey, don't patrol tonight, stay century here. And basically giving them thinking that for operators, it's more like them playing Starcraft, they can tell the robot what to do. And we provide a really good interface for them to be able to tell the robot to do different things. And that might be go patrol this floor that might be stay here that might be go recharge, okay, and that high level decision making, we will have to be made by our human operators who are very highly trained, and know our customers. And then our state machines that we build are basically these primitives for the robot to be able to do these block size to actions. But it's the human that's deciding sort of which of these should be done.

Interesting. So the human at a very high level, it's almost like picking the mode of whatever the robot will do. You do. It's like a delegation effectively, exactly. And then the robot will go and it will execute this fairly. This this decided behavior and you have all those primitives, those smaller action, control loop things go around this desk, go look in that unexplored territory. These kinds of things. Yeah, Yeah,

yeah. And so we want to build the system that I think it's, it's, we really consider our system. It's the combination of human and the robot human operators. Yeah. And that's really it's those put together that allow us, allow us to provide a great service for our customers, I think, you know, we've seen self driving cars being worked on for 10 years now. And they're just barely starting to actually give, you know, get out into the market, and give taxi rides to people. And for us, what really attracted me and my co founder to this idea was that we could use this combination of humans and robots to immediately get into the market and start providing value, like right away. Yeah, right away, even though pieces of the robotic technology weren't there yet. And now, as all this technology gets better, what it does is it increases our ratio of how many robots each one of our operators can manage.

1 post with the elevators tag