In this episode, Audrow Nash speaks with Melonee Wise, former CEO of Fetch Robotics and current VP of Robotics Automation at Zebra Technologies. Melonee speaks about the origin of Fetch Robotics, her experience at Willow Garage, her experience being acquired by Zebra Technologies, challenges in the warehouse setting, on autonomous cars, and on the future of robotics.

Hi, everyone, welcome to the sense think act podcast. I'm your host Audrow Nash. Today I speak with Melonee Wise, former CEO of Fetch Robotics and now VP of Robotics Automation at Zebra Technologies. We talked about her experience with Fetch robotics, the differences and similarities between mobile route robots and a factory setting, and autonomous cars, and her outlook on the future of robotics. I really enjoyed this interview. First, it was really nice getting to know Melonee. And also I found it very interesting to hear about the different stages that FX Fetch robotics went through before being acquired. Specifically, they started as a hardware company building the robots, which only lasted five months or so. And then they were a software company. And now, Melanie is describing them as a data company is very interesting. With that, I hope you enjoy. Hi, Melanie, would you introduce yourself?

Yeah, I'm Melonee Wise. I'm the former CEO of Fetch Robotics now the VP of Automation, Robotics Automation at Zebra,

So that's really exciting. Would you tell me a bit how Fetch and Zebra are related?

Yeah, so I started Fetch robotics about seven years ago. And this week, we actually closed on an acquisition of zebra acquiring Fetch robotics. And so Zebra Technologies acquired Fetch to create a new robotics automation division. and Fetch makes autonomous mobile robots for the logistics and manufacturing industries.

So going into Fetch a bit more congrats on being acquired. That's so exciting. And you said it was ahead of your timeline, your 10 year, whatever being your highlight, it was actually acquired in seven years. Would you tell me a bit more about Fetch. So it's for warehouse related robots, but just I guess, background information on the company.

Yeah. And to be clear, when I said I had a 10 year vision, What I meant is when I, when they first started talking about it, Fetch robotics to investors, I kind of I kind of made sure that they were in it for the long haul, because robotics is such a hard problem. So I was like, Hey, you realize that this is a 10 year problem, it's not like you're gonna get an acquisition, like you would in a software company and, you know, pump a couple dollars into flip into an IPO or an acquisition a couple years. And so, so that's there wasn't really like, I was like, Oh, I'm gonna sell the company in 10 years, it was more like, you know, when you go and try and convince people that robotics is something interesting to invest in, you need to help temper their expectations. Because, you know, if you look at the car industry, we've been saying, since, I don't know, 20, or 27, like 2007, that we're gonna have autonomous cars next year. And we've been saying that every year for the last, you know, 14 years.

So I so seven years ago, I started Fetch and I started the company with three other co founders, Derek King, Eric Dyer, and Michael Ferguson and we all worked at Willow Garage together. We were part of the robot development team had at Willow Garage. It's a team that that basically for about a two year period built a lot of different robotic prototypes to look at different industries that that were of interest to Willow Garage because but two years before it ended, the founder that started Willow kind of said, well, what's next? Congratulations, you did open source, although that's kind of a funny statement to make. But he kind of said Congratulations, you did some open source. Now what you know and, and Steve cousins at the time said, Hey, we need to go look at you know, the home and then industrial applications. And it's really funny when you look at you know, the people that worked at Willow Garage and, and the companies that were then formed by the former employees of Willow Garage, they all kind of reflect the different areas that we investigated in the last two years. And we investigated everything from like grocery retail to manufacturing to hotels, for example, Steve went and started savvy oak and did hotels and, and the guys in AI that started Fetch robotics. We we looked at at the manufacturing and logistic verticals and we saw it as a an interesting problem to solve. And, and when you look at that the reason that was interesting to us is we thought it was attractive, tractable problem, like most warehouses and most manufacturing facilities are what you might call semi structured. But there's also a large, like, dynamic happening in the industry where finding new employees is very hard. And even before the pandemic, there was a large labor shortage.

Yeah. For factories and warehouses. Before the pandemic, there was about a 600,000 person shortage. So there were 600,000 open jobs,

I don't know that statistic off the top my head, but I do know, before the pandemic there were, there were about 6 million jobs available. And the job openings in the manufacturing and logistics industry represented about 10% of all open jobs in the United States. Yeah, crazy. Yeah. And for a long time, the United States was running at a deficit for labor, right? So we had, for a long time, we only had about five or 6 million people unemployed, and we only had about five, we had about five or 6 million jobs available. And so it was a close thing. There were actually we were running what you might call a negative employment rate, where there just weren't enough jobs, there weren't enough people for all the jobs.

So that's interesting. I mean, you hear about robots taking jobs and things like this. In this case, it's the complete opposite. It's there are not people for that. But we need more people to fill out the roles so that the companies can be more efficient. That's very interesting.

And I think that's one of the key reasons why over the last couple of decades, there's been this strong push for automation. And why you're seeing in a lot of industrialized nations, you're seeing a lot of push to get more and more automation so that they can stay competitive so that they can grow their GDP, etc, etc, etc. And so that led us to be interested in creating Fetch and how know the

so that that is the case now, just before COVID. Fetch is seven years old, or so. Was that the case then, too? And how has that changed over time?

Yeah, so that that trend was very much true before. before when we started Fetch, there was just a huge labor gap in the industry. And it what's even worse is it's coupled with a turnover rate of about 40%. So super

high, basically. What kinds of jobs are these? Like? What are the what is a person and these jobs do further day at work?

Yeah, so they, they can do anything from like packing out a box to walking around in 110 degree warehouse picking boxes off the shelf? Yeah,

they can be assembling small parts or machinery they can be. I mean, our robots are doing the delivery task of someone's job and that and that's the other thing that conflate some of the problems is if you look at it, when they're looking at automation to offset labor, it's not typically like this robot is going to play replace this person 100% there's just there really isn't the possibility to do that in most cases today. And so we're why exactly um because if you look at it, there's there's a lot of tasks that make up your job. So if you look at say, me think a pickers job, if we look at what a picker does,

are those a person that picks things out of bins and then puts them into boxes, like an Amazon person?

Yeah, that's that's the overall they fulfill orders from customers. And so there's, there's a lot of parts of their job. So there's the part of the job where they first prepare all the boxes, and then they put them on a cart. And then they push that cart to a location in a warehouse where there's a shelf and they pull something out of a box, and they put it into a box on that cart. And they're like scanning each of those locations to make sure that that thing that they're they're picking is correct, etc. And they do that through an entire warehouse and then they come back and they typically offload the boxes from that cart on to an area where either separate person or that same person packs it out. So they take it from those boxes, confirm that all the the items in it are correct, and then get it ready for shipping. And that means putting like paperwork in it and basically shoving it down a line. So someone can put a label on it and tape it. But if you look at it, the picking task is very hard. It's not like there's a lot of robots today, they can just wander up to a shelf and pull an item out of a box, especially when they're, let's say they're their t shirts. And they're, I don't know, baby onesies. And they're packed in a box, you know, that is probably only meant to hold 50. But there's 500 of them in there. And so it's a pretty complicated task from a robotic standpoint, whether it's manipulation or vision. And so we today can't do all of the task. And so there's some things that we replaced completely, like we don't have them walk all that distance, they'll meet the robot or something like that, or, or we change the roles like the now if you have a robot that where the person used to configure the cart and put all the boxes on the cart, maybe one person now is just responsible for doing that for every robot, as opposed to that person individually beer sponsor responsible for their cart. And so the the jobs change what the roles are change, and and really, it's about taking out the part of their job or the set of tasks that they do that can be done by a robot. And I, I think that there's good analogues in kind of the technical world, you know, if you if you look at, if you look at maybe writers, or journalists, they previously had researchers, and they had secretaries, and they had all these, all these other tools or resources available to them. And now, many journalists, their entire job has been reduced down to doing all those things themselves. But now the tool they have instead of a person. is Google, or, or some search tool. Yeah. Yeah. And, and so I think there there's plenty of analogues for kind of this transition from people doing tasks or part of our jobs for us to us now using automated tools.

Yep. So if I understand correctly, basically, the shipping process has a whole bunch of different components that must get done on the way. And what you do is you say, oh, a robot might be very good at this one spot. And so by adding robots into the system, you free up people from doing that one task. But it's kind of it's not like saying a robot replaces a person. Because the there are other things that the people can focus more on, when the robot frees up that one, that one area with this kind of thing. So that's why it's like apples to oranges to say one robot in is not one human out, kind of

gigantic shortage, so you can just shift the people elsewhere? Probably.

Yeah, that's the other thing is, is when you have such a labor shortage, you you're not displacing anyone, you're just trying to fill in the gaps.

Mm hmm. Yeah. And make the few people that are there as efficient as possible. So that they can you can get as much work done as possible, socially? How did you guys find the So you mentioned kind of the detailed process of finding or of what occurs in sending out pick and place this kind of thing? or picking products out? How did you find the exact part of that where you thought a robot would be a good fit?

Yeah, I think that you know, if you look at it, some of this stuff that that we did, it will Oh, it became clear that that when you look at manufacturing and logistics, whether it's picking or it's it's assembling something, there's always a process in which material has to come from some obnoxious Lee faraway part of the facility to another part. And it became very clear that there was a high transportation cost. And there's been plenty of studies done for like labor analysis, looking at different different jobs, like the fulfillment job that I just described, or a manufacturing job. And typically walking time or transport represents upwards of 50% of the person's total time. And so that's a that's a very compelling statistics when statistic when you're looking at Should I make a mobile transport robot

It seems like to me that seems like almost a simple way of identifying low hanging fruit that can be solved with robotics things. I'm thinking of like nurses spend a large period of their time commuting from patient to patient, and then company, hello t robotics, where did an internship previously, they were saying how a huge they would have physical therapy or caretakers come to people's homes in Norway, and they spent like 60% of their time commuting from home to home. And these seems like it seemed like really good robotics applications to me if you can just put a robot in there and have it do the commuting wabbit do away with their commuting in some way. Okay, so you looked at this, and you saw this number? 660? He said, 50% of their time walking? Yeah. Which is crazy. So you started there? And

then we started with building mobile robots? Uh huh. And then and then it came down to now you have a mobile robot, you have a base now what do you have to put on it to make it productive for the application. And if you look at Fetch today, we offer three major platforms, and have different payload capacity. So 100 kilograms, 500 kilograms and 1500 kilograms. And then we put tops on it to basically attack the, the different problems in the warehouse. So we have, for our small and big robots, we have cart Connect, which allows us to move around carts, we have roller top accessory, which allows us to interface with conveyor, we have standard shelving, it's kind of boring, but you know, like the shelf with the touchscreen on it. Stuff like that. I mean, it really comes down to the accessories for the hardware. I mean, if you look at that, and like, the vision and and what we were trying to build from, from day one, you know, we started as a hardware company, and as all robotics companies go, you quickly become software companies. I mean, we spent like a hot five seconds as a hardware company. And I mean, you work in Open Robotics, and like the whole business, their software. Yeah. And so we spent a large part of our middle years as a software company in over the last couple of years, we've been transitioning more towards being a data company as well.

So Wow, that's the next level. Yeah. And so building

out our data platforms, building out our data capabilities, things like that. But that's still an ongoing process, because you know, you have to build a pretty large data like to do anything.

So I imagine a lot of like interest in cloud robotics at the moment with this is what you mean with the data? Yeah,

also. So if you look at it, if you if you look at it, the middle years, when we were becoming a software company, that's when we became we built our cloud robotics platform. So all of our robots are actually controlled from the cloud, they have their autonomous units. But if you want to tell them to like, go from point

A to decision making, the lower level control is done on the robot problem

was safety, navigation stuff is done on the robot. But the task level commands like yeah, go here, do this come from the cloud. And we did that, because it makes getting set up on site a lot easier. If you, if you look at like many of these facilities, they're kind of out in the boonies, they don't have good IT support, stuff like that. And if you come along with a very complex thing, like a robot, and you're like it Oh, by the way, where do we install the server? It kind of adds a lot. So we went off to the cloud. But one of the advantages of that is, we also save the robot data. And we're building capabilities on top of that data. So forklift tection, we can tell when forklifts are speeding, we can tell when, when they get too close to people, things like that. And so those are some of the capabilities that we're starting to build on top of our data platform. But the important thing is, is you need a lot of training data, you need a lot of real world data to do anything. And I think it's it's one of the reasons why when people talk about autonomous cars, I don't think they understand the scale and the scope of what we're talking about when they when they talk about like, Oh, yeah, it'll just machine learn it. It's like Yeah. Yeah. Or they they'll say like, the car has the AI if I if I hear AI one more time, right now

I know. It's true for so I want to go back and talk about the hardware company but just tying in this for the data company and talking about kind of this of the challenge for for autonomous cars. So you mentioned warehouses, semi structured environments is probably your data is a lot nicer than what you get. Would you just talk a bit about the difference between the data you're getting and the type of data needed for like an autonomous car? And how much how difficult that will be?

Probably. So I think I think the thing is, is is, I wouldn't say that the problem we're solving is any less difficult than that of autonomous cars. I would say they have unique idiosyncrasies,

So we're basically building a ton of indoor cars for the warehouse. The differences is we don't have weather, they have weather, but they typically their sensor package costs about $200,000. Our sensor package costs about 18 $100. So there's order of magnitude difference in the cost and capability of the sensors that we use.

So that becomes a massive difference your sensors, is it like a LIDAR and a few cameras? Or

Okay, is that 1800? And it's like, okay, that's one LIDAR and through?

Well, that's, the bigger robot has to LIDAR and eight cameras. But yeah, I mean, it's still not a $70,000 velodyne, you know, and so so there's, there's that there's one, the the kind of sensing tools that we have, are very different. And so although although they have somewhat of a harder problem to solve from weather and some other aspects, they have better budgets, and sensing tools and fidelity, to deal with some of those environments that we we don't. The other thing that's very unique about our environment, that's not true for cars is our environments are far more dynamic in some ways. While while roads can be very dynamic, and, and things like that, it's, yeah, they have lots of rules of the road. And pedestrians don't just end up in major highways on a regular basis. Whereas if you look at warehouses, there's like main highways, but people walk into them all the time. The other the other, the other thing that you have to understand is that our vehicles typically go around 1.5 to two meters per second. So we're talking like three to four miles an hour. But some of the vehicles we have to operate around like forklifts can go up to eight or nine miles an hour, or even faster. And so it's very, it's a very difficult problem when you are in a scenario in which vehicles around you are going much faster than you. And then there is the forklift problem.

cars don't have to deal with this notion of of a piece of equipment being around them, that has tines that are very thin, very

hard to detect the like probes at the top of

the fork of the tines before, and they can be at any height at any time. And

so they're super hard to sense probably to Yeah, they're

super hard to sense. They can be at any height. And there's a lot of things like we don't, cars don't have to deal with a lot of overhanging things. Roads are built such that nothing is going to be hanging in the middle of the road. It's not like that in manufacturing. So

what are you just taking a plane or whatever. So that makes it super difficult to do a lot with computer vision? Probably. Yeah. And that's challenging, because you have to recognize all these different circumstances.

Yeah. And so I'm not saying that that the autonomous car problem isn't hard. I'm saying it's very hard. But I think sometimes people tend to look at the problem we're looking at, it's like, well, it's got to be easier than autonomous cars. And it's like, well, it's it's not it. It's different. And, and there's things that constrain our problem solving, and things like that. And then the other thing that is is very true, is our industry is heavily regulated from a safety perspective. And autonomous cars are not yet. They're not heavily regulated yet. I mean, there's some regulation, but there's no, there's no you have to do this test to prove that your autonomous car is safe, etc, etc. Sorry, I actually need to stop and take this call really quick. Okay.

So we were talking about the differences between the data and that basically the problem of autonomous cars versus for manufacturing, logistics. Can you tell me a bit more about the data differences From these perspectives,

yeah, I will say that from a, you know, data perspective, you know, autonomous cars are using a lot more data, that's what makes us a little bit more open to, to being able to port and move all of our data to the cloud. And you do like algorithms and some of these things in the cloud, which a car may not have some of the luxury of doing. I mean, that's, I think one of the big things that's going to happen in the next decade or two is we realize what it's going to take to make the practical deployment of autonomous cars real is the sheer amount of data, you need to just store, the map of the local area drive there that you drive on a daily basis is, is massive, and what data networks are going to support just that capability, you know, everyone's going to have to have, you know, fiber to their house, practically, to be able to like overnight, download the local area updates and things like that. And then, if you want to go on a long trip, you know, say, from San Jose to Tahoe, I can only imagine what the strategies are going to be for, like a four hour drive. So for four and a half hour drive, the the data strategy is gonna have to be to do that the nice thing for warehousing and in our robots is it's a million square feet, that's about as big as the problem gets. And so we definitely, we we have, we're running into the multiple source problem. And I think right now, in the autonomous car space, they're still struggling with the single source problem of how do you get just data out of one car, and we're dealing with, okay, the data that we have to move isn't ginormous, but we have 3050 100 500 robots that we're, we're transmitting data from. And that's a lot of data. And so it's a pipe problem.

Yep. So with autonomous cars, they're just now thinking how to locally do it. So how can I make this one car do drive from one place to another autonomously, and you're going, we have these robots driving around this entire environment. They're all doing some sort of mapping of this environment. Like what maybe there's an obstacle that has occurred. And so then they all coordinate based on this, they go, Oh, that one path is blocked, I'm gonna have to go around, and then it reroutes

that, yes, as part of our coordination, we also use peer to peer Bluetooth. But I was just speaking more from the sheer amount of like, loading the data, and then post processing. If you if you will get today I think most companies Google way Mo, all those guys use, quote, unquote, sneakernet, which is putting a hard drive in a mailbox and sending it in the mail.

Oh, yes. That's crazy. The heart? What do you mean exactly? Where are they doing that? So like, well,

so you say, company has 30 autonomous cars that their field testing?

Oh, rather than actually putting it on the cloud and sending it and uploading it, they actually just physically mailed the device because of bandwidth constraints.

They take out the hard drives and mail them. That's crazy. sneakernet

Yeah, I heard I heard some companies are uploading more than 40 or 50 terabytes a week? Wow. Yeah. So a lot of data.

So you, so um, what it sounds like, to me, you have a good bit of faith in these machine learning approaches in the long game. So like 1020 years from now, for? Is that what you think? Or because I mean, data is not terribly important if you're going to do classical approaches for things, but you're finding in the warehouse setting that it is important to have a lot of data. Just Can you talk a little bit about this?

Yeah, I think that there are are some types of semantic behaviors that we care about, that we need models for. So we need to know what the thing is. And the problem is, is that there's a lot of variation in the world. And so if you tried to build a classically trained model for every object in your, you know, I guess traversable space, it would be very hard.

And by classically trained, would you talk about about what that means?

You have 1000 images and data examples of say a Volkswagen bug from 1976. And you put that all into an algorithm and it spits out a model and And it's really good that one detector is really good at processing Volkswagen bugs. And so if you were to look at like the most, I guess you might call it asinine way of approaching the problem is you could say, I'm going to build a specific model for every car on the road every year because they change year to year. And I'm then going to spin up parallel detectors for all of these things. And, and maybe want to know, the car model, maybe you don't, but maybe there is some material reason why you want to know this. Right now, I think they reduce it down to car, truck or other. But But say you want to do this this thing, then the problem is, is you're probably going to have some set of detectors that all say that they found the thing, right, so if you have 1000 detectors, and 15 of them, say I found the thing, then you need to put it through another set of detectors and voting system to decide which one of the things that says they found the thing is right,

or some approach to figure out exactly what is the one you listen to? Or what's most likely. Okay.

And so the the challenge with that is it takes massive amounts of data. And that's why people are very old,

like 1000 detectors or something like this, because it's that one problem. But there's a lot more and you have to make distinctions between that or is it the first model, but just a bunch of times? Because you could probably do that too, and then just have a confidence and wait to confidence?

Yeah, those those models aren't as good because so

you want it all in one model, basically, yeah. So it learns the discrimination between I think

the problem is, is if you look at it like a good a good way to think about, like, our way that we've been doing learning for a long time, is, for a very long time, we built models, and systems where we said, this is the thing you care about, this is the thing you care about, we kept showing it, the thing they cared about from all angles. But if you look at the way people learn, it's a little bit different. We we learn, this is the thing that you care about. But we also see on a regular basis 1000s of examples of this is not the thing you care about. Hmm, interesting. Yeah. And so but there's a problem of trying to prove the negative when the space of things that you're trying to prove the negative on is, is infinite, right. And, and so this is one of the problems with, like machine learning in general. And, and even if you want to show, this is the thing that you care about, say we wanted to show a phone, right, this is the thing you care about, and specifically this phone. So it's blue, it has some yellow buttons. You know, a person probably only has the tolerance to take 50 pictures 100 pictures of this phone, ad nauseum. Yeah. And one of the reasons that convolution neural nets are so interesting is is it It allows us to take the 100 images and make it into like 10,000 images. Because that's what it is, the purpose of it is it basically allows us to take the pieces of all these things, split them up, put them back together and use them to train to train systems. And that's why so it's a way of artificially creating more data. And it's found that that artificial data that we create results in high, high confidence confidently trained models, typically, now, okay, there's a lot of tuning and a lot of mumbo jumbo. Yeah, it's kind of a flasher, but definitely the same time. But a lot of those systems, again, are only trained on positive examples. I'm positive. And and this is this is in general, a bigger problem, theoretically, with all of robotics is because the negative examples that can be very bad, they can have very bad outcomes, like how do you teach a robot not to hit someone when hitting them could kill a person, right? So, so not it? Whether or not training with only positive reinforcement is good. Sometimes you can't produce negative reinforcement data, although I have seen work coming out more and more on how you build simulation models to create negative reinforcement data, for example. But

probably that got us all the way there, because you can overfit the simulator pretty heavily.

Yeah, yeah. And so that's the problem. As far as all sorts of problems overfitting weird edge cases, you know, things like

and so how this relates. So, if I understand quite correctly, how this relates in the autonomous car problem or in the way Your house problem, you are using some sort as you get all of this data, and you train different models on it. And you use that effectively for perception to help the robot understand the world and what's going on around it.

So I didn't know say that a person is a person. And so then the robot can exhibit different behaviors appropriate for that person. So, for example, the one thing that's the most mature in our navigation stack is robot, the robot detection. So we know when something is another robot. And that allows us to implement certain behaviors within the robot that make them more effective. They they will, one give priority to other robots based on what those robots are doing. They will, you know,

really cool. So it's going, Oh, it's a robot. But it's also going, Oh, I can understand from the context of what I can sense what this robot is doing. And that's a higher priority than me, kind of thing. That's interesting.

And until then, that extends into things like a forklift. One of the things we were talking about earlier, it's forklifts are very difficult. Well, one of the more difficult scenarios with a forklift is the tines can be all the way up in the air, they can be 15 feet up in the air well above the robot, and the robot can't see it. Of course, it can't, it's very high up near. Now, if you know what's a forklift, you can know not to drive into that area. Even if you can't see what's there. And so that's one of the examples in which, if you can know what things are, if you submit to glean know that this is a person

you can inform your policies of don't drive right in the spot where the tongs made come down, right, this kind of thing.

Yeah. And that, that is that is what the majority of applications for machine learning autonomous cars are also for is semantic understanding. This is a ramp, this is a person, this is a crosswalk, this is a stop sign.

Now there's one thing that I've seen, which is interesting to me, is ontologies and ontologies. It's like a map that shows how all of the different parts that you might see in an environment relate to each other. So if I was an autonomous car, it could be like, this is a road, this is a sidewalk, the sidewalk might have a person or a dog on it, or something like this. So then it'd be like road road has a sidewalk, sidewalk has a person, this kind of thing. It's a relationship between all these, do you? And this is kind of a manual way to specify all the different components. Yeah. Do you guys use ontologies? Or? So is it interesting?

If you look at that, and topologies in our environment, our district and this is one of the problems I was saying is like, there's a shelf, there's a ioway next to it, but people and everything else can be in a box can be in a trash can be in it. person can be in it. So it's a very flat ontology. This is a shelf. This is a floor. Yeah, so i think i think i agree that ontologies can be very, very helpful for creating kind of semantic maps and relationships. But they they don't always apply well in our environments.

Unfortunately, it's very complex. Yeah, yeah. Gotcha. Okay, going back a little bit. No, it's cool. This is all very interesting. Going back to being a hardware company. Yeah. Can you tell me a bit about when you guys started when you were a hardware company?

Oh, yeah. Yeah. I mean, we were hard. We were a hardware company for about five months.

In the seven years that have been so fun, yes. Crazy. Because

that's, that's when we didn't have any robots. So we had to make the robot. Yeah, we had to make the robots. The good thing is, is we had a really stellar team of co founders who, who all knew a lot about hardware. And it was really funny. I think one of the more kind of interesting experiences for me is is and I think for a lot of the the guys they started competing with is, you know, we while we were will abroad, we probably build five or six different types of robots together. And, you know, we learned a lot of lessons sometimes they didn't power on sometimes they have problems.

Yeah, yeah, all the things. But when we, when we started Fetch, we had so much experience like when we built the hardware, while we design the hardware, we sent it to his job. It came back from the shop we put together turned on it worked fine. And what was really funny is there were some employees who had been in other robotics companies who had been at other hardware companies. And like, we're like, what do you Like, this timeline is insane. All of this assumes that the hardware is going to turn on and work and not catch on fire. And we're like, Well, first, it will never catch on fire. We're not that bad. But second, second, we'll be fine. And it was, it was an interesting thing, because I think one of the things that I had developed is a somewhat like, what you might call robotics amnesia, where it kind of started forgetting that, that hardware could ever really be kind of shitty. And so and so like,

give us a nice position to be in when you really are like, you're like, Oh, I'm surprised that this didn't work. Yes, everything has just been working.

Yeah. And so it was, it was definitely an interesting juxtaposition. As, as the four co founders were kind of like, Oh, you know, we got this, this isn't going to be a problem. And then some of the new employees that we hired early on, were kinda like, this rodeo before. And never, it's never this move. So it was, it was cool. We were a hardware company for a very short amount of time early on me then eventually cycled back and we built the bigger robots, we became a hardware company briefly again. But the part of the company became a hardware company. It wasn't like, everyone there full time was just doing fucked.

Yeah. So then, how did you go about, like, the manufacturing process and everything for like, so once you worked out the designs, how do you go into actually making lots of the robots? Like I assume that that's quite a big thing?

It's actually relatively simple. Because it's not like we're bending metal, someone else does all of that stuff. Like they build the sheet metal. So you do the all the CAD designs, you do all the drawings, you send it out to a shop, they fabricate it. And really what you're doing at a robotics company, if you're doing assembly is you're doing assembly and test. And not to say that we're not manufacturing, but we're not bending metal. And so yep, the parts are putting together Yeah, we're putting them together, and then upgrading them, and then making sure that they turn on. And so it's, it's actually relatively easy to build or assemble, let's call it, robot hardware, especially in the in the case of the smaller robots, the larger robots get a lot more complicated because of the safety systems and things like that, that are required. But for smaller robots, almost all the parts in in smaller remotes are custom. And so we have a lot of control over it and comes in, we put together and worked.

Gotcha. That's interesting. I've heard some companies that I've talked to I've had a lot of difficulties, like finding suppliers and working with it. And I've heard things like the CEO will fly every rec to or some some person in the company will fly every three months to say, the Schengen area of China, and work with them on the manufacturing. Did you guys have to do these kinds of things? Or?

No, we predominantly made everything in the United States. And anything that wasn't made in the United States, we had us partners that handle all

that. Interesting. Can you speak a bit about that the manufacturing in the US because I'm not honestly terribly aware of many robotics. things being made in the US? Yeah, sure. new info to me.

Yeah. So if you if you look at it, if you look at the freight 100 or 100 kilogram platform, it has, it has about 300 total parts about 100 unique parts. And the the things that are made not in the United States are things like the laser scanner that's made in Germany by the, the skins, those are injection molded in in China, but they're designed in the United States, actually, the molds are designed in the United States. And then they're built in China, but and then our wheel motors are based off of a hub motor that you see in like scooters and stuff like that. And that was clever. Yeah. And we did the we do the engineering design in the United States with a company that we use their their Sai motors, they make custom motors, and then they own parts of factories in China. So in terms of the extent of the componentry, that's like fabricated or made not in the United States outside of the computer, let's call it it's all made in the United States. So we designed our own PCBs, though. prefabricated at a PCB fabrication place in actually California. All the sheet metal is bent in, in many, I mean, the bay area has a lot of manufacturing capabilities. And all of the sheet metal is I mean, one of the shemale vendors is literally blocked from our office. They they kind of scooted down the road. And so all of all that in the paint, you know, the any of the other things like powder coating, stuff like that, it's all done locally. So if you if you look at it, unfortunately, the the a US commerce defines defined in the USA by cost of goods. So because we have a laser scanner that's made in Germany, it we can't call the robot made in the USA, we can say design in the United States, but but if you look at it on a per

like because the laser scanner is such a large component of the whole robots cost. Yes. Yeah. Can you can you tell how much one of the robots costs? No. Okay.

Sorry. That's a closely guarded secret. Okay. Okay. Yeah. But if you look at it from a total part count, sense, the majority of the parts are made in the United States. And so most of the labor and things like that, and the jobs are being created in the United States. But there's this technical way of judging. You know, if

Melonee Wise's website

Fetch Robotics

Zebra Technologies

Fetch from startup to exit

In this episode, Audrow Nash speaks to Disha Sarawgi and Mohammed Kabir, who are Corvus Robotics’s Head of Special Projects, and Co-founder and CTO, respectively. Corvus makes a robotic drone for warehouse inventory management, which may greatly reduce inventory loss. They speak about flying their drone in warehouses, working with customers, safety, fleet management, how Corvus is funded, and they give advice to aspiring entrepreneurs.

This is a conversation with Disha Sarawgi and Mohammed Kabir. Disha and Kabir both work at Corvus Robotics. Disha is the head of Special Projects and an early employee at Corvus, while Kabir is a co founder and the CTO at Corvus. Corvus robotics is a Boston based startup that makes a drone to fly around warehouses and help with inventory management. Their drone, the Corvus one seems to promise to reduce inventory loss in warehouses, which currently happens to a staggering rate. As you'll find out in the interview, they speak about flying their drone in warehouses working with customers, safety, fleet management, how Corvus is funded, and they give advice to aspiring entrepreneurs. This is the Sense Think Act Podcast. I'm Audrow Nash. Thank you to our founding sponsor Open Robotics. And here is my conversation with Disha and Kabir. Kabir, would you introduce yourself?

Yeah, sure. So I'm Kabir. I'm the I'm the co founder and CTO of Corvis robotics.

Hi, I'm Disha. I work as head of Special Projects and General Projects. I lead basically all the product lines here at Corvus.

And she's also one of our early employees, one of the very first

We do have about 15 people across engineering and on engineering goes

to the shot when what number employee were you?

Yeah. She's the second employee. Right after you come here.

I see. So tell me about Corvus. Yeah, about what is the mission and everything.

Yeah, sure. So when we started out, sometime in 2017, or so we saw this a whole bunch of robotics companies, newly formed robotics companies, really kind of doing great things in the warehouse and industrial automation space. And generally, my background is that I come from a real robots. So I really love flying things and making drones. So when we started the company, we kind of were like looking for use cases, which we could automate kind of putting together this industrial automation space, and aerial robotics. And I think we found a pretty good niche and decided to pursue it, which is warehouse inventory management. That's kind of our singular, large focus throughout the company. So all our product lines are kind of trying to solve for that one specific thing, which is warehouse inventory. And that's where the idea for purpose one was born. What purpose one is, it's a drone, which which does, which helps warehouses do inventory. This is a huge market, warehouses lose millions and billions of dollars worth of goods from inside their own facilities every year. And this is the problem we set out to solve.

Why is that? Why are they losing millions and billions of dollars? Right, good, right.

So the way warehouses do inventory management today is whenever they send a person out with like a handheld barcode scanner, and then this person goes up and down the racks. So these are huge facilities that can be millions of square feet. So this person goes up and down the racks with this handheld barcode scanner, and basically scans barcodes on the on whatever stuff they have on the shelves. And then they basically write down or somehow tie back into their warehouse management system where this thing was. Now in variably, because there's a human involved in the loop. This introduces a lot of errors. And typically, I think the nationwide average for kind of the inventory accuracy across warehouses in the US, it's something about 40 to 60%, which is actually quite bad. So they lose half of their stuff just because they don't know where they put it. How could

that be that they lose 42 They only retain 46 to 60%. Yeah. So so what happens is like, why why are these red? Why such a large number of loss?

Right? So um, basically, the way inventory comes into warehouses, you basically unload from trucks and Then there is a person who is taking it from these truck bays and then putting it in their shelf somewhere. And this person is responsible for kind of doing an incremental scan. So they can. So basically, we can input into their warehouse management system, where this particular thing was booked. So it's basically all deltas. So it's, it's as things coming.

What do you mean, deltas? It's, it's every, it's the change effectively?

Yes, yes, it is the change. So every time there's a change, they update their warehouse management system, but invariably, because there isn't like a global view of the warehouse, per se. Errors build up over time.

Why? Why are they're building up, though? I mean, if I just put something there and leave it there, it should stay there. And it shouldn't matter that I only have Delta's what's going on? Are people stealing some of the packages that people move? Knowing or like, what's creating 60% of the law?

So I think it's a bunch of human factors at the end of the day, I wouldn't say that stealing is a big one. But I'm sure it is an actual problem. One of the main things we've seen is people just put it in the wrong place in the first place. So basically, you say that you're putting it in whatever aisle a slot B, but then you put it in aisle a slot, see the one next to it? And they just don't care about where they put it at the end of the

day? Because you mean the person who puts it there? Like, yeah, doesn't matter.

Yeah, they're processing like 1000s of orders a day. So it's really hard at even at the human level to kind of keep up that 100% accuracy.

I mean, it seems like getting one and two wrong if you're getting that level of like loss, which is kind of nuts. Yeah. Yeah.

It's a huge problem. It sounds ridiculous, but it is what it is. And that's that's what we set out to solve.

So then, what your so tell me a bit about how you're solving it with your Corvus one, drone.

Sure. So what premise one is, it's basically a full cycle autonomous solution. It's a drone, and this charging dock, which we go and install in facilities. So it sits at the end of warehouse racks. And it does scheduled flights on its own, and then goes out into the aisles, scans inventory as it's flying, it finds barcode scans them comes back home lands, and then we process this data and then send it to the customer, usually through some sort of digital data dump. And what this does is we're basically taking that person who was doing this with a barcode scanner, and we put the barcode scanner on the ground. And then we can do this repeatedly because well, it's a drone day don't get tired.

Yeah, so you're thinking of this. So if you were thinking of the diff, so it's kind of like in edge detection, and image or something like this. So you only get like, I don't know, you're only getting the difference, it's not a very clear way to see the whole state of the warehouse, right. And so now what you're saying is your drone flies around periodically. And so you can kind of get updates over time of where all the packages are. Exactly.

And if you do this frequently enough, we can basically get that global view of the warehouse basically every day, depending on the size of the facility. Okay.

Yeah, generally, like do I do that? Generally, inventory counts are taken by houses maybe once a year. But with the drawing, we're able to get the entire inventory counts multiple times. So God gives you a great idea of like historically, timestamps data, which you can go back to and refer to when like locate every item at the warehouse at any instant.

Gotcha. Very interesting. Now, how I'm thinking about this crazy loss. that puzzles me honestly, that it's so large, and I imagine like goods are twice as expensive as they would be if they didn't have this loss. Is that true? Would you think? Like, that's probably included in the cost of things? Yeah.

So the way the business model for warehouses work is they just write this loss off. So they account for that loss as they're moving stuff around.

still in their facility somewhere. It's they just don't know where.

Hmm. Interesting. So that with the perspective of fraud, flying The drone around, often, you can kind of get a better sense doesn't require the human to place the package as accurately or in the specific section so you can manage it, or at least in the one that they say it's placed in, because I would imagine, it would be the reason for the error. So then from that, you can kind of see if things are missing, or like if it was here, and now it's not you at least have a small window of time to investigate where it has gone to, or where, or something like that. Yeah,

it's not always just miss loading of items, but sometimes, because the humans in the loop, people also forget to just scan because they're taking care of 1000s of orders a day. And it's, it's easy if you're also on the ground level, but warehouse racks go up to like 20 feet, or 20 to 30 feet. And getting up a forklift scanning, and then coming down going to the next track at the same height is not an easy feat. So human error does play a big part of this as well.

I see. Yeah, that's a good point that they're so high that these packages are just stored with layer on layer on layer of packages. Is that the motivation for the drone?

Oh, badly. Yes. It's like humans can do inventory tracking at ground level, probably at the same rate. But as the height increases, the complexity increases. And that's where the drone comes in. And we get a

diversity increase, because you have to have some sort of ID or something to lift the person and you need a person Manning, or thing that tries and holds the basket. I'm picturing one of those like, things that they put up power lines, or go and work on the power lines with, I don't know how,

yeah, it's actually quite close. So they have forklifts or bucket trucks. And basically, there's a person in a harness clip to that. And then they go up in that, and they're running the barcode scanner. And it's, it's actually quite dangerous. Yeah, it is quite terrifying. Like, we've we've been in them just to get a feel for what it's like, do this annually. And it's more or less and were

lifted by Yes, yes. That's awesome. I like that. You guys do that? Yeah. How was it terrifying? It's it's, I'm just imagining, like dangling trying to scan packages. That sounds incredibly difficult. Yeah. Okay. So, being a drone really helps you get to those high packages very easy, it's almost free, because you just have to pilot it. Is there other other advantages to having a drone, rather than like a robot with like a real big telescoping pole, or something like this? Right. So

we've, when we started, we've actually looked into all of the options. Starting from like a lower complexity, like ground robot is obviously easier to build, right? Because it's not flying. But it turns out that when you get your poll 20 feet high, it becomes really hard to make the robot not tip over, just from the weight of the ball. And then if you want to kind of conquer that you need to make your robot like the you need the size of the base, you need to put in more batteries more weight. And it's like it's a pretty bad trade off at the end of the day, if you want to do anything of that kind.

Okay, yeah. And then so you basically with the drone, the Corvis. One, you have the drone flying and it has effectively a barcode scanner attached to it. And it's just pointing that at the packages. Is that right? Yes, exactly. And so now it's flying around from package to package just scanning all the different visible barcodes? Exactly. Hmm. Okay. So that's very interesting. How, how far into how many systems are deployed? And like, where are you guys at in terms of deploying yours to different warehouses your system to different warehouses?

Right. So we have a bunch of customers, we don't publish the exact number on systems deployed. But it is significant enough that we're getting very interesting fleet insights from them.

So order of magnitude is like 10 100. It's on the order of 100. Yes, that's really exciting. 100 robots out there. 100. Warehouses,

And when you say fleet, insights into fleet management, you're meaning multiple Have your drone systems in a single warehouse? And how to like urgently go about scanning them? Yep. Yeah. Gotcha. That's tell me a bit about that. The different things you're realizing at a fleet level of controlling your drones? Sure.

Let's see. So in general, the way we design our system, it's quite scalable. So if you want to scale to a larger facility, we can just add systems basically. And they talk to each other, the charging docks can talk to each other and kind of coordinate. And one of the things we're just starting to roll out is basically this orchestration engine, which has a global view of the entire warehouse, and then it can basically automatically tasked groans to go and scan certain parks either. Maybe it's a high traffic area, which we are kind of inferring from previous flights, that if there's a lot of change, and we can see that in the data, we can, we can cast more drones to kind of focus on that one area, and things like that.

That's really cool. Yeah, you should do anything to add

a bottle. Now, we also give kind of a high level control to the people at the warehouse as well. We have scheduling online, where, whereas managers can schedule, what time they want to draw to go and scan what location, so and then the charging of orchestration figures out which John was done there, and autonomically does all the missions and comes back and charges again.

So that scheduling that you mentioned, Keisha, it's like you try to avoid when people are in the area, or what kind of why would I want to schedule the flights of these drones.

So there are times in the warehouse where there's too much traffic in an area, and you'd rather dedicate the drawing to less traffic area, the friendships during the day, were like, probably night shifts have X number of people around, and you want to get a scan of whatever was moved during the day. And that works out better for people. So we give customers that level of control to decide what and how they want to use this product as well.

Gotcha. And could be are that what you mentioned for it, recognizing high traffic areas that have a lot of change and trying to do them? Or I was imagining? Just I didn't understand what about the fleet management was extremely interesting before because it was like, Oh, you just patrol certain regions. But then if a certain region has a lot more change, you want to make more passes of it, because you want to hire more temporal data. That's very interesting. Yeah. Gotcha. How so now that you have 100 robots out there or so like, order of magnitude? How is this affecting? Package loss?

Right, right. Right. So one of the things we've begun seeing is that we've actually been able to increase the inventory accuracy for warehouses significantly. It really depends on a per facility level, how much we can help just because facility dynamics, the workers, their, their existing practices. We don't have any specific numbers, which we want to share this point, just because it's extremely facility specific. And even like any vague ballpark number idea will be wrong. So

you can have a specific instance of the thing. Yeah, but I mean, maybe you have like an average, or something or like a median level of improvement. Right. So

like, double it, because the huge, like 60% loss is crazy to me. And if you keep an additional 30%, that would be amazing. I just walk

across, right, yeah, that is actually quite close to what we almost guarantee that if your facility is using Corvis drones 95% Just kind of the baseline you can expect to achieve within months. Yeah, and this involves working really closely with every customer because again, as I said, every facility is different. Everyone uses different barcodes. There's there's a lot of moving pieces there.

See? So let's see, can you tell me a bit about working with these customers?

and maybe the chef you have a perspective On this, I want to try to get us all talking about the same amount.

Sure, so, okay, since the team is pretty small right now, like, we are the ones who actually go to the deployment as well. And it's good to have engineers on site as well. Because if we build products that people cannot use in production, that it makes no sense. So going on site interacting with customers, we get a sense of what they want, and we build accordingly. So like onside, Devon been instances where customers have told us stories, like they spent hours on a Sunday night trying to locate one package, which was lost. And that's why like, and if there was something on Sunday, that sets them back for the rest of the week, because their week starts on Sunday. And that's where our drones have been able to get them a good amount of improvement like that is we have all the data stored. And we can tell them exactly where, what, where what package was seen. And it has them significantly improved their buffers and dedicated with others still actually getting packages to avoid. So

that's cool. So does this mean that you have to tie the data that you guys collect into their existing package, localization infrastructure, so like figuring out where all the packages are, and the map of that I imagine it's like a grid or something like a two 3d grid where they say it's in here, or at this location on this shelf, and you have to like, locate the package, and then map it to how they actually represent the location of the package. Is that true?

Oh, yes. So every facility has their own warehouse management system, which is, like, where every location is tagged as, depending on the rack. And it's not.

So it's like a sticker is on the so say, we have three shelves or something high, that each shelf of those will have a sticker or something that says this is shelf one, this is shelf two, this is shelf three. Yeah. And so that's how you localize them.

Yeah. So it also is like, maybe a number as to their eyes than their faces, like the two faces on each eye. So then, the next level is the face. And then they go by the level of the rag. And then each level also has number of slots. So the a hierarchical naming of each pallet position in the warehouse, which to keep track of and the warehouse management system. And we use the same mapping, and we work with customers to integrate the data we collect into their management systems as well.

So it's a big key value problem effectively with the key being the label of the shelf, and then the value being whatever packages are on it kind of thing. That's when I guess so if that's the case, it might not be I basically, was it challenging to kind of work with all of the different warehouse management setups that all of the different warehouses have? Or was there a challenge there?

Sure, yeah. This is just a huge pain generally integrating with the consuming, but the number of different systems out in the wild. We have a couple of ways of integrating life from the very simplest version, which we can basically support for everyone is we just give them a digital data dump. It's something like a CSV or something of that kind, which they can. And everyone says, yes, yes, yes, men is in commerce,

it can be processed like an Excel sheet. Exactly. It's like it's essentially an Excel sheet. Yeah. And then.

And then we can also kind of go into slightly more advanced integrations where we have some API which they can pull for data, or we can go the other way, and we can push data to their API's. This is again, very facility specific, very customer specific, and it's essentially a custom integration for every new facility.

We want to give as much value to the customers as we can, so we try to integrate with whatever their existing processes are and make them make it as easy as we can.

See, so now, getting into your drone a little bit more. How I like just how large is it? It has the autonomous charging dock just telling you bit about a bit more about it. What sensors it has, what it how large what it looks like, everything should start.

And this is a cordless one we're talking about. Correct? Great. Yeah. Yeah.

So it's basically two components. We have the landing pad, which is the charging dock for us. And the drone. The charging station is about like, one meter or 2.7 meters, maybe. And the drawn is probably, like 3.4. Yeah, five

Okay, so for now, just kind of, yes, landing station is four feet or five feet, like a square kind of thing. And I saw it in a video. And I don't know if it's changed, but it's like a fiducial marker, like a QR tag. And then you have the charging dock in the middle of it. So I'm assuming that you're counting that in the five foot width of the station? Yes, yeah. Okay. Yeah, go ahead.

So these are the mountains at a height where forklift can go through underneath it, and we're not causing any obstruction to general warehouse traffic.

How did you What was the lesson learned for why you are mounting on the Bub forklifts? I assume that was like, there's some story there.

So yeah, like, we genuinely love the landing pad and the drawn to like at the end of a warehouse racks. And there is also a high traffic area for anybody who wants to go into the aisle or come out of the aisle. And we do not want to cause any obstruction for them to take like a large radius around the racks. Because some of them are also narrow passages, which is difficult to get through. So we generally mount them at a height where the drone can also operate without obstructing the traffic underneath it.

Did like a get run over or where people just frustrated that it was like a long path around or

so we've had some close calls before before we kind of realized that we should move it up even higher. So one of the things about forklifts is that they kind of have two modes of operation. So one is when they're just traveling along the ground, and therefore service fully at ground level. And there's another thing where if you have to take packages off from a higher act, they basically extend up and then the whole thing goes up in the forks also go up with it. Now what happens is, even though forklift drivers instructed to not move around at full extension,

sometimes even meaning the fork all the way up the exact right.

Yes, that is correct. So they're never supposed to actually be moving around about the whole thing extended in the forks all the way up, because as you can imagine, that's pretty dangerous. stuff. Exactly. And but it's actually it does happen sometimes. And then well, as you can imagine, they could hit our landing pad or they could hit other stuff. Um, that is why we tried to position it at such a place which minimizes any

of this kind. So it's kind of you have like a shelf side and then a few feet up, you'll have your actual landing pad, which is five foot by five foot, approximately one point So would you say seven meter

yeah. Okay, so some square that suspended up there, like a little bird's nest. Yeah. And that that has the charging station there. You're Corvus one drone. Does it have like guards around the propellers? Or like how is it set up? So it's a quadrotor. It has four rotors. And each of those rotors has a propeller spinning. That's how it lifts up. Do the propellers have guards around them? So that they don't like hit anything? Or is it not a problem?

That actually, there's a story behind that too. So probably the story around Yes, indeed. Indeed. So when we started we actually did used to have propeller guards around the propellers. But after like doing 1000s of flights, one of the things we realized was that If there is any kind of collision with anything, the propeller guards don't really help in the first place, they actually end up. So the reason they don't help is whatever material you build your propeller guards out of, they typically have some amount of flex to them. And what can happen is your propeller guard can kind of bend and hit your propellers instead. And that's kind of a no win situation for everyone. Because then the drone, there's like a catastrophic failure.

Just falls down. Yeah, pretty much smashes on the warehouse floor. Okay, pretty much.

So the way we started looking at the problem was how can we prevent any kind of a situation where there could be a collision in the first place. And kind of what we've converged through is our economy stack maintains this safety envelope around the vehicle. And we basically got rid of the the hardware prop guards, and the software makes sure that the drone is just never close enough to get into any situation of that kind.

Gotcha. Okay, so you found basically, you could just stay far enough away from everything so that you don't need pretty much any sort of propeller guards and the propeller guards weren't even that good, because they flex and they would just go into the propeller, and then it would crash. So they'd be like, for protecting fingers or something where I would stop it. And I would hit the propeller guard, but then the propeller would hit the propeller guard, and then it would fall. So my fingers would be okay, but the drone would still suffer the same color. I don't know falling from air. Why not just use really rigid? I guess. So you've eliminated the need for it. I just wonder about like, really rigid propeller. But maybe they're heavier, they reduce your flight time or exactly. Yeah,

that's, that's pretty much it. If you're going to design with propeller guards in mind, you need to kind of add a lot of extra weight to the vehicle, which might be acceptable for for certain companies, or certain people working on drones. But for us, as since we kind of designed around the problem, it just was never a thing.

Yeah. And then the shot. Can you speak a little bit to the are you doing something with the cable?

Ah, yes. Yeah, these lame to have the computer turn off. Can you speak a little bit to like safety concerns of customers? Because I guess I'm getting at what I'm getting at with the propeller guards. Is I wonder if people in the factory would want something like that. But just general safety things, safety concerns of the customer? This kind of thing?

Oh, yeah, definitely. So safety is something we take very seriously because it's drawn with models running at like 20,000 rpm, and we do not want to collide into anything or hurt anyone or not. So we have a bunch of software and hardware fail safes for ensuring safety. We have bunch of audio visual cues that we be provide while taking off by landing wire in case of

2 posts with the warehouse tag