In this episode, Audrow Nash speaks to Christian Fritz, CEO and founder of Transitive Robotics. Transitive Robotics makes software for building full stack robotics applications. In this conversation, they talk about how Transitive Robotic's software works, their business model, sandboxing for security, creating a marketplace for robotics applications, and web tools, in general.

Episode introduction

Introducing Christian and Transitive Robotics

Fleet management + networking challenges

Problems in fleet management

Business model

Using MQQT for message passing

Developing your own capabilities

Testing capabilities

Sending data: authentication and authorization

Sandboxing for safety

Discussing web technologies

Open source business models

Scaling, finding market fit, and taking investment

Plans in starting a marketplace

Plans to grow the team + remote work

Web and robotics

Web technologies on the backend

Future of robotics

This is a conversation with Christian Fritz, who is the CEO and founder of Transitive Robotics. Transitive Robotics makes software for building full stack robotics applications, meaning that it aims to make connecting your robot to a user interface easier. Some reasons you might want to do this are to remotely monitor the health of robot fleets. See what a robots camera sees, or to tele operate a robot? Transitive Robotics does this while solving the hard networking problem of robots potentially coming into and out of connectivity. I enjoyed talking to Christian. His perspective in forming Transitive Robotics seems to have been very pragmatic. And in my opinion, Transitive Robotics is building a tool that is very needed by the robotics community. It was also interesting to hear his thoughts on growing a robotics app marketplace, which may be in Transitive's future. This is the Sense Think Act Podcast. I'm Audrow Nash. Thank you to our founding sponsor, Open Robotics. And now here's my conversation with Christian. Christian, would you introduce yourself?

Hi. I'm Christian Fritz. I'm the founder of Transitive Robotics. People may know a little bit from my career, where I'm coming from, I used to be the head of software at Savioke. And for those who don't know, Savio, said hope makes a really cute delivery robot for hotels and hospitals. And it was, it was a really fun ride. For me. Being there, I started when there were about five robots in the field. When I left there were over 100. And it was a really an eye opening experience. For me, my background is in robotics, I've done Robocop back in 2001, I've seen all sorts of versions of robotics, including before ROS, which was bearish. And then at Savioke, I learned about not just the robotics piece, but the full stack that went on top of that. So that's really kind of my background, I was leading the application development team there, which basically, I always described it to new candidates or employees as as follows. The application development team gets a robot that knows how to stay localized and not bump into anything, and turns it into value to the company. So what that meant was all of the HDRI was done by our team, anything that involves just a sense of ambition, right, like running a delivery, the notion of delivery, writing elevators, calling phones, notifying users, all of that was done by a team. And it was, it was a really interesting experience. Because I found out that while ROS is really great, there's, there's so much more that you need in order to deploy a successful robotics fleet in the real world. And so that's, that's how I came to start trying to

know, what exactly are you doing at Transitive? Like, how are you solving this deployment problem and the problems that you kind of saw from your work, it's.

So it's heavy, we spend a lot of time building, you know, those those full stack applications, data synchronization between robots and the cloud at the front end, visualizing elements of the robot, such as maps at laser scans, and sending data back and forth and doing all of that DevOps. And it was really interesting for me to find out that other people at other companies like fetch robotics and places that we spoke with, were basically building the exact same thing, right. So we, we felt that, that that's not a good idea. And granted, there are some solutions out there that help try to offer you the kind of a one stop shop for fleet management and DevOps. And some of them are actually done by friends of ours. And they approached us and said, We have to, to see whether we would want to use them. But the problem that we had with them was that they all are sinking, like standalone platforms. So really, they give you the choice of do I build my own fleet management? Or do I outsource that. And the issue that we had with that was that when you outsource that your hands are tied, if the thing is not perfect, you cannot extend? And if there's something that you need, you still have to build it separately. And then you have, well, first of all, some type of work. But also you're duplicating your data, landscape, right? Some data will live in the one side or the other in

your domain in so software you're using for fleet management, and then whatever features you build custom yourself kind of outside of that. For your own. Yeah,

that's exactly right. Yeah, exactly. Right. So you will have separate users on different accounts. You'll have different roles. You'll have complex some data about the robot. Yeah. And so what struck us was that what people really want is the ability to just take individual complaints and plug them into your existing web application. A good example of that is low latency video streaming, almost everybody wants to see what's the ability to see what the robot is currently seeing. And it's a complicated application or capability to implement. Because there's some really great technology like the one that we're using here right now, which is why Partisi, which is running in the background, it's very complex. And a lot of robotics companies don't want to get in the weeds don't want to become experts, they don't want to get into the weeds. So I tried to give were basically developing these things as individual components. That's the one thing that's different about us from from the existing offerings out there. And the other one is that we believe in open source. So we actually following an open core model, with a platform that we're developing will be open source, and anybody can use it, and we hope people will, and they will build cool things on top of that. And this is really kind of a philosophical thing for us. We think that ROS has done such a good job at uniting the robotics community by giving us a common platform that we can build packages on. And so we can share, right? So if you if you build a really awesome navigation functionality, and thinking of your last you can you can share that with the world and the world will be a better flex, right. And I think ROS has done such a tremendous job at establishing that framework or platform on based on which we can share it. And we would really love to do the same now for the for the full stack. If it's open if it's open source, and it's easy to implement new capabilities that we really think that people will start contributing back. And we can finally stop all reinventing the wheel and building the same.

Yeah, for everyone who has a fleet of robots that needs to manage them in some way. So now one thing I'd like to do just kind of going more into detail, what does it mean to do fleet management? And what are some of the challenges with fleet management?

Well, so I'd start perhaps a little bit lower and start with why why sleep management or anything to do with robotics is not just another react app, or something else where you can use common tools out there. And they don't just solve the problem. The big one is really that these robots as much as they seem to behave like servers are not really servers because they run around, they lose network connectivity, they gather data faster than anybody can process them. And so you have some unique requirements in creating a uniform picture of your fleet in the front end. And this is not very unlike what some of the cloud providers offer with their IoT

when we say that you mean like a web browser or some other web browser?

Yeah, it's um, yeah, I guess I should, I should say that I'm asking I'm a no, it's a it's a good point. Because in robotics, and especially in bras, a lot of Qt applications are still being fronted. We think that if you're dealing with a broad set of users who are distributed, have different backgrounds and so forth, using web technologies, really the way to come. Because everybody knows how to use them. Obviously, most of the software being developed these days, runs in some on some form of web technology. And it's just a very lively and flourishing ecosystem. So using web tools, I think it's the best approach for building these front ends. And so at that point, you have to create a link between your your robot gathering data and user sitting in front of a browser.

Okay, so for fleet management, webs, web related tools are really good. Because they let you it's like it's better. There's really good tooling around there. There's a good ecosystem, this kind of thing, and I guess, getting into kind of the fleet. So and then actually, you were saying that a server, like you can't just treat it like a normal sir, a robot like a normal server. And so because of that, you have Transitive robotics, you're trying to make it so that you can, in a sense, treat a robot like a server while addressing some of those challenges.

Yeah, that's exactly right. I think it kind of comes down to the core abilities that the platform gives you as a developer. And one of them. I think that's perhaps the most important one is the data model that it gives you. And what we're doing there is actually not very different from what some of the cloud providers offer in the IoT solutions at all on AWS, they call them IoT shadows. In a jury, they call them desired and reported state of a robot, the idea is always the same that you have some form of data model that tells the robot this is what I want you to be like, whatever that is configuration deployment. So as a robot to get into, basically. So it's, it's saying I want to think of it as you think of it as a state. Yeah. At the state can have all sorts of aspects to it, right? Like which software to run, what users to have, what configuration values to use, which cloud to connect to, and, and then the problem is your robot. Sometimes it's not just not online, right. So you have to make sure that you reliably synchronize that data to the robot, and also get the get the data back. On the coming back side, you have that same issue that you may want to monitor your robot while it is traversing through some areas where it has very poor connectivity. But you don't want the front end experience to be chopped up, right? You want the user to have a smooth experience and still see the robot, even if right now you can't get signal from it. And that's, that's why it is not a server, you can't just run a web server on it and have to reuse it connect to it directly.

Is that is that kind of the biggest challenge in this? Would you say where it's, the robot comes in and out of connectivity? And it makes it so that you can't make a lot of the assumptions that people would make about, like web servers? In general?

I think that's a good way of putting it. Yeah. So

what do you what do you do in the case? How do you how do you make a smooth experience when a robot is popping in and out of connectivity?

I think really, the key here is an intelligent data synchronization where the robot essentially updates the cloud, whenever it has connectivity. And then the cloud serves the serve the users front end, right, so the user always sees data, sometimes data will be a little bit delayed, because the robot hasn't gotten an update through to the cloud. But the cloud will be able to tell, hey, I haven't gotten a heartbeat from this robot in 10 seconds. So what you're seeing maybe maybe a little bit old, but from the user's perspective, pages, load fine. Everything will be there. And you will always see the latest data that you could possibly get independent of when you show data, some people think you could just run a web server on the robot itself, and then connect and then the problem is sometimes you try to build a patient that just doesn't know because your servers down or not connect you.

You mean literally host the website from the robot.

That's, that's a lead. So let's take an alternative. Some people are this into the robot harvest or like a non ROS users is a visualization tool for ROS. And it's it's an awesome tool. But it's, it's not really meant to be used remotely for for several reasons. One, it sends a humongous amount of data because it actually thinks that the connection is very good and fast and sending duplicate data is not a problem. And it also just sends a ton of data. So there's not a lot of filtering typically going on. Which means that remote experience with harvest can be very choppy, and the robot goes in and out of connectivity sometimes. So if you lose that you won't have a lot of disruptions, you're losing interest. So I think some kind of caching of the data in the cloud. That's one, reducing the amount of data that you send from the robot to the cloud is another Oh,

yeah, that's an important point, reducing it so that you don't actually spend that many resources on kind of, I don't know, perhaps you can get by with less. And then you have less requirements of your connectivity to your network.

Yep, that's exactly right. A good example, here is health monitoring. A lot of your your listeners may be familiar with how ROS solves the diagnostics problem where there is a nice hierarchy of message definitions that allow us to share diagnostic information about your robot such as set of values, battery stage, and so forth. But the way that ROS publishes status, here's an update, here's an update, here's an update, even if nothing changes, right. And that's fine with ROS, that's not ROS is designed on the robot. And that's, that's good. It's actually a good approach on the robot, but not for the cloud, because you don't want to repeat messages if they're literally the same as the previous one. So even just duplicating sequences of messages will already save you a ton of data.

Yes. So how I'm understanding what it sounds like you're describing. So if you want some sort of way of monitoring your robot, your robot has information it's driving around, it may move in and out of networks. What you're doing is you're placing a cloud component so some computer somewhere else that's talking to the robot. This is basically it's keeping up the whole time. And it's displaying whatever the most relevant data is on there. And then I suppose, well, actually, and then you have the user who looks at whatever the cloud is doing. So the cloud has its own host, its own website is hosting. And the user can look at that and get to see the state of the robot. And then I assume on the robot, you have something that's saying, this is necessary information to send up to the cloud. So if it if it's overheated message that's being sent, you don't bother sending it again, because of limited bandwidth effectively, to try to send data to the cloud. Like we only want to prioritize and send the most important updates, probably, to the cloud. Yeah,

that's exactly right. That's exactly right. The way that these decisions are made is that we believe in vertically integrated capabilities. So I've described so far is this very fundamental data synchronization layer, but then what you build on top of that, we think in terms of apps, right, so let's take a remote Tailor of capability. A lot of people need that on the robot. So that involves sending video from the robot to the to the front end, displaying the video on the front end, and sending back some control signals from the from the joystick. The Transitive is structured is that we have this concept, very strong concept of packages, similar to ROS. But what's special about Transitive is that inside a package, you don't just have components that run on the robot, or the code or the front end, you have all of them together. And what's great about that, is that you can you have an end to end exchange. And if you want to change the way that things are visualized on the front end, if you need, say, some new data that you didn't have in the previous version, it's fine, because you're going to publish another version of the code that runs on the robot and push us to the cloud as well. So this gives you some really nice. We think of this as cross device dependency management. Yeah, you don't have to worry about oh, well, we just upgraded our front end. So we now also need to upgrade our backend. It's all it's all taken care of.

Yeah. Yeah. Interesting. Okay. So going back. So some of the challenges with fleet management, we have, basically the connectivity issue where it's going in and out of the network, we have, perhaps, the, the so you have the robot, it's sending data back and forth. The robot may or may not be connected, you have health monitoring, and things like this checking the status of the system, and you have updating it, which you've kind of just mentioned, you can ship things as components, and it's updated universally. Are those kind of the other additional challenges to fleet management, and network updates. And the second, but yes, those are there additional ones that are kind of critical and fleet management, health monitoring.

Actually, if you don't mind, I'm going to share a kind of famous Greenbrier course. So this is a list that we've put together off things that we needed at one point or the other, in the past. And it's grouped by categories of kind of apps or capabilities that you might need to run a fleet successfully. So there's the category of DevOps and fleet management software, super modular uploads a video is Iraq, prospect view and health monitoring, and it goes observability is a big one. alerting, right. A lot of people like to integrate with pager duty and so forth. But then there's also things like application logic, right Task Entry to a queuing of tasks, user management, account management, and a lot of those things are really not that much fun. Yeah, so that and they're not really they're not differentiating to a robotics company, right? So we were we validated this with a lot of CEOs and CTOs of robotic startups. And they always said the same thing. There are certain things that we would like to focus on. Often, in the ml or computer vision space, we were talking with an apple picking robotics company, you can probably guess what that was. And they said, well, our our secret sauce is in our ability to identify grasping points from the computer vision, we would never want to outsource that. And that's where we want to differentiate ourselves. But there are so many things on your list like CI CD, like anomaly detection, that's not that's not important to us. We would love to outsource that. And that's how we think about Transitive a, a both a platform to develop those kinds of capabilities and then also a company that will that will offer them

yes to you So where are you guys currently in this because the list of things, there must be like 40 items or some large number of things that all seem really important to shipping a robot application. Do you have all these implemented? Or is it like a set of them? Or we're yet?

Yeah, no, not at all. This is the aspirational slide so to say. And it's also meant to, perhaps give people ideas of what they could develop and contribute. This is really the motivation to do all of that. And the way we're going about it is we want to develop the platform First, open sources, and then start building the ecosystem to develop all of them. Right now, if you go to our website, there's three capabilities that you can try out right away. And they are some of the ones that I've talked about. So far. The low latency video streaming, remote telehealth and, and health monitoring. Everything else?

So TBD? Yeah. So it's, so you have this list, you kind of gathered this list, and I assume maybe it was somewhat iterative in the process. But I assume this really informed the data model that you're using, and the development of the system so that you could fit all these cases under your framework.

So that's exactly how we're thinking about it. And

that's really cool. So the, I'm kind of imagining this is, like, I'm mapping it to like web concepts, it's eventually going to be like the hosting service. For robotics applications, probably you have your specific logic running on your robot. If you're using the ROS ecosystem, you get a lot of things for free, that might be good enough for your applications, like navigation or move it for motion planning of arms, these kinds of things, and then you have this to make it so you can quickly build your web interface to monitor your application for the people that should make sure the robots are running or I don't know, whatever it is that's engaging and engaging with your robots, or whoever it is, yes, whatever they're doing.

We're thinking about this in two different ways. One is to get people started more quickly. So they don't need to build these things. And then sometimes our investors ask, well, aren't you going to lose those people, once they grow to a certain scale? And they will, they will not care? Because they are now they now have 200 employees, and they will build it themselves? And my answer to that is they still not want to. And the reason is, like my counter example is slack. Yeah. Why do people use Slack? It's not impossible to do. In fact, it's pretty trivial. However, are you really going to bake it quite as well as slack? And that's all I do. Where, exactly, it's, it's the value of specialization, right. And in the end, you will use the best solution out there, startups in especially in robotics are actually not as price sensitive as you may think. And that's because the bulk cost of robots all of the operations is not that cheap. So if they have to shell out $20 a month for a capability, there's really well, who cares, they're spending more on the LTD Bill per robot. Right. So I actually think it this is something that I've felt with ROS, too, is that I think ROS would actually benefit from more entities having commercial a business plan behind them and commercial backing, so that the contributions that are being paid by those companies to ROS have have a longer a longer outlook, right? There's, there's some real money behind it, we can be certain that they will be maintained. And there's somebody on the hook for making sure that they that they do that they work well. Yeah, there was this really, I think I wrestled last year somebody presented this paper, I think it was called something like it takes a village, I forget the details, you remember that? I don't think I thought it was really interesting, because they did a study of ROS packages, and how many which are used, like how many how many people use them, and how well maintained they are. And it was really striking that there are some like the core modules that are they're all being used by almost everybody. But then the vast majority of packages, and of course, there's a ton of packages, they are not maintained very well. So that puts users in a difficult spot. And in deciding of can I really rely on this package? Or do I have to fork it if I want to if I want to use it. And I think that the JavaScript community does a very good job at that with their with the NPM package manager where with every package that you can find on their site, they will show you how about how well does it maintained me to how many commits Have there been made lately? How popular is it? How many people use it? And I think some other metric and I think I think that's very helpful quality, some sort of They bought the holiday

trick. I don't, I don't know how to get that, but it's quite cool.

I think it's very helpful. And when I choose libraries to use, that's definitely something that I look at. And Oh, for sure, knowing that there's others who use it is sometimes perhaps the most important factor. Because I know that if I run into issues, I will not be alone, there will be others who will be who are also invested

in it. And you can feel a little more secure in that other people might pick up some of the slack, you can pick up some of the slack, but it's not entirely on you to maintain if the developer just goes silent, and maintaining it. Yep. Gotcha.

And I think if we can stand something like that up here, we're obviously taking the perspective of the full stack. But I think some of those ideas translate into into other ROS packages as well. And one thing that we haven't touched on yet is you could have a rust package be part of one of these apps, right? You can, the way that we actually designed our deployment mechanism. These packages can contain whatever you want. Some people cringe when they hear whatever you want, because that sounds dangerous. And we can talk about that in a second. Yeah. Yeah, so we we've designed this from the bottom up as a framework that

what are the components? So in any one of these packages? Or in any of these components? It has the robot arm? Or what are the parts of it that are involved? And how do they talk? And then sandboxing sounds very interesting to talk about, too.

Okay, so let's, let's dive a little deeper up. Before we before we get into that, I think I should mention that we're using TGT as the underlying communication mechanism for just about everything. I think that will help our conversation here.

I don't know much about that. I don't know anything about that.

Okay, so MQ TT is a Pub Sub mechanism, where you have topics that are hierarchical, separated by slashes, just like a path in Unix,

very similar to ROS topics, it seems like from the description so far, in a sense that it's good topics, and then it's hierarchical, like namespacing, and this kind of thing.

That's right, I think the main difference rough topics is that they don't have this notion of a message definition. Any message in the end is just a buffer, or a string, like a character buffer. And then on the receiving end, where you will interpret

And other than that, I mean, it's a very well developed technology, it's actually underneath all of the IoT, cloud offerings like AWS, GCP, and Azure, all built on MQ TT. So it's sort of battle tested. And it's meant for IoT devices. So for low resource usage, very high throughput, communication. And unlike ROS, it was really meant to be cross device, like the typical IoT application, you have a large fleet of small devices that all send data to the cloud, and from there potentially onwards. So this seems like a very good technology to build this on. And then our packages, they are exactly like you suggested, they actually have three components, there's the code that will get installed on the robot, there's the code that will run in the cloud in a Docker container. And then the third piece is, are any sort of front end components. And there we are using this new technology called Web Components, it's not a very good name, because it's not very unique. But web components is a thing. And it's very nice, because it's front end framework agnostic, you can use it in React or Angular, or even without any of that. So it really just is like a custom tag in HTML. So you have lots you've loaded that component to that component, you can just type, you know, pointy brackets, the name of your component, and it will appear with where you want it to be. That's what we use for the front end. And then underneath, they all communicate through through MQ TT, in a in a namespace that is not just specific to the capability, but actually even to the version of the capability. And this is what allows us to create this cross device dependency management. This is this was a very big pain point that that I've faced at one point in my career where if you update the cloud, you may be forced to also update all of your robots, depending on what you've changed on the cloud. Now, guess what? Your robots if you have more than 100 robots, at no point in time, a lot of your robots be online at the same time. So when do you make the switch? Right? And so there's different ways to deal with it. Some people

get both, and you just have them send their version messages. That's how we are solving it except super cool. And then you kind of delete it once they're all on board with the most recent version.

Nobody has been talking to me be on that version for a month. So just got to shut off that Docker container.

That's super cool. Okay, so you use MQ TT, this gives you that really nice version messaging behavior that makes updates really cool. And I really like the idea of kind of it trickles in as the robots become available. Okay.

So that that tells you how the three components in the, say the three again, just explicitly, the robot, yep, the cloud and the UI. Okay.

And so for these components, one thing, so are they all? You get them to talk? Like, so if I'm writing one of these components? How does it look? Like how can I test all of these? What do you call with the components within the component? Or how do you refer to the the web robots and cloud parts of component?

We haven't, we should get that terminology. We don't have a good name for them yet.

Gotcha. That component makes me think React components, which is kind of it's a, and then that makes me think just the web part of what your thing is. But

we could call them modules. And well, actually, on right now we're referring to this whole thing as a capability capability. Okay. And then the three underneath are packages or modules. But okay.

Gotcha. Okay. I like the word capability better for discussing it just so it's not as overloaded in my head. So how do you write a module? And how do you write a capability and test all these modules? How do they work from a developer's point of view?

Some of this is still being developed right now, we have not yet made the open source release. And one thing that we want to get done well, is the SDK, which is kind of what you're asking about, right? So there will be tools that will allow you to create new capabilities, quickly in the sense that they will, I'm taking a lot of inspiration from again, how things like reactive office where you have like a CLI tool, that's where you might say, Transitive, create new capability, whenever it will, it will set up the directory structure and you know, the package json and the web, web pack, whatever you need, in order to compile all of those. Yeah, we do use I guess, I might have to say that we do use NPM exhaustedly, and for those of you who don't know, NPM is the node package manager. It's a very nice tool. A lot of people think like, it's only for node. But it turns out, you can put anything you want it to that package, it has a lot of advantages over say Debian packages. That was probably too much detail to go into, but I'll be

talking about it just a bit later in the interview. Yeah, good.

So once you have these NPM packages, there's, you know, scripts that you can have for NPM publish to publish those capab those those packages have your capability into a into a private registry or an Artifactory store. And from there, your robots will be able to pull them and and run them the connections that you were asking about for communication, where they will be set up through the further data synchronization. So from the from the developer's point of view, all you're manipulating is actually one big JSON object. And within that JSON object, you can decide which sub paths you want to subscribe to which ones you want to publish. But you will always be dealing with a JSON object as if it was just on your machine, you know,

the thing that's being passed between the modules in the capability correct. So you're sending that JSON object around and that JSON object. So that's just a data structure for viewing everything. JavaScript Object Notation, or whatever it stands for.

That's what it stands for. And data data data synchronization that I mentioned previously, basically just takes care of making sure that what the robot sees from that tracing is exactly the same as what what the capability in the front end sees, and you don't have to worry about it, you just manipulate it on the robot. And you will can see it like that on the on the front end. For those of your listeners who know how, you know, reactive web programming works is this is a reactive data source. So if something changes here, your React component will update. dynamically, you don't have to pull you don't have to worry about it. I simply will take care of the push. And in the moment there's a change on the robot. You will see it on the front end without ever folding.

So what that means is at the second row About sends a new image from the camera is your component updates and shows that new image correct a web component.

I wouldn't necessarily use images or an example here, okay, because that is more complex said for our web RTC and like with good compression. But the example still holds. If you if you could do that you could send an image over that channel. Another example would be like, let's say the battery charged, if your battery charged, that's probably a better example. It will show up right away on the front end.

That's really cool. So how if I if I'm developing this, like if I if I'm writing my own capability, how do I test that all of the three modules are working well, for the capability? Is there any like, I don't know, the thing that makes sure that all of them are working well together, that data is being sent around that I can test my components, my q&a module for the web browser locally, these kinds of things, like a storyboards for the mind, all these things. Component,

there's a lot of tooling, yet, so some of these automated checks that you're alluding to don't yet exist. But there's definitely the developer workflow and the documentation on how to run it locally in a development server. So that you can you can try this out. And I think that's a good way to get started, you can use a simulator to run you know, your, your, your robot, whenever we use turtle sim a lot, which honestly is enough for a lot of work we want to do, and then you open the local web page on your machine, and will you will be able to see whether or not things are working.

Gotcha. That seems super cool. And what is the so in the cloud component? So there's a web, the cloud the robot, what does the cloud component do?

So the cloud component is actually a very module. The Cloud Module, there's, there's obviously what you what you want the functionality to be there can be visualizations, and so forth. But before it can get there, what the what the Transitive wrapping for that, for that module does is twofold. One, and that's very important is authentication and authorization. Because if you want to embed this, this component in a third party web application, you need to get the appropriate permissions to actually show the data of the robot, right? I mean, not, you don't want the camera of your robot to be publicly available on the internet. And obfuscation is not enough, you want some really strong authentication, the way that we solve that is through JSON web tokens, which is a security technology, it's it basically allows you to take a JSON object and sign it with an asymmetric or symmetric key depending on what you choose. And we use that to basically get permission from the back end of your service, whatever you're embedding it in, to show certain things to the user. So you will basically give your user a signed JSON web token that says this user has permission to see this device, this capability for this and that long. And then it will be embedded in the in the webpage that your user goes to that webpage will talk to the transit server, whether you host it yourself or whether you use our transit products hosting. And it will let us know whether or not we are allowed to show the users that that kind of data. So that's our Yeah, what are they authorized to see that and that's that by itself. So it's a big pain point that a lot of people struggle with just this. For sure. It's complex. And we've seen some very successful companies offering authentication as a service like Okta. And for good reason. It's, it's not a lot of fun. And if you make a mistake, a lot of at stake, right, your company, you're sharing too much data than you then you wish you were.

Yeah. And I think that's an area where it's like, oh, you need to go be an expert at to do well. And so it's nice to specialize there and have another company that just does this really well. But so you don't have to worry about doing this for your application. So the cloud component for any of these capabilities you're making is going to handle the authentication and authorization. I imagine that's kind of like baked in to what you're building. So they don't probably have to handle that when writing their application. Does it also handle is it on the robot that it's running? Like, I know that sounds silly, but is it the thing that sends the information up to the cloud? Or is it actually running on the cloud? It's actually,

yeah, the human way the authentication is going to be processed,

or I mean, I guess. So the authentic authentication. That's I guess, I'm just I'm a little confused, because so for that, I would think that that would be something that you would set up very well. And maybe all the code lives in the cloud component. And the person hardly who's the developer hardly touches the cloud one it just like, because I don't think like, if I want to use this to write capability, I don't probably want to do the authentication and authorization side. So I'm imagining that's probably generated or exists in something that's already being used, like my code is using.

Yeah, that's part of the the essence of the framework. And it's actually a feature of the MQ TT brokers that we're using, where you can have custom authentication and authorization, which which we try to offer. So the way it works is that the the web component actually establishes a web a MQ TT connection over web over WebSockets. Okay. And in order to authenticate for that, for that connection, it will send its Simes JSON Web Token. The MQ TT broker will interpret that and say, first of all, yes, you are who you claimed you are, because this is a valid JSON, valid DWT. And also I've decoded that JSON web token. And in it, it says you have access to the following topics. And this can be read or write access. And that's all you need. Right? Because again, this is the beauty of relying on MPT for everything. Now, you know, that the broker knows which messages it is allowed to send to you, and which ones you are allowed to write. And then it kind of goes into the ether and and everybody connected to that. That's it. Yeah, it's, yeah,

that's magic. That's so cool. I feel like it's taking such a painful problem and making it a lot less painful. So just so the cloud, if I'm a developer, using writing my own capability to cloud component, I will probably have the ability to use the default way of doing authentication and authorization. Or I can write a custom one, if I desire. And that's kind of all the cloud component does, or is there anything else?

Well, I mean, the other thing that the cloud component does is, of course, the data synchronization piece, right? So that in the cloud component, you have immediately access to you can subscribe to a subset of the of the topics that you want, and they will enter data will magically appear on your doorstep. And then you can visualize it in whichever way you want. So that you can really focus on on just that, right? How do I, how do I present the data. So you know, I mentioned your little play, I like to think in terms of black boxes, if you're a web component, your black box that you receive, after all of the magic has happened, you receive a JSON object. And what you're responsible for is outputting, a nicely rendered HTML page or whatever or react whatever you want to use. You have visualize that data, right? Health Monitoring is a good example. Again, like you get a JSON object that represents the health of your fleet. And you decide to show that as a table with red and yellow and green and and what have you. Right? That's that's really the that's really what people want to use web components for. They don't want to spend too much time on the on the plumbing, as I call it. Yeah, they just want to visualize it. And then in the reverse the same right, so your, your joystick opponent says the user has been pressing the forward button, now you need to send that to the robot. So you will send the appropriate data into the onto on that topic where you know, the robot will be

listening. Gotcha. That's really cool. What does it look like on the robot? How, like, how do you set it up and what's actually running on your robots when you actually are using this.

So the the way to get started ended depends a little bit on whether you want to use the hosted solution that we offer or whether you want to sell photos. But in the hosted solution, you just go to our website and create an account and you will get a curl script that you can run on your robot, which will install a Transitive robotics agent. This is just a small little piece of software that runs on the robot. And it's primarily responsible for checking whether you are running whether which packages are installed, which packages should be installed, and then installing them and starting them. And then once it is told to install a certain capability, it will fetch that from the repository and install it and run it in a sandbox. The Sandbox is one of the other pieces that is really important to the way that we're thinking about this because we want to enable third parties to develop for you. We think that sandboxing is really important.

Tell me tell me what sandboxing is. I know what it means, but just like describe it.

So the issue is always that when you're buying software from a third party, you you may not want to trust it 100% So you want to reduce basically the damage it can do you want to reduce the permissions that it has on both iOS And Android has a very, very strong sandboxing model. So if you install an app on your phone, it won't be able to read the password from your from your other apps, right, which is kind of important. Yeah. And the same thing here, right. And in the ROS world, people are used to using sudo, to install ROS packages. And that's a very honorable thing to do, as opposed to give that much trust. But if you are dealing with a much larger ecosystem, you may want to be a little bit more careful about the trust that you give and not run everything as root. So a lot of people wouldn't even want to run it in the normal user space, because even the running user could have could have secrets and so forth. That's one thing. The other thing is you want to separate those the capabilities running on your robot from one another so that they cannot see each other's data. Otherwise, the authentication and authorization is kind of goes out of the window, because somebody could install everything. Yeah, and a malicious capability could just listen in on everybody else. So you're solving this. Through a number of technologies that all borrow from Docker, we do not use Docker on the robot, because we don't want to use require that as a as a as a requirement. But we use some of the same technologies, we use the Linux namespaces, using the unshare tool, to to isolate things. This allows us to do pretty cool things, honestly, without requiring sudo on the on the robot, for instance, we hide a lot of folders on the drive, we can hide the home folders. And then we're also using overlay Fs, which this technology exhaustively used by by Docker, to minimize the number of copies of things that you need on the system. So if a capability has a requirement for a ROS package, it will it will first check is that already installed. And if not, it will install it in an overlay on top of what's already there. This way, we don't have to install ROS in each and every component that we have, which would happen if we were running in Docker containers, right, there would be a lot of duplication, which we can, which we can elegantly avoid.

That is really cool. So you see if you have it elsewhere. And then if you have it elsewhere, you use it from that location. If not you install it. That's really cool. So what about handling things? So that that handles? You mentioned a lot of things? And I don't I'm not that familiar with all those technologies. But it can you hide things like the environmental variables that one capability can see versus another one? Or it just like all of the different things you give it access to only a small part of the file system. Yeah. How does all this work? It sounds so cool.

Right? Now we go with a one size fits all type sandbox, you don't actually customize it, we just we just reduce it down to what we think is all you need. If you don't see the environmental variables of your user, you you're either home folders or hidden certain sensitive folders in the in the file system such as var lib, or from you as much as possible, like the data, if you're, if you're running a database, it would live there and he wouldn't be able to see that. And you explicitly

opt in to permissions with this, or is it implicit in that, like, oh, this app needs this? And then we'll give it that? I'm not sure it's explicit. So

so right now we, we think actually, we have the permissions down to what most capabilities will will need we there could certainly be some people have asked us about things like user management on the robot, right? This is clearly something that would need administrative rights. And we don't currently support that. We are currently only supporting things. It's a what one way to think about this, especially for your audiences ROS and up if you a lot of capabilities run a ROS node, they're able to talk to the other ROS notes. And that's honestly all they need to do. Yeah, it's it's none of their business what I have on my on my desk, right? That's that's not what we're here to do. So, all of that accesses is going to be very, very limited. Other than say, you know, what you need for, for for ROS operations, such as the message definitions, right. But most of the file system will be hidden from

good I saw I'm imagining if I had some node that so I have a module on my robot, and that's sending up video. And maybe I want to send some sort of thing down from the cloud, to say, hey, we're a bit bogged down right now. Reduce the video quality you're sending up or something like this. And maybe it's netstat and an environmental variable, like, could you give it access to that? Or maybe there's a better way to go about doing that kind of thing?

I would have to think about it. If by setting an environmental variable, you mean one that would also be written, read by all of the other capabilities, like some sort of shared space? Yes, not currently possible, there is no shared drive space. So another piece of technology that we've taken a good look at is snaps on on Ubuntu. And they, they solve this very elegantly to where they have, of course, you know, containerized, or sandbox capabilities, but they also have some, some shared folders, right, in which you can collaborate if necessary. So that that certainly, that certainly exists. In the example of the video streaming, I would, I would argue that that's the capability itself, that then reduces that.

But yeah, you probably wouldn't need to do it the way I described. Because yeah, the capability would probably just do it. And it'd be setting a variable inside the module, robot module. That would be probably the best way to do it.

And actually, so we do have that video capability. And we use Web RTC, of course, and one of the beauties of web RTC is that it will actually do that automatically, right? That's so Turman, I can't get those frames through, we're going to reduce quality for a little while, and and it will adjust to that, which is very nice. And really one of the reasons also to use Web RTC

for industry. Do you think that this idea of sandboxing will catch on more generally, in robotics? Like I mean, if I'm using a bunch of software from the internet, you never know if something has been maliciously included? So

yep. Yeah, I, I think so. I mean, if you've been following what Ubuntu has been doing with their push for four snaps, for instance, they had a person that was also very active in the Rust community. And he was in charge of ROS and snaps on the canonical side. And he was making some very good points as to why snaps are a good solution for that because they are sandbox, like you say, right, they are snaps for, for those of your listeners who don't know, basically, or you can think of them as Docker containers that run apps on on Ubuntu now, people will yell at me for describing it like that. Because of course, in reality, it's more sophisticated, and it actually doesn't use Docker. But for that, for the event, who knows a little bit of Docker, that's roughly how it works. And so you get all of the beauty of isolation, and, you know, reliable requirements, like dependency, dependencies, and so forth. So I think it's useful. I'm very curious to see how this will evolve, and whether the sandboxing will get into the way of development, whether people will prefer to install their capabilities, quote, unquote, privileged. But I think it's the I think it's a, it's a requirement for more broader cooperation, like you said, if we want to enable third parties to run, and it's our fleet of robots, right. I mean, for it from the perspective of a business owner in robotics, they're very conservative about what gets to run on the robot. Because if you screw up, like, these are implications, there's a lot of implications for on the security side, but also, even just on the operational cost, if for some reason, your robots one morning, don't do it anymore, or, or you know, they dropped from the internet, you can't reach them, somebody is getting on a plane flying somewhere. Very expensive.

Gosh, uh huh. One thing a little bit of a tangent, but one thing that I thought was really interesting, and I am hoping more things start to go this way. There's Dino, which is a TypeScript library or TypeScript runtime environment. And Dino, for us. It's kind of like Node, but better suited for TypeScript also runs JavaScript. But if you explicitly opt in to every permission that you want to use, you can say all if you want, but so you can say this should only be accessing these folders, this should only have access to this environmental variable. And all of the software that you use can have these permissions set, which is quite cool. Because I mean, if you're running software, there's the chance that it's like trying to access something you don't want it to use. And this would be a nice way to bet that hey, there was an error or tried to access some sensitive location. We denied it and the program exited.

That's pretty cool. I had heard about the though I didn't know that they have that feature. That's uh, I'm actually gonna take a look at that. That's that sounds really useful.

Yeah, I'm sponsoring site quite a few you know, projects cuz I really like the direction that they're going But the whole node ecosystem is really, really, it's just so much there, that it's hard to use the you know, for anything that you really want to build, like, I've just been doing it for, like, command line tools that helped make my job at open robotics easier to try them. And you know, but lately node is kind of thing.

Gotcha. So it's not a replacement for Node js at this point.

Not yet. But there are quite a few packages that are like, I mean, I don't know, there's like an express like thing for hosting web servers, and routing. But if you want to do like, really nice date time stuff, or I don't know, there's just a bunch of missing stuff. Or you can go look at the CDN serialization or whatever it is, I don't actually know what that does. But you can go some packages can be translated over to it, but it has a different import setup. So it doesn't work flawlessly with existing node things, which is a bit of a bummer. But the ecosystem is growing. And there seem to be a lot of really smart programmers doing stuff on it. And building tools.

Yeah, I mean, that's really, I think, key to a framework platform. You name a working ecosystem of developers, right. And my favorite example there that the reason why Node js is so popular and works so well is because it's so popular and works so well, right? Like people use us developing more things. And because they develop more thing, it's the network effect, right? And then more things. So you can find a library for just about anything if you don't need to develop it, yourself. There's downsides, of course, and we've seen the news about some, you know, either hostile takeovers of packages, where people have have stopped maintaining it. Whereas sometimes even malicious changes that people made sometimes out of spite or break everything, yeah, yeah. And those issues are, of course, serious, and we need to take care of them, and it'd be worried about them. But it's a very cool technology to build on.

Oh, definitely. Yeah, I find it. So I mean, working at Open Robotics, we use a lot of C++ and Python. And type like years and years of using Python. But I really like TypeScript, because defining the interfaces is really nice. And it helps me I feel like I'm much more productive in it. And so now I'm getting familiar with the node, or NPM, packaging, and just downloading libraries and software from it. And I'm very impressed. And I feel like it's a, it's a lot more capable, in my opinion, than a lot of the Python libraries. And there are obviously exceptions, like, like, pandas is a really nice Python library that I don't think has a good analogue in TypeScript or JavaScript. But just it's an interesting space.

It absolutely is. And I like to play dog to that. Note, J S is actually a bit faster than Python. And that's because JavaScript is compiled just in time, it's not actually interpreted, it's just compiled on the fly, which makes the execution a lot faster. And then I can, you know, say a lot of positive things about NPM. As a package manager, it's, it has a lot of features. And it has a lot of capabilities that other package package managers have within the past leg. So for instance, one big one is having multiple versions of the same package in the repository, you would think that this is a trivial thing. But DVM, for instance, doesn't, doesn't really support that apt, you're not really supposed to have multiple versions of the same package in the in the repository. But it is important

for your application where you're trying to wean robots off of version, when they're when you're updating them. This is, this would be very useful.

Next time, it also for the for strict versioning. I think that strict versioning, of strict, strict semantic versioning is a real benefit. Where you in npm date, there's this concept of a lock file, where you will save I'm going to lock down my versions. And when I commit to get is exactly this, if you execute this, you will get the exact same result, no matter what any of the dependencies have done in the meantime, in terms of updating, you will get the exact same. And and this is really helpful when you're dealing with an ecosystem like NPM, where you have hierarchies of dependencies that can be very, very deep, right? You can super nest 2000 packages being installed as part of your of your somewhat simple application. And that's, that's a benefit, right? You don't want to reinvent all of those 2000. But you do want to be certain that if anything happens to either of them, maybe they introduce a bug in the new version, your system doesn't go down. So really locking down the versions and deciding Yes, I want to update or I don't want to update it. Is, is very powerful. And for that you need to be able to have multiple versions of the same package in the repository. The other one is, of course, all of the books like pre installed post install, pre uninstalled post, uninstall, pre publish all of those kinds of scripts that you may want to run as different types of the software lifecycle, they are very powerful they can really allow you to do I mean, us? They are, they are very powerful in Debian packages as well. But because NPM has them you can do, you can pretty much install anything through NPM packages that you have, because the payload is just a tarball. And then you run these scripts when when it gets installed. And at that point, you can, you could install things out of the tarball that say you Debian packages, or play system files if you have the permissions to do that. So it's very, it's very versatile tool. And I think it's, it's a real good technology to build on and share.

Let's see. So now segwaying, a little bit. Tell me a bit about your open core model at Transitive.

Yeah, this is still still in flux. But I am a big believer of open source businesses, I think that there are some very good examples that we've seen in the past. And so we, we really want to make this open source model work. And from everything that I've learned. There's, there's basically two things that work really well from, from an open source business model point of view, one is testing. So in our case, we are offering a hosted solution where the cloud will be taken care of by us, you can still, like I said, have the components and embed them in your own web applications, if even a web application is running somewhere completely different. But you don't have to worry about it, setting up the cloud, dealing with the maintenance and administration of that storage of data and whatever your capabilities want to do. That's the that's the one model. Some of our customers have asked about self hosted solution. So we are offering that as well. And this is still a work in progress. So that has not been released. And the business model in that case, is premium applications. So it's it's really similar, I guess, to Android, again, right? Where Android is open source, you can install it on any phone you want. But then the capability the apps in Android can also be free, or they can be they can be premium. And I think this is a good model, right? It allows people to first of all get started with it. They it provides developers with a marketplace where they can distribute their their their contributions, their work, they don't have to worry about, you know, advertisement and and billing and and all of that. And on the receiving end, you know, the the companies using those capabilities will have an ecosystem with writings with commercial support behind it that they can that they can rely on. So I think that this is a model that can that can really work.

Gotcha. And so are you experimenting between those two models? Or your you'll think you'll kind of see what capabilities you have? And then possibly add? Actually, it seems like it makes sense to do the Android model for what you're talking about, where it's and then you're doing both in a sense, because you are providing the hosting, but you're also possibly going to have specific capabilities that people can opt into, by purchasing. Yeah, that's,

yeah, that's exactly right. And both are evolving, we'll have to see which one will be more popular and how, for instance, what we are still keen to find out as to what extent will people prefer the hosted solution versus self hosting? There's definitely pros and cons with both. Some people really don't like the idea of relying on a third party, especially if it's a startup, right? Because it multiplies to whatever No. That's why the ability to self host is certainly desirable. Others really don't want to take care of the scaling of that infrastructure and would much rather outsource that. So this is mongos applet service, which I think is fantastic. If you if you just want to set up a database, you don't need to set it up yourself and manage it. But I Yeah, time will tell really. And we're curious to hear from from more people what they think. And also, on the premium app site, there's this additional persona of the third party developer, right. Are people interested in developing for this platform? I think so. I've spoken with a lot of people who have been working on specific capabilities, a lot of them in the ml or computer vision space that they want to market to robotics companies. And they like the idea to not having to do all of the legwork in terms of marketing it and figuring out deployment and so forth,

having more of an app store third approach. Yeah, that would be super cool. I would love if we had that kind of thing in the robotics ecosystem. Just like ROS really solves a lot of problems, making it easier for people to build applications. But it's hard to get it outside of your robot, as you're saying, as we've kind of motivated with the whole first part of this conversation. And so if it was like, you just try to get you would just add different capabilities on through an app store like setting and you can use them elsewhere. That would be super cool. Yeah, that's it. Gotcha. How have you guys been funded so far? And like, Would you talk a bit more about the team and everything,

the team is very small, you're looking at it. God's just, it's, I'm the only one fully committed right now and doing technical development, I have a short, I have a small list of advisors. First and foremost, my previous co founder, Bob Bauer, he was at Willow Garage, and he's still advising me. But he's not a developer. So there's not too much for him to do right now. And funding, it's, it's completely bootstrapped right now, where, where we're profitable in the sense that I'm doing consulting work for customers, helping them both doing custom development, and then helping them figure out how to use Transitive as well, which is a good model, it's, as you can imagine, you don't get rich doing that. But it's, it's, it keeps the lights on. And one thing that I've seen with robotics companies, one, one way that they have failed, was that once they take money, they have to spend it once you once you because your investors want to use that money to hire a bigger team and scale faster, which, which means that you are lacking in a certain certain product, a certain business model, and a certain market fit. And if the market fit is not great, then you're trying to scale a business that was not meant to be, then you're making you're you're digging yourself a hole. And I really want to be certain that we have absolutely nailed market fit that we have gotten adoption that we have validated that this is something that people want before we take on money. And so yeah, I consulting that actually solves two problems here. Because, number one, yes, there's some money coming into the bank. And number two, we are actually getting validation from our customers. So this, this really lends itself to an iterative process that will make sure that we will build the right product for the market. Yeah. And I think we would get into race, no problem. We've had conversations with investors, they, they like the idea that we are basically indexing on the robotics market. And what they mean by that is, rather than so they have the choice of which companies to invest in. And they know that in robotics, the differences are stark, right? One company may be doing apple picking the other one maybe doing drywall construction, another one may be layout printing, and another one is doing delivery on sidewalk, another one on the street, if they don't know what to invest in, they all know that robotics is going to go up. And there's plenty of studies by IFR and all of those API research that tells us it's going to go through the roof. It's it's it's a phenomenal technology. And it's only sort of at the start, but they all want to invest and they don't know where to invest. So what they like about platforms like this is that they know that if robotics will succeed this company,

as well, gotcha. That's really cool. I really like bootstrapping, too, as a model. In general, I get it. Because one of the other things with if you take investment is you kind of commit yourself to a timeline that you will have some sort of liquidation event within some period of years, like typically less than 10, maybe closer to five years that the investor would like to see a return on their investment. And that's tough, because as you said, you have to like really commit to the current fit, and it lends itself to a certain type of product. Yeah, and that being said, like cobalt robotics, they're doing the investment route, and they're doing a phenomenal job as far as I can tell. Which is really interesting. If you position everything well. And it'll be cool to see kind of if you start accepting investment, and how that helps you scale. But once you found that really good market fit, that seems like a very good approach. So me.

Yeah, I'm keen to get into that point. And I do have a concrete plan on how to use that money in the employee to to foster third party development. I think there's, there's there's some really interesting models that you can that you can follow. Yeah, I think that there are a lot of capability. developers out there and I love this idea of, of the marketplace because it has inherent network effect. So more capabilities We have of high quality, the more people would want to use it, the more people who come to our, to our app store to look for capabilities to what people want to develop for it. And so that's, you know, that's the classical network effect that can that can actually justify exponential growth. And investors always want to see exponential growth. And I always ask them, Well, based on what, right, it's not, it doesn't just happen, right? There needs to be some sort of mechanism behind that. And it's not bouncing a QR code on the Super Bowl. In Super Bowl x, right? That's, that's a lot of money. No, you have to have something where the classical Atlas one, right if the ad plus one user has a better experience than all of the end before and makes the experience all of the end users before that better. That's network effect. That's why you have these these conversions effects where everybody will use the same social network, right? If, if you're the owner, if you and I go on to a social and create a new social network, and we're the only users Well, you don't have to have great conversations, but but it's not as useful as if they're more friends of ours on that same thing, right. And so for me, one that we had, it gets better. And I think, you know, marketing marketplaces have that dynamic, inherently built in, they're also hard to pull off. That's where money can be really helpful. But I think first and foremost, you have to make sure that it that it goes about it the right way and openness. And I think it's really the the key to all of that.

Christian's LinkedIn

Transitive Robotics' Website

Transitive Robotics on LinkedIn

Transitive Robotics' Twitter

Sign up for Transitive Robotics' Beta

Contact Transitive Robotics

Full-stack Robotics and Growing an App Marketplace

They Still Wouldn't Want to Make It

Business Models for Open Source Software

Openness and Market Fit

Tent Poles

Web Tools and Robotics

In this episode, Audrow Nash interviews Adrian Macneil, Co-founder and CEO of Foxglove. Foxglove makes Foxglove Studio, an open source visualization and debugging tool for robotics. Adrian speaks about the origin of Foxglove, Foxglove’s business model, web and robotics, and gives advice to those interested in getting more involved in robotics.

Start

Introducing Adrian and Foxglove

Origin of Foxglove

Developer productivity and the future of robotics

Building the Foxglove team

Why Adrian’s optimistic about opportunities in robotics

On putting 1000 engineers on a problem

Role of open source

Foxglove’s business model

Why Foxglove “Studio”?

How name Foxglove?

Explaining Foxglove Studio

Reflecting on his work at Coinbase and Cruise

Timeline for autonomous cars + levels of autonomy

Advice for those interested in robotics

Info on getting in touch and contributing

Outro

Hi, everyone. Welcome to the Sensing Think Act Podcast. I'm the host Audrow Nash. And in this episode I interview Adrian Macneil, co founder and CEO of Foxglove, Foxglove makes Foxglove Studio, which is an open source visualization debugging tool for robotics. In this interview, Adrian and I talk about the origin of Foxglove, Foxglove's business model, web technologies and robotics, the timeline for autonomous cars, and Adrian gives advice for those interested in getting more involved in robotics. I really enjoyed talking to Adrian, he has an interesting perspective on technology and seems to be involved in interesting technologies, right as they're coming up. For example, He was an early employee at Coinbase, and Cruise. As always, a big thank you to our founding sponsor, Open Robotics, and I hope you enjoy. Hi, Adrian, would you introduce yourself? Hi, I'm

Adrian Macneil, founder and CEO of Foxglove.

Now, tell me a bit about foxglove what it does and what problem it's solving. And then I'd love to get into the backstory of that after.

Yeah, definitely. So foxglove is a developer tools company for robotics. We have our main product right now is an open source visualization platform for robotics. And it's browser based cross platform. We can allow you to really dive into your robotics data understanding of your sense of editing and planning, visualize things like point clouds, images, overlays, markers, bounding boxes, plots, and drill into kind of raw messages

and you guys you interface tightly with the robot operating system or ROS ecosystem, correct?

Right? Yeah, correct. So we we work very well with ROS. We don't we're not strictly limited to Russell though. That's definitely our best supported platform right now. We do allow you to connect other types of data sources like for example, we can connect just to a velodyne LIDAR and and talk directly to that sensor and visualize data coming off the sensor. But when you're in a ROS environment, we can obviously make a connection to ROS and then we have a lot of custom visualizations that help you understand ROS messages like point clouds and images and other things like

how does it compare to existing ROS tools like ARV is our ROS visualizer for visualization?

Yeah, so yeah, so we have I would say the biggest difference is that it's cross platform and using a web based stack that was something you know, we can get into the origin a little bit that that was something that was kind of a key underpinning of building this technology is that we wanted to work with a browser based stack which means that they can run cross platform we actually have desktop apps that you can install on your computer that run on Mac Windows and Linux equally well and then you can also access it in the browser. So that you know that makes it like very easy to just jump into an environment and start debugging it also kind of unlocks that something that I think is as important to me as breaking robotics development out of being stuck with like Linux desktops and you know Linux is great but a lot of robotics development now is very the software the developing is like very tied to the platform that you're developing on and you know if you're using ROS Vision X you know ROS galactic you need to be using Ubuntu 2004 and you know that's the only choice you have you pretty much stuck with that for your development handle and get developed well

it does work so galactic and many of the other ROS 2 distributions also work on Windows not very well I think tier three support for Mac this kind of thing Yeah, but

yeah, it's getting there I did say yeah, you're right.

But this was this was definitely which was kind of the origin of like foxglove was a rough one. Or Weber's Yeah, we

support both ROS 1 roster Yeah. But yeah, for sure our windows support is out there. Although you know, most of the people I've seen doing ROS development on windows are using the windows subsystem for Linux so they're actually still running you know, the actual Linux version. Oh, that's fine. And then yeah, and then Mac actually you know, Mac came out I think ROS 2 initially came out and said we're gonna suppose a lot of work. Yeah. Like no one kind of stepped up to volunteer. Yeah, so the and the actual you know, code that you're writing big Linux only is fine, right? Oh, we want to break out of as the developer tools so there's no reason that you can't be on a Mac developing inside a Docker container. But But your actual tools to like visualizing things, it's, you know, those graphical tools are going to be easier if you can just connect to them from a browser or connect or, you know, fire up a native emulator.

what's the what's the big win? Or what are some of the big wins doing it browser based as opposed to like qT? or something, which is what ARV is uses?

Yeah. Yeah, yeah. And I think I should jump into some of the other things to talk to the browser based thing. And then also some of the other things that kind of differentiate us with with Office and Matthew two things. browser based, I think, you know, there's a few points there. One is that it's it's inherently cross platform, you know, the browser is a really great environment where, you know, as long as you fire up your browser and go to a website, you can pretty much expect the same experience, regardless of what platform you're on. That makes it really good as you're scaling a robotics team. You know, if you start out with just one or two people, you're working on a project, that's, that's, you know, it's fine if you've got a pretty custom environment set up on your computer. But as you scale up that team, you want people to just be able to, you know, on board start working on the code base, you know, being able to like visit a URL and look at things directly is really great. And then another point there is that it makes it really easy for you to access remote data as well, right? So we can, you know, get into talking about data storage a little bit as well. But a lot of the times, you know, the state of the art and robotics today, a lot of people are just storing, you know, huge 100 gig bag files on a computer, and then I've been trying to debug it. And that's ROS bag. You want to share this, my friend? Yeah.

And ROS, too. It's an SQL lite database. Right. And I forget what it is in Roscoe, and

yeah, it's just a custom format, bag. Bag file format. That's just basically your ROS message. But having

a big monolithic data storage, basically. Massive.

Yeah, yeah, yeah. And so those can get really huge if you're dealing with multiple cameras, especially if you start getting into LIDAR and point clouds and radars and sensors, bossier. We're also recording all of the other messages that ROS is sending back and forth. And so, you know, a lot of people are working with these files locally. But sharing that among team members is really challenging. It's, you know, a case of like, either just get someone to wait here over and have a look at your desk. Or you know, you can't, you can't just like share a link in slack and have someone else click on that and immediately start looking at it. So that's, that's where we kind of see the browser's really unlocking these workflows where you can just click on data.

And is that talking about having data storage happening in the browser?

Yeah, yeah. So not always, I mean, the data storage happening on a server? Yeah. For sure, yeah. Yeah. But yeah, once you get to having a data stored in a centralized location, and the browser can just easily be streaming that, you know, it just makes the whole internet experience really, really compelling.

Let's see. So I feel like this is a good time to get into the origin story of foxglove. So can you tell me where it started?

Yeah, sure. So I spent the last five years prior to this cruise, building self driving cars. And I led basically all of the developer tools and infrastructure there. So when I joined, it was, you know, a very small company, maybe 40 4050 people or so over the over that period, we grew into the sort of 1000 plus range, I think it was around 1700, when I left out, and you know, over that time, or the engineering team grew right from, from 20 people to well over 1000. And so, you know, they they started, as you know, I'm sure a lot of our products company, so it was a ROS stack originally, on ROS 1, and you have a bunch of people just, you know, contributing code, building features, testing things out. But as you scale that and 100 and 1000 plus engineers, do you really need to start getting a lot more disciplined around the developer productivity? What tools are you making available to people? What are the workflows that you encourage people to do? You know, the less you standardize things, I guess the more pain you're in for as you start getting into 1000 plus engineers contributing to a project.

Yeah. And so crews had an internal effort to make something for visualization, rather than using the standard ROS 1 arvis. Or our Qt graph. Yeah. Anything?

Yeah, for sure. It started out as as visit Accutane. As I mentioned, they started out on a ROS 1 platform, positive ROS definitely gotten kind of switched out over time and replaced with in house things. But in the early days, everyone was just using, you know, the standard tools like ours. And the project that we that we kicked off with visualization there. I think we started back in about 2017. It came out of a hackathon actually it was not instead of a top down approach of you know, this is something we think we need to invest in, it was just something that some engineers really felt needed to exist and really felt would would kind of streamline the workflow. So yeah, that was kind of the origin of width is which, you know, some people may be familiar with, they presented with us at ROS con in 2019, I think it was eventually open sourced in 2019. But back in 2017, it really just started as, you know, some engineers on the team feeling that that we could, you know, we could do better on a couple of aspects. One is, you know, bringing these tools to the web where it's a lot easier for, for, you know, people to contribute to, like easier to find engineers.

Easier to contribute to it or easier to, for people, why is web easier for people to contribute to?

Yeah, I mean, I guess, you know, that could be a bit of a loaded statement, because obviously, if you're familiar with c++ development and development, then it's going to be a lot harder for you personally, to contribute to but you know, on a scale of, of larger commercial engineering teams, it's, on average, a lot easier to find front end and JavaScript engineers who can build web applications. There's just, you know, a much larger pool of people that you can draw

from, would you guess the order? Is there like, there's 10 times more people that are familiar with JavaScript than c++ or I mean, just ballpark for these kinds of jobs? Yeah, I think it's 10. Or, like this order of magnitude,

probably in the Bay Area. That wouldn't be far off. Yeah. Yeah. Yeah. Yeah. I haven't seen any numbers specifically on it. But yeah, anecdotally going out to hire a front end engineer versus a c++ engineer who has experience experience. Yeah, Qt experience and things like that. Like for sure there's a lot of really talented c++ engineers out there, but not so much. On the visualization side, there's this argument that I've, you know, done game development and things like that. But yeah, just just much easier for, especially once your company, you start building a lot of internal tools, usually a lot of internal tools are web based. And I don't want to push on, on being easy to contribute to is like, the main thing, but it's that combined with the fact that anyone at the company can just hit a URL and access that you don't need to worry about like deploying this app across your team. And also, you know, there was something that was important to us was being able to like unify these various tools into one interface. So

yeah, yeah, exactly. So are you able to bring

speak to what each does just a little bit?

Yeah, yeah. So avars is primarily a 3d based visualization tool. So you can go in and you can, you can open up some data, and you can look at where your model is sitting in the world. And you can see sensor data. And you can see, you know, three name markers, 3d bounding boxes, and things like that. You can see things like your transforms like the shape of your robot, and you know, where your robot is relative to something that it's trying to act on.

And if I understand correctly, it's not a 3d simulator. It's more just 3d visualization. There's no physics engine associated with it.

Right? Right. Right, exactly. So it's purely taking necessarily information from a robot, and then just showing you, you know, how your robot is thinking, and the world is gonna have an insight into your robot. And then you have, you know, acuity is sort of a collection of tools that you can open a standard one on Windows, or you can kind of open them in a sort of attached panel configuration. And it comes with a handful of other tools, which can give you things like reading diagnostics, from a robot reading direct messages from a robot, reading plots from a robot or reading plots, but displaying plots based on data from your robot. And then you start seeing that there's other tools in law recently that have come out like plot juggler and things, you know, a separate again. And so yeah, that was kind of one of our insights, I

And one of my early in one of my early days there I have a vivid memory of attending we so we had Friday demos, basically, every weekend, people could jump up and kind of just show something called what they've built. And someone from the QA team jumped up and demoed the script that they've written that would basically take a bag file and then open it, start ROS in the background, and then launch about like 12 different windows on your screen and rearrange them into position so that you'd be ready to start debugging. And I was like, you know, and they got a standing ovation for that right because it was so much. Such a massive time saver compared to Compared to like, manually going through that process and you know, taking a backfile and wanting ROS and then opening these various windows, you know, I just remember that so vividly because it's, it's, it's, you know, like seeing the pain of how difficult it is just to get to the point where you're actually getting your work done. It's like, this is not, this is just like admin before you actually get

20 minutes of work before you can start work.

Right. Right. And, you know, and that's actually sort of another difference that another point of differentiation we took is that, you know, the tools like others are, they're very coupled with ROS. So you have to have ROS, you run ROS, in the background, you stone, your ROS stack, and then you start others, and others will talk to ROS, and it'll show you kind of exactly what's happening. And if you want to play back a bag file, playback recording that you have, then you need to play that back into ROS. And then kind of you'll see it as like a byproduct of

Right, right. Everything's kind of going over the ROS network. And, again, when you start thinking about different workflows across the company, you have QA engineers, and they have a look at right.

So they want to talk up to good quality, and we're doing things well.

Okay, right, right, exactly. And if they're taking let's say, there's an incident where, you know, something that happened on the road, right? We pulled out this or not, we got too close to a cyclist, or we, you know, whatever we did, and you've been assigned to triage, an engineer has been assigned to that kind of triage this and figure out, maybe not like, what is the root cause just yet, but where do we think the likely root causes? You want to be able to just like, open that kind of incident directly and start looking at it? You

know, you're saying that the incident? Yeah, yes. rosbank file or okay.

Right, right. But you want to, like, what you actually want to do is get into visualizing that incident as quickly as possible. And you don't, you know, the fact that there steps like downloading a bag file,

which is huge. codebase, which is huge, probably. So it probably takes

uploads are often slow. You get it from a thumb drive.

Okay, yeah, tell you all about No. So it's just a process, right, kind of take that back file, and you bring it down to your computer, and then you open up, you know, check out the code base, and you go back to the version of the code base that you had last week that was actually running on and then you launch ROS launch hours, and like, now you're looking at the data. And, you know, we just wanted to simplify the whole process. So it's like, you click on the link, and then you're looking at it immediately.

Yeah, that is way easier, where you can just open it. And so you, and then this gets into the cross platform and everything because they could be on any computer. And they don't need to spend their money for OS installed. And they don't need to have ROS bag installed.

Just run it on a Chromebook or something and just, you know, immediately jump into things.

Gotcha. Okay, so then I see the motivation for this quite clearly. So you were at cruise, and you were working on web servers. And you had this experience where someone automated some of the setup that they might go through when trying to do some debugging. And that was like an aha moment for you. And then is that how web is started?

Yeah, and, you know, to clarify, I guess I, I was the director of engineering, so I didn't like contribute much, or anything to the webfest card itself that the team was under me, but all the talented engineers did the work not, not me. But you know, I very much agreed with the vision of of what they were trying to achieve there. And I think, you know, we, the team that was working on that there was some really talented engineers, they kind of made a proof of concept. Like I said, at a hackathon. They went around, you know, a handful of different teams, and kind of went team by him and said, Hey, you know, what problems you're having with your tools? How can we solve them with your visualization, and just incorporated all of the various workflows that were happening across the company into this one tool. And so, you know, over that period of a year or so, it just became kind of an indispensable that everyone was using at the company. And that really just opened my eyes up to you know, how much of a productivity boost a good quality visualization tool could be. And, you know, that's what ultimately led us to consulting.

Gotcha. And so you open sourced web is and then now you are with foxglove. Can you tell me a bit about that transition

Yeah, yeah, for sure. So we open sourced my visit cruise in around 2019. The goal there was to, you know, give back to the community. And, you know, also just to put some of that code out there as as, you know, engineers working on projects that companies quite often you want to just put, you know, anything that's not sort of like, tied critical IP to the company. Yeah, yeah, critical company secrets, it's always nice to be able to publish things. And we published a lot of open source code occurs around, you know, even just unrelated things, like how we're managing our Kubernetes infrastructure and things like that. But, you know, we put those out there, and it was available for the community to use. And, you know, they presented a Roscommon 2019, I think, and, you know, picked up a bit of steam. Certainly people were out there using it. But I guess, you know, here's where I would draw differentiation between non open source coders create an equal, I guess there's a difference between kind of code that is public versus actual communities that are being built. And, you know, I don't vote Cruz for this in any way. I think Chris is absolutely making the right decisions for their own prioritization and things. But, I mean, I was like part of these decision making but but Chris doesn't have time to build features and weathers that are not useful to crews, for example, you know, they have the project out there, but like, no one is building ROS 2 support and the weapons are known as

stack now. I mean, the stacks kind of, you know, hitting off in a different direction entirely. So, you know, it's the code was out there, and it was available for people to use, but there was never a kind of a pretense that crews wanted to sort of invest heavily in building a community and adding features that were like, not useful to them at all. Like I say, this is, you know, absolutely the right decision. If you're, if you're, the purpose of your company, is get a self driving car going in San Francisco, you know, the purpose of the company is not build unrelated kind of open source communities and invest heavily in that sometimes that happens, you know, if you're a company the size of Facebook, and then you build something like react exactly like, shot, you have hundreds of 1000s of employees, and you can spare a small team of 30 to work on XYZ thing, but

it's crazy. It's really that small. Yeah, that makes sense. But yeah, I didn't know. It could Yeah, or something. Was that just guess yeah.

But yeah, it's Yeah, it's, it's when you're a sufficiently large company, you can kind of afford to make those bets. But for the majority of companies out there, you're, you know, when they're open sourcing code, it's usually sort of isn't code that we wrote, use it if you like it, but we're not supporting it. still focused on my internal needs. Yeah. Makes sense. Yeah. So that's, you know, that's what ultimately led to foxglove. We had a lot of chances, the windows team with courage and leadership and several other weapons team are working Foxcroft now. But we wanted to make a clean break, I guess we, you know, another bit of context, as to the weapons, as it exists today is pretty tightly coupled with a lot of internal crew systems. So the open source version of weapons is only you know, only about half of the code and weapons is actually open source, right. And the rest of it is special custom source that exists inside our crews. And those two are quite tightly intertwined. It's not sort of super easy. For us, as founding foxglove, it's not super easy for us to go and kind of contribute back to that project without having access to all of the closed source things. Yep. Makes sense. So we made the decision, when we founded Fox five at the start of this year that we wanted to kind of carry on weathers in spirit. And you know, we use the we forked the code base originally, which means we took that code base, and we're basically saying, you know, where versus here, it's, it's the open source version of web, this is not really undergoing a lot of significant changes. Right now, most of the team that still works on that occurs is primarily focused on career specific additions that are not open source yet. But you know, we said we're gonna, we're gonna just, you know, we want to continue this in the spirit, but we're going to, it's going to become a different product over time. And we don't want to sort of create kind of confusion there. We just want to say, Hey, we're building Foxtrot. We're using some of the stuff that was done in the past. we're incorporating a whole bunch of Elon ideas, but we're not kind of being held back by what's of company technical. Yeah, exactly, like company interests, or just like, you know, technical limitations around how how projects are kind of tied to internal code.

2 posts with the web tag