In this episode, Audrow Nash interviews Brett Aldrich, author of SMACC and CEO of Robosoft AI. Robosoft AI develops and maintains SMACC and SMACC2, which are event-driven, behavior state machine libraries for ROS 1 and ROS 2, respectively. Brett explains SMACC, its origins, other strategies for robot control such as behavior trees, speaks about the challenges of developing software for industry users and hobbists, and gives some advice for new roboticists.

Start

Introducing Brett and SMACC

Events in State Machines

Clients and Client Behaviors

State reactors

Explaining dance bot + hierarchy in states

Recovery states

Origins of SMACC

SMACC and market pull

Robotics domains using SMACC

A problem to push the limits of SMACC

Making ROS packages smaller

SMACC for industry users

Making SMACC easy to use?

Control in many robotics applications

Comparing state machines to behavior trees

Future of SMACC

Advice for those starting out in robotics

Links and getting involved

This is a conversation with Brett Aldrich who is the CEO of a Robosoft AI. Robosoft AI develops and maintains SMAC, which is a library that aims to make it easier for roboticists to control robot behavior. One of the key ideas in SMAC is that a finite state machines, we'll be referring to finite state machines a lot in this interview, so I'll explain what they are. Before we start. Imagine that you have a lawn mowing robot. Imagine that that robot is powered off, and that you turn it on by pushing a button, it boots up and drives to a field to mow. When it gets to the field, it starts mowing. Once it finishes mowing, it drives home and powers off. This robot has states like powered off, booting up driving, and mowing. The robot also has transitions that occur on events, such as when the robot is turned on by you pressing the power button, or the robot arrives at the grass to mow these transitions change the robot from one state to another. For example, the event of arriving at the field to mow could transition the robot from the state of driving to mowing. So summarizing, a finite state machine is something made up of states like driving, and mowing, and transitions when the robot has reached the destination, or when it has finished mowing. This is the sense think act Podcast. I'm Audrow Nash. Thank you to our founding sponsor open robotics. And now here's my conversation with Brett Aldrich. Would you introduce yourself?

My name is Brett Aldrich, and I'm the author of Smak. State Machine library for robotic applications written in C Plus Plus, I'm also the founder of robust soft AI. Robot soft. We're the keepers of the smack tool chain. We're also the creators of the smack to runtime analyzer. A graphical application lets you visualize your state machines both statically and at runtime.

Hmm. And what so can you start at a high level and explain what SMAC is, and then what its motivation is? Sure.

SMACC is a state machine library for robotic applications. I'd like to describe it as dealing with the highest level of task planning and management on a robotic system. It has many unique features. And.

Okay, so it's a, it's a high level state machine library that helps you figure out so you get to control your robotics application with it.

It's a control, it's an architecture for control. And a couple of things about it. You know, when you're writing your state machine, what one it it provides a tight integration with. With ROS, allowing you to generate events for ROS topics, services actions right out of the box. It also offers a reference library of state machines that are guaranteed to compile and run right out of the box.

So by reference library, you mean a bunch of state machines you can already use for specific things. Okay.

Exactly, exactly. With, with simulation included for many of them. Gotcha. It also offers a client library. And this, this is part of the architecture of SMAC. That allows ROS users to use ROS packages like move it to or nav to right out of the box. And it also provides a pathway for other ROS packages. And for the interfaces of those packages to be developed by specialists that are then tied to your state machine it all of these features have the effect of dramatically lowering the bar for new developers to get started in ROS, and then in programming in general controlling their robot. I can go on quite a bit longer about this, but I would say so it's a good high level overview of what SMAC is,

is it kind of so that if I if I'm using smack, I can kind of compose more complex behaviors out of almost like blocks that will affect their behavior and I can, I don't know, like, if I have this sensor, I can run use this block to send out the data? Or is that how it looks kind of like a bunch of blocks that I can compose together?

Yes, yes, you have, you have state you have states, right? That you can let us to divide your problems. You have hierarchy, which does much the same thing. You also have the concepts of orthogonal, which are like boxes that allow you to modularize the physical subsystems on a robot.

Okay, and by that you mean kind of control them independently or use them into basically, right. Can you give me a more an example, I suppose?

Yeah, absolutely. In fact, I'm gonna show a robot that that was recently built using smack as an example. So this robot is from field work robotics, it was built by them along with ourselves in Bosch, India. And as you can see, there's four robot arms and a couple of trays, and navigation based, so looks like a fruit picking robot or exactly. It picks raspberries. Yeah. And it was based on the work of Dr. Sutton's at the University of Plymouth and his soft robotics lab.

So that just to describe it for the audio listeners, it's kind of like a big moving cart with four different arms. That probably it that's on wheels and probably drives through and picks stuff for it. Would you say? That's, that's a good description of it, I think to add Exactly, exactly. Okay, so use this.

So each orthogonal, contains an arm. So there's this orthogonal one with each arm. Another thought for the base, there's probably two more orthogonal is for there was like, like a packaging. Not exactly sure what you call it, almost like a gantry. A small gantry inside for for packaging, the raspberries. You probably also have your cameras with perception, all of these would would go into an orthogonal using a model of let's say, the human body. You might have orthogonal for your eyes, another two orthogonal for each arm. Yet another orthogonal, so is

it? Can you think of it kind of like a degree of freedom? Yes, in a sense, yeah, you can. Yeah, cuz, so I mean, I'm thinking of so orthogonal, which means, like, it has a technical definition, which I think is it's the dot product is zero, or something like this, right? So it basically means perpendicular, but it allows for higher dimensions. Right? And so it's like a new dimension, a new dimension and state space kind of thing.

You know, I, the term that is I use it comes from comes from boost HR. And it was always tied to physical subsystems. But actually from boosted, charted, inherited originally from David Harold's 1987. Paper on state charts. So in 1987, David Harrell came up with state charts. And he developed that formalism that I guess you'd call it an extension of the formalism of state machines while working as a consultant, Israeli Aircraft industries, and in particularly, in particular working on an avionics system. And so in that, in that context of a fighter jet, each orthogonal basically was tied to a physical subsystem. Hmm. Yeah. As opposed to, I think a more like a mathematical interpretation, which is, I think, where you are going more

with yes, that's where I know the word from I haven't heard it. And this in this this state machine sense. Okay. So then, so you have all these different orthogonal, I'm thinking of them kind of like a general degree of freedom, where multiple so if you said a full arm is one, it's kind of like a subsystem. And then maybe you want to do things with it. And maybe that subsystem has several degrees of freedom, like a shoulder and an elbow and a wrist, which is controlling it independently of other parts of the system. Is that kind of the core part? It's an orthogonal is something that is controlled independent of other parts of the system? Yes. Okay. Yes. So I'm thinking generalized degree of freedom. So multiple degrees of freedom

And can you have can you have orthogonal in orthogonal? Like, if I have an orthogonal that's my arm? Can I have that broken down into orthogonal is that are my shoulder, wrist and elbow? Is that kind of this hierarchy? Okay,

so I think the short answer is no. The orthogonal is can be quite complex, and so you deaf We can have multi joint, real, you know, arms in

But one of the things that that that we did we being Pablo and I in terms of the robot, sorry, what? Oh,

sorry, I thought we were still talking about the robot for picking

things in Yeah, not for that robot in particular, but in, in SMAC, as a library and viewing smack in a historical context. One of the things that that we we did was, we took the orthogonal concept that David Harrell and Andreas Huber Dawn, put into, into, you know, instantiated essentially in boost a chart, and we simplify, and because up until that point, so I think the world of Andreas Huber Donny, but he didn't, he never really used the orthogonal and the orthogonal in a, in a, in like a context of a PC really don't make that much sense. And so we took them and began to use them, and we saw shortcomings with them. And so we, we simplified the orthogonal structure and, and then what we did was we added this, this architecture part of clients and client behaviors. So inside of each orthogonal, so visualizing, and orthogonal is a box, right, a box inside an eight box, there are generally two to further boxes inside that have a client and a client behavior. The client is has a lifetime of the entire state machine. So if you if you have something that you have some piece of code that you want to effect that robot in every state, then you just put it in the client. So this would be something like, let's say, you've got a connection to a robot arm. And if at any time you lose that connection, you want some event to be thrown, you put that in the client, the client behavior contrast,

oh, it's like a global state in a sense. Yeah,

there is there was a blow valatie to it. It's a different type of state, though, in that. Typically, in smack, we're talking about states in a control sense. The states that are stored inside clients are more like, like an image state. This gets confusing actually, what it's like when you hear about something like state estimation, it's more like an image in a sense, right? So that part is stored, whereas the control states are always explicitly controlled within smack the state machine back to the talk. So if you've got clients that are scoped to the life of the state machine and client behaviors, they're only scoped to the life of that state. And and you can have more than one client behavior at any one time. And I would say that that's where really the modularity comes in. That's basically how you interact with the orthogonal is through the client behaviors, the client that sets up, you know, would say, all the communication and all the kind of stuff that you want throughout the lifetime of the state machine. But the particular behaviors are in are in these modular client behaviors that

so the client is like, it's the manager of, but so how do you call it so you have the client and the client is in a? What was the word again, orthogonal? That so you haven't or thought so you have like a global state, and then from the global state, you have your orthogonal, your orthogonal is some piece that can be controlled independently of all of the other orthogonal, and then inside that orthogonal, you have a client, and that client is going to control that orthogonal to run several different behaviors given the state. Am I understanding correctly? Not the client

and the client behavior. But okay, what's the sorry? What's the client behavior? The client behavior is different in that its lifetime is scoped only to that state?

And how are we talking about states? And this You were saying something about images? And what is the state in this?

Yeah, we're talking about it thinking from a control structure. So going back from like, a global, yep, the state machine is essentially synonymous with the application layer. Right? So the state machine is your application. And there's one state machine on any one robot I mean, one time, okay. Many libraries have like where you start nesting state machines inside of each other. I don't, we don't do that with you don't like it. Yeah, okay. I think it it creates problems with data with routing the data and other problems. So, so you have a state machine that It is your application. And within that state machine, you have a series of states, and they can be grouped in different levels of or higher of hierarchy. So let's say let's say back to

that vegetable picking robot. Like that would be a nice concrete example, to talk about this. Okay.

Okay, yeah, we'll use that. And also, we'll blend in an example that I heard about it for the DARPA subterranean challenge. So in starting out, or in one of these teams that I've had the pleasure of speaking with people involved, they started out with a four state state machine. Okay, so four states. So in that, in that example, you would have one state machine with four states moving forward, nor hierarchy at this point, I

see, I see one of my points of confusion. So state in robotics, and like a control sense can mean like, the thing that defines where the robot is. So this joint has this degree, this angle, this joint has the next one. So I was trying to understand this, but you're using state as an in a state machine. So is a specific state that the state machine is in. And that's what we're talking about when we're saying stuck in this. Okay. So then we're saying we have a finite number of states, those control basically the behaviors, or we can do different behaviors in different states. And we move through from one state to another as the operation proceeds or something like this based on Okay, exactly, I see an ambiguity, okay, that I was confused by. But it is it's a confusing thing. And especially, yeah, we'll just have to say state for state machines or something and states for controls or something. And that will help keep me clear with it. Or I'll try to make it explicit anyways. Okay, so you have your state.

So this is back to this example, we have four states, right? So, and we'll just name them state, 123, and four. So in each of these states, going back to the example of that field, work robotics robot, we would have, let's say, Okay, we got one orthogonal for each arm, one for a base, one for the two gantries, that's for six, seven, and let's say one for perception eight. Okay, so in each of those four states, you're going to see eight orthogonal 's, that okay, inside you, each one will have a client in every state. And then, depending on what behaviors we want that robot to express, we arranged the client behaviors accordingly. And so it provides like a modular control structure for each subsystem of the robot.

Gotcha. So if you are, yeah, and the state

basically serves is like, this gets more into like the sub state architecture of smack, but those, the client behaviors, and the clients throw events based on what happens outside, you know, so smack is a single node, operate back to this example. Smack is a single process operating at runtime. And you're also going to have, you know, other processes on this system, you're going to probably have nav two, you're going to have move it. One with perception. And so when those other nodes communicate via let's say, ROS topics or services, the the SMAC clients will turn those messages, which you'll specify into events. And then those events are that's cool ordinated by the state, like, like a conductor would, you know, on this event, we transition to this state, at the end of the day states are a lot of people, you might you might incoming to state machines, you might think that states, that's the state machines are about states, and they're really not they're about events. And okay, it's through events that you begin the subject of states almost, it's like, it begs the question, you know, like, when you have an event, you have what was before and what's after, right?

And then can you define an event? Can you define event,

an event is general explanations. It's like some type of discrete.

So in ROS, it would be have I received this message or not sure. Is the event is the message occurring, this kind of thing.

are they action completing or something like that? So if I'm using naff to, and I want to move from one spot to another spot, the action completing of me moving to the To location could be the event. So it says I'm in this location event has occurred, I then occur.

And then when that event is occurred in some in smack in a state Yeah, you specify often where you will like what the next state you'll transition to is.

So are we always transitioning on events? Or should you always go to another state, or you can probably stay in the same state and have several events. I'm just trying to think of more complex behavior. But so are events always used? They're not always used for transitions, but they

not always but and here, we get into the minute you start using events, you start using more. One of the another one of the features we added to I'm saying these are things that I think are interesting, historically insensitive, like what's unique about smack, you know, versus a blue state chart. And what happened before we we brought in the concept of state reactors.

Let us define a few things just because there's a few things that I don't quite understand before we get into state reactors. So I don't quite understand the difference between clients and client behaviors.

Sure, it comes down to the lifetime client, oh, or lifetime to the state machine. And behaviors have a lifetime specific to that event.

Oh, okay. So so, so client behaviors are just for behaviors? Or how does it what why are they named that? Let me just see, I think my smoke alarm is gone, or some smoke alarm?

I do hear a beep beep. We just filter it out in the thing. So long as I don't smell smoke and whatever. Yeah. Okay,

you do get out of there. Yeah, for sure. Just gets into like the systematic architecture of code layout SMAC offers. So let's say you have something that you want that robot to do, like some type of event that you want to respond to in a particular way. But you want that to be throughout the like every state of the state machine, you put that code in the client.

I see. Okay, so and then. So you're kind of you generate these client behaviors, when the event happens. And that will take care of a specific, it's the kind of created and destroyed on a specific event, whereas the other one occurs all the time, I'm thinking of them, like, kind of simple, but for like web pages, or something like you always want to load this one thing, it's going to happen no matter what page you do. So it's like the header or something, that's always going to be the same. And then your client behaviors would be the very specific things that are on a specific route, in this case, yep. Is that a fair way to think about it?

It's not very good, and the only nitpicking thing I would have there is that it's there not so much. The client behaviors aren't created with events, they're created with new states. So one state to another, that's when the client behavior is created at runtime.

Okay, I see. Okay. And then you're gonna so there was another thing that you were going to introduce?

Yeah, state reactors. So this is another type of this is all part of the the sub state architecture. So like, once once we're in a state and looking down at the pieces inside of it, what state reactors do is so

what so in the sub state what composes a sub state? Is it so other states within states or are there so others?

We do have we do have? We do have hierarchy? Yep. But when when we're talking about the the, but to in the hierarchy, one thing that that is unique and smacker? Or maybe, I don't know, what's one feature? You we break down the library into having container states and leaf states and it's the leaf state that has the orthogonal and the full substate architecture, V. Okay. The container states don't by default. You can turn them on. It's smack we follow the philosophy of we do things in a certain implementation that you know, as we see that we think is best but we try not to turn off like other ways of doing things because some of the early clients that we had really challenged my views of how they would interact with the system. And I'm thinking to you, Bob peetha, gray, Orange. They, they did things that I, what, what are you doing? You know, it took me back. But, uh, but it totally valid way of running things. So we try to leave those those sorts of things open anyway, back to the hierarchy. So. So generally, there's always a leaf state that has orthogonal and other things like that a state reactor is is a just another one of these objects. And what it does is it receives events and throws its own event. And so it basically brings in second order events, the first ace, so we came up with it. This is in like, mid 2018. And we just begun the library in like, at the end of 2017. And we were contacted by the company here.com. And they were working on a self driving car at the time that it had a very complicated array of six, like super high end cameras, was for like a self driving car that would map you know, as well, and what, what they wanted it to do was, you know, navigate from one point to another, when it started up, it wanted to make sure all the cameras were ready to go, and then drive, if anything happened, stop, bring all the other cameras back on line and go. And that led me to create the state reactors. And so what we did is we have a state reactive called all events go, and it's a runtime object that resides inside the state. And it receives events from camera one, that camera to event and once it's received it from all then it throws its own event, and then the state transitions. It's become such a popular, I'd almost say it's a state machine design pattern. You know, we call it S T acquire sensors. And I don't know what percentage of state, you know, we have in the reference library, we have certain state machines that are extremely simple. It's just like this one back and forth. So they don't have it. But I think for all of the other state machines that we've like, if there's a simulation video, I'm pretty sure all of them have one of these SD acquire sensor states and very early on in the execution chain.

Gotcha. Okay, so you have those that will kind of take information and they take in all the events, and then when a specific condition is met, they may event they may emit another event, and that can be used to trigger some sort of transition or whatever might occur. Correct. Okay.

They also, one other thing too about him, that's I don't know, that might be interesting for the specialist out there. They they've set the stage for us to move ahead with cogeneration in a new way, one of the things that's always cool state machine libraries,

when you try to play code, or Yeah, well, boilerplate,

but also, like graphical cogeneration. That one of the problems that that has are always complicated, that is callback functions. But with state reactors, you avoid that by with, you know, through these second order events. And so yeah, and that's part of our plans over the over the long run, you know, to build out an entire tool chain along with the the ability to right now we have this Mac to runtime analyzer, but we want to create a smack to writer where you can, you know, graphically align your state machine and have it auto generate the code, or at least generate some of the code, you know, like, like a lattice framework type thing. And, yeah, the only thing about that is that, in my opinion, you have to do that last. Because if you do that too early, and you've got some problems in your, you know, in your architecture, you basically baked it in place. So, yeah, so that's one of the things I'm really hoping to accomplish with SMAC, in this current year is basically getting more and more use cases, to a point where it's like, okay, no, no, we've seen you know, we've got an idea about every which way that the library could conceivably be used. I don't know if we'll get all the way there, but,

but if you get a lot of them, then you have a fairly general solution, and then it can be used for a lot of things. That'll be great. Yeah. So one thing that I don't quite understand, well, I guess I'm having trouble mapping The idea is together. So I understand this architecture with the client behaviors and you have your orthogonal and you have your states and things like this. Can you explain kind of so like a simple application, say say like that video of the Fetch, sorting, the Fetch, in simulation, sorting cubes by color, okay, like, how do we go from these states? And then the orthogonalize in and then how, just like an idea of how all the systems work together to create some reasonably complex behavior? Or can be a simpler one, if that's kind of complex?

Yeah, maybe we'll start with the dance bot example. We'll kind of dive into that one a little bit. All right. Sure. In so for those who, you know, who haven't seen it, the best spot to see the dance bot example I can like it. Yeah, is in is in the the ROS world 2021 presentation on smack to It's the opening video how we link that all together.

And just to describe the video a bit, it's a robot that it's in a sort of maze, and it goes to a location and then it drives until it hits a wall, and then it goes back to its original location. And then it goes out again, but slightly different angle from where it was heading. So it makes kind of like a star pattern. And where it goes, and it bumps each of the walls around it, and then it goes back to its original location, and then it goes to some other location to start doing the same pattern there. And I believe the intention with this is to map out an environment.

Not exactly it's oh, there is there is a mapping that, that, that that does happen. But it's really more about floor coverage. We were modeling, you know, how how a robot might, you know, cover a floor space in in modular ways. In that one, the robot navigates to to global waypoints and begin, proceed to perform dinners of superstate this is, so now we get into hierarchy a little bit. So we've got our leaf state. And then you can have these containers. These are like levels of hierarchy, which we have shorthand for, which I'll go into right now. So the first level of hierarchy we refer to as super states. The next level we refer to as mode states. There's no difference,

per se, it's just because it's a hierarchy. So super states is like high level control. And that's like fundamentally what the robot is doing. It's mapping it's driving to some location.

In your application, I think I think the most common use for for Super states is for loops, for loops, or like, like sequences, you know. So in that application, it moves to the first global waypoint. Okay, it's to execute a super state, that is a radial pattern. And you can set the parameters of that radial pattern in the superstate. So I think in the video, it showed something like 16 rays,

and the 360 divided by 16. And then it just went out exactly what each of those angles,

okay, but you would just go in there, and let's say, put in 32. And so it's parameterised. Exactly, exactly. It does three of those super states in a row, at which point it navigates to a fourth way point, and we haven't execute an F pattern, same kind of thing, although it's like, I think it's a series of seven states that execute this repetitive F pattern. Again, though, designed for floor coverage. Then it goes to a final waypoint where it's more of an S shaped pattern. So five total super states and five global way points, all the super states parameterised. In this case, we're only using two or thought well, let me think there might be more than one orthogonal in there. But in terms of like the crucial orthogonal for that example, there's two, there's the, the the LiDAR, and to cool. Yeah, in that particular example,

and nav to handling the navigation of the robot from one spot to another, okay, and controlling the motors and everything to drive it around. Exactly, right. Okay.

Yep. Yep. In other in other state machines, we might use, you know, something like ROS 2 control the same kind of a similar way that that move it relies on ROS to control the move the robot. And we've had we've built examples and navigation in ROS, one where it uses the ROS 2 or the ROS control draft controller. But yeah, it's you know, you start using like your standard ROS stack to get around. Yeah.

And then so if we're in superstate one we're in the radial star. Okay, one. What does that look like from the mode states, so not the super states, but now we're in Super state one. And so super state one is composed of mode states, as you were saying, with the shorthand, yeah. So how does it look? They're like, what, what's going on down there? And how do we get closer to

the CLI that example? Like, in depth, or just ballpark? Yeah, I believe the way it has is that we've got a mode state called Run, and we have another one called recover. And where I see that architecture evolving, in going, you know, in terms of real world is that one of the things that SMAC offers is very advanced recovery sequences, and capabilities. So I imagined, and I think one of the things that SMAC one of the most important things that SMAC offers and in why it's disruptive, is that it allows for a new order of magnitude in the complexity of robotic missions, particularly age missions. So So for instance, let's let's take an example of a fixed wing aircraft, right. So you'd have one mode for taxing on the runway, another mode or takeoff, another mode for I don't know, you know, the your initial flight, where you're finding your right heading, stuff like that. And for each one of those modes, you're probably going to want to have a separate recovery state, you know, separate recovery mode state, where dependent, so let's say you're in takeoff, right, and something wrong happens, then you switch from that mode.

So the appropriate recovery for that will take

off recovery, and you begin your, your, you know, error recovery procedures there. And as that continues to expand, you just, you end up with like greater, greater needs of hierarchy. It also brings in the question of where your data goes, another unique feature of SMAC is that every state is a class and every container is a class. So you can scope your variables and other things to the lifetime. And that's needed. You know, perhaps you need a variable across every state in in a certain mode. Perfect, you put it in the mode, you can also do you we also have the opportunity, or the option of global variables. But

yeah, okay. So interesting. And so, can you tell me a bit more about the real? So you're saying, I guess, the origins of smack and how it was born, from, I guess, a real application and then how it's evolved since? Yeah,

it's in the oranges of smack it's hard for me to put a finger on it i i would say it started the same. So I graduated from UCLA with a degree in biochemistry. And one of the the, my first job outside of outside of university was under a NASA grant at the UCLA UCLA Medical Center in the department of Head and Neck Surgery under a brilliant researcher named Dr. Larry Hoffman. And I, I ran one of the research labs there. I forget the exact title, but that one of the labs was basically mine. And what we were doing was it was this insane program where we were dissecting the inner ears of we were, we were injecting, like dyes, and then dissecting the inner ears of mammals, mainly chinchillas and mice. And then we were studying the organelles of the inner ear, the you trickle and the sack you will and stuff like this.

I don't know anything about this. Okay. It's just an our little parts of the object. Yeah,

even then. And in dissecting these organelles, we would then visualize them, you know, under fluorescent microscopes, and imagine like, I'm holding up for the listening audience. I'm holding up like my hand, it's like an Organon. It's like they would, they would have these Nerf patterns that would kind of follow through these organelles. And we were studying those in an effort to create better rocket guidance systems, which I have no idea if it ever, you know, resulted in such things, I have a feeling gyroscopes do a pretty good job. But it got me it brought up.

So it was this this was like, fundamental. So like, one thing that might be similar is I think it's rapid random exploring trees, R R T, or something like this. Yeah. So it's, it's almost like an you're trying to identify an algorithm that controls these nerves spreading through them. And then maybe that algorithm would be useful in applications, like, guided missiles, or whatever it might have been.

Yep, that's exactly right. And that's, and so it opened, expose me to to concepts like this. neural nets was talked about. I got to program a little bit of Fortran. Also, decision trees were part of that. And it, it, it began an interest that I had, like, even in terms of office hours, where I just started, I started sketching out these kinds of systems, you know, like, and then, in looking back on it, it was clear to me that I was drawing out state machines, although my early my earliest state machines, it's like, each state was a neuron. And so I was imagining these chains of neurons and how they would process information. And you know, what, where that was just like, yeah. And so anyway, that was, that was where, like, some of my earliest interest in that arose. Later, I went, I went back to school to get a degree in math and theoretical computer science at the University of Irvine, and which I did not complete. But in some of the early courses, you know, I was exposed to set theory automata theory. And so I had

no, any idea what that latter one is. Oh, automata

theory. That's like, that's like the, that's like the math behind things like finite state machines. Okay. Yeah. You get there into like, whether or not state machines are Turing complete stuff, like

proving things about them. Theoretically. Yeah,

exactly. Particularly when you start getting into advanced state machines with event queues. You know, and like, how then how that maps so there was that that part as well. I moving on later, I remember I got there was a when I started believing that robotics was the next wave that it was in I believe this still that robotics is going to be the next boo and probably the biggest one in my lifetime. I feel like that's going to happen to within the next 10 years. And so as I was looking forward to that there was a book called UML of real time systems and I read that book like I I've gotten rid of it since but I don't know I could almost pry tell you on a particular page what's on it and sketching too at that time though, I although I was so interested in them I never would have thought that I could have written the state machine library like that I just I would have thought that well and at this time this is really before ROS you know so like the systems were more closed off I would have thought that that was part of some giant proprietary stack I see Yeah. Anyway my my first project you know that involved ROS was creating what the company is real robotics and we were creating level line cable reels that we're going to provide mobile robots with tethered AC power for which you know more

It's a level wind meaning level wind level meaning that if we've got a real here it goes like a fishing exactly like a fishing line fish yeah, there's like a goes like this. So it winds perfectly every time. Cool. And I've got a picture too of like what, what we had in mind for the system holding it up for the camera right now and it's on top of ClearPath robotics Ridgeback robot so you can see the box that it is the real there's a yellow it's meant to be Like a nine gallon Shopback that goes on top in this application here, I'll show it from another angle.

Okay, so it's, it just looks like. So it looks like one of those small clear paths, or maybe not small, one of those clear path robots that has kind of the wheels underneath. And then there's a big cube on top of it, and it kind of looks like there's so I guess that's the Shopback cylinder that sticks out of the cube.

to get started with this. So the problem was to do level cabling. So you wanted to basically have the robot drive, and it would leave a cable behind it. And then you could wind back on the same path, and it would organize that cable as it retracted it, basically,

which, which required tight coordination with navigation. And, and then also tight coordination with what would be really the state machine of whatever the plan was to basically sell those to, to OEMs. You know, and so we knew

that OEM is original equipment manufacturers. So let's say so that's like,

yeah, go ahead. Well, yeah, somebody like Dyson vacuum or something, you know, more probably more industrial, something like 10 internal fills, but But yeah,

whoever was going to meet or know them, but I see. Yeah.

And so we knew that we were going to so we built, you know, we started with that software stack. And we moved all the way up. And we really did some beautiful work there. Some notable people to contributed on that project, Dave Coleman wrote something Oh, project. Yep. Yeah. He was this was before picnic, he was a graduate student at University of Colorado Boulder, and he wrote some of the original ROS control code. Vince Magyar, also worked on the project. He's the maintainer of ROS 2 control. A mathematician by the name of ROB Hanna from UCLA. And I did some, we mapped out all the mathematics where, imagine that level line system, you know, the goal is to know exactly how much cord is being produced with every encoder tick, but it's changing all the time, depending on how many, you know, layers of cord are on the real position of the level line carriage where the robot is the robot rotating. And so we did all that. I focused on the microcontroller code mainly. And that's kind of how I that that was more my background is I came into smack I was more like an embedded programmer. And that time, I thought that C 99.

That was my belief in anything involving templates. I looked it like, you know, I didn't like that at all. Yeah, Pablo country project too. And there was also a developer, from the former ClearPath developer named Devin Nash, he did some really great work on the project, too.

Anyway, it's cool to see. It's funny to me that it's such a small community, in a sense, as in these names keep popping up for a lot of endeavors. Okay, so you worked on this robot. And so this was with real robotics. And this was the application. Yep. And smack was born from this.

Correct? Yeah. Because we knew that we were going to have to integrate with whatever the OEMs were using for their top level coordination, because you have to coordinate you'd have to coordinate the movements of the reel with the navigation, you know, the navigation of the robot, which we accepted. You know, we knew we were going to have to do custom work like that. But in order to, you know, develop our own demos, the idea of a state machine came in, also early on, I'm going to say so we started this in like 2015 2014 2015 by 2016, maybe even a little earlier, I realized that smash was not up to the task,

Oh, yes. Smash was smashed was a state machine library developed by Jonathan Bullrun, I believe in 2009. And it was a state machine that that powered the PR two examples. Yeah. I think very highly of that library. I think he did a number of very, very innovative things that that inspired SMAC probably the most important was Tying the events to actions, you know, like, like, like, like in the example you were describing to navigation finally gets to a goal that, you know, that then creates an event and you transition to the next state. Exactly. There were things though about smash that never fully. So I want to give credit to Jonathan Boren like for like, it was a, it was a creative and in essence a leap forward. I think in terms of though, like, it was never really suitable for like more of a production level of robotics. The concurrency always had issues, the data porting is another subtle problem. Anytime you see, in my opinion, anytime you see data porting and state machines, run,

1 post with the control tag