22. Ocean Monitoring and Climate Change, with Ben Raanan
In this episode, Audrow Nash speaks to Ben Raana, who is a software engineer at the Monterey Bay Aquarium Research Institute (MBARI). MBARI uses robots to monitor and explore the oceans to better understand climate change and other topics. Ben speaks about how robotics is enabling ocean research, the challenges of putting a robot in the ocean, how their robot platform works, and how they are working towards long-term deployments.
Links
Comments
Outline
- 0:00:00 - Start
- 0:01:15 - Introducing Ben and MBARI
- 0:06:16 - Understanding human impact on the ocean
- 0:17:24 - Using robots for scalability
- 0:32:38 - Collecting ocean data
- 0:49:19 - Describing their ocean robots
- 0:51:49 - Power management
- 0:53:52 - Varying the buoyancy
- 0:55:57 - Their robot’s cost
- 0:58:09 - Communicating with the robot
- 1:00:49 - Failure management + safety precautions
- 1:05:32 - Growing the fleet
- 1:14:06 - Simulation with Gazebo + open source
- 1:26:42 - Future of missions
- 1:32:12 - Advice for those looking to contribute to ocean research
Transcript
The transcript is for informational purposes and is not guaranteed to be correct.
(0:00:02) Audrow Nash
This is a conversation with Ben Raanan who is a software engineer at the Monterey Bay Aquarium Research Institute, or MBARI, for short. MBARI uses robots to monitor and explore the oceans to better understand climate change, and other topics. In general, getting ocean data is extremely expensive. Depending on how far the area of interest is from the shore, it may require a large boat with a full crew. MBARI is working to make this process easier and much cheaper by having robots stay at sea for weeks at a time getting data on different ocean events. This generates a lot more data and will hopefully greatly improve our understanding of our oceans. I found it exciting to talk with Ben about how MBARI is looking into having robots stay at sea almost indefinitely, charging their batteries and uploading their data all while at sea. This could enable fleets of robots to continuously be at sea giving us a much richer picture of the health of our oceans. I'm Audrow Nash. This is the Sense Think Act Podcast. Thank you to our founding sponsor Open Robotics. And now here's my conversation with Ben Ron on. Hi, Ben, would you introduce yourself?
(0:01:18) Ben Raanan
Sure. Hi. My name is Ben Ron on a oceanographer and robotics software engineers here at the Monterey Bay Aquarium Research Institute. Or we're in MBARI, for sure. We're in Barcelona, California. I've been here with the Institute for probably coming on years now. Working as an engineer, a bedroom about my background, actually, motion career about 20 years ago, I was a special warfare operator and a combat diver in the Israeli Navy. And spent a good chunk of time with the teams there. And later on, got my undergrad in r&d tax and biotechnology back home in Israel. And then came here to get my Master's in physical oceanography. At that point, I got involved here with the Institute through an internship, and then I just refused to leave. Yeah, so now mostly what I do is I work development and operations with autonomous underwater vehicles. And so I have a few projects running, but I think you're interested in looking for talking about
(0:02:30) Audrow Nash
awesome. So what kinds of projects do you guys do at MBARI?
(0:02:37) Ben Raanan
Yeah, so I guess maybe we before we dive into the projects, can tell you a little more about embargo in general, I think, sure, you may be familiar, but others may not. So like I said, we're in Moss Landing Fornia in the Central California, say probably about an hour south, from San Jose, on the coastline, in the Monterey Bay. And in MBARI is actually a pretty unique Ocean Research Institute, in the sense where we're a private nonprofit. So the institute is not an academic Institute, it's kind of been founded as an alternative to what an academic Institute can be for ocean research, in bars funded almost exclusively, through the David and Lucile Packard Foundation. And David Packard, which is the P and HP and Hewlett Packard, he founded this, yeah, he founded this institute in 1987. So we just celebrated our 35th birthday. And the mission back then was to, like I said, offer an alternative to traditional academic research institutes, and take on challenges that require development of technology in order to, you know, answer some of the scientific questions that marine scientists have. And so that's kind of been the goal of Institute and also remove a lot of the, I guess, barriers, especially when it comes to funding. And so scientists and engineers here get to focus more on the doing and less than they're proposing. And so we have an internal proposal cycle, in which we, you know, for our budget, and then we get funded through Packard Foundation, which is phenomenal, it's absolutely great. And so, the type of the way the institute is structured is we're actually kind of like a three legged stool. And our trifecta, never skip on an opportunity to say that word. And so, so, the Institute has about 220 employees. We are divided into science, engineering, and Marine Operations. And we work very closely together. So Though the barrier or the the boundaries between the groups is very blurred, you know, you got a lot of scientists that are very much engineers and engineers that are scientists and they work together to find the questions that need to be answered. And then develop the technology that allows us to go out and get that. And then the Department of Marine Operations has the specialty in the knowledge and how to deploy the assets and manage, you know, so you know, make sure everything comes back safely, to the best of their ability. And so that together is what I think really makes the institute special in its ability to come up with very challenging questions, develop the technology to try and gather the information that's required in order to answer those questions. And then actually go in the field, gather the data, and and you know, start working on the answers. And so that's kind of like a cycle that we exercise here pretty regularly. And it's it's all good fun. So I guess that's the institute is the that's the answer to in in in a nutshell.
(0:06:16) Audrow Nash
And what kind of problems are you guys typically like also the in Hewlett Packard, so Packard set this foundation, or set this up? So he received or he sends you guys a good number, a good amount of money? To do the research? What kinds of problems? Are you guys most interested in? Solving? Or, like, what what kind of problems are you going after? I guess, yeah.
(0:06:44) Ben Raanan
So if you walk around the hall here, you'll find scientists pretty much any ocean research discipline that you can imagine. So we have geologists, geophysicist, chemical oceanographers, biological oceanographers, any kind of type of marine research really, that you can imagine, is happening here. And so at the core, we're trying to describe the ocean environment and go after questions that also have to do with anthropogenic impact, that's driving change in the ocean environment,
(0:07:23) Audrow Nash
what does that mean? So generic, is that
(0:07:28) Ben Raanan
is human generated? So, yeah, so that really relates to some of the challenges that are faced with climate change, and how emissions for example, impact our environment. And so, the ocean is actually a really important piece and buffering the heat on the planet. And so, in order to understand better the processes that govern heat transfer, how carbon dioxide that gets emitted into the atmosphere is gets absorbed and back in the ocean and and buried in the in the ground, that's, you know, the carbon cycle. So, the Institute is trying to target research in order to acquire data and figure out how what governs these processes, what are the rates, and then through this research inform larger models that have to do with you know, climate change and more of a global scale in the global scale sense. So, those kinds of questions they really drive research in all the layers of the ocean. So, from the atmospheric to you know, the upper water column, where the gases kind of mix and then how the biology interacts with the different chemical properties of the water, different physical properties of the water, they take in some of this carbon and use it to build their bodies right through photosynthesis, and that later on feeds the larger organisms and cascades through the foodweb and eventually, those kind of fix organic compounds are either pooped out or die and sink to the bottom of the ocean right and get buried there. And so that okay, is the carbon cycle in a nutshell and we focus on research that tries to describe each of those sections in detail, and then
(0:09:46) Audrow Nash
yeah, gotcha. So, you have just to see if I understand the carbon cycle, you have carbon in the environment, then you have small like algae or something, some small things. that will consume the carbon and sunlight in this kind of thing and the photosynthesis happens, then they go through this whole process where they're eaten by fish or something. And somehow, they like eaten by one fish that small, a bigger fish eats that whatever, eventually something dies in there. And then it sinks to the bottom of the ocean. And what happens when it's at the bottom of the ocean? It kind of just stays there for a while. And you get kind of like, on the bottom of the ocean, or
(0:10:31) Ben Raanan
Yeah, so it gets it gets vented is sediment, and ocean sea floor, really is, I guess the storage of that moment, until, you know, either gets, then you're starting to look at geological processes on you know, what happens there? How? Yeah, so how does the ocean seafloor evolve over time? And when does that carbon make it back into the atmosphere? And so I guess the point is, yeah, so we describe a lot. And I guess the point to really kind of take away from this is that these are processes that happen over many, many, many timescales and over very different, yeah, are very different spatial scales as well. And so at MBARI, we kind of talk about it in the sense of, you know, going from molecules and atoms to ecosystems and back. And so the type of science that we do hear goes from, but understanding the microbial processes that you know, what type of nutrients are required in the ocean in order to make these biological plankton or bacteria flourish and take in these gases, then, all the way up to how does it later cascade into the food web? And what are the timescales that are associated with those things, and so that, on its own poses many challenges, because in order to collect relevant information, to describe these processes, you really have to be at sea a lot, and have to be there not only for the duration, but also in the right place, and cover and span, you know, the area that's required in order to to really capture those fundamentals. So
(0:12:35) Audrow Nash
if I understand correctly, it sounds like you guys are basically trying to model so the carbon cycle and what's occurring to accurately model that and have a better idea for climate change, you need to be involved at something through the whole spectrum, which is a massive process that happens everywhere in the ocean, at all different times, with all different creatures, there's lots of things involved. And so because of this, this makes it so you guys have a very large research mission. In a sense, like, there's a lot of things that you need to understand, or there's a lot of monitoring that needs to happen. So that you can begin to refine your model. So you can have a good idea of this process, and maybe its impact, or maybe its changes over time. And these kinds of things.
(0:13:27) Ben Raanan
Exactly. And the idea is that eventually, those type of this type of information, can cascade into decision makers for conservation management, and other type of, you know, decisions that impact our environment, but also can affect some of the likelihood, for example, fishermen, and so on.
(0:13:49) Audrow Nash
So that sounds incredibly lofty as in a huge, huge challenge to basically take on climate change. Is that motivating everything at MBARI? Or are there other problems that you guys are interested in to? Like, clearly enormous? And that's is that everything under that umbrella? Or are there other umbrellas?
(0:14:09) Ben Raanan
Well, there are other themes as well. I mean, you have researchers here, for example, that are behavioral ecologists, and they're interested
(0:14:16) Audrow Nash
in I don't even know what that means at all.
(0:14:19) Ben Raanan
Oh, they're, they're interested in in describing, you know, how certain species behave under underwater and so. Right? So you can imagine, for example, if you're interested in studying squids, especially if you're interested in studying DP squids, then, again, getting access to to the environment where those squids are, is an enormous challenge. And one that's very suited for institute that is, you know, capable of developing the technologies and also executing the operations for getting the right information and so on. You know, that's another type of research that we do here. We also have a whole mapping division, we use vehicles to go in and do surveys of the ocean seafloor and map the ocean seafloor, and then return to see how it evolves over time. So that's a whole kind of branch of geology that we host here. So really, it's it's very hard to try to capture all the activities that go on here and the Institute, I'd say, climate change has definitely been one of the main themes and impacts of climate change on the ocean environment has been one of the main themes that have governed a lot of the proposals and the projects that have been here. But it's not the only one, you know, they're there. Other projects that, you know, maybe don't fall exactly under that umbrella. But that is definitely the the lead theme.
(0:15:55) Audrow Nash
Gotcha. And largely ocean related. I mean, well, entirely ocean related, it sounds and then it's also, I mean, if you have better ideas of how these creatures behave, maybe you have a better idea of how they are as actors in the system, which helps you model these things. And if you have better maps of the environment, perhaps that also helps you in environmental monitoring, I could imagine, which will go back into climate change, in some sense, because it's such a large problem, that having an accurate model has so many pieces to
(0:16:27) Ben Raanan
correct. And then I guess I'll add that embargo has been a major player, you know, since the 80s, collecting this information. And so at this point, we've gathered enormous quantities of data. And we are working pretty hard to make this data available for others to use, and also process it here and have that multi-decadal perspective, in order to quantify change in the environment. So in other words, we are establishing a baseline starting, you know, from some point, and looking and observing how the environment changes over time. And so another big mission of embargo is to, you know, safeguard this trove of data and make it accessible. And so I think that's another thing that the Institute is unique in its capacity to offer. So
(0:17:23) Audrow Nash
that's really cool. Now, so that's awesome. What are some of the ways that you guys have been using robots and these different parts of kind of the mission of MBARI?
(0:17:36) Ben Raanan
Right, so um, you know, we refer to oceanography here as a contact sport, like I mentioned, in order to gather the data, you have to be in the environment. And so as you know, traditionally, oceanography for decades and has been done using ships. And that's great. I think that ships and humans are always going to be part of this type of research. And inquiry, indeed, operates to large ships that go out and collect data and take expeditions at the sea. But I think there's also a realization that, you know, because of these temporal and spatial scales in which the phenomenon that we're interested in studying happen, we're just not able to
(0:18:27) Audrow Nash
get all the clients to be everywhere all at once, for sure. So it's much easier to have robots go do it, for
(0:18:33) Ben Raanan
sure. Not impossible, but it is cost prohibitive.
(0:18:35) Audrow Nash
Yes. But that also human error and all sorts of other things, too.
(0:18:42) Ben Raanan
Sure. I mean, robots make mistakes, also, especially people that operate them. So, in fact, because, you know, traditionally, this type of research has been done using chips. MBARIs also put a lot of resources into calibrating the measurements. Okay. So you've done it collected data, in a certain way, for a certain number of years, and now you're switching into a more automated collection system. And so how does that data compare? And what are the calibration factors that you need to apply to go from, you know, manually collected systems into using data that's collected by automated systems. And so that's, so making
(0:19:27) Audrow Nash
sure that we have consistency from humans taking it too. Maybe if you can automate it and have a robot do it, you want them to be apples to apples, not apples to oranges. So you have this big break? where the data is, like, completely different and hard to draw conclusions from?
(0:19:44) Ben Raanan
Exactly. So you don't suddenly think that there's, you know, some kind of climate crisis that happens to coincide with the time you started using certain robots. So so we've been seeing through this transition. Now, some of the environments that we researched which are very hard to get to. So, for example, kind of the average ocean depth here in the Pacific is about four, four and a half kilometers. So, so just those are
(0:20:12) Audrow Nash
four and a half kilometers. That 45 football fields deep, roughly or,
(0:20:19) Ben Raanan
rightly so. Yeah. And a lot of weaker but yeah, sorry. Yeah, so so so that's very deep. And when you think about, for example, the type of pressure Vironment that we live in. We live at one atmosphere of pressure, if you go to space, you're almost at zero. But if you go about 30 feet down into water, then you've already doubled the pressure. And every 30 feet, you add, you add a whole nother atmosphere. So
(0:20:52) Audrow Nash
those many atmospheres are down there, then
(0:20:55) Ben Raanan
we'll have to do the math.
(0:20:59) Audrow Nash
So 4.5 kilometers down, correct.
(0:21:02) Ben Raanan
It's one atmosphere per 10 meters. So if you have 4500 meters, then you'd be at
(0:21:14) Audrow Nash
divided by 3030 feet per or 10 meters is one atmospheric, correct?
(0:21:19) Ben Raanan
Yeah. So 450 atmospheres, I believe is the number.
(0:21:23) Audrow Nash
Yeah, that's an easy one. Yeah, doing math doesn't always
(0:21:27) Ben Raanan
work. But especially not not. Well, you're being recorded.
(0:21:30) Audrow Nash
Not well, sir. So 450 atmospheres?
(0:21:35) Ben Raanan
Correct. So I guess the pressure is immense. And so in order to take humans into those kinds of environments, and support life, human life, in those environments, you know, it comes with a huge overhead. And it's not risk free. And there are systems that do that. You know, we have deep sea sobs. Woods Hole Oceanographic, I believe, operates. Alvin, which is a human submarine that's able to go to those kinds of depths and and collecting through there. But it's really scalable. And you say,
(0:22:08) Audrow Nash
on average, just average depth is that much, right? What's the what's the deepest? Like, I guess we probably don't have a very clear idea. But what so we do. Okay, yeah. So how far down does it go?
(0:22:21) Ben Raanan
It's over 11 kilometers. taller than the mount, the tallest mountain, right? So and that's 1100. atmospheres. Correct. And I believe that's the Mariana Trench, off the coast of Japan, which is, so yeah, it's pretty deep, but humans have reached there. And vehicles have gone down there as well. And so it's not out of their reach. So it's not very scalable. Right, I guess it's the point. And, like I mentioned, again, a lot of the phenomenon that we're also interested in, in studying are very patchy and episodic in nature, meaning, you can imagine the wind blows a certain direction, the currents do a certain thing, the nutrients emerge, and then suddenly, boom, that creates an opportunity for something to occur. And you're either there, or you're not. And sometimes you can be, I'm sorry,
(0:23:19) Audrow Nash
give me an example. Like what might an event that you're trying to see occur be like an algae bloom of some sort, or
(0:23:27) Ben Raanan
Yeah, that'd be a classic example. And so typically, the way this occurs is winds blow, they push coastal waters away from shore, and pull in deeper waters that are much richer in nutrients. And that allows for this community to community of microorganisms, plankton, and phytoplankton, to really flourish. And that mostly happens on the, you know, upper part of the ocean where there's access to sunlight. And so when those conditions come together, and those blue dunes are, you know, you're either there to catch it, or you're not. And you either have a platform that's able to go out and collect this information, or you don't. And so, I guess, once that window of opportunity closes, and typically that's on the order of, yeah, weeks, spanning 10s of kilometers, at least in the coastal environment. Yeah, that then your window of opportunity is is gone and you've lost your chance to collect the relevant data. And so having having autonomous platforms that are able to persistently be in the ocean and collect information is a huge advantage. And so we kind of segwayed away from the deep ocean back, you know, to the upper water column and we operate automated In automated systems in both, but you either need a very a system that's capable of operating into depths and in those deaths, or you need a system that can operate up in the water and the upper water column that's, you know, capable of having that persistence and endurance to cover, you know, those, those those areas. And so in both cases, that those type of problems lend themselves very well, for automation, and for use of autonomous platforms. And so MBARI has developed. You know, at the beginning, we were very focused on getting to those steps. And so MBARI was one of the pioneers in developing remotely operated vehicles, there was a big kind of cube like, vehicles, remotely operated vehicles are over. Okay. Which, yeah, and so, those are tethered vehicles, and they're piloted remotely from a ship. So those still require a ship to be in the area. And then bears
(0:26:05) Audrow Nash
able, correct, yeah, going from the robot to the ship. Yeah. Yeah, correct.
(0:26:11) Ben Raanan
And so in borrowing has been invested heavily in developing those kinds of debit technologies, and also coming up with ways to manipulate, you know, the environment there with robotic arms, and so on, so forth. And then more recently, we focused on using autonomous vehicles to extend the duration. So cutting the tether, focusing on platforms that can sustain themselves in the environment, with a remote operator that only supervises periodically. And that really gives you a ability to design without counting on having a ship in the area. And B, it also opens the window for scalability, right? So
(0:27:04) Audrow Nash
because you don't need that big stack, to maintain and be running, like you don't need a whole ship, going to the one location because you just have a bunch of these making their way around the ocean, monitoring things
(0:27:16) Ben Raanan
correct. And you can also have an operator that rather than being fully engaged in operating a machine is now checking in periodically just to make sure things are happening as they should, and maybe providing instructions for correction. And so
(0:27:29) Audrow Nash
the grocery store checkout model is everywhere, these days, for this kind of thing, where you have like the one person watching the 10 machines that were it's just it's so interesting to see it here to where you have one person kind of supervising, but not paying attention all the time to all of the robots in your groups, but just watching a few that periodically come in and out of contact, and they manage maybe giving high level directions or something like this. It's so interesting, this is becoming everywhere, it seems.
(0:27:59) Ben Raanan
Yeah. And I guess the so there are some aspects of doing marine robotics that I guess they're very different from terrestrial robots, or aerial robots in a sense. So I think, you know, the underwater environment is a communication denied. Environment, meaning, you know, once you go on your, the radio frequencies that we use for localization and radio in between our vehicles diminishes, at that point, we're only the only chance to our only option to communicate with the vehicles mystically using sound underwater. And that presents a whole nother set of challenges. And that technology has come a long way. But it's still very limited, especially compared to way lower bandwidth
(0:28:44) Audrow Nash
I would imagine. Correct. So you're, you're talking about far less information can be sent,
(0:28:49) Ben Raanan
right bits per second, and then three limited in range with typical range, you know, I'd say a few kilometers, is kind of typical. Yeah, so So the vehicles really need to be self sustained. And they need to know what to do and operate on their own. And so that that's a challenge. But at the same time, you know, as humans, we're not really able to be in that environment, it's hostile for our bodies. So any kind of system they can put out there and can execute a task that's useful is going to be an improvement of what you know, on top of what a person can do. And I feel here in terrestrial robotics, we're always up to your challenge, because, you know, you got to do as well as something that as well as a human where you're already in an environment where humans can function fairly well. So so we have it, you know, many challenges of one one had but then, you know, in some, in some ways, it's kind of easier to make fast progress and push the needle further than what you would be able to Have you used in environments where humans are capable?
(0:30:05) Audrow Nash
Yeah, because it's so cheap. So I'm a little confused. But it. So you're saying, because these environments are so challenging to get people out there, and there's budget restrictions and things because like, you can't have a ship go to each location you need, or you're interested, but you can have a robot that kind of manages itself much more cheaply. And so because it's so hostile, and because of the scalability problem, there's a lot of room for advancement with these robots that are going at the kind of the shallow depths and monitoring things autonomously.
(0:30:42) Ben Raanan
Correct, or? No? Yeah, I mean, there's, there's just pretty much any kind of, if you can get a system to execute any kind of task out there at sea, you're already pushing the needle forward, you're improving over what a human can do. And so and so I guess that's one advantage of doing marine robotics, but it might be the only one, because it really is very hard. It really is. Environment.
(0:31:09) Audrow Nash
Oh, yeah, totally. I imagined it's like, almost, if not as difficult as like putting a robot in that, like, if you put a robot in space, it's like tremendously difficult to get it up there. But once it's there, and it's in a harsh environment, it's that's a big challenge, for sure, like putting a rover on the moon, or putting a rover on Mars or putting something in space. Like that's tremendously challenging, given the harshness of that environment, and that the remoteness of it. But this is kind of similar in that the environments like water is really powerful. It's very cold, it's very rough. There's like, I don't know, bunch of creatures all around, I don't know, I'm imagining like,
(0:31:48) Ben Raanan
we did have vehicles in the past. So vehicles have returned with shark bites in them and teeth embedded in wirings. But that's actually a really good comparison. But I guess I would argue that, you know, because of the pressure and space versus pressure at sea, we've already talked about that. So we call that yeah, you need to be able to sustain those pressures, which, you know, is a challenge, but also, even a vehicle on Mars. Sure, you have a 20 minutes to 20 minutes back delay in your communication, but you're still able to communicate with the vehicle at all times. Versus Right, exactly. Versus underwater platform, where once the vehicle is under you typically lose contact. So yeah,
(0:32:36) Audrow Nash
that's very interesting for that. It's definitely a very hard problem space. One thing just to give it because this is one thing that I don't quite understand yet. What kinds of things because these robots are out there, and they're monitoring something. But what kinds of data are they collecting? While they're out there? Like, specifically? And I'm sure there's a lot of things you could look at, but like the nature of the type of data, you're collecting, something I don't yet understand?
(0:33:02) Ben Raanan
Sure, yeah. So we have platforms, different platforms are geared toward collecting different types of information. And so if you're looking at an ROV, that's a remotely operated vehicle that has the option to manipulate. And so you little hands, right, exactly. Okay, so you can do specimen collection, for example, at the deep sea. We also use those type of platforms to do surveys of the seafloor using video multibeam sonars. And also for mapping and also, now, LIDAR. And
(0:33:43) Audrow Nash
that works under I mean, I guess it makes sense that it works underwater. I imagine there's a new set of problems and challenges with that.
(0:33:49) Ben Raanan
Yes, it's very challenging. I'm actually not working on those projects. But we have a whole team here that's developing that capability. And it's pretty cool, actually. Oh, yeah. So so they've developed actually this really cool device. It's, I can share a video later. But it's an articulating sled that connects to the remotely operated vehicle. And it allows the vehicle to conduct a survey. So you can imagine it flies over a constant altitude. But then regardless of how, with the bathymetry, that's what the bottom of the ocean. So for example, now you're coming up against the wall, the sled will articulate itself. So the sensing is always pointed directly at the seafloor. And so that allows you to get continuous surveys regardless of what the topography is doing underwater. And with LIDAR imagery and sonar, you get very high resolution data. And so you can start describing species to the genius level using imagery. Cool. And then by returning to those points, you're able to track how you know things evolve over time. And so that's something we do and Those, that's one type of information that we gather and bury those images and videos that we collect are one of the biggest, I guess, databases that we have is those videos that these vehicles collect. And we have also some projects going on here now to expose some of that information and make it bring it to a format where you know, it's right for automation. And so so that's one type of data, then we have two autonomous vehicles that also carry those deep sea vehicles. So also geared towards mapping as a primary task. And they also carry these big sonars, and are able to do the surveys along the sea bottom, at depth, and retrieve, you know, high resolution maps of certain areas. And so that's another type of data that we collect. And then, in addition to that, we also have what we refer to as oceanographic sensors. So most typically, there's going to be sensors that collect temperature, salinity,
(0:36:10) Audrow Nash
you know, that's how salty it is. Right? Correct. Second one, salinity,
(0:36:14) Ben Raanan
salinity, correct. And the way Yeah, so that's kind of like your, your basic, but then you have also specialized sensors that are able to tell you the concentration of certain nutrients in the ocean water, for example, like nitrogen, nitrates, sensors, things like that. And then we also have even more specialized sensors that are able, for example, to sense the type of bioluminescence. So they have a chamber that past Yeah, so so you can see how much bioluminescence occurring in the body of water that you're traveling through. And so that's another type of
(0:36:53) Audrow Nash
does that mean like an algae will produce light at night? Or something? I said analogy, or would it be like fish in a really deep ocean? Or both for?
(0:37:02) Ben Raanan
Yeah, so they bioluminescence is light emitting organisms. And so that actually happens in a very wide range of species across the entire ocean. So it's pretty ubiquitous. And we have a lab here that studies, you know, how it happens. And what are the evolution pathways that lead to different types? That's really cool. Yeah. So so that's another type of, you know,
(0:37:32) Audrow Nash
is it often in deep sea animals? Or do you see it in like surface algae? Or like, I don't know if like the ocean lights up at night? With algae? I could imagine that. But I wonder if it's true. No,
(0:37:42) Ben Raanan
it is true. So larger organisms that that have bioluminescence are typically found deeper. And they use bioluminescence for various reasons, it could be for attracting a mate for trying to capture prey for deterring prey. And you see different patterns and different colors being used for different purposes. In the shallower water, there are many types of algae that have bioluminescence, for different reasons. And we actually have a project going on here to try and identify what type of allergy you're observing just based on the bioluminescence patterns. So yeah, so it's actually pretty ubiquitous in the ocean and happens in all kinds of
(0:38:36) Audrow Nash
just out of curiosity, like, I don't really know anything about this, but why so I can understand animals having bioluminescence? Because they are, like, as the main reasons you've mentioned, hunting, or meeting or something like this, like it's beneficial for those reasons. But plants. I can't like algae is a plant to me. And I can't understand like how that would affect like, if there's one algae and it has a beautiful glow, it doesn't probably help that algae meeting with the other one, or maybe it's some reactive. What's the what's the reason why algae has bioluminescent glows?
(0:39:13) Ben Raanan
Well, I don't have a straight answer. There's
(0:39:15) Audrow Nash
many. Yeah,
(0:39:16) Ben Raanan
there are probably a few. It could also be for prediction avoidance. If they all fluoresce all at once,
(0:39:24) Audrow Nash
I guess it looks like we are waste or something at some stage.
(0:39:29) Ben Raanan
If you try to imagine trying to capture one little particle, but suddenly, a light show blows up in front of you. I think it would be pretty hard to focus just on one single cell. But I'm not sure that's really the reason I want to get back into that
(0:39:47) Audrow Nash
being some sort of thing. It's avoiding predators. I just intuitively I don't I don't have a good model for why that would be the case. And it's fascinating that that exists.
(0:39:57) Ben Raanan
But right, yeah, yeah. And so Steve haddock, Dr. Steve had a career in MBARI. He leads research here on bioluminescence. And so we can go ahead and look up services research and probably get some answers. But he's, he's great.
(0:40:15) Audrow Nash
So, those sensors to your bioluminescence sensors, and you mentioned a whole bunch of other ones so salty and temperature and different nutrients correctly. So those are the kinds of things you're collecting data about. And I guess, also the video one. So if you if you see an algae bloom, what kinds of things are you trying to sense about that, I suppose the nutrients is how salty it is the temperature, but then also maybe video of the event, and then or
(0:40:46) Ben Raanan
Well, yes, for a biological shark for studying those type of microscopic organisms, we do have some imagery based payloads that we use in order to try and identify species.
(0:41:00) Audrow Nash
So you mean like various cameras, camera setup, or something that you include on the robot. So you have your various payloads that have various sensors that you might include on your robots,
(0:41:12) Ben Raanan
okay. And in addition to that, we also have the ability to acquire water samples autonomously at sea. And so we have a whole lab here, that's actually, that's their focus, and they've developed a payload on the environmental sample processor. And this is actually probably the most advanced type of it's not really a sensor, but it's more like a payload we refer to it, it's able to take in 60 water samples. So you can imagine some kind of like, a system that has an intake and is able to filter water, while the vehicles deployed at sea. And then also has some automation that's built in to this payload, to, you know, use the right reagents in order to fix fixate that, that sample. So it doesn't continue to change after you've filtered the water. And then that system also how that would
(0:42:12) Audrow Nash
be like if you had some sort of bacteria in it, or something that were like continued to do, whatever. So it like, you'll basically sterilize everything or kill everything in it. So you can look at its components. And then you can easily subtract that reagent out when you are doing your tests on it. So you know, kind of your baseline, and you can see what it was at the time. Ah,
(0:42:36) Ben Raanan
exactly correct. And then these payloads also have some analytical modules that are able to tell you things about the sample that you just acquired at sea. Cool. And so at the extreme case, and this is still hasn't been miniaturized to the point where it runs on a vehicle, you can run this type of sample and actually target specific genes. So do almost like a genomic analysis on the sample that you just acquired, and radio back to shore.
(0:43:10) Audrow Nash
What are the genes of what is it literally genes of something? Are you like getting an algae sample and looking at it streams? Or what is it the what? Oh, it's okay. So it's an algae that you're synchronizing effectively. That's super cool.
(0:43:24) Ben Raanan
Yeah. And then that allows you over time and space to see, you know, what was the? What was the community, the microbial community? What was it looking like? And over time, how is how has it evolved? And that, you know, is, is really useful for research and understanding how things change over time and with conditions. So as conditions change, how does the microbial community react, but also, it's very useful for monitoring. So you can example for example, you can have, you can imagine seeing a system like this deployed somewhere, and looking for a certain species of algae that's harmful for the environment, because it produces some toxin. And that's kind of like an early warning system. So again, geared toward not only doing research, but also conservation and resource management. So that's another another piece there. And so, you know, that generally, that payload kind of personifies, I guess, one of the major themes or research that have been going on here at the Institute, which what we refer to as taking the lab, to see, and being able to do sample processing, and do as much of the analysis that we can at sea and inform the vehicle autonomously, on how to proceed. And so, taking the lab to the ocean to the sea is has been a major theme here. And the type of development is also you know, it's always geared to if you if you could do it here at sea or on a ship, how can we take that same analysis and put it on a platform and so Oh, this, this this water sampling code that it just described, for example, now we're at the point where we're adding these modules that are able to also analyze the samples and have the vehicle, do this type of analysis at sea, and radio the results back to shore.
(0:45:15) Audrow Nash
Gotcha. Okay, so they radio, the radio the results back. So this is for like, one of those surface monitoring robots. So it'll kind of like stay at some depth, that's not very deep, collect some information. And then it goes in chills at the very surface in the water, and it'll radio back the information for you guys to analyze, or do something with? And probably that's how you communicate with it for like, send it commands or something, as well, right?
(0:45:49) Ben Raanan
So yeah, so the vehicles actually, they can also use on the sensory inputs to inform their, you know, actions. So our planning algorithms, for example, that look at sensor input, and decide how to steer the vehicle autonomously, for example, in order to try to stay with some kind of maximum concentration, where you know, you're sensing where most of the algae is, or where the highest concentration of algae is. And you can use optical sensors, to try to see how much chlorophyll is in the water using fluorescence. And so then you can use this information and driving the vehicle to do kind of like, you can imagine a gradient climbing algorithm that takes a vehicle from a, you know, an edge and drives it into center of the patch. And then when it reaches the center of patch, it will trigger a water sample. So that's something that we do here pretty typically. And then, in addition to that, I guess you mentioned communication. So yes, the vehicle surface periodically to surface with shore. But those type of applications, especially if you want to stay with a certain part of the water mass, meaning, let's say, because everything is moving in the ocean constantly, right, you've got currents carrying carrying these microorganisms. And if you lose that patch of water, and in order to go up to the surface and communicate well, when you come back down, you might be looking at our community. And so those type of applications really lend themselves well to multi vehicle constellations. And so that's been another focus here is this multi vehicle type of operations. And so in the example that we just discussed, you'll have one vehicle that is doing this graded climbing and looking for a maxilla. And then once it finds the highest concentration of chlorophyll, or whatever it is, it's looking for, it kind of just parks it underwater, and stays with that water mass and drifts. And the issue with that is, you know, after a while, you're not going to know what a vehicle is, you are not know you're not going to know what type of information it's looking at. And so a second vehicle will typically track this type of sampling vehicle, smart. And then one is at
(0:48:16) Audrow Nash
the surface looking down and the other one is drifting with this algae Max.
(0:48:22) Ben Raanan
Well, so that's one way to do it. And we also use autonomous surface vessels in order to track underwater assets. But we typically also have a second underwater vehicle that's profiling around it and tracking it, it's surfacing periodically, and it can relay information from the diving vehicle. And also, it gives the scientists more contextual data, because usually the second vehicle is kind of parked at one little part of the water column. And in order to get the full contextual data of what's going on, you also need a second vehicle in order to survey and get
(0:49:00) Audrow Nash
the bird's eye view are the ones in the forest. And one is that the trees kind of thing.
(0:49:04) Ben Raanan
Exactly. And so multi vehicle applications have been, you know, a main theme here. And I guess that kind of leads us toward maybe talking a little bit about the sim. Yeah,
(0:49:17) Audrow Nash
one sec. Just before we go into that, what are these vehicles look like? So specifically talking about the surface monitoring ones where we're looking at algae, right? What are the what are the robots look like?
(0:49:31) Ben Raanan
Yeah, so typically, they're kind of torpedo shaped vehicles. That's the most efficient way. And the vehicles that we work that I work with, specifically are kind of small size, so they're probably two to two and a half meters long, depending on the type of payload that they have. And the diameter is about 12 inches. And there it basically looks like torpedoes, I can show you A picture here maybe? Sure.
(0:50:05) Audrow Nash
I'd love to talk a little bit about how they work and then get into the SIM stuff. Because the sim stuff is very interesting. For sure.
(0:50:12) Ben Raanan
So this is a picture of one of our vehicles, those are test class long range EVs. So, like I mentioned, these are upper water column vehicles, they're pressure rated for about 300 meters. And what's special about them is really their endurance, we decided to
(0:50:29) Audrow Nash
fire extinguisher,
(0:50:31) Ben Raanan
pretty much, yeah, if you ever seen a picture of it there, then it looks pretty much like that with a bit with a mask on top, and that's our communication antenna. Yeah, so go ahead, stop sharing it. But so that's the long end up now. I guess, like I mentioned, we embark has really been on the forefront of development of these platforms. And so on that vehicle that I just showed that this class long SUV has been developed and borrowed from scratch. And it's also manufactured here, and operated here. And so everything about that vehicle was designed here in house, these vehicles are special, they're special in their capacity for their high endurance vehicles, meaning they can cover about 1500 kilometers. And stay at sea for say, up to two to three weeks, which is unusual for vehicles at that size. Especially for ones that have propeller and they're able to thrust on their own. There are other types of vehicles that don't have thrusters and have longer endurance. But this vehicle is very special and unique. And that's capacity to stay up there.
(0:51:50) Audrow Nash
So it doesn't do any, it basically you have a battery, and that battery will last for two to three weeks. And that dictates how long you can do it doesn't do any power generation or like go up to the surface and have solar panels fold out or
(0:52:05) Ben Raanan
something. But it does. I mean, the power management system is very efficient, efficient. Yes, every every aspect of that vehicle is designed is really geared toward efficiency from the shape to the tail cone and how you know, the the hydrodynamic flow, and how water passes around it. It's going to set to minimize drag. And this vehicle specifically is also special because it can change its buoyancy. Meaning it most people that put very expensive pieces of equipment in the ocean, they make them buoyant. So if the agent stops working, we'll come back up. Well, this one is able to actually change its buoyancy at sea and fly neutral. And that actually gives you a huge efficiency boost, because you're not pushing against. Yeah, this upward force. And so then, you know, comes the question, okay, if something does go wrong, how do you manage that, and so their safety and power management are kind of the heart of this vehicle. And so both both have, you know, systems that are designed specifically in order to optimize the routers, the vehicle powers off systems automatically when it doesn't need them, even including extra readers. So it's very smart management on board. And, you know, when you you know, insurance in general, it's kind of death by 1000 cuts, right? Try to optimize around each parameter, eventually, you get a system that is able to persist and be out there for those type of durations. And again, the requirements for those durations come back to the original processes that we've discussed kind of in depth upon not intended.
(0:53:53) Audrow Nash
So many ocean ponds are possible for sure. So with this, when it adjusts its buoyancy, the way it does that is it takes on water, right, like so it will have little cavities or something in it, or how does it work?
(0:54:08) Ben Raanan
Yeah, well, actually. So our vehicle doesn't take in water, it uses oil. And so it can pass oil into a bladder that's external to the sealed compartment. And by doing that, so if you're taking oil from the sealed compartment, and pushing it out to a bladder that can inflate, you're actually changing the volume of the vehicle without changing its mass. And by definition, that makes it more buoyant. And when you want to become less buoyant, you just push the oil back into the seal compartments. And that deflates the bladder. And the vehicle
(0:54:49) Audrow Nash
then becomes less of a thing. The volume changes in the mass stays constant and that changes the buoyancy of the correct Is it is it a lot of energy to like compress the oil Remove it inside to the oil. What?
(0:55:05) Ben Raanan
It's not oil, it's not cool. It's non compressible oil.
(0:55:08) Audrow Nash
It's like some sort of vacuum thing to move it from one spot to another.
(0:55:12) Ben Raanan
Yeah, we have a valve that opens and then a pump that sits inside the seal compartment and can push oil outside. And it's designed in a way that, you know, it can push against pressure, even at depth. And then there's a lot of also some safety built in there, you know, what happens if suddenly you can't push against the pressure. And so there are some valves that are set to close automatically and make sure that water never makes it back. In in case of a failure. So like I said, failure management is one of the big things. Exactly. So pretty much every aspect here. Yeah. Go ahead. So.
(0:55:58) Audrow Nash
So you don't want it to fail, because you lose your robot. How much does one of these robots actually cost? Like ballpark?
(0:56:08) Ben Raanan
Yeah, sure. So I guess these vehicles also, were designed for what we refer to as science users. Okay. So if you're a researcher at a lab, you should be able to buy one of these and, and, and operate it on your own with a reasonable budget. And so that was kind of the requirements going into there. And so the vehicles, you know, they're well, actually, not sure if it's a good idea for me to disclose the exact
(0:56:39) Audrow Nash
like, like an order of magnitude $50,000 robot platform, I'd
(0:56:45) Ben Raanan
say probably an order of magnitude more.
(0:56:47) Audrow Nash
Wow. So it's like ballpark 500,000. So half a million almost for one of these. Yeah. And that's affordable for large research institutions. I take it well, if
(0:56:59) Ben Raanan
you consider what it costs to operate a ship. Yeah, most of
(0:57:06) Audrow Nash
it costs in order to do a ship, like right, so I guess, sorry, go ahead, because it seems expensive. But I have no experience in this. So no real ability to estimate this.
(0:57:18) Ben Raanan
Right. So a big ship, especially one that can operate an ROV, you're probably looking at a few tenths of $1,000 per day. Wow. So if it's a bigger ship, you're around 50 to 60k. Easy per day of operation. And so half a million. I mean, it might sound a lot, but when you think about it, and ship. Exactly. And so that's the idea, then, once you if you can get the costs in for the vehicle, the operational costs are very, very cheap. You can deploy these vehicles with, you know, tiny, little boats, and then you're stuck with the, you know, whatever communications bill for radium satellite communication, or you know, your, your 4g cell. So,
(0:58:08) Audrow Nash
so they the operational origin is, so if they're doing, like 4g cell service for transferring data, does that mean that you have to stay close to coast with these? And that's typically how you're using them? Because I would imagine if you're in the middle of the ocean, like very far from shore, you're in the middle of the Pacific. There's no cell towers out there. There's no cell service, but you did mention satellite. So I suppose you could use that?
(0:58:35) Ben Raanan
Yeah. So we have, we refer to the we have two modes of operation. We, when you're outside of cell range, we refer to that as over the horizon. When you're within cell range, you know, why not take advantage of cell communication and do big data dumps when you can? I don't know if a region is very slow, and bandwidth limited, and so
(0:58:58) Audrow Nash
slow is really slow, like ballpark.
(0:59:02) Ben Raanan
So let's see to read the impact it is 340 bytes. And you get about I'd say if you're lucky, three to four per minute. When the surface Yes, so it's tiny. And so we play a lot of games on board system, with refreshing, and all sorts. Yeah, and also not only compression, but also selecting what information to send what's the most relevant piece of information that needs to be sent? What needs to be sent first versus, you know, if you're only able to get one SBD? One ready, pack it out? You know, what is it going to be? And so, we work a lot with the scientists in order to understand the use case. You know, get what is the crucial information that they need for decision making while the vehicles deployed and then configure the vehicle appropriately and send it over? And, you know, hopefully not spend most of our time sitting on the surface sending packets data. Exactly. So time on the surface is rescue vehicle because it can get over. That's probably the biggest risk for these autonomous platforms
(1:00:06) Audrow Nash
getting run over. Not sharks. Yeah. All right.
(1:00:10) Ben Raanan
But so it's time not doing science. So totally.
(1:00:16) Audrow Nash
Okay, so you have these, what have you long range? What was the next part? autonomous underwater vehicles? Or what was the thing you were saying? Correct?
(1:00:27) Ben Raanan
Yes, it's the long. So these, these vehicles are long range autonomous underwater vehicles. So LR EVs for short.
(1:00:36) Audrow Nash
LR UVs? LR LR UVs. Yeah. doesn't roll off the tongue terribly.
(1:00:44) Ben Raanan
You know, we've had other people try to come up with different ways. We're kind of stuck with that one.
(1:00:51) Audrow Nash
Okay, I'm just gonna say long range underwater vehicles easier. Okay, so you have these, Whoa, you're mentioning that you have a bunch of safety precautions on them? Because so they're half a million dollar platforms. I'm sure that with the cost of about you make the money back, but it's still super expensive. So I think institutions probably wouldn't like to be replacing them because they keep getting lost at sea, literally. Right. Can you talk about about the safety?
(1:01:21) Ben Raanan
So I guess I'll say that we have eight of these platforms here in MBARI. And there are others coming up online. I was holding on other places where they're building at the moment. And between the fleet, we have probably nearing 35,000 hours of sea time. And seems like that's a lot. That's a lot.
(1:01:47) Audrow Nash
Let's see, how many how many days at sea is that between the eight? So that's 1000 divided by 24. Right?
(1:01:57) Ben Raanan
So So that's like, four years? Four years worth of? Yeah, it's four years of sea time, which isn't crazy. That's a lot. And dirt. Exactly. I was gonna go knock on wood. But yeah, right. We've never lost one. Yeah. And so, again, the reason is, because MBARI has a lot of experience. And we've basically taken all of our experience and knowledge and built it into the hardware level of these vehicles. So this vehicle has so many safety features built into it. In the hardware level, meaning the motherboard itself, if CPU fries. The motherboard itself has the capacity to turn on the radio and communicate, you know, so you know, fire up a device that drops a weight to make the vehicle positively Oh stays and we have an emergency, it's going? Well, no, it will come back to the surface. The drop weight, attaches
(1:02:58) Audrow Nash
reduces the weight. And so it goes exactly, yeah, it's like cutting your weights when you're scuba diving. So that exam you can start going on.
(1:03:06) Ben Raanan
And that's exactly the idea. And then we have transponders and all kinds of other acoustic devices onboard the vehicle, and then emergency beacons, so we can, we can always find them. So the vehicle will try as hard as it can to come back to the surface because that's where it can be located. And once it's on the surface, even if the battery is completely depleted, their secondary radio, right, that will that will communicate with shore and send its position. And so that's one aspect. We also have very sophisticated ground fault detection systems, meaning, you know, if, if there's a cable or something like that that's exposed to ocean water, that can cause corrosion, that can cause corrosion, just because Victor city starts interacting with different parks like the whole. And so we have a very strict monitoring system for that. We also have, you know, in general, you want the system to be reliable and fault tolerant. So you can continue to do operations, if the vehicle has to surface every 10 minutes, because if you've countered an issue that it can't resolve on its own, well, operationally, that doesn't scale. And so we have also a health monitoring system on board that's able to power cycle different subsystems and try to change balls. And so yeah, a lot of thought and effort has gone into making sure that the system will keep doing what it's supposed to do for as long as it can, and only come back to the surface and communicate with sure when it absolutely has to.
(1:04:39) Audrow Nash
Gotcha Awesome. That's so funny that it power cycle, all the things. It's just it's like that tried and true wisdom of just like if it's not working, just power cycle it. I love that you have the ability to do that, kind of remotely with the robot, right?
(1:04:53) Ben Raanan
Yeah, it'd be a step forward or it would flip RX and TX.
(1:05:00) Audrow Nash
And because of that, so just to explain, if anyone didn't get it, it's the transmit and receive wires. So like a lot of times when you're building stuff you mix up, which is the transmitting and receiving the flip, it all works magically. Right. So that would be so funny.
(1:05:15) Ben Raanan
Yeah, that's kind of like the classic mistake. But yeah. So with all that, and we I guess, I also say that we've operated these vehicles now in both Pacific and Atlantic oceans, and also in the Great Lakes. So their tried and true vehicles. very persistent, very reliable.
(1:05:33) Audrow Nash
So and you mentioned you have eight of them. That's, I mean, that's that's in between them, they have tons and tons of hours, is there a plan to like scale it massively and get like 80 of them, or 800 of them? Or, like, so these are all over monitoring everything all at once? Or? Well? Yeah, where do you see this going in the future?
(1:05:55) Ben Raanan
Right. And so I guess, you know, we're currently I think doing multi Vehicle Operations, has still a lot of overhead. And so figuring out, I guess, at first order how to operate so many vehicles, in conjunction, and how to automate the processes, ongoing research on MBARI, and I think in a lot of other places, we've nailed down two to three fleets of two or three and doing collaborative, multi vehicle operations at sea. With you know, that many vehicles, more than that, you start running into bottlenecks. For example, if they're all using the same acoustic channel to communicate, well, that's kind of like a habit, right? Only one person can talk at any time. And so you get where
(1:06:43) Audrow Nash
you say, You're, like, I'm talking to this one, and then the message, but it's inefficient.
(1:06:49) Ben Raanan
Right, exactly. So so that's another piece where we have some ongoing research, to figure out how we can scale that. But then also, the logistics of operating a fleet like that is also you know, so you have two weeks of battery, and you need to if you have many of those, they really need to be self sustained. And XE is
(1:07:11) Audrow Nash
some sort of power generation method.
(1:07:15) Ben Raanan
Correct. And so Barbie has an ongoing project, we have a, what we call the energy booty to it's, it's a boy that uses energy from waves. And so that charges a battery, and then in conjunction with
(1:07:32) Audrow Nash
that, so is it kind of like it rides along like maybe six feet down, or something, and then there's the buoy moving up and down, creating motion, and then you're using metal icicle energy, this
(1:07:44) Ben Raanan
is a whole separate platform that generates power. And then I was gonna say, and then in addition to that platform, it also has what we refer to as a docking module. And so the idea is for the vehicles to be able to home back to this device, dock on it, and charge the battery. And we've made some really good progress. So we have a boy that, you know, manufactures energy from waves, and stores it. And now we've been, we, we've actually licensed some technology from Northrop Grumman. And they've developed this technology they call Niobe, calm, which is this metal that's based on niobium, that is self healing. So it can coat itself with a very thin layer that doesn't transfer electricity. And so it interacts when when you power it in interacts with seawater, and it creates this coat round the metal. And so that's kind of like a self sustained piece of equipment that you could leave in the ocean. And then when the vehicle comes in docks on it, it could expose and just peel off that tiny little thin layer, and in connect to the power source in charge its battery. And so we've been working very hard to, you know, see that through, and we've done some sea trials, and you know, it works, there's still a lot of kinks to iron, but that's for sure, no, eventually, the the dream is to be able to have a large number of these vehicles out at sea where they circulate in common, charge their battery, do data dumps and then go back out to sea. And then you can even imagine maybe that wave we is something that's mobile, and maybe you can direct that thing in ordinance. And so I think that type of constellation, where you have an energy source in a in a data sink, where you know, vehicles can come in and charge the battery and do big data off floats and then go back and do tasks. Is is kind of where we're headed with this, what we refer to as kind of like a mobile observation platform.
(1:09:54) Audrow Nash
And then once you have that, then scaling is a lot easier. It's like maybe the two hardest challenges for this would be the communication as part of this, where you're limited and how you can send to multiple ones. And also you're very limited by satellite radio, I wonder if like Starlink, or anything will help any of that. That'd be quite cool. No, no.
(1:10:15) Ben Raanan
Yeah, there are still other technologies that are coming up, that will speed up over the horizon. Communication. So we're excited to see how this technology is gonna play
(1:10:26) Audrow Nash
out, that'd be super cool. And then the second one would be power generation, because you have to retrieve it after the battery is nearly dead, or around the time that the battery's nearly dead. But if you could somehow charge with these systems, or whatever it might be, then you could just stay out and continuously revolve them. And then it's a lot less going out and grabbing it, which I assume is still quite expensive to do, because you need to take a ship. And the ship has to have like a big crane or something to grab it or whatever it might be.
(1:10:58) Ben Raanan
Sure. And I mean, you don't assume for these vehicles, you don't necessarily have to have it. Yeah, but But if they're far out, I mean, if you're in a coastal environment, that's great. But if, if it's happening, if you want to do research for that, or if it's really just really far away. That's an issue. Yeah. And so we're also looking into developing autonomous surface vessels that are able to do deployment and tow out, that'd be so cool. Yeah. So that's another kind of direction that some researchers here are looking at at the moment.
(1:11:32) Audrow Nash
You mentioned earlier that one human operator can work with a few different of these robots that are out how does that look?
(1:11:40) Ben Raanan
Yeah, so we've put a lot of effort into developing a user interface, that's a web based interface that allows you to control the fleet and interact with it. And so we basically have a website that gives you all the information you need in order to operate the vehicles tells you where they are to you on a map, to see what mission they're running, and you're able to interact and queue up commands and send them to the vehicles through that web interface. Like I said, the vehicles were, you know, really designed with the end user in mind. So when we have big expeditions and experiments out here, the scientists are the ones that are actually running the vehicles and not the engineers. And so we have a training program that we, you know, developed in order to train people to use them with our web interface. And we actually have a number of projects here that are geared toward, you know, improving the user experience, and, and rethinking a little bit how we can do these types of multi vehicle operations. With a single user kind of commanding the fleet, again, doing 234. That's definitely manageable with a single operator. But more than that, I think it'd be a bit of an overload. A lot of ends up being, you know, what we're identifying the bottleneck is the ability to predict what is going to happen next, the how the schedule is going to work together, how missions. So basically, look into the future and understand what what, what are the next few steps that the fleet is going to do? Currently, it's easy to understand what's happening at the moment, but it's a little harder to understand, you know, how things are going to evolve. And so that's another piece where we're trying to add in, you know, models that are maybe a way to extrapolate out into the near future, at least, and give the operator an idea of what the fleet's going to be doing, not only what it's doing at the moment. So,
(1:13:45) Audrow Nash
gotcha. Awesome. It's very interesting. It's cool. It's cool to imagine scaling this, I don't know, it's really neat to me to have an operator controlling these at a high level and making the plans and it's neat to consider how that interface might be and how it'll change over time as you increase the efficiency for an operator. Very cool.
(1:14:07) Ben Raanan
Yeah. And I guess, you know, because the timescales we're talking about, for these vehicles for their deployments, I mean, that's weeks potentially, doing development on the system is very difficult, because once they go to the, there's a lot of overhead, everything's very slow, and so fast. And then real time simulation really plays heavily into what we do. And so I feel like, you know, I think that having those capabilities also, especially in the fleet context, are going to be monumental into developing disabilities to not only do the multi vehicle operations at sea, but also how to control them. So
(1:14:47) Audrow Nash
that's tell me a bit about the simulation. Right.
(1:14:51) Ben Raanan
So this project is something that we jumped up a couple of years ago. It's The idea was, you know, we talked a lot about multi vehicles, and how difficult Axi operations are and risky. And so the idea was to develop in a development environment for these vehicles that will allow us to test and the closest way to the conditions that we experienced out at sea, you know, the, the challenges and the bottlenecks for these types of operations. And so, we had a simulator that was built in that we developed the system. But we felt like it was time for a facelift. So it wasn't really I mean, it's a command line tool. So it didn't really scale well, to the motor vehicle scenario. And so we looked at a few options of you know, what it would take to upgrade it. And eventually, we came up with this idea, maybe we should, you know, bring in an external platform, and we started looking around. And that's when we came across a gazebo and a condition. And we also had some folks who opened robotics come here and visit and started forming these relationships. And so from here to there, we decided to launch this project to build a multi vehicle underwater simulation environment for LR UV in gazebo, and ignition. And we got funded through the Institute in order to do exactly that. And we partnered with open robotics, and really contracted the engineering time to do the development for this purpose. And so it's been a fun ride, working with the team here at open robotics. You know, we had some, we weren't really sure how it was gonna work out, we absolutely had to have the framework work with the flight code of the vehicles themselves. Typically, ignition is geared more toward applications that use Ross, where our vehicles run such tiny embedded systems that at the time, it wasn't really possible to even get Ross loaded on it. And so the integration with our software was key. And so we focus on that. And then the second biggest requirement was the fast in real time capability. And so we also had concerns with that. And that was part of the reasons why. So we went to ignition or gazebo classic, is the runtime is much quicker, and you can change different aspects of the system to expedite the runtimes. And so we started with integration built the vehicle model for L or u v, which is now available to download an app fuel and we can incorporate the links maybe at the bottom here, when we're done. And, and so we slowly started building the the pieces and the different plugins that were required. A big piece that we recently completed was kind of like custom sensor environmental data. So your ability to take detailed information from, you know, very sophisticated models that model circulation and temperatures and you know, all the different ocean parameter, as well as the biological activity that that relates to those conditions, and inject those into the system and how the vehicles operate within that environment that not only has, you know, the water properties, and waves and wind and all those things, but it also has kind of scientific sensor data that the vehicle would encounter, as if it's in the field. And so that's a really nice aspect of the system that I feel we've done a little bit different from the way things are more traditionally done, where you have kind of like a lyrical model that describes how properties change using some function. And another piece that we incorporated was an acoustic model. And so vehicles can communicate acoustically in the simulated environment, in order to do these multi Vehicle Operations, like I mentioned before, acoustics and coordination is a big piece. And so we have that model with the kind of like standardized interface that allows the different vehicles to send tracking messages between each other, and also data. And so so that's modeled and you know, so all the different obviously, the vehicle model is very detailed. So the vehicle has, you know, we talked a little bit about the verbal buoyancy system that we have on board and so that pose some some modeling challenges because that's unusual, especially in a simulation environment a gazebo for something to change its buoyancy in that way, and so we had to build some infrastructure into ignition itself in so come up with the right plugins in order to support that type of performance for the vehicles. And, you know, it's it's pretty cool and works for Very well, we're able to achieve fast in real time. runtimes have about 100x factor of time, which is great. Yes. And so even with a few vehicles in the simulation, you're still at, you know, 60 to 80x. So, you know, 24 hours of simulation translate into 24 minutes of runtime in the simulator, which is great. And it's something that, you know, it's it's a useful simulator, for sure. Yeah. And so that's been really good. We're still working on now, adding, you know, there's a whole nother part of the vehicle sensing capabilities that we haven't really touched on that has to do with how the vehicles navigate underwater. And so we're now more focusing on those aspects. And so we currently have the ability to simulate the type of experience that we've described earlier, where vehicles, you know, are searching around for different ocean properties and doing water samples and all that stuff. But now, okay, what about navigation. That's another use case that, you know, could benefit a lot from from this type of simulation. And so we've worked to add bathymetry using levels actually. So depending on the position of the vehicle in the simulation, it can load a chunk of seafloor. And so that makes it pretty scalable. And you can imagine titling huge regions, and have them, you know, loaded dynamically and have in a memory efficient way.
(1:21:32) Audrow Nash
So that you'll get what you need, or only what you need at the time. And that moves over to the next tile, and you load that tile and this kind of thing, or you load a few tiles around where it is, but it never sees the edge of the tile. Okay, yeah,
(1:21:44) Ben Raanan
we ran into that. So the tiles actually. Yeah, so So that's working great. And now we're working on some, some more acoustic sensors that have to do with sensing the bottom. So they're what's called Doppler velocity loggers, those are used for navigation, it can tell you how fast the vehicle is flying over the bottom. And then it becomes a question of odometry, you know, keeping track of integrating those velocities into position. So you're able to use those sensors for navigation. And so that's kind of the next big use case that we're aiming the sim at, is automated navigation, what we refer to. And we have Yeah, we have some cool use cases here at MBARI. There's a lab that's focused on doing terrain relative navigation that I think a lot of folks have heard of, in the context of, you know, the lander and Mars where they use during real time navigation to localize understand what they are before they landed. And so we do the same using the seafloor, using maps that were acquired with the really big, much more expensive vehicles later, you can load those in and use that in order to localize yourself based on a cemetery where you are going to like Sam says, and so. So that's another aspect. I guess one more cool piece in that is that together with the simulation plugins, and all the pieces that the folks that open robotics have working very hard and doing a tremendous job doing, those are all going to be open source and the RTR. And so that's going to be available for the community to use and leverage. And we're hoping that folks pick pieces up, you know, and use them and improve them. So pull requests are welcome, I guess, launch around open source community. But in addition to that, we're also going to be releasing a Docker image that has a version of the vehicles flight code. So folks can, you know, play around with both and operate the vehicles with the flight code in the loop. And in addition to that, we also have an interface to the vehicle that's a network based interface. And so the idea is, for folks that can't have access, for example, to the flight code, they're still able to send commands using this interface to the vehicle and derive it from their application. And so we refer to that as the backseat driver. And so the idea will be eventually that you level with those Docker images to create your own application that's able to get telemetry data from the main vehicle application, feed it into your app for processing, whatever your use case is, and issue commands back to the vehicle to you know, pilot within the the environment. And in theory, if everything's right, you should be able to take that backseat application that you've developed and put it on a Jetson or whatever that's mounted on board L or UV inside the home and have it pilot vehicle at sea. And so that's awesome. Yeah, so the idea is very much for this whole package to kind of be something that's deployable by single use. are so they can do development again for their back seat application or whatever using this interface that we've come up with. And later take the same application, put it on board and have it work at sea.
(1:25:13) Audrow Nash
That seems awesome. I like I like that. And it sounds like last, I haven't been very involved in this at all at open robotics. So just hearing about it is very cool. But it's, it sounds like you guys have done a tremendous amount of work to make it so that you have a realistic ocean environment, for your simulation for your robot in simulation to drive around and interact with. Very awesome.
(1:25:40) Ben Raanan
Yeah. Yeah, we I mean, there's always room for improvement. And, you know, we really focus on building, I wouldn't say it's the best ocean model, right? There are definitely ways to do things better. And we hope that over time, things will improve. But I think that goal was to kind of close the loop to get you to an environment that is close enough to the real thing that allows you to work on I'd say, kind of guidance level parts, right. So not so much the control, but maybe a layer above.
(1:26:18) Audrow Nash
And you wanted to do. Right, exactly,
(1:26:20) Ben Raanan
exactly. And debug and work in a multi vehicle setting. And so, like I said, per use case, I'm sure that will continue to improve the system. But at this point, it is it's pretty good. It's good enough to definitely be able to test out some of these scenarios. And much faster and real time capability is a godsend, really nice. Yeah. So so that's really cool.
(1:26:45) Audrow Nash
What do you so thinking forward? Where do you imagine this whole project? And kind of the work you're doing at MBARI? Going? So maybe like five years, 10 years in the future? Where Where do you think it will be? What do you hope it will be?
(1:27:02) Ben Raanan
Yeah, so I guess we're still at a point. So the way I think of it here is that we've done a tremendous job, getting a persistent platform that's reliable. And it's taken us the project for for LRE. V has been running since 2008. So even before I've just Institute, but it's taken a while to get to the point where the system is now stable, and very, very reliable. And so now, I guess, sensing and planning are the two big. Next, I guess challenges that we have, okay, so everything we do on the system is still pretty rudimentary. So we have really cool planning algorithms that, you know, climb gradients, but they use pretty rudimentary math, there's a lot more that we can do to
(1:27:55) Audrow Nash
improve, like, keep getting more algae, like if you keep seeing more, it's good. And if you don't, it's you've reached the center or something.
(1:28:04) Ben Raanan
Right. So boundaries, the algorithms that we're using are, I guess, there's room for improvement. They're great. Don't get me wrong, we do tree work. And yeah, but yeah, but but there's so so that's one piece where, you know, the autonomy piece where the vehicles can help plan and do and do things in conjunction together, is, I think, where we'll see big advancements, I'm hoping that, you know, the system that we described earlier, where we have kind of like a power system, yeah, observatory is something that, you know, will come into reality. And I think the same is also going to be paramount, and developing those type of systems. Because, like I mentioned, there are lots of challenges, communication, and other aspects that currently, we just aren't sure how we're going to solve. And, and being able to simulate everything, and much faster real time, it kind of lends itself to iterative methods and other other techniques that, you know, will hopefully help resolve and close some of these gaps without having to go to see every time and, and the overhead that goes into that. And so I think I think this mobile observatory, if we can be there in 10 years, I think that's going to be pretty amazing. So cool. Yeah. And then I guess another piece where I'm really hoping to see this project evolve into is, you know, we've put tremendous effort into building an infrastructure. That's, you know, making its way into upstream into gazebo and ignition at the core of it. And so I think, you know, what I would really like to see is ignition being adopted widely by the marine community, and other folks coming in and using this, you know, at MBARI. We're sure 100% that we can make good use of this technology and this application, but we're really hoping that others also pick it up and use it and contribute back to it and forget or whatever it is. But use it use it, you know, we're also lucky in the capacity that we have access to the ocean. But there could be some brilliant minds out there that may not have that type of budget or just straight up, you know, don't have access to the ocean. And so we're hoping that that will increase exposure, people will be able to develop these types of applications, and perhaps even communicate with us and we can get them on the vehicles to run et Cie. So that's also very exciting aspect of this project they'd like to see evolve and curious to see what kind of collaborations and partnerships we form, with this sim kind of being as the is the platform that connects, you know, because it's out
(1:30:45) Audrow Nash
in the community. So yeah, you get a bunch of people that can try it out and start to get involved and see if they fit well. And if they really would want to do this, that seems wonderful.
(1:30:56) Ben Raanan
Yeah, yeah, hopefully. So and I mean, there are so you know, other Institute's that, you know, already are doing this type of work. There's very little overlap, even within MBARI, different platforms, different code bases. And so there's a lot that we could do for standardization, and then bringing, you know, putting out different parts of the software, or even just interfaces to make things, you know, a little more interchangeable. And if I work very hard developing a terrain relative navigation system, you know, I'm hoping that somebody else can pick it up and use it as well. Totally. So that's another piece.
(1:31:34) Audrow Nash
Yeah, it seems like this is the way things go. Typically, when it's a space, that's more exploratory, like, there are very few people or few organizations doing work, like you guys are doing. Therefore, there's very little standardization around it. But think, I think creating the sim is a wonderful thing, because now a bunch of people can try it, and they can run their code on your platforms in the simulation. And then there can start to be more buy in from the community around this and more investment in ways of doing things which leads to standardization. So I am optimistic for the future, and kind of seeing where this will go. I think that'll be awesome. Yeah, hopefully. Do you. So what advice do you have for people that are say, say, like a college student or something that would like to be involved in this line of work? What advice for them Do you have?
(1:32:33) Ben Raanan
Well, so I was very much in that, you know, position. When I first started here, like I mentioned, I started as an intern here at MBARI. And I actually don't have a formal engineering background, my background is in marine science. So you know, I think you just have to want it and be curious. And then seek out, be outspoken, don't wait for the opportunity to come to you. So your quick story that, you know, at the time, when I did my internship, there was actually wasn't budget for me to come here and be an intern. And I already had met with a pi. And we had discussed a potential project and was very excited for the opportunity. But then suddenly, there wasn't enough budget to host me. And so that was the point where I was like, Okay, I'm gonna make this happen. And I reached out and wrote some blogs and ended up raising enough money to be able to fund my own internship. Right, so, but I guess the the piece is beyond the money, it's being proactive, and, and really wanting to take part, I think, also, intellectually, we'd like to see more people from different disciplines, especially more can engineering based in computer science base, and come to do ocean research. Because there's a lot of room for that type of knowledge to make it into our community and contribute. So if you're looking to be very impactful, and help understand what the environment is doing, and how we're impacting the environment, and how it's changing, this is a really good field for you to come in, and apply expertise or your research or just develop your interests and in contribute, and so internships are a great way to start. But a lot of the people that work in this community, because they're so they believe, you know, they want to see progress. They're very friendly, and they're happy to help. So get on our website, poke around, send an email, you know, it's it's always great to hear from people. And if we can help we, we try to. So, yeah, I guess, being proactive and not being deterred. You know, there's all kinds of reasons why not to do things, but if you are curious, just go ahead and shoot or know or learn something new and Push forward, eventually you get there. I don't know. Maybe that's a really broad piece of advice. But that's what I got. It sounds
(1:35:09) Audrow Nash
wonderful. Lastly, do you have any contact info or links or anything you'd like to make sure that we share with this episode?
(1:35:19) Ben Raanan
Sure. So I guess, obviously, there are simulation framework that we're developing, which is already open and out there. And so we'll include the links, that's actually I know SRF repo or C, these days. And we'll go ahead and share that. My information can be found on the bar website, and I'll share that as well. I'm also on LinkedIn, Twitter, all those great platforms. So feel free to reach out if you're interested. Especially if you want to use the sim or if you have an idea for a vaccine application that you'd like to see running at C mon are taught this class longer. AGVs Let's talk. Yeah, that's, that's pretty much it. I MBARI has a wonderful website that's full with data in interesting stories. So feel free to visit there. But that's pretty much it.
(1:36:13) Audrow Nash
Okay, awesome. It's been wonderful talking to you.
(1:36:16) Ben Raanan
Thank you very much for having me.
(1:36:18) Audrow Nash
Hi, everyone. Thank you for listening to this conversation with Ben Ron on. Thank you again to our founding sponsor, open robotics. See you next time.