Designing and Controlling Safe Self-Driving Systems

It's great to be here. Thanks very much for providing a forum for us to describe what we've been doing generally in our company. And I'd like to start out. I'd be remiss, I think, to ignore the vehicle on the slide here.

So this is a self-driving or an autonomous eRod. So we started selling these eRod vehicles in 2016. And we sponsored a project at ETH Zurich where students were able to take the eRod and turn it into an autonomous robotic vehicle.

A little bit of an update: I just recently stepped into the role of head of research at Kyburz and so what's involved in that role? It's really one of the greatest jobs in the world, I think. I get to come up with ideas and plans for the future. But I also get to do the work myself. So I've been a MATLAB user for, I think I figured about 18 years now. I took a little break when I was teaching. And now I'm really happy to be back at it.

So maybe before I jump into everything, I'll describe a little bit about Kyburz. The vehicles in the top left of your screen might look a little bit familiar to some of you, especially those who live in Switzerland. And the vehicles in the bottom left might start to look familiar to the Australian contingent in the room. We just sold 1,000 vehicles from Switzerland to Australia. Essentially, our major products are last-mile electric vehicles. So this is a map that shows the distribution of vehicles around Switzerland.

The company's owned and operated by Martin Kyburz since 1993. We're about 100 people sitting just outside of Zurich. And the team that I'm leading, it's quite a small team. So we have an overseeable set of resources. And I want to come back to that point as we go through these slides because I feel that's important for the context of why we're here.

In motivating the question of autonomous systems—so we're talking about autonomous delivery systems here at Kyburz. All of our vehicles are focused on the last mile or the first mile of the logistics chain. And that industry's been changing.

So classically, you've had one post person delivering mail to many people because that's how the system's worked for centuries. However, that business, as a result of electronic communication and just how much is being communicated by the internet now, has been declining. So the customers are feeling a certain amount of pain at their margins.

Because they have other revenue streams, for example, parcel delivery—the problem there is that the economics aren't the same. So you have one human acting for another human for a particular amount of time. So two to three minutes in each delivery, that person is working for one other person. So it doesn't scale as nicely.

Business is increasing. There are more parcels being delivered. But it's just not as easy to generate revenue that way for our customers. So they've been looking for new alternatives here. They've been looking for things to kind of backstop that problem, one of which being robots.

So two years ago, we started a program to see what we could do there. The challenge here is that these robots have to be all kinds of things. They have to be better, cheaper, faster, more reliable, and perhaps even somehow more personal than the human delivery agent. So there's a lot of paradoxes. It's a very challenging task. But I'll describe, hopefully, in this presentation how we've chosen to go about it.

So these are our prototypes to date. The first prototype, the red box, the second vehicle we built, was designed for use as a mobile depot station. So not sure if you're aware of this, but bags of mail are driven by a diesel truck around the city and dropped off for postal workers to collect and then deliver. And that prototype here, the red prototype, was designed to displace diesel trips. So we still have the model where a human comes along and meets the vehicle. But the driver who comes along and delivers the bags of mail is no longer required.

The second prototype—it's tough to see in the slide the scale difference here, but it's actually a rather large vehicle. It's 2 1/2 meters long, almost. Weighs almost 500 kilograms when it's loaded. It was sold to the Norwegian Post. It was a collaborative project with a partner called Buddy Mobility in Norway. And basically, the idea there is actually to completely replace the postal workers in the chain. So the vehicle's loaded at a logistics center, and it delivers the mail. And I'll show a few images of that happening later in the presentation.

So those are two business models. And essentially, we looked at our palette of options and decided to focus on a general-purpose model for our next prototype. So the sort of the concept sketch there, the gray vehicle, is a vehicle which is not yet built.

But essentially, the design is such that you'll have a pallet or a box which can contain bags of mail, or locker boxes, or whatever you'd like to transport on that surface. So essentially, it's a multipurpose autonomous delivery system. You'll notice we're not delivering people at this stage. So I think that's an important distinction to make.

When I talk about the challenges—so here are the three major categories of challenges that we are interested in solving with our autonomous systems. So the availability problem, I'll describe in some detail. So I'll just mention it here. You have to be as good at delivering parcels or mail as a human.

You have to be as safe or safer, and you have to prove that. That's a key point there. And we also, when we're developing these things, we need to be able to simulate—this came up a little earlier in the day as well—we need to simulate difficult to validate situations in the real world. So you have to come up with things which, in simulation, you can test to be sure that your control strategy works.

So let me focus on the first problem. I have a little bit of a thought experiment to present here. So you have this person. They're delivering 300 parcels per day. They have a standard working day and a standard salary.

And now we want to look at this robot, which is going to come along and deliver the mail for them and you can just do some quick math in your head there and say, okay, I need about 2  1/2 robots to make up for that parcel delivery so I'm looking at over 100,000 Swiss francs initial purchase cost. What's the amortization on that? So maybe it's 2, 2 1/2 years as well, and then you have your money back for this purchase.

But what about the ugly secret in autonomy? What about that team of people who's sitting in a back room with video cameras watching the system, ready to step in at a moment's notice and help save a robot from a situation it doesn't understand? What about those guys? Who pays them?

And so what we did when we looked at this small toy economic problem—and I think I just got the numbers wrong, so I can give you the correct model numbers after the presentation—what we discovered is that you can amortize these systems if you disengage, which basically means the human has to intervene once per day, but never after three per day. So essentially, you have this very tight requirement as to how much you can intervene in the robots' operation.

So what does that mean for our design? So essentially, what we've chosen to do then is to pack all kinds of sensor technology and computing power into these platforms with the understanding and the realization that we shouldn't have people getting involved at any time. So as you look down the columns of this table, you'll probably see that there's always a green and a yellow point, which means we always have some fallback, whether we're localizing or whether trying to avoid obstacles in these various scenarios.

And of course, this table is much wider than I've shown here. But just as an example, I thought I'd put some context in play. Things like transparent obstacles to handle, or cute, soft, and fuzzy obstacles that we don't want to run over—these are things that we're really thinking about in the layout of our sensor systems and designs.

So that's the goal. And how do we accomplish this? So the challenge for us in this scenario is to come up with an efficient workflow to allow a very small team to accomplish what major large corporations are doing. I have a team of two programmers. So there's lots of work to be done.

And essentially, we start with our specifications. So we write those out, linked up in the Model-Based Design paradigm that we're using. We do our coding. And then we generate that code. So we basically autogenerate our PLC code, which we were then going to compile and put onto our controller. And this allows us to achieve a level of safety which normally would be more difficult to obtain for our control systems.

I'm not going to belabor this point because Mohammed did such a good job explaining what the value of state flow could be for your work. But I just wanted to show one simple example of a diagram which we actually use to test some ideas. And I want to kind of contrast that to how—I think I couldn't have done as good a job as Mohammed did with slide 34 of his presentation, where he compared the standard code to the Stateflow model. But this is just so much more effective at handling that complexity.

So if we have all these sensors working redundantly, there's an enormous amount of complexity which we have to handle. So especially when you come to various controllers in various situations—sorry, this slide is very zoomed out in the Stateflow modeler. It's because it's actually one of the controllers that we're using. So maybe that picture, if you zoom right in, you can—[LAUGHS]. Anyway, the idea here is that we can handle, then, all of that complexity in a way that's testable and debuggable for a small team. So this is really helpful for us to be able to model in this kind of way.

So I'm going to switch gears now a little bit and talk about our approach to safety. So we're building machines, not vehicles. That's a key point here that I'd like to make because it means that we don't have to abide by the very, very stringent functional safety rules, necessarily. But we choose to do it voluntarily.

And the reason for that is that a lot of the best practices that we see coming out of automotive makes sense. It's a wise thing to do at this stage because at that point, we know we're at the state of the art no matter what. So one of the sort of key strengths, I would say, of Kyburz is our ability to get new vehicle concepts on the road. So this is part of the strategy in the autonomous system space.

And this is our process for our V cycle, for those of you in automotive, for defining our specifications and then testing. I'll take just a quick minute to talk about how this looks. So at the unit levels, right down to the bottom of the V, we're going to test simple unit tests in Simulink directly using the toolbox.

One level up, we have a nice way to take the generated code that's now been imported into Code Assist and co-simulate the test bench that we've generated, also automatically. So that's kind of a really nice way to kill two birds with one stone because we can then check whether the code is consistent and also whether it achieves the function that we wanted at a module level.

And then we take one step up. And it was mentioned, I believe, in Jason's presentation, about Voyager and ROS. So we also are using ROS, but only the Gazebo Multiphysics environment. And that's nice because it hooks up well with the Robotic Systems toolkit, which then helps us to simulate overall system behavior. So we can look at sensor echoes. We can introduce noise into our systems. And I'll show a few videos of us having fun in Gazebo later.

What I would like to mention, though, here is that the toolchain that I showed earlier was designed in order to get code into our vehicles which can be certified. So we certainly want to use toolchains where you can get that code through the process with TUF and the rest of the certification bodies that need to happen. And I'm a little skeptical if you can get a controller that's running completely in ROS through the gate. But I could be proven wrong in that regard.

So this is how the example looks for our just simple serial communication tests at the unit level. So we have this controller function block which has been autogenerated from our controller. And we're checking whether the CRC is evaluated correctly, and then from the inertial navigation system. And if we've had any problems, then we basically fault out.

So this is my last section. I do like to leave some time for questions at the end here. So what I'm looking now towards is the sort of simulation of corner cases. So we have this Hazard and Risk Analysis, which we perform for our vehicle systems. And we identify all sorts of failure modes, check their probability of occurrence, how bad it is if it happens, and how controllable it is.

So in this case, with autonomous systems, controllability is always the worst case because theoretically, nobody's there. And then we just look at some of these cases, look at the example of trying to cross a road. So if our vehicles are designed for sidewalk operation as well as road operation, what happens when we come to a light, a crossing light? How do we get across that road?

Or inclement weather, at the very bottom—what happens when we are looking through that fog and rain with our sensors? How is it possible for us to get through? And then finally, the most difficult challenge for a robot is, what happens if it's a really hot day, and you have no arms to hold the ice cream? So this is kind of the test case here that we're also very concerned about. I think that this fellow was just super hot and wanted to take a swim there.

Our approach here, as I mentioned, is to link our code, our controller code in Simulink, to ROS, so to Gazebo. And you're seeing on the right here one of our tests. This was taken to evaluate our avoidance with laser. And as you can see, it's something that probably didn't make it into a production—catastrophic failure, very hungry robot. But essentially, what's really nice about this tool is we can make code changes in very tight iteration cycles and in a somewhat agile way because, of course, we're doing everything specification-based. But to test certain ideas, this is a really useful approach and saves us a lot of time.

And in terms of what that means for the company, like any company, we have stringent deadlines. This was our first prototype. And this is the first real live delivery. And I think I had, maybe, if we counted nicely, about a week to prepare the entire controller code.

So I think who you're watching on the left there is one of our most important customers’ CEOs. So she came to visit us at our facility. And I think I was only able to develop this system in time because of this toolchain that I was using. So it really does work. It is effective. And it's important, if you want to hit your deadlines, to get through those things.

So I'm just going to wrap up here. I promise to leave some time for questions. So coming back to the first point, so we have this really complicated system we're trying to control. We're trying to make sure that a robot with eight different range sensors in three different location systems is behaving consistently and have high availability. And in order to do that, we have to manage a relatively hugely complex supervisory controller. And so to test that and debug that, it's very useful to have a Model-Based Design approach.

We're really making good use of the toolboxes that are available to us to ensure that we're following our norms. One of the major concerns with those norms is documentation. So you have to document your spec and your testing. And sort of taking this structured approach is really helpful.

And finally, I showed how we're able to simulate some corner cases that allow us to make sure that our robot's going to behave itself in the real world because ultimately, when we deployed—I should've mentioned this earlier. We sold the locker box model to the Norwegian Post as a demonstrator. And so now it's out in the real world, and we have to make sure that it actually works.

So with that, I'd like to wrap up and say thanks a lot. And hopefully, I can throw the cube over at someone.