Duckietown Competitors Highlight – Robohub

At ICRA 2022, Competitions are a core a part of the convention. We shine a highlight on influential competitions in Robotics. On this episode, Dr Liam Paull talks concerning the Duckietown Competitors, the place robots drive round Rubber Ducky passengers in an autonomous driving observe.

Dr. Liam Paull

Liam Paull is an assistant professor at l’Université de Montréal and the top of the Montreal Robotics and Embodied AI Lab (REAL). His lab focuses on robotics issues together with constructing representations of the world (corresponding to for simultaneous localization and mapping), modeling of uncertainty, and constructing higher workflows to show robotic brokers new duties (corresponding to by simulation or demonstration). Earlier to this, Liam was a analysis scientist at CSAIL MIT the place he led the TRI funded autonomous automobile venture. He was additionally a postdoc within the marine robotics lab at MIT the place he labored on SLAM for underwater robots. He obtained his PhD from the College of New Brunswick in 2013 the place he labored on sturdy and adaptive planning for underwater automobiles. He’s a co-founder and director of the Duckietown Basis, which is devoted to creating participating robotics studying experiences accessible to everybody. The Duckietown class was initially taught at MIT however now the platform is used at quite a few establishments worldwide.

Hyperlinks

transcript

Abate: [00:00:00] Good day all people. That is Abate. Subsequent week is ICRA and a core a part of this yr’s convention goes to be robotics competitions. So we’re going to deep dive into among the influential robotics competitions on the market. with a few brief spotlights on a number of completely different ones this week, we’ll be speaking to Dr. Liam Paul, the co-founder of the Duckietown competitors.

Hey Liam, welcome to Robohub. Might you give us a bit little bit of background about your self?

Dr. Liam Paull: Positive. My identify’s Liam Paul. I’m a professor on the college of Montreal. I’m additionally the president of the Duckietown basis and one of many co-founders of that venture.

I did my PhD in in new Brunswick. After which I did a postdoc in MIT, which is the place this Duckietown factor began. And now I’ve been a proffer about 5 years or so.

Abate: Yeah. So at the moment really we actually need to dive into the Duckietown competitors. Um, so might you give us a bit little bit of [00:01:00] details about the way you began it, what your motivations have been?

Dr. Liam Paull: Yeah. So, I imply, the Duckietown factor is one thing that’s type of taken on a lifetime of its personal, for certain. It began as a category initially, it was used for instructional functions, however then in some unspecified time in the future alongside the best way we thought that it might have additionally worth as, as a scientific benchmark. And so we began to see if we might reformulate and repurpose the platform to host these these competitions.

And the primary one was it NeurIPS. And I need to say 2018 after which we’d achieved a minimum of one at ICRA and some at NeurIPS and it’s kind of one thing that’s actually actually gathered the motivation, I believe actually is it’s all about attempting to carefully benchmark robotic algorithms. And this can be a fairly, it’s really a fairly [00:02:00] onerous job.

Quite a lot of robotic analysis is completed in some particular lab with a really particular setup and is kind of onerous to breed. And so we needed to construct a really standardized however very accessible platform that individuals might simply get their palms on, simply, put their algorithms on, and that we might one way or the other like examine all kinds of algorithms in some.

Standardized and like truthful, truthful manner.

Abate: Yeah. So what’s the precise problem that they’re competing for and the way does it, how does it look?

Dr. Liam Paull: Yeah, so that is advanced over time, however the primary premise is, is, is. Largely the identical. In order a part of the Duckietown platform, we now have the vehicles, that are these little, little vehicles which you could construct, however then there’s additionally an setting by which they function.

And the setting is [00:03:00] made up of like yoga mats and duct tape and indicators that we’ve like printed and stuff. Um, however the concept is that it’s very standardized and really reproducible. To you or me, like, it appears to be like like a small metropolis. Prefer it’s a really simplified view of a metropolis, nevertheless it’s one thing that approximates one way or the other a small metropolis and the challenges are very in complexity, however principally concerned the robots navigating on this metropolis.

And we will. we will fluctuate the complexity by having completely different typologies of town intersections. We are able to have completely different obstacles, we will produce other automobiles. And so the complexity can actually develop. Um, however probably the most type of like primary, elementary, like a PR factor that an agent ought to be capable to do is like drive down the street within the metropolis, keep away from obstacles and keep of their lane type of factor.

Abate: Yeah. So what was the motivation behind the identify Duckietown?

Dr. Liam Paull: That’s an [00:04:00] attention-grabbing, that’s an attention-grabbing one as properly, really. So just like the ducky not too many individuals know this, however the ducky branding, not solely does it, it predates the Duckietown venture, nevertheless it additionally has an ICRA connection. So the opposite co-founder of the venture his identify’s Andrea Censi and now he’s at ETH Zurich.

And I believe the yr earlier than Duckietown began, he was… I neglect precisely what the title was, nevertheless it at present this push for everyone to submit movies and so they have been going to attempt to sew all of those movies collectively to make like a promo video for the, for the convention. And Andrea got here up with the concept each video ought to have a rubber ducky in it kind of for a variety of causes.

However I believe that partially, it was like for scale and likewise for like some type of coherence between the completely different movies. So they might do like enjoyable cuts and stuff in between the movies, however one way or the other the branding of it similar to completely exploded. After which after we began this venture, [00:05:00] like earlier than anything, the one constraint was that it needed to have like rubber duckies concerned.

I… I don’t know… Simply kind of occurred that manner.

Abate: Yeah,

no, it’s nice. As a result of once you like grounded in one thing, that’s like a enjoyable idea it makes it far more participating for individuals to, to need to do it.

Dr. Liam Paull: Yeah. And there’s additionally a side of I imply, my view is that some, some robotics particularly is type of portrayed in a sure manner.

And I believe that like Hollywood has one thing to do with this. Scary, not like both it’s like Terminator are going to return and kill you, or it’s scary within the sense that it’s going to take your jobs or no matter. And I believe, yeah, ultimately a part of, a part of the motivation behind this like type of enjoyable, playful type of factor was that we might break this mildew a bit little bit of attempting to make one thing that’s tremendous quick and tremendous scary and tremendous massive or no matter that possibly this could enchantment to.

Totally different people who find themselves possibly not [00:06:00] drawn to the, like, let’s construct a giant, quick, scary factor, however as an alternative, you understand, additionally need to have the ability to like categorical themselves one way or the other by like by their work. And I believe yeah, I believe that’s additionally been, been a part of it and has been type of, type of profitable.

Yeah.

Abate: And so the competitors now it’s been working for, is it a decade or two?

Dr. Liam Paull: It’s not, no, it’s not that lengthy. I believe it’s, I believe the primary iteration was in 2018. So I believe we’re at like, across the five-year mark. Um, however the five-year time. Yeah. The primary iteration of the category at MIT would have been one thing round 2016.

I believe. So the venture itself has most likely been round for six or seven years, however the, the, the competitors itself possibly solely 4. Hm. Yeah.

Abate: So what have been among the, the real-world advantages that that you simply’ve seen out of the competitors?

Dr. Liam Paull: Yeah, that’s an ideal query. I imply, I believe with Roberta [00:07:00] robotics, I imply, a part of our you understand, philosophy is that robotics ought to contain a robotic.

And I believe particularly in newer previous, there’s been this big pattern in the direction of like machine studying and deep studying. Kind of algorithms. And I believe these algorithms actually have big potential, however once you attempt to put a few of these algorithms on robots, you see among the, among the type of nitty-gritty particulars that you simply possibly didn’t take into consideration actually have a huge impact, you understand, like how the latency of your system you understand, the way it’s coping with.

asynchronous singles versus synchronous alerts, like treating time, you understand, non-model defects and issues like friction and slippage and issues like this. And so for lots of the parents, I believe like the actual, like the actual world profit has been that, wow, they actually have gotten an appreciation for simply how, how robust it’s [00:08:00] to, to construct these methods.

After which once you have a look at like what, though we’re not all the best way to having, you understand business, autonomous automobiles. I believe which you could get some type of an appreciation for simply how outstanding, what has already been achieved. You recognize, it truly is when you think about all of the completely different items that need to work collectively and the way sturdy all of them need to be.

Yeah.

Abate: And I can think about over time, you understand, completely different applied sciences have taken extra curiosity within the eyes of roboticists and that the method that the completely different individuals competing has modified fairly a bit as properly.

Dr. Liam Paull: Oh, for certain. Yeah. In the beginning, I imply, we very a lot noticed fairly conventional what I’d name like classical.

Not as a result of they’re previous, however simply because it’s like the best way that issues was achieved, type of like stacked that had the very normal abstractions of like, you understand, notion and state estimation and planning and management, and now far more we’re seeing opponents [00:09:00] attempt to remedy this. And to finish machine studying kind of methods, whether or not they’re primarily based on extra like imitation studying paradigm leveraging knowledge that we make accessible, or whether or not they’re utilizing the simulator primarily.

And simply attempting to do like reinforcement studying stuff. Type method after which switch their brokers that the actual, the actual robotic, these, I, I nonetheless suppose it’s like stays to be seen at this level at this juncture, like which one is definitely higher at fixing the duty. However one factor that’s undoubtedly true is that the scholars within the opponents appear to be far more they discover the, like, I believe the machine studying type of method is extra interesting at this level.

It’s type of like this scorching, scorching matter, I suppose.

Abate: Oh, that’s attention-grabbing. So it’s possibly it’s extra interesting, however possibly it’s not essentially as of proper now leading to a extra success for the opponents.

Dr. Liam Paull: Yeah. I imply, the best way that I view it, particularly like from a say a scientific standpoint is that [00:10:00] particularly on this setting, all the things’s very well specified a very well engineered answer with little or no studying goes to be very onerous to be.

you understand, the potential advantages of extra studying primarily based methods or that they need to be capable to be extra sturdy to various situations, be capable to generalize in kind of a extra, a easy, extra S simpler technique to completely different environments. And so, yeah, it’s, it’s not, it’s not at all times simple. It’s not at all times simple to love we now have we now have to think twice about even simply what the metrics we’re going to make use of.

to check, you understand, these completely different algorithms, like, is that simply the one which, you understand, drives the quickest? I’m unsure that’s the very best, you understand, that’s the very best metric. Um, there’s all these different elements about like robustness and skill to generalize, to completely different like eventualities and issues like that.

And in these instances, the [00:11:00] machine studying options possibly do a bit.

Abate: Yeah, no, it’s an attention-grabbing level about overfitting your answer to particularly the competitors setting, apart from like whether or not or not that’s one thing that you simply actually need to do as a choose to say whether or not or not this can be a higher answer, it is perhaps higher on this competitors as a result of it was quicker… however ought to the impediment course change a bit, the topology change, now, possibly it’s not so sturdy.

Dr. Liam Paull: I believe that is really the central problem in constructing robotic competitions. It’s very troublesome to construct a robotic competitors. That’s like not hackable in some sense which you could’t win by simply actually overfitting to the specifics of that specific of that specific setup.

And so, yeah, I imply, I believe. You hit the nail on the top there it’s that is the massive problem for certain. And [00:12:00] attempting to construct like actually good robotic benchmarks.

Abate: Yeah. In order you, as you consider subsequent yr’s competitions have you ever guys ever thought-about possibly doing a not releasing the map and having it’s a bit extra of a shock and have a bit extra randomness related?

Dr. Liam Paull: Yeah. So we, we now have, we now have sometimes achieved that. Like, we now have a kind of a, like a, a validation set that individuals get the outcomes and so they can see all the things. After which what they’re really evaluated on as like a held out take a look at set that they don’t see. However what we’re desirous about doing this yr, So sometimes what we’ve achieved is we’ve had kind of like possibly two or three major challenges, just like the lane following problem, the lane following with obstacles, problem, and the lane following with intersections problem or no matter.

And every one in all these challenges is, has its personal outlined metrics. Like how lengthy you survive for, or how far you’re touring in a sure period of time, kind of like normal stuff. What we’re going to do that [00:13:00] yr is we’re going to. Have a sequence of ranges successfully which can be simply more and more complicated and more and more troublesome.

And every one in all them possibly has like some, some stage by way of the metrics that you must obtain to ensure that it to be handed. However what we’re attempting to do is definitely alleviate the overfitting to any particular type of like particular job and stage. You’re going to have an excessive amount of extra. B constructing a basic goal agent that’s in a position to do moderately properly in a, like a very like various like environments of various complexity and growing complexity.

And so I, that is our, that is our subsequent try, really at type of attempting to alleviate this, like over-fitting to the specifics of the, of the the precise like problem or no matter.

Abate: Thanks.

transcript

tags: Algorithm AI-Cognition, Algorithm Controls, Competitors-Problem, podcast