CyberRunner: How We Open-Sourced the AI That Beat Humans

Speaker 0: Lab, and it's an open source project. And I just wanted to give you a quick review of what we did and show what it can do so you can also go try it at home. It's an open source project. Has anybody played this game before? Right? Speaker 0: So that's amazing. So now the talk becomes more relevant. What we did is we basically plugged an agent to play this learn to play this game. So what do you think what it did? Anybody? Speaker 0: Okay. I definitely mastered it, so that's what Titus knows. This is a guy, in the eighties, Paul Simmons. He's he's, it was, almost like a religion when people were playing it. It's in a TV show. Speaker 0: They show how the how the game is played. You have motory skills. You have 2 knobs that you control with, and your eyes is your perception, and you actually can, play it and, like, go until the end. It's not impossible. Some people have done it. Speaker 0: Also, I've known people that have done it. What is happening inside your brain is a very interesting factor, but that's not what we're talking about. We're gonna abstract everything else, which is in the physics. So this guy is amazing. He did that. Speaker 0: Right? How long has he done it? No noticed that. And then we've got a bunch of people and we put a camera on top of it and we collected a lot of data. So we did we gave 6 hours of time and we, asked people from scratch to learn to play this game. Speaker 0: How much did you pay them? Fair rates. So and then the question is, whatever they've played, we've collected, can it learn like, can artificial intelligence powered robot learn to play the game? This is the question that we asked. Then the next question is, how quickly can it learn and how proficient does it get? Speaker 0: So these are questions. Right? Like, you can memorize a certain thing and you can just play it back, but the game doesn't allow you for that because there's so much dynamics. There's stiction and friction happening in the ball when the ball is moving around. So we built this system, which is exactly modeling what a human would be interacting with. Speaker 0: So there's so much of nonlinearities that are part of the system, and we kinda, like, build it as an open source system. Right? So the bottom part is just to build out. You have a camera on the top and 2 motors and that's, like, how you can build it. So 1 camera and 2 motors. Speaker 0: And that's how we set up the system. I just wanted to quickly go over the architecture of how we train this. Right? So with the physical system, we take a camera image, we encode the reward, and we take the ball position, plate angles, and the path direction as observations along with a 64 by 64 image, and that's it. We let it run on the system, and it starts to try to learn. Speaker 0: So that's what it does, for the first few minutes, and it starts to kind of, like, try to use the motors, try to learn things. The the model that we use or it's a model based RL, so we kinda learn the word, model. So all the physics behind it is kind of encoded into the into the model. And then, after 6 hours, as the humans did, we let the policy roll out on the main like, the the system, and this is what happens to it. So it's basically now just a rollout of what is happening. Speaker 0: And, yeah, it's pretty amazing. It also gets, like, very interesting, modes of velocity where you can really use the walls to get get out and yeah. So that's that's a human. That's what it does. This is a project at ETH. Speaker 0: You can open GitHub and give us a star so many people will go look at it. And so my, ours, like, PhD who built this is also happy that it's getting recognized, and I'm asking, and that's it. If you have any questions, I'm happy to to answer. Thank you very much, That's quick. Ruad? Speaker 0: Did you put the ball back manually every time? Is it, like, 6 hours of you doing this? Yes. So, actually, if if we actually go to that's a good very good question. So Thomas, when he built it, he didn't want to invest the time on engineering to actually do that. Speaker 0: So he didn't do it. And once we once, like, we started trying to open source it, this is, like, the documentation. We actually have a reload mechanism here. So what it basically does, it's a a lock which hits the ball with a solenoid, and then we have a track built. The marbles get transported back. Speaker 0: So that's the reset for the environment. We built a better version so it's not so noisy, and you can put it on your desk, and you can let it learn. Right? Yes. Speaker 1: Yeah. So basically, the first, user the first videos of user will kind of, like, first give the first cue for the model for the learning. Speaker 0: Basically, the model learns from scratch by interacting from the system, and that was just a comparison of how well does the model do. And yeah. So that's kind of, like, the storyline of yes. What were the most important features you think, like, was the model to learn or learn more efficiently? It so currently, we just give exactly what the human sees, and then it learns. Speaker 0: It's a I think it is 70,000,000 parameter size Müller, and it basically learns most of the things as how the ball interacts with the wood. When it comes back, it generalizes to learn all of the interactions that it can just repeat that in different parts. So when you see the model learning, you actually see that when a certain maze repeats but in orientation, it still generalizes that it just learns that part faster. So you yeah. Nice. Speaker 0: Let's keep the questions for Flauster so we can move on with the Thank you very much. Alright. Adrian, your turn.