Clawloop - the unified learning api

Speaker 0: And I was fascinated all my life about reinforcement learning. You have these agents interacting with the world, doing stupid things, dying, all things I don't have to do. So super fascinating. And now we have all these agentic systems everywhere, n and n, OpenClaw, so many ancient builders, but rarely any of them is closing the loop on actually learning. So this is what I tried to achieve with ClawLoop, this unified learning layer. Speaker 0: And oh, it's a bit small. So it's essentially a framework which integrates with different agent builders, like OpenClaw, n8n, or LightLM. And then this agent really consists of 2 different parts. There's, like, the model weights and there's the harness. And now we are able to close this loop and we collect interaction traces or we even get environments for the agents. Speaker 0: And then we train the agents Aswin bait and or harness space to get better and better. And ideally, we get, like, this feedback loop to work wonderfully, and everything is good. And we did, like, this is basically the Speaker 1: oh, no. It's too big. Speaker 0: Yeah. It would we're going to address it perfectly if it was, like, in reinforcement learning agents instead of me controlling this slide. But, essentially, what we do, we have, like, the harness, which consists of the agent, l m, detects, like, a query, and maybe it has a playbook. Then it's interacting with the word, connecting a trajectory. And afterwards, we reflect upon this trajectory and form memories how to behave better, and these insights are then adapted to a playbook that's also curated. Speaker 0: And in pre in future iterations, the Robert is adapted in the harness space by the memories. And here, like, 2 things which we did by this. The 1 was, like, in an with some hospital, we trained, like, medical agents to do medical question answering on electronic health data. Because of this, I can't show it because it's running on some GPU locked away, not accessible by the Internet. But there, we train, like, some small model to perform as, like, 30% better than all 4 mini, and it performs, yeah, quite well. Speaker 0: And here, I have something on just BFCL, which is, function calling benchmark task. So it's just given a set of different functions. The agent has to figure, okay, like, which tools do I have to call with which arguments. And then there's, like, okay, there's, like, some demo how it's supposed to work. Like, this was also not done with codex, but codex reviewed. Speaker 0: So at least something. So you can see, basically, the story that we form different memories while it is running. And then these memories are retrieved and appended to the task, and then the playbook grows. And then over time, you can imagine there's like the opportunity to have a lot of very different training mechanisms in the harness and the model layer space. And, ideally, I would want to train some hyper agent that's capable of training any agentic system Aswin any harness. Speaker 0: But, yeah, there's still some way ahead. So there's some GitHub repo of this in case you want to try it. And, yeah, let me see know how it goes. Share it. Start if you want. Speaker 0: And there's some actual training run. Let me see if it's No. I think let's just leave it with it. It's I would need another moment to find the actual training run. Yeah. Speaker 0: But essentially, it's just like learning layer connecting different agents to close the feedback loop in either harness Robert space. Speaker 1: Thank you very much, Robert. Do anybody have any question to Robert? Yeah. What's the hardware you train the, squadron wheeler? Speaker 0: This p 200. Speaker 1: Okay. How big is it? Speaker 0: 16. Speaker 1: Okay. Would you find it? Speaker 0: Classified. Speaker 1: Yes. Yeah. As the name suggests, you have a did you plug it to OpenClaw? Yeah. Speaker 0: Like Is Speaker 1: it for memory management? Or Speaker 0: Yeah. If you use like, if you go to this GitHub repo, there's this, like It's already ingested. Yeah. Yeah. Then I run it with OpenCloud, and I have this running on, like, some headset instance, and integrating with this OpenClaw headset instance. Speaker 0: And I think if you go with MakeMini or whatever your OpenClaw set up is, it should be just the same. And then you can, like, do this memory formation. And then I think it's quite good that it learns, like, your traits and, like, your oddities and, like, how I prompt it, which is quite cool. Speaker 1: Yeah. Alright. 1 last question. There's this paper from Stanford, UC Berkeley called agentic context engineering, which describes a similar pipeline. Is it based on that, or how is it different if you're aware of the word? Speaker 0: Yeah. Agentic context engineering was 1 of the motivations for this. Like, if you see what your harness is, it's essentially like a prompt, a bunch of tools, and you have this memory of playbooks. And agentic context engineering is mainly about, like, how do you form the playbook, and they only generate new memories. Like, they don't kick them out, and they don't do credit assignment, which is somewhat difficult. Speaker 0: There's, like, Stanford. They have also use a model paper. There, they do a bit that, like, they discontinue memories and eventually kick them out. Was just like adding more and more and more. It's a bit too much. Speaker 0: And I think, yeah, there's, like, this big trend now also with a couple of hyper agent things from Meta, also Stanford. And I think this is basically my business to get some cloud hosted Meta agent, and then this could be offered as some burning of the service.