Full transcript
0:00I want to show you how I personally have been using AI lately, just in case there are any tips or ideas or things that inspire you and a couple projects that I've been working on. Like having AI autonomously solve some cybersecurity wargames and capture the flag challenges, but we'll get there. You know my style, I want to show you the
0:18whole thing. And there are two keywords here because I say I'm using this personally, so this is just local in my own home lab playground sandbox. And I say how I'm using it lately, so not everything is done. Not everything is finished, not everything is perfect, but I thought it would still be cool to show you. So first things first, let's talk
0:35about the hardware. I don't want to be using AI on my main computer, like my desktop, because if you run AI in YOLO mode or like dangerously skip permissions, full access to do whatever it wants, which is where you get the awesome AI superpowers, well, then it's a little bit risky that I don't want on my main PC. So you think, okay, we'll
0:55put it inside of a virtual machine. And I've done that for a little bit of time, but it became kind of tedious. Like you spin up a VM, you get access into your environment, you're back in your workspace, then you can get to work. But I don't want or need a Mac mini or whatever is kind of all the rage
1:10recently. It just needed to be some compute. So I've literally been using this little device here, this tiny like handheld laptop. This is the GPD Pocket 2 and it's running Linux Mint just so I have Linux on the host because that seems to work really well for bash commands that your AI agent might want to run. And that's it. It's just this.
1:31And honestly, that's all it really needs to be because that box is just making API calls or like HTTP requests to OpenAI or Anthropic servers. And with that, let me talk about the model. These days I am just using GPT 5.5. And I've told the story before, I've gone through the saga like I think a lot of us have.
1:50Way back when, kind of just first copying and pasting in and out of ChatGPT on the website, then I played with Cursor when that was out and about and awesome and then Claude code that was super incredible exciting open code I played with Opus 4.5 hit the streets and that was phenomenal but I think a lot of the changes between what Claude
2:07has been up to in 4.6 and 4.7 just weren't quite there. So I have hopped over to GPT 5.5 and that has been awesome. I like to frequently use plan mode. I like to set it on a low reasoning because honestly that model can just spin and go off where it doesn't always need to. Sometimes I've seen folks even recommend using no
2:26reasoning and when you use fast mode it's like lightning. So since I am currently on the open AI train using GPT 5.5 it's natural look I want to use this thing in codex. For one thing that lets me use my subscription and I'm not paying as I go like paying per token with API usage it's just all bundled
2:43into the subscription. I got to be honest I don't know like my token counter the usage because I just oh go up to the limits and those are really generous. Using the monthly subscription with GPT 5.5 low reasoning and occasionally fast mode when I'm impatient I don't think I've ever actually hit the limits other than one time when I set like the autonomous CTF
3:02war game solver just let it run forever and it worked until like 5% remaining on the weekly limits and then I just stopped it and waited a day because okay it was just about that timing it lined up. So the next part to talk about is the harness and you know that GPD pocket device that little laptop that I was
3:19holding that just sits in the closet and I'm not going to interact with that literally on the keyboard. I want to remote into it. So I could use SSH and then maybe some team up little decorations if I were using the codex CLI but I got to be honest I don't like using the CLI. I don't want to use the
3:36command line because I do want a little bit more access. I like having the graphical user interface so I can split screen or pull up my browser or use things that just have visuals to them like oh having images rendered in chat. Like the AI can present them to me and I can easily provide images to it. So the
3:51first thought is to use the Codex app, like the GUI, the graphical user interface. And that is pretty good now. Like they added the browser, they added a little bit more files, you can see the diff and everything. It's cool, but it does not run well at all on Windows, which is the operating system that I just tend to use naturally for getting
4:08work done and being at my desktop. Opening the settings page makes it crash for me. I can't multitask all that well cuz I have to click and move in between each thread or chat, and it doesn't even have the remote support to like SSH connect over to that GPD pocket that I have in the other room. So, Codex GUI is
4:24for the moment out the window, and I had been using Cursor for quite some time. But Cursor got a little annoying but just how frequently it would just sort of die or break or have some connection issues in the middle of the conversations. It was also super slow. It would get sluggish when I'd start to do more work. I did use Theo's T3 code
4:41and even T1 code for a little bit. Shout out Maria. But that didn't quite have the ask user question tool for like plan mode, and it wouldn't be able to render images in chat. And I wanted something to be able to like dynamically create threads in like a programmatic way so that I could just let it surface up new
4:55work or new activities as the AI is doing what it would do. I hacked around with it for a little bit cuz it is open source and that's the fun, but it just seemed like I'm tripping over myself when I just didn't have what I wanted in the first place. So, I experimented with Zed. That also seemed a little too
5:09clunky for me, and now I'm just sort of resorting to VS Code. With all that talk out of the way, we can finally jump over the computer screen and I can show you some of this stuff. But while we're chatting about how AI is really changing a whole lot of cybersecurity work, let me tell you about some other really cool
5:24innovation thanks to the sponsor of today's video, Wiz. Cybersecurity has accelerated, especially in the AI era. Threats move faster than ever, and defenders have to move at machine speed to keep pace with today's attackers. But to maintain precision, you need context of everything that's part of your security stack so you can protect your
5:45organization with agent-powered security solutions. Wiz has introduced three new AI agents to tackle our industry's toughest problems. Continuous testing and evaluation with a red agent to find and uncover real, complex vulnerabilities and risk within your environment. And a blue agent to triage,
6:05investigate, and analyze threats and bubble up what actually matters. And a green agent to resolve and remediate everything that truly needs action. Visibility with Wiz is still agentless while defense is agent-powered. These three agents combine their efforts to proactively defend your entire security
6:25posture. And you can trigger them with your own controls and automations and workflows that you define. Even with help from their AI assistant Mika. So, you stay in the driver seat, but the agentic engine sets your security posture on cruise control. Then, as you build your own AI agents, the Wiz AI application protection platform
6:46seamlessly helps secure your AI footprint wherever you build. You get the visibility and trust that the agentic systems you set in motion are safe, secure, and streamlining your security operations. See how Wiz secures every layer of AI applications and enables you and your team to move at the speed of AI. You can see it all in
7:09action with a personalized demo of Wiz with my link below in the video description. jh.live/wiz. Huge thanks to Wiz for sponsoring this video. All right. So, VS Code is the surface that I've been using lately to interact with AI and AI agents. I believe they now have this like agent mode editor
7:30that I haven't really played with, truth be told. Uh I think it kind of gets me stuck into like their chat with like GitHub Copilot and things that I'm not all that interested. So, so far I've just been sticking in the editor mode, but I've been using these extensions to be able to actually have remote SSH. Here you can see what's installed. I do
7:48use remote, so that that way I'm able to actually access and connect to the GPD Pocket, the dedicated device I have set up in the other room. So, I can make that a little bit bigger for you to be able to see things. This is handy and convenient because it's functionality that Cursor gives me, but I think for
8:03the moment, at least at the time recording, the Codex app and GUI on its own doesn't quite give me. So, that's installed locally on my host, my desktop, my Windows machine, and then on the GPD Pocket 2 itself, I have the markdown preview enhanced added in so that I would be able to actually view markdown in a little bit more of a
8:22beautiful way, and of course, Codex. Claude is down here, too, but I am in less of that these days. With these extensions installed, since I have such a super simple interface and can easily make a whole lot of changes, I can just open up the sidebar to use Codex here, but that's not really the best UI for
8:40me, cuz it's stuck on one side or edge, and I have to click into or navigate in between all of the other threads. And moving in and out of those with the mouse getting in the way is just another little nuisance. So, I actually haven't been using the sidebar that comes with the Codex extension. I kind of leave
8:56that closed, and then just open up a new Codex agent here inside of the editor, so that it's essentially just a pane in a tab in the editor. This is just the usual chat interface to send any prompt or start any turn. I have this set to full access, as I told you, low reasoning for GPT 5.5, or we could
9:15adjust the speed if we wanted to. But, the benefit here is that I can more easily multitask because I could just open up another pane or another editor, and then drag in a new Codex window just to be put right there. That split-screen process isn't as seamless as I would like. Like, I use the hotkey control K
9:33and control O to open up a new agent and window here, and then if I ever wanted to, right, we could open up another split sidebar, or we could drag it to any part or piece, any top or bottom or corner, or even move it into any other piece that we would like. That just kind of lets me multitask and have the
9:50multiple agents up and running, which can be nice and convenient, but your mileage may vary. That's just kind of what I like to do to simulate a split screen tmux terminal, and so I have access to a ton of stuff if I'm just hardcore vibing getting that out across many different threads without needing to switch back and forth between them.
10:08It's been funny, I think I found my own like cognitive limit, like me as a human being the bottleneck to rapid robot machine speed. I can really only babysit maybe three or four of these. When I'm multitasking, that's probably the limit, otherwise I'm just kind of all over the place and too scatterbrained to do a good job. And this isn't perfect by any
10:25means, it's still a little bit clunky, but then I can adjust the color theme or do anything that I might like to make it pretty, but then I can easily take a look at my files, explore any of them to look at the real code itself, or even open up a web browser so that I can interact with
10:41anything that it's working with here, and I could pop that out into its own window, or again have that split screen, essentially giving me even more flexibility than what I would have otherwise had within the Codex app itself. That is extremely convenient because while I have Codex building things again remotely now on that GPD laptop that I could just let run and go
11:00as I move or leave or go out and about in the world on mobile. If I ever wanted to, I could forward a port and then have access to anything that it wants to serve or still make visible to me. This GPD Pocket 2, along with the other devices, are interconnected with Tailscale in my own home network. Then,
11:19even on the go, if I'm just on my phone, I could still connect into or look at some of the apps that it's serving or try to connect and maybe work with Codex. I don't get VS Code on the phone, I guess I could resort to Codex CLI from there using Termux or Termius or whatever. But for real work, I do like
11:35being at the desktop and being able to multitask kind of like this. For actual prompting, I have been using WhisperFlow. Maybe I'm a boomer, but I'm digging the whole like speech-to-text dictation thing. You'll be able to see it at the very bottom of the screen, just the indicator while I've triggered transcription. So as I'm speaking, it will be able to record and get the
11:52words. And I've actually set that as like a hotkey, not just on my keyboard, but as an extra mouse button that will automatically press enter and send once I'm done transcribing and sending the words that I'm speaking. That is super convenient because I'm really cognizant, look, I think and I speak faster than I type. And while I am trying to
12:10articulate an idea for this robot LLM AI to try to read my mind, like understand what I'm thinking to be able to then create it in its own canvas. Well, I like to just dump and talk and give it as much information to really try to get across my idea. Hey Codex, say hi to YouTube super quick. Hi YouTube. Thanks
12:30so much. Next, let's talk about the workspace, like the open directory, the folder or the project that you're operating in with AI. And I have a lot to talk about in this one, and I'm really genuinely curious of your opinion, your input, and how you tend to do this and how you work with AI, like your own scaffolding infrastructure and
12:48personal AI ecosystem. Now I know, I think I might trigger a few folks with this, but here is the folder that I'm in. You'll notice there are a lot of other folders, and inside of them even more files that are in like a human-readable title case including spaces for words. Now that's not normally what someone would opt for,
13:06right? I know, hey, the programming purists, originally I was thinking, oh, let's do this all lowercase, let's do kebab style with dashes or hyphens or minus symbols in between each one or underscores, right? And I was there for a little bit, but I got to thinking, look, I kind of want this to double as both a human operator surface as well as
13:24AI. So if it's doing natural English like large language model stuff, I don't really see an issue with it kind of doubling as okay, now a presentable layer for me to be able to have like human logic to look into and explore whatever I'd like. And I have this folder set up as both an Obsidian vault so that it has Obsidian Sync and that
13:44way it can kind of update across all of my other devices if they have Obsidian installed. And we've got pure easy natural markdown capability all in there and a GitHub repository. So, I have a little bit of a backup like I can push, pull, and have a sort of like cloud equivalent of saving my progress. And I'll be honest, I use this folder every
14:02single time. I always open exactly this static directory. I always want to be like anchored in my Obsidian vault because I treat that as literally the second brain, not just for myself, but so that AI and robot has that whole surface. And that is very intentionally like designed and structured so that this could double as both a GitHub
14:22repository and an Obsidian vault and human presentable so that if I were to ever like replicate this with like an HTTP or HTML like web browser front end, I can kind of easily see what's happening where and when and how in all of these different places. And this folder is where I dump everything, where I let AI dump everything. It doesn't
14:44have to throw code in here for projects or software. It could use something external that's still on the file system cuz the GPD Pocket, that device is all on its own that it would like. And I know maybe there's going to be an argument earlier the thought that like, "Hey John, you're killing. You're burning. You're ruining specific
14:59contexts in the context window." But I tend to think a context windows are going to skyrocket, right? That's just going to get bigger and bigger and the models are going to improve. Uh I figure, look, I just want this to kind of be home base, HQ, and this is the anchor that just about everything at least focuses in on. And then I have
15:18like controls over the runtime or the kind of uh structure of everything that I like to have happen here with a little bit of at least scaffolding of skills and markdown and then some sort of front end if I ever wanted to view this entire AI ecosystem or like personal AI infrastructure that I might refer to it
15:34as and being able to see and browse to it all. And I don't know if some folks are wondering, okay, how does this compare to like open claw or Hermes agent or whatever other like always on AI agent? And obviously like this, right? Obsidian vault, GitHub directory opening within VS code and still running codex is not an air quotes always on AI
15:53agent, but I don't always see open claw or Hermes agent or others that in the same way. And that sure, at least there is some like background daemon running and waiting to receive your prompts and inputs and that way you could have it do things in the future like scheduling, but you could do the very same just
16:09without that. So, I haven't really been using open claw or Hermes agent or any of the newfangled big bang buzzwords. But to give it some more of that power, I did end up stitching this together so that this folder Obsidian vault and GitHub repo also sits on top of a self-hosted N8N installation. So, that that way I could still coordinate
16:28workflows or draft them and assign them again with just natural language speaking it into the robot here. And that's been a really interesting thing because I could then have all the same connectors like Slack, like Gmail, like Notion, like ClickUp, like whatever, whatever, whatever, all those integrations that like a Zapier or N8N might provide and I can architect them
16:46and AI and robot can architect them entirely in simple JSON. That even includes some semblance of a credential manager for at least like passwords or secrets or how you connect to those apps and gives me the power to still schedule things to happen, scheduled workflows, any orchestrations, anything that I might want to create whether it's enabled or disabled, like all of these
17:06things that we might want to have with specific timers. The again, just a folder representation and then the N8N workflow as to how it's all built out and information that's kind of appropriate for each one of them and the n8n workflow itself. Having n8n embedded as like a self-hosted solution is actually really cool because it does help me find the balance between what I
17:26would have considered like, okay, the balancing act really of non-deterministic wibbly-wobbly AI timy-wimy wibbly stuff of like, okay, how it could accomplish something in just pure reasoning and inference versus deterministic code like hard procedural line-by-line chronological operations of syntax. I know that I got myself stuck
17:48in repeatedly trying to recreate or rebuild things that would allow me to connect to different tools because I wanted my own. I didn't want to trust some other weird MCP that's out on the internet and I still wanted my own capabilities and wrapping around it the way that I could harness it however I want. n8n helps solve for that both on
18:06the apps and integrations and secrets and credentials. Though I've played with having like a local command line pass or password manager password storage. And that way I can connect and any sort of deterministic workflow that I want while still stitching it and tying it together with the non-deterministic AI capability. And the coolest thing is that I could make some sort of like n8n
18:25to Codex host control bridge. So, whenever it wants or needs any step that should require AI for reasoning and inference and deductive thinking, right? I don't use their static flat AI reasoning block or that node. I actually pivot it back to my own to say, "Look, hey, call my Codex. Call my current
18:46vault and inner vase and environment here so that that way again you're using my subscription. You have access in all of the files and folders and knowledge and skills that are present here. And I could trigger that at any moment, at any point, with any workflow, with any schedule, with any operations." So, with that I can tie together really well
19:04things like Slack or Pushover to have mobile notifications in a smart, strong, and sturdy way. Whenever anything might present itself to me to a certain like human in the loop degree, I tend to call that a handoff surface because it's presenting me something handoff to me, and then I'll just kind of bundle those up and pile them in there. For both the
19:22knowledge and the place for if you want to chip away at code, this is how you can route or guide yourself to go find them in that AI way. That has been one of my biggest lessons learned actually because I think I'm like a little stuck in my ways of oh wanting to think of the traditional programming development like
19:38oh define create some beautiful architected system or application and software to solve all the problems that I want to have fine-tuned and tested and all the conveniences of libraries, dependencies, blah blah blah, beautiful elegant systems and proper production code when I really don't need that or want that and I get wrapped around the axle and stuck with it when I just want
20:00hey AI, can we move quick on this? Can I get the outcome and the job accomplished? I felt that I really needed to lean into skills and non-determinism work. So, we actually genuinely like told it as part of okay, this AI ecosystem, make a boatload of skills and tie those together as to how we should actually consider each of
20:20these and what they do. That's all mixed together here in the vault so we could still track down at least sort of its own map and orientation and let the model and AI guide itself to find what it might need at any point. AI ecosystem is for the concept and idea of this home base. Archive history is just where hey,
20:37some stuff isn't as important, but maybe we just want to keep track of it. I clear that out with some other scheduled workflow. Control planes are things that I want to be able to actually interact with. Those are things that I have in my mind even like conceptually a pillar or pedestal of functionality or what I have
20:52proven now AI can do and I've captured that capability and I want to be able to still keep working with it. We'll talk about the Codex goals super duper soon, but there's some other things in here right of course and end to Codex as I was alluding to and end in and some other things. Even like having
21:08a brokerage to use different web browsers. Because things like browser use or computer use, well, I'm not really wanting that to happen again on my own real computer, but I want to be able to have the GPD Pocket 2 and AI just run and do whatever I kind of ask, still sometime needing the web browser. Things like the knowledge reference or
21:26how we store durable learnings, local projects or things I'm just chipping away at. My devices is some little bit of like device fabric, so it's cognizant aware of the other laptops or computers or servers or things that are just kind of in my own home, right? My personal lab and homegrown environment, and all connected by Tailscale. So that it could
21:43work or use any other resources as needed. Now, finally, after all that talk, I'm sorry, let's get into the more interesting stuff like security research here. And a couple of these are like unorganized random assortment of files here. But the wargames and labs is really what I wanted to focus in on because that was kind of a neat and cool
22:02proof of concept here. A lot of people have been starting to use AI to find vulnerabilities or do cybersecurity research. Maybe it's operating as a SOC analyst, like security operations center, to do triage, to do investigations, or maybe just legitimately looking for bugs, doing bug hunting for bug bounty, finding zero days, maybe kernel driver stuff, cyber
22:23deception, maybe honeypots, maybe just casting infrastructure out of the internet and seeing what happens. There's so much that you could do with it, but I didn't want to boil the ocean to start. Cuz I think it's one thing to just say, "Hey Claude, find me a vulnerability." And then that's it, like, okay, a single turn prompt. But I
22:39feel like you really need some sort of scaffolding, not to use the AI word, but it needs a system, it needs an engine, there needs to be a sort of framework. I want it to be able to work autonomously, I want it to be able to record itself, like see and capture, and so we have historical artifacts, literal all the
22:56breadcrumbs and puzzle pieces to replicate what work was done, and then have proven human observable proof that something was accomplished. Even on top of those, another problem to solve is having the LLM and AI just keep working over and over and over again until the job is done, until it has a finding. Because it's one thing for the AI to
23:17chip away at it, but then if it just at any point throws its hands in the air says, "Okay, I'm done working. This is what we could do next, what we would do next, or I give up, I didn't find anything." Well, then you need to like shepherd it along and babysit it, which is annoying and impossible if you wanted
23:32this to run at all times, even while you're asleep. I wanted this to happen autonomously. I want the AI to keep working forever. Like, don't stop. Don't end a turn. Just keep turning after you finish the turn. Use all the tokens. I don't care. We're on the subscription. So, there was a thought to put this thing in a loop, and that's what a lot
23:51of people called a Ralph loop, right? Where you just Oh, even just slam the same prompt over and over and over again until it's trying to move closer and closer to accomplishing some objective. But, that wasn't super smart. So, now OpenAI and Codex has put together a goal. You might have seen that {slash} goal feature. Right now, at the time of
24:09recording, that goal feature to have your AI LLM GPT 5.5 spin and keep working and infinitely until it accomplishes a goal is an experimental feature in the Codex CLI. You might have seen it if you turn that on in your like config.toml, you can use a {slash} command for goal to just do something ad
24:29nauseam until some criteria is met. This would be incredibly cool for security research, finding a bug, finding a vulnerability. Or, with me, just trying to put this in kind of a small poor man's brain for the moment, uh how about just solving some CTF challenges to get a proof of concept. Now, I can't use goal within VS Code, but because we can
24:51orchestrate so much here, why not just have it use Codex goals via orchestrating them to call and keep working from the Codex CLI. I could have a goal work and run and then bubble up its own artifacts and like its own report to surface that to me or I could orchestrate them so that I could run multiple goals even at the same time or
25:12like in serial in sequence. Once one is done, start another. After I got that sort of goal orchestration concept and put together with some code to be able to manage that, I created what I kind of just call conceptually infinite workers. Where it'll keep working through goals, however many I want, and it can just keep doing them
25:30to infinity because there is no real goal criteria. I literally want to set this in motion and just have it do everything with an unbound completion state like if I were to say solve all TryHackMe war games or complete everything in the CTF, it could just keep going on its own knowing how to move on and I don't need to say or know
25:53how many TryHackMe rooms there are or cybersecurity CTFs are in this event. It just goes. Hey Codex, can you tell me and explain to me what our Codex goals are and how we've harnessed them well inside of our AI ecosystem here? We have the Codex relay and we have the Codex orchestrations and we have infinite workers. Could you give me like a brief
26:14description of what all those are and how they work? This has been just one of the coolest things to have on top of Obsidian vault, GitHub repo, N8N workflows, deterministic and non-deterministic skills and everything, scheduled capabilities and I haven't even told you about the sweetest thing quite yet. Yes, briefly our Codex goals are the way to do it. All right, sweet.
26:35Giving you a little quick crash course probably better than I did here. Yeah, so this is the uh sweet synopsis about how we break it down and how I've conceptually and again I keep saying and I've tried to even codify that idea in the language that I use to speak to robot and codex here is like, I want a
26:54captured capability. Once I've proven something and codified it as a skill, now that AI can prove reliably perform a task, we build them into all of our power in our ecosystem here. So, let me tell you about that other aspect though, and you might have seen it, right? Ludus. This is the part that has been super duper important for what I wanted
27:13have turned into real genuine security research. Like solving a couple CTFs or playing some online war games is a little bit kitschy, but cool. Hey, it spins up maybe Linux. It's already running Linux, but wouldn't it be great if it could actually have more computers or virtual machines at its disposal. So, I have on another server, not the GPD
27:34pocket cuz that's just a tiny trinket toy laptop, but a beefy server machine with huge amount of RAM, big hard drive, blah blah blah, all the resources to be able to have a whole cyber range and control what's Proxmox under the hood, but then just simple AI native, I'm only speaking English natural language, it
27:54can build and create an entire network and create any virtual machine running Windows, running Linux, using FlareVM to do analysis, maybe using Kali Linux to solve a couple war games, doing anything that I might like that AI can drive. Kind of cool. Having that local playground and sandbox for a whole Ludus VM range allows me to spin up what I
28:17would consider like an operator enclave for like, okay, anything that you build or use for TryHackMe or Hack The Box or Over The Wire. Again, these small little primitive ideas so that once we do real security research, it could have everything in its own little environment. And it's tied together with the web browser lease so it can use
28:36that. It can take screenshots. It can record what it's doing with a transcript in the terminal, like the script command simply or with ASCII cinema. And so when I point it at any security research, and again, I'm just using the war games as an example here. Say we ran through over the wire and all of those different war
28:51games, it has been able to make the files and folders for each individual one that it has worked through and completed. And we tell it, "Don't look at public write-ups. Do this thing yourself using the goal to work and solve that challenge and then an infinite goal to move on to the next one. Move on to the next one. Keep
29:09solving these." Say you put that pipeline and engine into like, "Okay, now go find me real vulnerabilities." I think it's going to be cool. I've started to build out the corpus for that, but it's not quite there yet. But, at the very least, I think this concept is neat. And let me show you some of the stuff that it's cranked out. Hey Codex,
29:26we have previously done some work in security research where we've just been working through different war games. Things like Over The Wire and things like TryHackMe. And you put together, honestly, what looked like a sort of a blog site where you had a control surface for me to be able to see and watch your current active tasks and the
29:43goals that you were working on. And I'll be able to actually look through and see the reports for everything that you previously already solved. Could you bring that back up, get it kind of top of mind, and open and start that service and tell me what port that it's running on so I can go review and see everything
29:57that you've built from that control plane. Let's see what it brings back up to the surface here. Good, good. You found it. Security research, war games, and the control plane site for us to be able to explore it. While Codex tracks it down, I want to show you some of the Codex goal relay notifications that I would just get in Slack. Hey, uh sorry
30:16for the edit jump cut, different shirt, different time. My original recording process, like initial recording session, got interrupted. So, returning, resuming, starting recording again just a few days later. So, I have some updates to share with you, but this is my Slack notifications or a handoff surface, the Codex goal relay. And this is where I need to add some of those
30:36disclaimers in that look, this isn't going to look all that good. This is going to be a little bit ugly. This is not perfect. You'll see some duplicate messages, you'll see some testing, experimentation, and the very, very start of this is kind of where it kicks off. Me figuring out how I'd like to have these notifications and awareness.
30:53Uh originally experimenting with just different reports kind of present in the message itself. Um and a lot of these are pretty ugly, pretty tough to read, pretty tough to look at, and not extremely valuable. So, this is me still just kind of figuring it out, letting it do some testing, some smoke tests, responding in threads, having a little
31:13bit of uh display. Remember, at this early point, this is me experimenting, trying to figure out, "Hey, how do I want this to be represented to me?" I thought, "Why don't we try it as like a PDF that it could render and generate out of its markdown report?" Didn't really like that. Eventually just thought, "You know what? Why don't we
31:28make this some sort of button that points to like an HTTP front end or a website, like a blog site, where it's able to write up everything that it's doing?" And this is really cool. Once I finally got that going, I thought, "You know what? Cool. Now try to solve a couple Try Hack Me rooms." Just as an
31:43example. And it went through a couple of them. I tried to tighten that up so that it could then figure out, "Ooh, okay, actually just make sure you include the name, exactly what you were working on." And then it seemed to move smooth enough that I could just try to put this on other war games, like Over the Wire. And
31:59you'll see again, I'm getting duplicate notifications, like three different pings or two for level five, six, seven, eight, nine. And I'm like, "This is actually moving now." I've set the goal in motion that I do genuinely now have this autonomously solving some cybersecurity research war games. Maybe occasionally an issue that it bubbles up to me that I'm able to then go back to,
32:19go see what the robot is doing. But the wildest thing is that I could set this in motion now, and it would just keep going. I don't know how well you can see the timestamps, but this is like throughout the day. And then once it finishes with Bandit, it says, "All right, let me go now move on to Natas.
32:36Let me go try a different war game from that war game provider. And it does each and every one of these in to the point you can see 5:00 p.m. 6:00 p.m. I let this thing run overnight and that was the craziest coolest thing because I just woke up and all of Over the Wire has been completed. I don't need to
32:53scroll through this entire thing for you. I'll kind of put this on auto scroll, but look at how cool this is. You can see Krypton, you can see Leviathan, Narnia, Behemoth doing some like low-level binary exploitation things. And it's like this is wild. It's now 3:57 a.m. I am asleep and I've set it in motion and it has just been able
33:12to go. Little bit of time, 6:14 a.m. I realize, okay, it's now completed maze level nine and that was in like what the website Over the Wire suggested the progression of you as a human player to go through these war games. It didn't tell you about man page, but I just set that in motion and then I'm wondering,
33:29"Ooh, can I have this now parallelize? Do other CTFs or war games or research kind of concurrently?" So, I set it on picoCTF. Solving a couple of these, going back into man page levels for Over the Wire and now they're intermingled because I can now set it to go tackle multiple. Runs through Formula One, keeps going through picoCTF. Goes on to
33:51do the Vortex war game. Now, getting the mobile notification that it was done and it completed its work was cool and all and I think I hope that's neat for you to see, okay, just the series and saga of it running through things is pretty cool. But, I know we're more interested in, okay, what is the actual write-up?
34:06What did it produce? What did it show me? What is in that interactive report? So, let me go ahead and forward the port that it is hosting some of those on on that control dashboard. So, now I'm easy able to access that. So, let me show you how crazy cool this thing is. I have the active work and like what is running now
34:23for infinite workers or orchestrations just to be able to list them out and for me to be able to see them. And then some of the other walk-throughs and write-ups that it's been doing over on the side that I could explore into or any other cards to see how this has been going through things and you can see 233
34:37different war game levels and it solved about 102 of them for just what it's been pulling and queuing up. You can see some CTF time events that it has already tried to stage or at least register for. Again, just experimenting with CTF stuff, but I'll show you that more in a moment. For now, let me see if I could
34:52actually set another one in motion here for us to see if it solves anything by the time I'm showing you all this. Hey Codex, could you please continue working with our pico CTF infinite worker, set that goal in motion so that we can solve more pico CTF challenges. Choose a couple easy simple ones because I do want to be able to present this to
35:08YouTube within a matter of moments. Let me see if it can get that cooking for us. All right, yeah, he is now cruising. Okay, it actually found a previous challenge that it did want to work through a second time just to make sure it got it right within the operator Ludus VM range and then it found a
35:24couple other challenges that it could try to work through. But, it did kickstart the active worker and now it is going to keep solving pico CTF for us, so that is pretty cool. And while that's working on solving some challenges, let me show you all of the war games here. Now, this is neat. There are a couple that it has already staged,
35:41but you could see the ones that it has started to solve and work through like bite man C zero, a couple different try hack me rooms and some of the man page vortex old previous levels from bandit over the wire. Now, I want to show you these reports, but even then being able to browse through this and you could see
35:57I tried to set up a lot of the filters and searches, all that functionality at the very top is very cool. Like let me go look for over the wire, a couple of these and the latter ones it wasn't able to pull an image out of, but some of the other original ones, let me expand the page view here to about 100. You could
36:13see it did grab just sort of a hero image for like that write up in blog post. Not perfect because again, I am just setting this on autopilot. I'm letting it do everything it needs to without me being involved whatsoever. So, let me find the very original initial write-up here, the first one that it tried to create. Do we have a
36:31bandit level zero for me to show you? Yeah, so this is the very start and it's not good, right? You can see a couple weird gross artifacts that are just kind of being left in. You could see at least some of the syntax that it uses here though, because it's recreating everything that it's doing from its use
36:48of the terminal. Let me jump to another one that's a little bit easier read. Level 19 here has a good set of screenshots that it's tried to crop through. And this is when okay, now I've helped it understand this is how I like the write-up and walk-through presentation. We could see a little bit more of the communication, what it's
37:04doing, how it's doing all this, and then of course the flag or the answer or what we actually want to see as proof that it has been solved. Let me jump back to the previous page and not the best UI UX, right? Again, Codex is the thing that generated this. So, for the people screaming, you know, the front-end
37:20functionality is not the best. Here's like a new thumbnail level four. So, this is doing a little bit more hardcore work. SSH only binary exploitation level. Official page does not provide a hint. So, it's trying to make sense of things and this is kind of crazy because hey, binary exploitation level has no stack canary, no PIE, no executable
37:39stack. So, they go through and figure out some of the shell code that they might need. They put this thing all together and then run it and then do everything. And it's working. Like, there are a couple that are blocked or it has staged as like a candidate, but for a gist, it is doing uh it. It It's
37:59doing the whole thing. It's working through the process here, taking screenshots, knowing it what it needs to do on the command line, reflecting that all. Let me not search for OverTheWire, but let me actually just change the provider to whatever we want and you could see kind of all the functionality that I've kind of given myself here. Let
38:16me toggle this to TryHackMe so we could see a couple of these and >> [laughter] >> Low-fi would be a good one because that is a room that I had prepared. Folks might know this was original challenge that I made for a past Nam Con CTF, but you can see it's doing some local file inclusion and it's literally getting all
38:33the screenshots and everything to prove and validate, okay, that work has been completed that I just spoke into existence and it would run and do the thing. Even with the images and everything that we need, here's the hack to win that it's using the attack box on its own. You can see the split screen kind of open up and then how it tackles
38:52that challenge. Just small simple stuff to just validate that it works and how it actually progressed through all of this. Some of the screenshots being duplicate, again, not the best, not a perfect cropped here, but it's doing it. I mentioned I had some updates from the past few days that transpired, so I'll show you those, but first I want to
39:11check in on our robot to see how he is doing with the current picoCTF run. I'm using speech-to-text with WhisperFlow again, so you'll see that indicator at the bottom of the screen. How is our picoCTF infinite worker research doing? Has it solved a new challenge? What's the latest? What is it working on right now? Can you give me an update? Do we
39:27need to move on to a different challenge for the sake of speed to have something to show? Cool. It's double checking it right now. I'm hoping it pops up the Slack notification like while we're recording live because that'd be pretty cool. Oh, it did solve it, but it hasn't marked completed yet because it is still, okay, carving through the
39:43artifacts and clean up. This will still pop a Slack notification for us, correct? So we'll be able to see it. Yeah, it hasn't fired it quite yet, but I'm hoping it will soon. All right, well, let me keep showing you cool stuff. So, back on the Slack feed here just to kind of have a quick summary of
39:57what it was doing. The other day I did try to finally point this at real software to try to find vulnerabilities, weaknesses in genuine applications, not just funny, silly, haha internet points, internet games. And this I didn't get working all that well. You can see there's a significant amount of spam as it did try to keep evaluating the lab
40:16environment and not actually proceed through real genuine research and it just kept telling me hey I'm still going I'm still doing it over and over and over again and that's fine that's fine I just like mute notifications on this chat and just let it keep spinning but while I was waiting a little bit bored I said hey shouldn't you have been
40:32registering and like auto solving some CTFs on CTF time just cuz I was wanted to see if it could thought that was neat so I pointed again at TJ CTF which it didn't automatically register for but now it did as an experiment cuz I told it hey we should get back up in action on that and you can see it tried to
40:49solve and go through real genuine challenges for TJ CTF that was running over this past weekend oh IT'S ALL THE CHALLENGE IT'S ALL THE challenge new MESSAGE DOWN BELOW >> [laughter] >> LET ME GO right back to the overview page up here do we have one is there a write up yet where is it at it didn't
41:07build out the write up for me yet he went to go do new challenge but I don't see the bite mancy one write up quite yet I saw the slack notification but I don't see a write up being updated and adding into the control plane dashboard where's it at we found the write up itself it was updated in the challenges
41:24folder the gap appears to be servicing and indexing yeah can you tighten that up please all right it's syncing the write ups right now it should have done that to begin with but this is where I'm super curious about your input your take and your opinion if you're willing to help me out in the comments cuz I feel
41:39like well building systems like this building the sort of engine does feel like there are a lot of moving parts I don't know if I'm overcomplicating it over engineering it but the benefit of me to be able to see and look through and read and watch and observe this is something that I really value so putting all of this capability
41:57to be able to take screenshots record yourself have the script transcript that you could recreate for the write up in the handoff service doing debugging doing validation again I'm finding the balance between like the non-deterministic reasoning that I want the AI to be able to supervise and like add oversight to and then the deterministic okay present this to me in
42:18a way that I can actually see it and give yourself enough guardrails to make sure this all actually happens. Okay, it looks like you figured it out. Status was just set to completed but it should have been marked as solved and now it should display it. Oh, yeah, I don't even have to refresh. You're starting on old sessions and you're beginning that
42:35walk through right up now. But what about Bitemancy number one? Oh, wait, he might have just updated the original write-up because that existed before and it was just doing the Luteus Foundation now from the range. So, bad crop on that screenshot, better on the second one, but does complete the challenge, goes through the actual task and completes
42:57it. >> [laughter] >> Okay, can't complain and we can see it getting started on this old sessions challenge and making further progress on that. So, fingers crossed we'll see that one complete. Anyway, let me show you some of the other stuff because I was alluding to the fact that I was exploring this with uh some software. It
43:14is not working at the moment. It is not actively researching with a infinite worker right now, but I just for playground's sake was experimenting, "Hey, how well could you do this pulling down and installing and setting up like PaperCut or CrushFTP or just things that I would be curious about. And this is neat. CrushFTP thinks it did find a
43:36vulnerability or at least a finding, right? The approach here is having it look through with certain amount of hypotheses to explore those as like actual atomic units of work. And this is gross. Again, that this display output is not ideal here, but little bit of a feed to see what it's doing where and when and why and how and then some of
43:56these hypotheses that it is exploring and what it might still be wanting to explore and track down for potential issues. Can we find RCE? Can we find SSRF? Can we find arbitrary file read, arbitrary file upload, blah, blah, blah. Kind of sweet. It thinks it had a finding, so I just want to look and see what it actually produce here. Again,
44:17not the prettiest sort of front matter, but it is thinking there are things. I need to go through and do my own manual human validation of this, but again, just the concept, just the idea, setting this in motion, having it be a factory to just crap out potential bugs or research leads is wild and cool. Being able to
44:39present this all to me in a way that I could just dive in and go see, oh, is this actually legitimate? Is something that I would never imagine. I feel like I'm at the very, very start of this, and maybe I'm behind. I know a ton of people are already doing this kind of themselves, and maybe they've built out
44:53better systems or pipelines or engines for this thing, but I'm still experimenting and trying to figure out, how do I do this myself? Let me get back to the war games because I wanted to show you TJ CTF as like a real little capture the flag that was live just the other day. And shout out, love you guys.
45:08I'm sorry. Hope you didn't mind. But we had this now register autonomously for the CTF, sign up and play, and then it explores all the challenges, and it took a big screenshot here of literally all of the challenges, and it kind of duplicated the screenshot over and over and over again. Uh I would have really loved the crop, making that a little bit
45:27easier to read. So I still need to fine-tune this, but okay, it just went for treasure hunt, and it looks like it hasn't specifically broken these out into each individual challenge for different CTF time CTFs quite yet. So I need to improve on that, but it was able to solve just a simple, okay, exploration of web challenges. And then
45:45it tried to do some cryptography challenge. Again, giant screenshot of the interface. Let's scroll through that. But it uncovers this and finds the bit of code that it needs to unravel it and recreate it, and it solves the challenge. It's validating from its own profile page the amount of solves that it has and its current points and then
46:06it moves on to another more graphical one which was very cool to see. Okay, it was able to now put together and like OCR to a certain extent what is in these images and that is just cool. I didn't really let the TJCTF run for too long cuz again all just kind of proof of concepts here. This is playground fun
46:23experimentation just to see if it could and very obviously it can. I don't see a lot of value in using something like this to be able to genuinely play or participate in CTFs but using this as like a stomping ground for genuine learning for it to go through any number of these old past challenges to use that
46:43as research data like a corpus of intelligence to now point it at any other amount of software to go find real vulnerabilities and have something that it could go use to reference and guide itself through some similar scenarios. It's like building up its repertoire of skills the same way we do as people. And I tried to tell it like give it enough
47:01guardrails to say, "Look, you're not looking for any online write-ups. You're doing this yourself. You're solving it personally." And we can see like their stream of thoughts, the agent transcript to like explore exactly what it's doing when and where. And this isn't all that pretty. That's kind of a still disgusting and I need to clean up that
47:18UI UX but it will provide like recommendations as like, "Oh, how could we make the lab even better? How could we make our process even better?" If it's missing some sort of tools, if it needed some other access or other capability to actually accomplish something autonomously. And fingers crossed we'll get a completion of this old sessions picoCTF challenge. Is the
47:37picoCTF infinite worker still working? Uh is it completing now that old sessions picoCTF challenge? The neat thing is that I can just kind of ask the AI, "Hey, what's it doing? Is it almost done? Can you give me kind of an ETA? I don't want to have YouTube waiting for too long. It's already a long video." This is hysterical. Ah, maybe it's uh
47:55worthwhile talking point uh automatic compaction of the contact window. Like obviously, right, I have a ton of stuff in this Obsidian vault and this folder and sometimes that could bloat up the context window, but auto compaction is good, in my opinion. Really genuinely curious to hear your thoughts in the comments. I hope you give me some info
48:11and insight from this video. Oh, it has the flag. Okay, cool. So, it's just finishing it up and then it'll crap out a write up for us. So cool. I'm waiting for the Slack notification. Oh, it finished. It finished. It finished. >> [laughter] >> All right, back on the control
48:31dashboard. Look at that. Short web challenge about the risk of sessions that stay valid for far too long. Prompt describes a site where the user can log in once, never has to log out again. There is some good screenshots of it starting up the instance, exploring it, and finding a couple sessions. >> [laughter] >> And it just replays it and that's it.
48:52Okay, so obviously, 5, 10, however long that took, is longer than it needed to for like, okay, if we were to be going through this challenge all on our own. But setting this thing in motion, maybe to play alongside you, is still just mind-blowing to me. And yes, obviously, we are absolutely at the point where, look, AI can do this. AI can find
49:13vulnerabilities in software. AI can work through the security research exercises, war games, CTFs. And it's not that the world is changing because of AI. Look, the world has already changed and I want to be part of the party. I want to have a seat at the table. So, I'm playing along. And I hope you are, too. And I
49:30hope maybe this video gave you some ideas, gave you some inspiration, and uh, hopefully didn't leak too much of myself. Thank you so much for watching, everybody. Really hope you enjoyed this video. Really hope you are still wrestling with your robots, hanging out, playing with AI. And I hope you're giving some love to all the incredible new AI innovation in the security
49:48industry. Of course, big shout out to Wiz for sponsoring this video and their incredible work. Link below in the video description to hang out with them and please do all those YouTube algorithm things. Like, comment, subscribe. I'll see you in the next video.