Full transcript
0:00All righty, hi everyone. Thanks so much for tuning in. Look, I'm super excited to finally get a chance to chat with Estelle, Estelle Roulin, threat intelligence researcher over at Flare. And my goodness, Estelle, you've been doing some super cool stuff. I know, running around for different conferences, doing research, Black Hat, Def Con, etc. And I think you've been
0:18cooking up something really slick to dig into stealer logs with some LLM or AI magic, is that right? Can you fill in the gaps? But I guess first, hey, let me know who you are and what you're up to. >> Hi guys, I'm Estelle. I'm a cyber threat intelligence researcher. It's basically a big term to say I'm a data scientist
0:33applied to anything cybercrime because it's it's fun and intriguing, you know? So, I really love playing with shapes and colors to give data a bit of life and so we can see what's happening instead of just reading a thousands of lines. So, yeah. Recently, in the last year, I've been moving around a lot to present some cool stuff and uh mainly
0:51with the use of LLMs and AI to cybercrime data. And here in this case, to stealer logs. For those who don't know, stealer logs are the product, basically, of info stealer malware and it's essentially the whole numerical identity of a user from a compromised device. We have plenty of things. We can have passwords, the files, family photos
1:11from the computer, browsing histories, softwares and processes. So, it's very complex. It's really like a gold mine of information. The only thing that's annoying when you're an analyst or a researcher is that there's so much going on inside those stealer logs that it takes ages to basically analyze and to understand how the user got infected. That got us thinking we could basically
1:32use an LLM and AI to do the job a human would take many, many hours to do. So, that's basically the genesis of the idea behind what we call stealer lens. Essentially, we give the raw stealer log and then the analysis magic is happening and at the end you can get, basically, how the infection happened and all the
1:51evidences. What's really interesting here it's basically the same way a human would do it. We essentially taught the LLM how to do it. So, it's basically a human logic behind the analysis, so you know what to look for. We had to teach or to tell the LLM, "Hey, for the softwares, if anything has like kind of a weird name or for the processes, and
2:11you can see like a hash type of name, then maybe that's suspicious." The same way goes for analyzing the browsing histories, like file sharing platforms or YouTube videos mentioning crack or a hack could be essentially suspicious. Here, the biggest challenge we had was to transmit that knowledge through the LLM and make sure they could apply it to
2:32all the different sections and artifacts from the log and do not hallucinate because we do not want like a fake hypothesis, obviously. So, now it's computing. We give it a log and then here you can get some information on what's going on. So, here you can see we are analyzing the browsing history from all the browsers available in a stealer
2:51log, so it's processed. Then we have analysis where we cross-reference the browsing history with the software installed. What's interesting is if you have a browsing entry about, for example, a YouTube video on a GTA enhanced mod menu and then 5 minutes later you have a software that was installed named GTA enhanced mod menu,
3:12then that's a really big sign that it could be the reason behind the infection. So, we did a cross-analysis of both of those artifacts. Then we go on to the screenshots. Some stealer logs include a screenshot, which is basically an image taken at the moment of infection of what was happening on the screen of the user. It's funny at first
3:31sight, but it's really interesting cuz we can see what was happening. So, here you have the image analysis performed. We have a description, essentially. So, here we had a screenshot showing a YouTube video and a title here in probably Spanish about a Google Samsung unlocking file. Then we had the URL from the the screenshot and then we have the
3:51initial vector, which here was nothing in particular. Then we extract from the system's data and the path of the stealer log, the infection date, and then we have the local date that was saved on the computer. Once we have analyzed everything, we output an infection hypothesis, which is here. And here the answer is basically infection
4:11happened because the user most likely downloaded a malicious file. This guy is tagged as a Samsung FRP unlock tool. Then we have the theme, which is essentially a strain, saying of the same thing. We have a suspected malicious file and here no any file was identified. And then you have a suspected URL leading to a potential malicious file. Underneath you have the
4:32supporting details. So, it gives you evidence to back the hypothesis. Here we have the browsing history saying that we had extensive activities around the downloading of Samsung and unlock tools. We had suspicious processes with random names. We had suspicious software installed including phone unlocking tools commonly abused. Then you have the infection dates, which matches some
4:54period of intense activity of downloading and browsing something related to Samsung and so on and so forth. So, that's the first part of stealer lens. It gives you the hypothesis, basically. The second part is we developed an agent with which you can talk about the case. So, if you have any question, what was data analyzed, if you want some more further detail, you
5:13can ask anything. So, what I will start with is ask what's the data that was used and available. Here we had system infection details, the browsing history, we had a process list, some software, but we had no clipboard data available. So, it kind of gives you some context on what was used in the data infection um hypothesis. You could ask for a summary,
5:33which is essentially giving you back um the hypothesis in case you asked too many questions and you wanted to get it back. And then you can ask some natural language uh questions. For example, what are the five browsing entries closest to the infection moment? Then the agent will go and look into all the files and analysis that were performed and then it
5:53will give you an answer and hopefully you can have your basically what you're searching for. Here, it's saying that in the default Brave history, we had five entries of browsing close to the infection date, which was on the 17th of June. And then you can see search for software and then you have the URL again, FRP tool, which is really
6:11relevant in our case and all this. It also gives you observations, significance and when it's lacking the data, it will also tell you, "Hey, this might be happening, but I'm lacking some details, so I wouldn't take my word for it." The next thing you can ask to this personal hypothesis is a competing one cuz sometimes the main hypothesis it's
6:29giving you could be flawed and so you could basically discuss, "Hey, maybe that's false. Could you give me an alternative to what you were saying? Give me a competing infection hypothesis." What's really interesting is we tried to basically add some personality trait, quote unquote, to make sure it evolves as an analyst and can recognize its flaws and basically
6:50the bias as well, so it can have the limitations and also the counterpoints when needed, so you're not completely biased or there's no hallucinations going on. Here, it's giving you another infection hypothesis about a remote access software because it had a screen connect software installed on the device. How this competes with the primary hypothesis basically focused on
7:10the self-execution of a backdoor unlocking tool, but then maybe both scenarios were possible and it would need some more investigation from the human to make it sure which one is the best. Here you have the limitation of the hypothesis. There's no direct evidence provided in the data to confirm remote access was used for the compromise. And then you have actionable
7:29recommendations. I think it's cool because analyzing a stealer log takes up a lot of time and usually you don't have the time for two different people to analyze a single log. And so sometimes the analysis can be quite a lonesome process. So, having an agent to discuss can help you order your thoughts and get another opinion without you spending a
7:48few hours explaining all your findings because they are already given to the agent as as context, basically. So, if if I may, just to kind of zoom out, I think what you have here is like normally, oh, when you have a stealer log on its own, it's just could be bundled up into a zip archive from all
8:04the different folders and files and text documents that it has, but that's the sweet collection of whatever the victim and the target had on their computer, like the software running, the screenshot at the moment they're infected, any other information. But because, just as you mentioned, that's so much stuff, manually going through it, I guess what you can do with your
8:23tool, with this stealer lens, is then try to walk back and figure out how were they infected, like actually, and kind of interrogate everything else on the system at that point, which is super cool because then you got that insight and understanding of what were they doing. And you could, just as you did and you showed, like, "Look, this
8:43YouTube video was totally the one that maybe, probably could have been the infection point, right?" And even then, that's another open door for more threat intel of like, "Oh, I know that's bad. I I see this one." You're collecting across this even at scale. So, this is super cool. Yeah, definitely. I think it's really interesting as well cuz when you look at
9:02inside the log, you see basically that everything that is going on you can check, essentially. So, it helps you pinpoint what's to be checked. For example, here you can see the screen that was given to the agent and you can see it's the exact video with the title. You can see there's the URL. But on the same time, when you search for basically
9:21like the URL that could lead you to a file, it helps you pinpoint that one because when you go inside the history on some of the logs, you have so many entries. And if I had to check every single one by reading them, it would be like basically a nightmare. But then here it was pinpointed in the analysis
9:39of the agent that this one was really suspicious. So, then it helps you narrow down what is essential for the further investigation and then that's even more time gained, essentially. Yeah, and you can pinpoint that malware specifically if that is there, so that's so cool. I do see a couple uh and I forgive me, I know I'm eyeing a little bit of your
9:58Visual Studio Code here, um, but I see Nova rules. Were you doing anything with the sweet O AI text to determine what's good, what's bad, what's ugly? Do you have that mixed in a little bit? But the code here is so cool. Yeah. So, we, when we presented the screenshot description pipeline in Black Hat this year, uh, we met Thomas Roccia,
10:18which is the author of the Nova rules framework. For this demo, I had to disable them >> [laughter] >> cuz I wanted to ask some, you know, red team stuff and I wanted to get some details. But essentially, we put the Nova framework in the code to make sure that if we one day release the whole thing, uh, we could check for any
10:37question that was being asked to Stealer Lens so that the question was not harmful to the person from the stealer. So, are we trying to jailbreak from the agent? Yes, no. Are we asking too much PII? Yes, no. Those are the things that we wanted to prevent if we were to basically release some of those. The Nova framework is really, really good.
10:57Honestly, it catches 100% of what I've tried and doing for the the red questions, quote unquote. So, I'm excited for this and to add new rules as well. Have you tracked down any other really cool discoveries, any findings, any other, I don't know, sweet stories to tell from using this and kind of seeing it in action? Are there any cool
11:17case studies? Yeah. So, Quinn, Quinn is a stealer log from a bad guy that I analyzed personally, manually, last year. And I wanted to see if basically Stealer Lens was as good as a human to do the analysis. Well, it's funny when you have a stealer log from a bad guy is you can see what they were doing on the
11:36computer. Ooh, okay. So, this infected a threat actor, right? Is that the case? Gotcha. Exactly. So, if I go and show you the log from Quinn, the screenshot here is pretty self-explanatory. So, Quinn was infected using a LinkedIn checker, uh, that they downloaded from a forum. So, you we'll see that in the
11:56analysis and it's just the perfect testing subject because then you can you don't have any like remorse to check some more red kind of stuff. So, here you can see it analyzed and it found that it was a LinkedIn checker and then we had some really interesting processes about a proxy and then you have not much going on in the log besides that cuz it
12:16was not that complete from a minds. But I wanted to check on the processes cuz if we ask for a competing hypothesis, it won't mention those those processes. And I remember when I was studying this log, the opsec was very terrible. So, that made it a really interesting log when I was trying to learn who the user was, but I was also very, very
12:37confused on how it got infected because on one side you had the screenshot where you could clearly see you had a LinkedIn checker and on the other side you could see so many of those processes about proxy something. And I couldn't get to the end of it, to be honest, as a human. And I think it showed how the analysis
12:54is going on inside the agents is pretty truthful because even there, when you ask for a competing hypothesis, it will give you something about proxy and it's quite convincing and it conveys the same doubt I had when I was studying basically this log. So, that was really interesting. I also, I won't be showing it here, but for the purpose of this
13:14same investigation about this log, I was deanonymizing the user and I thought I might just automate this deanonymizing idea in the same way and that proved to be efficient as well because those logs are not the funniest to find the infection theme, but it's the funniest to find the opsec that sometimes can be as bad as a grandma, for example. So,
13:35that was kind of a funny discovery when I when I use it. Hey, super sorry to interrupt, but real quick, I did just want to add a little bit more context as to the sheer size and scale of stealer logs and all this data. This is actually a sweet new update from Flare. Their new collection page actually breaks down all
13:53the different stealer logs, chat messages, community events, and other information out on the dark web that they're collecting. This is all the telemetry and insight that Flare collects to help track down your identifiers or your exposed attack surface on the dark web. And look at all this stuff. You can see what, like 24 million events over in the last 3
14:14months, 12 million events for stealer logs, and you can drill down into this. Take a look at see details. You can see the change over time, even some of the categories of like Redline info stealer, Luma C2. Actually being able to see the rise and fall of some of these different malware families. Things like chat messages that are shared in different
14:32Telegram groups or even Signal groups they're tracking now. But look, this is like a billion Telegram events in the last 3 months. And we can click view events to pivot to their global search here and then drill down, interrogate any one of those. Think of ransomware leak sites, think of these dark web marketplaces, even in the source
14:49directory, like we could drill down to get a quick summary of the different communities, when they're tracking what events at what timeline, from where. It's pretty slick. Even at the very top in the navigation, you could see some of the specific Telegram channels that they're tracking. That's a lot. Actually, remember we could take a look. We can go see the details for Telegram
15:09here. Almost 4 billion events, 3 billion of that being just this year. That is absurd. And the named breaches tab even clues you in as to some specific like newsworthy, headline-making different data breaches. And what credentials might be included with that. That's actually pretty cool to get like a description and a fine understanding as
15:29to what PII was leaked from there and if any of your credentials are present. And my last one here, I know a whole lot of folks have been asking for a sweet feature to submit a collection request to Flare. So, in case you're ever wondering, is Flare tracking this specific Telegram channel or this specific dark web onion site, now you
15:46can actually straight up ask them, make it super easy. Look, I want to say track this other website, this other resource, and then you could have that added into Flare's data set. Super cool. That routes directly to their collection team so that all of their researchers, including Estelle, can start to go through it, maybe hunt down some more of
16:02those stealer logs, maybe do some of that sweet LLM AI processing and all the awesome stuff that Flare is doing to get threat intelligence out and about and to protect your exposed attack surface. You know I'm a fanboy. I hope you give Flare some love. Link below in the video description. Let's get back to Estelle. Yeah, that makes me wonder, I guess,
16:19what else or what future or what further, I don't know, are there more development ideas or more things you want to add to this? Are you feeling like, oh, this is sort of done for this research effort? Or, I mean, I'm bouncing [clears throat] all over the walls thinking like, whoa, you could make like a stealer log MCP server, get
16:35that into some whole workflow to do this again in like a factory to analyze each and every one of these that you all, Flare, of course, is tracking all the time. Yeah, so right now I we have those two tools, which is Stealer Lens analyzing how someone got infected. And then we have a red team tool to get the
16:53identity of the bad guys from the stealer logs. Those two are very separate and I don't think I would combine them because they have two different purposes and we don't want them to make the red team one public in any way. But the Stealer Lens is really interesting cuz we could basically use them on any stealer log to give more
17:09intelligence on, hey, this is why it has been infected and then you can spread the word on saying, hey, we have this much percentage of infection that are due to this kind of trap. Um, right now I'm working on a campaign tracking project where we use this kind of pipeline on tens of thousands of stealer logs and then we did some clustering and
17:29a bunch of data science magic and now we're able to track some campaigns over time and alert when there's a new one coming up. This is really interesting because it's basically a weather system daily to say, hey, here is today's weather on info stealer. Here are the traps you should be avoiding and then we can basically tell the public, hey,
17:47today is looking pretty cloudy on the Roblox or on the Counter-Strike Global Offensive sector, you know. So, I think it's a really good way to tackle the threat of info stealer and so we can spread the awareness to avoid the infections. That is so cool. >> [laughter] >> Awesome. What do you think, Estelle? Is there more to showcase? Do you have any
18:05other sweet fireworks? Or, I mean, I'm blown away already. I don't know how much more you have up your sleeve, but this is so cool. I think this is the best visual I can give you for for Stealer Lens, to be honest. It's more textual than anything, but yeah, the essential is here. >> I'd love to ask, if it's okay, I know I
18:22I think a little bit of your background was really studying, if that's okay for me to get into, the criminal behavior, criminal psychology, a little bit of how did you get into this and then how did you find, you know, maybe this route and this angle using AI, which is super cool and super powers in its own right. Can
18:40you tell me a little bit about that upbringing or how you got into threat intelligence in this super sweet method here? >> Yeah, so when I was in high school, I specialized in arithmetic and basically mathematics and I was doing some computer science as well and then I got bored of it and got interested in psychology and more specifically
19:00criminal psychology. And so, that got me into studying criminology in Canada and then I ended up thinking, hey, criminology about physical crime was really boring to me cuz it was studied for like maybe 50 years now. And so, I decided to combine criminology with the mathematics to go into cybercrime, which was new and exciting to me. So,
19:21I got into this basically space linking two very different subjects and that got me into being a data scientist for cybercrime data and I think it's really helpful cuz most of cybercrime data is just text and I'm too lazy to read 25,000 lines of text. So, I ended up doing it for myself in the beginning because I thought, hey, those algorithm
19:42and graphs could look cool about this criminal phenomenon on the web and then it spiraled into using LLMs to automate tasks I was doing, uh, repetitively. so that got me into using shapes and colors to basically explain the cyber threat landscape and using LLM to automate some workflows that I thought would be automatable. Well, hey, honestly, I
20:04think that is a genius approach because I'll be the first to admit when I have or see or look at a stealer log, normally I I know all the info is there. Like, oh, the processes that are running, the software that's installed, maybe their browser history. But because there's just so much there, I scoff and move it aside and don't really care.
20:23Like, I look for the screenshot cuz I think that's cool and flashy. But having all of that immediately ripped through with an LLM is so cool. And then you could do that for, as you mentioned, tens of thousands or hundreds of these just to get that in mass. So slick. >> [laughter] >> Yeah, it's definitely exciting to be in
20:40the era where LLMs are going to used in very different scenarios. But I think one thing to remember is the LLM can't think for you. So, you have to explain every little micro step you're doing as a human in your mind for a certain like task. Usually when you we're human, we have heuristics as in we know the
21:00definition of what is something financial or what is something suspicious. And then we hope the LLM will have that same definition. But if we don't explain that definition, it's going to be a problem because the LLM will have its own and then it's kind of a it becomes an issue. So, you have to really dissect basically how you do
21:17things as a human so that you could teach the LLM on how to do it if you want like a human-like output. It's really something you have to wrap your mind around. But as long as you can do that, it's going to be an amazing time with LLMs. Heck, yeah. Well, thank you, thank you, thank you so much, Estelle.
21:32This is awesome. I really appreciate you showing me. I think it's a phenomenal idea. I'm excited to see it grow. I know you're going to do so many other cool incredible things with it. And Flare always just has this extremely impressive data set of everything that you're tracking. So, big shout-out, big props, kudos to you all. Thank you again
21:48and again. Anything more for us to dig into or hey, any call to action, any sweet stuff? Hey, go check out Flare, what they're up to. >> [laughter] >> Yeah, check out Flare, check out the research and hopefully we have some more graphics and cool stuff to show you a very soon. All righty. Well, we'll have some links in the video description. You
22:05can take a look at what Estelle is up to just as well. But thank you so much, everyone. And we'll catch you next time. Thank you.