Full transcript
0:00Recently, I've been playing with the Nuclei tool from Project Discovery. And I'll be totally honest with you, I've never really used Nuclei before. Like, I've heard the name and I knew it exists, but I always just thought, "Oh, that's more in the bug bounty world that I'm not usually in." And honestly, I just never tried it. But now, I realize
0:18Nuclei is the tool that I have always dreamed of. Like, conceptually, the idea, scanning for different kinds of vulnerabilities or findings and things in mass, it's what I always wanted but never knew it already existed. So, Nuclei is awesome. Let me show you. I'm here on their GitHub repository, and of course, I'll leave a link for that in
0:38the video description. Nuclei is a fast, customizable vulnerability scanner powered by the global security community and built on a simple YAML-based DSL, enabling collaboration to tackle trending vulnerabilities on the internet. It helps track down vulnerabilities and applications, APIs, networks, DNS, cloud configurations, and
0:58a heck of a lot more. But Nuclei boils down to using these super simple YAML-based templates, like a configuration file, or maybe you might be used to in sort of YARA or Sigma or really just a lot of tools now that use YAML to define anything it could do and make it extremely extensible. But rather than YARA or Sigma or whatever, this is
1:20like fully in place for looking for vulnerabilities, scanning for different findings that are worth exploring a little bit further. Now, we could do this from the command line. could kick the tires and get this up and action if we were to install Nuclei. They include a guide in their documentation, and it is super easy to install. It is
1:39literally a single go command to go install. You do need the latest version, but we can go get that super easy. So, to play with this, I'm inside of my Kali Linux virtual machine. I can hit control-alt-t on my keyboard to open up a terminal, F11 to full screen, control mouse scroll to zoom in, and let me
1:56paste this in. But you'll notice I don't have go installed, so I can super simple go ahead and install that with golang-go. It will prompt for my password. I can use Kali as that is the default, and we'll hit yes to get this installed. Once that is installed, I'll paste in the go command again to go ahead and
2:14install Nuclei. It takes a little bit. There are a lot of dependencies because this tool has so much power and so many different features. But once this is done, I'm back at my prompt, and I could try to type in Nuclei, but it's not yet in my path, and I don't want to install it from the repositories cuz we just did
2:30it via go. But now, it should be in a go directory, kind of in your home profile. Inside of the bin or binaries folder, we have Nuclei right here. I do actually want to add this to my path just because I had just installed go and got this set up. So, I'm going to edit my shell configuration. I'm in Kali Linux, so
2:48just zshrc. At the bottom of this file, let me just export the path environment variable, set that equal to the current value of the path, but with just probably my home Kali go bin. Go beans. All right, save that file. Let me reset or just probably dot slash our zshrc.
3:09Now, no matter where I am, even if I were in my home directory, I should be able to run Nuclei, and you can see that's highlighted here. All right, Nuclei version 3.5.1. The first time you run this, it won't have the templates installed. So, it will go ahead and grab them. There we go. That'll already be downloaded and
3:27available in the directory, and take a look at this. Current Nuclei version, we all running the latest cuz we just downloaded it the way we should have. Same thing with the Nuclei templates, those were just downloaded, but there are almost 9,000 templates now, which is absurd, 8,701 time recording. Back over in the GitHub repository, we could take a look at the
3:48command line flags with the dash H or all the help output, and there is a ton of stuff that you could do, providing a target, maybe a list of targets, things that you'll exclude, and you could provide the templates and what you will do or even filter on different sort of responses and results. There are so many
4:08different options for configuration and how you set the behavior of this tool. This goes on forever, and I don't think I could really cover all of it, but look at all the stuff that you could tweak and tune as you use Nuclei. But now that we have it installed, let's go ahead and do something. Let's use the Nuclei
4:24scanner. Back on our command line, we could use Nuclei with a dash target to specify a target or dash U like you might have seen for a shorthand notation. So, we need a target, something to scan. So, for this demo, I'm going to use some of the labs that come from Hacking Hub. Hacking Hub is another online training provider or
4:42cyber range, tons of different labs or hubs to be able to practice, sharpen your skills, and learn more cybersecurity. Put together by some awesome people in the scene, you might recognize Ben Sadeghpour, Namsec, Adam Langley, Build Hack Secure, Todd Bailey, and there's me. I guess I'm there, too. But one more thing we're super stoked about is a brand new Nuclei masterclass
5:03course, and I do have a voucher for it if you want to snag that for 50% off. There's a code link down below in the video description, and if you want to learn a little bit more Nuclei, you get all that sweet education from Ben Namsec himself, and it's really walking through a ton of real-world scenarios. Over five
5:21sections, 27 lessons, almost 2 hours of that practical training, and a ton of stuff to dig through. So, I hope you go take a look, link below in the video description. And 50% off. You can use the code at Nuclei50, or the link should automatically apply it down below. And I will say, last one, sorry, this is going
5:38to be included in a lot of the sweet Black Friday deals that Hacking Hub is offering. There're going to be some sweet bundles with the Linux for hackers, regex for hackers, hands-on web application exploitation, blind XSS, cross-site scripting, and of course, this sweet Nuclei masterclass. So, there'll be a whole ton of promo for Black Friday stuff. Hope you get your
5:57hands on it, link below to video description. All right, let's finally do the thing. I'm going to open up Firefox, my web browser, just to show you what web page we're going to be up against here. Hacking Hub has spun up a domain for me, aquamarine.ctfio.com, and I do want to go to the dub dub dub front end on there. So, we have
6:18Neurocore Labs, advancing neuroscience through blah blah blah this stuff, but there's not a whole lot for us to be able to see just from the page itself. So, why not use Nuclei to see if it could scan and uncover as much information as it could. Let me try to do that. Let's use dub dub dub aquamarine.ctfio.com,
6:39and let this thing go. Might take a little bit of time, but take a look at this. From the 8,701 templates that it's been able to use, it actually ended up just shrinking that down, clustering of some of the templates down to 1,825 or so. And we'll talk more about the templates in just a sec. But first, I wanted to have you see
6:58this thing in action. Oh, tracking external service interaction. Oh, also detecting maybe a web application firewall. It does see Nginx running behind this thing, right? And we could kind of go validate that. Like, when we connected to this web page, if I actually take a look at the browser tools, let me open the network tab, and if I tried to reload this page,
7:20it should be able to give me one of the headers here that tells me, yep, this is Nginx. Validating that manually, it at least saved us the step, right? But imagine a tool that does that scanning, check, verification, and all that for any different asset across however many of them you want. That is Nuclei. Look, it even saw, okay, some simple robots.text
7:42endpoint. So, slash robots.text, in case we forgot to go take a look, told us there is a stats page. There's a research page. Let me jump to that. Let me get to slash robots.text. Sweet, it found all those. What is at the research page? Ooh. How about stats? >> [laughter]
8:02>> Okay, cool. I found a flag. Nice. Whoa. Okay, even got a email address here. Saw that from the front page, it was able to just extract that out. Found the Nginx version from that header, and a lot of different output for security headers, like strict transport security, X-Frame-Options, Content-Security-Policy, blah blah blah.
8:24So, this scan completed in just about 4 minutes. Granted, I sped up the video and cut up a lot of the waiting. About 24 different hits based off the different templates. But let's go take a look at what some of those Nuclei templates really are. Remember, Nuclei, the first time that we ran it, it automatically downloaded, saved, and
8:42stored these Nuclei templates in our home directory, and we could take a look what all this stuff is made up of. Just moved into that directory, we could LS and bebop around in the terminal, but honestly, it might just be easier to try and take a look at the whole thing in our text editor. So, I'm going to open
8:59up Insecure Blind Text. And look, realistically, this is a GitHub repository, so we could clone and download these really easily. But you can see how it's kind of differentiating a lot of the different kinds of endpoints, whether it's something that it's scanning in the cloud, like Azure, AWS, GCP, even code, like specific syntax or vulnerabilities, real CVEs
9:21that have been out and about in the wild. This is kind of awesome. Hey, here's a specific CVE for an authentication bypass in GitLab. Maybe some PortSwigger references, some blogs and articles that are out and about, and then how this all comes together. There are tons of others here. Here's some Erlang or SSH remote code execution. Here's Crush FTP. And this goes on and
9:43on and on. Whenever there's a new vulnerability that like falls out of the sky, I always hear, "Hey, there's some sweet new Nuclei templates that are out and about." Just really quick, like within a couple days or a week. There's some scanning that could be done even specific to files, like sensitive keys or information that might be included in
10:00any syntax or any other information that you could track down in some of the code, like the real software and syntax that's behind the applications you're interacting with. Look at all of these. Holy crap! Stripe API key? Yeah, that's got to be a finding. And they're using regular expressions here to extract this stuff out. Well, they even have a
10:20malware subdirectory. WHOA! >> [laughter] >> ANYWAY, I WANT US TO chat a little bit more about some of the HTTP stuff. Like when we're connecting to a web app, looking for vulnerabilities, looking for stuff that might be out and about. Can I find the robots.txt one, something super simple? Let me hit control G and try to go to robots.txt.yaml
10:43or YAML, the robots.txt endpoint, and take a look at how these things work. I know we're just looking at a simple example here, but I wanted to show you the syntax. We've got an ID defined that's like the name that was displayed in the nuclei output. An info section, a little bit more metadata, like the name of what you're tracking, the authors,
11:01the contributors, severity metadata, and maybe if you want to include the references or links in there to help others, like analysts or for folks working through this. But the HTTP section here is the most important tidbit, because this is where you define different requests. Like how is nuclei going to scan, interact, and look for vulnerabilities or whatever they might
11:22be based off of different methods. Like you can make a post request or put request or get request, simplest case here. And then the URL or the path that it goes to. They have sort of a template-looking like syntax for base URL, and then where you'll go. You can even define how many times you're willing to redirect, and then you define
11:43different conditions for matchers. And you define the matchers themselves or what is it that you're trying to match and find and validate really exists so that you see, okay, this is present. This finding really exists. I know this is simple robots.txt example, but we'll move on and elevate to write sort of even maybe a little bit of our own
12:04syntax or see this in action in just a sec. Here we're looking for the disallow word, just like we saw when we browsed that page. And the content type, like the MIME type or header here for robots.txt, that matches status 200 code. Of course, we want to see the page successfully returned to us. So, this all makes sense. It's pretty logical.
12:25And you just define it naturally in this YAML syntax based off of the requests and matchers that you define. And while there are almost 9,000 different templates that we could go explore and get some syntax ideas for, we could create our own, maybe learn a little bit of the YAML syntax, get a feel for the schema. But while they're explaining a
12:45little bit about the templates, they actually note you could use their totally free AI-powered nuclei templates editor that's all online at Project Discovery's website. So, for a little bit more of our demo, I know I've been pointing us at www. aquamarine, but there is another that I want to showcase, labs.aquamarine.
13:06We can see it's just another flat, silly, stupid engine X landing page from default install, but there might be some more secrets here. We can scan it with nuclei, but I do want to show us how we could just spit together our own template and get another feel for the syntax. Maybe we cheat a little bit, we
13:22use our AI crutch, but look, that's the way the world needs these days. And while I'm cheating, let me break the fourth wall. I know there is a hidden dangling .git folder there, a directory that's exposed for a GitHub repository or Git repository. Now, obviously this already exists as a nuclei template, but I want to walk through the steps so we
13:41learn. So, let me prompt our nuclei templates editor AI here, scan for a .git directory exposed on an HTTP or web endpoint. Let me click send, and look at this thing. Okay, it gives us a default like boilerplate one here. Let me zoom out a little bit so that's kind of easier to read. But then it start to
14:03generate it. Oh, it already did it. Exposed Git directory was the ID that it gave us, and maybe it's going to look for a .git directory. It uses one of these HTTP requests, just like we expect here, with the method get, path that's provided, and the matchers are expecting a 200 HTTP status code in response with
14:25a like directory indexing list. Let's try that, and we can just go back to our terminal. Let me actually make a git.yaml or whatever. We can paste all this in. I hit control V, hit control S, close it, and let's use nuclei now with the labs domain, and let's use that -t with our
14:46git.yaml file in the current directory. Found it, but didn't find the .git directory. Hmm. Well, let's do some learning together here, and again, I'm intentionally dragging us against the wall here, because I think it's important for us to go see, really, how does this make sense? We don't get a 200 status code response.
15:09So, that matcher isn't the right match for our nuclei template. 403 forbidden. We never get a 200. So, let's change this up. What else could be in like a .git folder? Normally, it's the caps. So, how could we change up our rule? How could we change up our
15:30template? Let me ask AI, change the approach here and look for a .git/config file. How does it work with just that as the input? Should change up the template that we're working with here, and ooh, yeah, okay. Just modified exposed Git config. It runs a get request to look
15:50for the base URL/ .git/config with a 200 response code on that, and if it has some words here that make sense, you could decide, you can determine are these other conditionals for our matchers, do they really make sense for what we're looking for? Are you willing to accept if we were to remove these potential false positives cuz maybe
16:12there's something else hosted at a .git/config, whatever the case may be. But let's try this. Back in our command line, let's modify our git.yaml, and then change this. Let's go rerun nuclei with that info, and it got it. It found it. Okay. You can see the hit for our exposed Git config. But let me tell you, we didn't have to
16:33do any of that because, as I mentioned, nuclei already has a template for this. Let me close this out, and then let me look in the folder here for Git, and there are a lot of these that could be any sort of understanding. GitLab, Git Tea, looking for certain things. Git logs exposure, though. Oh, actually finds kind of a similar approach that we
16:54had. Well, AI did, right? HTTP requests for base URL.git and logs head. And then it tries to track down some of the pertinent information including like some of the regular expressions that might be here. There are tons of these, so let's go see what our actual scanner, without using any of our own custom templates, but just the however many
17:15thousands already exist on labs.aquamarine and this endpoint. So, I'm not providing any other tags, I'm just running that. Let's see what it finds. Closer and down templates again. I do tend to see this take a little bit of time because it's going through a lot of different templates. You could use -v, turn on like verbose mode to see all the
17:35different kinds of traffic that it's sending there, all the scans that it's performing. But there we go. Oh, a couple hits again on the external service interaction just to get us started here. Will we find Git? It's worth noting, too, we could also limit the amount of templates that it's using or specify a certain amount or any kind
17:52of filters that we want. But oh, it tracked down the Git config here. And then a lot of the missing HTTP headers. But we got Git. There's the Git logs exposure just as well. So, since it did find a Git repository exposed on that website, could we use a tool to dump that entire Git repository? Could it
18:11loop through and find any of the objects? And there are a lot of resources for this, and I know I've covered these plenty before in other videos. But let's use pip to get Git Dumper. And then we could try and pull down the entire repository and see what secrets or sensitive info might be in there. Let me actually move into a
18:29temporary directory for us to work in. We'll just call it forgit. Forgot. Okay, that's fine. Typo, I don't care. Then let's actually specify a virtual environment for us to work in so we can cleanly install this pip library, and then I can pip install properly. Okay? Now, can I run Git Dumper? I can. Okay.
18:52We want to do that on our labs.aquamarine.ctfio.com, and we can just put it in an output directory. Ah, yep, I forgot the schema. You're right, that should include an https: // to start that. And will it track it down? Cool. Downloaded all here. Sweet. So, now inside of an output
19:14directory that was created, we can see what would have been here. There is a secret admin folder, and maybe if we wanted to go find stuff in this directory, we could. Or we could just throw it to nuclei. Because remember, there are tons of scanners, tons of templates here, even for like local files. And it might be able to track
19:34down any of the certain kinds of secrets that could be stored in a GitHub repository or in any sort of data that you're looking at. So, let's see. Is the syntax here nuclei with the target being like this folder? And I think I could specify -file to denote there will be files in there, I think.
19:54Yeah. Yeah, yeah, yeah. Okay. >> [laughter] >> It found a lot of stuff. It found a SendGrid API key. Let me zoom out. There are a lot of hits here. SendGrid API key, Slack API key. You can see the full value. Okay, maybe a little bit fake for that. Credentials disclosure, PHP stuff in action. Whoa. Maybe these aren't all
20:14that helpful, but it does point us at okay, what file and what actual location should I go explore, but the others it could just rip out the secrets and find the sensitive detail right away. Let's go look at the templates for those because I'm super curious what sort of regular expression are they using to track that down. Here's one and
20:36depending on all that muscles you got for with regular expressions, you can see the SG SendGrid beginning portion here and then a list of character options like all the alphanumerics, about 66 possible repetitions of that. And then maybe if it has newline characters or quotes, double quotes or single quotes behind it at the end of a line potentially. Dang, that's so cool.
20:58Again, obviously that's just one of many the nuclei could track down and all the different templates that it might be able to search for. One last firework for you, Ben or NamSec had showed me this sweet repository from Optiv with mobile nuclei templates. It includes some stuff for potentially sensitive things you might track down on Android or for other keys. Some of the things
21:20maybe there's some duplicative material than what is already in some of the nuclei templates that we have. But if you're interested, we could use some of these templates in the keys directory. They have a lot of other options. Amazon, Facebook, GCP, Google, LinkedIn, Mailchimp, SendGrid, blah blah blah. So I'm curious, will this track down
21:41anything else in our repository for that secret folder as opposed to the others? So we could pull this down and then try and use this with nuclei. Again, super easy. We could just pass it in as the templates to use. So nothing fancy. I'm just going to get clone this repository and then I can use nuclei like I have
22:02before specifying the files that we're going to be looking for, but let me actually specify -t for other templates and let's use the mobile nuclei templates and keys directory. Will that come through? Yeah, significantly less templates, right? Didn't have a lot of other noise, but reliably found the SendGrid API key and Slack API key here.
22:25And much faster, right? And this is only scratch the surface of all the stuff nuclei can do. So listen, I got to start using nuclei more often because my goodness, that's so stinking cool. I love the idea of just small, simple, easy, but smart YAML files like to configure these templates, right? That could scan and validate and do all this
22:47testing, hunt for these findings in an easy way. So you can automatically discover what you're up against and you don't need to go through all of that enumeration yourself. Hey, if you caught the bug just like I did, I know I for one I'm going to go through that new hacking hub nuclei masterclass and course because my goodness, this thing
23:03is just too cool. Link below in the video description, 50% off nuclei50, all the sweet Black Friday deal shenanigans. Thanks so much for watching this video. If you enjoyed this video, please do all those YouTube algorithm things like comment, subscribe and I'll see you in the next video.