Full transcript
0:09Hey everyone, thanks for joining us for
0:11the next session of our spotlight on
0:12GitHub advanced security series. My name
0:15is Anna. I'm your producer for this
0:17session, an event planner for Reactor
0:19joining you from Redmid, Washington.
0:22Before we start, I do have some quick
0:24housekeeping.
0:25Please take a moment to read our code of
0:27conduct. We seek to provide a respectful
0:30environment for both our audience and
0:31presenters.
0:33While we absolutely encourage engagement
0:35in the chat, we ask that you please be
0:37mindful of your commentary, remain
0:39professional and on topic.
0:41Keep an eye on that chat. We'll be
0:43dropping helpful links and checking for
0:45questions for our presenters to answer
0:46live.
0:48Our session is being recorded. It will
0:50be available to view on demand right
0:52here on the Reactor channel.
0:55With that, I'd love to turn it over to
0:56our presenters for today, Lupita and
0:59Matt. Thank you so much for joining.
1:02>> Thank you.
1:04>> Welcome back everyone to another series
1:07for Reactor. We're so excited to come
1:10together to show you more of what we
1:12have for application security
1:14specifically. Here at GitHub, we're all
1:16about collaboration, not compromise. And
1:18as we know, there's no silver bullet in
1:21application security. And so we're going
1:23to go ahead and start off with a piece
1:26of who I am, who Matt is, what this era
1:30of vibe coding looks like today, and how
1:33we're here to help you secure by
1:35default. So go ahead and have Matt's
1:37screen shared out, and we'll start from
1:39the top.
1:41>> Awesome. Okay,
1:44a little bit about who we are. Um, hi
1:47everybody. I'm Matt. I'm a solutions
1:48architect here at Endor Labs. uh
1:51formally before this I was in uh cloud
1:54security vulnerability management. I was
1:57a early SE at a uh another A application
2:02security scanning tool provider uh who
2:04shall remain nameless uh before that
2:07turned into the specialist for uh
2:09software composition analysis there was
2:12a former software engineer had my
2:14masters in computer science originally
2:15from Boston but I'm in Wayne New Jersey
2:18now with my two beautiful daughters and
2:20my beautiful wife
2:22um I love golf woodworking carpentry and
2:26held Divers 2. If anybody plays Hell
2:28Divers 2, you know, drop your gamer tag
2:30in the in the chat and we'll get on. Um,
2:34yeah, it's just a little bit about me.
2:35Uh, now for Lupita and I'm an enterprise
2:39application security executive,
2:41basically an overlay to all sales folks
2:43who specializes in GitHub advanced
2:45security. I was formerly at Veraricode
2:47for about seven years, IBM Appscan for a
2:50couple years. I was an early SC turn
2:52specialist. I actually started my career
2:54as a software engineer. I majored in
2:56electrical engineering and computer
2:58science. I'm from Portland, Oregon, if
3:00you can't tell by the photos here. I
3:02love the outdoors, photography, salsa
3:04dancing, camping, dirt bike riding. So,
3:08love the Pacific Northwest. Um, I see
3:10someone's from Mexico in the chat. Feel
3:12free to drop from wherever you're from.
3:15>> Awesome.
3:18So, what are we going to talk about
3:21today? um
3:24vibe coding a little bit. So, first off,
3:28things are changing. A lot of things are
3:30actually changing and it's like really
3:31how we build it, what we build, and who
3:34builds it. Okay,
3:36I don't think I have to tell anybody
3:38here that like how we are actually
3:40building stuff is
3:42significantly changing. What was it
3:44before, right? You get stuck on a
3:46problem, you think about what you have
3:47to do, right? you get stuck on a
3:48problem, you go to Stack Overflow, you
3:50kind of see what you can do, you bang
3:51your head against the wall, but
3:52eventually you figure it out. Now, it's
3:55kind of like, if anybody's seen the
3:56charts of Stack Overflow usage, it's
3:58gone down like that, but the usage for
4:01all this these co-pilots have
4:03skyrocketed. And tools like, you know,
4:06VS Code with GitHub Copilot built in
4:09have done a really good job of like
4:11bringing that to the developer right
4:13where they're integrated, right, where
4:14they're actually working. So that way
4:17it's like okay I can just rely on this
4:19kind of assistant to help me. So we say
4:22not an assistant a co-author. I think
4:24it's you know I think it's a little bit
4:27of both if you ask me. So it's you're
4:30doing a lot more prompting but you can
4:32we'll talk about kind of the the
4:34potential negatives about that as well,
4:36right? Um
4:40what we're actually building. So this
4:43notion of
4:45AI native application applications was
4:48not like a thing a few years ago right
4:50where it's like okay at first everybody
4:53was just building like chat bots and
4:55stuff like that but now and still today
4:58I mean all this stuff is so new my head
5:00spins every five minutes because it's
5:02like a new thing comes out right now
5:04people are starting to figure out okay
5:06well instead of just a chatbot I can use
5:09the this LLM to do this other thing I
5:12use agents. I can use a lot of
5:13automation. Obviously, I still think you
5:16need a human being somewhere along the
5:19line there. Like, and I will get this
5:22out of the way right now. I don't think
5:23AI is going to take any of our jobs
5:25anytime soon. Um, I took a few ML
5:28classes back in grad school and I'm
5:30telling you right now, it's it's not as
5:33scary as it seems. Okay. So, kind of
5:36what we're actually building is people
5:39are kind of viewing it as like AI first.
5:41Again, I think it's more AI assisted and
5:44there's a lot of like embedded
5:45intelligence. It's it's another tool in
5:47the box that you can use to build
5:49things, right? Um, but it's makes you
5:52more powerful. I will say it makes you
5:54more powerful of like how you're
5:55actually utilizing it, how you're
5:57actually building it and things like
5:58that. And then who builds it? Again,
6:02this whole intern who never sleeps
6:04thing, you know,
6:06it's helpful, but I don't think again
6:08it's going to take anybody's job anytime
6:10soon. So there's a lot of autonomous
6:12code writing. Obviously, you're like
6:14collaborating with a tool that can talk
6:16back to you rather than kind of looking
6:18it up and doing all the work yourself.
6:19That's how I kind of view it as. And
6:22we're going to talk about different like
6:24prompts and how to really how to really
6:26vibe code uh securely to make sure
6:29you're not just putting a bunch of AI
6:31swap into your code, right? Um
6:35what is vibe coding?
6:38I won't read this entire definition, but
6:40the the way I kind of view it view vibe
6:44coding is, you know, the one kind of
6:46sentence is it's essentially yoloing on
6:48steroids where it's like you're just
6:51kind of some people think like, oh, you
6:53do let the the LLM do most of the work.
6:56It's literally vi the kind of main
6:59definition that Andre Kaparthi wrote was
7:02essentially you literally just let the
7:06machine do everything and you just kind
7:07of let it let go of any kind of
7:09interference or anything like that and
7:12you're just okay I'm going to accept
7:14everything right now. I'm going to
7:16accept all the vibes if you will. And uh
7:19you can I'm sure everybody here has kind
7:22of seen that doesn't always work out so
7:25well. Um and we'll talk more about that,
7:27but just wanted to say, okay, what
7:30actually is vibe coding in case nobody
7:33has heard about it, but um this is a
7:36really good definition of it from the
7:37man himself who actually pioneered this
7:40kind of stuff.
7:43And what do we actually need to know
7:44about these code assistants? Right? So
7:48they're you can kind of see they're
7:49enabling programming for
7:50non-programmers. Like my wife who I'm
7:52telling she's a lawyer. She hates
7:54technology. She loves writing like
7:57little scripts to figure out. Okay. Um
8:00like she handles some of the budget at
8:02the household. So she's like oh I
8:04figured out a I asked Chachi BT to do a
8:06script to help us figure out how to how
8:09we can afford this new addition. Right.
8:10Like she actually like loves using it.
8:12So when my wife is like programming who
8:16she will admit to you she hates
8:18technology
8:20the opposites attract right um I think
8:23that's a that's a very cool thing in my
8:25opinion I think that's actually really
8:26nice right so it can accelerate
8:28productivity and however it can also
8:31accelerate security risks because my
8:33wife has no idea about the security
8:34implications of anything that she's
8:36writing nor does she want to honestly
8:40um they these LM lens the tools they'll
8:43transform ideas to code you know cursor
8:46GitHub copilot within VS Code all of
8:49that they can really kind of help you
8:53kind of bring your ideas to life and
8:54it's really good for like prototyping I
8:56love it for prototyping it's really nice
8:59but I kind of you got to kind of go in
9:00with the mentality of listen there's
9:02probably going to be stuff wrong here
9:04I'm gonna have to kind of help it along
9:05a little bit and it's trained on public
9:08GitHub repositories all these LLMs which
9:10is a very good thing because it's a lot
9:13of open source and there's a lot of
9:15really good open source out there but it
9:17could also be not so good because with
9:20SCA obviously you can see this is why
9:23CVEes are there are vulnerabilities in
9:25open source right
9:29I'm not going to get into this but it's
9:30programming versus software engineering
9:32I think vibe coding is more on the
9:34programming side of thing rather than
9:35the software engineering side of thing
9:37if you want more information on this it
9:39is actually a really good uh book uh
9:41from Google software engineering at
9:43Google. It just kind of really describes
9:46okay well what is the difference between
9:47programming software engineering um
9:51another just another consideration to
9:52take with vibe coding
9:56and uh I don't know if anybody saw this
9:59this was what March yeah back in March
10:02where you know you get kind of all these
10:05these folks who think oh I'm just gonna
10:08write an app and deploy it and I'm gonna
10:10get rich quick right and This person
10:14here went ahead and tried just vibe
10:16coding something together. Oh, my SAS
10:18startup, right? Turns out he got
10:20absolutely wrecked because people are
10:23going to, you know, they're creating
10:24random stuff. They're um maxing out API
10:28keys. They're they're bypassing the
10:31subscription. Like this is what happens
10:33when somebody who this person admits is
10:37not technical tries to just go ahead and
10:40vibe some vibe code something on their
10:42own. Um, and as you can kind of see for
10:45now, he's he's actually stopped because
10:49these are the implications and the
10:50ramifications that a lot of people don't
10:52understand or realize if they've never
10:54programmed before is that a lot more
10:56goes into it than just like some code in
10:58a couple files, right? So, you can't
11:00just vibe code your way out of
11:02everything.
11:05So, all this talk of like, oh, just
11:08have, you know, the AI deploy to prod
11:10kind of thing, right? just have AI do
11:12everything. It's not going to happen
11:14anytime soon. And this is kind of why
11:17because let's this is kind of the the um
11:21the anatomy of like an attack if you
11:23just let AI do everything, right? And
11:25it's kind of interesting because AI
11:27starts and AI finishes as well.
11:29>> So you have a dev, hey AI, I want you to
11:32write some method or function that goes
11:35ahead and uh goes to the database, pulls
11:37some data, puts it into a leaderboard or
11:40something like that. So the AI code does
11:41that but it doesn't give you any au
11:44authentication for the SQL database and
11:47it doesn't give you any rate limits and
11:49the SAS AI says oh yeah this looks good
11:52and the code review says oh yeah sure
11:55this looks good right and then it's
11:57shipped and then what what can happen is
11:59an attacker can use AI fuzzing to look
12:02for endpoints to then get to those
12:04endpoints to find out that there is no
12:06authentication there are no rate limits
12:08and then go ahead and gain access to
12:11Do I think it's getting better? Yes.
12:14However, there's a lot more when it
12:16comes to
12:19software than it is just regular old
12:21code that you kind of interface with.
12:24Like software is very complicated. It's
12:26very complex and everybody writes
12:28software so differently and every
12:30company does it so differently and
12:31everybody it's just very very unique I
12:34will say in the sense that like I've
12:37been doing SCA for six or seven years
12:39now. I've worked at thousands of
12:41companies. I've never ever ever seen two
12:43companies have the same tech stack and
12:45do things the same way
12:48because it's just it's people. Everybody
12:49does things differently, right? So
12:52can't just blindly trust
12:55AI.
12:58Um, here are some stats. So, 75% of
13:02developers are using AI co-pilots.
13:05Again, I'll give you a good analogy
13:07here. If I'm building an IKEA
13:12built-in, right, like the one behind me,
13:15I could use that little Allen wrench
13:17that they give you and spend hours and,
13:20you know, be really angry
13:23at how difficult and pain in the butt it
13:25is, right? Or I could use a drill and
13:30just go much faster, have a much
13:32smoother experience. That is what AI is.
13:34AI is the drill when it comes to
13:39programming or doing software
13:40engineering with AI. It is meant as
13:42another tool, but a lot of people want
13:44to use the drill rather than the Allen
13:46wrench, right? Of course, you would
13:47never want to use the Allen wrench. So,
13:4875% of developers are using AI
13:50co-pilots.
13:52This was a stat based off of the paper
13:54that's referenced below, uh, accelerate
13:57accelerate state of DevOps report from
14:00Morgan Stanley in collaboration with
14:02Microsoft. Um
14:0540% of the code is AI on GitHub is AI
14:08generated. Is that public repos? Is that
14:11private repos? I'm not too sure. I don't
14:14know if the paper actually says it or
14:16not, but that is the stat that is
14:18available.
14:20And 62% of AI generated code has issu
14:23issues. For me, it's like 80%.
14:26But um that is what folks have found on
14:29average here in this report. So there
14:32are going to be issues associated with
14:34it. Okay.
14:37So we at Endor when all this started to
14:40really come out did a little experiment
14:43and we said hey make you know an
14:45application to track a board game
14:47collection and let's just see what
14:49happens from like a security
14:50perspective. Okay.
14:53And right off the bat it added
14:58JavaScript, right? a lot of JavaScript
14:59in this. Um, use a Python back end, use
15:03a React front end. Off the bat, 1300
15:06dependencies were added. 65 of those
15:08were critical and 700 SAS issues. Took
15:1110 minutes, but then it didn't work. So,
15:13we started toying with it a little bit
15:14more, started playing with it. And when
15:16we specifically asked, hey, uh, I want
15:19you to reduce the number of dependencies
15:21in this, it actually added more and went
15:24all the way up to 1,600 dependencies
15:26that were added. it it can't really get
15:29that context of like okay if I add these
15:31dependencies maybe I'll hopefully I
15:33won't get as many but when it said okay
15:35I'll reduce the number ones replace you
15:38know the the functionality of this
15:40dependency with that dependency didn't
15:41kind of realize oh yeah that could
15:43actually add more dependencies with this
15:45other one because it brings in you know
15:47more transitive dependencies so we
15:49thought that that was pretty pretty
15:50interesting
15:52there so from this experiment what did
15:56we learn non-determinism is good is a
15:59good thing and a bad thing. And this is
16:02essentially what it boils down to is
16:04like you'll never kind of get the same
16:06answer the same exact way. Um so the
16:10same prompt can produce completely
16:12different results and that's really by
16:14design. They they don't want to produce
16:15deterministic output. Um because a lot
16:18of it really is like statistics and
16:20linear algebra for when it comes to all
16:22this LLM stuff. So it the the
16:25non-determinism can be good in the sense
16:28of you can generate a lot better ideas
16:31or a lot more ideas, look at things
16:32differently, but it can also be a bad
16:35thing because if you want to run the
16:36same exact prompt and expect the same
16:40answer, like that's not going to happen
16:41kind of thing.
16:44It's it's the whole free from the if
16:45then the non-determinism actually allows
16:47for that creativity like I mentioned. Um
16:51and then the model dependencies as well.
16:53So like one of the things I like to do
16:56is I like to use a couple different
16:58models while I'm programming and like
17:00have them act as certain roles if you
17:03will. Like for example, one could be
17:05like a security engineer, one could be a
17:07developer and it's like the developer
17:09model goes ahead and outputs something
17:11that I want to implement. I'll then pass
17:13it through the security model to be
17:14like, hey, what what best practices from
17:17a security standpoint am I missing here?
17:18What's the authentication look like? Are
17:20do I have that rate limiting? Things
17:22like that. and I'll kind of have them
17:23work together uh in that sense too. So
17:27we I think I've found anyway personally
17:30like OpenAI is very good at the security
17:34persona but Gemini is really good at
17:35like the developer persona. That's just
17:38something I found pretty pretty
17:39interesting.
17:42And you can kind of see here
17:45the same exact prompt, two different
17:48models, the same exact prompt, GPT4.1
17:51and then cloud 3.5 sonnet gave us
17:55completely different dependencies. So
17:56one gave us five extra dependency or
17:59three extra dependencies that probably
18:01didn't need and you can kind of see the
18:03result here within VS Code like all the
18:05different errors and all the different
18:06issues. Meanwhile, GBT4.1 just gave us
18:10two, which makes it more manageable.
18:14Why did it pick five instead of two? Why
18:16did Claude do that? Non-determinism. We
18:19don't know why it actually did that, but
18:21just pretty interesting. The same exact
18:23prompt, two completely different
18:24results. Some similarities, but even the
18:27version numbers are completely
18:28different. You can kind of see here
18:30Claude did 2.24.0 or 0.24.0
18:33and then this one did uh 0.29.0.
18:37Pretty interesting.
18:41The next thing that we learned, um,
18:43expect the unexpected, right? Small
18:45features can have a really big impact.
18:46You kind of can even see it in here,
18:48right? Even this little feature or this
18:50little app that just adds, it's just
18:51like a board game collector
18:54could add thousands of dependencies like
18:56what we saw.
18:58um the vulnerability multiplication.
19:01It's funny like depending on again the
19:03model and the prompt. This can either
19:06produce good results in the sense of oh
19:08it's not adding too many but maybe it's
19:10missing some functionality or really bad
19:13results in the sense of it's just adding
19:15anything it can think of. Oh, I might
19:17need this so I'm going to add that. Oh,
19:18I might need this. I'm going to add that
19:20and just add everything in there that it
19:23can.
19:24And what that h when that happens you're
19:28kind of your risk for vulnerability
19:30skyrockets because if you had a
19:32dependency that has 25 transitive
19:34dependencies you do that 25 times it's
19:37just exponential growth in the number of
19:39dependencies and the number of risks
19:40that can be introduced
19:42uh and risk beyond risks beyond CWES and
19:46CVEes.
19:48So there's there are things that models
19:52could introduce
19:54that a SAS SCA scan like the business
19:56logic stuff like won't pick up. It's
19:59almost like a security code review would
20:02pick up if you will.
20:04it can introduce risks like that that
20:06are just beyond like the kind of known
20:08the common weakness enumerations in the
20:10CVEEs that can say okay
20:14um maybe oh I wanted to add change the
20:18way I handle PII data or something like
20:20that right so it's not just the CV CWES
20:24and CVES that you kind of have to worry
20:25about a little bit with these models but
20:26it's also like the business logic and
20:28the flows that it could also change or
20:31add in that sense
20:36Secure prompts. This is how
20:39I like to view prompting. It's like a
20:42like a PRD or like a design document. If
20:45you don't start with these secure
20:47prompts, then what'll happen is down the
20:51line, obviously, it's not going to take
20:53that context into consideration and it's
20:56going to introduce potential risk. So I
20:58like to treat my prompts as like design
21:00documents or PRDs or even like an RFC if
21:04you will. So then that way it kind of
21:06has that context. All these models now
21:08have like memories introduced in them.
21:10It can have that memory. We we'll talk
21:12about rules, things like that that you
21:14can implement to really take advantage
21:16of this. But if you start with secure
21:19prompts just like you would with, you
21:21know, programming, right? You want to
21:23start with secure design that will lead
21:26to more secure code kind of down the
21:28line there. And this is a good example
21:29of that where a write a Python script
21:32that takes a host name as input and
21:34pings it four times. Very very simple.
21:37Where if you give it more of a security
21:39conscious awareness we can say you know
21:42hey write a p a secure Python script
21:44that takes a host name validate the
21:46input to avoid security issues. You can
21:48see it actually gives that.
21:51So secure pumps are a good way to view
21:54or good are good to view as like a
21:56secure by design if you will
22:00down the line there.
22:02We're going to talk about rules in a
22:04second and kind of like test-driven
22:05development. What's old is new again.
22:07Test- driven development was uh was kind
22:10of gone now it's back. It's it's pretty
22:12interesting to kind of see that uh
22:14ignore rules as well. So these kind of
22:18rules and these templates that you can
22:19actually add for these LLMs to refer to
22:23within projects or within organizations,
22:25whatever have you, these are actually
22:27super duper important. We're going to
22:28dive a little deeper into that in a
22:30second. But like the rule files can help
22:33guide the LLM. And again, that's like a
22:36a a design document that the LLM can
22:39kind of refer to when they're going to
22:41do that. And like this is an example of
22:43one where hey in our organization this
22:46is how we handle user inputs andization.
22:48So like anytime a developer is saying
22:51okay I want you to add a method or
22:52function that does this the AI will say
22:55okay sure I want to go refer to the rule
22:58that you or your organization has set
23:00for me to make sure I'm adhering to this
23:04proper way of handling input
23:06sanitization. And we'll cover that in a
23:08second.
23:10This is probably my favorite one because
23:13it's very interesting. Um,
23:16these models by themselves,
23:20they're all trained on I don't want to
23:23say old data, but like it's not data
23:26that was updated earlier this morning.
23:29You know what I mean? So like these
23:30these models, they're trained up to a
23:33certain date. CVEEs don't stop because a
23:36model has, you know, stopped training at
23:39a certain date, right? CVS are always
23:41always
23:43going to be coming out. It's just a fact
23:45of life, death taxes, and there's always
23:47going to be vulnerabilities.
23:50MCP servers fix this because it will
23:53actually get context, and we'll show
23:55what an MCP server looks like here in a
23:56second. it'll get context from whatever
24:00provider
24:02in order to give good data about what's
24:06potentially in the past right so
24:10this example here right so cursor
24:13co-pilot whichever one GPT right GPT
24:17here gave these two dependencies well
24:20the data cutoff was June of 2024
24:24so these were the latest versions that
24:26it just picked
24:27was from April and March of 2024.
24:30I don't even know what version UV UVORN
24:34is up to now, but I'm sure new CVs have
24:37been released for these and it's what a
24:39year and a half later. I'll guarantee
24:41you there are CVS that have been
24:43introduced in later versions, later
24:45versions of these. And this is just two
24:48dependencies, right? So
24:52the model kind of cut off date or the
24:54data cut off date, you can't just
24:57inherently trust the LLMs because they
24:59don't have the most relevant data. So
25:01that's like super important for CVS.
25:05Super important.
25:07Um, if folks don't know what an MCP
25:11server is, I'll just quickly touch on
25:12that.
25:14MCP server model context protocol. The
25:16way I like to look at it, think about it
25:18as an API for your LLM. I think that's
25:21like the simplest way to look at it
25:23where the MCP server within uh VS Code,
25:28right? You have Copilot within VS Code.
25:31The MCP server is going to go ahead and
25:34actually get data from whatever tool you
25:36you call out. So in this case, I'll show
25:39in a second, Endor Labs embedded MCP
25:42server with GitHub Copilot within VS
25:45Code. So when we go ahead and like add a
25:48method or something like that, it's
25:50going to go and grab the relevant
25:51context from Endor Labs to say, "Okay,
25:54well actually uh GPT4.1 I added this
25:57dependency. Oops, it has a vulnerability
26:00in it." But Endor is telling me, "Oh, I
26:02can just upgrade to this version. It
26:03won't break my code and I'll be good to
26:04go kind of thing." So, an MCP server, I
26:07think, is a good way to think about it
26:10as like an an API for your for your LLM
26:13or for your co-pilot, if you will. And
26:16we'll cover rules here in a second. I'll
26:18show what that kind of looks like. But
26:21these are really like, hey, how can we
26:23actually securely adopt AI code
26:25assistance? Honestly, the big three are
26:28secure prompts, security standards with
26:30rules, and then add some kind of MCP
26:33server to help along with that. And I'm
26:35going to show um VS Code here to kind of
26:40show what that looks like.
26:43So, by the way, this is I'm going to
26:44open source this afterwards if anybody
26:46wants to play around with it or anything
26:48like that. But you give it a prompt. I'm
26:52giving Copilot here. GPT5 Mini. I mean
26:55VSGO with Copilot. I'm giving a prompt.
26:56Hey, create a to-do list app with a
26:58React front end Python backend. Similar
27:00to what we actually looked at. You can
27:02see it's going to kind of go through its
27:03paces here. Okay. I'm going to make a
27:05requirements.ext a Python file here. But
27:09now I'm going to go ahead and check for
27:13vulnerabilities with Endor Labs. And
27:15that's because what I did was in
27:20there you go copilot instructions. In
27:23the co-pilot instructions,
27:26I said, "Hey, ensure newly added
27:28dependencies or check for
27:29vulnerabilities and projects always
27:32stays clean. I want you to check for
27:34these files and then run the MCP server
27:36andor CLI tools. Run the check
27:38dependency vulnerabilities tool. I want
27:40you to look at the language dependency
27:42name version. I want you to do all this
27:43stuff, right?
27:45And then you can actually give it
27:50secure specific to SCA. Hey, I want you
27:53to look at these things in this way,
27:55right?
27:57And the reason why I think these are
27:58super powerful and super key is because
28:01like I said earlier, everybody is so
28:04different and no two people write code
28:06the same way. Okay? And no two
28:09organizations have the same kind of
28:11policies or the same kind of workflows
28:13or anything like that. So this allows
28:16for people to say, "Hey, I know in like
28:19the general sense, this is how you would
28:21go about testing stuff." But we use
28:23Gradle version 4.5 for some older stuff.
28:27And the way we use our wrapper around it
28:29is with this and that. I've seen I've
28:32seen too much of this kind of stuff. And
28:36you know, somebody decides to build
28:37something a certain way 10 years ago and
28:39it's still there. Everybody does things
28:41so differently. So what we want to be
28:44able to do is to have the flexibility to
28:47give to the LLM and the context to give
28:49to the LLM. Okay, I want you to test
28:51this way or look at it that way. Da da,
28:53right? So
28:56you know, Copilot does its thing here.
28:58The LLM is checking for is writing some
29:00code and then it's going to call the MCP
29:04server
29:06andor MCP server to say, "Okay, I want
29:07you to check this version of this
29:09dependency that I had. Oops, it does
29:10have a vulnerability in it. What about
29:13this one? Uh, nope. This one doesn't
29:15have a vulnerability in it. I'm good
29:16with that one. And then kind of keep
29:18going. React. There's one with like a
29:20ton of vulnerabilities in it that I want
29:21to find. Uh, of course it'll be probably
29:24Yeah, this one. Like this version of
29:26Vite fits, however you say it. Uh, npm
29:29version 5.1. This has 12 vulnerabilities
29:32in it. So, right off the bat, and by the
29:34way, I haven't scanned anything. I
29:36haven't done anything. Endor is giving
29:37me all this context because of our
29:39database.
29:41I'm just writing my code, right? So now
29:47what's happening is the LLM is getting
29:49the context from Endor saying, "Hey, in
29:52this npm package or in this Pi Pi
29:54package, there's a ton of
29:55vulnerabilities in it. Uh, this one has
29:58one vulnerability has multiple. Next,
29:59I'll fetch the details for these two.
30:02I'll go ahead find out the information."
30:05And again, this is all from Endor. It's
30:07just getting the context. It's doing its
30:09thing. Okay, I found the issues. Now,
30:12what I want to do is I actually want to
30:14change the versions of it. I want to get
30:16the context. Is it going to be a
30:17breaking change? All that stuff. And I
30:20want to change the versions to say,
30:21okay, instead do those versions. No
30:25vulnerabilities in here and no
30:27vulnerabilities in here. We get some in
30:29here, but it'll keep going. There we go.
30:33Get the details. Have it do its thing.
30:37It'll say actually let me do 5.5 and
30:40then it says zero. So you can even see
30:43it went and it said oh okay let me go to
30:46this version said try that nope that has
30:49more vulnerabilities in it. Let me go to
30:51a different version. Oh this version
30:53doesn't have vulnerabilities in it.
30:56And it kind of keeps going and going and
30:58now the ben the kind of huge benefit of
31:01this is I haven't pushed any code yet. I
31:04haven't done anything other than just
31:06write some code and copilot's taken care
31:08of it for me. Anyway,
31:12I've gone ahead and I blabbed a lot. Um,
31:16I haven't been able to see the chat, so
31:18I'll take a look at the chat while
31:19Lupita presents here now. Um, but any
31:22questions from anybody?
31:27>> Yeah, I think we have a couple of
31:28questions from you. We can go for the
31:30top.
31:31I think the first one that came in was
31:34regarding
31:36if the models get better, will the code
31:38get better?
31:40>> You know, I'm going to answer that
31:41question with another question.
31:47If the models get better, will the code
31:49get better? I mean, it could, right? But
31:51at some point,
31:53things plateau with these LLMs. And
31:58again, it's getting trained on all
32:00open-source data. So the the way that
32:03the code gets better is that, you know,
32:06the training data that it gets has no
32:08vulnerable code in it, which will never
32:09happen. Um, people ask also all the
32:13time, well, how do I secure the code
32:15that my AI code generator writes? How do
32:19you secure your code now that you write?
32:22Same way. It's all code. Um,
32:26it's a good question. I don't know the
32:28answer to that question. I'm not sure. I
32:30mean, there'll be improvements, of
32:32course, but the with LLMs and with
32:35machine learning and with all this kind
32:36of stuff. It's only as good as the data
32:39that it's trained on. And if it's
32:40getting trained on all the open source
32:42data, there's always going to be
32:43vulnerabilities on in it. You know,
32:46maybe in 10 years or whatever, we'll
32:49have no insecure code and we can all
32:51retire. But uh until then there's always
32:54be insecure code and it'll just kind of
32:57keep kind of going around and around. So
32:58it's like okay now all this AI code is
33:01being generated based off of open source
33:04code that had vulnerabilities in it.
33:05Well now that will be used to train the
33:07models. And if you really think about
33:09it, all this kind of AI slop that's
33:11being used to train the models, is that
33:13going to turn into even worse models? So
33:15it could go the other way around. Um
33:18but I don't know if that answers your
33:20question. It probably doesn't. That's
33:22but the truth of it is is it's a really
33:25hard question to answer.
33:27I suspect they will get a little bit
33:29better, but I don't think they'll be
33:31perfect.
33:33>> And then are security rules just a guard
33:35rail.
33:43>> They are one of many, right? You can
33:46never just have one
33:48place to have a guardrail in the world
33:50of security. Like if you kind of think
33:52about a secure SDLC, right? Yeah. You
33:55know, I'm in the world of more at the
33:57code level with the application security
33:59code and hey, I want to go ahead and
34:01build and I want to break a build,
34:03right? That's one guardrail. Oh, log 4j
34:06break the build, right? That's one
34:07guardrail. I then have a W. That's
34:10another guardrail. I then have this.
34:12This is another guardrail.
34:14So, it is one of many guardrails, I'll
34:17say. But the point of this guard rail is
34:20that you catch it literally
34:22if I'm thinking in my head of a secure
34:24SDLC like the second earliest portion
34:27the first earliest is like design or
34:30even like the person's brain right as
34:32they're writing the code or thinking
34:34about the code or designing the project.
34:37The earlier you catch these things the
34:39better. So if I can put another
34:40guardrail there that does not impede
34:42with developer productivity at all but
34:44does nothing but like help them write
34:46more secure code, I will happily put up
34:48that guardrail as long as it because I
34:51was that developer that got a thousand
34:52Jira tickets and never fixed anything
34:55security-wise because it would just get
34:57in my way. As long as it just doesn't
34:59get in my way and it fixes it for me and
35:02I don't have to really think about that
35:04security stuff,
35:06that's a great guard rail to have. And
35:08again, it's just one of many guardrails.
35:10I hope that answers that answers your
35:12question.
35:13>> Yeah. And I think that answers the next
35:15question. Is the instructions or rules
35:17the only way or are there more ways? So,
35:19it sounds like there's more ways.
35:22>> Yes, there's there's a lot more ways.
35:24The these instructions and these rules
35:27like these here, this is the way to do
35:29it for when the specific use case I'm a
35:33developer. I'm in the IDE. I'm utilizing
35:35co-pilot to help me write code. I want
35:39to add a new method. Okay, help me write
35:41a method that does this. Every single
35:43time it does that, you set up a rule to
35:45say, by the way, check for dependencies.
35:47This is in this specific use case. This
35:49is the way to do it. You then have
35:51several other areas within the secure
35:53SDLC, the software development life
35:55cycle that you can implement these kinds
35:57of things.
35:59>> Awesome. Um, and then the last question,
36:01how does Endoor Labs integrate with the
36:03GitHub platform in practice? Does it
36:05work through GitHub actions or APIs?
36:08>> Both. We have a GitHub app and we
36:11integrate directly with GitHub advanced
36:12security. So, if you're a GitHub
36:14advanced security customer and um, you
36:17know, you want to have a little bit more
36:19of a robust SCA tool, we will integrate
36:23right in there. We'll implement our
36:24reachability analysis and it will all
36:27show up in the GitHub advanced security
36:28dashboard. So you can have GitHub
36:31advanced security in GitHub with GitHub
36:34CodeQL being results being put in there,
36:36but also Endor Labs SCA results being
36:39put in there too with our reachability
36:41and all that good stuff. But yes,
36:44>> so I think that brings us to a great
36:46transition to the GitHub portion of this
36:48session. So we can go ahead and share
36:50out my screen. So we talked a little bit
36:53about the vibe coding area and what it
36:56entails and we saw that guy post about
36:58you know keys being taken and used in
37:01the wrong way and obviously dependencies
37:03being introduced exponentially
37:04increasing the risk and so from a
37:07platform perspective as Matt mentioned
37:10uh we're a better together right best
37:12and breed bringing all of the different
37:13solutions at play so that you have a
37:16single pane of glass and so I'm just
37:17going to quickly touch on some of the
37:19GitHub portions of the application
37:21security program that you could
37:23implement should you have both
37:24solutions. Um, but really this session
37:26was was more focused on just like the
37:29state of the state, what it looks like,
37:30and then how you can start thinking
37:32about securing your code by default. Um,
37:35next week's session will actually be an
37:37update on all things gas, but this will
37:40at least give you a 10,000 foot view of
37:42what we include within our solution. So,
37:46we're going to go ahead and start with
37:48secret protection. Um, it used to be
37:51called secret scanning, now we call it
37:52protection, but the sense of being
37:54preventative, right? We want to prevent
37:57exposure, catch secrets before they're
37:59even committed. We want to reduce that
38:01attack surface, keep those developers
38:03moving. As you heard Matt say, he
38:05doesn't stop to fix any security
38:07findings. He is not alone. That is very
38:09common. And so, the ultimate key thing
38:12here is just that, you know, brand
38:13reputation, protecting it from any
38:15potential incident or breach. So with
38:18that, I think it's it's super important
38:20to just kind of understand like what
38:22exactly does our secret protection
38:24cover. Uh we do the full get history
38:26scanning, collaboration spaces,
38:28documentation, public sharing. We're
38:30looking for generic secrets, provider
38:32specific patterns, and custom patterns.
38:34So great breath and depth of what all
38:37could be potentially exposed.
38:39I always say like it's important to
38:42focus on the ones that are most like
38:43important. maybe they're valid, they're
38:46currently, you know, active. So, when we
38:49do detection, we want to provide the the
38:51lowest amount of noise so that when a
38:53developer does get a a ticket item that
38:55they can take action on it. Uh we
38:57partner with over 150 service providers
39:00to do that validation. And then you can
39:02also use co-pilot power detection for
39:04generic passwords so that you can create
39:06your own.
39:08Validity checks, like I mentioned, are
39:10are the key sort of component to
39:12prioritization. It helps you focus on
39:14the ones that are active today so you
39:16can take action on those. Sometimes
39:18developers will have a hard time like
39:19picking where should I start. This kind
39:21of gives them a a a great starting point
39:24and then if there's any potential reason
39:26why they don't want to fix that finding
39:28there is alert dismissal with approval
39:30process in place so you have that
39:32history that audit trail so nothing gets
39:34swept under the rug.
39:37Right. The pro the secret protection
39:39promise here is prevention first. So, as
39:42Matt mentioned, shift left as much as
39:43you can with your dependencies, with
39:45your secrets. Give the developers the
39:48resources they need to succeed. That
39:49way, we streamline remediation and
39:51enhance developer productivity. Your
39:54brand remains protected,
39:56right? For us, if you've never used
39:58secret protection for any GitHub users,
40:01we do have a free assessment. You click
40:03the button and it shows you at a high
40:05level what secrets lie within your
40:07organization. So, this is available
40:09today free of charge. You can go in
40:11there, just click that button, and it'll
40:13give you a highle report. It won't show
40:14you the details until you actually turn
40:16on the product, but at least it gives
40:18you an idea and it can help drive
40:21urgency within the organization if
40:22someone is trying to get convinced of
40:25the need of of this product,
40:28right? And then code review. As we see
40:31more AI generated code in conjunction
40:34with our developers, I always say the
40:36industry doesn't have a detection
40:38problem. It has a remediation problem.
40:41So everything Matt showed you was prior
40:43to even scanning. This was like the most
40:45preventative protection that you can put
40:48in place. But then once it gets into the
40:50flow of events, you want to make sure
40:53that regardless of the, you know,
40:55scanners that you have in place and
40:57everything that's being analyzed that
40:59there's a plan of action for the
41:01remediation. And with that comes our
41:04co-pilot. So within code scanning, we're
41:06going to be looking at your first party
41:08code or code generated by AI. For those
41:10of you not familiar, this is powered by
41:12what is known as code QL. It basically
41:15does a semantic code analysis that
41:17treats code as data. So it's it makes it
41:19querable. Um we have over a 100 native
41:22security rules that were created by us
41:24and we publicly display them. So they're
41:26transparent. You create your own as
41:28well. I come from different companies
41:31where they didn't display that and it
41:33was more so of just like accept whatever
41:35was analyzed without really knowing or
41:37being able to look under the hood to see
41:38what was actually being detected. So
41:41here we we like to provide that
41:43flexibility and transparency so that you
41:45have that deep contextual understanding.
41:47That way we reduce false positives. We
41:49want to make sure that we're not just
41:51finding flaws though that we're actually
41:53fixing them. So what that means is now
41:56with C-Pilot autofix which is separate
41:59from the copilot you saw earlier on
42:01that's helping you generate the code.
42:03Copilot is just powering our GitHub
42:05advanced solution to allow you to
42:08quickly fix findings directly in the PR
42:11and within historical alerts. So what
42:14we're seeing is that 45% of suggestions
42:17that are being created by the co-pilot
42:19autofix are being accepted automatically
42:21within the PRs which is reducing
42:23meanantime to remediation. That's a
42:26metric most companies use to assess how
42:29you know well a solution is doing. And I
42:32always say, you know, depending on like
42:34where you are today and kind of where
42:36you want to be, um, I think if if you
42:39look at like the types of flaws that are
42:41being introduced, for example, cross-ite
42:43scripting is one of the most common
42:45ones. 80%
42:48we're seeing an 80% reduction with SQL
42:51injection is about 90% reduction in the
42:53medium time to remediation. So that's
42:56pretty awesome. Here's a little snapshot
43:00of what autofix looks like. So within
43:02the pull request, it'll show you what
43:04code it changed and then you can either
43:06accept it, edit it, or just, you know,
43:09add your own or or or skip out of it. It
43:11totally depending on kind of your flow
43:13and what you think is right. So you have
43:15full control. We still need that human
43:17as Matt said.
43:20And then there's another component to
43:22our solution which is security campaigns
43:24to help you burn down security debt. So
43:28once you kind of think about what your
43:31goals are as a business, let's say you
43:33want to meet OS top 10, SAS top 25, PCI
43:36compliance, it it's really important to
43:39have that bridge between developers and
43:41security folks to align on what the
43:44outcome looks like. And with campaigns,
43:46we made it easy to collaborate with both
43:49teams so that the security team can say,
43:52"Hey, by the end of the month, we want
43:53to have all applications adhere to OAS
43:56top 10." We're seeing that developers
43:58are two times more engaged when they
44:00have direction on what they need to
44:02focus in on first. And we're seeing that
44:0455% of alerts are actually being
44:06remediated compared to just like 10%
44:09when there is no direction. And so with
44:12that, we always say secure. Here's a
44:15snapshot of it. We always say that
44:16security campaigns and autofix go better
44:19together. So there's less friction
44:20between both parties and it dramatically
44:24reduces your security at scale. So that
44:27increases your your software security
44:29posture by I think it was like 5x.
44:34So in essence um as Matt mentioned we
44:38become the kind of command center where
44:41you can integrate your findings from
44:44Endor from whatever third party tool
44:46you'd like. So you have a single pane of
44:49glass of your first party code findings,
44:52your dependencies, your secrets all
44:54within the landscape of your platform so
44:58that you can drill in to the metrics,
45:00see where you are, track deployment
45:02status and progress and prioritize the
45:04vulnerabilities that need your attention
45:06first.
45:09So I know that was a highle overview of
45:11GitHub advanced security. It was more so
45:13just a touch on what could be used after
45:15the fact uh when vibe coders go wild and
45:18start to implement applications that may
45:21not scale. So having all of these pieces
45:23to the puzzle will essentially help
45:25developers build the most security
45:28strong applications for the future.
45:32I'll stop right there. See if there's
45:34any questions in the chat.
45:39Otherwise, that wraps up our session for
45:41today.
45:52Yeah, feel free to reach out. Like, a
45:55lot of this stuff is so new. I don't
45:58know about you, Lupita, but like
46:00sometimes it gives me a headache. And I
46:01live in Jersey, which apparently is in
46:04data center valley, and my electric bill
46:06has gone through the roof. So, I'm
46:08trying to figure out ways to not use AI,
46:12but like if this stuff is is so it's
46:17changing so fast and it can get really
46:20complicated. Um, so if anybody has
46:23questions, please do feel free to reach
46:25out. Um, we're we also have a secure
46:28prompt kind of like library that if
46:30anybody wants, I'm happy to share. Um,
46:33lot of resources that both GitHub and
46:36ourselves have to to help anybody out.
46:40>> Absolutely. And Matt, I just wish that
46:4210 years ago I had a co-pilot to vibe
46:45code. Maybe I would have stayed a coder.
46:47>> Actually, you know what? I I I have the
46:50opposite opinion because I'll use it too
46:53much, then I forget too much. Like, how
46:55the hell do I do this again?
46:57So, I'm glad I like in grad school, our
47:00professor made us use Notepad as our
47:02IDE.
47:03>> Yeah.
47:03>> And I will never forget a semicolon for
47:06the rest of my life. So, it's um
47:09>> it's uh the repo for the first demo.
47:11I'll share that out.
47:13>> Um for sure. And uh it's it's just so
47:17much it's so much information. So,
47:21>> absolutely.
47:24And I know um both companies are always
47:26recruiting. So if you go to the careers
47:28page and find something interesting,
47:29feel free to reach out if you have any
47:31questions. Hopefully that was helpful
47:33for everyone. Like I said, next week's
47:35session or in two weeks, I apologize, we
47:37will go through a gas overview in
47:40detail. So if you'd like to join us
47:42then, feel free. And as always, we thank
47:45you for your time, your comments, and
47:47look forward to seeing you on the next
47:49sessions.
47:50>> Good to see you all joining us. Awesome.
48:02Thank you all for joining and thank you
48:04to our speakers. This session is a part
48:06of a series. To register for future
48:09shows and watch past episodes on demand,
48:12you can follow the link on the screen or
48:14in the chat.
48:15We're always looking to improve our
48:17sessions and your experience. If you
48:19have any feedback for us, we would love
48:21to hear what you have to say. You can
48:23find that link on the screen or in the
48:25chat.
48:27We'll see you at the next one.
48:29[Music]
48:47[Music]
49:10Hey hey hey.
49:12[Music]