Full transcript
0:12Hey, welcome everyone to our webinar.
0:14I'm just watching the participant count
0:17um creep up, so I'll give it a few more
0:18seconds um to let people join.
0:45All right, I think we'll get going.
0:48We've got a good number here already. Um
0:50so, welcome everyone to the webinar. My
0:53name is Katie Coleman. I'm the managing
0:54director for Equal Experts North
0:57America. Today, we're talking about AI
0:59in the SDLC from experimentation to
1:02real-life delivery impact. Um so, thank
1:05you for taking the time to be here. Just
1:07as a quick introduction, one of the
1:09consistent challenges we're seeing at
1:11the moment is how do you scale AI in
1:14software delivery. So, this is really
1:15about moving beyond individual engineers
1:19or product managers using an AI tool
1:21with some good prompting to moving into
1:24more a repeatable governed embedded
1:27practice across a single team and
1:29multiple teams.
1:31So, my colleagues Andy and Mike are
1:32going to talk you through a practical
1:34approach today, including a demo. We'll
1:36be using uh Claude Code for that.
1:39Um but, quickly before I hand over to
1:42them, I just wanted to give you some
1:43context about who we are, Equal Experts.
1:46We're a global um tech consulting um
1:49company. We focus on helping large
1:52organizations deliver better software
1:55faster with less risk. And really what
1:57makes us different is our model. We have
2:00a senior only team, so about 90% of our
2:03people have 10 years of hand hands-on
2:06experience or more. In fact, there's
2:09lots have 20 plus years. So, we don't
2:11have juniors, we don't have a pyramid
2:13structure that a lot of our competitors
2:14have, and we really work directly with
2:17our customers' teams, working side by
2:19side, embedded. So, that means we're
2:21much closer to the outcomes rather than
2:24providing advice from a distance.
2:27We're We're employee owned, which we're
2:29very proud to share. So, we don't have
2:31some of the same external investor
2:32pressure that other companies have. Um
2:35so, it means we don't overemphasize on
2:38growing from a financial perspective,
2:40but we start small with our customers,
2:42build trust, and go from there.
2:45Um so, today
2:47we're not trying to sell you a product,
2:49but this is really about how do we
2:51um
2:53how AI changes the way we build software
2:56itself. So, that's across engineering,
2:58product, testing, documentation, and so
3:00on. So, we're really seeing firsthand
3:02with a lot of our customers that they're
3:05even if they're seeing some results,
3:06it's kind of patchy, and they really
3:08want to see some more consistency across
3:10that.
3:11So, on that note, Andy and Mike will be
3:14sharing everything with us today.
3:17We'll run for about 30 minutes, and then
3:19we'll have 15 minutes of Q&A. So, please
3:21use the Q&A function. You can drop
3:23questions in there at any time, and you
3:25can upvote other people's questions. So,
3:27go through that, keep an eye on it, and
3:29then we'll revert into that after about
3:3230 minutes. Okay? Over to you, Andy and
3:35Mike.
3:38Thank you very much, Katie.
3:40Hi, everyone. Thank you so much for
3:42being here with us today.
3:44Um my colleague Andy and I are going to
3:46talk about a really popular topic that
3:48Katie has already mentioned, using AI to
3:51develop software.
3:53Everybody's talking about this, but we
3:55have our own perspective we'd like to
3:57share on that, and specifically we want
3:59to talk about the challenges that
4:00enterprise companies have in adopting AI
4:03for software delivery, and it introduce
4:06some keys for moving beyond those
4:07challenges.
4:10So, first let's just just a quick intro.
4:12I'm Mike Mitchell. I'm product and
4:14strategy principal for Equal Experts
4:16here in North America. I've spent 30
4:18years in enterprise software as an
4:20engineer and a product management leader
4:23uh building and leading world-class
4:24product teams.
4:26And hey everyone, I'm Andy Vase,
4:28technical principal here at Equal
4:29Experts. I've spent my career leading
4:31teams and working hands-on with
4:32enterprise platforms and engineering
4:34teams. Uh really excited to be here
4:36today and share how AI has transformed
4:38how we work and how we've helped our
4:39clients transform how they work.
4:43Thanks, Andy. All right, so
4:46let's just dive into the problem.
4:48The problem we want to talk about is
4:50that enterprise companies are not
4:51getting the results they hoped for from
4:54AI and software delivery today, and
4:56there've been industry reports about
4:58this since last summer. You may remember
4:59the uh often-quoted MIT report saying
5:0395% of AI projects aren't really
5:05producing any return.
5:07Um and there've been
5:08more reports, even though that has been
5:10debunked a little bit, there's been a
5:12lot more reports about the same problem
5:14since the first of the year confirming
5:15that this problem is really still
5:16happening, and it's a little like
5:19setting out to bake this fancy Gremlin
5:20cake and uh ending up with whatever this
5:24is.
5:25Companies are just not getting the full
5:26effect of using AI in their software
5:29development life cycle.
5:31So, we've been exploring this problem a
5:33little bit more deeply with companies
5:35that we know, and we've seen that they
5:37tend to fall into one of these four
5:39stages of maturity.
5:41There's unaugmented, which is where we
5:43all started
5:44uh and where we were several years ago
5:46now, before AI, just using standard
5:49human processes developing software.
5:52Initial steps towards adopting AI tend
5:54to start with giving individual team
5:56members subscriptions to an AI tool. For
5:58example, Copilot is a common example,
6:01and letting them build in their own way.
6:04The phase beyond that is
6:06cross-functional teams using AI together
6:08with intentionally designed structure
6:10and workflows and some shared
6:13methodology. And then of course, the
6:15vision that we all have is the agentic
6:16future where AI anticipates and delivers
6:20on our needs with minimal direction, um
6:22it still produces high-quality code,
6:26but very few organizations are close to
6:28that today, really.
6:30What we hear from most organizations is
6:32that they're currently stuck somewhere
6:34between individual and team
6:36augmentation, and they're struggling to
6:38move beyond that. And there can be many
6:40reasons for that beyond simply not
6:42knowing what to do technically to to
6:44achieve that, from politics to cost to
6:47security concerns. And we don't intend
6:49to minimize the difficulty of solving
6:50for those, but if you can just take that
6:53next step towards maturity with team
6:56adoption of AI, you can really start
6:58seeing the real benefit of AI at scale.
7:02So, I'm going to turn over to Andy now
7:03to introduce some key ingredients to
7:05start making that move.
7:09Thanks, Mike.
7:11And so, after working with AI across
7:12clients, we we've really seen a
7:13consistent pattern. And the gap between
7:16expectation and impact isn't really
7:18about the language model. It's a lot
7:19more about the operating model and how
7:21you ultimately put this new delivery
7:23stream into practice.
7:24And so, if you want AI to accelerate
7:26software delivery, three foundational
7:28elements have to be in place. First,
7:30shared context or knowledge. Um AI can't
7:33operate in a vacuum. It really needs
7:35explicit version version knowledge about
7:37your architecture, standards, and domain
7:39to get the optimal result.
7:41Second is governance and rules. Clear
7:44guardrails for where AI can act, how it
7:46should act, and the constraints it must
7:48follow when it produces output. And
7:50third, experts in the loop or architects
7:52in the loop or humans in the loop. Uh
7:54but AI augments, it doesn't replace
7:56judgment. Your architects and engineers
7:58define intent, define the trade-offs,
8:00and the approval points at the right
8:01stages. And when you get these three
8:03things working together, you get
8:05repeatable, reliable impact.
8:09And so, let's dive first into shared
8:10context and what we mean by that. And
8:12so, when we say shared context, we're
8:14not talking about documentation for
8:15documentation's sake. Uh shared context
8:18is a living understanding of what you're
8:20building and why. Product intent, domain
8:22rules, architectural decisions, and
8:24constraint. Think of it as your product
8:26or your organizational blueprints that
8:28not only humans can understand, but AI
8:29can as well. And so, most of us
8:32throughout our careers have probably
8:33experienced different artifacts, systems
8:35landscapes, product specs, journey maps.
8:37Um you produce it and they go stale
8:39almost the minute they were produced
8:40because no one really keeps it up to
8:41date. Um
8:45is with AI, you not only can you keep it
8:47up to date, but it's actually critical
8:49and foundational to getting um the most
8:51out of it. And where that context lives
8:54really does matter.
8:56Um and so, we've heard a lot about MCP
8:58and going out externally to grab
9:00information and pulling it into the work
9:02stream. Um while you certainly can do
9:04that, uh we believe context needs to
9:06live within the repo or exactly where
9:08the LM can grab it in a language it
9:10easily understands to avoid delays,
9:12avoid context noise, and really
9:14streamline how it's what it's accessing
9:16and what it means and how it's using to
9:18shape its outcome.
9:19Um and so again, context includes your
9:21code, your tickets, your design
9:22artifacts, but it really needs to be
9:24intentionally structured, curated, and
9:27you'll hear this theme um said
9:28throughout this presentation. Um when
9:31you're delivering a feature or
9:32delivering something new, it's not just
9:34about being code complete, it's about
9:35being context complete as well. And so
9:37when you're merging your branch, you're
9:39not just merging your mutations to the
9:41code, but you're also mutation
9:44merging mutations to your context. And
9:46when you treat context this way, it
9:47really becomes a connective tissue
9:49between AI and people, and it helps keep
9:52things aligned.
9:53You know, one question I get asked often
9:55is this is great. AI is helping you
9:57build code, but what happens when
9:58something breaks? What happens when
10:00institutional knowledge fades or the
10:02system evolves beyond what anyone
10:04remembers? And the answer again isn't
10:06just shared context. It's context that's
10:08been continuously maintained and
10:10embedded into development life cycle
10:12itself.
10:13And when these become living artifacts
10:15that evolve with the system, they become
10:17a reliable source of truth that you can
10:19then plug into either your developers or
10:21to an LLM. And so when there's a bug, a
10:23production issue, a new feature request,
10:26you can feed that up-to-date context to
10:28the LLM and understands exactly what the
10:30current state of the system is without
10:32having to go search through code and
10:34search through documents and external
10:35systems. And this is what helps to
10:37really keep AI aligned with reality. It
10:40helps prevent drift and stale
10:42assumptions, and it makes it helps to
10:44make recommendations that reflect how
10:46the system actually exists today and not
10:48how it existed months ago.
10:51And so this is just an example of what
10:53we mean by context. It's a usually a
10:55markdown file. It's explicitly curated.
10:57It can be derived from external
10:59artifacts in the form of synthetic data
11:01that it represents, but it is lives in
11:04your code base. It's explicit and brief,
11:07but generally pretty information dense
11:08because it needs to contain all the
11:10relevant pieces of information that LLM
11:12will need to understand whatever piece
11:14of context you're looking at. And then
11:16as I mentioned before,
11:18you know, going externally via MCP or
11:20API increases complexity and most
11:23importantly floods the context window
11:25potentially with noise or unnecessary
11:28information that might actually shift
11:30and alter the put you hoping to get.
11:35And so our next principle is governance
11:36and rules. Um
11:38and you know, if shared context defines
11:40what we know, governance defines how we
11:42actually operate. So it's not about
11:44prompts, it's about being explicit,
11:46standards, templates, guardrails, CI
11:48checks baked directly into the workflow.
11:51Um we keep these again in the repo along
11:53with the code because that's where
11:54execution happens and it makes it easy
11:56for LM to grab it and use it while it's
11:58working.
11:59And the goal isn't to micromanage AI,
12:01it's to remove any ambiguity before it
12:03acts and you're going to see that as a
12:04consistent theme as we talk about specs
12:06and spec driven development. It's really
12:08shaping the input so you get the output
12:11you're looking for. And when you do
12:13that, the output stops being random and
12:15it starts aligning with your
12:16architecture, your security posture and
12:18your standards. And this is what
12:19ultimately enables scale, not just
12:21within its use, but scale across the
12:23team because when you have a consistent
12:24source of truth for both your context
12:27and your skills or your rules, now you
12:29can have different teams operating in a
12:31common way,
12:33all adhering to the same organizational
12:35and project level standards while still
12:37allowing flexibility to shape what
12:39they're doing at a more micro level if
12:41needed. And of course, these are always
12:43curated and defined by your architects
12:45and experts and and and they evolve
12:48along with the code as well.
12:51And so again, here's an example of
12:53governance and rules. Um it's just a
12:54markdown file that lives in your repo.
12:57It's explicit and intentional. It's
12:58curated and maintained by your experts
13:00and most importantly, it's maintained
13:02and updated regularly. Um a question I
13:04got asked previously was, what happens
13:06when our security policy changes at an
13:08organizational level? That's a perfect
13:10example of how you want to be able to
13:11take that global organizational change
13:13in security posture, standards,
13:15requirements and make sure it gets
13:17cascaded down and reflected into your
13:19rules and governance so your teams are
13:21now operating under those new
13:22constraints.
13:27And then our last principle is experts
13:28in the loop. Um, notice the play in our
13:31name. It's not accidental, but of course
13:32your experts and architects, but it's
13:34really accountability by design. And so
13:36as AI takes on more execution, there
13:38needs to be explicit points where humans
13:40own the decisions, trade-offs, and
13:42risks. Because if something goes wrong,
13:44um, accountability doesn't sit with the
13:45model, it sits with the organization or
13:47the operator who ultimately produced and
13:49checked in that code. And the challenge
13:51with AI is it often looks good enough at
13:53a glance and it's really tempting to
13:54just accept its output, but that's
13:56exactly where failure creeps in. So the
13:59goal isn't an exhaustive manual review
14:00of everything, it's defining where
14:02judgment matters, design intent, product
14:05decisions, architectural decisions, risk
14:08acceptance, and embedding those
14:10checkpoints directly into your workflow.
14:13And you'll see us demonstrate parts of
14:14that today both on the product and the
14:16delivery side. And when you do this from
14:18the start, review actually becomes
14:19easier and quicker. AI does a lot of the
14:21cumbersome work and experts really focus
14:24on validation and direction. And just as
14:26importantly, this builds trust for
14:29teams, for leadership, and for clients
14:31that AI is accelerating delivery without
14:33eroding responsibility.
14:37And so we've talked about the problem
14:39unmet expectations for AI and the three
14:41foundations, shared context, governance,
14:43and experts in the loop. And with those
14:45in place, AI stops being just a coding
14:47accelerator and it really starts
14:48changing how we think about the life
14:49cycle itself. And the biggest impact is
14:52actually upstream before a single line
14:54of code is written.
14:55And so if AI is going to be grounded in
14:57and aligned with our standards, that
14:59context has to have start with clear
15:01product intent, their requirements, and
15:03a clear definition of what done and good
15:05looks like. And that's where shifting
15:07left really begins. So I'm going to hand
15:09it back over to Mike to walk through
15:10what that looks like on the product
15:12side.
15:14Thank you very much, Andy.
15:17So the next thing we're going to do is
15:19show an example feature moving from
15:21definition to delivery using some of
15:24these techniques that Annie was just
15:25talking about.
15:27And we're starting at the left end of
15:28the delivery cycle with requirements
15:30definition. And here's a process map of
15:33what it looks like to use our pillars in
15:35this part of the process. And one of the
15:38biggest goals in building out the
15:40structure is automating the tactical
15:43busy work that lots of PMs are tasked
15:45with today. And that automation is is
15:48important for two reasons.
15:51First, AI is also accelerating
15:53engineering teams. So, product managers
15:56need to keep pace with that new demand
15:58or they're going to become the new
16:00bottleneck. And we don't want the
16:02we don't want that bottleneck just to
16:04move somewhere else in the life cycle.
16:05It's not going to speed everything up.
16:09Second of all, we need to automate
16:12to open up more time for making sure we
16:14get the strategy right. And also making
16:17that strategy more available to AI. And
16:20that means spending more time on the
16:22activities in the bottom swim lane here
16:24that are more strategic and review
16:26activities.
16:27Remember, AI accelerates you in whatever
16:29direction you're pointing. So, we need
16:31to make sure that we're pointed in the
16:32right direction or we're going to make
16:34bigger mistakes and we're going to make
16:36them faster.
16:38AI also consumes that strategy
16:40differently than people do. So, we need
16:43to make we need to build that strategy
16:45directly into our shared context so AI
16:47has access to that throughout the
16:49development life cycle and understands
16:51what strategy it's building towards.
16:54So, in our swim lanes here, we're
16:55showing governance and rules across the
16:57top. So, these are in the form of AI
17:00agents and skills executing the
17:03activities that we're asking it to
17:04execute throughout this process.
17:07Human in the loop on the bottom where
17:09we're doing strategic activities and
17:11we're reviewing the outputs. And we've
17:13got the shared context in the middle.
17:15So, those are those artifacts that
17:17humans and AI are collaborating on to uh
17:21drive through the the life cycle. And
17:24here we are, we've got these in a loop
17:26so that we're iterating on those. We're
17:28constantly refining them.
17:30And in this particular demo, we're going
17:33to show the right hand side of this.
17:38So, we're not going to show the entire
17:40um requirements definition process
17:42starting from discovery, creating all
17:44these initial documents and things.
17:46We've already completed those in our
17:47repo using AI. We've built all of these
17:51things, and we're going to show uh
17:53refining a brand new feature and going
17:56through that refinement loop and
17:57approval with a human
17:59um before we feed it into the
18:01engineering uh part of the life cycle.
18:09So, before we move to that, I just
18:11wanted to to explain what the
18:13application is that we're building on
18:15top of. So, we're building a feature for
18:16this particular application, which is
18:18very very simple. It's a survey
18:20application that we built for a previous
18:22um event.
18:24There's a survey. There is an admin page
18:27with a login. And that's really it. The
18:29admins can just see how many surveys
18:32have been answered.
18:33And that's all they're able to do today.
18:35Very very simple application. You're
18:37looking at pretty much the entire thing
18:39right here. And we're just going to
18:40build one simple feature on top of this.
18:45So, before I go into the demo itself,
18:48this is going to be a video. And it is
18:51And it moves pretty quickly. So, I want
18:53to describe what we're going to see on
18:54the next page before it starts so that
18:57you're all oriented. So, on When I start
18:59the video, you're going to see a split
19:01screen. And on the left hand side,
19:02you're going to see GitHub, specifically
19:05GitHub issues where we're tracking uh
19:08the feature tickets that we're building
19:09for this project. And on the right hand
19:11side of the screen you're going to see
19:12Slack, which many product teams use for
19:15communication within the team.
19:18So, that's what's on the page here. So,
19:20I'm going to go ahead and start our
19:22video.
19:24There it is. We've got GitHub on the
19:26left and you'll see a typical Kanban
19:27board, which I'm sure most of you are
19:29familiar with.
19:31And in our Kanban board,
19:33we've got our backlog status here and
19:35we're going to create a brand new issue
19:37in the backlog to display results for a
19:40single survey response.
19:42So, we want to be able to see a survey
19:44response that someone is has entered and
19:47we're just going to give it a really
19:48simple one-sentence description here.
19:50Definitely not sufficient for a typical
19:52development story, right? Stories should
19:54be much richer than this, but just as an
19:57example for the sake of time, we're
19:58going to do a very, very simple story
20:00here and show you how we're going to
20:01refine this with AI. So, we're going to
20:03allow someone to see a respondent's
20:05answers.
20:06And
20:07we know that that's not sufficient and
20:09so we're going to
20:11check the status to needs refinement and
20:13that is going to tell the system
20:15to to move this into needs refinement
20:18status. It's going to kick off an
20:19automation in GitHub actions using
20:22Claude code to refine this particular
20:25ticket. And you'll see it's it's started
20:27a thread in Slack and it says it's
20:30reviewing the context now. What does
20:32that mean? So, if you jump back into
20:34GitHub for a moment, we've got a context
20:36directory in our GitHub repo
20:39that has a bunch of things in it. It's
20:41got context
20:42talking about the business requirements
20:44for the product.
20:46It's got a product spec for the product
20:49that's been built already. So, this is
20:51what already exists in the product
20:52today, including some screenshots there.
20:55It's got target personas. It's got our
20:58tech stack.
20:59All that context is available for AI and
21:02it's also got our governance in here in
21:04the form of agents and skills. And so,
21:06we've got a product agent that's been
21:08defined. It's got a a mission that it's
21:10trying to achieve. It's got a bunch of
21:12resources available to itself to use in
21:15accomplishing this goal. Some of those
21:17resources are skills, and it's got a
21:19Slack conversation skill as well as a
21:21story management skill. And the story
21:23management skill contains a template for
21:25what stories should look like.
21:28So, that's how it's going to create the
21:30stories.
21:31So, it's finished reviewing the context,
21:33and now it's going to start a thread
21:35that I'm going to open up. And it's
21:37asking me some questions
21:39that it needs to refine the story
21:41further. And the first one it's asking
21:43is who's developing this? Is it an LLM
21:45or a human developer? And we found that
21:47this is important because those stories
21:48look different.
21:49LLMs have all of that context available
21:51to them all the time, and they can keep
21:53it in their
21:54in their proverbial head. So, it doesn't
21:57need as much in the story itself. So,
21:59the format can be a little bit
22:00different. I'm going to answer those
22:02questions. It's going to come back and
22:03ask me more questions.
22:05This is sped up
22:07a little bit. I'm not You don't have to
22:08wait for the LLM to answer.
22:11Um so, it's going to I'm going to answer
22:13these additional questions it's got for
22:16me, and it's going to take these back.
22:18And it's ultimately going to
22:20produce
22:23a refined story for me to approve.
22:26And here's the story that it sent me.
22:30And if we start at the top here and look
22:32through, we've got a typical story in in
22:34a format that you'd expect.
22:36It's got some additional business
22:38context. It's got some metadata here.
22:40Acceptance criteria
22:42uh in behavior-driven development
22:44format.
22:46That given-when-then format there. It's
22:48got some non-functional requirements,
22:50some additional quality checks.
22:53No open questions notice because we
22:55answered those in the Slack thread in
22:57our conversation.
22:59And it's asking me to explicitly approve
23:01to finalize the story. And so, I'm going
23:03to type approve and send that back.
23:06It says this It's been refined and
23:08labeled, so it's we should see it show
23:10up in the refined column now. And if we
23:13open up this story, it's now populated
23:16that issue with the story that we've
23:18just approved.
23:20In the comments, it's got a checklist
23:22that's showing what it accomplished
23:26uh
23:27in the Claude automation, and it also
23:29has a conversation log that shows the
23:32conversation it had with me in Slack
23:35with timestamps and who it was
23:38conversing with, as well as the story
23:41that was approved. It's right in there.
23:43And my official approval that this is
23:47what I wanted to go into the issue, so
23:49that we've got an audit trail now of
23:51everything that AI helped me with in
23:53producing this particular story.
23:58And so now it is ready to turn over to
24:00engineering. And uh before I hand it
24:03back to Andy, I just wanted to mention
24:04that this was it I recorded this in real
24:07time. I cut out, as I mentioned, kind of
24:09the long waits waiting for the LLM to
24:11come back to me, but other than that, it
24:14is uh
24:16exactly as I went through it when I
24:18recorded this.
24:19In real life, it this took about 10
24:21minutes or so. It cut it back to about 5
24:23minutes um with the additional part
24:25added to to walk through the context.
24:28So, it can move very, very quickly when
24:31you're using AI to kind of take out all
24:33the burdensome, cumbersome pieces that
24:36you're normally dealing with on a
24:37day-to-day basis. So, with that, I'm
24:39going to turn over to Andy to walk
24:40through the rest of the process.
24:43Thanks, Mike. And what I love about that
24:45is we're not just shifting the workflow
24:46left, we're actually shifting how we
24:48engage further left to our stakeholders.
24:51I don't know if you've ever experienced
24:52a stakeholder that you just can't get to
24:54go into Jira and update a ticket or
24:56refine one. Uh but that's one of the
24:58great things about AI is you can
24:59actually take that whole workflow,
25:01simplify it, and actually move it to
25:03where they're already working. We use
25:04Slack, but you could have just used
25:06Teams. Um that's but I really enjoy that
25:08part. Thank you, Mike.
25:10All right, cool. Back to the engineering
25:12fun, right? Um and so, our next step,
25:14now we've got a well-refined feature, is
25:17to get back to the fundamentals of
25:18software development, which could be a
25:19little boring for some, but I think it's
25:21really important here as we work with
25:22AI. Um because before you write any code
25:24human AI, we want to be explicit about
25:26what we're building and why. That's the
25:28idea of spectrum in development, the
25:30term I'm sure most of us have probably
25:31heard before. And if you think about how
25:33teams evolve from more waterfall style
25:35delivery into Agile, some of that rigor
25:38naturally got lost. We optimized for
25:40iteration and speed and relied on people
25:42to fill in the gaps as we went. And that
25:44worked when humans were doing most of
25:45the work, but with AI, it doesn't quite
25:47work the same way because you don't
25:49really want the AI filling in the gaps
25:51and making assumptions. And so, bringing
25:53that discipline back isn't about going
25:54backwards. It's about creating clarity
25:57and intent before asking it to execute.
25:59The spec itself becomes a checkpoint.
26:01Experts or architects can review it,
26:03validate direction, catch issues early
26:05on before coding anything. And once it's
26:07defined, it doesn't matter whether
26:09developer or humans or an LLM, you're
26:11creating something deliberate and
26:12intentional.
26:14Um and again, this is just an example of
26:16what a spec would look like. It lives in
26:18your repo with your code base. It ends
26:20up It's a good practice to link it back
26:22to the Jira ticket or a GitHub issue
26:24whatever item that it's actually solving
26:26for. Um it contains regurgitation of
26:29requirements. It defines how you're
26:31going to solve the acceptance criteria,
26:33what new classes you might want to
26:34introduce. It's essentially a blueprint
26:36or a real technical spec that you want
26:38to sign off on before it building
26:40anything.
26:42And so, once our spec's defined,
26:44implementation really becomes a
26:46structured orchestrated workflow. Uh we
26:48hand the spec to the LLM and it
26:49generates the back-end changes or
26:51front-end changes, Whatever the spec
26:52called for.
26:54Um and if the spec's too large, by the
26:55way, we can break it up into logical
26:57pieces, but the flow is generally
26:59consistent. Generate code, um generate
27:02tests, and then validate against the
27:03spec. And throughout that process, the
27:06shared context informs decisions,
27:08governance enforces patterns, and
27:10predefined skills keep the output
27:11aligned with our standards.
27:13Um experts then review the results. You
27:16could review it in a PR, you could
27:17download it locally to your IDE, run it,
27:19approve it, or tweak it. And what's
27:21really cool about that is if you're
27:22using Quad Code or CodeX or whatever CLI
27:24tool you like, um you're using the same
27:27context and rules locally as you did
27:29kind of with this more agentic stream
27:31that we're kind of showing you. And so
27:32they actually complement each other. Um
27:34and so if you do want to iterate on the
27:36code, you can locally as well. But you
27:38don't have to go create a whole new set
27:40of context, rules, and skills. They're
27:41already there for you to use and tweak
27:43as you go along.
27:44And I think this is actually where some
27:46of the biggest time impact shows up.
27:48Because we're not spending time writing
27:50boilerplate or rewriting code we've
27:51written hundreds of times. Um we're
27:54defining intent and guardrails,
27:56reviewing the output, and making sure
27:57it's exactly what it would have been had
27:59we written it ourselves.
28:05Oh, perfect. Sorry. Um
28:07so once implementation is complete, we
28:08move into validation. Um the test cases
28:11are generated directly from the
28:12specification and the acceptance
28:13criteria that we defined um during the
28:15requirement stage. So we're validating
28:18exactly what we intended to build. Um
28:20from there we analyze edge cases, run
28:22the regression suite to make sure
28:24changes don't break existing behavior.
28:26All this is guided by shared context, a
28:28specification, test strategy, and the
28:29evolving test suite. And importantly,
28:31humans stay in the loop. Engineers
28:33review the results and approve the suite
28:35before any changes move forward. So
28:37validation becomes a governed,
28:39repeatable step in the life cycle,
28:41rather than something ad hoc we do at
28:43the end.
28:46And this last piece I really want to
28:47emphasize what I think makes what we're
28:49talking about today fairly unique unique
28:52um and it's the idea of treating um
28:54context as as if it were code or part of
28:56your code base as well. And so you hear
28:58people talk about context, but for us it
29:00means something very very specific. The
29:02artifacts you create at the beginning of
29:04the projects don't become stale. It's
29:06not a one-time exercise, it's part of
29:08the life cycle, and with every feature
29:10release, you're not just merging code,
29:12you're also merging the context updates
29:14themselves. So if you just new
29:16endpoints, DTOs, architectural
29:17decisions, your architectural docs get
29:20updated, your specs get updated, and
29:22your context files get updated. And a
29:25feature isn't complete until it's code
29:27complete and context complete. We really
29:29want to emphasize that point. Because
29:31that's where this becomes really
29:32powerful. Um you end up with true
29:34end-to-end traceability from the
29:35original requirement to the spec that
29:37satisfied it to the code that
29:38implemented it to the updated artifacts
29:41describe the system. And that's
29:42something a lot of organizations have
29:44not yet mastered this idea that it's a
29:46full cycle thing as opposed to a linear
29:48development workflow. And when context
29:51evolves with the system, you minimize
29:53drift, you can see what's changed, why
29:55it changed, and you can all understand
29:57the gaps that were ultimately closed by
29:58delivering a feature.
30:00And so I'm going to take you through a
30:01really brief demo here of what that
30:03looks like practically and how easy it
30:05is to do that. Um so Mike, if you don't
30:07mind hitting play, please.
30:12Fantastic. Um cool. And so as Mike
30:15showed you earlier, we have agents. In
30:17this case, we'll create a specific
30:18context manager agent. We like to use
30:20agents sometimes for specific tasks cuz
30:22you can do unique things. They run in
30:23their own context window. They can call
30:26other agents. You can also specify which
30:28model you want to use for a task. So in
30:31your context agent, maybe you want to
30:32use a more recent in-depth reasoning
30:34model just so it can better create
30:36update the skills in context. Um but
30:38ultimately, it's the agent definition
30:40here itself and within it it says which
30:42skills to use um to manage context and
30:46here again is our context management
30:47skill. It talks to the core principles,
30:50what the sort what it should look for
30:51its source, where it should be putting
30:54the updated context files, whether it
30:56should be updating your Claude that MD
30:58or your copilot MD files, right?
31:00Depending on what you're using. And then
31:02of course here is like our context
31:04directory itself. We check this shape
31:06generically just for the purpose of this
31:07demo. Claude kind of wants you to do one
31:10way and Codex wants you to do it similar
31:12way
31:13but ultimately it's just markdown files
31:15in a repository usually with an index
31:17file that's showing you what files
31:19govern which process
31:22but these are again maintained as a
31:23final step of our release pipeline once
31:25we're happy with both the code and the
31:27validation.
31:29Um
31:30just kind of showing you what like a
31:31teaser of an adjunctive feature looks
31:33like, right? And so
31:35we have this all hooked up to GitHub
31:37actions and so they're triggering
31:39actions that call our agents that
31:41effectively generate the context or do
31:43different phases of this
31:45of our life cycle here.
31:47Um and then what's really cool is when
31:49you then come back to your feature
31:50branch
31:52you get a full record of everything
31:54that's really happened
31:56along the way and so you look at this
31:58this this feature branch and it has the
32:00spec that was created, it has the code
32:02that was created in the commit related
32:03to it, the QA that was created and this
32:06final step here which is the context
32:08that was ultimately mutated to represent
32:10the changes that were created by that
32:12feature branch. It's all self-contained
32:14and self-referencing back to the initial
32:17um request that asked for it.
32:22Cool.
32:23And so kind of just bring this all
32:25together, right? This is an example of
32:27what maybe a very forward-looking
32:29adjunctive life cycle might look like.
32:32In the magenta you have kind of your
32:34agents facilitating things and you have
32:36your context there at the bottom and a
32:38top and sort of that aqua blue color. Um
32:41you have the different skills and
32:42governance that are shaping the output
32:44as you work through this delivery
32:45stream.
32:47Um and what's really cool is when you
32:48create these foundations, it's it's it's
32:50helping us develop code as part of what
32:52we just showed you, but what it also
32:54does is it enables you to really shift
32:56the process further left and you
32:58leverage the same context and skills in
33:00other places that you might want to
33:01work. My favorite example is
33:02prototyping, right? Um so if you have
33:04the context and it has your like design
33:07artifacts in there and you have your
33:08skills that have your design you know
33:10your your your different um you know
33:12media guidelines for how to produce
33:14stuff, you could use something like
33:15Figma make or other tools and start
33:17prototyping where product and and the
33:19business side can start prototyping and
33:21seeing what they're asking for, how it
33:23could look and feel, and it could remain
33:25consistent with everything else that
33:27you're producing. So that could be an
33:28additional artifact they could hand
33:30further downstream um to eliminate any
33:32ambiguity and validate um the user
33:35experience themselves.
33:36Um and as you can see, there are
33:38different steps here with orange
33:39representing the different stages where
33:41humans should be making decisions. And
33:43one question I get is what's the big
33:44difference between this and how we work
33:46today? Um this is if you can imagine
33:48agents or an agentic workflow pulling
33:51work as you go from left to right. Um
33:54you could have humans doing all those
33:56magenta boxes where you see agents,
33:57totally fine. You could be doing that in
33:59your terminal using the same context and
34:01skills. Um but I know a lot of folks are
34:03imagining a world where you wake up and
34:05you have a PR to review and you and your
34:07day reviewing the code um
34:09that got generated along the way. Um and
34:12so
34:13So this is just kind of bringing
34:14everything together and giving you guys
34:16a full picture of what life in the
34:18future might look like or in the present
34:20for
34:21um some forward-looking organizations.
34:26And so last but not least, I really want
34:27you all to kind of think about what this
34:29um slide represents.
34:31Um context as the foundation, skills as
34:33a layer above, agents above that, and
34:36human oversight in the middle. And what
34:39I think sometimes gets lost on us as
34:40engineers and product individuals is
34:42what we're we're building things with
34:43that what's immediately in front of us,
34:45but we're actually laying the foundation
34:47to create an agentic future, not just
34:50for development, but for the
34:51organization as a whole. Because the
34:53context and skills we're creating might
34:55be useful for development, but the
34:56marketing user might also want to tap
34:59into those same context text files and
35:00skills when they're creating their
35:01journeys. Or you might have an executive
35:03that wants to understand a product, why
35:05it was built, how it was built, what are
35:07some of the non-functional requirements
35:08or business requirements within it.
35:10Well, that lives in the context, and
35:11they could just pull up their local
35:12version of Claude, tap into the shared
35:15context and skills layer, and start
35:17asking their own questions about what's
35:18there. And so, I really want to leave
35:20everyone with this kind of picture in
35:21mind as we're mostly technical folks, I
35:23think on this call. Um, what we're
35:25creating isn't just for us, it's really
35:27to help usher in the future state of an
35:29agentic world where um, we're leveraging
35:32these tools and these foundational
35:34principles to accelerate other areas of
35:36work throughout the organization.
35:41And so, that's it. I just want to say
35:42thank you. I'll hand it back to Katie
35:44here in a moment, but appreciate
35:45everyone's time, and um, hopefully you
35:47all learned something.
35:49Awesome. Thank you, Andy Mike. That was
35:51great. It's always good to to hear the
35:53detailed behind all of this. Um, we know
35:55there's a lot of talk about um,
35:58a lot of the Advantage AI can bring, but
36:00seeing this specifically laid out here
36:02is really useful.
36:04So, we've got about five questions.
36:05We've got um, just under 10 minutes to
36:08respond. So, we'll go through those
36:09pretty quickly. We've got some votes
36:11here, so I'll go in vote order. Uh,
36:13starting from a question from Ray. Um,
36:16do you recommend any particular
36:17framework like Spec Kit or Compound
36:19Engineering plugin?
36:23Um,
36:25so, we try to
36:27Well, so
36:28we are happy to share what we would
36:29recommend, but we generally try to take
36:31a fairly agnostic perspective on how we
36:34implement this and really anchoring on
36:36those sort of three principles. Um what
36:38we found is every organization is not
36:40unsurprisingly a little bit different.
36:42The tools they could use, what they
36:43could bring into their stack. We tend to
36:45work with a lot of large enterprise
36:46clients that have really restricted
36:48environments. Um so, um maybe happy to
36:52connect with you offline, Ray, and share
36:53some of the tooling that we've done. Um
36:56but but yeah, I mean we we try to
36:58create principles in a workflow that can
37:01translate to whatever stack an
37:02organization itself is using and adhere
37:04to to their world without introducing
37:07with with minimal introduction of any
37:09new tooling if we can.
37:11Great. Thanks, Andy.
37:14Um all right. Next one up from Rodrigo,
37:17we have um a question on I have been
37:19having issues creating something like
37:21this when consulting for multiple
37:23clients. Each one has its own rules and
37:25defining rules to rule them all have
37:27have been an issue. Um some don't even
37:30have version control. Just wondering how
37:31do you manage this issue?
37:36Mike, you want to take that or happy to
37:37do that? It's a good one. I appreciate
37:39the question, Rodrigo.
37:41Yeah, it's it's definitely tricky. Um
37:44so
37:45I mean really need to take each client
37:47individually. So So, your company is is
37:50going to be the thing that you're
37:52focused on, right? So So, what are the
37:54challenges within your organization? I
37:55mean
37:56we'd like to start small.
37:58Right? So, we like to find a a project
38:00with the team that's very passionate
38:02about what they're doing and develop
38:04something that works for that team and
38:06then expand from there. Right? So,
38:09trying to go across the entire
38:10organization and find something that
38:12works for everybody at the same time is
38:14usually going to be very challenging.
38:16It's just a big change management
38:18problem regardless of the type of change
38:20you're trying to introduce. So So,
38:22starting small, finding a good example
38:26team that can execute something and get
38:28it to work and show the results is the
38:31important first step to get everybody
38:33sort of on board, and then you can take
38:34and build on that from there.
38:37Yeah, and maybe I'll just add um one
38:39thing that like I've seen us do
38:41practically very recently is creating
38:43like um a single repository, like a
38:45global repository that has kind of
38:47global skills, global context, global
38:49rules, and then that as a sub-module
38:52into the different projects you might
38:54work on. And so, Rodrigo, in your case
38:55where you have different clients, you
38:57might have your toolkit that you build
38:58out in a central repository, and you
39:00would import that to get started quickly
39:02with a given client that you're now
39:03working on. Um and then within that repo
39:07that you're using for that specific
39:08client, if there are unique
39:10variations of whatever it is they need,
39:12then you can have that live locally
39:13there. Um so, that's that's that's one
39:16option.
39:18Great. Thanks, Mike and team. We'll move
39:20on. I'm conscious of time. We've got a
39:22few more minutes left. Uh question from
39:24Rajnish. Hi, Rajnish.
39:26Um I'm I'm curious, what are the best
39:28practices for evolution, major changes
39:31to the shared context? For example,
39:33adding observability across the system.
39:36Mhm.
39:38Who wants to take that?
39:42I'll give it a stab, and Mike, maybe
39:43you'll color it in, anything I might
39:45have missed here. But, yeah, I mean,
39:46it's a good question. Um
39:48managing changes to shared context is
39:51probably the most challenging part in
39:53all this, and how do you like stay aware
39:55of changes and cascade them down, right?
39:58Especially when they don't live um
39:59within the within the immediate
40:01vicinity.
40:02Um observability, I I'm not sure what
40:04you mean by that. I mean, I mean, I know
40:05what you mean, but I don't know if you
40:06mean it observability in terms of
40:07context or observability in terms of
40:09code. Um if it's in terms of just
40:12context and source documents that feed
40:14into like a local repo's context,
40:17Um
40:18There are a couple of ways you work
40:20through managing that. There are some
40:22tools in that case that are pretty good
40:24that can observe changes in files and
40:26cascade them downward and trigger
40:27actions
40:29um
40:29in terms of code large code changes.
40:34Um
40:39Well, so
40:40large code changes I guess is a relative
40:42question. You One of the One of the core
40:45our philosophies about all this by the
40:46way is that
40:48if you want to use AI to develop
40:49software, you should be conscious of its
40:51limitations and how to best use them.
40:54And so if you have some one giant
40:56monolithic code base, uh AI is going to
40:58struggle to like really comb through all
41:00that. Not that it can't, but it's going
41:02to be, you know, a lot to comb through
41:04to then go and evolve context.
41:06Um taking a more of a modular
41:08microservice-based approach to how you
41:10tackle a monolith or decompose it and or
41:12how you handle future project will give
41:15you the ability to manage context and
41:18observe code changes and update the
41:19context based on that
41:21um
41:22more easily and clearly.
41:25Great. Thanks, Andy. We can always
41:27follow up with Rajneesh as well.
41:29Um I'll move on from that just so we can
41:31get through some of these other
41:33questions, but thank you, Rajneesh.
41:36Um Okay, so the next question from
41:38Amitabh I think is something we can just
41:39share afterwards is please share the
41:41GitHub repo you referenced here so we
41:43can send that out as a follow-up
41:46I presume.
41:48Great. And then uh Shivi, where does a
41:52program manager projects manager sit in
41:54here? So maybe this is a good question
41:56for Mike cuz I know this is a topic
41:57close to your heart.
41:59Yeah, it's very interesting. I mean,
42:01obviously this is
42:03it's very organizationally specific,
42:05right? So um
42:07you know, some organizations have
42:09uh a more um
42:12modern product operating model. You
42:14know, other organizations are
42:16a little bit more old-fashioned and have
42:19project managers throughout.
42:21So, I think typically they're they're
42:23going to have their we're going to start
42:25with them in their current role, where
42:27they're at.
42:29And the reason I was part of this
42:31presentation is because that stuff that
42:33happens early on at the very beginning
42:36is super important, right? It's the the
42:39amount of leverage that you have as a
42:41project manager or as somebody defining
42:44what it is that you're building
42:46uh increases exponentially once you add
42:48AI, right? Because everything that
42:50happens after that is accelerated so
42:51much. So, it's very very important for
42:54for that person to have a a prominent
42:56role in what's being defined
43:00um and to be very active in helping to
43:03build the context out. Um and so
43:07so, I'm not sure what you mean by where
43:09they sit, but I think that there I I
43:11think that you know, there's been a lot
43:13of talk about how these these positions
43:16are going away.
43:18It's just not happening at all. Like the
43:20the openings for product managers, for
43:22example, are actually increasing.
43:23There's more open product manager roles
43:26in the industry
43:28just overall in all industries than
43:31there was before AI was out. So, and
43:34it's just continued to increase, which
43:35is very surprising, right? I don't think
43:37most of us would have predicted that.
43:38So, there's a huge role for them to play
43:41in this and and it's becoming more and
43:43more important to get this stuff right.
43:46So, um
43:47so, I think that it's it's super
43:48important, but where they sit is
43:50definitely the left end of this process
43:53is
43:54is becoming the more important piece of
43:56it, which is what we're trying to get
43:57across here in this presentation today.
43:59Great. Thanks, Mike. Um we're going to
44:02squeeze in one last one cuz there's a
44:03few people have voted on this one. Um
44:05this is from an anonymous attendee, but
44:07how long um
44:09might it take for this setup to be up
44:11and running and introduced to a new
44:13company?
44:14Good question. That's a good question.
44:16One minute to answer this before the
44:17webinar ends.
44:20I would say it's pretty quick to get
44:22going like the foundations of it and
44:23start seeing immediate impact. I mean
44:25like you can get a baseline there in
44:27like a week and probably 2 weeks you
44:28have a pretty good curated set of
44:30contacts skilled and governance.
44:32Um the largest hurdle is the human
44:35element and so implementing the change
44:38management around working in a new way
44:41and thinking about contacts as part of
44:43your code base. And so that requires
44:45repetition, training, and curation and
44:47then making sure your skills and and
44:50whatever you learn along the way
44:51evolves. So just getting a baseline out
44:53the door is actually pretty quick. Um
44:55but then it's sort of the ongoing
44:57maintenance and the change management of
44:59that that usually takes
45:01um you know
45:02be a month or two to really like cement
45:05and get folks um driving.
45:08Great. Thanks, Andy. I'm sorry we've got
45:09a few more questions, but the uh webinar
45:11is going to come to a an end. What we'll
45:13do is we'll follow up. Uh we'll take a
45:15copy of these questions and follow up
45:17afterwards. But thank you everyone for
45:18joining us. Thank you, Andy and Mike.
45:20It's really insightful talk. Appreciate
45:22it. Thank you. Bye.
45:24>> Thanks, everyone. Thank you, everybody.