Full transcript
0:00Hi everyone. Thank you for joining us
0:01today for prompting 101. My name is
0:04Hannah. I'm part of the Applied AI team
0:06here at Anthropic. And with me is
0:08Christian, also part of the Applied AI
0:10team. And what we're going to do today
0:12is take you through a little bit of
0:13prompting best practices. And we're
0:15going to use a real-world scenario and
0:17build up a prompt together. So a little
0:20bit about what prompt engineering is.
0:23Prompt engineering, you're all probably
0:24a little bit familiar with this. This is
0:26the way that we communicate with a
0:28language model and try to get it to do
0:30what we want. So this is the practice of
0:32writing clear instructions for the
0:33model, giving the model the context that
0:35it needs to complete the task, and
0:37thinking through how we want to arrange
0:39that information in order to get the
0:41best result. So there's a lot of detail
0:43here, a lot of different ways you might
0:45want to think about building out a
0:46prompt.
0:47And as always, the best way to learn
0:49this is just to practice doing it. So
0:52today we're going to go through a
0:53hands-on scenario.
0:54We're going to use an example inspired
0:56by a real customer that we worked with.
0:59So we've modified what the actual
1:00customer asked us to do, but this is a
1:02really interesting case of trying to
1:03analyze some images and get factual
1:07information out of the images and have
1:08Claude make a judgment about what
1:10content it finds there. And I actually
1:13do not speak the language that this
1:15content is in, but luckily Christian and
1:17Claude both do.
1:19So I'm going to pass it over to
1:19Christian to talk about the scenario and
1:21the content.
1:22>> So for this example that we have here,
1:24it's
1:26intended So so to set the stage, imagine
1:28you're working for a Swedish insurance
1:29company and you deal with car insurance
1:32claims on a daily manner.
1:35And the purpose of this is that you have
1:36two pieces of information.
1:38We're going to these in detail as well,
1:39but visually you can see on the
1:41left-hand side we have car accident
1:43report form
1:45just detailing out what transpired
1:47before the accident accident actually
1:48took place. And then finally we have a
1:50sort of human-drawn
1:53a sketch of how the accident took place
1:55as well. So these two pieces of
1:56information is what we're going to try
1:58to pass on to Claude. And to begin with,
2:01we could just take these two and throw
2:02them into a console and just see what
2:04what happens. So if we transition over
2:06to your console as well, we can actually
2:08do this in a real manner. And in this
2:10case here you can see we have our shiny
2:12beautiful Anthropic console. We're using
2:15the new Claude 4 solid model as well. In
2:17this case,
2:19setting temperature zero and having a
2:21huge max token budget as well as it's
2:23helping us make sure that there's no
2:25limitations to what Claude can do. In
2:27this case you can see we have a very
2:28simple prompt to setting the stage of
2:30what Claude is supposed to do in this
2:31case, mentioning that this is intended
2:34to review a an accident report form
2:37and eventually also determine what
2:40happened accident and who's at fault.
2:42So you can see here with this very
2:43simple prompt, if I just run this, let
2:45me go to preview.
2:48We can see here that Claude thinks that
2:52this is in relation to skiing accident
2:55that happened on a street called Schepam
2:57gatan, which is a very common street in
2:59Sweden. And in many ways you can sort of
3:01understand this innocent mistake in the
3:03sense that in our prompt we actually
3:05haven't done anything to set the stage
3:07on what is actually taking place here.
3:10So his sort of first guess is not too
3:12bad, but we still notice a lot of
3:13intuition that we can bake into Claude.
3:16So we switch back to the slides,
3:19you can see here that
3:21in many ways prompt engineering is a
3:22very iterative empirical science. In
3:25this case here we could almost have a
3:27test case where Claude is supposed to
3:29make sure that it understands it's in
3:31the car or vehicular environment,
3:33nothing to do with skiing. And in that
3:35way you iteratively build upon your
3:38prompt to make sure it's actually
3:39tackling the problem you're intending to
3:41solve. And to do so, we go through some
3:44best practices of how we we at Anthropic
3:46break this down internally and how we
3:48recommend others do so as well.
3:51So we're going to talk about some best
3:52practices for developing a great prompt.
3:54First we want to talk a little bit about
3:57what a great prompt structure looks
3:58like. So you might be familiar with kind
4:01of interacting with a chatbot, with
4:02Claude, going back and forth, having a
4:04more kind of conversational style
4:06interaction. When we're working with a
4:08task like this, we're probably using the
4:10API and we kind of want to send one
4:12single message to Claude and have it
4:14nail the task the first time around
4:16without needing to kind of move back and
4:18forth. So the kind of structure that we
4:21recommend is setting the task
4:23description up front. So telling Claude,
4:25what are you here to do? What's your
4:26role? What task are you trying to
4:28accomplish today? Then we provide
4:30content. So in this case it's the images
4:32that Christian was showing, the form and
4:34the drawing of the accident and how they
4:36occurred. That's our dynamic content.
4:38This might also be something you're
4:39retrieving from another system,
4:40depending on what your use case is.
4:42We're going to give some detailed
4:43instructions to Claude. So almost like a
4:45step-by-step list of how we want Claude
4:48to go through the task and how we want
4:50it to tackle the reasoning.
4:52We may give some examples to Claude.
4:54Here's an example of if some piece of
4:56content you might receive, here's how
4:57you should respond when given that
4:59content. And at the end, we usually
5:01recommend repeating anything that's
5:02really important for Claude to
5:04understand about this task. Kind of
5:06reviewing the information with Claude,
5:08emphasizing things that are extra
5:10critical, and then telling Claude,
5:12"Okay, go ahead and do your work."
5:14So here's another view. This has a
5:16little bit more detail, a little bit
5:18more of a breakdown. And we're going to
5:19walk through each of these 10 points
5:21individually and show you how we build
5:23this up in the console. So the first
5:26couple things, Christian's going to talk
5:28about the task context and the tone
5:29context. Perfect. So yeah, if we begin
5:33with the task context, as you realize
5:35when I went through the little demo
5:35there,
5:37we didn't have much elaborating what
5:39what what's the scenario Claude was
5:41actually working within. And because of
5:42that, you can also tell that Claude
5:44doesn't necessarily needs to guess a lot
5:46more on what you actually want from it.
5:47So in our case, we really want to break
5:48that down, make sure we can give more
5:50clear-cut instructions,
5:51and also make sure we understand what's
5:54the task that we're asking Claude to to
5:55do.
5:56Secondly as well, we also make sure we
5:59add a little bit of tone and title.
6:01Key thing here is we want Claude to stay
6:03factual and to stay confident. So if
6:07Claude can't understand what it's
6:08looking at, we don't want it to guess
6:10and just sort of mislead us. We want to
6:11make sure that any assessment, and in
6:14our case we want to make sure that we
6:15can understand who's at fault here, we
6:17want to make sure that assessment is as
6:19clear and as confident as possible. If
6:21not, we're sort of losing track of what
6:22we're doing. So if we transition back to
6:25the to the console, we can jump to a V2
6:28that we have here. So I'll just navigate
6:31to V2.
6:33And you can see here
6:34I'll also just illustrate the data cuz
6:36we didn't really do that last time
6:37around. Just to really highlight what
6:38we're looking at. So what we're seeing
6:40here is this is that
6:41car accident report form. And it's just
6:4417 different checkboxes going through
6:47what actually happened. You can see
6:48there's a vehicle A and vehicle B both
6:50on the left and right-hand side. And the
6:51main purpose of this is that we want to
6:53make sure that Claude can understand
6:54this manually generated data to assess
6:58what's actually going on. And that is
7:00corroborated by, if I navigate back
7:02here, to the sketch that we can
7:04highlight here as well. In this case,
7:06the form is just a different
7:09data point for the same scenario.
7:12And in this case here I want to bake in
7:13more of the information into our version
7:15two. And by doing so, I'm actually
7:18elaborating a lot more on what's going
7:19on. So you can see here I'm specifying
7:21that this AI system is supposed to help
7:24a human's claim claims adjuster as
7:26reviewing car accident report forms in
7:29Swedish as well. You can see here we're
7:31also elaborating that this is a
7:32human-drawn sketch of the incident and
7:35that it should not
7:37make an assessment if it's not actually
7:38fully confident. And that's really key
7:40because if we run this, you'll see that
7:43And you can see it's the same settings
7:44as well, Claude 4, our new shiny model,
7:46zero temperature as well. If we run
7:48this, we can see here what actually
7:51happens.
7:52In this case,
7:53Claude's able to pick up that now it's
7:56relating to car accidents, not skiing
7:58accidents, which is great. You can see
7:59it's able to pick up that vehicle A was
8:02marked on on checkbox one and then
8:04vehicle B was on 12.
8:07And if we scroll down though, we can
8:08still tell that there's some information
8:10missing for Claude to make a fully
8:12confident determination of who's at
8:14fault here. And this is great. This is
8:16pertaining to the task we have set. Make
8:18sure you don't make anything any claims
8:20that aren't
8:22factual and make sure you you only sort
8:24of assert things when you're when you're
8:25confident. But there's a lot of
8:26information we're still missing here
8:28regarding the form, what the form
8:31actually entails. And a lot of that
8:33information is what we want to want to
8:34bake into this LLM application as well.
8:38And the best way of doing so is actually
8:39adding it to the system prompt, which
8:41Hannah will elaborate on.
8:43So back in the slides, we have the next
8:46item we're going to add to the prompt.
8:47And this is background detail, data,
8:50documents, and images. And here, as
8:53Christian was saying, we actually know a
8:54lot about this form. The form is going
8:56to be the same every single time. The
8:57form will never change. And so this is a
9:00really great type of information to
9:01provide to Claude, to tell Claude,
9:03"Here's the structure of the form you'll
9:05be looking at." We know that will not
9:07ever alter between different queries.
9:09The way the form is filled out will
9:11change, but the form itself is not going
9:12to change. And so this is a great type
9:15of information to put into the system
9:17prompt. Also a great thing to use prompt
9:19caching for. If you're considering using
9:20prompt caching, this will always be the
9:21same. And what this will help Claude do
9:24is spend less time trying to figure out
9:26what the form is the first time it sees
9:28the form each time. And it's going to do
9:30a better job of reading the form because
9:32it already knows what to expect there.
9:36So another thing I want to touch on here
9:38is how we like to organize information
9:40in prompts. So Claude really loves
9:42structure, loves organization. That's
9:44why we recommend following kind of a
9:46standard structure in your prompts. And
9:48there's a couple other tools you can use
9:50to help Claude understand the
9:52information better. I also just want to
9:53mention all of this is in our docs with
9:55a lot of really great examples. So,
9:57definitely take pictures, but if you
9:59forget to take a picture, don't worry.
10:01All of this content is online with lots
10:03of examples and definitely encourage you
10:05guys to check it out there, too. Um
10:08Anyway, the So, some things you can use
10:10delimiters like XML tags. Also, markdown
10:14is pretty useful to Claude, but XML tags
10:16are nice because you can actually
10:17specify what's inside those tags. So, we
10:20can tell Claude, "Here's Here's the user
10:22preferences. Now, you're going to read
10:24some content." And these XML are letting
10:26you know that everything wrapped in
10:27those tags is related to the user's
10:29preferences, and it helps Claude refer
10:31back to that information maybe at later
10:33points in the prompt. Um So, I want to
10:35show in the back in the console how we
10:39actually do this in this case.
10:41And Christian's going to pull up our
10:43version three. So, we're keeping
10:44everything about the other part of the
10:46user prompt the same, and we've decided
10:49in this case to put this information in
10:50the system prompt. You could try this
10:52different ways. Um we're doing it in the
10:54system prompt here. And we're going to
10:55tell Claude everything it needs to know
10:57about this form. So, this is a Swedish
10:59car accident form. The form will be in
11:01Swedish. It'll have this title. It'll
11:03have two columns. The columns represent
11:05different vehicles. We'll tell Claude
11:07about each of the 17 rows and what they
11:10mean.
11:11You might have noticed when we ran it
11:12before, Claude was reading individually
11:15each of the lines to understand what
11:16they are. We can provide all of that
11:18information up front. And we're also
11:20going to give Claude a little bit of
11:21information about how this form should
11:23be filled out. This is also really
11:25useful for Claude. We can tell it things
11:27like
11:28you know, humans are filling this form
11:30out, basically. So, it's not going to be
11:31perfect. People might put a circle. They
11:33might scribble. They might not put an X
11:35in the box. There could be many types of
11:37markings that you need to look for when
11:40you're reading this form.
11:41Um we can also give Claude a little bit
11:43of information about how to interpret
11:44this or what the purpose or meaning of
11:46this form is. And all of this is context
11:49that is hopefully really going to help
11:50Claude um do a better job analyzing the
11:53form.
11:54So, if we run it,
11:56everything else is still the same. So,
11:57we've kept the same user prompt down
12:00here. Oh, your scroll is backwards from
12:01mine. Uh
12:03the You have the same user prompt here.
12:05Still asking Claude to do the same task,
12:07same context. And we'll see here that
12:10it's spending less time. It's kind of
12:12narrating to us a little bit less about
12:14what the form is because it already
12:15knows what that is, and it's not
12:17concerned with kind of bringing us that
12:19information back. It's going to give us
12:21a whole list of what it found to be
12:23checked, what the sketch shows. And
12:25here, Claude is now becoming much more
12:27confident. With this additional context
12:29that we gave to Claude, Claude now feels
12:31it's appropriate to say, "Vehicle B was
12:34at fault in this case." Based on this
12:36drawing and based on this sketch. So,
12:37already we're seeing some improvement in
12:39the way Claude is analyzing these. I
12:41think we could probably all agree if we
12:43looked at the drawing and at the list
12:45that vehicle B is at fault. Um so, we
12:47like to see that.
12:49Uh so, we're going to go back to the
12:51slides and talk about a couple of other
12:53items that we're not really using in
12:55this prompt, um but can be really
12:57helpful to building up building up your
13:00prompt and making it work better.
13:02Exactly. I think um one thing that we
13:04really highlight is examples. I think
13:06examples or few shot is a mechanism that
13:09really is powerful in steering Claude.
13:12So, you can imagine this um
13:15in in quite a non-trivial way as well.
13:16So, imagine you have scenarios,
13:18situations, even in this case, concrete
13:21accidents that happened that are
13:24um tricky for Claude to get right, but
13:25you with your human intuition and your
13:27human label data
13:29um is able to actually get to your right
13:32conclusion. Then you can bake that
13:33information into the system prompt
13:35itself by having clear-cut examples of
13:38A, the data that that it's supposed to
13:39look at. So, you can have visual
13:41examples. You can use base64 encode a
13:44a um
13:45an image and have that as part of the
13:47data you're passing along into the
13:48examples. And then, on top of that, you
13:50can have the sort of depiction or
13:51description, rather, of how to break
13:53that down and understand it. This is
13:55something we really highlight and and
13:56emphasize in how you can sort of push
13:59the limits of your LLM application is by
14:01baking in these examples into system
14:03prompt. And this again is sort of the
14:05empirical science of prompt engineering
14:06that you sort of always want to push the
14:08limits of your application and get that
14:10feedback loop in where it's going wrong
14:12and try to add that into system prompt
14:14so that next time when example that sort
14:16of mimics that
14:17um takes place, it's able to actually
14:19reference it in its example set.
14:22You can see here as well, this is just a
14:23little example of how we do this. Again,
14:26really emphasizing the sort of XML
14:28structure that we we um we enjoy. It It
14:31gives a lot of structure to Claude. It's
14:33what it's been fine-tuned on as well. Um
14:35and it works perfectly well for this
14:36example. And in our case, we're not
14:37doing this just because it's a simple
14:39demo, but you can realistically imagine
14:41if you were building this for an
14:42insurance company, you would have tens,
14:45maybe even hundreds of examples that are
14:46quite difficult, maybe in the gray, that
14:48you'd like to make sure that Claude
14:50actually has some basis in to make the
14:53verdict next time.
14:54Um another topic we really want to
14:56highlight, which we're not doing in this
14:57demo, is conversation history. It's in
14:59the same vein as examples. Uh we use
15:02this to make sure that there's enough
15:04context-rich information is at Claude's
15:06disposal when it when when Claude's
15:08working on on on your behalf. Um in our
15:12case now, this isn't really a
15:13user-facing LLM application. It's more
15:15something happening in the background.
15:17You can imagine for this insurance
15:18company, they have this automated
15:19system, some data is generated out of
15:21this, and then you might have a human in
15:23the loop at towards the end. If you were
15:25to build something much more user-facing
15:27where you'd have a long conversation
15:29history that would be um relevant to
15:31bring in, this is a perfect place in the
15:34system prompt to include because it
15:36enriches the context that Claude Claude
15:38works within. Um in our case, we haven't
15:41done so, but what we do is in the next
15:43step is try to make sure we give a
15:46concrete reminder of the task at hand.
15:50So, now we're going to build out the
15:51final part of this prompt for Claude,
15:53and that's coming back to the reminder
15:55of what the immediate task is and giving
15:57Claude a reminder about any important
15:59guidelines that we want it to follow.
16:01Some reasons that we may do this are A,
16:04preventing hallucinations. Um so, we
16:06want Claude to
16:08uh not invent details that it's not
16:10finding in this prompt, right? Or not
16:12finding in the data. If Claude can't
16:14tell which form is checked, we don't
16:16want Claude to take its best guess or
16:19invent the idea that a box might be
16:20checked when it's not. If the sketch is
16:23unintelligible, the person did a really
16:25bad job drawing this drawing and even a
16:27human would not be able to figure it
16:28out, we want Claude to be able to say
16:30that. And so, these are some of the
16:31things we'll include in this final
16:34reminder and kind of wrap-up step for
16:36Claude. Uh remind it to do things like
16:38answer only if it's very confident. We
16:39could even ask it to refer back to what
16:42it has seen in the form anytime it's
16:44making a factual claim. So, if it wants
16:45to say, "Vehicle B turned right," it
16:48should say, "I know this based on the
16:49fact that box two is clearly checked."
16:52Or whatever it might be. We can kind of
16:53give Claude some guidelines about that.
16:55So, if we go back to the console,
16:58we can see
17:00the next version of the prompt. And
17:03we're going to keep uh we're going to
17:04keep everything the same here in the
17:06system prompt. So, we're not changing
17:07any of that background context that we
17:09gave to Claude about the form, about how
17:11it's going to fill everything out. We're
17:12not changing anything else about the
17:13context and the role. We're just adding
17:16this detailed list of tasks. And this is
17:18how we want Claude to go about analyzing
17:20this. And a really key thing that we
17:22found here as we were building this demo
17:24and when we were working on the customer
17:26example is that the order in which
17:28Claude analyzes this information is very
17:29important. And this is analogous to the
17:32way you might think about doing this if
17:33you were a human. You would probably not
17:35look at the drawing first and try to
17:37understand what was going on, right?
17:39It's pretty unclear. It's a bunch of
17:40boxes and lines. We don't really know
17:43what that drawing is supposed to mean
17:44without any additional context. But if
17:46we have the form and we can read the
17:48form first and understand that we're
17:49talking about a car accident and that
17:51we're seeing some checkboxes that
17:53indicate what vehicles were doing at
17:54certain times, then we know a little bit
17:57more about how to understand what might
17:58be in the drawing. And so, that's the
18:00kind of detail that we're going to give
18:02Claude here is to say, "Hey, first go
18:04look at the form. Look at it very
18:05carefully. Make sure you can tell what
18:07boxes are checked. Make sure you're not
18:09missing anything here. Um make a list
18:11for yourself of what you see in that,
18:14and then move on to the sketch." So,
18:16after you've kind of confidently gotten
18:18information out of the form and you can
18:19say what's factually true, then you can
18:22go on
18:23and think about what you can gain from
18:26that sketch,
18:27keeping in mind your understanding of
18:29the accident so far. So, whatever you've
18:31learned from the form, you're trying to
18:32match that up with the sketch. And
18:34that's how you're going to arrive um at
18:36your final uh at your final assessment
18:38of the form.
18:41And we'll run it.
18:47And here you can see one behavior that
18:49this produced for Claude. Because I told
18:51it to very carefully examine the form,
18:53it's showing me its work as it does
18:55that. So, it's telling me each
18:57individual box, is the box checked? Is
18:59it not checked? And so, this is one
19:01thing you'll notice as you do prompt
19:03engineering. In our previous prompts, we
19:05were kind of letting Claude decide how
19:07much it wanted to tell us about what it
19:09saw on the form. Here, because I've told
19:11it carefully examine each and every box,
19:13it's very carefully examining each and
19:15every box. And that might not be what we
19:17want in the end. So, that's something we
19:19might change. Um but it's also going to
19:21give me these other things that I asked
19:22for in XML tags. So, a nice analysis of
19:25the form, the accident summary so far.
19:28It's going to give me a sketch analysis,
19:30and it's going to continue to say that
19:32vehicle B appears to be clearly at
19:34fault.
19:35In this In this example, it's pretty
19:36simple example. With more complicated
19:38drawings, more uh less clarity in the
19:41forms, this kind of step-by-step
19:43thinking for Claude is really impactful
19:45in its ability to make a correct
19:47assessment here.
19:49Uh so, I I we'll go back to the slides,
19:51and Christian's going to talk about the
19:53last kind of piece that we might add to
19:55this to really make it useful for a
19:57real-world task. Indeed, thank you so
20:00much. So, as Hannah mentioned, we sort
20:03of set the stage in this prompt to make
20:05sure that Claude's really acting on our
20:07behalf in the right manner.
20:09And a key step that we also add towards
20:11the end of this prompt, which I'm going
20:12to show you in a second, is a simple
20:14sort of guidelines or reminder part as
20:16well. So, just strengthening and
20:18reinforcing exactly what we want to get
20:19out of it. And one important piece is
20:21actually output formatting. You can
20:23imagine if you're a data engineer
20:24working on this LLM application, all
20:27this sort of fancy preamble is great,
20:29but at the end of the day, you want your
20:30piece of information to to be stored in,
20:33let's say, your SQL database, wherever
20:34you want to store that data, and the
20:36rest of it that is necessary for Claude
20:38to sort of give its verdict isn't really
20:40that necessary for your application. You
20:42want the nitty-gritty information for
20:44your application. So, if we transition
20:46back to console, you'll see here that we
20:49just added a simple important guidelines
20:51part. And again, this is just
20:53reinforcing the sort of mechanical
20:56behavior that we want out of Claude
20:57here. Want to make sure that the summary
20:59is clear, concise, and accurate. Want to
21:01make sure that nothing is sort of
21:03impeding in in in Claude's assessment
21:06apart from the data it's analyzing. And
21:08then finally, when it comes to output
21:09formatting, in my case here, I'm just
21:11going to ask Claude to wrap its final
21:13verdict. All other stuff I'm actually
21:15going to ignore for my application and
21:16just look at what it's actually
21:17assessing. And that is I can I can use
21:19this
21:20if I want to build some sort of
21:22analytics tool afterwards as well, or if
21:24I just want to get a cut
21:26determination, this is a way I can do
21:28so. So, if I just run this here, you'll
21:30see it's going through the same sort of
21:32process that we've seen before. In this
21:33case, it's much more succinct because
21:35we've asked it to be to summarize its
21:37findings in a a much more
21:38straightforward manner. And then
21:39finally, towards the end, you'll see
21:41that it'll wrap my output in these final
21:44verdict XML tags. So, you can see that
21:46during this demo, we've gone from a
21:48skiing accident to sort of unconfident,
21:52insecure outputs from perhaps a car
21:55accident in the second version to now a
21:57much more strictly formatted, confident
22:00output that we can actually build an LLM
22:02application around and actually help,
22:05you know, a real-world
22:08car insurance company, for example.
22:10And finally, if we transition back to
22:12the slides, another key way of shaping
22:17Claude's output is actually putting
22:19words in Claude's mouth, or as we call
22:22it, pre-filled responses. You can
22:24imagine that parsing XML tags is nice
22:26and all, but maybe you want a structured
22:28JSON output to make sure that it's JSON
22:31serializable and you can use this in a
22:34subsequent subsequent call, for example.
22:36This is quite simple to do. You can just
22:38add that
22:40Claude needs to begin its output with a
22:42certain format. This could be, for
22:44example, a
22:45open square bracket squarely bracket,
22:47for example, or even in this case that
22:49we see in front of us, this would be an
22:51XML tag for itinerary. In our case, it
22:53could also be that final verdict XML
22:54tag.
22:55And this is just a great way of again
22:57shaping how Claude is supposed to
23:00respond
23:02without all the preamble if you don't
23:03want that, even though that is also key
23:05in shaping its output to make sure that
23:07Claude is reasoning through the steps
23:08that we wanted. So, in our case here, we
23:11would just wrap it in the final verdict
23:12and then parse it afterwards. But you
23:14can use pre-fill as well.
23:16Now, finally, one step that I would like
23:19to highlight here as well is that both
23:21Claude 3.7 and especially Claude 4, of
23:23course, is
23:25has a hybrid reasoning model, meaning
23:26that there's extended thinking at your
23:28disposal.
23:29And this is something we want to
23:30highlight because you can use extended
23:33thinking as a crutch for your prompt
23:35engineering. Basically, you can enable
23:37this to make sure that Claude actually
23:38has time to think. It adds these
23:39thinking tags and the scratchpad.
23:42And the beauty of that is that you can
23:43actually analyze that transcript to
23:44understand how Claude is going about
23:46that data. So, as we mentioned, we have
23:48these checkboxes where it goes through
23:50step by step of the scenario that
23:52transpired for the accident. And in many
23:54ways there, you can actually try to help
23:56Claude in building this into the system
23:58prompt itself. It's not only more token
24:00efficient, but it's a good way of
24:01understanding how these intelligent
24:03models that don't have our intuition
24:05actually go about the data that we
24:07provide them. And because of that, it's
24:09quite key in actually trying to break
24:10down how your system prompt can get a
24:12lot better.
24:13And with that said, I think I'd like to
24:16thank all of you for coming today. We'll
24:18be around as well, so if you have any
24:19questions on prompting, please please go
24:21ahead. I know there's a prompting
24:23>> You want to learn more about prompting?
24:25In an hour, we have prompting for
24:26agents. And right now, we have an
24:28amazing demo of Claude plays Pokémon, so
24:31don't go anywhere for that. And as
24:33Christian said, we'll be around all day,
24:34so I know we didn't have time for Q&A in
24:36this session, but please come find us if
24:38you want to chat. And thank you guys for
24:39coming. Thank you so much.
24:42>> [applause]