Full transcript
0:00We've got another cool, brand-new model
0:02to test. All of these AI companies seem
0:05to be really competing with each other,
0:07trying to one-up each other at every
0:09turn, and the only winners are those of
0:11us that actually use the technology that
0:12they are putting out there because of
0:14this rapid rate of improvement. Now, I
0:17recently did a video about Claude Opus
0:194.6. I found it to be a marginal
0:22improvement, but actually maybe not even
0:24that, maybe just more or less the same
0:26as Claude Opus 4.5. But today, we're
0:29talking about Claude Sonnet 4.6, which
0:32is the next upgrade for the Sonnet
0:34model. Now, in the past, back in the
0:36Claude 3 days, Claude Sonnet ended up
0:38eventually passing Opus as being the
0:40best model, especially once we got to
0:42Claude Sonnet 3.7, which was a fantastic
0:44model at its time. It did some things
0:47I'd never seen an AI model do before. Of
0:49course, now many models can do what that
0:51model did, but at the time it was really
0:54awesome. And so, I do think that some of
0:56these little minor improvements that we
0:58see to these models can actually have a
1:00big effect. Today's video, we're going
1:02to go through my standard test prompts
1:04for Sonnet 4.6. And uh because we've
1:08just done Opus 4.6, I think it's best
1:11that we do a a small comparison between
1:13the two, just to see is 4.6 at the same
1:16level as Opus 4.6. And uh if so, you
1:19know, that's great because it means that
1:20we could have a much cheaper model.
1:25>> [snorts]
1:27>> So, let's start with the specs. First of
1:28all, Claude Sonnet 4.6 is the same input
1:31and output price as the previous
1:33versions of Claude Sonnet, which is $3
1:35per million tokens of input and $15 per
1:37million tokens of output. So, this is
1:40great because it's uh it's a kind of
1:42that's in the standard mid-tier range
1:44for as far as price goes. It's not that
1:47expensive, not that inexpensive. Uh you
1:49can get a lot in it a lot more
1:50inexpensive than this, but you know,
1:52it's pretty standard middle-of-the-road
1:54price. And definitely a lot cheaper than
1:56Claude Opus, which I believe Yeah, Opus
2:00is a little bit more at $5 per million
2:02tokens of input and $25 per million
2:04tokens of output. Not as big of a
2:06difference compared to past versions of
2:09Opus, but uh still more expensive than
2:12Sonnet. All right, the other big thing
2:14about Claude Sonnet 4.6 is is the fact
2:16that it now has a 1 million token
2:18context window. But, if you recall my
2:21last video on Opus 4.6, this is not
2:24necessarily as impressive as people
2:25think it is. Just being able to
2:27theoretically, technically contain that
2:30many tokens in its prompts is not
2:32necessarily mean that it's good at
2:34retrieving all of the information in
2:36there. I've put like entire books into
2:39these models and asked it to do a
2:41chapter-by-chapter summary, and it can't
2:43do it. It'll give me like maybe 12 out
2:46of the chapters that I put there, but
2:47it'll skip chapters, it'll miss some, it
2:49won't get all the way to the end, you
2:51know, it'll do stuff like that because
2:52it's not really able to cohesively hold
2:56all of that information at one time.
2:57Models are getting better at this thing,
2:59so we might see that change in the
3:02future, but so far the context window is
3:04not really as impressive to me as how
3:07well it does with a small amount of
3:09context, and I generally tell people not
3:11to let their context get a little higher
3:13than 30,000 or so. If you're getting
3:15higher than that, you have a
3:16watered-down prompt, it's not actually
3:18going to perform well, and you'll be
3:19spending more. If you summarize a lot of
3:22your content that you're putting into
3:25your
3:26prompts, the context should say,
3:28that's going to perform a lot better,
3:30generally speaking, and it'll be
3:31cheaper. Anyway, that's my little high
3:32horse there. I'll get off of that, and
3:34let's actually go and start looking
3:36through some of these prompts. Now, I
3:38intentionally give really simple prompts
3:41for most of these. Couple of people can
3:43have come up to me recently and said,
3:44"Why don't you like give it samples of
3:46your own writing and ask it to do all of
3:48this and stuff?" And I'm like, "Of
3:49course, I do that when I'm actually
3:51writing with these tools for real. But
3:53for these prompts, I want to see how
3:55well does it do right out of the gate
3:57without any of that super prompting
4:00techniques that we use.
4:02Like if I say it right a few ad
4:04headlines for this, I know I could give
4:06it examples of high-performing ads and
4:08the output that I will get is going to
4:10be way better. But if I just say, "Give
4:12me 20 ad headlines for this with no
4:14other context." and then it also gives
4:16me amazing ads, then I know that the
4:18model itself is much more powerful. So,
4:21that's just my my thoughts on that and
4:23the reason I do these prompts really
4:24simply. And this first one is the
4:25logline prompt where I ask it to give me
4:2820 loglines for an epic fantasy series.
4:30And back when I did Claude Opus 4.6,
4:33there were a few that I actually thought
4:34were really good. And looking through
4:35this list, none of them are bad per se,
4:38like they will all create a they're all
4:41kind of a cohesive story, they all make
4:42sense. I'm still seeing a lot of the
4:45same tropes coming up over and over
4:47again, like the cartographer here is
4:50always there, a disgraced knight is
4:52always there. But these aren't
4:54necessarily bad given the fact that I
4:56gave it no context. If you give it more
4:58context, then it'll give you better
4:59ideas. But they are kind of generic for
5:01what I gave it. But there are a few that
5:03are pretty good, like this one here, a
5:04young grave digger grave digger who can
5:07raise the dead must build an undead army
5:09to overthrow a tyrannical emperor, but
5:11every soldier she resurrects returns
5:12with a singular desire to kill her.
5:14Yeah, that's not a bad a bad one right
5:17there. Kind of simple, you obviously,
5:18especially for epic fantasy, you would
5:20need to pull in a whole lot more ideas
5:21to make it a fully fleshed out thing,
5:23but it could be a good place to start.
5:25All right, the next big prompt is the
5:27outline prompt. This is where I give it
5:28a an outline template and just and then
5:31like a one paragraph description of a
5:34book that I want to write, a concept,
5:35and then let it run wild with those two
5:37elements. First of all, it gave me a
5:39couple of things that I thought were
5:41really interesting. It gives me these
5:43core story elements, like a reference
5:45sheet, which I didn't ask it for, and I
5:49sometimes like when AI gives me things
5:51that I didn't ask it for. Sometimes I
5:52don't though, because obviously if it
5:54does that, that means that it was it's
5:57maybe not following the prompt as well.
5:59But in this case, I thought this was a
6:00really good introduction to bring
6:03together just to kind of keep it in its
6:05own memory as it's writing the rest of
6:07the thing. Hopefully that results in a
6:09more cohesive outline. And looking at
6:11this first one here, here's the
6:13prologue. This is a pretty length
6:15description of a prologue, and looking
6:17through it actually is really
6:18interesting. It's a little bit less
6:19generic than I'm used to seeing, so
6:21that's good. A lot of times these
6:22prologues can be very vague, like, "Oh,
6:25we see an image of such and such." But
6:27then it doesn't really there's no story
6:30to it, you know? This actually does tell
6:32a kind of isolated story that sets the
6:34tone of the story, but isn't really
6:36about the protagonist, uh which is what
6:38your prologue usually should be. Tells
6:40the story of this guy named Sarah Bohl,
6:43a middle-aged soldier in Thornhaven's
6:45gray livery running. He is clearly a
6:47trained fighter, but he is terrified. He
6:48has been expelled through Thornhaven's
6:50great gate. We see it sealed behind him
6:52with the sound like a mountain
6:53swallowing itself, and now he's alone in
6:55the darkness outside. And it kind of
6:56goes through his little brief story,
6:58which does not end well. And then in
7:00chapter one, we finally get to the
7:02actual protagonist, Latham Volgerous. I
7:05gave the first name here, but the
7:06Volgerous it made up. One of the things
7:08I often look for with this prompt is the
7:10save the cat moment, which uh I in my
7:13template tell it to do in chapter one.
7:15And usually it gives me a really like
7:17random save the cat moment that doesn't
7:19have anything to do with a plot. And
7:21this one is a little better, it's not
7:23the best. Uh neither was the one for
7:24Opus 4.6, but it's a little bit better,
7:27and I think uh cuz it does kind of show
7:30the save the cat moment in a way that's
7:31more consistent with the character
7:32compared to other past versions of this
7:34that I've seen, cuz the character is
7:36like this Eric and Prince kind of a
7:37thing, but then he has this moment where
7:39he gives some food to a hungry little
7:42girl. but the way it describes that it's
7:44it's more than just like a passing thing
7:46and it's much more consistent with his
7:47character, which I thought was good. I
7:50would probably still change this myself
7:52just to be something a little bit more
7:53interesting, but it is an improvement.
7:55And then fast-forwarding here a little
7:57bit to look at the inciting incident
7:59chapter, he this character is summoned
8:01before his father, but it's not a
8:03private meeting. It's a formal audience
8:05in a great hall with a full council
8:06present and an honor guard, and he's
8:09falsely accused of some things that he
8:10did and I like that it says here, "These
8:13charges are technically true, but they
8:15are framed as sedition." What I like
8:16about this, the reason I like this is
8:18that it's clearly drawing on what it
8:19wrote in the past chapters and it's a
8:22little bit more cohesive than sometimes
8:23AI does. A lot of times with this
8:25outlining prompt, the AI models will
8:27sort of it's almost like a separate
8:29story for each chapter that have a
8:31little bit of cohesiveness together, but
8:33a lot of it is very like episodic and
8:35stilted. This feels like it's a complete
8:38outline that it's connected together.
8:40And I'd have to go through and read the
8:41entire thing, which is let me check on
8:43how long this prompt goes for this
8:46outline goes for. All right, so this
8:48prompt
8:50was almost 18,000 words, which is a
8:52very, very, very, very long output. I
8:56think this is actually pretty good. Back
8:58when Claude Sonnet 3.7 was out, that
9:02model at its time was by far the best
9:05model for outlining. It was really,
9:07really good at it compared to other
9:08models of the time. I think Sonnet is
9:10actually still really, really good at
9:11outlines. I feel like it's almost
9:13slightly less creative than Opus, but as
9:17a result sticks to your stuff better
9:19than Opus. Opus can sometimes go in wild
9:21directions when you're outlining with
9:23it. Quick pause, if you're an author who
9:25wants to start writing more books, I put
9:27together my complete AI prompt pack that
9:29helped me and many others in my
9:30community write a book in a month. It's
9:32completely free. Hit the link in the
9:34description, join the free waitlist for
9:35Story Hacker Gold, then you just answer
9:37a couple quick questions and I'll send
9:39you the full prompt list plus a few
9:41extra free goodies. I'll also email you
9:44when I open the paid StoryHackers Gold
9:46group so you can join if it feels right.
9:48There's no commitment to join though and
9:49you'll still be able to get all of the
9:51freebies you signed up for either way.
9:52All right, back to the video. So, on the
9:54whole, I think this is actually a really
9:56good results. I'd say this is arguably a
9:58little bit better than what I got out of
10:004.6 Opus, which does do a few good
10:02things. I do feel like the prologue I
10:04got with Opus was it was good but it was
10:07also kind of the same sort of story that
10:09I've heard many times with this prompt.
10:11They're certainly not bad compared to
10:13each other. They're like neck and neck
10:14on this one. But given the slight
10:16inexpensiveness of Sonnet, I might go
10:18with Sonnet on this one really if I'm
10:20doing outlining tasks. So, just
10:22something to think on there. Let's move
10:24on to the basic prose prompt with both
10:26of these. So, this is where we're really
10:28going to compare between Opus 4.6 and
10:30Sonnet 4.6. This is just to see I do
10:33this exact scene every time that I do
10:35these tests. The reason I like doing an
10:37epic fantasy with this is because AI
10:39typically struggles with anything is set
10:41in a historical type time frame when
10:43compared to contemporary. AI is usually
10:45a lot better at contemporary stuff like
10:47a contemporary romance or an urban
10:48fantasy or something like that. You're
10:50much more likely to get good results in
10:51the prose out of the AI for those
10:53compared to something a little bit more
10:55ancient. So, if it can do this, I have
10:57high confidence that it can do other
10:59genres as well. Now, just looking
11:00through this, there's an interesting
11:02moment here that is not prompted but
11:04that is actually really, really good
11:06here where he says he had been he had
11:08been seven when he left Varloden with
11:10the expedition. Seven members that was.
11:11He counted them every night by habit as
11:13Overseer Merritt had insisted, "Count
11:15your people before you count your
11:16supplies." And he had counted them every
11:18night since lying in the dark and
11:20watching the numbers reduce themselves.
11:21Six, four, two, one. I like the emphasis
11:24on one. Haldir looked back. He knew
11:26better than to look. Every account,
11:27every training session, every greybeard
11:29at the society hall who had actually
11:30crossed the Ash Fell said the same
11:32thing. You look back and you slow down
11:34and you slow down and it's done. These
11:36again are little details I did not
11:37prompt it for which can be a good thing
11:39if they fit with your world. He looked
11:41back anyway because something in him
11:43needed to know how much time he had left
11:44to exist in. They were closer than he'd
11:46hoped, closer than they had been at
11:48midday when he'd been allowed to when
11:50he'd last allowed himself to count. Two
11:52dozen at least, perhaps more the edges,
11:53gray shapes swallowed by gray distance.
11:55They moved the way the hungry always
11:57moved, that wrong shuddering gait, limbs
11:59snapping forward
12:00jerks as though controlled by someone
12:01who had been told about walking and
12:03never seen it done. They made no sound
12:05he could hear over the wind. They never
12:06did. That was the worst of it somehow,
12:08the silence, the absolute indifference
12:10of their pursuit and as though consuming
12:11him was simply a thing that would
12:13happen, patient and inevitable as
12:15weather. I don't like that last phrase.
12:16That's a little AI sloppy but that was
12:19actually a pretty solid paragraph
12:20overall. If we compare that moment to
12:23the same part of the same prompt with
12:25Opus 4.6, scattered across the darkening
12:27horizon like stones come loose from the
12:29earth itself, gray shapes move against
12:31the gray land. At first glance, they
12:33might have been nothing, tricks of the
12:34fading light, outcrops of rock shifting
12:36in the heat haze that still clung to the
12:38ground. A rock did not move like that.
12:39Nothing living moved like that. They
12:41came in stuttering, broken surges,
12:43bodies jerking forward in spasms that
12:45covered ground with a horrible deceptive
12:46speed, limbs snapping into motion as if
12:48yanked by invisible strings then
12:50freezing then surging again. Two dozen
12:52at least spread in a loose and
12:53tightening arc across the waste behind
12:55him. Looking through this, I almost
12:57think that Sonnet 4.6 is better. I'm
13:00genuine like this is the first time I've
13:02read through this. I'm genuinely
13:03surprised right now. Now, you'd have to
13:05work with these things a lot more to
13:06kind of get a sense of their quirks but
13:08I do feel like there's a lot more like
13:10sloppy AI sloppy phrases and the kind of
13:14an over drama in the way that Opus 4.6
13:17explains this but yeah, I think this is
13:20actually pretty good. Let's try the next
13:22one which is the same scene but using a
13:25a advanced prompt that it does include
13:27more prompting
13:28examples of my style and cetera. Let's
13:30find that same moment. He looked back.
13:32They were there. Of course they were.
13:34Two dozen scattered across the shelf
13:35rock in a loose uneven line maybe 400
13:37yards behind him. The light was bad now
13:38and their gray skin swallowed what
13:40remained of it. They barely separated
13:41from the stone just shapes in the dusk
13:43that moved wrong. Their jerkiness was
13:45what always betrayed them first, not the
13:46stumble of something tired but a
13:47hitching stop-start and pull like
13:49puppets on a drunk man's string. Some of
13:51them had already fixed directions
13:53straight at him. Haldor turned back to
13:55Vardan and walked faster. All right, so
13:57looking at that now, I don't think this
13:59one is quite as good. So maybe we just
14:01got lucky on that first one. There are a
14:03couple of AI slopisms like just shapes
14:06in the dusk that moved wrong, you know,
14:07these kind of like overly dramatic
14:09sayings. Their gray skin swallowed what
14:12remained. But still not bad and still
14:14maybe better than what we were getting
14:15out of Opus. So that's interesting to
14:17me. All right, the next one is the
14:19dialogue prose template and this is one
14:21where I give it another scene, same
14:23story but one with more dialogue. So
14:25let's look at what Opus gave us first.
14:28Let's start here. A man died at the
14:30gates tonight, he said. His boots rang
14:32against the stone floor as he crossed
14:33the hall. Two of the king's guards
14:34shifted at his approach. Not
14:36threatening, not yet. Attentive in the
14:37way of men trained to read the weight of
14:39a footstep. All right, there was a lot
14:40of prose there. Like, I know, said
14:41Store, King of Vardan. You know? As
14:44Lathen stopped at the base of the dais,
14:45his jaw tight. Then you know the gates
14:47stay shut. You know we watched him die.
14:49That's the second time in a month,
14:50father. The second time we've left
14:52someone out there to be taken by the
14:53hungry while we stood behind our walls
14:55and did nothing. All right, so let's
14:57look at that same moment here inside of
14:59what Sonnet 4.6 gave us. They left him
15:01to die. Lathen's voice carried the full
15:03length of the room. He didn't soften it.
15:04Again, the king regarded him with a
15:06peculiar stillness he had perfected over
15:08decades. Not calm, Lathen knew, but the
15:09shape of calm more like armor. There's
15:12some AI slopisms in there. More like
15:14armor. A lot of metaphors and stuff like
15:16that. Around the throne, the usual
15:18constellation of advisors and courtiers
15:19shuffled and looked away. There were
15:21procedures. A man died screaming at your
15:23gate, Father. Laethan crossed the
15:25polished stone floor, each footfall
15:26deliberate. I heard him. Half the city
15:28heard him, and we did nothing. He
15:29stopped at the foot of the dais. This is
15:31the second time in a month. The law, you
15:33make the law. So, looking at this, I
15:35think this is close. This is a little
15:37bit of a tough one cuz it's actually
15:39pretty close. But, it's actually close
15:41enough that I wouldn't say that Sonnet
15:43is worse, especially for the price tag.
15:46The fact that this is almost half the
15:48price of Opus, not quite, but almost, is
15:51encouraging, I think. If we can get
15:53results these good this good for half
15:56the price of Opus, it might be worth it
15:57to me. Let's take a look at the Editing
15:59Pros template. So, this is where I give
16:01it the same scene that was written with
16:02a really poor AI model and ask it to
16:05rewrite it to improve the pros. The hall
16:07stretched before him, vaulted stone,
16:08cold torchlight, the smell of old smoke
16:10and older power. His father sat at its
16:11far end upon the ash wood throne,
16:13watching his son's approach with a
16:15peculiar stillness of a man who has long
16:16since learned to mistake patience for
16:18wisdom. Okay, that sounds very AI-ish
16:20written there. Father, Laethan stopped
16:22at the foot of the dais. We need to
16:23speak about the gate. Do we? It wasn't a
16:25question. I don't think that line was in
16:27the original, and I kind of like it. A
16:29man died last night. He died on our
16:31doorstep screaming our name while we
16:32stood on the walls and watched. The
16:34words tasted like ash in his mouth. Oh,
16:36man, the number of times I've heard of
16:37words tasting like ash. You were there.
16:40You heard him. King Storr's expression
16:41did not change. It really did. The law
16:42was upheld. The gates remained shut.
16:44Order was preserved. All right, so this
16:46is not great. This It's better than the
16:48original text that is used, but better
16:52to just rewrite the scene with new
16:54prompts than to try and improve on
16:57existing writing. All right, so the next
16:59one is add headlines prompt, and let's
17:02see what Opus gave us. It gave us things
17:05like he was told the world outside the
17:07mountain was dead. He was lied to about
17:09everything. A prince thrown out to die
17:11discovers the kingdom he loved was built
17:12on a lie. First of all, these are all
17:13kind of long, and looking at Sonnet
17:16here, these look pretty long as well.
17:18They told him the world outside the
17:19mountain would kill him. They were
17:20right, but not for the reasons he was
17:21warned about. A prince, a betrayal, and
17:23a world his father made very sure he
17:25would never survive long enough to
17:27understand. He thought the monsters
17:28outside the wall were the most dangerous
17:30things in the world, then he met the
17:31humans. All right, that's fun. So, yeah,
17:33I would say these are about the same as
17:35what Opus gave me, nothing particularly
17:37groundbreaking here. And they're all
17:38kind of long. Usually headlines are
17:40better shortened. Yeah, a lot of these
17:42are really long and definitely Uh uh one
17:45thing that it did do is it kind of
17:48grouped some of these together. As noted
17:50here, headlines 4, 7, and 12 create a
17:52mystery. 3, 11, and 19 flip reader
17:55expectations. So, that was a nice little
17:57touch, but overall I wouldn't say that
17:58that's a particularly better than Opus.
18:01All right, and then we have the email
18:02newsletter prompt. I have some exciting
18:04news to share. My newest book, The Crown
18:05of Hunger, is officially releasing on
18:06December 21st, and I couldn't be more
18:08thrilled to finally get this one into
18:10your hands. A little over dramatic. This
18:12story has been living in my head for a
18:13long time, and I poured everything into
18:15it. Yeah, I I wouldn't use this myself.
18:18If we look at what Opus gave me, "I
18:21honestly can't wait for you to read it.
18:22This one's been a labor of love." Yeah,
18:24this is not much better. It might be a
18:26little I think it is a little better,
18:27but still really over the top. Last but
18:29not least, we have the SEO article super
18:31prompt where I give it a whole bunch of
18:33other articles on the web on how to
18:35write fantasy and ask it to write an
18:37article that's 4,000 words. The main
18:39thing I'm looking for here is how well
18:41did it actually stick to the 4,000 words
18:43thing because usually that's that's a
18:45number I came to specifically because
18:47that's what I'd write. So, Opus wrote
18:484,500 words. So, it exceeded my target,
18:51and Sonnet wrote 3,200 words. So, it
18:56actually did not hit the mark, although
18:57that 3,200 is still pretty beefy. I'm
19:00I'm okay with that. Let's look through
19:01this. Okay, lots of dashes. Uh Claude is
19:05particularly bad at dashes. Of course,
19:07they're all bad at dashes. But looking
19:09through this, this looks okay. Well, a
19:12lot of the problems I see with other
19:14models is that they write it almost like
19:17a just more fleshed out outline where
19:19there's not actually a whole lot of
19:20substance talking about each element.
19:22And here there is, so that's good. Yeah,
19:24I think this is a decent article. So,
19:26I'd probably give the edge to Opus just
19:27because it did exceed the 4,000 words
19:29that I asked for, but this is fine as
19:31well, honestly. All right, that's it for
19:34Sonnet 4.6. If I had to give you my
19:37initial gut reaction to this, I think
19:39it's actually pretty solid. I I might
19:41even consider swapping it out in some of
19:42my automations for areas where I'm using
19:45Opus. It is certainly not free of all of
19:48the problems that we have with AI,
19:50different AI-isms, and such, but a lot
19:53of those can be filtered out with other
19:54prompts and stuff to get rid of all of
19:56those, and I'm working on an automation
19:58that will automatically do that for
19:59people, but it's nice to not have them
20:01in the first place and not have to
20:02filter them out in the first place. But,
20:03at the very least, we've got something
20:05that's pretty decent here, and given
20:07that it's almost half the price of Opus,
20:10I would strongly consider using it a
20:12lot. Do I think it's better than Gemini
20:133? I don't know. And as I'm recording
20:16this, Gemini 3.1 just came out, so now I
20:19have to do a video on that. You'll see
20:21that probably tomorrow as you're seeing
20:23this. As this video came out, I'll
20:25probably do the the Gemini 3.1 tomorrow
20:27cuz I really do like the Gemini 3 for
20:29writing, but I do seem to be in the
20:31minority for that as I've come to find
20:34from running some tests and some polls.
20:36So, don't listen to me. Go ahead and do
20:38your own tests. See what you discover
20:40about this model, about Opus. How does
20:41it compare to Opus? And let me know in
20:43the comments below what your experience
20:45has been. I'd love to see that, and I'll
20:46see you in the next video.