Full transcript
0:00In this video, we're going to be
0:01reviewing GPT-2 which a lot of places on
0:04the internet are calling the best AI
0:07detector out there. We're going to
0:09actually see if that's true with a
0:10number of different tests that I'm going
0:11to run and just review it generally for
0:14pricing and other features besides AI
0:17detection to see what is, you know, is
0:19it a good service? Is it worth the
0:21money? All of that. Now, we all know at
0:23this point if you've been following this
0:24channel that AI detectors are
0:26notoriously bad, but I do kind of have a
0:29reason for wanting to go through and
0:30find which ones are the best ones
0:32because I'm currently building systems
0:34to improve on AI writing in such a way
0:36that it also humanizes it and I want to
0:39find the best AI detector so that I can
0:41properly measure do some of these
0:44automations that I'm creating make a
0:45difference. And GPT-2 is definitely one
0:48that I heard about a lot. You'll hear it
0:50a lot if you search for like best AI
0:52detector out there. It's one of the most
0:54commonly cited ones. So, I thought I
0:56would give it a shot.
1:01So, let's take a look. This is their
1:03home page here and you can actually get
1:05some free AI detection here just right
1:08here in their little window here if you
1:09wanted to test it out. This, however,
1:11will be their less advanced AI detector.
1:15It's not going to be as good as the one
1:17you get when you pay for it. But, let's
1:18look at pricing and there's a couple of
1:20things here. Now, one thing I do like
1:22about GPT-2 is it does have a free
1:24option which allows you to look at
1:2610,000 words per month. So, if you're
1:28just testing this out to see you know,
1:30does it work for my use case or you
1:33barely use an AI detector but
1:35occasionally need to, this is something
1:37that you might want to look at. This
1:38will give you a basic scan, but it does
1:40allow for three free advanced scans per
1:42month. Now, 10,000 words isn't a whole
1:44lot. You'll get through that very
1:45quickly. In fact, just for the test that
1:47I ran for this video, I quickly
1:48surpassed that. So, if that is you, then
1:51you might need to go up to the $12.99
1:53per month that's billed annually and
1:56it's $23.99 when billed monthly. This,
1:58however, gives you 3,000 words per
2:00month, which is a lot closer to what you
2:02would probably need. You get the
2:04advanced scan for all of that. You get
2:05multilingual AI detection. So, if you're
2:08writing in a different language and you
2:09want to detect AI use in different
2:11languages, you get that. And you can
2:13download these AI reports. For the
2:15professional, you get 500,000 words per
2:17month plus everything else in the
2:19premium and then a couple of other
2:21things like scan up 250 files at once.
2:23So, if you have any sort of bulk
2:25scanning needs, you might need this one
2:27for that. But for most people, you're
2:29either going to occasionally use the
2:30free version here or if you really need
2:33something a little bit more substantial,
2:35you get the subscription. Unfortunately,
2:37I don't like subscriptions for this. I
2:38recently reviewed originality.ai. One of
2:41the things I liked about that one is
2:42that they have a pay-as-you-go system,
2:45ala carte. And that's nice because I can
2:47just like pop 20 bucks in there and be
2:49like, "Okay, I'll just use this when I
2:50use this." And then, you know, that way
2:52I don't have to worry about did I go a
2:54whole month without even using this tool
2:56at all? And you don't have to worry
2:57about monthly recurring expenditures
3:00adding up along with all of your other
3:03author tools and Netflix and Disney Plus
3:05and all of these things. So, that's one
3:07of the downsides, I think, of GPT-3 is
3:09that there is no ala carte option like
3:10there is with originality.ai.
3:12Still, $12.99 I'd say is probably a
3:15reasonable amount for what this does. Of
3:17course, you have to get the annual
3:18version to do that, a little over $130.
3:22So, that'd probably be about $150 or so.
3:25I'm just doing some quick math in my
3:26head. All right, so that's GPT-3. So,
3:28what does it look like when we actually
3:30log in to GPT-3, especially if you have
3:32a advanced account? Well, it's going to
3:34look something like this. You can come
3:36here if you have multi-file upload
3:39thing, you can do that here. But most of
3:41the time, you're just going to go here
3:42and say new scan and then paste your
3:44document in here or you can upload it
3:46from Google Docs or just upload a Docs
3:48file or something like that, which is
3:50nice to have all of those different
3:51options to upload your documents,
3:54especially if you have really long
3:55documents like you have a whole book or
3:57something that you want to check, you
3:58could do that. Of course, that's going
3:59to use up your credits real fast, but
4:01you can do that. Anyway, you just paste
4:03your thing here and then you select what
4:04kind of scans you want to do. And one of
4:07the things I like about GPT-3 is they do
4:08have a lot of options here just besides
4:11AI detection. Of course, they have their
4:13advanced AI scan, which is what I was
4:15using for all of all of my tests. They
4:17can also text for plagiarism checking,
4:20for AI hallucinations. It could be
4:21really useful if you want to like go
4:24through and check the claims that are
4:25made. And this will actually, I think,
4:27also help you find sources, which is
4:29nice. You can even ask it to do this to
4:31give you writing feedback to help you,
4:34the human, to be able to incorporate
4:36feedback and improve your text, which I
4:37thought was also a really nice thing. Uh
4:39there's also this create custom reviewer
4:41option, which I thought it was
4:42interesting, but I'll get to that in a
4:44little bit later. Quick pause. If you're
4:45an author who wants to start writing
4:47more books, I put together my complete
4:49AI prompt pack that helped me and many
4:51others in my community write a book in a
4:53month. It's completely free. Hit the
4:54link in the description, join the free
4:56wait list for Story Hacker Gold, then
4:58you just answer a couple quick
4:59questions, and I'll send you the full
5:01prompt list plus a few extra free
5:03goodies. I'll also email you when I open
5:05the paid Story Hacker Gold group so you
5:07can join if it feels right. There's no
5:09commitment to join though, and you'll
5:10still be able to get all of the freebies
5:12you signed up for either way. All right,
5:14back to the video. Anyway, actually no,
5:16let's get into it now. So, you can get
5:17it here on the left here, you can go to
5:19custom review, and this allows you to
5:22create essentially a feedback avatar.
5:26So, let's say, you know, a lot of people
5:27using GPT-3 are instructors and
5:29teachers. So, let's say we want them to
5:32have like a an emphasis like a teacher
5:35with an emphasis in American literature
5:36or something like that. Then you can put
5:37that in here. You can attach files, so
5:40it has things like rubrics, score
5:42sheets, criteria, writing it so it could
5:44know what the writing assignment is. All
5:46of this stuff very useful, I think, if
5:47you're an educator, because basically
5:49what this does is it allows AI to grade
5:53papers.
5:54To grade stuff that was given to you by
5:57your students. And I think it's kind of
5:59ironic because this tool seems to be
6:01really targeted at educators. I think
6:03it's kind of ironic that we have a tool
6:05that is designed to detect if a student
6:08has used AI in their homework, while
6:10also offering the educator a chance to
6:13use AI for their homework, which is to
6:15grade papers and other documents, other
6:17assignments. Because you can do that
6:19with this tool. You can absolutely take
6:21all of your stuff, if you're a teacher,
6:23plop it in here, and let it do the
6:25grading for you. Now, I actually don't
6:26think that's a bad thing. If you have a
6:28really good rubric and the assignments
6:30are very well spelled out, I think I
6:32would actually want the AI to do the
6:35grading instead of a human most of the
6:37time. I would trust an AI to do that
6:39better. The reason is, first of all, it
6:41has to have a good rubric and clearly
6:44defined goals and criteria. But, here's
6:47the thing. Humans, especially when it
6:48comes to something like grading papers,
6:50as someone who uh was an English major
6:52and had a lot of papers criticized by a
6:54lot of different teachers, it's so easy
6:57for like every teacher is different. I
6:59could submit the same quality of work to
7:00a bunch of different teachers, and it'd
7:02all be different. And, you know, and
7:04sometimes teachers have bad days,
7:05they're really tired, they've got a lot
7:07on their plate, maybe they're just
7:08having a bad day and they're extra
7:10critical that day. I actually think, and
7:12I know I'm getting kind of philosophical
7:13here, but I actually think AI would be
7:15more objective. And if you have your
7:17rubric straight, they could actually be
7:19pretty accurate, too. And I think a
7:21teacher is better off focusing more on
7:23the teaching. I don't get much value out
7:25of a teacher grading my papers. I get
7:27more value out of the instruction that
7:28they give me, uh either face-to-face in
7:30a classroom setting or one-on-one. Like,
7:32that's the kind of thing that is
7:35meaningful, like mentorship from this
7:37person. But, the actual day-to-day
7:39grading of stuff, I think AI's fine for
7:41that. Anyway, I'll get off my
7:42philosophical soapbox there, but I just
7:45think it is kind of ironic that we have
7:46this tool that will help you grade
7:47papers while also making sure that the
7:50students aren't using AI because of
7:51course that's wrong. Anyway, let's go
7:53into the actual detection and results
7:57from all of the different prompts that I
7:58gave it. So, I gave a bunch of different
8:00AI and human samples to kind of see how
8:03well it does cuz I want to look at does
8:05it detect AI well enough? Does it have
8:07false positives? That sort of thing. And
8:10so, first of all, I had gave a pure AI
8:12fiction piece and it came back at 66%
8:15likely AI. So, or at least 66% of it is
8:18likely AI. This is inaccurate. The whole
8:21thing was 100% AI, so you would expect
8:23it to be 100% AI. Interestingly, over
8:26here I have a more heavily edited work
8:29of AI. So, a chapter that was completely
8:31written with AI, but then went through
8:32several editing passes with more than
8:34one human where almost every sentence
8:37was modified in some way by one of the
8:40humans, right? That one it thought was
8:4287% AI. So, I think it's funny that the
8:44100% AI was 66 and the heavily edited AI
8:48was 87. This is consistent although with
8:51what I got out of originality.ai which
8:53was 64% for the pure AI and 100% for the
8:56heavily edited AI which was surprising
8:58to me. This one is more accurate because
9:00it got a little bit higher on the pure
9:02AI and a little bit lower on the heavily
9:04edited AI cuz I'd have expect more like
9:0650% or so for the heavily edited AI. We
9:09had a pure AI a nonfiction piece and
9:12this one it got a little bit more
9:13accurate. It was at about 95% was its
9:16estimation. And the thing I've found
9:18already a bit with these two tests that
9:20I've run so far with originality.ai and
9:22GPT-2 is that nonfiction seems to get
9:25flagged more often as being AI. Because
9:27let's move on to this next couple here.
9:29I have a human written piece, completely
9:32human written and it flagged none of it
9:34as AI. So, it got this part correct.
9:36That that part was not AI. Then I had
9:38human-written nonfiction that was
9:40completely human-written. I know for
9:42sure because I wrote the article in
9:44question here, and I wrote it the
9:45traditional way before AI came around.
9:48And GPT-2 flagged that as 53% AI, which
9:52is an improvement on originality.ai,
9:55which flagged it as 99% AI. But still, I
9:58would have expected zero because there
10:00was zero AI in the formation of this
10:02particular article. So, while it did
10:04better here than originality.ai,
10:06it's still way off for nonfiction as
10:09opposed to fiction, which it generally
10:10got. And then I gave it some Sherlock
10:12Holmes text right here. That's what this
10:14column is. And then also a segment from
10:17the Bible. And both of those it
10:19correctly got that there was zero AI
10:21involved here. So, overall, I think this
10:23is actually pretty good so far. I've
10:26only reviewed two of these, but so far
10:28GPT-2 is does appear to be the more
10:30accurate of the two across several of
10:32these benchmarks. And I'm going to go
10:34ahead and add more of these as we go
10:36along with these reviews. But as far as
10:38my overall thoughts on GPT-2, if you're
10:40an educator, it seems to be a good bet.
10:42Things I don't like about it is I don't
10:44really care for the pricing structure.
10:46Although it is fairly standard, I still
10:48don't care for it. I think if you're not
10:50an instructor, not a teacher, it lacks
10:52some features that I would want.
10:54Originality.ai, for example, had more
10:56features leaning towards those who are
10:58SEO content creators and that kind of
11:00thing, online articles. And some of
11:02those would be really handy to have
11:03here. But at least it does check for
11:05hallucinations and plagiarism. It gives
11:07you writing feedback. It has the AI
11:09scanning, of course. And all of that.
11:10Obviously, the most important thing is
11:12how accurate is the scanner. And it
11:13seems to be pretty good so far. Although
11:16still certainly not perfect as I think
11:19we can expect for all of these. So,
11:21those are my thoughts on GPT-2. Let me
11:23know your thoughts working with this in
11:24the comments if you've worked with GPT-2
11:26before. And I will see you in the next
11:28video.