Full transcript
0:04My name is Lyanna Davis. I am the chief
0:07programs officer at Wiki Education and
0:11I'm really excited to have a great group
0:14of panelists with us today for this
0:16webinar on generative AI and or
0:21Wikipedia in the age of generative AI.
0:23Let me get that get that right. Um
0:26uh let me share a few notes before we
0:29begin here. So let me first talk about
0:32wiki education. So we run a series of
0:35programs connecting subject matter
0:37experts to Wikipedia in our flagship
0:41program. We run the Wikipedia student
0:44program where we engage college and
0:46university faculty who want to assign
0:49their students to write Wikipedia
0:51articles as part of the coursework. And
0:53you can learn more about that program at
0:57teach.wikedu.org.
0:59And my colleague Colleen will hopefully
1:02be dropping some links to these in the
1:04chat and once maybe the introductions
1:06are done so that we have uh some space
1:08to to see them as well. Um and then we
1:11run a second program that where we run
1:15courses teaching people how to edit
1:17Wikipedia in a particular content area.
1:21And we actually are recruiting for a
1:23number of those courses right now. So if
1:25you have ever wanted to learn how to
1:26edit Wikipedia yourself, I encourage you
1:29to visit learn.wikedu.org.
1:32And learn.wikedu.org
1:34is the homepage where all of the courses
1:37are being offered. And we've got ones
1:39right now on um uh politics, civics, and
1:43democracy ahead of the midterm elections
1:44in the United States. We've got one on
1:46American history. We've got one on
1:49nonprofits in the United States and then
1:50we also have one that's open to
1:52international participants around
1:54disability and health care. So I
1:56encourage you to sign up for one of
1:58those if you are interested in that. A
2:01general note, we will talk a lot today
2:04about generative AI and I want to
2:07acknowledge upfront that AI means more
2:10than chat bots like chat GPT. And so for
2:13the purposes of today's discussion, we
2:15are focusing on large language model LLM
2:18powered chat bots like chat GBT or
2:21claude copilot Gemini etc. Um and the
2:24panelists and I may use them and may use
2:26the term AI as sort of a shortcut to
2:29refer to these chat bots in the context
2:31of this discussion. But I do want to
2:33acknowledge there's much more to
2:34artificial intelligence than just text
2:36generation. And that's just what we're
2:38we're talking about as the focus of AI
2:40today.
2:41Um, so with that, I'm going to ask each
2:44of our panelists to introduce themselves
2:47and just share your name and where
2:50you're joining us from. And, uh, to make
2:53it a little more fun, please share the
2:55last Wikipedia article you read. Um, and
2:58so I am going to start with Oliver.
3:03>> Hi everyone. I'm Oliver W. I teach art
3:07history at Boston College. Um, the last
3:10Wikipedia article I read was early early
3:14this morning. I was looking at the
3:16article on Verazy's painting, The
3:19Wedding at Cana, which some people might
3:22know because it's in the same room at
3:24the Louv as the Mona Lisa, but it's the
3:26one on the other side of the wall. So,
3:28it's the the painting that probably more
3:31people have turned their backs to uh
3:33than any other painting in the world,
3:34but it's a pretty cool painting in its
3:37own right.
3:39Looking forward to the conversation.
3:41>> Great. Thanks, Oliver. Next up, Whitney.
3:45>> Hello. My name is Whitney Lou James and
3:47I teach first year writing at Notre
3:49Dame. Um, and the last Wikipedia article
3:52I was looking at was about trades. I got
3:53into a little bit of a Wikipedia um
3:56wormhole there.
3:59Okay, Sage.
4:02I'm Sage Ross. I'm the chief technology
4:05officer at Wiki Education and I'm
4:07calling in from Seattle and the last
4:10Wikipedia article that I read was uh loo
4:14prunis bernomi uh which is a cute little
4:18yellow mushroom that loves to travel
4:20around the world in uh tropical soil
4:24potting mix.
4:26>> Excellent. Thanks, Sage. And last but
4:28not least, Max.
4:30>> Everyone, I'm Max. I'm the CEO and
4:33co-founder of Pangram Labs. We do uh AI
4:36generated content detection. I'm sure
4:38I'll be telling you more about this now
4:40or later. Uh I am calling in from a
4:43hotel in San Francisco and the most
4:48recent Wikipedia article I read was this
4:51list of highest grossing musical theater
4:53productions. Uh it's The Lion King by a
4:56lot.
4:59>> That's great. I personally have enjoyed
5:01watching the Lion King. So [laughter]
5:03that's that's good to know. Good to
5:04know. Well, thank you everyone. I'm I'm
5:06really glad to to have you here today.
5:09Um so I want to start us off by asking a
5:12series of questions of our panelists. I
5:14encourage panelists to jump in and
5:17answer questions. Just because I'm
5:18asking one to somebody doesn't mean that
5:20other folks um haven't a great uh answer
5:24for that question as well. Um, and then
5:27we'll have time at the end for Q&A. So,
5:30for those of you who are just joining
5:31us, please add those questions into the
5:33Q&A box on Zoom and not the chat. Um, so
5:36I'm going to start with Sage as um as
5:38our our first uh question here. So,
5:42Sage, you're both Wiki Education CTO,
5:44but you're also a longtime Wikipedia
5:46editor yourself. Uh, drawing from your
5:49experience, how has AI generated text
5:51changed how Wikipedia works?
5:54Yeah. So, uh, with my Wikipedia editor
5:57hat on, I would say that like the
6:00biggest thing in broadstrokes, I really
6:03had to keep myself from writing too many
6:05essays, um, about this topic. But um, I
6:08think the undermining of trust is kind
6:11of the the biggest thing that is
6:13affecting Wikipedia with the advent of
6:16so much easy LLM text that can put onto
6:20it. So like Wikipedia is built around
6:23the idea of assuming good faith but not
6:26necessarily assuming competence. So if
6:29somebody shows up, we typically assume
6:30that they're here because they want to
6:33help the project of building the
6:34encyclopedia. And um traditionally
6:38someone who writes about a given topic
6:41um that's kind of evidence that they one
6:44care enough about that topic to learn
6:46something about it that wasn't already
6:47on Wikipedia and two that they've taken
6:51enough time to sort of think about it
6:52and process it into expressing their
6:55their ideas and what they know in
6:56writing. Um, and so if they don't
6:58actually know that much about it, um, it
7:01was usually obvious. Uh, if they didn't
7:03take time to express what they know, uh,
7:06in, you know, like understandable terms,
7:08that would also be obvious. So, as a
7:10Wikipedia editor, if I see something
7:12that I know a little bit about, um, it's
7:14pretty easy for me to get a sense for
7:16whether the person writing it knows what
7:18they're doing. um if it makes sense in
7:21broad strokes, if it uses the kind of
7:23language that signals familiarity with
7:25that topic, that's a strong signal that
7:27says, "Hey, unless I see something else
7:29that points in another direction, I can
7:32probably trust that this editor uh you
7:35know, like is here for the right reasons
7:37and um knows something about what
7:39they're doing." Um and so AI just does
7:42such a good job at making plausible
7:44text. um it does make sense in
7:46broadstrokes often and it does use that
7:49same kind of language that implies
7:50familiarity with the topic. So it really
7:53breaks down those signals that Wikipedia
7:56editors rely on to know how much trust
7:59to put in uh another person's writing.
8:01So it also kind of like throws out that
8:05uh default assumption of good faith um
8:08in many cases because it reduces the
8:11cost uh for sort of bad actors. So uh in
8:15the past you know it's always been a
8:17important target for people doing
8:19various kinds of trying to affect the
8:22information landscape to to get involved
8:24with Wikipedia. But one uh learning how
8:28to write in Wikipedia's style the kind
8:30of neutral point of view just the facts
8:33that kind of thing that's been a
8:34non-trivial cost for a lot of people
8:36looking to do like self-promotion
8:39marketing electioneering PR and so on.
8:42Um, and uh, you largely don't have to
8:45pay that cost if you're using AI because
8:48it can it can do a plausible job of kind
8:50of capturing uh, encyclopedia tone. Um,
8:54and then it also makes possible to do
8:56sort of more subtle manipulations at
8:58scale. So um, whereas if you had to
9:01write everything that you were trying to
9:02get onto Wikipedia, there was a high
9:04investment cost and when an editor said,
9:06"Hey, no, no, you can't do that." and
9:08undid it. Um, you've sort of lost what
9:11you invested there. Um, AI lets you do
9:14things at a much higher scale where you
9:16just throw things at the wall and and
9:18some of it may stick. Um, and so it also
9:21makes possible and kind of incentivizes
9:24um, more subtle manipulations
9:27um, in part because of the way that like
9:31AI is kind of displacing traditional
9:33search. And we see that AI responses
9:37often pick up um new edits to Wikipedia,
9:42Reddit, other things like that very
9:44quickly. Um and so it's really become uh
9:46much more of an SEO target, even if it's
9:49not the kind of main topic at hand. So
9:51if you kind of insert the little fact
9:53that you're trying to get out into uh
9:56the the sort of conversation or the
9:57ecosystem in a tangentally related minor
10:00thing that no one would be likely to
10:02read anytime soon that can still get in
10:05front of people via um kind of chat
10:09driven or chatbot driven search.
10:13>> Thanks Sage. Yeah, there was a recent
10:15New York Times article that uh talked
10:17about the a time is 12 minutes from the
10:20point when a fact gets inserted into
10:22Wikipedia and when it starts being
10:24regurgitated by chat bots um when you
10:27ask a question which is a pretty
10:29incredible sort of turnaround time
10:30there. Um, so, so Sage, taking off your
10:33Wikipedia editor hat, uh, literally and
10:36figuratively here. Um, can you share a
10:39little bit more about how we at Wiki
10:41Education have addressed the rise of AI
10:43usage among our program participants?
10:46>> Sure. Um, since uh, last summer, this
10:50has really been a big focus for us. Um,
10:54and
10:55we have kind of two prongs of it. and
10:57one of them is aimed at sort of the the
11:00pedagogical side. Uh so we've developed
11:04um several training modules that are now
11:07the default part of what students go
11:09through before they start editing
11:12Wikipedia. Um that tries to do as good
11:15of a job as we can of giving them
11:18context about hey how do LLMs really
11:21work? Um what you know what are their
11:23failure modes? why like how do they
11:26behave when you try to write
11:28encyclopedia articles with them? Um the
11:30short answer is that they often do very
11:33poorly at that. Um even though the text
11:35seems so plausible. Um, and so like
11:38giving them giving students the kind of
11:40intellectual tools and perspective to
11:43not take what they get out of a chatbot
11:45at face value, but but think about it
11:47critically in the way that an
11:49encyclopedia editor who's really focused
11:51on, hey, I want to establish what is
11:54real, what are the facts, what what do
11:56sources say about this? um as sort of a
11:58baseline, equip them to approach what
12:01comes out of an LLM the way that we
12:03think uh a sort of savvy Wikipedia
12:06editor ought to. Um and then the other
12:09side of it um is that we've implemented
12:13um AI detection. Uh so we um did kind of
12:18a survey of hey what's the AI detection
12:21landscape and is it useful enough to
12:24actually put into practice? Um, and we
12:27were highly concerned about basically
12:29like, hey, we don't want to do something
12:31where we're going to be constantly um,
12:34sort of accusing people of using AI when
12:36they didn't. Um, and uh, we settled on
12:41panggram as sort of the best option for
12:44that. And so we now in near real time
12:47when students are editing Wikipedia um,
12:50check what they're doing through Pang.
12:53And we're increasingly using that as
12:56kind of a signal that says, "Hey, this
12:58is worth a second look." Not that this
13:00is automatically where something going
13:01to if it has a hit, then oh, it's AI and
13:05so it's going to get nuked. rather it's
13:08a one opportunity for uh you know
13:12reaching the student and re-emphasizing
13:15the importance of not using AI and why
13:17it's likely to be um sort of a a a bad
13:20thing for Wikipedia but also focusing
13:23down on what we've found is the kind of
13:26key critical failure mode that AI
13:29consistently does very poorly at and
13:32that is kind of the correspondence
13:34between the text of an article and the
13:37citations. So, um the better AIs now are
13:41are pretty good at um writing text that
13:46has a lot of accurate facts in them. Um
13:48and also uh will happily provide
13:52citations to uh like sources that are
13:56often um right on target in terms of
13:59yeah, this is a good source. This is the
14:01kind of thing you would want to use to
14:03write about this topic. Um but
14:06essentially the uh specifics of hey this
14:09sentence uh is backed up by this source
14:12uh is where AI usually completely breaks
14:16down. Uh and so we have some automated
14:18emails and then we also have um kind of
14:21our staff uh available to take a look
14:24and we try especially to keep um
14:27unvalidated uh sort of likely AI content
14:31um out of Wikipedia's main space. that's
14:33kind of our priority uh right now.
14:36>> Great. So I think um there there's lots
14:39of questions around um Pangram and I'm
14:42excited to have Max here who is the CEO
14:45of and founder of Pangram. So Max, I
14:48think AI detection is obviously a hot
14:50topic in um in the public discourse
14:54right now, but particularly in the
14:55higher education sector. Um I think a
14:58lot of our audience today may not quite
15:00understand how it works. Um so can you
15:03share a little bit about how pangram
15:05determines whether a text is AI
15:06generated or not?
15:09>> Sure. So pangram is a machine learning
15:12model. Uh it's a classifier model. So
15:15it's not generative. It's not generating
15:17text like chatgpt but instead it's like
15:20when you're on Google Chrome and it
15:22notices that the web page you're reading
15:24is Spanish. Um it's like uh Google's
15:28like language detection model is very
15:30similar in a sense to Pangram's AI
15:32detection model. Uh so how does it work?
15:36Uh well so we we create a training set
15:39and we're going to use this training set
15:41to teach our classifier model um to
15:46infer the author of the text given what
15:49it's seen. So we start with a corpus of
15:54human examples. These are of texts from
15:582022 or earlier before chat GPT. So
16:02maybe for example is like a uh a
16:07Wikipedia article on Moby Dick. Uh and
16:10then what we could do is is we ask a
16:13random AI model to also generate a
16:16Wikipedia article on Moby Dick. And then
16:20when we feed these into panggram, uh
16:22during the training process, we teach
16:25panggram learns through a bunch of small
16:27signals and word choices and sentence
16:29choices what are the differences between
16:31this human written article about Moby
16:34Dick and this AI written article about
16:36Moby Dick. And so we do this over the
16:39course of millions and millions of
16:41examples. And through the power of um
16:46uh machine learning and using a
16:49transformer to understand the text, we
16:52are able to infer whether the author of
16:55this text was AI or not. And in fact,
16:59Tanggram actually internally we've we've
17:01learned that it actually can likely
17:03infer the difference between something
17:05that's claude written and something
17:07that's chat GBT or Gemini written
17:09because all of these AI models have
17:12pretty distinctive styles. uh and and
17:16we're able to
17:18the the minimum amount of text that we
17:20need to classify um a text is 50 words
17:24but by 200 300 500 Pra is very very
17:27accurate and very confident um in its
17:30classification
17:32uh so that's one side of things the
17:34binary side which is AI or human but
17:37Pangram today actually goes a bit
17:39further pangram can tell you the degree
17:41of AI use roughly
17:43Uh, and how it does that is it also we
17:47also teach it with editing prompts. So
17:49let's say I take again this Wikipedia
17:51article on Moby Dick and then I ask chat
17:54GPT, hey improve this, make it better.
17:58So then then we're going to go clause by
17:59clause and label every clause and say
18:02either this clause was in the original
18:04text. This clause is modified from the
18:06original text. We'll call that AI
18:08assisted and this clause is completely
18:11new that has information that was not in
18:14the original text. And we'll call that
18:16AI generated. And then so we're going on
18:18a clause level and teaching the pang
18:21model that you know this clause is new.
18:23It's AI generated. this clause is um old
18:26so it should be classified as human
18:27written and so on and so when you look
18:30at the pang score on mixed text often
18:33times it'll say a couple sentences are
18:35AI and then this sentence is human and
18:37that's because it's doing this it has
18:39learned on a clause or sentence label to
18:42classify um
18:45sentences as AI or not
18:48the the very short version I'm happy to
18:50to go more in depth and answer any
18:52questions
18:54Great. Thanks, Max. And I know we'll
18:56we'll be getting back to you and have
18:58some additional questions coming in on
18:59the chat. So, I think there's a lot of
19:01interest in this topic area, but in the
19:03interest of time, I'm going to move on
19:04for now. Um, so I'll start with Whitney,
19:07but Oliver, please feel free to build
19:09off this question, too. Um, so how have
19:12you handled students who have been
19:14flagged by Pangram through our program
19:16who have used generative AI for their
19:18Wikipedia assignments?
19:20Um, in short, I would say very
19:23carefully. Um, I work with first year
19:26students and, um, there's a lot of
19:30anxiety about generative AI, not knowing
19:34when they can use generative AI
19:36generally across classes, um, and just
19:38being scared that they're going to be
19:40accused of using AI. So, there's a lot
19:43of fear. So, in the emails, you know, I
19:46think it's really great. Wiki Ed has a
19:48lot of great communication about what
19:50LLMs are. There's a really wonderful
19:52training, you know, saying clearly, do
19:54not ever copy and paste. Um, and so
19:57that's really helpful and it's really
19:59helpful to be transparent upfront. Um,
20:02but when those emails do come through,
20:04my students do experience some anxiety
20:07there. So, as Sage was saying, you know,
20:09it's really I'm reaching out to them and
20:10saying, "Hey, this has been flagged. You
20:12know, let's talk about it. Let's see
20:14what's going on here." And across my
20:17classes since Pangram has been used by
20:19Wiki Ed, I've actually had very very few
20:23um pieces flagged and a I would say
20:27maybe a third of those were actually
20:29texts that they had copied from
20:31Wikipedia and were going to edit and
20:34they got flagged from previous text
20:36within Wikipedia being uh Gen AI
20:40produced. And when that happens, that's
20:42a really great teaching moment because
20:43they are so frustrated and feel like
20:46they have been, you know, kind of like
20:48somebody's not following the rules.
20:50Somebody's using this this text and
20:52they're not writing and they feel that
20:55um sort of emotional response to it. And
20:58I can always use that as a time like
21:00yeah, that's why it's so frustrating
21:01when people are using Genai on Wikipedia
21:05and why we really need to be following
21:07um these rules. So I think you know as
21:09Sage was saying opening it as a
21:11conversation. I always make really clear
21:13to my students too that um this is a
21:16rule for all of Wikipedia and that
21:19panagram is being used on their
21:21Wikipedia assignments only. Um I'll have
21:24to admit that when Wiki Ed started using
21:27Pangram I did have a little bit of
21:29trepidation because I take my students
21:32intellectual property really seriously.
21:34So, I don't use um software like Turn It
21:38In or other kinds of play plagiarism
21:41detection because you're giving your
21:42students intellectual property away. I
21:44also don't put my students uh content
21:47into LLMs because of that. And so, I
21:50talk to my students about that. But
21:51Wikipedia, we are kind of giving this
21:54away, right? We're giving our time,
21:55we're volunteering our time, we're
21:57volunteering our ability to access text
22:00that other people can't access. And so I
22:02think it makes a lot of sense in this
22:04space to be using um AI detection or
22:08looking at it much more critically than
22:10I would in other spaces. And so I think
22:12it's important to sort of lay that out
22:14for my students and and be transparent
22:17and talk about when and why it's being
22:19used. So that's kind of how I've
22:20approached it. [clears throat]
22:24>> Just to add on to what Whitney said, I
22:27think my approach has been similar. I am
22:31hesitant to make direct accusations
22:35when I get noticed about pangram
22:39flagging a student's text even though I
22:42know from all the the research into
22:45pangram that false positives are
22:47exceptionally rare. I think it's max can
22:50confirm but something like one in
22:5110,000. Um,
22:54so
22:56I I feel fairly confident when I see the
22:59flag text as AI that it probably is AI
23:03generated. But even so, I don't want to
23:05get into a long debate with a student
23:08about whether or not AI was used.
23:11Instead, I focus more on the problems
23:15that I can find in the text itself. The
23:19problems that AI almost always
23:21introduces to Wikipedia articles. And
23:25the the big one there is um as Sage was
23:28talking about earlier, the issue of
23:30verifiability and use of sources.
23:34without exception the the text that has
23:38been flagged as AI generated
23:41through this panggram integration I'm
23:44always able to find some way in which
23:48the text fails the verifiability
23:50standard um that might be that the cited
23:55source doesn't actually mention the
23:58facts that the student has added or
24:01maybe the cited source does talk about
24:04that topic but not on the pages that um
24:08that the student had referenced. Um or
24:11sometimes the student will be citing a
24:15source that does talk about that
24:17subject, but it's it's really not one of
24:20the authoritative sources on the topic.
24:22Um, one of the things that we can get
24:25into more maybe later is is how when
24:30LLMs are pushed to provide citations to
24:35back up the claims that um are in in the
24:37text they generate, they are limited to
24:41text that's accessible to them on the
24:43internet. So often the most authorit
24:46authoritative sources on a topic that
24:48I've worked with students to um to make
24:51sure they're emphasizing in their
24:53research often those aren't things that
24:55chat GPT can look at directly because it
24:58might be in a book that's not fully
25:00digitized online or maybe it's uh an
25:03article that's behind a payw wall. Uh,
25:06and so often times this text that gets
25:09flagged, even if it is citing a source,
25:12it's citing some online source that's
25:15not what the student and I had agreed
25:18upon when coming up with a list of of
25:21um, good bibliographic references. It's
25:23not what we said that the student would
25:25use. So placing emphasis on that
25:29bibliography stage of the project is
25:32often really helpful for setting a
25:33foundation for any later conversations
25:36about about AI use.
25:39>> Yeah, thanks Oliver. And I think that
25:41that lines up with what Sage was saying
25:43about sort of how Wiki Education is
25:46framing this not as sort of an
25:48accusation of wrongdoing, but instead as
25:51a invitation for the student to show you
25:55as the instructor where they actually
25:57found that information and go through a
25:59verification exercise because I think
26:02you know what we find then is when we
26:04target the sort of problematic outputs
26:07of the generative AI usage, then we're
26:10able to have a more structured
26:12conversation with students around what's
26:14wrong with the text versus just saying,
26:16you know, oh, you used AI and so that's
26:18bad. We're we're saying you used AI and
26:21that's bad because the text is no longer
26:23verifiable or this sentence has a
26:26citation, the information is not
26:28actually in that citation that that
26:30you're claiming it is. And so, you know,
26:33being able to move past that sort of
26:34accusation, I think moves it into a much
26:37more constructive dialogue in the
26:38classroom. And you know, and that's
26:40where we've been really grateful for um
26:43for Pangram as as a tool to enable us to
26:45sort of flag that. So, so I think
26:48Oliver, you talked a little bit about
26:50sort of the the conversations around um
26:53the use of AI and Wikipedia text, but I
26:56know you you've talked about how it has
26:58also impacted other classroom
27:00discussions such as the relationship
27:02between knowledge, representation, and
27:03authenticity. Can you talk a little bit
27:05more about that?
27:08>> Sure. I think one of the kind of fun
27:11surprises about needing to talk about AI
27:15early in the semester connected to
27:18course policies and the Wikipedia
27:20assignment is that actually a lot of the
27:22issues that come up in those policy
27:25course policy conversations end up uh
27:28resurfacing when we're thinking about
27:31topics in the course content. Yeah, I'm
27:33an art historian. So u examples might be
27:38when we talk about the invention of
27:40photography and the effect of
27:42photography on painting students notice
27:45parallels to how we talked about the
27:47introduction of AI and AI's effect on
27:49writing. And similar to how
27:52um with the introduction of AI there are
27:56predictions about oh this makes human
27:58writing obsolete. So too with the
28:00invention of photography, you have u
28:03critics and even artists worrying, oh
28:06this means the end of art. Uh painting
28:10uh painting is no longer needed because
28:13we have a machine that can create um
28:15realistic pictures. But of course we
28:17know painting didn't cease to exist. And
28:21so students then pause and think about,
28:23okay, um maybe the um the historical
28:28material we're studying um isn't
28:31instructive for thinking about this
28:33transition we're going through
28:34technologically today. I think also when
28:38we talk about works of art that try to
28:41persuade the viewer that they're
28:44truthful um and reliable, students
28:48sometimes make the connection to our our
28:50conversations about AI. So if we look at
28:53a painting like um like the the famous
28:56painting of Washington crossing the
28:58Delaware
28:59um which many American students know u I
29:04can ask how does this painting make you
29:08think that it's it's trustworthy that
29:10that this really happened and students
29:14will often point out how
29:18how smoothly it's painted how detailed
29:20it is how the ropes on the boat, you can
29:23see individual fibers and the the ice
29:26and the water is so sharply painted. Um,
29:29but then we'll talk about how well you
29:31can simulate a lot of those things. So,
29:33does that really make the painting
29:35reliable
29:37as as a guide to what actually happened?
29:40And then sometimes students will make
29:42the connection to the Wikipedia
29:43assignment and and they'll say, well,
29:46with with Wikipedia and AI, we talk a
29:48lot about how to really verify whether a
29:52text is trustworthy. You can't just go
29:55on stylistic grounds of like, oh, it
29:58sound it sounds authoritative. You
30:01actually have to cross reference with
30:02sources. So maybe we should read
30:06accounts of Washington actually crossing
30:08the Delaware or look at documents and
30:10compare um uh and and we will then
30:15discover oh there are things that don't
30:16match up like famously in that painting
30:18the American flag on the boat didn't
30:20actually it wasn't the American flag at
30:22the time of Washington crossing the
30:24Delaware. So that's all just to say that
30:29I I'm actually I I started off feeling
30:33kind of resentful that I had to devote
30:36so much course time at the beginning of
30:38the semester to AI policy conversations,
30:41but it's actually uh been a really
30:44useful way to connect some of the
30:46courses larger themes to to things that
30:49are going on in our own time.
30:53>> Thanks Oliver. I love that of just, you
30:55know, using a an element of your course
30:58syllabus anyways to to then talk about
31:01kind of representation and authenticity
31:03and verifiability in numerous contexts.
31:07So, I think finding ways to kind of
31:08connect that into the the course subject
31:12is um is a really important uh
31:15pedagogical tool right now. Um Whitney,
31:17I I want to come back to you um for for
31:20one additional question here um around
31:23this concept of verifiability. So, how
31:26has centering verifiability helped
31:28students understand why it's so
31:31important to not use AI when drafting
31:34Wikipedia text? And is this different
31:36from a traditional essay or research
31:38paper that you might otherwise assign?
31:41Um, so I think I'm going to start with
31:43this second one. I actually second
31:45question. I actually replaced a more
31:47traditional
31:49um research. It was a lit uh lit um a
31:53literature review style essay. We're
31:56tasked with teaching students about
31:57academic writing right early in the
31:59first semester which I think we can all
32:01agree academic writing is is many things
32:03and AC in many different disciplines uh
32:06and even within them and one of the
32:08things that is so important is citation
32:10right and I think that that is one thing
32:12that Wikipedia and academic genres have
32:16in common and when I was doing the
32:17literature review essay students were
32:19really frustrated because I was kind of
32:21coming in and being like you have to
32:22site this you have to cite this and they
32:23were like why why why I was like well
32:26because this is what we do [laughter] um
32:28in academic writing you know and for me
32:30that was a very persuasive argument um
32:33not so much for my students and so when
32:35I took that out and brought Wikipedia in
32:38I think the real focus on verifiability
32:40and citation and using good strong
32:44sources and really thinking about those
32:46sources I've had um wonderful
32:49Wikipedians come in and sort of look at
32:52some of my student work and say Hey,
32:54these sources and sort of go through
32:56their sources and teach them about the
32:57sources that they're using, you know,
32:59um, and whether they're strong or not in
33:02the case of that article. And that is a
33:04really wonderful thing I think that
33:06Wikipedia has where you have all these
33:08people who are who are coming in and and
33:09willing to do some teaching, right, and
33:12teaching about that context. So, I think
33:15that um
33:17my students really appreciate the need
33:20for verifiability a lot more after
33:23having completed the Wikipedia
33:24assignment and working on Wikipedia.
33:26They understand much more that citation
33:29is something that builds their ethos in
33:32many different contexts and not just in
33:34those academic ones. Um, and as I
33:36mentioned before, when you know students
33:39are are getting flagged for AI use and
33:40it's from somebody else, that's a real
33:42lesson for them about why it's so
33:44important not to use generative AI. Um,
33:47and going back too to the points that
33:49have been made about how quickly content
33:51from AI is swept up and uh spit back out
33:56by these different um AI supported um
34:00search. That also I think makes them
34:03take their work a lot more seriously in
34:05understanding that you know this is
34:07going to circulate widely right this
34:09isn't just writing an essay for me as
34:11your teacher this isn't even just for
34:13Wikipedia unfortunately in a lot of
34:16instances it's going to be um traveling
34:19around and then I think that's a good
34:21conversation about well what are the
34:23responses on that AI supported uh chat
34:27right or uh search results um you know
34:30where is this coming from and and how
34:32can we sort of think about that. So, I
34:34think verifiability is is a really great
34:36thing to focus on in all of these
34:38aspects and has just been super valuable
34:41for me and my students.
34:45Great. Thanks so much. Um I think I want
34:48to go back to to Max now and I had a
34:51sort of prepared question that I'm going
34:52to modify slightly based on because
34:54we've got a lot of questions kind of
34:55coming into the chat around um around
34:57Pangram. So, my original question is,
35:00what are the long-term prospects for
35:02tools like Pangram keeping up with
35:05detecting fully AI created text? Um,
35:08specifically interested in hearing how
35:10you're addressing things like humanizers
35:13or other ways that people take AI text
35:16and make changes to it in an attempt to
35:18evade detection from software like
35:20pangram. I think I would also add based
35:23on some questions coming in from the
35:24chat sort of one of the common um things
35:27that I think a lot of instructors in our
35:29program here is like oh I just used
35:31Grammarly and you know the challenges
35:34obviously of tools like Grammarly is
35:36they used to just fix your you know your
35:38grammar and now they will actually
35:40suggest sentences for you and you know
35:42draft your text kind of for you and so I
35:45think the kind of differences of you
35:47know Grammarly is an AI tool and so
35:50using Grammarly, you know, can
35:53give you a positive on on tools if
35:56you're asking it to draft text for you.
35:57So, I guess I'm interested, Max, if you
35:59want to share a little bit about sort of
36:01Pangram's philosophy around the like how
36:04accurate is the, you know, is the
36:06detection, how will that change in the
36:08future as these AI tools become more and
36:10more integrated into daily life and um
36:14with the use of humanizers and other
36:15things like that.
36:18>> Okay, cool. Yeah, a lot to talk about.
36:20So, I'll start kind of from the top on
36:22Pangram's baseline false positive rate.
36:25It's about 1 in 10,000. That means for
36:2910,000 fully human written texts that uh
36:33are entered into Pangram, about one of
36:35them will be classified as any% AI. Uh
36:40with that said, um there are a lot of
36:43tools that are in daily life like
36:45Grammarly where Grammarly has kind of
36:48two modes as the nonAI mode where it
36:50just like will fix your grammar and fix
36:52your punctuation and that's the AI mode
36:54where you can do things kind of like the
36:55the prompts I talked about before where
36:57you can say improve my text, make it
36:59more descriptive, uh make it more
37:01formal. uh and these sorts of rewrites
37:04panggram is trained to call these AI
37:06assisted rather than fully AI generated.
37:09So if you write something yourself and
37:11then have Grammarly
37:13rewrite it in like you know Wikipedia
37:17register uh the pangram ideally will
37:20classify that as AI assisted. We've also
37:24run a lot of evaluations on English
37:27language learner text and I saw a
37:29question in there in the Q&A and so I
37:31just wanted to answer it by saying uh in
37:34our most recent technical report we've
37:37actually benchmarked on all of the major
37:39English language learner data sets and
37:42found Pangram had one false positive out
37:44of about 25,000 texts. So still low uh
37:49and still around the same order of
37:51magnitude as our um normal false
37:55positive rate. So the next thing to talk
37:59about is sort of just like the the
38:01future humanizers.
38:03Um
38:05yeah I I think basically here
38:10I guess two things to talk about is like
38:13AI models are getting better. They're
38:14getting more capable. Are they becoming
38:17more humanlike? Uh, I think that's been
38:20kind of a topic of debate. My
38:23uh personal opinion, most humanlike AI
38:27models were like GPD2 and GPD3, which
38:30were trained to mimic the distribution
38:32of human writing only. And today's AI
38:37models are instead um
38:40much more capable and they're tuned to
38:43try and find like the correct token.
38:44They're
38:46preferences are introduced in the
38:48training process which makes AIS sound
38:51the way they do. It's why Claude sounds
38:53so distinctive. It's why like you can
38:55read one paragraph of text from Claude
38:58Fable and it sounds almost alien and
39:00like a little bit different. And I think
39:02that's because of these preferences that
39:04are added during the training process.
39:07Um
39:09my
39:11Yeah. Yeah. And then the last question
39:13was on humanizers which is these are
39:15tools that are commonly used sometimes
39:19by students to take text and paraphrase
39:22them in a way that the goal is to bypass
39:26uh AI detectors. Uh so Pangram also
39:30trains on paraphrased AI text in order
39:34to be able to say this this is
39:36paraphrase AI text. this looks humanized
39:38rather than saying that like it it looks
39:42human because ultimately all of these
39:44paraphrasers are also AI models that we
39:46can also learn how they work.
39:50Um that was
39:53may maybe I'll answer one more thing on
39:56>> go for it. Yeah
39:57>> from the questions. Um there there were
40:00some questions around you know some
40:02people on Substack have uh made claims
40:07of AI detectors. Uh I think Michael Bach
40:10had a note that he ran one of his books
40:13uh written in 2019 through various AI
40:16detectors and it came back some of them
40:18said it was AI written. So how do you
40:20trust Pangram? I the thing to say which
40:24um
40:26partly it's kind of just like trust me
40:27but like like Pangram is really
40:29different than these other AI detectors.
40:30There's a whole bunch out there online
40:33varying degrees of accuracy. Most many
40:35of them are very inaccurate and many of
40:37them also sell a humanizer as part of
40:40the same package. So really their AI
40:42detector is there to sell their
40:43humanizer. We don't have a humanizer. We
40:46hire a bunch of really great researchers
40:49and we're working on solving the problem
40:51to the highest degree of accuracy. So
40:53even if your 2019 written book came back
40:56AI in other AI detectors, please try it
40:58with Pangram. I think you'll be happily
41:00surprised with the result.
41:05Thanks, Max. And Sage, do you want to
41:06talk a little bit about why Wiki
41:08Education chose Pangram in particular?
41:11>> Sure. Um,
41:14the false positives were kind of the
41:16central concern that I had when I was
41:20looking into AI detection in the first
41:22place. Um, and I had tried some of them
41:26several years before, um, and found that
41:29they were noisy enough to just like not
41:32be worth bothering with. Um, but, um,
41:36the the sort of like real thing that
41:39that
41:41uh, like sort of lit up uh, my mind for
41:46this and I think the rest of the team
41:48uh, was we ran basically like overtime
41:52articles that students had written from
41:55the beginning of our program like up
41:57through now. So started with kind of a
41:59small sample and say hey like if we you
42:01know run articles from uh 2015 from 2018
42:06from 2020 um and then like term by term
42:09um what does pengram say? And when we
42:12initially did that, um, I was floored
42:14that we literally found zero hits from
42:18before late 2022 when chat GPT launched.
42:22Um, now I would say that once we started
42:25putting it into practice and and like
42:27trying to run it not just on like, hey,
42:30this is we're going to check an entire
42:31article, but we're actually looking at
42:33the level where where students work. Um,
42:37it does get a lot messier actually. Um,
42:39so, um, I think that genre is, um, a
42:44really key factor in, um, how well any,
42:48uh, AI detection, uh, tool works, um,
42:51including Panggram. Um, and so like, um,
42:54I know that Pangram has been
42:56specifically trained on, um, Wikipedia
42:58articles among a lot of other genres of
43:00text. Um, and that they they see a lot
43:02of kind of like, you know, cross
43:05benefits from training on different
43:07kinds of text. So that it is pretty good
43:09also with with kind of genres that they
43:11haven't specifically trained against.
43:13But um when you're looking at sort of
43:16like granular edits of like hey uh I'm
43:19inserting or rewriting an existing
43:21paragraph on Wikipedia. Um I'm working
43:24originally with wiki text and so it's
43:26kind of already been transformed through
43:30um uh some degree of kind of like
43:32structured uh uh coding language. um not
43:36just kind of like pros that would match
43:39up to uh you know what someone might uh
43:41like be typing originally. Um like that
43:45can introduce some some things that can
43:47trip up um panggram as well as others uh
43:51related to sort of formatting um uh sort
43:54of like the it can be sensitive to
43:57things like kind of bulleted lists in
43:59the con in the middle of something
44:01that's the rest of pros um and things
44:03like that. Um, and so it took us a while
44:06to kind of tune, hey, these are the
44:08things where we think that it is doing a
44:10great job if we strip it down to just
44:12the pros, um, and sort of get as close
44:14to um, human writing as possible with
44:17what we're trying to check. And that
44:19means that we don't check some other
44:21things um like uh an annotated
44:25bibliography for example is is a kind of
44:28genre where uh uh uh we've found that
44:30pengram and other AI detectors as well
44:34don't actually do that well because uh
44:35an annotated bibliography is a mix of of
44:38text that is definitely not human
44:41written text right like a a sort of MLA
44:43citation is not AI generated but it's
44:47structured and has sort of like very
44:49specific patterns of usage that don't
44:52reflect sort of like human writing. It's
44:54it's like so mixes of structured text
44:57and unstructured text um tend to be
45:00where like we we have to be really
45:02careful with interpreting the results.
45:04Um but um as I said the sort of like
45:07baseline of hey it can tell with an
45:10extremely good degree of accuracy that
45:12none of these things written before the
45:14launch of chat GPT were AI generated.
45:17And then you see sort of a steady uptick
45:19term by term as more and more students
45:22are adopting it uh of of what
45:24panagramram shows. That was what made me
45:26say okay well this is probably if we
45:28fine-tune it and figure out what parts
45:30of the signal to look for um this is
45:33something that we probably ought to be
45:34building around. And it also comes down
45:37to like it's it's a cost benefit. even
45:40even a a sort of you know um extremely
45:43accurate uh detector that only has uh a
45:46false positive every once in a while. Um
45:50that one that one false positive can
45:52still be like really meaningful and and
45:54have negative impacts um on the one
45:57hand, but on the other hand like it's
46:00kind of existential for what we do to
46:04not be like having a ne negative impact
46:07on Wikipedia. And so we just simply
46:11couldn't keep up with the the difficulty
46:14of like factchecking and and and getting
46:19rid of of sort of the the slop that
46:22didn't represent sort of like meaningful
46:26intellectual engagement um uh at the
46:29scale that it was starting to happen.
46:30Um, so that's kind of the the balance
46:32that I think any use of AI detection has
46:35to to weigh is sort of like, hey, what
46:37are the costs? Um, and and like what
46:41what do you need to to be able to
46:44protect the system that you're trying to
46:45protect?
46:47>> Yeah, thanks Sage. I agree. I think
46:49that's a it's a super important point.
46:51And I know we are coming up on time
46:54here. So, if you have any additional
46:56questions, I think there's a lot of sort
46:58of very specific questions about Pangram
47:00in the chat that we can try to get to.
47:02Um, if there's any broader questions,
47:04please feel free to add them in the Q&A
47:06box. Um, I have one more question for
47:08our panelists and then we will turn to
47:10the the Q&A box. Um the question I have
47:13for our panelists is I think we've we've
47:14talked a lot about some of the kind of
47:16negative challenges of generative AI
47:19text but um as I mentioned at the top
47:21generative AI is more than just a text
47:24generator right so are there some
47:26positive ways um either Oliver or
47:29Whitney that you've seen students use
47:30generative AI in their Wikipedia work or
47:34Sage are there additional kind of ways
47:36that you've seen us encourage students
47:38to use generative AI not for text
47:41generation that has been helpful in the
47:42Wikipedia context um either for students
47:45in our program or for the broader
47:47Wikipedia community.
47:54>> I can I can get started. Um there were
47:59two two main ways that came to mind that
48:05are potentially productive for for
48:07students using generative AI. Um, one
48:11really basic one is sometimes when I
48:14have a student working on a research
48:18project for a Wikipedia article on a
48:20particular topic where some of the best
48:22sources are in a language that the
48:24student doesn't um doesn't understand or
48:27doesn't um understand well.
48:31It has been useful for translating
48:36specific passages for that student. If
48:38we can identify a really key passage in
48:41a text that we think, you know, this is
48:43one of the most authoritative sources on
48:45this topic. It's in this other language.
48:48Um, let's have it let's have um an LLM
48:53give you a literal faithful translation
48:56of this specific passage. Even that
48:59requires some caution. If a student
49:01gives a long text like a a 30-page
49:05article in a different language and asks
49:08for an English language summary, uh that
49:11introduces some problems because then
49:13the the student doesn't necessarily know
49:15for sure where that information came
49:17from within the source and and sometimes
49:20in summarizing
49:22the nuances of the argument get lost.
49:24But a translation of a short passage,
49:27that's been helpful. And then at times
49:31when a student uses generative AI to ask
49:36for additional sources the student might
49:38have missed, occasionally that will turn
49:40up sources that that the student had
49:43overlooked. And there again, it's just
49:46key to emphasize
49:49you have to then go get the source and
49:51read it yourself because you can't trust
49:54the the LLM to provide a faithful
49:58summary of it pointing to exactly where
50:01the information came from within the
50:03source. But as a starting point for
50:05research, sometimes it it can be useful.
50:07I'm always a little hesitant to to even
50:10permit that just because it's easy for
50:14students to then take the next step and
50:16assume that they can use generative AI
50:20to to actually read the source for them.
50:23Um, but for finding finding sources they
50:26might have missed, sometimes it has been
50:27helpful.
50:30>> Um, yeah, you can build on that. So, I
50:32think sometimes it can be occasionally
50:34useful for finding other sources. Again,
50:36I work with first year students, so I
50:38think one way that they find it helpful
50:39is um just
50:43search terms, key terms, so they can go
50:45find sources themselves. I know my
50:47students really struggle with that, but
50:48I tend to
50:51I'm I'm pretty uh critical and resistant
50:54toward generative AI. I think one of the
50:56best things that uh generative AI has
50:58done is actually have people have
51:00conversations about artificial
51:01intelligence and thinking about um
51:04inequality and how you know these are
51:08serious issues. I'm super psyched that
51:10everybody's upset about data centers
51:12being built in their communities and
51:14those kinds of things. So I think I
51:15think these are conversations that are
51:17long overdue and I'm I'm happy that
51:19they're finally coming to the four to a
51:21certain extent.
51:27Okay. Well, thank you all. I want to
51:30We've got a lot of questions in the Q&A
51:32and so I'll I'll start by just saying
51:33there's no way we're going to get to all
51:35of them here. Um but let me let me pick
51:37out a a couple that I think is not uh
51:40something we haven't quite talked about
51:42yet. Um so Thomas asked a question for
51:45Oliver talking or talking about
51:47authoritative sources without checking
51:50all cited sources and reading all the
51:52cited sources. Is is there some proxy
51:54that you judge reliable for not citing
51:56authoritative sources? For example, just
51:58a citation of an online article would be
52:00viewed as suspicious.
52:04Yeah. Um so if I understand the question
52:08correctly, it's getting at getting at um
52:10h how do you evaluate whether the
52:14student has actually engaged with an
52:16authoritative source on on the topic?
52:19Um, and one thing I'll say here is it
52:23really has helped for me to focus my
52:27students Wikipedia projects on areas
52:31where I feel I have some background
52:34knowledge on the scholarly literature.
52:36So, so I um
52:40know
52:42at least um the basics of who who are
52:45the the really um key figures writing on
52:49this topic and I'll notice if there's an
52:52article that doesn't sound sound like
52:55one of the big scholars on the subject.
52:58But the other thing I'd say is um a lot
53:01of this ends up just being a lot more
53:04work for me. I mean it assessing
53:07assessing whether the student has
53:09actually engaged with authoritative
53:11scholarly sources. It's required a lot
53:14of
53:15looking at the sources myself and
53:18double-checking. Um, it helps that I
53:21have a great university library. And so,
53:26um, I have on occasion checked out a
53:29book just to track down a reference to
53:33see did the student actually engage with
53:35this this um, this book that's cited
53:38here. Um, I'm glad that
53:43most of my student projects don't
53:46require that level of um of
53:49[clears throat] double-checking because
53:52most of them still don't get flagged
53:54because of all of our conversations
53:56earlier in this semester warning them of
53:58the dangers of generative AI. But there
54:01there are always a couple projects
54:03that do require
54:05going and and becoming a bit of an
54:08expert on that subject myself
54:12um in order to um in order to verify did
54:16did this student really look at the
54:18sources that are cited here and are are
54:20these the most authoritative sources on
54:22on the topic or are these just the most
54:24accessible sources online. Um, so there
54:28I would say it helps even even if I'm
54:31not a an expert on the subject that the
54:33student is writing about, I always try
54:35to guide my students to work on subjects
54:38where I at least really want to know a
54:40lot more about that subject in case
54:42something like this comes up because
54:43then it becomes sort of fun and
54:46interesting. um and and not just a a
54:49kind of rule enforcement thing where
54:52okay, yeah, I do need to double check
54:54some stuff in this student's in this
54:56student's project, but I've always
54:57wanted to know more about this painting
54:59anyway. So, u a trip to the library
55:02isn't really a big bother.
55:05>> That's great. And and yeah, one of the
55:07things we've heard anecdotally for years
55:10about this program is that ironically
55:12writing a Wikipedia article is often one
55:15of the first times students set foot in
55:17the library um on campus because they're
55:20forced to do a a deep dive into the
55:23entirety of the published research on a
55:26particular topic versus just what's the
55:28first like couple hits that you can get
55:31from a a search of a the library
55:33scholarly databases online. And um so so
55:36yeah, I love the the sort of idea of
55:38going back and and working with students
55:40on topics that um that are actually of
55:42interest to you. Um I think we're almost
55:45at time now and so I want to end with um
55:48one additional question here um for for
55:51all of our panelists just in 30 seconds
55:54or less. What is one step instructors or
55:57Wikipedia editors could take today to
55:59help protect information integrity
56:02online?
56:06contribute to Wikipedia.
56:08>> Well, the [laughter] perfect answer.
56:10>> I mean, I think that's one of the things
56:11that we can really make an impact
56:13because there is such circulation.
56:17>> Thank you. Anyone else want to jump in?
56:20>> I would say um
56:24become really rigorous about um the way
56:27that you um are transparent about
56:32using AI. Um, I I I've uh sort of tried
56:36to take it on to like anytime I'm doing
56:40anything with AI, which as as a coder
56:43now, I've started to find very useful.
56:45Um, I I try to be really rigorous in
56:48differentiating this is how I used AI
56:51and what AI I used. Um, and and sort of
56:55what human involvement was involved in
56:57in any given artifact. Um, and I think
56:59that kind of like establishing a norm of
57:02if you're going to use AI, it absolutely
57:05needs to be transparent. You need to
57:07explain how and why you did or you know,
57:09it may be obvious, but like you need to
57:11disclose that. I think establishing that
57:13norm um is is really critical right now.
57:18>> Yeah, I want to jump on and kind of
57:21second that. I think like we're we're in
57:23this time of norms are in flux. AI is
57:27this very new technology and it's
57:28changing monthtomonth and so people who
57:31are used to treating it more as a tool
57:33and starting to realize like this is
57:34actually like it it is an extremely
57:37powerful technology um it can definitely
57:40help make all of our lives much better
57:43but also I think it um enables if if you
57:47if people don't disclose properly if
57:49people aren't transparent about how they
57:50use it um then we could enter this
57:53culture of much greater distrust and
57:55we're trying to not have that. And I
57:58think yeah, you use AI and and use it um
58:03use it properly with with disclosure.
58:07>> Yeah. And I would endorse everything
58:12that's been said. I I would um say that
58:17I have been reluctant to use AI,
58:22but have come to the conclusion that
58:24actually using it
58:27to some degree every day actually has
58:29helped me really recognize its hallmarks
58:33in in student work, anywhere else that I
58:36encounter it on the internet. And I feel
58:38that I was doing myself a disservice by
58:41just pretending it didn't exist because
58:43now I actually feel like I can I can
58:47help um call attention to to its
58:50influence because I I notice some of its
58:53hallmarks just from interacting with it.
58:58>> Absolutely. Well, please join me
59:00everyone in thanking our four wonderful
59:03panelists today. Thank you Oliver,
59:05Whitney, Sage, and Max for joining us,
59:08for sharing um all of your your
59:11excellent insight. And I'm sure this
59:12conversation could continue for another
59:15hour, but we are over time right now. So
59:18um so thank you all for joining us
59:20today. And um please stay tuned. The
59:23next edition of our speaker series will
59:26be coming up in October. We're still
59:27finalizing the date, but um look for an
59:30email invitation for that. And thank you
59:33for joining us today. Thank you for the
59:35very active chat and the great questions
59:37in the Q&A box and I'm sorry we couldn't
59:39get to all of them. Um, if this sparked
59:42your enthusiasm for participating in
59:45Wiki Education's programs, you can learn
59:47more about teaching with Wikipedia at
59:49teach.wikedu.org.
59:51Or if you're inspired to edit yourself,
59:53um, heed Whitney's call and join one of
59:56our courses at learn.wikedu.org.
59:59Thank you very much to our panelists and
1:00:01to all of our attendees. Thank you all.