Full transcript
0:02hello participants welcome to the short
0:05video on test blueprints a guide to
0:09effectively planning your pre- and post
0:11assessments if you remember in the first
0:13video we talked a little bit about the
0:15need to create test blueprints when uh
0:18creating pre- and post Assessments in
0:20this presentation we'll talk about
0:22exactly how to do that if you took the
0:24creating assessment course with us
0:26before we refer to this as a table of
0:28specifications specifically when looking
0:31at one type of assessment for the
0:33purposes of this course we'll refer to
0:35it as a test blueprint specifically when
0:37designing pre and post
0:41assessments so what does this
0:43presentation cover we'll take a look at
0:45the objectives and overview how to build
0:48pre- and post tests by creating testing
0:50blueprints writing test questions and
0:53building the tests to the
0:55blueprint and we'll discuss briefly how
0:58do you administer pre- and post assess
1:01Ms our objectives for this presentation
1:05are to acquire a method that will
1:08accurately measure learning gains of a
1:10class or group how are you going to do
1:12this by learning how to create a test
1:15blueprint and then using that test
1:17blueprint to create assessment tools
1:19that will effectively measure learning
1:22and show student growth or
1:28gains let's start by discussing why and
1:31how to build pre- and post tests if
1:34you'll remember in the first video on
1:35this unit we talked some about the
1:37importance of using pre and post tests
1:40in your course in order to show uh and
1:44assess student learning progress let's
1:46take a look again at why we need to do
1:49this and how uh a testing blueprint uh
1:52can be
1:54helpful you want to be able to create a
1:56basis for comparison so what did your
1:59students know coming into the
2:01instruction or coming into the course
2:03and then what did they learn after
2:05instruction you want to be sure that
2:07your pre- and post tests measure the
2:11same content before and after
2:13instruction so you can see that learning
2:19growth again pre and post tests help us
2:22determine how much our students have
2:25grown you want to be sure that the
2:27assessments you use are the same level
2:30of difficulty each time it would not be
2:33fair to give students a more difficult
2:35test before or after uh or to cover
2:38different material or different content
2:41that you had not covered in class also
2:44you want to be sure that even though
2:46you're covering the same content you do
2:48not repeat questions on assessments
2:51creating an effective testing blueprint
2:54will help you do
2:57this so your you're going to start with
3:00a testing blueprint and you're going to
3:02use that testing blueprint for both
3:04assessments so you're not going to
3:06create two separate ones for each
3:07assessment no you're going to create one
3:10and from there write out the assessment
3:13questions that you'll want to use for
3:15both the pre and the post
3:19test so what is a test
3:22blueprint according to to Linda siski a
3:26test blueprint is an outline of the test
3:29that lists the learning goals that
3:31students are to
3:32demonstrate in other words it's a
3:34process or best practice strategy of
3:36being able to link assessments to
3:38learning goals so being sure that what
3:41you're testing students on what you're
3:44evaluating them on clearly aligns with
3:46your course goals and
3:51objectives here's a a look at an example
3:54test blueprint this is an example of a
3:57test blueprint for dental hygiene 101
4:00they use this test blueprint for both a
4:02pre and post assessment let's start by
4:04taking a look at the left hand column
4:06here program objective and learning
4:09outcome they've listed the general
4:12program objective an ability to apply
4:14knowledge of dental hygiene to personal
4:16care then they've listed the specific
4:20learning student learning objectives for
4:22this uh unit identify active ingredient
4:25in toothpaste explain why tee should be
4:27cleaned relate poor Dental Hy to pour
4:30overall
4:31health then you'll notice they've
4:34classified each objective by Bloom's
4:36taxonomy there are several taxonomies
4:38you can use choose the one that works
4:40best for you and your course they've
4:43identified knowledge comprehension and
4:46Analysis then they've decided to list
4:49the number of test items allocated to
4:52each objective for the first objective
4:55it is one second one two test items for
5:00the third objective which means that
5:03this per of the test is weighted more
5:06heavily than the others it will be
5:09important as you create a test blueprint
5:11to do this for yourself which objectives
5:13are more critical are going to be more
5:15highly weighted more important for your
5:17students to know um and provide the
5:20number of test items Point values and
5:22weight of the test uh allocate that
5:25accordingly at the end after all
5:28objectives are listed um and all uh
5:30weights have been determined your total
5:32should always uh equal to 100 and this
5:36is helpful for when you go to do the
5:38data
5:40analysis here's another short General
5:43example of a testing blueprint that sort
5:46of breaks things down a little bit
5:48further so on the left hand side you
5:50still have your student learning
5:51objectives which they've just broken
5:53down generally to SL1 SL2 Etc and so
5:56that teacher probably knows specifically
5:59what those objectives are it'll be
6:01helpful for you to actually list them
6:02out however unless you provide a
6:04key in the second column here the
6:07percentage of the test dedicated to
6:09measuring that objective just like in
6:11the previous one that we saw they've
6:13weighted it 10% to the first three
6:16objectives and then wow look at that 50%
6:19for objective number four so we know
6:21that whatever this objective is it is
6:24highly valued in this course it is
6:26highly critical so half of the test is
6:29dedicated to that
6:30objective here they've broken it down
6:32further the other the first blueprint
6:34sample didn't do this where they are
6:36listing the types of test items so
6:39they've listed multiple choice matching
6:41ordering
6:43Etc then again here the number of
6:45questions so 10 questions with only one
6:48point each per question uh down to
6:51there's only five questions for student
6:53learning objective number four but they
6:55are 10 points each so students are well
6:58aware and the teach teacher is aware
7:00that this is a highly valued
7:02objective again they've indicated the
7:05level of difficulty or cognition
7:07knowledge down through analysis and then
7:09here they've added something that I
7:11think is really useful for you as you
7:12create pre- and post assessments they've
7:14listed the test question uh numbers that
7:17are allocated to that objective so
7:19questions 1 through 10 are student
7:21learning objective number one uh down to
7:24questions 21 through 25 student learning
7:26objective number three as you build two
7:29compar aable test forms it'll be useful
7:31for you if you do the same so for
7:33example your pre-test um objective
7:36number one uh questions 1 through 10
7:38measure objective number one uh post
7:40test same thing so if you can align the
7:43objectives with the actual question
7:44numbers it will help when it comes to
7:46the data
7:47analysis and finally here since it seems
7:50that they have administered a version of
7:52the test they've listed the percentage
7:53of students who answered correctly this
7:55is definitely optional something that
7:57we'll look at a little bit more closely
7:59uh in week two when we discuss data
8:01analysis again just sort of a general
8:04example of how you can set up your
8:07testing blueprint as you um start to
8:09create your pre- and post
8:13assessments so let's review how do you
8:15build a testing blueprint start with
8:18your curriculum outline make sure you've
8:20listed your goals on your objectives for
8:23that unit that you're going to use and
8:25that you're going to assess students
8:27on Define your content categories so
8:30consider the frequency and criticality
8:32of each and specify weights for each
8:35category remember the weights should
8:38always add up to
8:40100 determine the total test length
8:42length so how many questions in total do
8:44you want uh on the test then determine
8:48the number of questions allocated to
8:50each content area or to each objective
8:52using the proportions previously
8:54calculated so if you know you only have
8:5610 questions and one question is only
8:59going to be worth 10% then that should
9:01equal one question for that one
9:05objective again just another quick look
9:08at the same testing blueprint sample
9:10you'll see here that they've done all of
9:11these things they've listed the
9:12objectives the percentage dedicated to
9:14the objective in this case they've added
9:17the um the types of test items the
9:20weight the level of difficulty and then
9:23also they've added um which question
9:25numbers are allocated to that specific
9:28objective so this is really
9:29comprehensive testing blueprint sample
9:32and it is located on our course Moodle
9:34page if you'd like to take a closer look
9:35at
9:37it so finally let's talk about creating
9:41the assessment tools you'll want to be
9:44sure that you write several questions
9:45for each objective identified again
9:48depending on the percentage and the
9:50weight that you've given to that
9:52specific objective code the questions to
9:54their objective and then obviously
9:56classify the questions by Apparent
9:59difficulty blooms mosenthal or some
10:01other uh taxonomy that you have uh used
10:04in your classroom that works for you and
10:08then build two comparable test forms by
10:10comparable meaning that they are the
10:12same level of difficulty um follow the
10:15blueprint for both so if you said you're
10:17going to have five questions for student
10:19learning objective number one then be
10:21sure that you have five questions on
10:22both the preest and the poest post test
10:26so again you're using the blueprint to
10:27create two different different forms two
10:30different assessments uh but that cover
10:32the same exact
10:36content here one more last example of a
10:38sample test blueprint um as you can see
10:41here uh they've listed the objectives
10:44the percentage uh that allocated to that
10:47objective the number of questions per
10:49exam the total number of questions in
10:51that area the criticality so how
10:53important or how critical it is to the
10:56learning and to the course and then the
10:59skill level so they obviously have their
11:00own coding system uh for their course
11:04and I just wanted to show you something
11:05a little bit different sort of outside
11:06of Education uh to show you that it is
11:08used uh when creating assessments of all
11:13kinds finally when it comes to test
11:15Administration some basic things to
11:17remember you want to be sure that you
11:19build both the pre and post test forms
11:21to the same blueprint I cannot emphasiz
11:23this enough it's one blueprint two uh
11:26separate uh assessments on the same
11:30content make sure that they're as
11:32equivalent as possible without repeating
11:34the same question
11:36items try to create similar testing
11:39environments both times you want to
11:40ensure that the the environment with
11:42which students take the test um is is
11:45appropriate and conducive uh to taking
11:47an
11:48assessment and let students know that
11:51they should see a growth in their
11:52learning from the pre-test to the post
11:55test um if they have been engaged and
11:58actively learning in class and
12:00participating in the instruction um so
12:03show them that this is a great form of
12:05feedback not just for you but for
12:07themselves and so that they can see as
12:08well what areas uh still need
12:11strengthening and what areas still need
12:13growth and where they have uh achieved
12:15uh Higher Learning
12:18gains let's do a quick review take a
12:21minute in your head to answer the
12:22following
12:24questions what are pre-post assessments
12:27used for
12:30that's right to assess student learning
12:34growth what is a test
12:38blueprint you got it it's an outline
12:41that aligns our curriculum goals with
12:43our
12:45assessments how is a test blueprint
12:50created good job slightly longer process
12:53but yes you'll have to start with your
12:54curriculum outline and goals and then uh
12:58create the the actual test blueprint uh
13:01looking at the categories and waiting
13:02each category and identifying the level
13:05of difficulty and the number of
13:06questions allocated to each objective
13:09and then you go ahead and start writing
13:10your questions based on
13:13that what are a few things to remember
13:15about pre-post
13:18assessments right the questions on each
13:21assessment should be different but they
13:23should be uh created at the same level
13:25of difficulty administered under uh the
13:29similar circumstances and environments
13:31and Ed as a basis for comparison so
13:33students can see their learning
13:36growth go ahead and please follow the
13:38Links at the bottom of this YouTube
13:40video to see examples of test Blueprints
13:42and the tests created from them to help
13:44you get ideas when you start to create
13:46your own testing Blueprints and your own
13:48pre- and post assessments thanks and if
13:51you have any questions feel free to
13:53email me I hope this video and tutorial
13:55have been useful for you