CS50P - Lecture 5 - Unit Tests

0:25DAVID MALAN: This is CS50's Introduction to Programming with Python.

0:28My name is David Malan.

0:29And this is our week on unit tests.

0:31Up until now, we've been writing a lot of code,

0:34and you might have been testing your code by running your program,

0:37and passing in some sample inputs, and running it again, and passing

0:40in some sample inputs, or you might have been waiting

0:42for us to test your code instead.

0:44But it's actually much better practice to get into the habit sooner

0:47rather than later of testing your own code using code of your own.

0:51In fact, whether you're writing a personal project

0:53or working in industry, it's very common nowadays to not only write code

0:57to solve the problems that you want to solve,

0:58but also to write a little extra code to test the code that you wrote.

1:03And that's what we're going to focus on today, writing our own test so as

1:06to be all the more confident, all the more certain,

1:08that the problems we have been trying to solve are, in fact, solved correctly.

1:12So let's rewind a few weeks now to a program we wrote a while back,

1:17namely to calculate numbers.

1:20And specifically, we left off with this calculator

1:23on trying to compute the power of a number, like x squared

1:27or where x might be two or three or some other number as well.

1:31Let me go ahead and resurrect that file by going into my terminal window

1:34here and running again code of calculator.py.

1:38And let me go ahead and pick up where we left off way back when

1:42by defining a main function here.

1:44And then in my main function, I did something like this.

1:46I said, x equals int of input.

1:49And I ask the user, what's x, question mark?

1:52And then I immediately went ahead and printed out something like,

1:55x squared is, and then I passed in as a second argument

2:00to print the result of calling a function called

2:02square passing in that value, x.

2:04Now of course, I haven't yet implemented the square function.

2:08So let's define that as well.

2:10Let me go down a couple of lines and define square.

2:12And it takes an argument recall, a parameter that at the time

2:15I called n, for number, so I'll do that again, though I could technically

2:18choose any name for this variable.

2:21And I recall, did this.

2:22I returned n times n.

2:25And there were multiple ways to do this.

2:27Squaring a number is multiplying it by itself.

2:30So I could also use other syntax here, but this

2:32is what we ultimately settled on, and then

2:34recall that I ultimately called main in order to kick off

2:37the process of running this program.

2:39So just as a test manually, let me go ahead and run

2:41Python of calculator.py and hit Enter.

2:44What's x?

2:45Let's start with 2.

2:47x squared is 4.

2:48I think that's correct.

2:49So let's run it again, just for good measure.

2:51Python of calculator.py, let's type in 3 for x this time.

2:55X squared is 9.

2:57And I think that's correct.

2:58And I might be feeling pretty good at this point,

3:00and I go off and submit my code to a course,

3:02or I post it on the internet for others to use.

3:04But I haven't really methodically tested this code.

3:07And it's not necessarily the case that it works entirely.

3:10In fact, I haven't really considered a number of corner cases.

3:13I went with some pretty obvious numbers like 2 and 3, but what about 0?

3:17What about negative numbers?

3:18What about any number of other infinite numbers?

3:20We're not going to test an infinite number of inputs

3:23to this, because the program would never halt,

3:25but we should test some representative inputs ultimately.

3:28But before we do that, let's get into the habit of making sure

3:31that main isn't always called.

3:33Let's adopt this habit, again, of doing If__name__=="__main__",

3:43only then should we execute main.

3:45And I'm doing this now proactively, because I

3:48want to make sure that when I import my square function, perhaps

3:53from another library, from another file, treating it as though it's a library,

3:56I want to make sure that main is not just automatically called itself.

4:00Now what do I want to do from here now that I've

4:03modified this program as follows?

4:06Let's go ahead and write a completely different program whose sole purpose

4:09in life is to now test this program.

4:12So I've got my actual calculator and calculator.py.

4:15I've readied myself to call main conditionally

4:18so that I can safely import one or more things from this file in another file.

4:23What should that other file be?

4:24By convention, I'm going to create a file that's called test_,

4:28and then because the thing I'm testing is this calculator itself,

4:31let's call this file test_calculator.py.

4:34That's going to give me a new tab, in which I

4:36can write a brand new program whose purpose in life

4:39is now specifically to test that program,

4:41but really that program's specific functionality.

4:43Built into that program is the square function.

4:46Let's focus on testing that function.

4:49So how do I access that function in this program?

4:52Recall that I can import a function from another file

4:55as though it's a library of my own, a so-called module.

4:58So I'm going to do this.

4:59From calculator, import square.

5:03I could go ahead and just import square itself.

5:05But then I would have to prefix my use of square recall

5:09by saying calculator dot everywhere, and it's just a little cleaner

5:12to just import the one function.

5:14And now let me go ahead and do this.

5:16Let me go ahead and define a function called test square.

5:19This too is a convention.

5:21If you want to test a function called square, your function for testing

5:25should be called test_square.

5:27Or alternatively, you could do square_test,

5:30I'll adopt this convention here.

5:31Now what kind of tests can we do?

5:34I don't dislike the tests I ran earlier, testing x equals 2 and x equals 3.

5:38But every time I want to test my program previously,

5:41I would have to do that manually.

5:43And that's going to get tedious.

5:44It's not going to be easy for someone else to test it.

5:45And if I'm actually working in the real world,

5:47it would be nice if I could automatically

5:49have my program tested again and again by having some automated process run

5:54my own code.

5:54So let's do that and take the human ultimately out of the equation.

5:58So how might I go about testing the square function

6:00that I've now imported per line one?

6:03In my test square function, why don't I do this?

6:05If the result of calling square of 2 does not equal 4,

6:11why don't we go ahead and print an error message,

6:13because I know that in the real world, 2 squared should equal 4,

6:17so if square of 2 does not equal 4, there's a bug in my program.

6:21There's a bug in my function.

6:22I've made a mistake.

6:23So let me go ahead and print something like that so I or someone else

6:26knows 2 squared was not 4, for instance.

6:30So I could print out anything here.

6:31What should I maybe next test?

6:33Let's do more than one test.

6:34Let's say if the square of 3 does not equal 3 squared 9, then let's go ahead

6:40and print out that 3 squared was not 9.

6:43So I haven't done any more testing than I did earlier.

6:46But I've baked those two tests, x equals 2 and x equals 3, into my own code

6:52here, so I can now run those tests automatically, if you will.

6:55Now, it's not enough to just define a function called test square.

6:59I actually, if I want to run this function, need to call it somehow.

7:02And our convention for doing that is the same as always.

7:06In this file too, let me define main.

7:08And main's sole purpose in life is going to be to test square.

7:12And now at the bottom of this file, as before,

7:15let me go ahead and adopt my convention of if__name__=="__main__",

7:23then go ahead and call main.

7:26So a lot of this is just boilerplate.

7:27We've seen this before, defining a main function

7:29and calling a function to kick off some process,

7:32now adding the conditional at the bottom of the file

7:34to make sure I'm only conditionally calling main, just in case I import

7:37anything from this file elsewhere.

7:40So let's see.

7:41Let's go ahead and test my code now.

7:43Let me go ahead and run test_calculator Python and hit Enter,

7:47and nothing outputs.

7:49Nothing outputs.

7:50But I think it's OK.

7:53I think no output is good, because look at my test square function.

7:56I'm not printing anything if all seems well.

8:00So let's demonstrate as much by going back to my calculator,

8:03and let me break it.

8:04Let me introduce a bug.

8:05Maybe I didn't even get it right the first time.

8:07Maybe my code originally looked like this.

8:09I wasn't thinking.

8:10I forgot my squares.

8:11And so I thought that the square of a number is n plus n,

8:15instead of n times n, so a reasonable mistake to make,

8:18perhaps arithmetically.

8:19Let me now go back to my test calculator,

8:21which I'm not going to change, but I am going to rerun it,

8:24python of test_calculator.py.

8:26I'm going to cross my fingers here, but for naught, I'm

8:29going to see immediately that 3 squared was not 9.

8:33Now what is it?

8:35Let's see, when your tests fail, how can we put our finger on what's wrong?

8:39It's a little interesting that I completely broke my square function,

8:42and yet only one of these tests is failing.

8:45It looks like this test, lines 9 and 10, is fine,

8:49because I'm not seeing that output.

8:51But of course these two lines, this test,

8:54is failing, because 3 squared is not 9 when I'm using plus.

8:57So just to be clear here, why is my function only partially broken,

9:03just to be clear.

9:04Why am I seeing only I error instead of two,

9:07even though the square function is now mathematically broken?

9:11SPEAKER 1: Because 2 plus 2 is 4.

9:13DAVID MALAN: Yeah, it's as simple as that.

9:14I just got lucky that 2 plus 2 is the same thing as 2 times 2.

9:18So this is one of those corner cases, and this

9:20is why it's good to be in the habit of not just testing one thing,

9:22but test several and make sure you're covering your bases, so to speak.

9:25So I got lucky here.

9:27And that explains why I'm seeing only I error,

9:29even though the function itself is flawed, but let me propose that there's

9:32another way we could do this, because honestly,

9:35if I extrapolate from this simple example, running not just two tests

9:39but 3, or 4, or 10, or 20 tests, you can imagine that, my God,

9:45the code is going to get so much more complicated than the function itself.

9:48Already, look, in calculator.py, the function in question is two lines long.

9:53And yet in test_calculator, the code in question is five lines long.

9:58I've written more code to test my code than I actually wrote original code.

10:01So the fewer lines of code we can write when testing code,

10:05I think the more likely you and I are to do it,

10:08because it's going to be literally a little

10:09less work and just fewer opportunities for mistakes.

10:12So what's another approach I can take here?

10:15it turns out in Python, there is another keyword that we haven't yet used,

10:19which is this here, assert.

10:21Assert is a keyword in Python and some other languages as well

10:25that allow you to do exactly that, as in English, to assert

10:28that something is true, to sort of boldly claim that something is true.

10:31And if it is, nothing's going to happen.

10:34No errors are going to appear on the screen.

10:36But if you assert something in Python, and it is not true, that is,

10:40the thing you're insert asserting, a Boolean expression, is false,

10:44you're actually going to see some kind of error on the screen.

10:47So let's go ahead and try this new keyword as follows.

10:50Let me go back to my code here.

10:52And just to make it a little simpler, let

10:54me propose that I use this new keyword as follows.

10:58Let me simply assert that the square of 2 should equal 4.

11:04So I've changed my logic.

11:05Instead of checking for not equals, I'm now

11:07asserting very loudly that it should equal 4.

11:11And then on one additional line, let me do the other test,

11:13assert that the square of 3 equals equals 9.

11:17And that's it, no indented print.

11:21I'm just going to assert more simply these two

11:24things that I want to be true.

11:26Let me go ahead now, with calculator.py still broken.

11:29I'm still using plus accidentally, instead of multiplication.

11:33Let me go ahead now and run Python of test calculator.py,

11:37crossing my fingers as always, but it's not going to go well this time.

11:41A whole lot of errors seem to appear on the screen.

11:44And if I scroll up here for this traceback,

11:46we'll see that the thing that failed was this line here, assert square(3) == 9.

11:53Now unfortunately, when you're using the assert keyword,

11:55it's not terribly user friendly.

11:57It shows you the files and the line numbers involved,

11:59but it does show you the specific line of code that failed,

12:02the assertion that failed, so to speak.

12:04It's now kind of up to you and me to infer from this, wait a minute,

12:08why is the square root 3 not equal to 9?

12:11So it's not super user friendly, but honestly, it

12:13was half as much code for me to write.

12:14It's just two lines, instead of those previous four.

12:16But notice this little remnant down here.

12:19This was an assertion error.

12:21And we have seen errors before.

12:23We've seen errors before when we've made other mistakes in our code.

12:27And in the past, what was our solution for catching those errors?

12:34How do we catch errors that seem to resemble this,

12:37even though we've not seen this one before?

12:39SPEAKER 2: Try and except.

12:41DAVID MALAN: Yeah, in Python, we can use the try and except keywords

12:44to try to do something, optimistically, except if something goes wrong,

12:48do something else instead.

12:49So this is a step forward, in that I can at least catch this error.

12:53But it's going to be perhaps a step backward, in that I'm

12:55going to end up writing, I'll admit in advance, a little more code instead.

12:59So let me go ahead and try this.

13:01Let me go back into my code here.

13:02And instead of just asserting, blindly, let

13:05me go ahead, as Tola proposed, and try to do this first assertion,

13:10except if there is an assertion error, like we saw a moment ago, then go ahead

13:16and print out something more user friendly

13:18that explains what actually failed.

13:202 squared was not 4.

13:23And let me go ahead similarly and try to assert that the square of 3

13:28equals 9, except if there's an assertion error there, in which case,

13:33I'm going to print out, more user friendly, 3 squared was not 9.

13:37So I've taken a step forward, but also a step back,

13:39because now I have more code.

13:41But I have at least introduced assertions and exceptions

13:44in a manner consistent with how we've seen in the past.

13:46When something goes wrong, you actually see an exception raised.

13:50Let me go ahead and run this version of the program now instead.

13:53Python of test calculator.py, crossing my fingers,

13:57it's still failed, because I'm seeing output.

14:00But we're back to at least user friendly output.

14:02So that's at least progress in some way here.

14:05But it's, again, more code than might have been ideal.

14:09And in fact, if we continue this further, what if we actually

14:11want to add additional test cases here as well?

14:14It seems like we might end up writing way more code than would be ideal.

14:18For instance, I'm testing 2 and 3 now.

14:21I should probably test some negative numbers as well.

14:24So why don't I go ahead and add in, for instance-- let me go ahead

14:26and copy and paste this.

14:28Let me try to assert that the square root of negative 2 equals

14:32equals 4, which should be the case mathematically.

14:34And if not, let me go ahead and change this

14:36to say negative 2 squared was not 4.

14:39And let me go ahead and copy paste this again,

14:41test another negative number, just for good measure.

14:44Let's test the square root of negative 3, which should equal 9.

14:47But if it doesn't, let's go ahead and say that negative 3 squared was not 9.

14:53And just to think aloud here, what might be another good value to test?

14:56I've tried 2.

14:57I've tried 3.

14:58I've tried negative 2.

14:59I've tried negative 3.

15:00I can't try infinite numbers.

15:01But there's at least something that's a little

15:03different in between those values.

15:04Let's try 0.

15:050 is an interesting case too, just in case something might be wrong.

15:08And why 0?

15:09I'm just going with instincts here.

15:11Odds are positive numbers are generally going to behave the same.

15:14Negative numbers might generally behave the same.

15:160 might be a little anomalous.

15:18There's no science to it necessarily, but rather considering for yourself

15:23based on your own experience, what are the potential corner cases based

15:26on the function you're trying to test?

15:28I'm trying to test something mathematical,

15:29so I want to test representative values.

15:31So let me go ahead and paste in one more try except block.

15:34Let's assert that the square of 0 should equal 0.

15:38And if not, I'll say something explanatory, like 0 squared was not 0.

15:43Now if I go ahead and run this, Python of test_calculator.py, and hit Enter,

15:48now I see multiple errors.

15:50And this is interesting.

15:51It's a bit of a clue, because notice that some, but not all,

15:55of these assertions are failing.

15:57The 1 for 2 squared is apparently OK, as we noted earlier.

16:02Recall that 2 squared happens to be 2 plus 2.

16:05So that bug doesn't really throw off our test,

16:07but it's a good thing we tested for 3.

16:09It's a good thing we tested for negative 2 and negative 3,

16:11because all of those tests caught this error.

16:13The 0 test did not notice, because 0 squared is, of course, 0, but 0 plus 0

16:18is 0.

16:19So we're getting lucky or unlucky there, depending

16:22on how you view the glass as half full or half empty here.

16:25We at least by way of having multiple tests caught this mistake somehow.

16:30So it would be nice, though, if we weren't writing so much darn code here,

16:35because notice what I've done.

16:37I have try, except, try, except.

16:39I have all of these assertions.

16:41I have a main function.

16:42I have this if conditional at the bottom of my file.

16:45Honestly, who's going to want to write 31 lines of code

16:49now just to test a two line function?

16:51No one's going to write test code like this

16:53if we're all writing so much more code to do the actual testing.

16:57So people have solved this problem.

16:59If you are in the habit of testing your code a lot, or wanting to,

17:02if I'm in the habit of wanting to test my code a lot,

17:04if everyone else in the real world is in this habit of wanting

17:07to test their code, why don't we create tools

17:09that make it a little easier to do so?

17:11And in fact, there is a mechanism for doing

17:14this, whereby we can use a tool that's popularly called pytest.

17:17So pytest is a third party program that you can download and install

17:21that will automate the testing of your code, so long as you write the tests.

17:26But what's nice about this library and others

17:29like it is that it adopts some conventions so

17:31that you don't have to write as many lines of code yourself manually.

17:35They do some of that automatically for you.

17:38Now this is a third party library.

17:39There's other libraries for unit tests, so to speak,

17:42that is testing units of your code.

17:44Some of them come with Python itself.

17:46We're proposing that we look at pytest today

17:48because it's actually a little simpler than the unit

17:50testing frameworks that come with Python itself.

17:53And what do we mean by unit testing?

17:54Unit testing is just a formal way of describing testing

17:57individual units of your program.

18:00What are those individual units?

18:01They're typically functions.

18:02So unit tests are typically tests for functions that you have written.

18:07Now what does this mean in practice here.

18:09Let me go back to my VS code here and let

18:12me propose that we simplify my test calculator significantly.

18:17I'm going to go ahead and delete all of these tests, which were accumulating

18:22to like 31 lines of code.

18:24And let's see if we can distill the tests to their essence, using pytest.

18:28From my same calculator program, let me still import square.

18:32So I do still need that line of code so that I

18:34can test that's specific function.

18:35Now I'm going to go ahead and define a function, just like I did before,

18:39as follows.

18:39I'm going to define a function called test square, again

18:42by convention, test underscore and the name of the function you want to test,

18:46though it doesn't have to be that way.

18:47And now I'm going to go ahead and make a few assertions.

18:50I'm going to assert that the square of 2 should equal 4.

18:53I'm going to assert that the square of 3 should equal 9.

18:57I'm going to assert that the square of negative 2 should equal 4.

19:01And I'm going to assert that the square of negative 3 should equal 9.

19:06And lastly for now I'm going to assert that the square of 0 should equal 0.

19:10So I'm still using the assert keyword, as I introduced earlier.

19:14And even though it was a little tedious to type those,

19:17it's only eight lines of code now.

19:18And they're so easy to type.

19:20It's not try and except and all of this.

19:22Wouldn't it be nice if something else, someone else,

19:26handled the try, the except, the printing, all of the standardization

19:31of actually running these tests?

19:33And that's where, indeed, pytest comes into play.

19:36Per the documentation for pytest, which can itself be installed with pip

19:40install pytest, which we've used to install other libraries in the past,

19:44you can look at the documentation here for all of its formal usage.

19:47But fortunately, pytest is pretty user friendly, as testing frameworks go,

19:51and it actually allows us to dive right in by just running pytest on the code

19:55that we've written.

19:56So if I go back to VS Code here and look at my test_calculator.py, which,

20:00notice, has no main function anymore-- it has no conditional.

20:04It has no tries.

20:05It has no excepts.

20:06It has no prints.

20:07It just has my few assertions-- pytest and other libraries

20:11like it are going to automate the process of running these tests for me

20:14and informing me on the screen whether or not any of those tests failed.

20:20So let me go ahead and do this.

20:21I'm going to go ahead and increase the size of my terminal window

20:24for a moment, just so we can see more on the screen.

20:26And I'm going to run not python, as I've been doing.

20:29I'm going to run pytest, which, again, is this third party

20:32tool for running tests in your code.

20:35I'm going to run pytest of test_calculator, so that same file.

20:39I'm going to cross my fingers as always and hit Enter,

20:42and we'll see that something has failed.

20:46Now admittedly, even though I do think you'll

20:48find that pytest is relatively simple to use,

20:51it's output, at least at first glance, is not necessarily super user friendly.

20:55So what are we seeing here?

20:56Notice at the very top of my window is the command that I ran after my prompt.

21:01Right below that is a single F in red, which means fail,

21:05so not very encouraging.

21:07I tried really hard here, but fail is my grade on this program.

21:10But let's see exactly what happened.

21:12If I look at this excerpt here under failures,

21:15you'll see that test square is the function that failed.

21:18That makes sense, because that's the only one I wrote.

21:20And you'll see here somewhat arcane output describing what the error was.

21:25So what you're seeing here is the first line of output equals equals 4,

21:28which is fine.

21:29There's no red error message below that, so that one's OK.

21:32But this line of code here assert that square of 3 equals equals 9,

21:36pytest did not like that assertion, because it didn't end up being true.

21:40In fact, per the red E at the start of this line,

21:44you'll see that I'm effectively trying to assert that 6 equals equals 9.

21:50Now, where did the 6 come from?

21:52Wait a minute, if my test involves this, notice that where 6 equals square of 3,

21:56this is saying that because I've called square, passing in a value of 3,

22:01it turns out it's return value is 6.

22:03And of course, mathematically, 6 does not equal equal 9.

22:07So that's why this is failing.

22:09Now, pytest is not as user friendly as telling you

22:13exactly why the bug is there or how to fix it.

22:16This is really just a clue to you what must be wrong.

22:19What you're seeing here is a clue that the first test passed,

22:23because there's no red error below that line of code, but this test failed.

22:26Somehow or other, your square function is returning 6

22:32when passed in 3 instead of 9.

22:34So at this point, you sort of put your detective hat on,

22:37you go back to your actual code, and you think

22:39about in calculator.py, how in the world is

22:42line 7 of my square function returning 6 instead of 9.

22:47And at this point, odds are the light bulb

22:49would go off above your head proverbially,

22:51and you would see, I'm using addition, instead of multiplication.

22:55But what pytest has done for us is automate

22:57the process of at least pointing out that error for us.

23:00And if I now go in and fix this-- let me go ahead,

23:03and the light bulb has gone off.

23:04I change the plus to a multiply.

23:08Now I'm going to go ahead, and after clearing my screen,

23:10I'm going to run not Python, but pytest of test_calculator.py,

23:15crossing my fingers again.

23:16And now it's green.

23:17And I see just a dot, which indicates that my one and only test passed.

23:21I'm good, 100% success with my test now after fixing that bug.

23:27Let me pause here and see if there's any questions.

23:30SPEAKER 3: So my question is, what if a user,

23:33instead of, because we are taking input from the user,

23:36what if the user is somewhat malicious and types in a string instead

23:41of an integer, or maybe he types in a float or some other data type?

23:46DAVID MALAN: Yeah, so what if the user, like we've

23:48seen in past examples, types in cat, instead of a number, when

23:51we're expecting an integer?

23:52How do we test for something like that?

23:54At the moment, I'm admittedly not testing user input.

23:57If I go back to my code here, notice that my calculator function, of course,

24:02has the square function that we keep testing and retesting.

24:05But notice that all of the user input is currently

24:08relegated to my main function.

24:10And admittedly, as of now, I am not testing my main function.

24:14So there could be one of those bugs.

24:15And in fact, there would be, because if the user types in a string, like cat,

24:19instead of an integer, like 2 or 3, then line two recall would actually

24:24raise a value error exception.

24:26So we've seen that before.

24:28So when it comes to testing your code, this

24:30is actually a good reason for having multiple functions in your program.

24:35Rather than putting all of your logic in just the file itself,

24:37rather than putting all of the logic in just main,

24:40it's actually really good, really helpful practice

24:43to break your ideas up into smaller bit-sized functions

24:46that themselves are testable.

24:48And what do I mean here?

24:49Square is perfectly testable.

24:52Why?

24:52Because it takes as input a parameter called n,

24:56and it returns as output in integer, which is going

24:59to be the square thereof, hopefully.

25:01It has a well-defined input and a well-defined output.

25:04It is therefore completely within your control in your test program

25:08to pass in those values.

25:09Now I will say, if you want to test whether square behaves properly

25:15when passed something like a string, like, quote, unquote,

25:18"cat," we could absolutely do something like this,

25:20assert that the square of quote, unquote, "cat,"

25:24it's not going to equal something.

25:26You can actually, using different syntax,

25:28assert that a specific exception will be raised.

25:31So if we were actually going to go back into our square function,

25:34improve it, and deliberately raise an exception, we could test for that too.

25:37But for now, I'm deliberately only testing the square function.

25:41I'm not testing for specific user input.

25:43But that's another problem to be solved.

25:45Other questions now on unit tests?

25:49SPEAKER 4: Do use the unit test to test code for the CS50 check?

25:56DAVID MALAN: So Check 50 is similar in spirit.

25:58Check 50 is a tool that we, CS50, wrote that is essentially doing something

26:03like pytest for the evaluation of students' code.

26:06It is similar in spirit, but think of Check 50

26:10as being an alternative to pytest, if you will.

26:12But it works a little bit differently.

26:14But same idea, pytest and unit testing more

26:17generally is a technique that is independent of CS50

26:19and is something that you can and should be doing on your own code, both in

26:23or outside of this class.

26:25How about one other question here on our unit tests?

26:31SPEAKER 5: My question is that is instead

26:33of writing four times, like as a square of, 2 squared 4,

26:37instead of that, can we write equals to in square brackets the numbers we want,

26:43instead of writing four lines?

26:44DAVID MALAN: A really good question, absolutely.

26:46Right now if I go back to test_calculator.py,

26:49it's indeed pretty manual.

26:51It took me a while to say and to type out those several lines,

26:54and you could imagine writing some kind of loop to just assert in a loop

26:58that this equals that, that this equals that, and so forth, using a list

27:02or using maybe a list or a dictionary or some structure like that.

27:05So yes, you can absolutely automate some of these tests

27:08by not just doing the same thing again and again.

27:10You can still use all of the syntax of Python to do loops.

27:12But generally speaking, your tests should be pretty simple.

27:16And in fact, let me propose that we improve upon even this design further,

27:21because at the moment what's not really ideal, when I run all of these tests

27:28when my function is buggy, is notice the output that I got.

27:32Let me reintroduce that same bug by changing my multiplication back

27:35to addition.

27:36Let me increase the size of my terminal window again.

27:39And let me run pytest again of test_calculator.py.

27:42So this is the version of my code now that has the bug again.

27:46So I'm going to see that big massive failure where

27:49this failure has been displayed to me.

27:52But this is not as helpful as it could be,

27:55because I have all of those other tests in my code.

27:58Recall that I had, what, one, two, three, four, five separate tests,

28:01and I'm only seeing the output of the first.

28:03Now, why is that?

28:04If we go back to my code here, you'll see

28:06that the first assertion that's failing, namely this one here, that assert

28:11of square of 3 equals equals 9, the other tests aren't even getting run.

28:15And that's not a big deal in the sense that my code is buggy, so one or more

28:19of them are probably going to fail anyway,

28:21but wouldn't it be nice to know which of them are going to fail?

28:24And in fact, it's ideal to run as many tests all at once as possible

28:27to give you as many clues as possible to finding your bug.

28:31So let me propose that we improve the design of my testing code

28:35now, still using pytest as follows.

28:38Instead of having one big function called test_square

28:41that tests the entire function itself with so many different inputs,

28:45let's break down my tests into different categories.

28:48And here, too, there's no one right way to do this.

28:51But my mind is thinking that I should maybe

28:53test positive numbers separately, test negative numbers separately, and test 0

28:57separately.

28:58I could think of other ways.

28:59I could test even numbers.

29:00I could test odd numbers or maybe some other pattern altogether,

29:03but separating this big test into multiple tests

29:07is probably going to yield more clues for me when something goes wrong.

29:10So let me do this.

29:11Let me go ahead and rename this function to test positive initially,

29:15and let me include in that function only those first two tests.

29:19Let me then create another function here called test negative.

29:23And in this function, let me test only negative 2 and negative 3.

29:27Then down here, let me do one more def of test_zero,

29:31and I'll just run one test in there.

29:33So I have the same assertions, the same five,

29:36but I've now divided them up among three separate functions.

29:39What's nice about pytest and other unit testing frameworks

29:43is that all three of these test functions will be run automatically.

29:47Even if one of them fails, the others will be attempted.

29:50That means that if one or two or three of them fail,

29:53I'll have one or two or three clues now for helping me find that mistake.

29:58So let me go ahead and again increase the size of my terminal window,

30:01just so we can see more on the screen.

30:02My calculator still has the bug, using addition, instead of multiplication.

30:07Let me go ahead and run not Python, but again, pytest of test_calculator.py,

30:12crossing my fingers as always, and now, oh my God,

30:14there's even more errors on the screen.

30:16But this in itself is more helpful.

30:19Let's work through them from top to bottom.

30:20So under FAILURES here, in all caps, which

30:22I know is not very encouraging to see failure when you're just

30:25trying to solve a problem, but that's what these frameworks do,

30:28under FAILURES, the first function that failed is test_positive.

30:31But here, too, we see the same clue as before.

30:34The first one, 2, the square of 2 equals equals 4, that one is fine.

30:38It's not erring with any red errors.

30:40But the next one is failing.

30:41So I know that square is broken when I pass in 3.

30:45What about down here?

30:46It looks like, unfortunately, my test negative function is failing too.

30:49Why?

30:49When I pass in-- oh, this is interesting-- here now, negative 2

30:53doesn't even work.

30:55So I got lucky with positive 2.

30:56But negative 2 isn't working.

30:58So that's a bit of a clue.

30:59But in total, only two tests failed.

31:02So notice at the very bottom, this summary, two failed and one passed.

31:07What's the other one?

31:08What was the third one?

31:09Test zero.

31:10So test zero is passing.

31:11These two are failing.

31:13And so that kind of leads me logically, mathematically, if you will,

31:17to the source of the bug.

31:18And just to be clear too, if you have a lot of tests,

31:20this little one line output is helpful, even though also a bit discouraging,

31:23fail, fail, and dot means pass.

31:27So there are the three tests just depicted

31:28graphically a little bit differently.

31:31Let me rewind now and go back in to calculator.py.

31:35Let's fix that bug, because let's suppose

31:38that I've deduced I'm using addition.

31:40I should have been using multiplication all this time.

31:42Let me now after fixing the bug yet again,

31:44let me go back to my big terminal.

31:46Let me run pytest of test_calculator.py, hitting Enter, crossing my fingers now,

31:51and dot dot dot means all is well.

31:53100% of my tests passed, all three of them.

31:56So now I'm good.

31:57It doesn't necessarily mean that my code is 100% correct.

32:02But it does mean that it has passed 100% of my current tests.

32:05And so it would probably behoove us to think a little harder about maybe

32:10we should test bigger numbers.

32:11Maybe we should test even smaller numbers.

32:13Maybe we should test strings or something else.

32:15The onus is ultimately on you to decide what you're going to test.

32:19But in the real world, you're going to be very unhappy with yourself

32:22or someone else-- maybe your boss is going to be very unhappy with you--

32:25if you did not catch a bug in your code, which you could have caught

32:29had you just written a test to try that kind of input.

32:33Let me pause again and see if there's any questions now

32:35on unit testing with pytest.

32:38SPEAKER 6: So if you wanted to test, like someone suggested before,

32:41user input as well as testing your function,

32:45do you do that within the same file?

32:47Or do you make separate files for different types of tests?

32:50DAVID MALAN: Really good question.

32:51You could absolutely make separate files to test different types of things.

32:55Or if you don't have that many, you can keep them all in the same file.

32:58At the moment, I've been storing all of my tests in one file for convenience,

33:01and there's not terribly many of them.

33:03But we'll take a look in a bit at an example

33:05that allows me to put them into a folder and even run pytest

33:08on the whole folder of tests as well.

33:11So that's possible.

33:12Other questions on unit testing.

33:14SPEAKER 7: So I've got two questions.

33:16So a couple of while ago, you just used an exception called--

33:22I'm not sure what it was-- oh yeah, assertion error.

33:26What exactly does that particular error catch?

33:30And my second question is, does the assert keyword

33:36stand out to the compiler, exactly tell them

33:39to insert this particular line of code?

33:42DAVID MALAN: Indeed.

33:43The assert keyword we're seeing and the assertion error we saw earlier

33:48are intertwined.

33:49So when you use assert and the assertion fails,

33:52because whatever Boolean expression you're using is not true, it's false,

33:56an assertion error, by definition of Python, will be raised.

34:00So those two work in conjunction.

34:02Those errors, those assertion errors, are still being raised by my code

34:06here when any of these lines of code fail.

34:09However, pytest, this third party library,

34:12is handling the process of catching those exceptions automatically for me,

34:16so as to give me this standard output.

34:18So we started today's story by really implementing unit testing myself.

34:22I wrote all of the code myself.

34:23I wrote main.

34:24I did my conditional.

34:25I did try and except.

34:26Honestly, it's going to get incredibly painful to write tests

34:29long term if you and I have to write that much code every time, especially

34:32when our function is this small.

34:34So pytest and unit testing frameworks like it just automate so much of that.

34:38Essentially, pytest adds the try, the except, the if, the prints for you,

34:43so you can just focus on the essence of the test, which

34:46really are these inputs and outputs.

34:49How about time for one other question here on unit testing as well?

34:52SPEAKER 8: So when we enter minus x or minus 5 squared,

35:00square root of that number comes up.

35:03But when we put 6.6 or 5.6, something like that integer,

35:07then line shows error.

35:11So what's happening there?

35:13DAVID MALAN: So I'm deliberately testing integers right now,

35:16in large part because I only want pow to operate on integers.

35:19And that might be conveyed in Python's documentation or my own documentation

35:23for that function.

35:24If you were to pass in something else, like a float,

35:26it turns out that floating point values in Python and other languages

35:30are actually very hard, if not impossible,

35:33to represent 100% precisely.

35:35And so if you are trying to compare it against some other value,

35:39there might be slight rounding errors as a result.

35:41I'm just inferring from what you've described,

35:43but I'm very deliberately now testing this function with only the inputs

35:47that I would expect.

35:48It might indeed throw other errors if other inputs are passed.

35:53Allow me to propose that we consider what should happen if square

35:56isn't actually passed a number.

35:58For instance, if I go back to calculator.py,

36:01and suppose that I, or perhaps someone else using my square function,

36:04simply forgets to convert the return value of input from a str to an int,

36:09as by modifying line to here.

36:11Now, something's definitely going to go wrong if I type in a str

36:14instead of what appears to be an int.

36:16For instance, if I clear my terminal here,

36:18run Python of calculator.py and hit Enter--

36:22let's type in cat as our value for x-- and of course,

36:26this raises now a type error.

36:27Why?

36:28Can't multiply sequence by non-int of type 'str.'

36:30What does that mean?

36:31You can't do cat times cat, because indeed, square is

36:35expecting that end will be some number.

36:36But that doesn't necessarily mean that square itself is buggy.

36:39But this does mean that if I expect a type error to be raised,

36:43let's test for that too, so that I know the behavior indeed works as expected.

36:47So let me go back to test_calculator.py, and let me go in add a fourth test down

36:53here.

36:53How about define test underscore, and I'll

36:56call this test_str, because I'm going to specifically and deliberately pass

36:59in a str for testing.

37:01And I want to in spirit assert that passing in something like cat to square

37:06will raise a type error.

37:08But we don't use the assert keyword for that.

37:10Rather, we need this.

37:11Let me go to the top of this file, and let me additionally

37:14import the pytest library itself, because it turns out

37:18there's a function in that library called

37:20raises that allows me to express that I expect an exception to be raised.

37:25And I can express that as follows with pytest.raises,

37:29and then in parentheses I can pass in the type of exception I expect,

37:33which is going to be a type error in this case.

37:35And now when do I expect that type error to be raised?

37:38Whenever I do something like calling square and passing in not a number,

37:42but something like cat.

37:44So now if I go back to my terminal window,

37:46run pytest of test calculator.py, this time having four tests,

37:51I should see that all four now are successful.

37:55Let's now consider how we could test code that doesn't just expect numbers

37:59as input, but actually strings.

38:02And let me rewind us in time here in VS Code

38:04to that very first program we wrote a few different versions of in hello.py

38:09that ultimately looked a little something like this.

38:11I had a main function that prompted the user

38:14for the value of a variable by asking them, "what's your name?"

38:18question mark.

38:19And then we went ahead and did something like hello,

38:21open paren, name, passing that user's name into a function called hello.

38:26Now that function hello recall ultimately looked like this.

38:30We defined hello as taking a parameter called to,

38:33the default value of which was world, and that function very simply

38:37printed hello, followed by a comma, and then whatever

38:41the name that had been passed in.

38:43And then we ultimately called main, but for now onward,

38:46I'm going to always add this if conditional,

38:48if name equals equals underscore underscore main, then and only then

38:53do I want to call main.

38:54So that's essentially what this program looked like in its last incarnation.

38:58How do we go about testing it?

39:00Here again too, I'm not going to test the user's input per se in main.

39:03I'm going to focus really on the module of code

39:07here that's of interest, which is the hello function itself.

39:10How can I go about testing the hello function?

39:14Unfortunately, even if I start by doing something like code of test hello.py--

39:19let me go about and start writing a test program--

39:22I could import from my hello program a function called hello.

39:26So a bit strange to see from hello import

39:28hello, but notice that on this line here, I'm importing from the module--

39:32that is the file called hello.py-- the function called hello.

39:36And how do I go about testing this?

39:40If I have a function like define test_argument like this--

39:46let me do this.

39:48So if I were to define a function like define test_hello, what could I do?

39:53I could call hello with quote, unquote, say, "David,"

39:59and then check if it equals, what, "hello, David."

40:04So would this work, this approach here?

40:07If I've written a test, called test_hello, that

40:10calls hello with an argument of David and then tests its return value,

40:14just like we've done for our calculator, would this work as written?

40:19And let me go back to in just a moment the version

40:22of hello that we're testing.

40:23So you can see that function hello.

40:25Here's the test.

40:27Here is the actual code.

40:29Would this test now work?

40:32Any thoughts?

40:34SPEAKER 9: I think the problem is that in the first version in hello.py,

40:38you're using the to argument that you first declared, when you declared

40:42the function instead of using the name.

40:47DAVID MALAN: That is actually not a bug here.

40:50So let me stipulate that in hello.py, this code actually

40:53does work as intended.

40:54And let me go ahead and test it manually, just to demonstrate as much.

40:57Let me run Python of hello.py, typing in, as my name, D-A-V-I-D, and I see,

41:03in fact, that it says, "hello, David."

41:05If, though, I were to change this program,

41:07and get rid of the name argument, get rid of the name variable,

41:11and just call hello, again, running Python of hello.py,

41:14this time I'm not even prompted, because I got rid of my input call,

41:17but it does behave as I expect.

41:19It does say "hello, world."

41:21So let me stipulate that this code in its current form is actually correct,

41:26but my test is not going to work as I'd hoped.

41:30And there's a subtle difference between my hello function

41:38and my square function that explains.

41:41Why might this test not work as intended?

41:45SPEAKER 10: Because it's not returning a value.

41:47DAVID MALAN: Yeah, exactly.

41:48Recall our discussion early on about functions.

41:50Functions can either return a value, like my square function hands

41:54you back the square of some value, or they

41:56can have side effects, sort of visual artifacts

41:59that might happen on the screen, like printing something out on the screen.

42:02And by definition, that's how print works.

42:05Notice that hello, it is short, but it's implemented ultimately

42:08using the print function, which does not return a value as I'm using it here.

42:12It instead has this side effect of printing something onto the screen.

42:15So it is not correct in my test function to check

42:19if the return value of hello equals equals hello David,

42:23because again, hello is not returning anything.

42:26It's printing something, that side effect,

42:28but notice, literally, it has no return keyword,

42:31unlike my square function, which did.

42:34So here's an opportunity to perhaps change

42:37how I go about implementing my actual functions.

42:41It turns out that as your programs get more and more sophisticated, more

42:44and more complicated, it tends to be best practice not

42:47to have side effects if you can avoid it,

42:50especially if you want your code to be testable.

42:52And in fact, I'm going to propose that we change my hello program to now work

42:56as follows.

42:57Let me go ahead and change this function to not print hello and then that name.

43:03Let me go ahead and literally return maybe

43:05an F string, which will clean this up a little bit, hello comma

43:09to close quotes at the end.

43:11So my syntax here is just the familiar f string or format string.

43:15It's going to return hello, world or hello, David or hello, whomever's name

43:19is passed in as that argument, but I'm returning it now.

43:23I'm not printing it out.

43:24So what needs to change up here?

43:27I could do something like this.

43:29I could say something like output equals hello

43:33and then print output in my main function.

43:35Or I can simplify that, because I don't really need that variable.

43:38I could instead just do this.

43:40I could still call hello, but I could immediately print out the result.

43:44And this version of my hello program now is actually more testable.

43:49Why?

43:50Because these assert statements that we're using,

43:52and we've seen thus far for our tests, are really

43:54designed to test arguments into functions and return values

44:00they're from, not testing side effects.

44:02So if you're doing equals equals, you're looking for a return value, something

44:05that's handed back from the function.

44:07So that's fine.

44:08If I modify the design of my program now not to just print hello,

44:11but to return the string, the sentence, the phrase that I want to construct,

44:17I can leave it to the caller--

44:19that is the function who's using this hello function--

44:22to handle the actual printing.

44:24Now what does this mean in my code?

44:25It means now if my hello.py looks like this,

44:28and hello is indeed returning a value, in my test_hello function,

44:33I can test it exactly like this.

44:35So let me go ahead and run pytest of test_hello.py,

44:38crossing my fingers as always, and voila, one passed.

44:42So I passed this test, because apparently the return value of hello

44:45does indeed equal "hello, David."

44:48Let's test the other scenario.

44:49What if I call hello without any arguments?

44:53Let's assert that calling hello with nothing in those parentheses

44:56similarly equals hello comma, but world, the default value.

45:00Let me now go ahead and run pytest of test_hello.py.

45:04And that too passes entirely.

45:07But there too, suppose that I had made some mistakes.

45:09Suppose that there were a bug in my code.

45:12It might not be best practice to combine multiple tests in this one function,

45:15so let's make it more clear what might pass or fail.

45:18Let's call the first function test the default to this function.

45:22And let's only include this first line of code.

45:24And then let's go ahead and define another function, like test_argument,

45:28to test this other line of code here.

45:30So now I have two different tests, each of which

45:32is testing something a little fundamentally different.

45:35So now when I run my code, it's still not broken.

45:38If I run pytest of test_hello.py, Enter, I've now passed two tests.

45:43And that's just as good as before.

45:45But if I did have a bug, having two tests instead of one

45:49would indeed give me, perhaps, a bit more of a hint as to what's wrong.

45:54Questions now on this testing of return values,

45:57when these return values are now strings instead of integers

46:00and why we've done this?

46:02SPEAKER 11: So my question is about function inside the function.

46:07Can we test that too or recursion we haven't seen?

46:14DAVID MALAN: If you have a recursive function, which we've not

46:17discussed in this class, yes, you can absolutely

46:19test those too by simply calling them exactly in this way.

46:23Recursion does not affect this process.

46:25How about one more question here on unit tests

46:27before we look at one final example?

46:29SPEAKER 12: When testing our arguments, can we

46:34use something like loops or inside of assets or for the values?

46:41DAVID MALAN: Absolutely.

46:42You can absolutely use a loop to test multiple values.

46:45In this case, for instance, I could do something like this.

46:48I could say for name in the following list of Hermione, say, Harry, and Ron,

46:57I could then within this loop assert that hello of that given name equals

47:02equals, say, the format string of hello, comma name,

47:08and then run all of these here at once by running, again,

47:13pytest of test_hello.py.

47:15It's still going to be just one test within that function,

47:17but if there's something interesting about those several strings that

47:20makes it compelling to test all of them, you can absolutely

47:23automate the test in that way.

47:24With that said, each of your tests should ideally

47:27be pretty simple and pretty small.

47:30Why?

47:30Because you don't want to write so much code,

47:32so much complicated code that your tests might be flawed.

47:35What we don't want to have to do is write tests for our tests and test

47:38for our tests for our test, because it would never end.

47:41So keeping tests nice and simple is really the goal,

47:44so that a reasonable human, yourself included,

47:45can eyeball them and just claim, yeah, that is correct.

47:49We don't need tests for our tests.

47:51How about one other feature?

47:53Suppose that we don't have just one test, but many different tests instead,

47:56and we want to start to organize those tests into multiple files and even

47:59a folder.

48:00Pytest and other frameworks support that paradigm as well.

48:03In fact, let me go ahead and test hello.py using a folder of tests,

48:08with technically just one test, but it would

48:09be representative of having even more in that folder.

48:12I'm going to go ahead and create a new folder called test

48:15using mkdir at my command line.

48:20And then within that folder, I'm going to go ahead and create a file called

48:23test_hello.py.

48:25Within this file, meanwhile, I'm going to test the same thing.

48:28So I'm going to go ahead, and from hello, import hello.

48:31And I'm going to go ahead and define a function like test default that

48:36simply tests the scenario where hello with no arguments

48:39returns hello, comma world.

48:41And I'm going to have that other function where

48:43I test that an argument is passed.

48:45And in this case, I'll choose an argument

48:47like asserting that hello, quote, unquote, David,

48:50equals, indeed, hello, comma, not world, but David.

48:54So in this case, I've just recreated the same test as earlier,

48:57but they're in a file now in a folder called test.

49:01Pytest allows me to run these here too.

49:03But to do so, I actually need to create one other file.

49:06Within my test directory, I need to create a file called __init__.py,

49:14which has the effect, even if this file is empty,

49:18of telling Python to treat that folder as not just a module, but a package,

49:24so to speak.

49:25A package is a Python module or multiple modules

49:28that are organized inside of a folder.

49:30And this file, __init__.py, is just a visual indicator to Python that indeed

49:36it should treat that folder as a package.

49:39If I had more code in this folder, I could do even more things

49:41with this file.

49:42But for now, it's just a clue that it's indeed

49:44meant to be a package and not just a module or file alone.

49:48What I can now do in closing is run pytest, not even on that specific file,

49:53but on a whole folder of tests.

49:55So if I run pytest of test, where the test is the name of that folder,

50:00pytest will automatically search through that folder looking

50:03for all possible tests, granted there's just those two in this one file,

50:07but when I run it now with Enter, I'll still pass those tests.

50:11I'll still get 100%.

50:12And I now have a mechanism, ultimately, for testing my own code.

50:16So whether you're writing functions that return integers or something else,

50:19functions that have side effects that could be rewritten as functions

50:22that return values, you now have a mechanism

50:24to not just wait for, one, someone like us to test your code

50:27and not just test your code manually again and again, which

50:30might get tedious, and you might make mistakes

50:32by not including some possible inputs, we now

50:34have an automated mechanism for testing one's own code that's

50:37going to be even more powerful when you start collaborating with others

50:41so that you can write tests that ensure that if they

50:44make a change to the same code, they haven't broken the code that you've

50:47written.

50:48That's it for this week.

50:50We'll see you next time.