Week 3

Week 3 - Automated Unit Testing and Debugging

Table of Contents

Due Dates & Links

To Read

Lecture Materials

Video Shorts

Notes from class

Monday Notes (1)

Monday Notes (2)

Wednesday Notes (1)

Wednesday Notes (2)

Note: Links will require you to log in as a UCSD student

Lab Tasks

As usual, we publish these ahead of time, but they aren’t guaranteed to be final until the start of lab on Wednesday.

This week in lab, you will find symptoms of bugs by writing tests, and then narrow down the actual bug.

Setup

Make a fork of this repository (it’s OK if it’s public):

https://github.com/ucsd-cse15l-w23/lab3

There are a few relevant files for us:

  • ArrayExamples.java
  • ArrayTests.java
  • ListExamples.java
  • LinkedListExample.java
  • FileExample.java

The files that end in Example or Examples have code in them with bugs for you to find – in ListExamples and ArrayExamples, all the methods have bugs.

  • In LinkedListExample at least one of the methods on LinkedList has a bug.
  • In FileExample, getFiles has a bug.

So many 🐛s!

The file ArrayTests.java has some tests for the methods in ArrayExamples.java. It uses a library called JUnit to run tests using methods called assertEquals and assertArrayEquals and other assert... methods. (Note: JUnit is already included in the repository so no need to install anything)

When we run this class with JUnit, it runs each method that has a @Test annotation on it, and reports the success or failure of the assert calls.

Since JUnit is an external library, it requires some extra work to compile and run. These two commands work well, and you should see output like the below when you run them:

MAC USERS:

local $ javac -cp .:lib/hamcrest-core-1.3.jar:lib/junit-4.13.2.jar *.java
local $ java -cp .:lib/hamcrest-core-1.3.jar:lib/junit-4.13.2.jar org.junit.runner.JUnitCore ArrayTests
JUnit version 4.13.2
..
Time: 0.006

OK (2 tests)

WINDOWS USERS:

local $ javac -cp ".;lib/hamcrest-core-1.3.jar;lib/junit-4.13.2.jar" *.java
local $ java -cp ".;lib/junit-4.13.2.jar;lib/hamcrest-core-1.3.jar" org.junit.runner.JUnitCore ArrayTests
..
Time: 0.006

OK (2 tests)
  • The * in the first command tells javac to compile all the .java files in this directory. It’s a shorthand for writing them all out, so it’s a useful notation to start using now whenever we have several Java files to compile.
  • The -cp command-line argument stands for “classpath”. Java uses this command-line argument to know where to look for classes. It takes paths separated by : / ; (dependent on your operating system), so the first place it will look is ., the current directory. After that, it will look for classes in the two .jar files in the lib directory. A .jar file is like a .zip file of a bunch of classes, and Java knows how to work with them.
  • While you might think the .. in the output below JUnit has something to do with a path, it’s actually printing a . for each test that runs. So if you run hundreds of tests, you can kinda watch the progress by seeing how the dots count up.

Task with your group: Try running the command of the operating system that isn’t yours. (For example, a MAC user trying the Windows command and vise-versa) Observe the output and discuss with your partner/group what happens and why?

Symptoms and Failure-inducing Inputs

Array Methods

The two tests we wrote for you in ArrayTests.java pass, but the two implementations of reverse methods in ArrayExamples.java have bugs! Write more tests in ArrayTests.java to demonstrate that the two methods, reverseInPlace and reversed, have bugs, and identify the bugs.

Write down in notes - For each, what was the failure-inducing input (the test)? What was the symptom (the output when the test failed)? What was the bug (the problem in the code)? Be as specific as possible when adding the following down to your notes:

  1. The code for your tests (copy it into the doc)
  2. The output corresponding to each symptom
  3. The code change you need to make to fix the bug

Checkpoint – After fixing the reverse methods, make a commit and push to your repository. It’s really useful to checkpoint your work this way; you will be able to see in the commit history each of the elements above as well. Week 1’s Key Definitions and Related Links has some basics about git, and this Github documentation describes pushing your changes to Github.

For averageWithoutLowest, you will do a similar process as above, making sure to note symptoms, failure-inducing inputs, and bugs. But, this time, your lab partner will brainstorm one of the tests for you to write, describe it to you at a high level, and then you will implement it. Complete these steps as described below.

  1. Write down in notes - Come up with a high level description of a test that your lab partner should implement (without using any code!). Also include: Which symptom/bug is this testing for? Why is this test useful?
  2. Implement and run the test devised by your lab partner

If you’re having trouble thinking of tests, try starting from the smallest possible inputs (an empty array), and then trying increasing sizes of arrays to structure your thinking.

List Methods

There are two methods in ListExamples, each of which has a bug. For this program, create a new file called ListTests.java with JUnit tests. In ListTests.java, write some tests to identify and demonstrate the bugs in ListExamples. You should carefully design your tests to get failing inputs, symptoms, and eventually identify the bugs. Keep in mind that most of the time we can find relatively small inputs that trigger the symptom!

Write down in notes – Symptoms, inputs, and bugs.

Checkpoint – Commit and push your changes, and don’t forget to add the new file! This is one of the most common mistakes I make with git that is annoying for my collaborators: I add a new file locally and forget to put it on Github for them to see!

Linked List Methods

The file LinkedListExample.java has an implementation of a linked list that is, in fact, buggy. We won’t tell you which method(s) have the bugs. Create a file called LinkedListTests.java and write tests in that file.

Note that for this case, your failure-inducing input requires a little more work to construct: you have to build lists and try out the methods. All of that work is part of the “failure-inducing input”, and it’s useful to have it all written down in code so that we don’t have to remember it or re-type it each time we want to run the test.

Write down in notes – Symptoms, inputs, and bugs.

Checkpoint – Commit and push your changes, and don’t forget to add the new file! This is one of the most common mistakes I make with git that is annoying for my collaborators: I add a new file locally and forget to put it on Github for them to see! This is repeated from last week.

Reflect, write down in notes – You’ve now had to create two new test files (one for ListExamples and one for LinkedListExample). What actions did you take in your editor to do this? How long did it take you? Could you or did you use copy/paste effectively to avoid lots of typing? Could you or did you use the up arrow in the terminal to “get back” earlier commands rather than typing them out again? Any other tricks you could use to make this more painless in the future?

Discuss with your group! The long-term goal here is to learn tips and tricks to take tasks that might be annoying 5-minute tasks and turn them into 30-second tasks. It really changes how you think about writing tests if the process of getting started takes less time!

File Methods

The file FileExample.java has an implementation of a method that works with the filesystem and, fortunately for our learning, it’s buggy.

This one is interesting because the definition of a “failure-inducing input” is trickier – this program won’t run without us having some files and paths available to try! There is a comment in FileExample.java showing an example file structure for this program. So the first thing you may want to do is create the file structure depicted in the comment in your repository! Then you can write tests that use FileExample.java. You can create this just using VScode’s “new file” and “new folder” buttons/options.

To help you read and understand the program check out the File documentation here:

https://docs.oracle.com/javase/8/docs/api/java/io/File.html

Write down in notes – Symptoms, inputs, and bugs. Remember that your input is not just the code, but also the test files you created.

Checkpoint – Commit and push your changes, and don’t forget to add the new files, including the files you created as test input! All these files need to be a part of the repository so that we can run the tests. (This is a particularly annoying one to realize you missed later – the test will fail because the input data doesn’t exist!)

Discuss – Out of all the bugs above, which was the most interesting bug you found? Have you ever made bugs like these yourself in your own programs? What about having JUnit tests written might be useful one or two weeks from now?

Bugs and Troubleshooting

Let’s test your knowledge of bugs and how to troubleshoot them!

You and your partner should design a bug that you think is not trivial to catch. Make sure you agree on what a failure-inducing input for it is. This could be a new method you make up, or an edit to one of the existing methods in the lab already.

Write down in notes - Then, with another pair who has also done this, swap the buggy programs you designed (you can copy-paste them into the notes doc to share). See if you can spot one another’s bugs, and catch them with a well-designed input. Were these easy or hard to catch?

Once both groups have found the issues, try again! Try to stump one another with plausible, but tough-to-find, bugs.

Share your favorites with your team and tutor. We’ll share a few of the best ones in class.

Lab Report 2 - Servers and Bugs (Week 3)

As with the first lab report, you’ll write this as a Github Pages page, then print that page to PDF and upload to Gradescope. There are 3 parts:

Part 1

Write a web server called StringServer that supports the path and behavior described below. It should keep track of a single string that gets added to by incoming requests. The requests should look like this:

/add-message?s=<string>

The effect of this request is to concatenate a new line (\n) and the string after = to the running string, and then respond with the entire string so far.

So, for example, after

/add-message?s=Hello

The page should show

Hello

and after

/add-message?s=How are you

the page should show

Hello
How are you

Show the code for your StringServer, and two screenshots of using /add-message.

For each of the two screenshots, describe:

  • Which methods in your code are called?
  • What are the relevant arguments to those methods, and the values of any relevant fields of the class?
  • How do the values of any relevant fields of the class change from this specific request? If no values got changed, explain why.

By values, we mean specific Strings, ints, URIs, and so on. "abc" is a value, 456 is a value, new URI("http://...") is a value, and so on.)

Part 2

Choose one of the bugs from lab 3.

Provide:

  • A failure-inducing input for the buggy program, as a JUnit test and any associated code (write it as a code block in Markdown)
  • An input that doesn’t induce a failure, as a JUnit test and any associated code (write it as a code block in Markdown)
  • The symptom, as the output of running the tests (provide it as a screenshot of running JUnit with at least the two inputs above)
  • The bug, as the before-and-after code change required to fix it (as two code blocks in Markdown)

Briefly describe why the fix addresses the issue.

Part 3

In a couple of sentences, describe something you learned from lab in week 2 or 3 that you didn’t know before.