Link Search Menu Expand Document
CSCI 1600, Fall 2023

Testing and debugging

Last day for on-time checkoff: Tuesday, 10/31

Prelab due 10/23/23 at 11:00 am

Writeup due 10/30/23 at 11:00 am

About

In this lab, you will come up with test cases to test your lab 5 code, based on a reference FSM. You will run these test cases and use the tests to debug your code. You will then be asked to reflect on the assumptions we made in testing. By the end of this lab, you will have seen how FSM-driven design influences unit testing for an embedded project.

Lab 6 Rubric

Resources

Materials

Same circuit as for Lab 5.

Steps

  1. Make sure that you still have your circuit and code from Lab 5, and follow these steps to set your code up for testing:

    1. In lab5.h, add the following line to the top of the file:

       #define TESTING

    2. In lab5.ino, comment out everything in your loop() function, and add the following line to the end of your setup() function:

      testAllTests()

    3. Compile your code (Sketch -> Verify/Compile or the checkmark button next to the upload button). Your Arduino does not need to be plugged in to complete this step. If there are compilation errors, double-check the steps above.

  2. Read the testTransition function in lab5_tests.ino and discuss how it works with your partner. When you have come up with a 2-4 sentence description, get it checked off by a TA.

  3. Work with your partner to combine your test case tables. For most transitions and non-transitions, choose one partner’s test case to keep. For four transitions of your choice, include both tests, such that you have 23 tests in total. To create more rows in the testing table, highlight the two sheet rows that make up one transition entry. Copy the rows. Right-click on the next blank row and select “Paste special -> Paste format only”) Make sure that the tests are numbered from 0 to 22, in order, in column A.

  4. Follow the instructions at the end of the prelab to turn your test cases into lines of Arduino code. Remember that your spreadsheet shouldn’t have any invalid input errors nor blank cells in order for this code to work. Also check the pinned troubleshooting post on Ed if the script won’t run. Paste the generated lines from cell B1 into lab5_tests.ino under the REPLACE THE FOLLOWING 6 LINES WITH YOUR TEST CASES comment. You will probably have to remove some quotation marks that Google Sheets automatically adds on to copied cells.

  5. Connect your Arduino, open the Serial Monitor, and upload and run the code. If you get to “All tests passed!”, congratulations. Get checked off by a TA. Be prepared for your TA to ask you conceptual questions about your test cases. Otherwise, debug your code, based on which transition fails. You may find that there is a flaw in your implementation, or you may find that your test case made an incorrect assumption. In the second case, you should correct your assumptions (asking a TA if need be), modify the test case, and re-parse your test cases as in step 4 above.

  6. Comment out the #define TESTING line in lab5.h; and uncomment your loop() code and remove the testAllTests() line in lab5.ino. Run your code and verify that the game plays as intended. If not, discuss what might have gone wrong and work on debugging your test cases and code (note that testTransition does not test that you called the outputs correctly, so even if your tests are correct and all pass, you should double-check that the outputs are being set correctly in each transition). Once the game is working, get it checked off by a TA.

  7. OPTIONAL if your tests pass well before lab is over, try one or both of the following bonus steps. If you get checked off for the bonus step, you do not have to answer the corresponding question of the lab writeup.

    1. Test of display functions: the tests you ran do not test whether the display is displaying the correct text/graphic. Write some code that mocks up the display functions (see the implementations of our helper functions under the MOCKED UP FUNCTIONS FOR TESTING comment in lab5_utils.ino for examples) such that they set global variable(s) that can be checked against expected values, as we talked about in class. You will need to modify the testTransition function definition and contents to check whether the expected outputs were correct. Write two test cases for transitions that use different display functions and explain and demonstrate your code to a TA.

    2. Test of non-deterministic functions: In order to make testing easier, we mocked out the non-determinism of the helper functions randomLocationOrientation, randomTurn, and shrinkBounds (see the implementations under the MOCKED UP FUNCTIONS FOR TESTING comment in lab5_utils.ino). Remove the #define TESTING line in lab5.h. Now modify your testing code to test that your test cases pass with the original, non-mocked functions.

      Hint 1: think of this testing in terms of the “properties” you need to check. For example, if a transition calls shrinkBounds, you know that the value of uxb-lxb before the transition will be 1 larger than the value of uxb-lxb after transition – what is a mathematical/Boolean expression you can write to test this change? Similarly, you know that for randomTurn, the orientation after the transition will never be the direct opposite of the orientation before the transition.

      Hint 2: you will have to modify the testTransition code in some cases. One way to do this is to hand-create array(s) of flags that tell testTransition whether, for a given test case, it is doing a direct value comparison or a property check.

      Once you are confident your code works, get it checked off by a TA. Your TA will check that you run your 23 test cases and that #define TESTING does not exist anywhere in your code.

  8. Turn in the code (entire Arduino project zipped up) on the Lab 6: Code Gradescope assignment