CSCI 265 Quiz 5 Fall 2024 Sample solutions

Question 1a: Stubs/drivers to test a function

Suppose we need to test a function in isolation, where the function has the following specifications:

Provide pseudo-code for stubs/drivers to suitably test the function, and a short explanation of how test cases would be run using your stubs/drivers.

Sample solution
For this one any decent pseudo-code or description is accepted, with the key points to capture outlined below.

We'll need a driver to call completionProb (written in the same language and compiled together, assuming it's a compiled language):

it reads (e.g. from stdin) the remaining, min, and max times to be used in this case and the expected result we expect to be returned from completionProb
it calls completionProb, captures the return value
it compares the return value to the expected one and displays pass or fail

No stub is needed (since completionProb is not known to call anything).

Running a test would thus consist of running the driver, feeding in the four numeric values (via stdin) and then observing/capturing the pass/fail result.

To automate the process we might: create separate files for each test case, named to reflect the nature of the test case and containing the four input values

A batch test script could then read each file in turn (e.g. read every file in a test directory), read and display the test case name (filename), and run the driver with the file contents redirected to stdin (e.g. ./driverx < filename) and displaying the final pass/fail from the driver.

Question 1b: Test cases

For the function and your stubs/drivers/process from (1a), provide 6-8 key test cases to test the functions (providing actual data values to be used in each test) and a short justification for why you picked those test cases as the key ones.

Sample solution test cases
remaining	min	max	expected	rationale
-1	1	2	?	erroneous val for remaining
1	-1	2	?	erroneous val for remaining
1	2	-1	?	erroneous val for remaining
2	3	1	?	min > max, error?
1	1	1	?	all equal (edge case)
1	2	3	?	remaining < min
2	1	3	?	remaining between min and max
3	1	2	?	remaining > max
? is shown for expected probabilities since I told you not to bother calculating those
a variety of other edge cases are likely, e.g. where two of three values are equal and where expected probability would be 0 (or very close to it) or 1 (or very close to it)
Note that testing for numbers that weren't reals (e.g. testing for text inputs) wouldn't make sense in a compiled/type-checked language since those couldn't possibly be passed as parameters to the function.

Question 1c: Testing the tester

Assuming we implemented the stubs/drivers/test cases you described for (1a) and (1b), describe a means we could then use to test that the stubs/drivers/test cases/process were actually correct.

Sample solution
Now what we're actually testing is the driver from (1a), so we'll need a driver to drive it: call this one DD for short.

We need to control the behaviour of completionProb to test how the driver works on good/bad implementations of completionProb, so we'll use a stub for completionProb that allows us to control what completionProb returns in any given test case, and we'll need to compile our (1a) driver with our completionProb stub. The stub might simply wait for the tester to specify (e.g. via stdin) what probability value it should return.

We want DD to run the driver on a test case and compare the driver's pass/fail result to what we expect in this situation: if the driver is correct then should it be saying pass or fail in this instance?

DD can be a script with the following behaviour:

we tell it (via stdin) the test case name
we tell it (via stdin) if the driver should be giving a pass/fail on this case
it runs driver,
- driver gets from us (via stdin, as per 1a) the remaining, min, max times we want to use and the expected result
- driver runs completionProb (the stub), which asks us to enter the desired value to return
- driver compares the value the stub returns to the earlier expected value and says pass/fail
DD checks if driver's pass/fail response matches expectations for this case

Question 1d: Impact of data input mechanism

Outline the changes that would need to be made to your (1a) and (1b) answers if we made the following change to the function specifications:

Suppose that the function only took the expected min/max times as parameters (e.g. the 12.1 and 40.7 in the earlier example), and that the target completion time (the 17.5 in the earlier example) is obtained as a text input value from the user.

Sample solution
For our (1a) the only real change is in the order we enter the data values: before we entered the remaining, min, and max times and the expected result before calling completion prob, now we'd need to enter min, max, and expected before it calls completionProb and then enter the remaining value when the completionProb stub expects it. (Plus of course the driver would only be passing the two parameters instead of three.)

For our (1b), since the remaining time is now taken as a text user input instead of being passed as a parameter (which would have been type-checked at compile time in the 1a version) we now have to worry about cases where the user enters non-numeric values instead of an actual real number.

Presumably the specs for the function would need to be updated to indicate what completionProd is supposed to do if given non-numeric input for remaining time, and we would need to add test cases that checked completionProd correctly followed those new specs.