Skip to main content

Is it acceptable to accept multiple results in a unit test? [Resolved]

I've noticed this while implementing a Rest-API today. We first define interfaces for each API method and the api contract for one endpoint allows (amongst others) two status codes: 200 and 204. Both may be returned when the request was successful, but the returned data isn't really important. The implementation might choose to return 204 and no additional data or a 200 if some additional, not really relevant data is returned (might be a string for debugging purposes like "entity 123 persisted").

Now, I'm implementing the API and know I want the implementation to always return such a text. But this might change in the future for whatever reason, so I thought I'll test the result like that (this is Java code but should be easy enough to follow):

assertThat(response.getStatus(), anyOf(is(200), is(204));

Is it acceptable to test my implementation like this, just because the interface allows both values for the same case (and it's not really important)? Or should I stay close to the actual implementation here (and change the test later, if I change the implementation)

Question Credit: looper
Question Reference
Asked October 6, 2019
Posted Under: Programming
4 Answers

I prefer to test the interface, not the implementation.

If your consumers have access to your code, tests are a form of documentation that is guaranteed to be up to date. If you only test for 200, consumers may come to rely on that, and changing to 204 (always or sometimes) is more likely to be a breaking change.

From a more selfish point of view, tests should be minimally specific (without compromising on the "guarantee" of correctness) because this makes them less brittle and easier to maintain.

credit: Jacob Raihle
Answered October 6, 2019

So if you get a status 200, that is correct and your test must pass. And if you get a status 204, that is also correct and your test must pass.

Let’s think hard about this...

Yes, your test must pass in both cases.

(This is ignoring the fact that a server must always be expected to return all kinds of errors due to not functioning correctly right now, which should probably not lead to “test passed” or “test failed” but “test could not be performed”).

credit: gnasher729
Answered October 6, 2019

I think you should have two different tests.

You return a different status code based on what the value is in the database. You have defined two different behaviours: when the database is in one state, you return 200. When the database is in another state, you return 204.

But this is a unit test of your web-service - so the test should be decoupled from the database. You can and should use some form of test double (mock, stub, etc). This way the tests not only document the different possible responses, but give example scenarios of when to expect each response.

If you find you really want to connect to a database, your test specification should set the database state, i.e. delete the table(s), populate some test data for the test, then run the test. But this is not preferred - if your database is unavailable, you won't be able to test/build your API.

When you talk about changing it to always return 200, you're not talking about a refactoring that changes the implementation without changing the behaviour. That's an actual behaviour change, so it's natural that you would update the tests at this point.

credit: Jeremy Hunt
Answered October 6, 2019
Your Answer