Hi Jacob! Great articles series! We've got a big project with async await and Combine and we've been struggling with flaky asynchronous testing. Your articles helped us a lot to make them stable. However, I have one question to this article.
We've found out that some tests were failing because we've been waiting for the @Published property and then we've been checking the value inside the sut, not the returned value from the sink.
As per Apple's documentation for @Published: "When the property changes, publishing occurs in the property’s willSet block, meaning subscribers receive the new value before it’s actually set on the property". So it looks like there is a slight chance of race and that checking the value after sink might check the "wrong" value, before it changes and I think we've seen exactly that problem in our tests because making a change - stabilised them (we have many many tests - around 2k).
So in your first example it would require saving what sink has returned into a variable, e.g. returnedBeers and then instead of checking XCTAssertEqual(sampleBeers, sut.beers) - check XCTAssertEqual(sampleBeers, returnedBeers).
What do you think? Does it make sense? Have you experience that as well?
Hi Jacob! Great articles series! We've got a big project with async await and Combine and we've been struggling with flaky asynchronous testing. Your articles helped us a lot to make them stable. However, I have one question to this article.
We've found out that some tests were failing because we've been waiting for the @Published property and then we've been checking the value inside the sut, not the returned value from the sink.
As per Apple's documentation for @Published: "When the property changes, publishing occurs in the property’s willSet block, meaning subscribers receive the new value before it’s actually set on the property". So it looks like there is a slight chance of race and that checking the value after sink might check the "wrong" value, before it changes and I think we've seen exactly that problem in our tests because making a change - stabilised them (we have many many tests - around 2k).
So in your first example it would require saving what sink has returned into a variable, e.g. returnedBeers and then instead of checking XCTAssertEqual(sampleBeers, sut.beers) - check XCTAssertEqual(sampleBeers, returnedBeers).
What do you think? Does it make sense? Have you experience that as well?