Imagine you're testing a service to creates, queries and deletes users. A fake version of that service might just be a wrapper on a HashMap keyed by ID. It might have several fields like some personal info, a hashed password, an email address, whether you're verified and so on.
Imagine one of your tests is if the user deletes their account. What pattern of calls should it make? You don't really care other than the record being deleted (or marked as deleted, depending on retention policy) after you're done.
In the mock world you might mock out calls like deleteUserByID and make suer it's called.
In the fake world, you simply check that the user record is deleted (or marked as such) after the test. You don't really care about what sequence of calls made that happen.
That may sound trivial but it gets less trivial the more complex your example is. Imagine instead you want to clear out all users who are marked for deletion. If you think about the SQL for that you might do a DELETE ... WHERE call so your API call might look like that. But if the logic is more complicated? Where if there's a change where EU and NA users have different retention periods or logging requirements so they're suddenly handled differently?
In a mokcing world you would have to change all your expected mocks. In fact, implementing this change might require fixing a ton of tests you don't care about at all and aren't really being broken by the change regardless.
In a fake world, you're testing what the data looks like after you're done, not the specific steps it took to get there.
Now those are pretty simple examples because there's not much to do the arguments used and no return values to speak of. Your code might branch differently based on those values, which then changes what calls to expects and with what values.
You're testing implementation details in a really time-consuming yet brittle way.
I am unsure I follow this. I'm generally mocking the things that are dependencies for the thing I'm really testing.
If the dependencies are proper interfaces, I don't care if it's a fake or a mock, as long as the interface is called with the correct parameters. Precisely because I don't want to test the implementation details. The assumption (correctly so) is that the interface provides a contract I can rely on.
In you example, the brittleness simply moves from mocks to data setup for the fake.
The point is that you probably don't care that much how exactly the dependency is called, as long as it is called in such a way that it does the action you want and returns the results you're interested in. The test shouldn't be "which methods of the dependency does this function call?" but rather "does this function produce the right results, assuming the dependency works as expected?".
This is most obvious with complex interfaces where there are multiple ways to call the dependency that do the same thing. For example if my dependency was an SQL library, I could call it with a string such as `SELECT name, id FROM ...`, or `SELECT id, name FROM ...`. For the dependency itself, these two strings are essentially equivalent. They'll return results in a different order, but as long as the calling code parses those results in the right order, it doesn't matter which option I go for, at least as far as my tests are concerned.
So if I write a test that checks that the dependency was carried with `SELECT name, id FROM ...`, and later I decide that the code looks cleaner the other way around, then my test will break, even though the code still works. This is a bad test - tests should only fail if there is a bug and the code is not working as expected.
In practice, you probably aren't mocking SQL calls directly, but a lot of complex dependencies have this feature where there are multiple ways to skin a cat, but you're only interested in whether the cat got skinned. I had this most recently using websockets in Node - there are different ways of checking, say, the state of the socket, and you don't want to write tests that depend on a specific method because you might later choose a different method that is completely equivalent, and you don't want your tests to start failing because of that.
The fakes vs mocks distinction here feels like a terminology debate masking violent agreement. What you’re describing as a “fake” is just a well-designed mock. The problem isn’t mocks as a concept, it’s mocking at the wrong layer.
The rule: mock what you own, at the boundaries you control.
The chaos you describe comes from mocking infrastructure directly. Verifying “deleteUserById was called exactly once with these params” is testing implementation, not behavior. Your HashMap-backed fake tests the right thing: is the user gone after the operation? Who cares how. The issue is finding the correct layers to validate behavior, not the implementation detail of mocks or fakes… that’s like complaining a hammer smashed a hole in the wall.
In the SQL example, unless you actually use an SQL service as a fake, you cannot really quite get the fake do the right thing either. At which point, it's no longer a mock/fake test but an integration/DB test.
Network servers are another such class and for most parts can be either mocked or faked using interface methods.
I would argue that (barring SQL), if there are too many ways to skin a cat, it is a design smell. Interfaces are contracts. Even for SQL, I almost end up using a repository method (findByXxx flavors) so it is very narrow in scope.
Of course getting overly pedantic leads to its own issues, much like the distinctions between types of tests.
At my last Java job I used to commonly say things like "mocks are a smell", and avoided Mockito like GP, though it was occasionally useful. PowerMock was also sometimes used because it lets you get into the innards of anything without changing any code, but much more rarely. Ideally you don't need a test double at all.
Check function XYZ is called, return abc when XYZ is called etc are the bad kind that people were bit badly by.
The good kind are a minimally correct fake implementation that doesn't really need any mocking library to build.
Tests should not be brittle and rigidly restate the order of function calls and expected responses. That's a whole lot of ceremony that doesn't really add confidence in the code because it does not catch many classes of errors, and requires pointless updates to match the implementation 1-1 everytime it is updated. It's effectively just writing the implementation twice, if you squint at it a bit.
In reflection heavy environments and with injection and reflection heavy frameworks the distinction is a bit more obvious and relevant (.Net, Java). In some cases the mock configuration blossoms to essentially parallel implementations, leading to the brittleness discussed earlier in the thread.
Technically creating a shim or stub object is mocking, but “faking” isn’t using a mocking framework to track incoming calls or internal behaviours. Done properly, IMO, you’re using inheritance and the opportunity through the TDD process to polish & refine the inheritance story and internal interface of key subsystems. Much like TDD helps design interfaces by giving you earlier external interface consumers, you also get early inheritors if you are, say, creating test services with fixed output.
In ideal implementations those stub or “fake” services answer the “given…” part of user stories leaving minimalistic focused tests. Delivering hardcoded dictionaries of test data built with appropriate helpers is minimal and easy to keep up to date, without undue extra work, and doing that kind of stub work often identifies early re-use needs/benefits in the code-base. The exact features needed to evolve the system as unexpected change requests roll in are there already, as QA/end-users are the systems second rodeo, not first.
The mocking antipatterns cluster around ORM misuse and tend to leak implementation details (leading to those brittle tests), and is often co-morbid with anemic domains and other cargo cult cruft. Needing intense mocking utility and frameworks on a system you own is a smell.
For corner cases and exhaustiveness I prefer to be able to do meaningful integration tests in memory as far as possible too (in conjunction with more comprehensive tests). Faster feedback means faster work.
Why is check if XYZ is called with return value ABC bad, as long as XYZ is an interface method?
Why is a minimally correct fake any better than a mock in this context?
Mocks are not really about order of calls unless you are talking about different return values on different invocations. A fake simply moves the cheese to setting up data correctly, as your tests and logic change.
The point is to test against a model of the dependency, not just the expected behavour of the code under test. If you just write a mock that exactly corresponds to the test that you're running, you're not testing the interface with the underlying system, you're just running the (probably already perfectly understandable) unit through a rote set of steps, and that's both harder to maintain and less useful than testing against a model of the underlying system.
(And IMO this should only be done for heavyweight or difficult to precisely control components of the system where necessary to improve test runtime or expand the range of testable conditions. Always prefer testing as close to the real system as reasonably practical)
The kind of mocks the OP is arguing against are not really a model of the dependency, they're just a model of a particular execution sequence in the test, because the mock is just following a script. Nothing in it ensures that the sequence is even consistent with any given understanding of how the dependency works, and it will almost certainly need updating when the code under test is refactored.
My point is that a fake doesn't magically fix this issue. Both are narrow models of the underlying interface. I don't still quite understand why a mock is worse than a fake, when it comes to narrow models of the interface. If there is a method that needs to be called with a specific set up, there is no practical difference between a fake and a mock.
Again, none of this is a replacement for writing integration tests where possible. Mocks have a place in the testing realm and they are not an inherently bad tool.
Mocking is testing how an interface is used, rather than testing an implementation. That's why it requires some kind of library support. Otherwise you'd just on the hook for providing your own simple implementations of your dependencies.
That's mocks in a nutshell. What other way would you use mocks?