On my work computer, I have VS2017 and VS2017 preview installed. Recently both stopped being able to discover MSTest unit tests, and Resharper also did not discover tests until I opened a file containing some tests.
There was nothing in the Visual Studio log windows that indicated problems, and it proved to be an issue whose cause was hard to find.
Unit test projects in VS2017 (not .NET Core) C# projects want to use the MSTest.TestAdapter and MSTest.TestFramework NuGet packages for the MSTest v2 libraries and integration. Earlier versions use something called Microsoft.VisualStudio.TestPlatform.Extensions.VSTestIntegration for the Visual Studio integration.
What was odd, is that the console version of the Visual Studio test runner was able to find the unit tests.
To use the console runner, open a developer command prompt in the unit test project’s folder and run
vstest.console.exe /listtests:bin\debug\settingsgenerator.tests.dll
The /listtests parameter makes it display the unit tests that it discovers.
Visual Studio uses two test discovery engines called “vstest.discoveryengine.exe” and “vstest.discoveryengine.x86.exe”. One is for X64 binaries and the other is for X86 binaries. They analyse the built projects looking for unit tests, but they actually use some plugins to perform the analysis. Andy my problem was caused by one of the plugins.
To get diagnostics logs from the Visual Studio IDE and the test discovery engines, it is necessary to add some configuration to their app.config files:
<system.diagnostics>
<switches>
<add name="TpTraceLevel" value="4" />
</switches>
</system.diagnostics>
The Visual Studio IDE binary devenv.exe has a config file called devenv.exe.config. It doesn’t have the above <system.diagnostics> element at all, so it needs to have it added. The two discovery engines have the XML element, but the default for TpTraceLevel is 0. It needs to be changed to 4.
After restarting VS, and reloading and rebuilding the solution, it will now generate three files in %temp%:
- devenv.TpTrace.log
- vstest.discoveryengine.TpTrace.log
- vstest.discoveryengine.x86.TpTrace.log
The first file contains logs for when the IDE starts and connects to the test runner service, e.g.
V, 20076, 9, 2017/08/08, 06:31:27.263, 860485603663, devenv.exe, TestRunnerServiceClient: Creating test connection. Informational: ------ Load Playlist started ------ Informational: ========== Load Playlist finished (0:00:00,0005014) ========== I, 20076, 9, 2017/08/08, 06:31:28.934, 860490029860, devenv.exe, TestRunnerServiceClient: Successfully connected to test runner service.
When it is done discovering tests, it logs:
V, 20076, 81, 2017/08/08, 06:31:36.592, 860510310219, devenv.exe, DiscoveryRequest.DiscoveryComplete: Starting. Aborted:False, TotalTests:31
Before I fixed the problem on my computer, it consistently found 0 tests. But the presence of those log entries showed that Visual Studio at least attempted to discover my unit test cases.
vstest.discoveryengine.x86.TpTrace.log also contained message indicating that it did indeed attempt to discover the unit tests, e.g. this message, which in the log file also contained a list of all the binaries that my solution builds:
TestDiscoveryManager: Discovering tests from sources
During the discovery, it even logged this message:
V, 19212, 14, 2017/08/08, 06:19:40.695, 858614302804, vstest.discoveryengine.x86.exe, DiscoveryContext.LoadTests: Done loading tests for Microsoft.VisualStudio.TestPlatform.Extensions.VSTestIntegration.MSTestDiscoverer
It was this message, when I compared the TpTrace.log files to the diagnostics log of the console test runner, that caught my eye:
When tests were discovered by the console test runner, it used a library called “Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter”, whereas the library was called “Microsoft.VisualStudio.TestPlatform.Extensions.VSTestIntegration” when discovering tests in the IDE.
My test projects have a NuGet package called MSTest.TestAdapter, since that is the new MSTest v2 library, so it seemed that the test projects were built with one TestAdapter implementation, and the discovery engine used another.
Indeed, the MSTest.TestAdapter NuGet package copies some binaries to a folder called
%temp%\VisualStudioTestExplorerExtensions\MSTest.TestAdapter.<version>\build\_common
But for some reason they weren’t being used by the IDE test discovery engine.
The solution to my problem: Copy the DLL files from that folder to the “Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions” subfolder of the Visual Studio installation folder and restart VS.
This had to be done for each VS instance (I also have the latest VS 2017 preview installed).
The cause: I don’t know. Perhaps it started when I performed a disk cleanup which emptied my %temp% folder, and thus deleted the VisualStudioTestExplorerExtensions folder.
But somewhere, VS must have lost some configuration that made it load the new MSTest.TestAdapter libraries, since re-installing the NuGet packages that place them in %temp%\VisualStudioTestExplorerExtensions did not solve the problem. Worth noting: My regular user account is not an admin, so it doesn’t have write permissions to the Extensions subfolder in the VS installation folder, perhaps that caused problems also.