Matthew,
Frist and foremost, thank you for writing Chutzpah, it's been useful for me at my organization.
We have configured Chutzpah in one of your projects. The test cases are discovered and run just fine. We're also able to run code coverage on the project. However, we are getting "Object reference not set to an instance of an object" dialog errors from Visual Studio when opening our solution (see attached screenshot). I tracked down the problem to Chutzpah.
I'm attaching a minimal case that reproduces the problem.
Basically, if you have an Entity Framework connection string in the web.config file of a solution, Chutzpah will trigger a database connection when collecting the project's items.
I debugged over chutzpah code and found the source of the error, this is the interesting part of the call trace:
...
System.Data.SqlClient.SqlConnection.Open
...
Microsoft.VisualStudio.Data.Package.DataConnection.EnsureConnected
Microsoft.VisualStudio.Data.Package.DataExplorerNode.EnumerateOrRefreshChildren
...
Microsoft.VisualStudio.Data.Package.DataExplorerConnection.TryGetProperty
Microsoft.VisualStudio.Data.Package.DataExplorerConnection.Microsoft.VisualStudio.Shell.Interop.IVsHierarchy.GetProperty
VS11.Plugin.VsSolutionHelper.GetPropertyValue Line 84 <<<<<<<<<<<
VS11.Plugin.VsSolutionHelper.GetProjectItems Line 48
...
Chutzpah.VS2012.TestAdapter.ChutzpahTestContainerDiscoverer.FindPotentialTestFiles Line 376
Chutzpah.VS2012.TestAdapter.ChutzpahTestContainerDiscoverer.OnSolutionProjectChanged
It looks like the call to IVsHierarchy.GetProperty can cause database connections to get open when the project is using the Entity Framework.
If you try to open the minimal case you won't get the "Object reference not set to an instance of an object." errors, as I have cleaned the connection strings in the web.config file. But you should be seeing a "Connect to SQL Server" dialog instead. Also, if you debug Chutzpah and break on System.Data.SqlClient.SqlException, you should be getting one on the GetPropertyValue method of the VsSolutionHelper class in Chutzpah.
Also, please note that the minimal case solution can be opened without errors when you disable the Chutzpah Test Adapter plugin. We have found a work around, removing or commenting out Entity Framework's connection string in the web.config file avoids the errors to be thrown. So, fixing this is not urgent to us. However, it would be nice if Chutzpah stops triggering database connections.
Miguel
Comments: I tried to repro this with your sample in VS2013 but I did not get the error. Does this still happen for you if you add a chutzpah.json file which specifies which are your test files?https://chutzpah.codeplex.com/wikipage?title=Using%20the%20tests%20setting&referringTitle=Chutzpah.json%20Settings%20File
Frist and foremost, thank you for writing Chutzpah, it's been useful for me at my organization.
We have configured Chutzpah in one of your projects. The test cases are discovered and run just fine. We're also able to run code coverage on the project. However, we are getting "Object reference not set to an instance of an object" dialog errors from Visual Studio when opening our solution (see attached screenshot). I tracked down the problem to Chutzpah.
I'm attaching a minimal case that reproduces the problem.
Basically, if you have an Entity Framework connection string in the web.config file of a solution, Chutzpah will trigger a database connection when collecting the project's items.
I debugged over chutzpah code and found the source of the error, this is the interesting part of the call trace:
...
System.Data.SqlClient.SqlConnection.Open
...
Microsoft.VisualStudio.Data.Package.DataConnection.EnsureConnected
Microsoft.VisualStudio.Data.Package.DataExplorerNode.EnumerateOrRefreshChildren
...
Microsoft.VisualStudio.Data.Package.DataExplorerConnection.TryGetProperty
Microsoft.VisualStudio.Data.Package.DataExplorerConnection.Microsoft.VisualStudio.Shell.Interop.IVsHierarchy.GetProperty
VS11.Plugin.VsSolutionHelper.GetPropertyValue Line 84 <<<<<<<<<<<
VS11.Plugin.VsSolutionHelper.GetProjectItems Line 48
...
Chutzpah.VS2012.TestAdapter.ChutzpahTestContainerDiscoverer.FindPotentialTestFiles Line 376
Chutzpah.VS2012.TestAdapter.ChutzpahTestContainerDiscoverer.OnSolutionProjectChanged
It looks like the call to IVsHierarchy.GetProperty can cause database connections to get open when the project is using the Entity Framework.
If you try to open the minimal case you won't get the "Object reference not set to an instance of an object." errors, as I have cleaned the connection strings in the web.config file. But you should be seeing a "Connect to SQL Server" dialog instead. Also, if you debug Chutzpah and break on System.Data.SqlClient.SqlException, you should be getting one on the GetPropertyValue method of the VsSolutionHelper class in Chutzpah.
Also, please note that the minimal case solution can be opened without errors when you disable the Chutzpah Test Adapter plugin. We have found a work around, removing or commenting out Entity Framework's connection string in the web.config file avoids the errors to be thrown. So, fixing this is not urgent to us. However, it would be nice if Chutzpah stops triggering database connections.
Miguel
Comments: I tried to repro this with your sample in VS2013 but I did not get the error. Does this still happen for you if you add a chutzpah.json file which specifies which are your test files?https://chutzpah.codeplex.com/wikipage?title=Using%20the%20tests%20setting&referringTitle=Chutzpah.json%20Settings%20File