New Post: FailOnError and TFS Integration

Hi,

Thanks for responding so quickly.

That is correct. The script error is not failing the build.

No, they are not reported in the build result.

A little bit more information about the error: it's 9/10 a reference file problem. The JS file under test, a.js, depends on b.js and either we didn't include a /// <refere... for it or we forgot to mock it within a.js

When an error like this happens within VS the tests aren't discovered. That makes sense. If we had a JS error such that the test file couldn't even be enumerated how could it discover the tests. If I right-click on the file and open in browser, we'll see a 404 error where it attempted to locate the file or something like that.

We're running 3.0.1.0 of both Chutzpah and the Test Adapter. I'm looking at the latest version to see if there's a change that would correct this. Please let me know if there is.

SO... my thought is to add some code like this to the ExecutionCallback object in the testSuiteComplete method. (Still installing VS 2013 otherwise I'd have tried it by now.)

if (testResultsSummary.Errors.Any())
{
    var test = new TestCase();
    test.TestName = "Chutzpah Error Check";
    frameworkHandle.RecordStart(test.ToVsTestCase());

    var testResult = new Models.TestResult();
    testResult.Passed = false;
    testResult.Message = "There was an error when running the tests through Chutzpah.";
    test.TestResults.Add(testResult);
    var outcome = ChutzpahExtensionMethods.ToVsTestOutcome(test.Passed);
    var result = test.ToVsTestResult();
    var testCase = test.ToVsTestCase();
    frameworkHandle.RecordResult(result);
    frameworkHandle.RecordEnd(testCase, outcome);
}

(Or maybe it's in the TestError collection in DiscoverTests.)
Thx, Joel