Roslyn analyzers: How to test

Roslyn analyzers: How to test

When you write a Roslyn analyzer, you should also write tests for it. The Roslyn SDK provides a set of NuGet packages that you can use to test your analyzers. In this post, I describe how to test a Roslyn analyzer using the Roslyn SDK NuGet packages.

First, Roslyn SDK NuGet packages are not available on NuGet.org. You need to add the following package source to your project file:

<RestoreAdditionalProjectSources>https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json</RestoreAdditionalProjectSources>

Warning Some older versions of the packages are available on NuGet.org. However, I recommend using the latest version from the Azure DevOps feed (more info).

Next, you need to add the following NuGet packages to your project file:

dotnet add package Microsoft.NET.Test.Sdk
dotnet add package xunit
dotnet add package xunit.runner.visualstudio

dotnet add package Microsoft.CodeAnalysis.CSharp.Analyzer.Testing --version 1.1.2-beta1.24169.1
dotnet add package Microsoft.CodeAnalysis.CSharp.CodeFix.Testing --version 1.1.2-beta1.24169.1
dotnet add package Microsoft.CodeAnalysis.CSharp.Workspaces --version 4.9.2

Note The Roslyn SDK NuGet provides multiple test-framework-specific packages such as Microsoft.CodeAnalysis.CSharp.Analyzer.Testing.Xunit. However, the recommendation is now to use the default packages and the DefaultVerifier (source, source)

Now, you can write tests for your analyzer. Here is an example of a test that checks if your analyzer produces the expected diagnostics:

[Fact]
public async Task Test()
{
    var context = new CSharpAnalyzerTest<MyAnalyzer, DefaultVerifier>();
    context.ReferenceAssemblies = ReferenceAssemblies.Net.Net80;

    // The code between [| and |] is the code that should produce the diagnostic.
    context.TestCode = """
        class [|Type1|] { }
        """;

    await context.RunAsync();
}

You can also test the code fix for your analyzer. Here is an example of a test that checks if your code fix fixes the diagnostic:

[Fact]
public async Task Test()
{
    var context = new CSharpCodeFixTest<MyAnalyzer, MyAnalyzerCodeFixProvider, DefaultVerifier>();
    context.ReferenceAssemblies = ReferenceAssemblies.Net.Net80;

    context.TestCode = """
        class [|Type1|] { }
        class [|Type2|] { }
        """;

    context.FixedCode = """
        class TYPE1 { }
        class TYPE2 { }
        """;

    await context.RunAsync();
}

#Marker syntax

If the analyzer only has one diagnostic, you can use [| and |]. Note that you can nest the markers.

var context = new CSharpAnalyzerTest<MyAnalyzer, DefaultVerifier>();
context.ReferenceAssemblies = ReferenceAssemblies.Net.Net80;
context.TestCode = """
    class [|Type1|] { }
    """;
await context.RunAsync();

If the analyzer produces multiple diagnostics, you can use {|RuleId: and |}

var context = new CSharpAnalyzerTest<MyAnalyzer, DefaultVerifier>();
context.ReferenceAssemblies = ReferenceAssemblies.Net.Net80;
context.TestCode = """
    class {|MyRuleId:Type1|} { }
    """;
await context.RunAsync();

If you need more control over the diagnostic, you can use {|#0: and |} or |#0}, where 0 is the diagnostic index.

var context = new CSharpAnalyzerTest<MyAnalyzer, DefaultVerifier>();
context.ReferenceAssemblies = ReferenceAssemblies.Net.Net80;
context.TestCode = """
    class {|#0:Type1|} { }
    class {|#1:Type1|#1} { } // Use the full closing (e.g. |#1}) if you have overlapping diagnostics
    """;
context.ExpectedDiagnostics.Add(new DiagnosticResult(MyAnalyzer.Rule).WithLocation(0).WithArguments("Type1"));
context.ExpectedDiagnostics.Add(new DiagnosticResult(MyAnalyzer.Rule).WithLocation(1).WithArguments("Type2"));
await context.RunAsync();

#Adding more source files

If you need to add more source files to your analyzer, you can use the TestState.Sources property.

var context = new CSharpAnalyzerTest<MyAnalyzer, DefaultVerifier>();
context.ReferenceAssemblies = ReferenceAssemblies.Net.Net80;
context.Sources.Add("file1.cs", """
    global using System;
    """);

context.TestCode = """
    class {|MyRuleId:Type1|} { }
    """;
await context.RunAsync();

#Using custom reference assemblies

By default, you can easily target most .NET versions using the ReferenceAssemblies class. But, you can also add a custom version of .NET if needed.

var context = new CSharpAnalyzerTest<MyAnalyzer, DefaultVerifier>();
// Default
context.ReferenceAssemblies = ReferenceAssemblies.Net.Net80;
context.ReferenceAssemblies = ReferenceAssemblies.Net.Net80Windows;
context.ReferenceAssemblies = ReferenceAssemblies.NetStandard.NetStandard20;
context.ReferenceAssemblies = ReferenceAssemblies.NetFramework.Net48.Wpf;

// Custom one
context.ReferenceAssemblies =
    new ReferenceAssemblies(
        targetFramework: "net9.0",
        referenceAssemblyPackage: new PackageIdentity("Microsoft.NETCore.App.Ref", "9.0.0-preview.3.24172.9"),
        referenceAssemblyPath: Path.Combine("ref", "net9.0"));

#Adding NuGet packages

If you need to test an analyzer that relies on a type provided by a NuGet package, you can use the AddPackages method.

var context = new CSharpAnalyzerTest<MyAnalyzer, DefaultVerifier>();
context.ReferenceAssemblies = ReferenceAssemblies.Net.Net80
    .AddPackages([
        new PackageIdentity("Microsoft.AspNetCore.App.Ref", "8.0.4"),
        new PackageIdentity("Microsoft.WindowsDesktop.App.Ref", "8.0.4"),
        new PackageIdentity("Microsoft.Windows.SDK.NET.Ref", "10.0.22621.33"),
        new PackageIdentity("Meziantou.Framework.FullPath", "1.0.12")
    ]);

#Adding additional files

If you need to add additional files to your analyzer, you can use the TestState.AdditionalFiles method.

var context = new CSharpAnalyzerTest<MyAnalyzer, DefaultVerifier>();
context.ReferenceAssemblies = ReferenceAssemblies.Net.Net80;

context.TestState.AdditionalFiles.Add(("sample.txt", "content"));

#Add .editorconfig files

If you need to add an editorconfig to your analyzer, you can use the TestState.AnalyzerConfigFiles method.

var context = new CSharpAnalyzerTest<MyAnalyzer, DefaultVerifier>();
context.ReferenceAssemblies = ReferenceAssemblies.Net.Net80;

context.TestState.AnalyzerConfigFiles.Add(("/sample.editorconfig", "name = value"));

#Settings the Output Kind to a console application

If you need to set the output kind to a console application, you can use the TestState.OutputKind property.

Tip Using a console application output as you can use top-level statements in your test code to reduce the test length.

var context = new CSharpAnalyzerTest<MyAnalyzer, DefaultVerifier>();
context.ReferenceAssemblies = ReferenceAssemblies.Net.Net80;
context.TestState.OutputKind = OutputKind.ConsoleApplication;
context.TestCode = "Console.WriteLine(\"Hello, World!\");"; // no need for a class or a method!
await context.RunAsync();

#Setting the parser options

If you need to set the language version for your analyzer, you can add a solution transformer:

var context = new CSharpAnalyzerTest<MyAnalyzer, DefaultVerifier>();
context.SolutionTransforms.Add((solution, projectId) =>
{
    return solution.WithProjectParseOptions(projectId, new CSharpParseOptions(
        languageVersion: LanguageVersion.CSharp4,
        preprocessorSymbols: ["DEBUG"]));
});

#Configuring compiler diagnostics

Most of the time, it can be useful to ensure the code you are testing compiles without errors. You can configure the compiler diagnostics using the CompilerDiagnostics property.

var context = new CSharpAnalyzerTest<MyAnalyzer, DefaultVerifier>();

context.CompilerDiagnostics = CompilerDiagnostics.All;         // All diagnostics
context.CompilerDiagnostics = CompilerDiagnostics.Errors;      // Errors
context.CompilerDiagnostics = CompilerDiagnostics.Warnings;    // Warnings and errors
context.CompilerDiagnostics = CompilerDiagnostics.Suggestions; // Suggestions, warnings and errors

#Disabling diagnostics

If you need to disable diagnostics for a specific test, you can use the TestState.DisabledDiagnostics method.

var context = new CSharpAnalyzerTest<MyAnalyzer, DefaultVerifier>();
context.CompilerDiagnostics = CompilerDiagnostics.All;
context.DisabledDiagnostics.Add("CS8019"); // Unused using directives

context.TestCode = "using System;";
await context.RunAsync();

#Testing a specific code fix

If the code fix provides multiple fixes, you can apply a specific fix using the CodeActionIndex or CodeActionEquivalenceKey property.

var context = new CSharpCodeFixTest<MyAnalyzer, MyAnalyzerCodeFixProvider, DefaultVerifier>();
context.TestCode = "class [|Type1|] { }";
context.FixedCode = "class TYPE1 { }";

context.CodeActionIndex = 1;
context.CodeActionEquivalenceKey = "key";
await context.RunAsync();

#Alternative syntax for simple cases

If you have a simple case, you can use the AnalyzerVerifier or CodeFixVerifier types

using VerifyCS = Microsoft.CodeAnalysis.CSharp.Testing.CSharpAnalyzerVerifier<MyAnalyzer, Microsoft.CodeAnalysis.Testing.DefaultVerifier>;

class MyAnalyzerTests
{
    [Fact]
    public async Task Test()
    {
        // Basic case
        await VerifyCS.VerifyAnalyzerAsync("class [|Sample|] { }");

        // With custom expected diagnostics
        await VerifyCS.VerifyAnalyzerAsync("class {|#0:Sample|} { }",
            VerifyCS.Diagnostic().WithLocation(0).WithArguments("Sample"));
    }
}

using VerifyCS = Microsoft.CodeAnalysis.CSharp.Testing.CSharpCodeFixVerifier<MyAnalyzer, MyAnalyzerCodeFixProvider, Microsoft.CodeAnalysis.Testing.DefaultVerifier>;

class MyAnalyzerTests
{
    [Fact]
    public async Task Test()
    {
        // Basic case
        await VerifyCS.VerifyCodeFixAsync(
            source: "class [|Sample|] { }",
            expected: [],
            fixedSource: "class SAMPLE { }");

        // With custom expected diagnostics
        await VerifyCS.VerifyCodeFixAsync(
            source: "class {|#0:Sample|} { }",
            expected: [VerifyCS.Diagnostic().WithLocation(0).WithArguments("Sample")],
            fixedSource: "class SAMPLE { }");
    }
}

#Get syntax highlighting in the test code

If you want syntax highlighting in the test code, you can help your editor by setting the language. There are two ways to do this:

Add a /* lang=c#-test */ comment before a string
Use the [StringSyntax("c#-test")] attribute on a parameter or a property

_ = /* lang=c#-test */ """
    public void Sample()
    {
        var x = 1;
        var y = 2;
        var z = x + y;
    }
    """;

Syntax highlighting using a comment

!https://www.meziantou.net/assets/syntax-highlighting-comment.png?v=d1d5

ValidateCode("public void A() { }");

void ValidateCode([StringSyntax("c#-test")] string value) { }

Syntax highlighting using the StringSyntax attribute

!https://www.meziantou.net/assets/syntax-highlighting-attribute.png?v=9ad7

#Additional resources

Roslyn SDK

**#Marker syntax**​

**#Adding more source files**​

**#Using custom reference assemblies**​

**#Adding NuGet packages**​

**#Adding additional files**​

**#Add .editorconfig files**​

**#Settings the Output Kind to a console application**​

**#Setting the parser options**​

**#Configuring compiler diagnostics**​

**#Disabling diagnostics**​

**#Testing a specific code fix**​

**#Alternative syntax for simple cases**​

**#Get syntax highlighting in the test code**​

**#Additional resources**​