如果在Docker下运行的.NET Core单元测试的代码覆盖率不到90％，则打破TeamCity的构建 [英] Breaking the build in TeamCity if .NET Core unit tests running under Docker have code-coverage less than 90%

问题描述

我最近一直在研究Docker，以及如何使用TeamCity在Docker容器中运行.NET Core单元测试，这是我的构建管道的一部分。我将此添加为Dockerfile中的最后一行，以便能够运行测试：

I have recently been looking at Docker, and how I can use TeamCity to run .NET Core unit tests in Docker containers as part of my build pipe-line. I add this as the final line in my Dockerfile to be able to run tests:

ENTRYPOINT ["dotnet", "test", "--verbosity=normal"]

这些Dockerfile然后在TeamCity构建和构建的组合文件中引用在命令行中使用docker-compose运行。

These Dockerfiles are then referenced in compose files which TeamCity builds and runs using docker-compose in the command line.

我现在已成功完成此工作。下一个挑战是，如果单元/集成测试的覆盖率小于90％（或其他值），请中断构建！

I have this working successfully now. Next challenge is to break the build if unit/integration test coverage is less than 90% - or some other value - no arguments about this please!

我正在成功使用coverlet.msbuild NuGet依赖关系可以作为我的构建的一部分来测量代码覆盖率。

I'm successfully using the coverlet.msbuild NuGet dependency to measure code-coverage as part of my build. This works fine in TeamCity too, and I see the output in my TeamCity build.

我通过在每个测试项目中添加coverlet.msbuild并进行更改来完成此工作。 Dockerfile入口点：

I got this working by adding coverlet.msbuild to each of my test projects, and changing the Dockerfile entry-point to:

ENTRYPOINT ["dotnet", "test", "--verbosity=normal", "/p:CollectCoverage=true", "/p:Threshold=90", "/p:ThresholdType=line"]

TeamCity构建输出显示了包含结果的ASCII表，但是到目前为止，如果代码覆盖率不够高，我仍然找不到找到破坏构建的好方法。如果代码覆盖率太低，TeamCity不会将其留给自己的设备标记为构建失败；如果覆盖率太高，这是合理的，因为这不是出于心理考虑！

The TeamCity build output shows the ASCII tables with the results in, but as of yet I've not been able to find a good way to break the build if the code-coverage isn't high enough. Left to its own devices, TeamCity doesn't mark builds as failing if the code-coverage is too low, which if fair enough as it's not psychic!

我天真的认为我可以在TeamCity中创建一个失败条件，该条件将检测以下文本的存在：

I naively thought I could create a failure condition in TeamCity which would detect for the presence of the following text:

'[Assemnbly]' has a line coverage '9.8%' below specified threshold '95%'

...使用如下正则表达式：

...using a regular expression like this:

has a line coverage '((\d+(\.\d*)?)|(\.\d+))%' below specified threshold '((\d+(\.\d*)?)|(\.\d+))%'

但是，当被测试的DLL引用其他被测试的DLL时分别地，它变得棘手，因为Coverlet.msbuild报告所有接触的DLL的覆盖率指标。例如，我有一个名为Steve.Core.Files.Tests的测试项目，用于测试Steve.Core.Files。但是，Steve.Core.Files依次引用Steve.Core.Extensions。我在自己的测试DLL中分别测试Steve.Core.Extensions，因此在测试文件时，我不在乎该DLL的结果。 TeamCity中的输出如下所示：

However, when DLLs being tested reference other DLLs that are tested separately, it gets tricky because coverlet.msbuild reports coverage metrics for all "touched" DLLs. For instance, I have a test project called Steve.Core.Files.Tests which tests Steve.Core.Files. However, Steve.Core.Files in turn references Steve.Core.Extensions. I test Steve.Core.Extensions separately in its own test DLL so I don't care about the results for that DLL when testing files. The output in TeamCity looks like this:

+-----------------------+--------+--------+--------+
| Module                | Line   | Branch | Method |
+-----------------------+--------+--------+--------+
| Steve.Core.Extensions | 23.5%  | 40%    | 40%    |
+-----------------------+--------+--------+--------+
| Steve.Core.Files      | 100%   | 100%   | 100%   |
+-----------------------+--------+--------+--------+

...因此即使基于23.5％的位，它也会失败，即使有问题的DLL是100％。实际上，这使得检查Regex失败条件的使用非常困难。

...so it fails based on the 23.5% bit, even though the DLL in question is 100%. This actually makes it very difficult to check the using a Regex failure condition.

为了使事情更加复杂，我使用单个动态Dockerfile在所有程序集中运行所有测试，两个原因：

To complicate things further I'm running all tests in all assemblies using a single dynamic Dockerfile, for two reasons:

1. 我不想每次都必须更改Dockerfile和docker-compose文件（和TeamCity）我添加了更多的项目和测试。

1. I don't want to have to change the Dockerfile and docker-compose file (and TeamCity) each time I add more projects and tests.

这些DLL之间有很多依赖关系，因此有必要一次构建它们并一起测试它们。

There are many dependencies between the DLLs so it makes sense to build them once and test them all together.

这意味着我不愿意将测试分开，以便每个测试都有自己的Dockerfile-我知道这将使我能够使用排除/包含标志来获取所需的行为。

This means I'm loath to split the tests up so that each has its own Dockerfile - I know that this would allow me to use the Exclude/Include flags to get the desired behaviour.

有人是否有其他想法可以解决此问题？

Does anyone have any other ideas how I can solve this please?

我希望我可以在每个测试项目的级别添加一个文件，以告诉它要覆盖哪些DLL-那将是最好的解决方案。失败了，因为我在项目和测试项目之间使用了严格的命名约定，我可以在我的 dotnet test 命令中添加一个开关以仅测试与以下名称相同的程序集测试程序集最后是否要减去.Tests位？

I'm hoping I can add a file at the level of each test project to tell it which DLLs to do coverage for - that would be the best solution. Failing that, as I use a strict naming convention between projects and test projects, can I add a switch to my dotnet test command to test only the assembly that has the same name as the test assembly minus the .Tests bit on the end?

预先感谢；

干杯，

Steve。

更新2018年9月7日：

因此，我的Dockerfile现在特定于每个单元测试项目。它们看起来像这样并且存在于测试项目文件的旁边：

So, my Dockerfiles are now specific to each unit test project. They look like this and exist in next to the test project files:

FROM microsoft/dotnet:2-sdk

# Set the working directory:
WORKDIR /src

# Copy the solution file and the NuGet.config across to the src directory:
COPY *.sln NuGet.config ./

# Copy the main source project files to the root level:
COPY */*.csproj ./

# Make directories for each project file and move the project file to the correct place:
RUN for file in $(ls *.csproj); do mkdir -p ${file%.*}/ && mv $file ${file%.*}/; done

# Restore dependencies:
RUN dotnet restore

# Copy all files so that we have all everything ready to compile:
COPY . .

# Set the flag to tell TeamCity that these are unit tests:
ENV TEAMCITY_PROJECT_NAME = ${TEAMCITY_PROJECT_NAME}

# Run the tests:
ENTRYPOINT ["dotnet", "test", "Steve.Core.Configuration.Tests/Steve.Core.Configuration.Tests.csproj", "--verbosity=normal", "/p:CollectCoverage=true", "/p:Threshold=95", "/p:ThresholdType=line", "/p:Exclude=\"[Steve.Core.Testing]*\""]

请注意，应该停止Steve.Core.Testing DLL的覆盖结果的exclude开关将包括在Steve.Core.Configuration的结果中，这是

Note the exclude switch which is supposed to stop the coverage results for the Steve.Core.Testing DLL being included in the results for Steve.Core.Configuration, which is the main dependency of the tests, and the project being unit tested.

我的撰写文件如下所示，并且位于解决方案文件旁边：

My compose file looks like this and exists next to the solution file:

version: '3.6'

services:
  # Dependencies:
  steve.core.ldap.tests.ldap:
    image: osixia/openldap
    container_name: steve.core.ldap.tests.ldap
    environment:
      LDAP_ORGANISATION: Steve
      LDAP_DOMAIN: steve.com
      LDAP_ADMIN_PASSWORD: Password1
  steve.core.data.mysql.tests.database:
    image: mysql
    container_name: steve.core.data.mysql.tests.database
    command: mysqld --default-authentication-plugin=mysql_native_password
    environment:
      - MYSQL_ROOT_PASSWORD=Password1
      - MYSQL_DATABASE=testdb
  steve.core.data.sqlserver.tests.database:
    image: microsoft/mssql-server-linux
    container_name: steve.core.data.sqlserver.tests.database
    environment:
      - MSSQL_SA_PASSWORD=Password1
      - ACCEPT_EULA=Y
      - MSSQL_PID=Developer  
  steve.core.email.tests.smtp:
    image: mailhog/mailhog 
    container_name: steve.core.email.tests.smtp  

  # Steve.Core.Configuration:
  steve.core.configuration.tests:
    image: steve.core.configuration.tests:tests
    build:
      context: .
      dockerfile: Steve.Core.Configuration.Tests/Dockerfile
    environment:
      - TEAMCITY_PROJECT_NAME

  # Steve.Core.Data.MySql:
  steve.core.data.mysql.tests:
    image: steve.core.data.mysql.tests:tests
    build:
      context: .
      dockerfile: Steve.Core.Data.MySql.Tests/Dockerfile
    environment:
      - TEAMCITY_PROJECT_NAME

  # Steve.Core.Data.SqlServer:
  steve.core.data.sqlserver.tests:
    image: steve.core.data.sqlserver.tests:tests
    build:
      context: .
      dockerfile: Steve.Core.Data.SqlServer.Tests/Dockerfile
    environment:
      - TEAMCITY_PROJECT_NAME

  # Steve.Core.Data:
  steve.core.data.tests:
    image: steve.core.data.tests:tests
    build:
      context: .
      dockerfile: Steve.Core.Data.Tests/Dockerfile
    environment:
      - TEAMCITY_PROJECT_NAME

  # Steve.Core.Email:
  steve.core.email.tests:
    image: steve.core.email.tests:tests
    build:
      context: .
      dockerfile: Steve.Core.Email.Tests/Dockerfile
    environment:
      - TEAMCITY_PROJECT_NAME

  # Steve.Core.Encryption:
  steve.core.encryption.tests:
   image: steve.core.encryption.tests:tests
   build:
     context: .
     dockerfile: Steve.Core.Encryption.Tests/Dockerfile
   environment:
     - TEAMCITY_PROJECT_NAME

  # Steve.Core.Execution:
  steve.core.execution.tests:
    image: steve.core.execution.tests:tests
    build:
      context: .
      dockerfile: Steve.Core.Execution.Tests/Dockerfile
    environment:
      - TEAMCITY_PROJECT_NAME

  # Steve.Core.Extensions:
  steve.core.extensions.tests:
    image: steve.core.extensions.tests:tests
    build:
      context: .
      dockerfile: Steve.Core.Extensions.Tests/Dockerfile
    environment:
      - TEAMCITY_PROJECT_NAME

  # Steve.Core.Files:
  steve.core.files.tests:
    image: steve.core.files.tests:tests
    build:
      context: .
      dockerfile: Steve.Core.Files.Tests/Dockerfile
    environment:
      - TEAMCITY_PROJECT_NAME

  # Steve.Core.Ldap:
  steve.core.ldap.tests:
    image: steve.core.ldap.tests:tests
    build:
      context: .
      dockerfile: Steve.Core.Ldap.Tests/Dockerfile
    environment:
      - TEAMCITY_PROJECT_NAME

  # Steve.Core.Maths:
  steve.core.maths.tests:
    image: steve.core.maths.tests:tests
    build:
      context: .
      dockerfile: Steve.Core.Maths.Tests/Dockerfile
    environment:
      - TEAMCITY_PROJECT_NAME

  # Steve.Core.Time:
  steve.core.time.tests:
    image: steve.core.time.tests:tests
    build:
      context: .
      dockerfile: Steve.Core.Time.Tests/Dockerfile
    environment:
      - TEAMCITY_PROJECT_NAME

在TeamCity中运行时，即使12个项目中有236个测试，它也仅报告两个项目中的7个测试。

When it runs in TeamCity, it reports only 7 tests from two projects (for some strange reason) even though there are 236 tests in 12 projects.

如果愿意，我很乐意通过电子邮件将TeamCity构建的输出发送给电子邮件。

I'd be happy to email the output from the TeamCity build if it will help.

有人知道我如何使我的测试重新运行吗？

Does anyone know how I can get my tests all running again please?

谢谢

Steve。

推荐答案

因此，唯一的解决方案是将每个单元测试项目拆分为自己的撰写文件，该文件包括该测试DLL所需的依赖项。 （例如，用于测试电子邮件DLL的mailhog，用于测试数据库DLL的SQL Server等）。然后，TeamCity使用单个脚本分别运行它们，如下所示：

So, the only solution was to split each unit test project up into its own compose file which includes the dependencies needed for just that test DLL. (e.g. mailhog for testing email DLLs, SQL Server for testing database DLLs, etc...). TeamCity then runs them all individually using a single script like this:

docker-compose -f docker-compose-configuration-tests.yml up --force-recreate --abort-on-container-exit --build
docker-compose -f docker-compose-configuration-tests.yml down --volumes --remove-orphans

docker-compose -f docker-compose-data-mysql-tests.yml up --force-recreate --abort-on-container-exit --build
docker-compose -f docker-compose-data-mysql-tests.yml down --volumes --remove-orphans

...

每个人都有其自己的Dockerfile，该文件构建测试DLL并设置DLL异常以进行单元测试。 TeamCity在一个构建步骤中吐出所有测试的结果，上面问题中提到的regex代码覆盖失败情况，然后正确地检测到未达到x％覆盖率的测试项目并破坏了构建。

Each one has its own Dockerfile which builds the test DLL and sets the DLL exceptions for unit test coverage. TeamCity spits out the results of all tests in a single build step, and the regex code coverage failure condition mentioned in my question above then correctly detects test projects that don't achieve x% coverage and breaks the build.

现在要弄清楚如何将代码检查（例如FxCop和StyleCop的现代等效项）集成到我的构建过程中...

Now to work out how to integrate code checking (e.g. the modern equivalents of FxCop and StyleCop) into my build process...

这篇关于如果在Docker下运行的.NET Core单元测试的代码覆盖率不到90％，则打破TeamCity的构建的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！