使用自定义插件并行运行测试
此示例说明如何创建一个支持并行运行测试的自定义插件。该自定义插件对测试套件的通过和失败断言数进行计数。为了扩展 TestRunner
实例,该插件会覆盖 matlab.unittest.plugins.TestRunnerPlugin
类的选择方法。此外,为了支持并行运行测试,该插件会创建 matlab.unittest.plugins.Parallelizable
接口的子类。要并行运行测试,您需要 Parallel Computing Toolbox™。
创建插件类
在当前文件夹中名为 AssertionCountingPlugin.m
的文件中,创建可并行化的插件类 AssertionCountingPlugin
,它同时从继承自 TestRunnerPlugin
和 Parallelizable
类。有关 AssertionCountingPlugin
的完整代码,请参阅插件类定义。
要跟踪通过和失败的断言数,请在 properties
块中定义四个只读属性。当前并行池中的每个 MATLAB® 工作进程使用 NumPassingAssertions
和 NumFailingAssertions
来跟踪运行 TestSuite
数组的一部分时的通过和失败断言数。MATLAB 客户端使用 FinalizedNumPassingAssertions
和 FinalizedNumFailingAssertions
聚合来自不同工作进程的结果,并在测试会话结束时报告通过和失败断言的总数。
properties (SetAccess = private) NumPassingAssertions NumFailingAssertions FinalizedNumPassingAssertions FinalizedNumFailingAssertions end
扩展测试会话的运行
要扩展整个 TestSuite
数组的运行,请使用 protected
访问权限覆盖 methods
块中的 runSession
方法。runSession
显示关于 Test
元素总数的信息,初始化插件用于生成文本输出的属性,并调用超类方法来触发整个测试运行。在测试框架完成对超类方法的计算后,runSession
通过调用辅助方法 printAssertionSummary
(请参阅定义辅助方法)显示断言计数摘要。框架在客户端对此方法进行一次计算。
methods (Access = protected) function runSession(plugin,pluginData) suiteSize = numel(pluginData.TestSuite); fprintf("## Running a total of %d tests\n\n",suiteSize) plugin.FinalizedNumPassingAssertions = 0; plugin.FinalizedNumFailingAssertions = 0; runSession@ ... matlab.unittest.plugins.TestRunnerPlugin(plugin,pluginData) fprintf("## Done running tests\n") plugin.printAssertionSummary end end
扩展共享测试脚手架和测试用例的创建
将侦听程序添加到 AssertionPassed
和 AssertionFailed
事件以统计断言数。要添加这些侦听程序,请扩展测试框架用于创建测试内容的方法。测试内容包括每个 Test
元素的 TestCase
实例,TestClassSetup
和 TestClassTeardown
methods
块的类级别 TestCase
实例,以及在 TestCase
类具有 SharedTestFixtures
属性时使用的 Fixture
实例。将这些创建方法添加到具有 protected
访问权限的 methods
代码块。
在覆盖创建方法时调用相应的超类方法。这些创建方法返回测试框架针对其各自上下文创建的内容。当使用 incrementPassingAssertionsCount
和 incrementFailingAssertionsCount
辅助方法实现上述方法时,将插件所需的侦听程序添加到返回的 Fixture
或 TestCase
实例。
methods (Access = protected) function fixture = createSharedTestFixture(plugin,pluginData) fixture = createSharedTestFixture@ ... matlab.unittest.plugins.TestRunnerPlugin(plugin,pluginData); fixture.addlistener("AssertionPassed", ... @(~,~)plugin.incrementPassingAssertionsCount); fixture.addlistener("AssertionFailed", ... @(~,~)plugin.incrementFailingAssertionsCount); end function testCase = createTestClassInstance(plugin,pluginData) testCase = createTestClassInstance@ ... matlab.unittest.plugins.TestRunnerPlugin(plugin,pluginData); testCase.addlistener("AssertionPassed", ... @(~,~)plugin.incrementPassingAssertionsCount); testCase.addlistener("AssertionFailed", ... @(~,~)plugin.incrementFailingAssertionsCount); end function testCase = createTestMethodInstance(plugin,pluginData) testCase = createTestMethodInstance@ ... matlab.unittest.plugins.TestRunnerPlugin(plugin,pluginData); testCase.addlistener("AssertionPassed", ... @(~,~)plugin.incrementPassingAssertionsCount); testCase.addlistener("AssertionFailed", ... @(~,~)plugin.incrementFailingAssertionsCount); end end
扩展测试套件部分的运行
测试框架将整个 TestSuite
数组分成组,并将它们分配给工作进程进行处理。每个工作进程可以运行一个或多个测试套件部分。要自定义工作进程的行为,请在具有 protected
访问权限的 methods
块中覆盖 runTestSuite
方法。
扩展测试运行器以显示单个工作进程运行的每个测试组的标识符以及该组中 Test
元素的数量。此外,将通过和失败断言数存储在一个通信缓冲区中,以便客户端可以检索这些值来产生最终化的结果。像所有插件方法一样,runTestSuite
方法要求您在适当的时机调用对应的超类方法。在本例中,请在初始化属性之后和存储工作进程数据之前调用超类方法。测试框架在工作进程上计算 runTestSuite
的次数等于测试套件部分的个数。
methods (Access = protected) function runTestSuite(plugin,pluginData) suiteSize = numel(pluginData.TestSuite); groupNumber = pluginData.Group; fprintf("### Running a total of %d tests in group %d\n", ... suiteSize,groupNumber) plugin.NumPassingAssertions = 0; plugin.NumFailingAssertions = 0; runTestSuite@ ... matlab.unittest.plugins.TestRunnerPlugin(plugin,pluginData) assertionStruct = struct( ... "Passing",plugin.NumPassingAssertions, ... "Failing",plugin.NumFailingAssertions); plugin.storeIn(pluginData.CommunicationBuffer,assertionStruct) end end
为了存储特定于测试的数据,runTestSuite
的实现包含对 Parallelizable
接口的 storeIn
方法的调用。当工作进程必须向客户端报告时,请使用 storeIn
和 retrieveFrom
。在此示例中,在从超类方法返回后,NumPassingAssertions
和 NumFailingAssertions
包含对应于一组测试的通过和失败断言数。由于 storeIn
只以一个输入参量的形式接受工作进程数据,因此结构体 assertionStruct
使用两个字段对断言计数进行分组。
扩展最终测试套件部分的报告
扩展 reportFinalizedSuite
以通过检索每个最终测试套件部分的测试数据来聚合断言计数。要检索为一个测试套件部分存储的结构,请在 reportFinalizedSuite
的作用域内调用 retrieveFrom
方法。要在缓冲区为空的情况下(例如当致命断言失败阻止所有工作进程写入缓冲区时)返回默认数据 (自 R2024a 起),请在调用 retrieveFrom
方法时指定 DefaultData
。将字段值添加到对应的类属性,并调用超类方法。测试框架在客户端上计算此方法的次数等于测试套件部分的个数。
methods (Access = protected) function reportFinalizedSuite(plugin,pluginData) assertionStruct = plugin.retrieveFrom( ... pluginData.CommunicationBuffer, ... DefaultData=struct("Passing",0,"Failing",0)); plugin.FinalizedNumPassingAssertions = ... plugin.FinalizedNumPassingAssertions + assertionStruct.Passing; plugin.FinalizedNumFailingAssertions = ... plugin.FinalizedNumFailingAssertions + assertionStruct.Failing; reportFinalizedSuite@ ... matlab.unittest.plugins.TestRunnerPlugin(plugin,pluginData) end end
定义辅助方法
在具有 private
访问权限的 methods
块中,定义三个辅助方法。这些方法累加每个运行测试套件部分中的通过或失败断言数,并输出断言计数摘要。
methods (Access = private) function incrementPassingAssertionsCount(plugin) plugin.NumPassingAssertions = plugin.NumPassingAssertions + 1; end function incrementFailingAssertionsCount(plugin) plugin.NumFailingAssertions = plugin.NumFailingAssertions + 1; end function printAssertionSummary(plugin) fprintf("%s\n",repmat('_',1,30)) fprintf("Total Assertions: %d\n", ... plugin.FinalizedNumPassingAssertions + ... plugin.FinalizedNumFailingAssertions) fprintf("\t%d Passed, %d Failed\n", ... plugin.FinalizedNumPassingAssertions, ... plugin.FinalizedNumFailingAssertions) end end
插件类定义
以下代码提供了 AssertionCountingPlugin
类的完整内容。
classdef AssertionCountingPlugin < ... matlab.unittest.plugins.TestRunnerPlugin & ... matlab.unittest.plugins.Parallelizable properties (SetAccess = private) NumPassingAssertions NumFailingAssertions FinalizedNumPassingAssertions FinalizedNumFailingAssertions end methods (Access = protected) function runSession(plugin,pluginData) suiteSize = numel(pluginData.TestSuite); fprintf("## Running a total of %d tests\n\n",suiteSize) plugin.FinalizedNumPassingAssertions = 0; plugin.FinalizedNumFailingAssertions = 0; runSession@ ... matlab.unittest.plugins.TestRunnerPlugin(plugin,pluginData) fprintf("## Done running tests\n") plugin.printAssertionSummary end function fixture = createSharedTestFixture(plugin,pluginData) fixture = createSharedTestFixture@ ... matlab.unittest.plugins.TestRunnerPlugin(plugin,pluginData); fixture.addlistener("AssertionPassed", ... @(~,~)plugin.incrementPassingAssertionsCount); fixture.addlistener("AssertionFailed", ... @(~,~)plugin.incrementFailingAssertionsCount); end function testCase = createTestClassInstance(plugin,pluginData) testCase = createTestClassInstance@ ... matlab.unittest.plugins.TestRunnerPlugin(plugin,pluginData); testCase.addlistener("AssertionPassed", ... @(~,~)plugin.incrementPassingAssertionsCount); testCase.addlistener("AssertionFailed", ... @(~,~)plugin.incrementFailingAssertionsCount); end function testCase = createTestMethodInstance(plugin,pluginData) testCase = createTestMethodInstance@ ... matlab.unittest.plugins.TestRunnerPlugin(plugin,pluginData); testCase.addlistener("AssertionPassed", ... @(~,~)plugin.incrementPassingAssertionsCount); testCase.addlistener("AssertionFailed", ... @(~,~)plugin.incrementFailingAssertionsCount); end function runTestSuite(plugin,pluginData) suiteSize = numel(pluginData.TestSuite); groupNumber = pluginData.Group; fprintf("### Running a total of %d tests in group %d\n", ... suiteSize,groupNumber) plugin.NumPassingAssertions = 0; plugin.NumFailingAssertions = 0; runTestSuite@ ... matlab.unittest.plugins.TestRunnerPlugin(plugin,pluginData) assertionStruct = struct( ... "Passing",plugin.NumPassingAssertions, ... "Failing",plugin.NumFailingAssertions); plugin.storeIn(pluginData.CommunicationBuffer,assertionStruct) end function reportFinalizedSuite(plugin,pluginData) assertionStruct = plugin.retrieveFrom( ... pluginData.CommunicationBuffer, ... DefaultData=struct("Passing",0,"Failing",0)); plugin.FinalizedNumPassingAssertions = ... plugin.FinalizedNumPassingAssertions + assertionStruct.Passing; plugin.FinalizedNumFailingAssertions = ... plugin.FinalizedNumFailingAssertions + assertionStruct.Failing; reportFinalizedSuite@ ... matlab.unittest.plugins.TestRunnerPlugin(plugin,pluginData) end end methods (Access = private) function incrementPassingAssertionsCount(plugin) plugin.NumPassingAssertions = plugin.NumPassingAssertions + 1; end function incrementFailingAssertionsCount(plugin) plugin.NumFailingAssertions = plugin.NumFailingAssertions + 1; end function printAssertionSummary(plugin) fprintf("%s\n",repmat('_',1,30)) fprintf("Total Assertions: %d\n", ... plugin.FinalizedNumPassingAssertions + ... plugin.FinalizedNumFailingAssertions) fprintf("\t%d Passed, %d Failed\n", ... plugin.FinalizedNumPassingAssertions, ... plugin.FinalizedNumFailingAssertions) end end end
创建测试类
在当前文件夹的名为 ExampleTest.m
的文件中,通过以下操作创建 ExampleTest
参数化测试类。此类产生 300 个测试,其中 100 个是断言测试,用于比较 1 和 10 之间的伪随机整数。
classdef ExampleTest < matlab.unittest.TestCase properties (TestParameter) num1 = repmat({@()randi(10)},1,10) num2 = repmat({@()randi(10)},1,10) end methods (Test) function testAssert(testCase,num1,num2) testCase.assertNotEqual(num1(),num2()) end function testVerify(testCase,num1,num2) testCase.verifyNotEqual(num1(),num2()) end function testAssume(testCase,num1,num2) testCase.assumeNotEqual(num1(),num2()) end end end
将插件添加到测试运行器并运行测试
根据 ExampleTest
类创建一个测试套件。
suite = testsuite("ExampleTest");
创建一个不含任何插件的测试运行器。以下代码将创建一个不产生任何输出的静默运行器。
runner = testrunner("minimal");
您现在可以添加任何您选择的插件。将 AssertionCountingPlugin
类的一个实例添加到运行器中,以并行方式运行测试。(如果对运行器调用 run
方法,您还可以在串行模式下运行相同的测试。)
runner.addPlugin(AssertionCountingPlugin) runner.runInParallel(suite);
## Running a total of 300 tests Split tests into 18 groups and running them on 6 workers. ---------------- Finished Group 1 ---------------- ### Running a total of 20 tests in group 1 ---------------- Finished Group 2 ---------------- ### Running a total of 20 tests in group 2 ---------------- Finished Group 3 ---------------- ### Running a total of 19 tests in group 3 ---------------- Finished Group 4 ---------------- ### Running a total of 19 tests in group 4 ---------------- Finished Group 5 ---------------- ### Running a total of 18 tests in group 5 ---------------- Finished Group 6 ---------------- ### Running a total of 18 tests in group 6 ---------------- Finished Group 7 ---------------- ### Running a total of 18 tests in group 7 ---------------- Finished Group 8 ---------------- ### Running a total of 17 tests in group 8 ---------------- Finished Group 9 ---------------- ### Running a total of 17 tests in group 9 ----------------- Finished Group 10 ----------------- ### Running a total of 17 tests in group 10 ----------------- Finished Group 11 ----------------- ### Running a total of 16 tests in group 11 ----------------- Finished Group 12 ----------------- ### Running a total of 16 tests in group 12 ----------------- Finished Group 13 ----------------- ### Running a total of 15 tests in group 13 ----------------- Finished Group 14 ----------------- ### Running a total of 15 tests in group 14 ----------------- Finished Group 16 ----------------- ### Running a total of 14 tests in group 16 ----------------- Finished Group 15 ----------------- ### Running a total of 15 tests in group 15 ----------------- Finished Group 17 ----------------- ### Running a total of 14 tests in group 17 ----------------- Finished Group 18 ----------------- ### Running a total of 12 tests in group 18 ## Done running tests ______________________________ Total Assertions: 100 88 Passed, 12 Failed
另请参阅
函数
类
matlab.unittest.plugins.TestRunnerPlugin
|matlab.unittest.plugins.Parallelizable
|matlab.unittest.TestRunner
|matlab.unittest.TestSuite
|matlab.unittest.TestCase
|matlab.unittest.fixtures.Fixture