Unit Test User-Defined Defects

As part of development and maintenance of the Polyspace® Query Language (PQL) defects you create, test your user-defined rules early and often. This topic shows how to unit-test user-defined PQL defects in isolation. Because PQL defects cannot be executed directly, you must inject them into a minimal test standard, package the standard as a .pschk file, and run the packaged standard against sample C/C++ code. The scope of the tests described in this topic is limited to a single coding rule. You can extend this approach to create unit tests for each rule in your coding standard and the test suite can serve as the test of the standard.

Required Artifacts

A PQL rule is a logical combination of various predicates that operate on C/C++ code and produces result. To unit test a PQL rule, you need:

  • PQL rule — The defect or predicate you want to test.

  • C/C++ code — Choose code that includes:

    • Test cases where the defect must occur.

    • Test cases where the defect must not occur.

    • Corner cases you want to validate

  • A test coding standard — PQL rules cannot run directly. The defect must be inserted into a minimal coding standard for testing purposes.

In this topic, you create unit tests for the defects created in the topic Detect Syntactic Issues Using Polyspace Query Language Syntactic Classes, where you define the defect in the folder SyntaxDefect. The defect is defined in these .pql files:

  •  SyntaxDefect/If_Statement/ImproperBraceforifStatement.pql

  •  SyntaxDefect/If_Statement/hasCompoundStatement.pql

  •  SyntaxDefect/If_Statement/getOpeningBraceLine.pql

  •  SyntaxDefect/If_Statement/getConditionEndLine.pql

  •  SyntaxDefect/If_Statement/TestRule.pql

Create Unit Test for Defect DoesNotHaveCompoundStatement

Create unit tests for the defect DoesNotHaveCompoundStatement:

  1. Copy If_Statement as a subfolder in a new folder PQLTests.

  2. In the folder PQLTests, initialize a new PQL project. At the command line, run:

    polyspace-query-language init
    This command creates the files main.pql and pql.json. After running this command, the folder PQLTests contains main.pql, pql.json, and the folder If_statement.

  3. In main.pql, add a test standard that includes only one defect from If_Statement. A PQL defect by itself cannot be check for an issue in C/C++ code. By packaging the defect under test in a standard, you can run a Bug Finder analysis that checks for only the PQL defect and test the defect:

    package main

// A test standard containing one test section and one defect under test
catalog TestDoesNotHaveCompoundStatement = {

    #[Description("Test Section")]
    section TestSection = {

        // Wrap the defect in a rule to make it runnable
        #[Description("If statement rules"), Id(TestRule)]
        rule TestRuleforDefect = {
            If_Statement.DoesNotHaveCompoundStatement
        }
    }
}
    This code creates a test standard that only contains the defect DoesNotHaveCompoundStatement from the If_Statement package.

  4. Package this test standard by running this command in the folder PQLTests:

    polyspace-query-language package
    This command creates the file TestDoesNotHaveCompoundStatement.pschk.

  5. Add C++ test cases in a file example.cpp:

    void foo (bool cond){
  int x;
  if(cond==true){ //expect-0-TestRule
    x = 0;
  }
}

void foo1 (bool cond){
  int x;
  if(cond==true) //expect-0-TestRule
  {
    x = 0;
  }
}

void foo2 (bool cond){
  int x;
  if(cond==true) //expect-1-TestRule
    x = 0;
}
    Annotate the expected violation of the rule as well as expected lack of violation using comments of this format:
    //expect-N-id
    Here N is the number of violation and id is the value of the attribute Id in the definition of the test rule. In this case, there is only one rule in the test standard and its Id is TestRule. In cases where you want to explicitly state that a defect is not expected, use an expect comment with N set to 0. In cases where you want to explicitly state that 1 defect is expected, use an expect comment with N set to 1.

    Observe that this code is annotated to expect a violation only when a if statement does not have a compound statement. The code also explicitly expects a lack of violation in the other cases. You can add multiple files with test cases that guard against false positive and false negatives. At this stage, the arrangement of the PQL files and C++ files in PQLTests is:

    PQLTests/
   If_Statement/                      ← copied folder containing predicates/defects
   main.pql                           ← test standard
   example.cpp                        ← test cases

  6. To run the test for the defect DoesNotHaveCompoundStatement, at the command line, enter:

    polyspace-query-language test example.cpp
    The test shows that the expected violations and expected lack of violations are present in the code, passing the test.

Create Unit Test for Defect ImproperBraceforifStatement

Similar to the defect, DoesNotHaveCompoundStatement, you can test the defect ImproperBraceforifStatement:

  1. Copy If_Statement as a subfolder in a new folder PQLTests2.

  2. In the folder PQLTests2, initialize a new PQL project. At the command line, run:

    polyspace-query-language init

  3. In main.pql, add a test standard that includes only the defect ImproperBraceforifStatement

    package main

catalog TestImproperBraceforifStatement = {  // changed standard name to reflect the new defect under test
#[Description("Test Section")]
	section TestSection = {
#[Description("If statement rules"), Id(TestRule)]
		rule TestRuleforDefect = {
			If_Statement.ImproperBraceforifStatement // new defect under test
		}
	}
}
    This code creates a test standard that only contains the defect ImproperBraceforifStatement from the If_Statement package.

  4. Package this test standard by running this command in the folder PQLTests2:

    polyspace-query-language package
    This command creates the file TestImproperBraceforifStatement.pschk.

  5. Add C++ test cases in a file example.cpp:

    void foo (bool cond){
  int x;
  if(cond==true){ //expect-0-TestRule
    x = 0;
  }
}

void foo1 (bool cond){
  int x;
  if(cond==true) //expect-1-TestRule
  {
    x = 0;
  }
}

void foo2 (bool cond){
  int x;
  if(cond==true) //expect-0-TestRule
    x = 0;
}

  6. To run the test for the defect ImproperBraceforifStatement, at the command line, enter:

    polyspace-query-language test example.cpp
    The test shows that the expected violations and expected lack of violations are present in the code, passing the test.

See Also