Creating intelligent testcases using uvm_report_catcher

In this blog, we will go through the verification method for erroneous scenarios which helps you to prevent your RTL from error cases and also helps you to find out bugs easily.

This blog is written based on UVM feature for changing severity of message by using “uvm_report_catcher”. For using this feature let’s take an example of verifying any IP. As we know, we are testing normal scenarios as well as we also verify the behavior of IP with the error scenarios to check that error related interrupt is asserted or not or error indicated register is updated or not.

Let's say, a sequence (“proc_crc_err_seq”) is developed to check CRC error condition and we are expecting error for it. Then, a normal sequence “proc_norm_seq” is driven to the IP and in that case we are not expecting any error.

After developing the test, we can run the simulation and check that error is injected properly or not. Also we check that, IP/VIP has reported error properly or not from log file and also from the waveform. But it is not possible for us to go and check waveform and log file every time.

In the regression we check that any test failed or not and we debug failures. Here, in this test we are expecting CRC error. So we can keep the test in the expected failure list. But we need to keep track few things like,

(1) CRC error generation is must.

For example, somebody has updated RTL or testbench and due to that CRC error is not generated. It will create a big issue later on as error scenario is not verified.

(2) There shall not be any error other than “CRC error”.

It is good to track whether RTL or testbench is generating any other error or not for the above condition.

(3) CRC error shall be generated in the first sequence only.

If RTL generates CRC error for a good sequence driven after a bad sequence, then it creates another trouble.

To overcome this problem, we can use “uvm_report_catcher” class. Below is the example, how to demote this error and also demonstrate how to overcome above three problems by implementing proper logic in the testcase.

Example:

class test1_demoter extends uvm_report_catcher;

  bit err_demoted;

  `uvm_object_utils(test1_demoter)

  function new(string name="test1_demoter");

    super.new(name);

  endfunction : new

  function action_e catch();

    if((get_severity() == UVM_ERROR) && 

        (get_message() == "CRC error generated."))begin

      set_severity(UVM_INFO);

      err_demoted = 1;

    end

    return THROW;

  endfunction : catch

endclass : test1_demoter

class crc_error_test extends uvm_test;

  test1_demoter demoter;

  …

  …

  function void build_phase(uvm_phase phase);

    …

    demoter = test1_demoter::type_id::create("demoter");

    …

  endfunction : build_phase

  task main_phase(uvm_phase phase);

    uvm_report_cb::add(env.ag.mon, demoter);

    crc_err_seq.start(env.ag.seqr);

    uvm_report_cb::delete(env.ag.mon, demoter);

    if(demoter.err_demoted == 0)

      `uvm_report_error(get_full_name(),”CRC error is not detected by the test”)

    norm_seq.start(env.ag.seqr);

  endtask : main_phase

  ...

  ...

endclass :crc_error_test

In above example, we assume that, "CRC error generated." error will be generated from the monitor when CRC error is injected and we need to demote it. For it, “test1_demoter” class is used to demote the error. “err_demoted” variable is used for checking whether error is generated or not in the simulation. "catch" function is used to change the severity of the error message. 

In the testcase, test1_demoter is instantiated and registered with the “mon” object handle (env.ag.mon => from where the error message will be generated) before driving the error sequence. After driving the sequence, we delete the callback object from “mon” object handle and checks that error occurs or not in the previous sequence. Then we drive normal sequence.

So in the simulation, when "crc_err_seq" is driven, "CRC error generated." error will be generated but as we demote it, it will be displayed as "UVM_INFO" instead displaying as "UVM_ERROR" and also the error count will not be incremented. If this error is not generated then, "err_demoted" variable will not be set and from testcase, we can detect that CRC error is not generated by RTL or testbench.

If any other will be generated then UVM_ERROR message will be displayed and we can track it.

As we delete the callback, if any error is generated for "norm_seq" then also UVM_ERROR will be generated and we can debug the issue.

So at the end of simulation, we are not expecting any error and our regression shall be clean.

Reference:

(1) Universal Verification Methodology (UVM) 1.1 Class Reference, June 2011