Creating intelligent testcases using uvm_report_catcher

In this blog, we will go through the verification method for erroneous scenarios which helps you to prevent your RTL from error cases and also helps you to find out bugs easily.

This blog is written based on UVM feature for changing severity of message by using “uvm_report_catcher”. For using this feature let’s take an example of verifying any IP. As we know, we are testing normal scenarios as well as we also verify the behavior of IP with the error scenarios to check that error related interrupt is asserted or not or error indicated register is updated or not.

Let's say, a sequence (“proc_crc_err_seq”) is developed to check CRC error condition and we are expecting error for it. Then, a normal sequence “proc_norm_seq” is driven to the IP and in that case we are not expecting any error.

After developing the test, we can run the simulation and check that error is injected properly or not. Also we check that, IP/VIP has reported error properly or not from log file and also from the waveform. But it is not possible for us to go and check waveform and log file every time.

In the regression we check that any test failed or not and we debug failures. Here, in this test we are expecting CRC error. So we can keep the test in the expected failure list. But we need to keep track few things like,

(1) CRC error generation is must.

For example, somebody has updated RTL or testbench and due to that CRC error is not generated. It will create a big issue later on as error scenario is not verified.

(2) There shall not be any error other than “CRC error”.

It is good to track whether RTL or testbench is generating any other error or not for the above condition.

(3) CRC error shall be generated in the first sequence only.

If RTL generates CRC error for a good sequence driven after a bad sequence, then it creates another trouble.

To overcome this problem, we can use “uvm_report_catcher” class. Below is the example, how to demote this error and also demonstrate how to overcome above three problems by implementing proper logic in the testcase.

Example:

class test1_demoter extends uvm_report_catcher;

  bit err_demoted;

  `uvm_object_utils(test1_demoter)

  function new(string name="test1_demoter");

    super.new(name);

  endfunction : new

  function action_e catch();

    if((get_severity() == UVM_ERROR) && 

        (get_message() == "CRC error generated."))begin

      set_severity(UVM_INFO);

      err_demoted = 1;

    end

    return THROW;

  endfunction : catch

endclass : test1_demoter

class crc_error_test extends uvm_test;

  test1_demoter demoter;

  …

  …

  function void build_phase(uvm_phase phase);

    …

    demoter = test1_demoter::type_id::create("demoter");

    …

  endfunction : build_phase

  task main_phase(uvm_phase phase);

    uvm_report_cb::add(env.ag.mon, demoter);

    crc_err_seq.start(env.ag.seqr);

    uvm_report_cb::delete(env.ag.mon, demoter);

    if(demoter.err_demoted == 0)

      `uvm_report_error(get_full_name(),”CRC error is not detected by the test”)

    norm_seq.start(env.ag.seqr);

  endtask : main_phase

  ...

  ...

endclass :crc_error_test

In above example, we assume that, "CRC error generated." error will be generated from the monitor when CRC error is injected and we need to demote it. For it, “test1_demoter” class is used to demote the error. “err_demoted” variable is used for checking whether error is generated or not in the simulation. "catch" function is used to change the severity of the error message. 

In the testcase, test1_demoter is instantiated and registered with the “mon” object handle (env.ag.mon => from where the error message will be generated) before driving the error sequence. After driving the sequence, we delete the callback object from “mon” object handle and checks that error occurs or not in the previous sequence. Then we drive normal sequence.

So in the simulation, when "crc_err_seq" is driven, "CRC error generated." error will be generated but as we demote it, it will be displayed as "UVM_INFO" instead displaying as "UVM_ERROR" and also the error count will not be incremented. If this error is not generated then, "err_demoted" variable will not be set and from testcase, we can detect that CRC error is not generated by RTL or testbench.

If any other will be generated then UVM_ERROR message will be displayed and we can track it.

As we delete the callback, if any error is generated for "norm_seq" then also UVM_ERROR will be generated and we can debug the issue.

So at the end of simulation, we are not expecting any error and our regression shall be clean.

Reference:

(1) Universal Verification Methodology (UVM) 1.1 Class Reference, June 2011

Introduction to encryption in System Verilog with Questa-Sim

In this blog, we will go through the encryption mechanism in verilog and systemverilog using Questa-Sim. The protection expressions “`pragma protect” for Verilog/SystemVerilog specify the encryption algorithm used to protect the source code.

For protecting the code, first you need to identify the portion of code which you need to encrypt and then use the “`pragma protect” directive in your code followed by compiling your code with “+protect” argument.

Example:

vlog +protect file.v or vlog +protect file.sv

By executing above code, it creates “file.vp”/”file.svp” in the “work” library which is a protected version of file and protected code will be encrypted. This file can be delivered to the end customer.

Here, you can give the file name during compilation option also.

Example:

vlog file.v +protect=protected_file.vp

After executing above code, it creates “protected_file.vp” which is a protected version of “file.v”. 

If specific pragma keyword is not specified then tool will use its default value.

Below are some examples and usages of syntax for encryption:

Figure 1

Figure 2

Here, encryption envelope which is encrypted using session key and recorded in decryption enveloped as data_block. A copy of session key is encrypted and recorded in key_block.

Below is the example to overwrite data_keyname and data_method to create decryption envelope from the encryption envelope.

Figure 3

Figure 4

Below is the example to overwrite key_keyname and key_method to create key_block for encryption of session key.

Figure 5

Figure 6

Same thing can be applied in the verilog code to encrypt any logic.

Reference:

(1) IEEE Standard for SystemVerilog— Unified Hardware Design, Specification, and Verification Language,IEEE Std 1800™-2012 

(2) Questa SIM User’s Manual, v10.3a

SED commands which can be used frequently to reduce coding time

This blog talks about the most usable “sed” commands in a day-to-day life to reduce coding time.

Nowadays, in IT industry many tasks are common and most of us are doing some repetitive jobs which consumes a lot of time. Some tasks are really very boring like “deleting words” or “inserting or replacing some text” in the file.

For one or two file, it is easy to do modification manually but when there is a number of files required same modification then we generally use “sed” and “awk” command which makes our life easier.

Following are the 10 most usable sed commands:

1. For doing search and replace in the file globally:

• sed –i ‘s/OLD_PATTERN/NEW_PATTERN/g’ filename

          Example: sed –i ‘s/Hi/Hello/g’ myfile

          Data in myfile before sed command:

              Hi

              How are you?

              Are you busy?

              Have you take lunch?

              I am going to mall.

          Data in myfile after sed command:

              Hello

              How are you?

              Are you busy?

              Have you take lunch?

              I am going to mall.

2. Appending any line after specific pattern matched in the file:

• sed -i "/PATTERN/a\  LINE_TO_BE_ADDED" filename

          Example: sed –i “Hello/a\ I am Munjal” myfile

          Data in myfile after sed command:

              Hello

              I am Munjal

              How are you?

              Are you busy?

              Have you take lunch?

              I am going to mall.

3. Deleting any line which contain specific pattern:

• sed –i ‘/PATTERN/d’ filename

          Example: sed –i ‘/Munjal/d’ myfile

          Data in myfile after sed command:

              Hello

              How are you?

              Are you busy?

              Have you take lunch?

              I am going to mall.

4. Deleting all lines from starting match to ending match:

• sed -i '/STARTING_MATCH_PATTERN/,/ENDING_MATCH_PATTERN/d' filename

          Example: sed –i ‘s/How are you/,/Have you take lunch/d’ myfile

          Data in myfile after sed command:

              Hello

              I am going to mall.

5. Case insensitive search and replace:

• sed -i 's/OLD_PATTERN/NEW_PATTERN/I' filename

          Example: sed –i ‘/hello/Hi/I’ myfile

          Data in myfile after sed command:

              Hi

              I am going to mall.

6. Inserting a blank line above any pattern:

• sed -i ‘s/PATTERN/\n&/g’ filename

          Example: sed –i ‘/going/\n&/g’ myfile

          Data in myfile after sed command:

              Hi

              I am going to mall.

7. Deleting multiple lines after pattern match with matching line:

• sed -i '/PATTERN/,+Nd' filename 

          Example: sed –i ‘/How are you/,+2d’ myfile

          Data in myfile before sed command:

              Hello

              How are you?

              Are you busy?

              Have you take lunch?

              I am going to mall.

          Data in myfile after sed command:

              Hello

              I am going to mall.

8. Inserting multiple lines in the file from another file:

• sed -i '/  PATTERN/r filename_to_add_lines' filename

          Example: sed –i ‘/Are you busy/r yourfile’ myfile

          Data in myfile before sed command:

              Hello

              How are you?

              Are you busy?

              Have you take lunch?

              I am going to mall.

          Data in “yourfile”:

              I am not busy.

              I just wake up.

          Data in myfile after sed command:

              Hello

              How are you?

              Are you busy?

              I am not busy.

              I just wake up.

              Have you take lunch?

              I am going to mall.

9. Converting multiple blank line to single blank line:

• sed –i ‘/^$/N;/^\n$/D’ filename

          By using above sed command, it replaces multiple consecutive blank lines with single blank line which looks better during reviewing the code.

10. Changing the pattern by grouping words:

• sed –i ‘s/\(.*\)PATTERN\(.*\)/\2PATTERN\1/g’ filename

          Example: sed –i ‘s/\(.*\) are \(.*\)/\2 are \1/g’ myfile

          Data in myfile after sed command:

              Hello

              you? are How

              Are you busy?

              I am not busy.

              I just wake up.

              Have you take lunch?

              I am going to mall.

11. Adding a new line before pattern match

• sed -i '/PATTERN/i LINE' filename

12. Adding a new line after pattern match

• sed -i '/PATTERN/a LINE' filename

Reference:

(1) http://www.grymoire.com/Unix/Sed.html

(2) http://www.computerhope.com/unix/used.htm

How coverage tracking helps in verification with Questa-Sim

This blog talks about the tracking of the functional coverage through Questa-Sim which helps users during verification. In verification, code coverage and functional coverage are a very important part and without it verification cannot be closed. With Questa-Sim, user can track the functional coverage report with respect to section of the specification.

Each protocol specification contains basic, intermediate and advanced information related to protocol in different sections. In the starting of any verification, basic functionality is checked and then moved on to intermediate and advanced functionality checking. Here there is a possibility to miss any verification of some basic functionality which creates architecture brake later on and that will create a big hurdle in verification as well as in RTL.

To handle this kind of problem, coverage tracking will be useful. For tracking, xml sheet is provided with each QUESTA VIP which contains information of different coverpoint and crosses covers from different section of specification and it shall be coded in the covergroup. It contains following information:

1. Section number, indicates section number from the specification
2. Title of the section, indicates title of the section
3. Description related to particular section, indicates description of the section
4. Link, Type, Weight and Goal

         Where,

• Link indicates the coverpoint/cross name with covergroup name to which it belongs,
• Type indicates coverpoint or cross,
• Weight indicates the weightage of that coverpoint/cross in the Covergroup,
• Goal indicates the target goal for a covergroup instance

Below is the snapshot of the xml sheet: 

Figure 1 Functional Coverage XML Plan

Figure 1 indicates the header of the functional coverage xml plan in which there are different columns for section, Title, Description, Link, Type, Weight and Goal are reside.

Figure 2 indicates the coverpoint and cross inside the Link column. Its goal is 100% in coverage and weightage is 1 for that covergroup. 

Figure 2 Functional Coverage XML Plan

How to use:

• Command to generate UCDB from xml: 

          xml2ucdb -format Excel functional_coverage_plan.xml testplan.ucdb

          (which creates “testplan.ucdb”)

• Command to merge all coverage:

          vcover merge coverage_merge.ucdb testplan.ucdb merged_test_coverage.ucdb

         (which creates “coverage_merge.ucdb” from testplan.ucdb and   merged_test_coverage.ucdb)

• Open Questa-Sim using “vsim &” command
• Open “coverage” window through Layout=>Coverage
• Open coverage_merge.ucdb file through “open” toolbar.
• Open coverage tracker through View=>Verification Management=>Tracker

How it helps in verification:

Let’s take example of UFS v2.0 specification. It contains total 15 sections.

Section 10 describes the packet type transmitted by UFS host and UFS device.

Section 11 describes different commands driven with command type packet.

Section 12 and 13 describes the various functionalities of UFS

Section 14 describes supported attributes, flags and descriptors.

Now during verification, the basic requirement is to check the transmission and reception of all packet types are supported or not by any IP. So, user can write tests and generate coverage report. Through coverage tracker, it is easy to find weather any packet transmission test is missed out or not.

If these kind of basic functionality is missed then it is good to cover it before checking any intermediate or advanced functionality. Because to cover most of the scenarios, it requires to transmits the different packets in specific manner. From the tracker, it is easy to track weather read/write operation is performed or no on attribute or flags.

Figure 3 Coverage Tracking Report

Figure 3 shows the coverage tracker. There is no coverpoint or cross for chapter 1 to 9. So they are seen disabled in the tracker. 56.85% functionality of section 10 is covered. It also shows each sub-section and its coverage.

Thanks to "Mentor Graphics" and "Mr. Yogesh Chaudhary (Mentor Graphics)" for helping in this blog.

Reference:

1. Questa Sim User Manual
2. JESD220B, Universal Flash Storage (UFS) v2.0