Sunday 27 September 2015

How coverage tracking helps in verification with Questa-Sim

This blog talks about the tracking of the functional coverage through Questa-Sim which helps users during verification. In verification, code coverage and functional coverage are a very important part and without it verification cannot be closed. With Questa-Sim, user can track the functional coverage report with respect to section of the specification.

Each protocol specification contains basic, intermediate and advanced information related to protocol in different sections. In the starting of any verification, basic functionality is checked and then moved on to intermediate and advanced functionality checking. Here there is a possibility to miss any verification of some basic functionality which creates architecture brake later on and that will create a big hurdle in verification as well as in RTL.

To handle this kind of problem, coverage tracking will be useful. For tracking, xml sheet is provided with each QUESTA VIP which contains information of different coverpoint and crosses covers from different section of specification and it shall be coded in the covergroup. It contains following information:
  1. Section number, indicates section number from the specification
  2. Title of the section, indicates title of the section
  3. Description related to particular section, indicates description of the section
  4. Link, Type, Weight and Goal
         Where,
    • Link indicates the coverpoint/cross name with covergroup name to which it belongs,
    • Type indicates coverpoint or cross,
    • Weight indicates the weightage of that coverpoint/cross in the Covergroup,
    • Goal indicates the target goal for a covergroup instance
Below is the snapshot of the xml sheet: 
Figure 1 Functional Coverage XML Plan
Figure 1 indicates the header of the functional coverage xml plan in which there are different columns for section, Title, Description, Link, Type, Weight and Goal are reside.

Figure 2 indicates the coverpoint and cross inside the Link column. Its goal is 100% in coverage and weightage is 1 for that covergroup. 
Figure 2 Functional Coverage XML Plan
How to use:
  • Command to generate UCDB from xml: 
          xml2ucdb -format Excel functional_coverage_plan.xml testplan.ucdb
          (which creates “testplan.ucdb”)
  • Command to merge all coverage:
          vcover merge coverage_merge.ucdb testplan.ucdb merged_test_coverage.ucdb
         (which creates “coverage_merge.ucdb” from testplan.ucdb and   merged_test_coverage.ucdb)
  • Open Questa-Sim using “vsim &” command
  • Open “coverage” window through Layout=>Coverage
  • Open coverage_merge.ucdb file through “open” toolbar.
  • Open coverage tracker through View=>Verification Management=>Tracker
How it helps in verification:

Let’s take example of UFS v2.0 specification. It contains total 15 sections.
Section 10 describes the packet type transmitted by UFS host and UFS device.
Section 11 describes different commands driven with command type packet.
Section 12 and 13 describes the various functionalities of UFS
Section 14 describes supported attributes, flags and descriptors.

Now during verification, the basic requirement is to check the transmission and reception of all packet types are supported or not by any IP. So, user can write tests and generate coverage report. Through coverage tracker, it is easy to find weather any packet transmission test is missed out or not.

If these kind of basic functionality is missed then it is good to cover it before checking any intermediate or advanced functionality. Because to cover most of the scenarios, it requires to transmits the different packets in specific manner. From the tracker, it is easy to track weather read/write operation is performed or no on attribute or flags.

Figure 3 Coverage Tracking Report
Figure 3 shows the coverage tracker. There is no coverpoint or cross for chapter 1 to 9. So they are seen disabled in the tracker. 56.85% functionality of section 10 is covered. It also shows each sub-section and its coverage.

Thanks to "Mentor Graphics" and "Mr. Yogesh Chaudhary (Mentor Graphics)" for helping in this blog.

Reference:
  1. Questa Sim User Manual
  2. JESD220B, Universal Flash Storage (UFS) v2.0

13 comments:

  1. Thanks a lot for putting my name in the blog.
    This is nicely written article which explains the coverage capturing of the protocol.
    Great effort Munjal!!!

    ReplyDelete
  2. Could anyone help me with a resource which list all coverage commands or steps to use it through GUI ? The Questasim manual doesn't seem to be available on google or may be I am using wrong keywords.

    ReplyDelete
    Replies
    1. You can get it from "Help" menu on the main window of the GUI.

      Delete
  3. Hello Munjal,
    Your explanation is very good and helped me use VM Feature of Questasim.
    I have one problem, I'm unable to link the coverage points (cvg, cp, bins etc) to the tests run. Can you please explain the linking portion? It would be a great help.

    I'm using the Questasim 10.3c version of the Questasim.

    Nevertheless your blog is easy to understand.

    Thanks and Regards,
    Pranesh

    ReplyDelete
    Replies
    1. Hi Pranesh,

      Thanks for going through the blog.

      First you need to validate the xml sheet with Questa add-ons. Once it is validated, you can use that xml during simulation.

      In xml, you need to take care for covergroup and cross name. It shall be same as implemented in code. If there is any extra space is added then it will not going to cover in the report.
      I don't know, this feature is how much stable with 10.3 version.

      Delete
  4. Hi Munjal,

    The article is nicely written .
    Can we annotate coverage numbers in hierarchical plans ? i.e. If I have separate plan for each block and I want to annotate it with result coverage data base , then is it possible ?

    Thanks & Regards,
    Bhavin Patel

    ReplyDelete
    Replies
    1. Thank you. Could you please elaborate more on it. You mean to say, you have different xml plan and you want to add all the generated ucdb into single ucdb. Is it so?

      Delete
    2. I have 3 questa test plan .
      1. Top Level test plan which is having test plan links for hierarchical test plans (Functional Coverage Plan and Test Plan)
      2. Functional Coverage plan which contains information about features and respective coverage items
      3. Test plan : It contains test case names , description and pass fail status .

      Now when we run regression and merge coverage data base with test plan , it generates final ucdb.

      And using result.ucdb we are able to annotate coverage numbers in top level testplan. But we are not able to annotate in other two plans. Can you give information about how we can annotate it in individual sub plans if it is possible ?

      Thanks & Regards,
      Bhavin Patel.

      Delete
    3. Up to my knowledge, you can provide the sheet name of the xls file while executing the command to generate ucdb. For exact command, you can take help from Questa team.

      Delete
  5. Hi Munjal,

    I see some weird scenario while generating Coverage report on QuestaSim. While writing the Single Coverpoint item I have mentioned the option.weight = 0, and for the Cross Coverpoint I have given the default option.weight = 1.

    What I observe post simulation is that even my Single coverpoint has weightage as 1. This is resulting in False Coverage Percentage.

    Please let me know if you got something for me on this.

    Regards
    XJL

    ReplyDelete
    Replies
    1. Hi John,

      I tried an example as per your comments and I am getting proper behavior. The coverpoint which has weight=0, is not considered in total coverage calculation. I am using Questa-10.5C_3, which Questa version, you are using?

      Delete
    2. If possible, could you please provide your example? My email id is, munjal.mmr@gmail.com

      Delete
  6. vcover report -details -html result.ucdb

    ReplyDelete