Software tools: Between the lines

The traditional V-shaped software development lifecycle

As applications such as lane keeping assist add to automotive complexity, Mike Bartley from TVS explains how to bring requirements driven verification to hardware

 

Automotive applications such as lane keeping assistants are leading to a rapidly escalating system-on-chip (SoC) design complexity, meaning engineers have to embrace a number of verification methodologies to ensure a successful tapeout. Many organisations involved in SoC design have evolved from using verification environments based primarily on directed tests to include constrained-random techniques and, most recently, metric-driven verification.

The inclusion of constrained-random and metric-drive verification has helped greatly in driving down the number of bugs that lurk in an SoC design by the time it needs to be taped out. The methods are effective at finding bugs that traditional directed tests do not find. Metric-driven verification focuses effort on parts of the design that risk being ignored and prevents tests from concentrating too much on functions that have already been checked. But these methods cannot satisfy all the needs of verification and ignore one essential part of the design: the requirements that the SoC needs to satisfy.

Requirements-driven verification has been a vital part of high-reliability designs for decades, providing a key part of the V-model paradigm that these projects employ. The V diagram describes the journey that the implementation and verification team needs to follow through a project. It starts with high-level requirements that are progressively decomposed into system functions expressed in hardware and software. As these functions are assembled into subsystems and ultimately the final system, along the rising edge of the V, tests ensure that each function behaves as expected. Finally, system-level tests at the end of the V tie back to the system-level requirements. If those are satisfied, the design can proceed to production.

Similarly, for SoC designs, the requirements driven verification and test (RDVT) methodology enables project progress to be tracked and analysed from the perspective of the requirements of the system. In this way, every functional requirement can be mapped to a demonstration of implementation. Furthermore, any verification or test activity that does not relate to a requirement can be easily identified and questioned. If it is not needed, it can be removed from the test database to avoid wasting resources.

As an example, consider an automotive product that might have a new requirement for a lane keeping assistant, which provides support to the driver to stay in lane. This is obviously a high-level requirement that can be broken down hierarchically to hardware requirements (for example, a video input stream) and software requirements that can identify the lane separation lines in the video input. These hardware and software requirements will be broken down further to atomic requirements, which can be tested individually. As those atomic requirements are then integrated, combined requirements can be tested until it can finally demonstrate that the requirement for a lane keeping assistant is working.

RDVT provide a number of advantages that can pay dividends not just in terms of product quality but time to market. With conventional approaches to SoC-design test and verification, the engineers responsible for building the test harness need to wait for implementation code. The RDVT methodology makes it possible to shift a large amount of the verification process left in the project. The provision of requirements early in the cycle makes it possible to analyse the overall structure of the design and the tests it requires and enable the team to build an effective verification plan.

The RDVT methodology improves management visibility of the project. As well as providing analysis of code coverage, metrics can support the project managers by showing how well the design is meeting its requirements over time. Used across multiple projects, the metrics can help identify common or recurring issues, letting managers spot potential problems earlier in the cycle and have them addressed.

Through RDVT, the project managers can understand verification status in terms of externally focussed customer requirements – how well the product will meet its goals – rather than less transparent internal metrics. Because the metrics reflect the completeness of the design, RDVT support improved decision-making because they expose real-time data on the project’s actual status and reduce the reliance on indirect measures such as bug-detection rates.

A common criticism of the requirements-driven approach is that it is inflexible and often manually intensive. These aspects have helped prevent widespread use other than in military and avionics projects where requirements traceability forms a major part of the standards that development teams are required by law to follow.

In most projects that do not have strict legislative or contractual constraints, the interpretation of requirements can change during the course of the project to reflect engineering realities – which can include enhancements made possible by optimisations as much as problems encountered during implementation.

The lack of flexibility is made worse by a lack of automation. Even though a wide array is available for analysing source code and executables, there are no tools that automatically track the results of tests as they apply to requirements. The common practice is to stop tracing requirements at the point at which tests are defined. Tools typically do not ensure that requirements have tests defined to match them and that those tests were successfully completed. This approach results in a gap between the requirements capture tools that are commonly available and the results provided by test-focussed products.

The manual mapping of documents produced by test tools to requirements is time-consuming and prone to error. The translation or moving of the data involved can result in errors or even corruption of databases and maintaining good visibility of requirements through the entire tree is difficult to achieve. IC design has a further problem due to its multidisciplinary nature – the communications across pre- and post-silicon views and the software and system domains are complex.

To enable RDVT, there is a clear need for tools to help automate the process of tying test results as well as specifications to requirements, and tracking results over time to ensure changes to the design do not cause results in requirements from being untested in the final version. Such an environment also needs to provide documentations to prove that the implementation met requirements for audit purposes.

Tools are now beginning to appear on the market that support the RDVT approach. These can be built on top of a relational database to track how a project is developing over time for every feature and requirement, and how these features relate to the tests that are used to measure their progress.

The advantage of such tools is that they can be independent of specific EDA tools and flows. They can import requirements from a range of sources, from Excel spreadsheets through to the output of specialised tools such as Doors or Visure and many others. And through support for test standards such as UCDB and UCIS as well as direct translation, test data from multiple sources can be incorporated into the RDVT methodology.

Any changes to the requirement during the lifecycle of a project can be managed through direct editing within the tool or through a partial import from a requirements tool. The tool performs an intelligent analysis of where in the hierarchy a changed requirement should fit and, if there is a match, compares all the attribute values to look for changes. Any changes need to ensure that goals or sub-features still satisfy the changed requirement, and will be flagged as suspect to ensure the user is pro-active in accepting that the requirement is satisfied and that key requirements are not missed as project needs change.

Support for partial imports and changing requirements means RDVT can support agile development environments. Agile is an effective way to develop products in an environment where the market requirements are not clear and can evolve with the development. By allowing requirements to change and modify the associated tests over time, RDVT are effective methods for such projects.

The adoption of any new technique is not without its drawbacks and RDVT is no different. One potential obstacle for adopters is acquiring the discipline to write effective requirements at an early stage. However, organisations that raise such an objection have larger problems and are unlikely to be able to develop systems that meet unspecified requirements. Because the team needs to set requirements early on, this additional phase may seem to increase costs. However, this early capture is likely to lead to savings in the overall development costs because bugs are more expensive to fix later in the cycle. RDVT provide a highly effective way of supporting the shift left in SoC design and verification.

 

Mike Bartley is founder and CEO of TVS

TVS

Recommend to Colleagues: 

Add new comment

Plain text

  • Allowed HTML tags: <a> <em> <p> <h1> <h2> <h3> <code> <ul> <ol> <li> <dl> <dt> <dd> <strong>
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.

Follow Us:

Twitter icon
Facebook icon
RSS icon