Last edited time

Mar 19, 2024 12:28 PM

Reading list

Prompt Engineering tips

Status

1 case

Tools

ChatGPT

tags

documentation

Case 1. Complicated technical standard. IEC 61513 Nuclear power plants – Instrumentation and control important to safety – General requirements for systems

Inputs: IEC 61513 Nuclear power plants – Instrumentation and control important to safety – General requirements for systems.

Model parameters:

1. GPT 4
2. Temperature 0

Tasks:

1. Analysis of standard
2. General description of standard
3. Elaboration of native requirements
4. Preparation of elaborated requirements
5. Assignment unique ID to each requirement

For the prompt preparation CO-STAR Framework has been utilized.

Breaking Down the CO-STAR Framework

C — Context — Sets the stage for the interaction, providing background information or the scenario in which the request is made.

O — Objective — Defines what the prompt aims to achieve, specifying the goal or the desired output from the language model.

S — Style — Specifies the desired writing or response style, guiding how the content should be presented or articulated.

T — Tone — Indicates the emotional character or attitude of the response, shaping how the message is emotionally conveyed.

A — Audience — Defines the intended audience or reader of the content, influencing the language, complexity, and approach of the response.

R — Response Format — Describes how the response should be structured, determining the organization and presentation of the content.

Plan preparation

Initially, LLM was tasked with drafting a standard analysis plan for requirements elaboration. This plan was subsequently refined by an expert in requirement management, along with a technical expert familiar with IEC 61513 standards.

Final plan is presented in App.1.

Objective was declared:

We have standard IEC 61513, necessary to decompose it to requirements and group them by categories based on IEC 61513 standard content clause by clause, applicable for the nuclear Instrumentation and control design process, result will be read by instrumentation and control expert, requirements will be used for the basic design of instrumentation and control for nuclear power plant construction.

IEC 61513 standard description and purpose.

In the second step, LLM was tasked with conducting a full standard analysis and providing a general description and purpose. This description helped LLM better understand the environments it was working in. A technical expert familiar with IEC 61513 standard assessed the results.

Result presented in App 2.

Native requirements preparation

For the test case was decided to use one section of IEC 61513 “5.4.2.6 Defence against CCF “

Text of section is presented App.3.

For the native requirements extraction we used context of general description and purpose and particular section of IEC 61513 and following prompt:

As a result we received native requirements.

Extracted native requirements are equal to original standard text, titles and IDs for each requirement are provided.

Elaborated Requirements

As a final step we asked LLM to provide elaborated requirements from native requirements.

For assessment of results we utilized following criteria:

1. Atomicity: This refers to the requirement being indivisible or unable to be further divided. Each requirement should address a specific aspect of the system or product being developed, and should not encompass multiple distinct functionalities. Atomic requirements help in better understanding, management, and tracking throughout the development process.
2. Consistency: Consistency ensures that there are no contradictions or conflicts among the requirements. All requirements should align with each other and with the overall goals and objectives of the project. Inconsistent requirements can lead to confusion, inefficiencies, and potentially costly rework during the development process.
3. Correctness: Correctness emphasizes the accuracy and precision of the requirements. Each requirement should accurately reflect the needs and expectations of the stakeholders, and should be technically feasible and achievable within the constraints of the project. Incorrect requirements can lead to misunderstandings, errors, and ultimately, project failure.
4. Unambiguity: Unambiguity refers to the clarity and lack of ambiguity in the requirements. Each requirement should be clearly defined and free from any vague or ambiguous language that could lead to different interpretations. Ambiguous requirements can result in misunderstandings, delays, and disagreements among stakeholders, impacting the overall success of the project.
5. Understandability: Understandability emphasizes the readability and comprehensibility of the requirements. They should be expressed in a clear and concise manner that can be easily understood by all stakeholders, including developers, testers, and end-users. Complex or overly technical language should be avoided to ensure that the requirements are accessible to everyone involved in the project.
6. Feasibility: Feasibility assesses the practicality and achievability of implementing the requirements within the given constraints, including time, budget, resources, and technology. Each requirement should be technically feasible and economically viable to implement. Infeasible requirements can lead to project delays, cost overruns, and dissatisfaction among stakeholders. Therefore, it's essential to evaluate the feasibility of each requirement before proceeding with development.

We prepared the elaborated requirements using the description and purpose of the IEC 61513 standard, native requirements, and the following few-shot prompt with examples of human-processed requirements:

Format requirements

As a result we received elaborated requirements in necessary format.

Validation

The results were assessed by a requirement management expert and an IEC 61513 expert. The requirements are of sufficient quality and meet the stated assessment criteria.

The final prompt, which follows the CO-STAR Framework, can be found in App 5.