Bitcrush Specialist in Embedded Software Testing and Quality Processes https://bitcrushtesting.com/en/ Bitcrush https://bitcrushtesting.com/en/ https://bitcrushtesting.com/image/brand/icon-1-1.png https://bitcrushtesting.com/image/brand/icon-2-1.png 2026-06-15T20:42:07Z Testing a Yocto-built Image in QEMU Bernhard Trinnes https://bitcrushtesting.com/ https://bitcrushtesting.com/en/blog/run-yocto-image-in-qemu/ 2025-09-25T19:00:00Z 2025-09-25T19:00:00Z With QEMU you can boot an embedded Linux system directly on your host PC without needing real hardware. For this walkthrough we use Poky, the Yocto Project's reference build system. It is compact, easy to extend and includes all the tools needed to get started building, running and testing Linux images.

The steps shown here are not limited to Poky — they transfer directly to your own images, for example on SoC platforms like i.MX. The workflow for building, booting and testing stays essentially the same; only the target machine and additional layers differ.

These characteristics make Poky an ideal starting point for gaining hands-on experience with emulators and integrating automated tests into the development process. This article covers the full workflow: from downloading the sources, through the image build, to booting in QEMU and running tests.

Prerequisites

  • A Linux host system (e.g. Ubuntu or Fedora)
  • git, tar and standard build tools (gcc, make, python3, etc.)
  • At least 50 GB of free disk space

Step 1: Clone the Poky Repository

git clone git://git.yoctoproject.org/poky
cd poky

Optionally, check out a stable branch, e.g. scarthgap:

git checkout scarthgap

Step 2: Initialise the Build Environment

source oe-init-build-env

This creates the build/ directory. Settings in conf/local.conf can be adjusted, for example the number of parallel jobs:

BB_NUMBER_THREADS = "8"
PARALLEL_MAKE = "-j 8"

Step 3: Build the Image

A minimal image:

bitbake core-image-minimal

Depending on your hardware the build can take several hours. The output is located at:

build/tmp/deploy/images/qemux86-64/

Step 4: Boot the Image in QEMU

Poky ships a launch script:

runqemu qemux86-64

This boots a virtual system via QEMU. A login shell appears (default user: root, no password).

Step 5: Run Tests

The Yocto Project supports a test infrastructure (oeqa). Custom tests can be added as Python test cases under meta/lib/oeqa/runtime/. Here is an example ping test:

from oeqa.runtime.case import OERuntimeTestCase
from oeqa.core.decorator.oetimeout import OETimeout

class PingTest(OERuntimeTestCase):
    @OETimeout(60)
    def test_ping(self):
        status, output = self.target.run("ping -c 3 8.8.8.8")
        self.assertEqual(status, 0, msg="Ping failed:\n%s" % output)

Step 6: Failure Analysis

If tests fail:

  • Check build/tmp/log/oeqa/
  • Start QEMU interactively with runqemu
  • Inspect log files in build/tmp/work/.../temp/log.do_*

Conclusion

QEMU provides a flexible way to boot embedded Linux images quickly and independently of real hardware. Developers can use it to trace the boot process, verify functionality and run automated tests. Poky is used here purely as an example for image creation — the same workflow applies equally to custom or board-specific builds, such as for i.MX.

This makes QEMU a valuable tool in embedded development: for early prototyping, continuous testing and stable release processes.

]]>
Software Requirements for Safety-Critical Applications Bernhard Trinnes https://bitcrushtesting.com/ https://bitcrushtesting.com/en/blog/software-requirements-for-safety-applications/ 2025-09-12T19:00:00Z 2025-09-12T19:00:00Z When developing safety-critical systems, software requirements are not just an item on a checklist — they are the foundation for everything that follows. From design and verification through certification to maintenance: the clarity and correctness of your requirements can determine whether your product succeeds or fails, especially when lives depend on how the system behaves.

Whether you are developing an insulin pump or a pressure regulator for a nuclear power plant, writing software requirements for safety-critical applications demands a disciplined approach. This article shows how to formulate precise, unambiguous and verifiable requirements that satisfy industry standards and support certification.

Writing High-Quality Requirements

Most requirements are still written in natural language (NL). NL is flexible and easy to understand, but prone to ambiguity. A phrase like "when needed" can be interpreted in multiple ways — and thereby cause critical defects. Safety-related development therefore needs rules and structure even in NL. A good software requirement is:

  • correct and clear
  • consistent and non-contradictory
  • testable and verifiable — it must define measurable criteria
  • free from conjunctions and exceptions
  • role-specific — it names the user type

Example structure:

The operator shall be able to [action] on the [object] in order to achieve [desired state].

Every requirement should define:

  • Who performs the action (user role)
  • What is done (task and object)
  • Why, or what result is expected
  • Who receives or uses the result

Style and Structure

A well-formulated requirement typically follows a Subject–Predicate–Object (S-P-O) pattern:

Subject Predicate Object
The system shall display the current heart rate.

ISO/IEC/IEEE 29148:2018 describes how to write clear and testable requirements and standardises the use of modal verbs:

  • Shall: Identifies a mandatory requirement.
  • Will: Describes a fact or intention, not a requirement.
  • Should: A recommended requirement; not mandatory, but deviations must be justified.
  • Must: Sometimes used for laws of nature or unavoidable consequences; should not be used in place of "shall" for requirements.

Best practice: use "shall" only once per requirement. Avoid compound requirements joined with "and", "or" or "with". Combining actions makes the requirement untestable and creates multiple requirements in one. Words like "is", "are", "was" or "must" are suitable for descriptive or introductory sections.

Every requirement should describe what the system must do, not how it is to be implemented. This is crucial for testability and traceability. Example:

The system shall use a MySQL database to store ECG data.

This describes the how, not the what. The choice of database is a design decision that belongs in downstream documents such as the Software Architecture Document (SAD) or Detailed Design Document (DDD).

Improving Requirements Through Controlled Language

One way to reduce ambiguity is to introduce a controlled language — a simplified subset of English for technical documentation. In aerospace, for example, ASD-STE100 (Simplified Technical English) is used.

Controlled language enforces:

  • a defined vocabulary
  • simple sentence structures
  • avoidance of synonyms or ambiguous terms

A style guide for controlled language enables consistent and easily verifiable requirements across multiple documents and roles.

Avoiding Ambiguity

Ambiguity can arise from lexical, syntactic or semantic issues:

  • Lexical ambiguity: A word has multiple meanings. Example: "bank" can mean a financial institution or a river bank.
  • Syntactic ambiguity: A sentence admits multiple grammatical structures with different meanings. Example: "medical history database" — is it a database for medical histories, or a historical database?
  • Attachment ambiguity: A phrase (often prepositional) can attach to different parts of the sentence. Example: "The state machine can be implemented as part of the driver for communication with the watchdog CPU." Meanings:
    • The driver handles communication and the state machine is part of it.
    • The state machine controls the communication.
  • Semantic ambiguity: A sentence is unclear in context, even without lexical or syntactic problems. Example: "The system shall stop the motor before it overheats." — Does "it" refer to the system or the motor?

Ambiguity often creeps in through vague terms, passive constructions or unclear references. Example:

"The system shall activate the warning signal shortly after fault detection."

What does "shortly" mean? Who activates the signal? Questions like these are red flags in a safety-critical context.

Avoid:

  • Relative terms: powerful, acceptable, fast, reliable
  • Unclear quantifiers: all, most, some, any
  • Passive voice: "is activated" vs. "the controller activates"
  • Temporal vagueness: soon, periodically, later
  • Compound requirements: two "shall" clauses joined with "and/or"

Every requirement must be atomic, verifiable and unambiguous.

Formalising Requirements

For higher safety integrity levels it becomes necessary to formalise requirements — translating them into a formal language such as PSL (Property Specification Language) or LTL (Linear Temporal Logic) that tools can check and simulate.

Formalisation enables:

  • complete analysis of boundary cases
  • traceability from high-level requirements to implementation
  • automated verification in safety audits

Practical Examples

Infusion Pump — Flow Rate Control

Natural Language (NL SR)

"The system should deliver the correct dose at the correct rate."

Problems: vague terms such as "correct dose" and "correct rate". No subject defined. Not testable.

Controlled Natural Language (cNL SR)

"When the infusion programme is started, the control unit shall deliver the programmed medication volume at a flow rate between 0.1 and 10.0 ml/h with an accuracy of ±5 %."

  • Subject: control unit
  • Action: shall deliver
  • Parameters: flow rate, dose, accuracy
  • Trigger: start of infusion programme

Heart Rate Monitor — Triggering an Alarm

Natural Language (NL SR)

"The system must trigger an alarm when the heart rate is abnormal."

Problems: what does "abnormal" mean? How loud or fast must the alarm be? When exactly should it trigger?

Controlled Natural Language (cNL SR)

"When the patient's heart rate falls below 40 bpm or exceeds 180 bpm and this condition persists for more than 3 seconds, the alarm controller shall activate a 90 dB alarm within 1 second."

  • Clear thresholds: 40 and 180 bpm
  • Time condition: more than 3 seconds
  • Action: activate 90 dB alarm
  • Time limit: within 1 second

Radiation Beam Activation with Hardware Interlock

Natural Language (NL SR)

"The system shall activate the beam when the operator starts treatment."

Problems: no mention of safety interlocks or conditions. "Activate" is unclear — switch on physically or merely prepare?

Controlled Natural Language (cNL SR)

"The system shall activate the radiation beam only when the gantry is in the correct position, the hardware interlock is closed, and the treatment console receives a valid 'Start' command from the operator."

  • Multiple necessary preconditions
  • Unambiguous subject and trigger
  • "Only when" logic accounts for safety

Requirements Review

Developing a safety-critical product is rarely the work of a single person. Before proceeding to the next steps in the V-model, the software requirements must be reviewed. But that is a topic for another time.

Further Reading

]]>
Testing for Functional Safety in the Context of IEC 62304 Bernhard Trinnes https://bitcrushtesting.com/ https://bitcrushtesting.com/en/blog/iec62304_testing/ 2025-09-12T19:00:00Z 2025-09-12T19:00:00Z When developing software for medical devices, safety is not merely a goal — it is a legal and ethical obligation. Among the many standards that govern this domain, IEC 62304 stands out as the central international standard for the lifecycle of medical device software. A fundamental part of IEC 62304 conformance is systematic testing for functional safety. But what does that mean in practice?

As engineers and software developers we are trained to think in terms of structure, reliability and traceability. This mindset aligns closely with the philosophy of IEC 62304.

Understanding Functional Safety

Functional safety ensures that a system responds correctly to its inputs, particularly in failure situations. In the medical domain this means: the software must not endanger patients or users, even in the event of foreseeable failures.

For example, a software-controlled infusion pump administering medication must handle sensor faults, communication failures and user errors in a way that prevents hazardous events. This is where functional safety comes in.

IEC 62304 and Safety Classification

IEC 62304 defines three software safety classes:

  • Class A: No injury or health hazard possible
  • Class B: Non-serious injury possible
  • Class C: Death or serious injury possible

Depending on the classification, the scope of testing and verification activities increases substantially. For Class C software, comprehensive test strategies must be implemented — including unit tests, integration tests and system verification with traceability to risk controls.

The Role of Testing

Testing in the context of IEC 62304 is not just about finding defects — it is about demonstrating that the software meets its safety requirements under all foreseeable conditions. Testing must therefore be:

  • Systematic: following a documented and traceable process
  • Risk-based: tests are prioritised according to potential severity of harm
  • Bidirectionally traceable: every test case is linked to a requirement and vice versa

Typical testing activities include:

  • Static analysis for early detection of potential defects
  • Unit tests to verify logic at the function level
  • Integration tests to confirm communication between modules
  • System tests with fault injection and boundary cases
  • Verification of risk controls, particularly in the area of hardware-software interaction

Traceability

Precision and rigorous documentation are well-known strengths of good engineering teams — and here they pay off. IEC 62304 demands traceability across the entire software lifecycle. Every requirement must be linked to:

  • Its origin in the risk management process (typically per ISO 14971)
  • The corresponding architectural or design element
  • The test cases that verify it

Tools such as Polarion, Codebeamer or open-source alternatives like ReqView and Robot Framework can support this rigour.

Integration with ISO 14971 and IEC 60601

Testing for functional safety cannot happen in isolation. It must be aligned with related standards, including:

  • ISO 14971 for risk management
  • IEC 60601 for electrical safety and essential performance

For example, a Class C software system might include a software-based failsafe that activates when a hardware sensor reports inconsistent data. This behaviour must be verified both in software (simulated sensor fault) and in system testing (actual fault injection).

Closing Thoughts

Testing for functional safety to IEC 62304 is not just a regulatory checkbox — it is a mindset. It requires a deep understanding of both the software architecture and the clinical risks involved.

A focus on process discipline, risk management and technical integrity provides a solid foundation for developing safe and reliable medical software.

In the end, good testing is not about covering every line of code — it is about being able to sleep soundly knowing that the software will not harm anyone.

]]>
Unit Tests That Add Value Bernhard Trinnes https://bitcrushtesting.com/ https://bitcrushtesting.com/en/blog/unit-tests-that-add-value/ 2025-09-12T19:00:00Z 2025-09-12T19:00:00Z Unit tests are a standard tool of professional software development. Yet they often end up as tedious formalities: tests that barely verify anything, break at every small change, or merely confirm the obvious. For unit tests to deliver real value, you need clarity about their purpose and benefit.

Common Problems with Unit Tests

  • Trivial tests: they only check getters/setters or mirror the implementation line by line. Value: zero.
  • Fragile tests: even minor refactorings cause failures, without any change in behaviour.
  • Missing focus: tests verify incidentals rather than the business-critical rules.
  • Unreadability: complex test logic that is harder to understand than the production code itself.

What Makes a Test Valuable?

A good unit test satisfies at least one of these conditions:

  • It covers business-relevant rules (e.g. price calculation, safety check).
  • It builds confidence in refactorings by testing behaviour rather than implementation.
  • It documents intended behaviour in a readable and traceable way.
  • It provides fast feedback when something goes wrong.

Principles for Adding Value

  1. Test behaviour, not implementation. Instead of assertEquals(42, obj.getValue()) just because getValue() happens to return 42, build a scenario that makes business sense.
  2. Keep tests independent. Every test must be able to run in isolation, without hidden dependencies.
  3. Use descriptive names. A test named shouldRejectInvalidPassword() is also documentation.
  4. Don't forget edge cases. Failure paths, boundary values and exceptions often reveal more than happy-path tests.
  5. Think about maintainability. Tests are code and need care. Clear structures and helper functions make them robust.

Example: Valuable vs. Trivial

Trivial:

TEST(UserTest, GetNameReturnsName) {
    User u("Alice");
    EXPECT_EQ("Alice", u.getName());
}

No real value — it only confirms that getName() returns what was set in the constructor.

Valuable:

TEST(PasswordTest, RejectsShortPasswords) {
    User u("Alice");
    EXPECT_THROW(u.setPassword("123"), std::invalid_argument);
}

This test verifies a business rule: passwords must meet minimum requirements.

Conclusion

Unit tests that add value focus on behaviour, rules and robustness. Their job is not to mirror the implementation but to build confidence and preserve knowledge. Poor tests are dead weight; good tests are investments in quality and maintainability.

]]>