Introduction

Since the Yocto warrior release, the RAUC (Robust Auto-Update Controller) mechanism support has been added to Yogurt. It controls the procedure of updating a device with new firmware. This includes updating Linux kernel, Device Tree, and root filesystem. It currently does not update the bootloader. For detailed information about RAUC, visitrauc.readthedocs.io.

This manual describes how RAUC is used and implemented on various PHYTEC platforms. Note, that different modules use different bootloaders and flash storage devices, which affects the way things are handled with RAUC. Make sure to read the correct sections fitting your platform.

System Configuration

RAUC can be used with both eMMC and NAND flash storage. It is not, by default, enabled but our example can be configured and activated with the instructions shown below. RAUC can be used in different update scenarios. As an example, we configured the BSP to use an A/B setup to have a completely redundant system (except for the bootloader).

The partition layout is defined in the /etc/rauc/system.conf file. As an example, this is how it looks like for i.MX 8M Mini with eMMC flash storage:

[system]
compatible=phyboard-polis-imx8mm-3
bootloader=uboot
mountprefix=/mnt/rauc

[handlers]
pre-install=/usr/bin/rauc_downgrade_barrier.sh

[keyring]
path=ca.cert.pem

# System A
[slot.rootfs.0]
device=/dev/mmcblk2p2
type=ext4
bootname=system0

[slot.boot.0]
device=/dev/mmcblk2p1
type=vfat
parent=rootfs.0
                     
# System B           
[slot.rootfs.1]      
device=/dev/mmcblk2p4
type=ext4            
bootname=system1     
                     
[slot.boot.1]        
device=/dev/mmcblk2p3
type=vfat            
parent=rootfs.1

Note, that the devices specified in the slots are different depending on the selected machine.

Design Considerations

In order to prevent the system from locking up, it may be a good idea to utilize a hardware watchdog. In case the Linux Kernel does not boot or another catastrophic event occurs that prevents the system from operating normally, the hardware watchdog then resets the system. By default, the hardware watchdog is disabled. To enable it, refer to the corresponding BSP manual that fits your SoM.

Initial Setup

To use RAUC, the flash device needs to be written and the bootloader must be set up to start the A/B system.

Flash storage

eMMC

To flash the eMMC with the correct partitions a line needs to be added in the local.conf file of the BSP:

# Select a preconfigured A/B system setup for SD/eMMC images.
#WKS_FILES_mx6 = "imx6-rauc-sdimage.wks"
#WKS_FILES_mx6ul = "imx6-rauc-sdimage.wks"
WKS_FILES_mx8m = "imx8m-rauc-sdimage.wks"

Enable the correct partitioning scheme by removing the comment as shown above. Build the image as usual:

host$ bitbake phytec-headless-image

Flash the .sdcard image onto a running target eMMC from the host:

host$ dd if=phytec-headless-image-phyboard-polis-imx8mm-3.sdcard | ssh root@192.168.3.11 "dd of=/dev/mmcblk2 bs=1MB conv=fsync"

Now the target is able to boot the flashed A/B system.

NAND

With raw NAND flash the kernel, device tree, and root filesystem are written individually. First, initialize the NAND flash with the correct volumes from the bootloader:

bootloader$ rauc_init_nand

bootloader$ edit /env/bin/rauc_init_nand

Now that the volumes exist, flash the kernel, device tree and root filesystem. The following script assumes that you have these files named as zImage, oftree and root.ubifs. It is possible to use either a TFTP server or an attached MMC device as a source. For setting up a TFTP server, see the corresponding BSP manual that fits you SoM.

bootloader$ rauc_flash_nand_from_tftp  # either using TFTP server
bootloader$ rauc_flash_nand_from_mmc   # ... or using external MMC device

The system is now ready to be booted.

Bootloader

U-Boot

After a successful boot, a U-Boot parameter needs to be set. This command is used to view the available parameters:

target$ fw_printenv

You should see this parameter along with others in the output:

doraucboot=0

To enable booting the A/B system with RAUC, set this variable to "1":

target$ fw_setenv doraucboot 1

The parameters can also be edited in U-Boot. Restart your board and hit any key to stop the automatic boot. The environment variables can now be viewed:

bootloader$ printenv

and set:

bootloader$ setenv doraucboot 1
bootloader$ saveenv

Boot into the system:

bootloader$ boot

Now you are able to install RAUC bundles on your machine with the booted A/B system.

Barebox

To automatically boot using the bootchooser framework, that selects the correct partitions in an A/B system, set the default boot target:

bootloader$ nv boot.default=bootchooser

Boot into the system:

bootloader$ boot

Now you are able to install RAUC bundles on your machine with the booted A/B system.

Creating RAUC Bundles

To update your system with RAUC, a RAUC bundle (.raucb) needs to be created. It contains all required images and scripts for the update and a RAUC manifest.raucm that describes the content of the bundle for the RAUC update on the target. The BSP includes a Yocto target that lets you build a RAUC bundle from your Yocto build.

To create the bundle with Yocto, run:

host$ bitbake phytec-headless-bundle

This results in the creation of a .raucb bundle file in deploy/images/<MACHINE>/ which can be used for updating the system as described later. There is no need to create a manifest.raucm manually as it is created automatically during the build of the bundle. As a reference, the created manifest would look something like:

[update]
compatible=phyboard-polis-imx8mm-3
version=r0
description=PHYTEC rauc bundle based on BSP-Yocto-FSL-i.MX8MM-PD20.1.0
build=20200624074335

[image.rootfs]
sha256=cc3f65cd1c1993951d7a39bdb7b7d723617ac46460f8b640cd8d1622ad6e4c17
size=99942000
filename=phytec-headless-image-phyboard-polis-imx8mm-3.tar.gz

[image.boot]
sha256=bafe46679af8c6292dba22b9d402e3119ef78c6f8b458bcb6993326060de3aa4
size=12410534
filename=boot.tar.gz.img

For more information about the manifest format, see https://rauc.readthedocs.io/en/latest/reference.html#manifest.

Updating with RAUC

To update the target system with RAUC, the RAUC bundle file previously created first needs to be copied to the board or to a memory device that can be mounted in Linux. One way is to copy the bundle file with scp, but this requires enough space left on the board's filesystem. To do this, boot the target board to Linux and connect it via Ethernet to your host PC.

On the host, run:

host$ scp phytec-headless-bundle-phyboard-polis-imx8mm-2.raucb root@192.168.3.11:/root/

On the target, the bundle can be verified:

target$ rauc info /root/phytec-headless-bundle-phyboard-polis-imx8mm-2.raucb

and the output should look similar to this:

rauc-Message: 12:52:49.821: Reading bundle: /phytec-headless-bundle-phyboard-polis-imx8mm-2.raucb
rauc-Message: 12:52:49.830: Verifying bundle... 
Compatible:     'phyboard-polis-imx8mm-3'
Version:        'r0'
Description:    'PHYTEC rauc bundle based on BSP-Yocto-FSL-i.MX8MM-PD20.1.0'
Build:          '20200624073212'
Hooks:          ''
2 Images:
(1)     phytec-headless-image-phyboard-polis-imx8mm-2.tar.gz
        Slotclass: rootfs
        Checksum:  342f67f7678d7af3f77710e1b68979f638c7f4d20393f6ffd0c36beff2789070
        Size:      180407809
        Hooks:     
(2)     boot.tar.gz.img
        Slotclass: boot
        Checksum:  8c84465b4715cc142eca2785fea09804bd970755142c9ff57e08c791e2b71f28
        Size:      12411786
        Hooks:     
0 Files

Certificate Chain:
 0 Subject: /O=PHYTEC Messtechnik GmbH/CN=PHYTEC Messtechnik GmbH Development-1
   Issuer: /O=PHYTEC Messtechnik GmbH/CN=PHYTEC Messtechnik GmbH PHYTEC BSP CA Development
   SPKI sha256: E2:47:5F:32:05:37:04:D4:8C:48:8D:A6:74:A8:21:2E:97:41:EE:88:74:B5:F4:65:75:97:76:1D:FF:1D:7B:EE
   Not Before: Jan  1 00:00:00 1970 GMT
   Not After:  Dec 31 23:59:59 9999 GMT
 1 Subject: /O=PHYTEC Messtechnik GmbH/CN=PHYTEC Messtechnik GmbH PHYTEC BSP CA Development
   Issuer: /O=PHYTEC Messtechnik GmbH/CN=PHYTEC Messtechnik GmbH PHYTEC BSP CA Development
   SPKI sha256: AB:5C:DB:C6:0A:ED:A4:48:B9:40:AC:B1:48:06:AA:BA:92:09:83:8C:DC:6F:E1:5F:B6:FB:0C:39:3C:3B:E6:A2
   Not Before: Jan  1 00:00:00 1970 GMT
   Not After:  Dec 31 23:59:59 9999 GMT

To check the current state of the system, run:

target$ rauc status

and get output similar to this:

Compatible:  phyboard-polis-imx8mm-3
Variant:
Booted from: rootfs.0 (system0)
Activated:   rootfs.0 (system0)
slot states:
  rootfs.0: class=rootfs, device=/dev/mmcblk2p2, type=ext4, bootname=system0
      state=booted, description=, parent=(none), mountpoint=/
      boot status=good
  boot.0: class=boot, device=/dev/mmcblk2p1, type=vfat, bootname=(null)
      state=active, description=, parent=rootfs.0, mountpoint=(none)

  rootfs.1: class=rootfs, device=/dev/mmcblk2p4, type=ext4, bootname=system1
      state=inactive, description=, parent=(none), mountpoint=(none)
      boot status=good
  boot.1: class=boot, device=/dev/mmcblk2p3, type=vfat, bootname=(null)
      state=inactive, description=, parent=rootfs.1, mountpoint=(none)

To update the currently inactive system with the downloaded bundle, run:

target$ rauc install /root/phytec-headless-bundle-phyboard-polis-imx8mm-3.raucb

and reboot afterward:

target$ reboot

With the success of the update, RAUC automatically switches the active system to the newly updated system. Now during reboot, RAUC counts the boot attempts of the kernel and if it fails more often than specified in the state framework of the system, RAUC switches back to the old system and marks the new system as bad. If the boot attempt to the kernel is successful, the new system is marked as good and the old system can now be updated with the same instructions. After two successful rauc install and reboot, both systems are updated.

Changing the Active Boot Slot

It is possible to switch the active system manually:

target$ rauc status mark-active other

After a reboot, the target now starts from the other system.

Use Case Examples

Use Case 1: Automatic Update from USB Flash Drive with RAUC

One of the most prominent use cases for RAUC might be an automatic update system from a USB flash drive. This use case is implemented in the BSP as a reference example. We combine only standard Linux mechanisms with RAUC to build the system. The kernel notifies udev when a device gets plugged into the USB port. We use a custom udev rule to trigger a systemd service when this event happens.

KERNEL!="sd[a-z][0-9]", GOTO="media_by_label_auto_mount_end"
 
# Trigger systemd service
ACTION=="add", TAG+="systemd", ENV{SYSTEMD_WANTS}="update-usb@%k.service"
 
# Exit  
LABEL="media_by_label_auto_mount_end"

The service automatically mounts the USB flash drive and notifies the application.

[Unit]
Description=usb media RAUC service
After=multi-user.target
Requires=rauc.service
 
[Service]
Type=oneshot
Environment=DBUS_SESSION_BUS_ADDRESS=unix:path=/run/dbus/system_bus_socket
ExecStartPre=/bin/mkdir -p /media/%I
ExecStartPre=/bin/mount -t auto /dev/%I /media/%I
ExecStart=/usr/bin/update_usb.sh %I
ExecStop=/bin/umount -l /media/%i
ExecStopPost=-/bin/rmdir /media/%I

In our reference implementation, we simply use a bash script for the application logic.

#!/bin/sh
 
MOUNT=/media/$1
 
NUMRAUCM=$(find ${MOUNT}/*.raucb -maxdepth 0 | wc -l)
 
[ "$NUMRAUCM" -eq 0 ] && echo "${MOUNT}*.raucb not found" && exit
[ "$NUMRAUCM" -ne 1 ] && echo "more than one ${MOUNT}/*.raucb" && exit
 
rauc install $MOUNT/*.raucb
if [ "$?" -ne 0 ]; then
	echo "Failed to install RAUC bundle."
else
	echo "Update successful."
fi
exit $?

The update logic can be integrated into an application by using systemd's D-Bus API. RAUC does not need to be called by its command-line interface but can be integrated with D-Bus.

Use Case 2: Security Measurement: Downgrade Barrier

As a second reference example, we will implement a security mechanism: a downgrade barrier. When you detect a security vulnerability on your system, you will fix it and update your system. The systems with the new software will now be secure again. If an attacker gets ahold of the old software update bundle, which still has a valid signature, the attacker might have the possibility to install the old software and still take advantage of the previously fixed security vulnerability. To prevent this from happening, you could revoke the update certificate for every single update and create a new one. This might be difficult to handle, depending on the environment. A simpler solution would be to allow updates only in one direction using a version check.

#!/bin/sh                                       
                                                  
VERSION_FILE=/etc/rauc/downgrade_barrier_version  
MANIFEST_FILE=${RAUC_UPDATE_SOURCE}/manifest.raucm
                                   
[ ! -f ${VERSION_FILE} ] && exit 1 
[ ! -f ${MANIFEST_FILE} ] && exit 2            
                                                                      
VERSION=`cat ${VERSION_FILE} | cut -d 'r' -f 2`                       
BUNDLE_VERSION=`grep "version" -rI ${MANIFEST_FILE} | cut -d 'r' -f 3`
                                     
# check from empty or unset variables
[ -z "${VERSION}" ] && exit 3       
[ -z "${BUNDLE_VERSION}" ] && exit 4                
                                                    
# developer mode, allow all updates if version is r0
#[ ${VERSION} -eq 0 ] && exit 0
                                             
# downgrade barrier                                          
if [ ${VERSION} -gt ${BUNDLE_VERSION} ]; then                
        echo "Downgrade barrier blocked rauc update! CODE5\n"
else          
        exit 0
fi            
exit 5

The script is installed on the target but it is not activated. You need to remove the developer mode line in the script to activate it.

Reference

Boot Logic Implementation

U-Boot Environment Variables

For U-Boot, the bootlogic that selects the correct partitions to boot from is implemented in its environment. As a reference, these are the most important U-Boot variables that are used for the A/B system with RAUC:

These environment variables are defined in include/configs/phycore_<SOC>.h in the u-boot source code.

Barebox Bootchooser Framework

For the barebox the bootlogic that selects the correct partitions to boot from is implemented using the bootchooser and state framework. See the barebox documentation for detailed information about these: Barebox Bootchooser Framework, Barebox State Framework.

First, the state framework configuration needs to be added to the barebox device tree. Check out the Customizing the BSP chapter in the Yocto reference manual. The state framework configuration is already included with our BSP for the supported SoC and can be directly included in the main barebox device tree. E.g. for i.MX6 based module:

#include "imx6qdl-phytec-state.dtsi"

Afterward, rebuild the image and flash the new bootloader.

The following device tree snippet shows an example of the state framework configuration used with the BSP. As can be seen, the EEPROM is used as a backend for the state information.

/ {
    aliases {
        state = &state;
    };
  
    state: imx6qdl_phytec_boot_state {
        magic = <0x883b86a6>;
        compatible = "barebox,state";
        backend-type = "raw";
        backend = <&backend_update_eeprom>;
        backend-stridesize = <54>;
  
        #address-cells = <1>;
        #size-cells = <1>;
        bootstate {
            #address-cells = <1>;
            #size-cells = <1>;
            last_chosen {
                reg = <0x0 0x4>;
                type = "uint32";
            };
            system0 {
                #address-cells = <1>;
                #size-cells = <1>;
                remaining_attempts {
                    reg = <0x4 0x4>;
                    type = "uint32";
                    default = <3>;
                };
                priority {
                    reg = <0x8 0x4>;
                    type = "uint32";
                    default = <21>;
                };
                ok {
                    reg = <0xc 0x4>;
                    type = "uint32";
                    default = <0>;
                };
            };
            system1 {
                #address-cells = <1>;
                #size-cells = <1>;
                remaining_attempts {
                    reg = <0x10 0x4>;
                    type = "uint32";
                    default = <3>;
                };
                priority {
                    reg = <0x14 0x4>;
                    type = "uint32";
                    default = <20>;
                };
                ok {
                    reg = <0x18 0x4>;
                    type = "uint32";
                    default = <0>;
                };
            };
        };
    };
};
  
&eeprom {
    status = "okay";
    partitions {
        compatible = "fixed-partitions";
        #size-cells = <1>;
        #address-cells = <1>;
        backend_update_eeprom: state@0 {
            reg = <0x0 0x100>;
            label = "update-eeprom";
        };
    };
};

To be able to boot from two systems alternately, the bootchooser needs to be aware of the state framework configuration. For each system, a boot script is required. For a system with NAND flash, the boot script of the first system may look like the following:

#!/bin/sh

[ -e /env/config-expansions ] && /env/config-expansions

[ ! -e /dev/nand0.root.ubi ] && ubiattach /dev/nand0.root

global.bootm.image="/dev/nand0.root.ubi.kernel0"
global.bootm.oftree="/dev/nand0.root.ubi.oftree0"
global.linux.bootargs.dyn.root="root=ubi0:root0 ubi.mtd=root rootfstype=ubifs"

The second boot script has the same structure but uses the partitions containing the second system. Machines with eMMC flash use similar boot scripts, albeit the mounting and boot arguments look different.

Run the following commands to create the required bootchooser non-volatile environment variables:

bootloader$ nv bootchooser.state_prefix=state.bootstate
bootloader$ nv bootchooser.system0.boot=system0
bootloader$ nv bootchooser.system1.boot=system1
bootloader$ nv bootchooser.targets="system0 system1"