L-1006e.A1 RAUC Update and Device Management Manual

Table of Contents

L-1006e.A1 RAUC Update & Device Management Manual
Document TitleL-1006e.A1 RAUC Update & Device Management Manual
Article NumberL-1006e.A1
StatusReleased
Release Date12.08.2020
ControllerCompatible BSP
i.MX 8M MiniBSP-Yocto-FSL-i.MX8MM-PD20.1.0
i.MX 6BSP-Yocto-i.MX6-PD20.1.0

Introduction

Since the Yocto warrior release, the RAUC (Robust Auto-Update Controller) mechanism support has been added to Yogurt. It controls the procedure of updating a device with new firmware. This includes updating Linux kernel, Device Tree, and root filesystem. It currently does not update the bootloader. For detailed information about RAUC, visitrauc.readthedocs.io.

This manual describes how RAUC is used and implemented on various PHYTEC platforms. Note, that different modules use different bootloaders and flash storage devices, which affects the way things are handled with RAUC. Make sure to read the correct sections fitting your platform.

Note

Note, that this manual contains machine-specific paths and variable contents. Make sure you are using the correct machine for your application when executing any commands.

System Configuration

RAUC can be used with both eMMC and NAND flash storage. It is not, by default, enabled but our example can be configured and activated with the instructions shown below. RAUC can be used in different update scenarios. As an example, we configured the BSP to use an A/B setup to have a completely redundant system (except for the bootloader).

The partition layout is defined in the /etc/rauc/system.conf file. As an example, this is how it looks like for i.MX 8M Mini with eMMC flash storage:

[system]
compatible=phyboard-polis-imx8mm-3
bootloader=uboot
mountprefix=/mnt/rauc

[handlers]
pre-install=/usr/bin/rauc_downgrade_barrier.sh

[keyring]
path=ca.cert.pem

# System A
[slot.rootfs.0]
device=/dev/mmcblk2p2
type=ext4
bootname=system0

[slot.boot.0]
device=/dev/mmcblk2p1
type=vfat
parent=rootfs.0
                     
# System B           
[slot.rootfs.1]      
device=/dev/mmcblk2p4
type=ext4            
bootname=system1     
                     
[slot.boot.1]        
device=/dev/mmcblk2p3
type=vfat            
parent=rootfs.1

Note, that the devices specified in the slots are different depending on the selected machine.

Updates with RAUC use an openSSL certificate to verify the validity of an image. The BSP includes a certificate that can be used for development. In a productive system, however, it is highly recommended to use a self-created key and certificate.

Design Considerations

In order to prevent the system from locking up, it may be a good idea to utilize a hardware watchdog. In case the Linux Kernel does not boot or another catastrophic event occurs that prevents the system from operating normally, the hardware watchdog then resets the system. By default, the hardware watchdog is disabled. To enable it, refer to the corresponding BSP manual that fits your SoM.

Initial Setup

To use RAUC, the flash device needs to be written and the bootloader must be set up to start the A/B system.

Flash storage

eMMC

To flash the eMMC with the correct partitions a line needs to be added in the local.conf file of the BSP:

# Select a preconfigured A/B system setup for SD/eMMC images.
#WKS_FILES_mx6 = "imx6-rauc-sdimage.wks"
#WKS_FILES_mx6ul = "imx6-rauc-sdimage.wks"
WKS_FILES_mx8m = "imx8m-rauc-sdimage.wks"

Enable the correct partitioning scheme by removing the comment as shown above. Build the image as usual:

host$ bitbake phytec-headless-image

Flash the .sdcard image onto a running target eMMC from the host:

host$ dd if=phytec-headless-image-phyboard-polis-imx8mm-3.sdcard | ssh root@192.168.3.11 "dd of=/dev/mmcblk2 bs=1MB conv=fsync"

Now the target is able to boot the flashed A/B system.

NAND

With raw NAND flash the kernel, device tree, and root filesystem are written individually. First, initialize the NAND flash with the correct volumes from the bootloader:

bootloader$ rauc_init_nand

Note

Note, that the initialization script may assume a different NAND flash size than your board is actually using. Make sure the correct size is being used. For this, execute the following:

bootloader$ edit /env/bin/rauc_init_nand

Edit the variable ROOTFSSIZE. The maximum size of the root filesystem must be half the NAND flash size, minus the size of the kernel and device tree volumes. Save and quit the file using Ctrl+D before executing the initialization script.

Now that the volumes exist, flash the kernel, device tree and root filesystem. The following script assumes that you have these files named as zImage, oftree and root.ubifs. It is possible to use either a TFTP server or an attached MMC device as a source. For setting up a TFTP server, see the corresponding BSP manual that fits you SoM.

bootloader$ rauc_flash_nand_from_tftp  # either using TFTP server
bootloader$ rauc_flash_nand_from_mmc   # ... or using external MMC device

The system is now ready to be booted.

Bootloader

U-Boot

After a successful boot, a U-Boot parameter needs to be set. This command is used to view the available parameters:

target$ fw_printenv

You should see this parameter along with others in the output:

doraucboot=0

To enable booting the A/B system with RAUC, set this variable to "1":

target$ fw_setenv doraucboot 1

The parameters can also be edited in U-Boot. Restart your board and hit any key to stop the automatic boot. The environment variables can now be viewed:

bootloader$ printenv

and set:

bootloader$ setenv doraucboot 1
bootloader$ saveenv

Boot into the system:

bootloader$ boot

Now you are able to install RAUC bundles on your machine with the booted A/B system.

Barebox

To automatically boot using the bootchooser framework, that selects the correct partitions in an A/B system, set the default boot target:

bootloader$ nv boot.default=bootchooser

Boot into the system:

bootloader$ boot

Now you are able to install RAUC bundles on your machine with the booted A/B system.

Creating RAUC Bundles

To update your system with RAUC, a RAUC bundle (.raucb) needs to be created. It contains all required images and scripts for the update and a RAUC manifest.raucm that describes the content of the bundle for the RAUC update on the target. The BSP includes a Yocto target that lets you build a RAUC bundle from your Yocto build.

To create the bundle with Yocto, run:

host$ bitbake phytec-headless-bundle

This results in the creation of a .raucb bundle file in deploy/images/<MACHINE>/ which can be used for updating the system as described later. There is no need to create a manifest.raucm manually as it is created automatically during the build of the bundle. As a reference, the created manifest would look something like:

[update]
compatible=phyboard-polis-imx8mm-3
version=r0
description=PHYTEC rauc bundle based on BSP-Yocto-FSL-i.MX8MM-PD20.1.0
build=20200624074335

[image.rootfs]
sha256=cc3f65cd1c1993951d7a39bdb7b7d723617ac46460f8b640cd8d1622ad6e4c17
size=99942000
filename=phytec-headless-image-phyboard-polis-imx8mm-3.tar.gz

[image.boot]
sha256=bafe46679af8c6292dba22b9d402e3119ef78c6f8b458bcb6993326060de3aa4
size=12410534
filename=boot.tar.gz.img

For more information about the manifest format, see https://rauc.readthedocs.io/en/latest/reference.html#manifest.

Updating with RAUC

To update the target system with RAUC, the RAUC bundle file previously created first needs to be copied to the board or to a memory device that can be mounted in Linux. One way is to copy the bundle file with scp, but this requires enough space left on the board's filesystem. To do this, boot the target board to Linux and connect it via Ethernet to your host PC.

On the host, run:

host$ scp phytec-headless-bundle-phyboard-polis-imx8mm-2.raucb root@192.168.3.11:/root/

On the target, the bundle can be verified:

target$ rauc info /root/phytec-headless-bundle-phyboard-polis-imx8mm-2.raucb

and the output should look similar to this:

rauc-Message: 12:52:49.821: Reading bundle: /phytec-headless-bundle-phyboard-polis-imx8mm-2.raucb
rauc-Message: 12:52:49.830: Verifying bundle... 
Compatible:     'phyboard-polis-imx8mm-3'
Version:        'r0'
Description:    'PHYTEC rauc bundle based on BSP-Yocto-FSL-i.MX8MM-PD20.1.0'
Build:          '20200624073212'
Hooks:          ''
2 Images:
(1)     phytec-headless-image-phyboard-polis-imx8mm-2.tar.gz
        Slotclass: rootfs
        Checksum:  342f67f7678d7af3f77710e1b68979f638c7f4d20393f6ffd0c36beff2789070
        Size:      180407809
        Hooks:     
(2)     boot.tar.gz.img
        Slotclass: boot
        Checksum:  8c84465b4715cc142eca2785fea09804bd970755142c9ff57e08c791e2b71f28
        Size:      12411786
        Hooks:     
0 Files

Certificate Chain:
 0 Subject: /O=PHYTEC Messtechnik GmbH/CN=PHYTEC Messtechnik GmbH Development-1
   Issuer: /O=PHYTEC Messtechnik GmbH/CN=PHYTEC Messtechnik GmbH PHYTEC BSP CA Development
   SPKI sha256: E2:47:5F:32:05:37:04:D4:8C:48:8D:A6:74:A8:21:2E:97:41:EE:88:74:B5:F4:65:75:97:76:1D:FF:1D:7B:EE
   Not Before: Jan  1 00:00:00 1970 GMT
   Not After:  Dec 31 23:59:59 9999 GMT
 1 Subject: /O=PHYTEC Messtechnik GmbH/CN=PHYTEC Messtechnik GmbH PHYTEC BSP CA Development
   Issuer: /O=PHYTEC Messtechnik GmbH/CN=PHYTEC Messtechnik GmbH PHYTEC BSP CA Development
   SPKI sha256: AB:5C:DB:C6:0A:ED:A4:48:B9:40:AC:B1:48:06:AA:BA:92:09:83:8C:DC:6F:E1:5F:B6:FB:0C:39:3C:3B:E6:A2
   Not Before: Jan  1 00:00:00 1970 GMT
   Not After:  Dec 31 23:59:59 9999 GMT

To check the current state of the system, run:

target$ rauc status

and get output similar to this:

Compatible:  phyboard-polis-imx8mm-3
Variant:
Booted from: rootfs.0 (system0)
Activated:   rootfs.0 (system0)
slot states:
  rootfs.0: class=rootfs, device=/dev/mmcblk2p2, type=ext4, bootname=system0
      state=booted, description=, parent=(none), mountpoint=/
      boot status=good
  boot.0: class=boot, device=/dev/mmcblk2p1, type=vfat, bootname=(null)
      state=active, description=, parent=rootfs.0, mountpoint=(none)

  rootfs.1: class=rootfs, device=/dev/mmcblk2p4, type=ext4, bootname=system1
      state=inactive, description=, parent=(none), mountpoint=(none)
      boot status=good
  boot.1: class=boot, device=/dev/mmcblk2p3, type=vfat, bootname=(null)
      state=inactive, description=, parent=rootfs.1, mountpoint=(none)

To update the currently inactive system with the downloaded bundle, run:

target$ rauc install /root/phytec-headless-bundle-phyboard-polis-imx8mm-3.raucb

and reboot afterward:

target$ reboot

With the success of the update, RAUC automatically switches the active system to the newly updated system. Now during reboot, RAUC counts the boot attempts of the kernel and if it fails more often than specified in the state framework of the system, RAUC switches back to the old system and marks the new system as bad. If the boot attempt to the kernel is successful, the new system is marked as good and the old system can now be updated with the same instructions. After two successful rauc install and reboot, both systems are updated.

Tip

When you update from a USB stick, make sure to remove the stick after a successful update before reboot. If not, an automatic update will be started after each boot. This is due to the "Automatic Update from USB Flash Drive example" you can find below.

Changing the Active Boot Slot

It is possible to switch the active system manually:

target$ rauc status mark-active other

After a reboot, the target now starts from the other system.

Use Case Examples

Use Case 1: Automatic Update from USB Flash Drive with RAUC

One of the most prominent use cases for RAUC might be an automatic update system from a USB flash drive. This use case is implemented in the BSP as a reference example. We combine only standard Linux mechanisms with RAUC to build the system. The kernel notifies udev when a device gets plugged into the USB port. We use a custom udev rule to trigger a systemd service when this event happens.

10-update-usb.rules
KERNEL!="sd[a-z][0-9]", GOTO="media_by_label_auto_mount_end"
 
# Trigger systemd service
ACTION=="add", TAG+="systemd", ENV{SYSTEMD_WANTS}="update-usb@%k.service"
 
# Exit  
LABEL="media_by_label_auto_mount_end"

The service automatically mounts the USB flash drive and notifies the application.

update-usb@.service
[Unit]
Description=usb media RAUC service
After=multi-user.target
Requires=rauc.service
 
[Service]
Type=oneshot
Environment=DBUS_SESSION_BUS_ADDRESS=unix:path=/run/dbus/system_bus_socket
ExecStartPre=/bin/mkdir -p /media/%I
ExecStartPre=/bin/mount -t auto /dev/%I /media/%I
ExecStart=/usr/bin/update_usb.sh %I
ExecStop=/bin/umount -l /media/%i
ExecStopPost=-/bin/rmdir /media/%I

In our reference implementation, we simply use a bash script for the application logic.

update_usb.sh
#!/bin/sh
 
MOUNT=/media/$1
 
NUMRAUCM=$(find ${MOUNT}/*.raucb -maxdepth 0 | wc -l)
 
[ "$NUMRAUCM" -eq 0 ] && echo "${MOUNT}*.raucb not found" && exit
[ "$NUMRAUCM" -ne 1 ] && echo "more than one ${MOUNT}/*.raucb" && exit
 
rauc install $MOUNT/*.raucb
if [ "$?" -ne 0 ]; then
	echo "Failed to install RAUC bundle."
else
	echo "Update successful."
fi
exit $?

The update logic can be integrated into an application by using systemd's D-Bus API. RAUC does not need to be called by its command-line interface but can be integrated with D-Bus.

Tip

Use Case 2: Security Measurement: Downgrade Barrier

As a second reference example, we will implement a security mechanism: a downgrade barrier. When you detect a security vulnerability on your system, you will fix it and update your system. The systems with the new software will now be secure again. If an attacker gets ahold of the old software update bundle, which still has a valid signature, the attacker might have the possibility to install the old software and still take advantage of the previously fixed security vulnerability. To prevent this from happening, you could revoke the update certificate for every single update and create a new one. This might be difficult to handle, depending on the environment. A simpler solution would be to allow updates only in one direction using a version check.

rauc_downgrade_barrier.sh
#!/bin/sh                                       
                                                  
VERSION_FILE=/etc/rauc/downgrade_barrier_version  
MANIFEST_FILE=${RAUC_UPDATE_SOURCE}/manifest.raucm
                                   
[ ! -f ${VERSION_FILE} ] && exit 1 
[ ! -f ${MANIFEST_FILE} ] && exit 2            
                                                                      
VERSION=`cat ${VERSION_FILE} | cut -d 'r' -f 2`                       
BUNDLE_VERSION=`grep "version" -rI ${MANIFEST_FILE} | cut -d 'r' -f 3`
                                     
# check from empty or unset variables
[ -z "${VERSION}" ] && exit 3       
[ -z "${BUNDLE_VERSION}" ] && exit 4                
                                                    
# developer mode, allow all updates if version is r0
#[ ${VERSION} -eq 0 ] && exit 0
                                             
# downgrade barrier                                          
if [ ${VERSION} -gt ${BUNDLE_VERSION} ]; then                
        echo "Downgrade barrier blocked rauc update! CODE5\n"
else          
        exit 0
fi            
exit 5

The script is installed on the target but it is not activated. You need to remove the developer mode line in the script to activate it.

Reference

Boot Logic Implementation

Tip

The implementation details described in this chapter serve as a reference guide. PHYTEC BSPs that have RAUC support include these by default and the changes are already incorporated.

U-Boot Environment Variables

For U-Boot, the bootlogic that selects the correct partitions to boot from is implemented in its environment. As a reference, these are the most important U-Boot variables that are used for the A/B system with RAUC:

NameFunction
BOOT_ORDERContains a space-separated list of boot-targets in the order they should be tried. This parameter is automatically set by RAUC.
BOOT_<slot>_LEFT

Contains the number of remaining boot attempts to perform for the respective slot. This parameter is automatically set by RAUC.

raucbootContains the boot logic that sets the partitions so the correct system is loaded.
doraucbootEnables booting the A/B system if set to 1 and disables it if set to 0.
raucslotContains the current boot slot used in BOOT_<slot>_LEFT.
raucargsSets the Kernel bootargs like console, root, and RAUC slot.
raucdevSets the eMMC as the boot device.
raucrootpartSets the root filesystem partitions of the device.
raucpartSets the boot partitions of the device.
loadraucimageLoads the Kernel image into RAM.
loadraucfdtLoads the device tree into RAM.

These environment variables are defined in include/configs/phycore_<SOC>.h in the u-boot source code.

Note

A change in the partition layout, e.g. when using an additional data partition, may require changing the variables raucrootpart and raucpart. Make sure to rebuild your image with the new bootloader environment after you have made the appropriate changes.

Barebox Bootchooser Framework

For the barebox the bootlogic that selects the correct partitions to boot from is implemented using the bootchooser and state framework. See the barebox documentation for detailed information about these: Barebox Bootchooser Framework, Barebox State Framework.

First, the state framework configuration needs to be added to the barebox device tree. Check out the Customizing the BSP chapter in the Yocto reference manual. The state framework configuration is already included with our BSP for the supported SoC and can be directly included in the main barebox device tree. E.g. for i.MX6 based module:

#include "imx6qdl-phytec-state.dtsi"

Afterward, rebuild the image and flash the new bootloader.

Warning

Be aware that by adding the state framework configuration, the first 160 bytes of the EEPROM are occupied and can no longer be used for user-specific purposes.

The following device tree snippet shows an example of the state framework configuration used with the BSP. As can be seen, the EEPROM is used as a backend for the state information.

/ {
    aliases {
        state = &state;
    };
  
    state: imx6qdl_phytec_boot_state {
        magic = <0x883b86a6>;
        compatible = "barebox,state";
        backend-type = "raw";
        backend = <&backend_update_eeprom>;
        backend-stridesize = <54>;
  
        #address-cells = <1>;
        #size-cells = <1>;
        bootstate {
            #address-cells = <1>;
            #size-cells = <1>;
            last_chosen {
                reg = <0x0 0x4>;
                type = "uint32";
            };
            system0 {
                #address-cells = <1>;
                #size-cells = <1>;
                remaining_attempts {
                    reg = <0x4 0x4>;
                    type = "uint32";
                    default = <3>;
                };
                priority {
                    reg = <0x8 0x4>;
                    type = "uint32";
                    default = <21>;
                };
                ok {
                    reg = <0xc 0x4>;
                    type = "uint32";
                    default = <0>;
                };
            };
            system1 {
                #address-cells = <1>;
                #size-cells = <1>;
                remaining_attempts {
                    reg = <0x10 0x4>;
                    type = "uint32";
                    default = <3>;
                };
                priority {
                    reg = <0x14 0x4>;
                    type = "uint32";
                    default = <20>;
                };
                ok {
                    reg = <0x18 0x4>;
                    type = "uint32";
                    default = <0>;
                };
            };
        };
    };
};
  
&eeprom {
    status = "okay";
    partitions {
        compatible = "fixed-partitions";
        #size-cells = <1>;
        #address-cells = <1>;
        backend_update_eeprom: state@0 {
            reg = <0x0 0x100>;
            label = "update-eeprom";
        };
    };
};

To be able to boot from two systems alternately, the bootchooser needs to be aware of the state framework configuration. For each system, a boot script is required. For a system with NAND flash, the boot script of the first system may look like the following:

/env/boot/system0
#!/bin/sh

[ -e /env/config-expansions ] && /env/config-expansions

[ ! -e /dev/nand0.root.ubi ] && ubiattach /dev/nand0.root

global.bootm.image="/dev/nand0.root.ubi.kernel0"
global.bootm.oftree="/dev/nand0.root.ubi.oftree0"
global.linux.bootargs.dyn.root="root=ubi0:root0 ubi.mtd=root rootfstype=ubifs"

The second boot script has the same structure but uses the partitions containing the second system. Machines with eMMC flash use similar boot scripts, albeit the mounting and boot arguments look different.

Run the following commands to create the required bootchooser non-volatile environment variables:

bootloader$ nv bootchooser.state_prefix=state.bootstate
bootloader$ nv bootchooser.system0.boot=system0
bootloader$ nv bootchooser.system1.boot=system1
bootloader$ nv bootchooser.targets="system0 system1"