Skip to content

Improve and document hook execution flow, add path mapping support #126

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 21, 2025
Merged

Improve and document hook execution flow, add path mapping support #126

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
910031a
engine: Add tagged path support
learmj Nov 17, 2025
a5ba486
core: map paths, re-route and extend hooks
learmj Nov 17, 2025
1bbfbef
doc: Refresh execution flow
learmj Nov 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
222 changes: 192 additions & 30 deletions bin/runner
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,42 +2,204 @@

set -u

op="$1"
source "${IGTOP}/lib/common.sh"

add_dir() {
local spec=$1 dir
if dir=$(map_path "$spec") && [[ -d $dir ]]; then
dirs+=("$dir")
fi
}

phase="$1"
shift 1

# Execute scripts from within these dirs in this order

# Script bundles in these dirs are executed in this order
dirs=()
[[ -n ${IGconf_device_assetdir+x} ]] && dirs+=("${IGconf_device_assetdir}/bdebstrap")
[[ -n ${IGconf_image_assetdir+x} ]] && dirs+=("${IGconf_image_assetdir}/bdebstrap")
dirs+=("${SRC_DIR}/bdebstrap")
dirs+=("${IGTOP}/scripts/bdebstrap")

case "$op" in
setup|extract|essential|customize|cleanup)
for dir in "${dirs[@]}" ; do
[[ -d "$dir" ]] || continue
if compgen -G "${dir}/${op}*" >/dev/null; then
for script in "${dir}/${op}"* ; do
s=$(basename "$script")
if [ -x "$script" ] ; then
if [[ $s =~ ^[a-zA-Z0-9-]+$ ]] ; then
echo "runner: $dir [run $s]"
env -C "$dir" bash "$s" "$@"
err=$?
if [ $err -ne 0 ] ; then
>&2 echo "runner: $script error ($err)"
exit $err
fi
else
>&2 echo "runner: $dir [skipped $s]"
fi
else
>&2 echo "runner: $dir [ignored $s]"
fi
done
add_dir 'DEVICE_ASSET:bdebstrap' # device hooks
add_dir 'IMAGE_ASSET:bdebstrap' # image hooks
add_dir 'SRCROOT:bdebstrap' # source tree hooks
add_dir 'IGROOT:scripts/bdebstrap' # built-in hooks


# Execute these hooks at the appropriate phase
declare -A HOOK_PHASES=(
# Filesystem
[setup]="\
IGROOT_device:pre-build.sh IGROOT_image:pre-build.sh \
DEVICE_ASSET:pre-build.sh IMAGE_ASSET:pre-build.sh \
IGROOT:pre-build.sh \
SRCROOT:pre-build.sh"

[post-build]="\
IGROOT_device:post-build.sh IGROOT_image:post-build.sh \
DEVICE_ASSET:post-build.sh IMAGE_ASSET:post-build.sh \
IGROOT:post-build.sh \
SRCROOT:post-build.sh"

# Image
[pre-image]="\
IGROOT_device:pre-image.sh IGROOT_image:pre-image.sh \
DEVICE_ASSET:pre-image.sh IMAGE_ASSET:pre-image.sh \
IGROOT:pre-image.sh \
SRCROOT:pre-image.sh"

[post-image]="\
DEVICE_ASSET:post-image.sh|IGROOT_device:post-image.sh \
IMAGE_ASSET:post-image.sh|IGROOT_image:post-image.sh \
IGROOT:post-image.sh \
SRCROOT:post-image.sh"

[sbom]='VAR:IGconf_sbom_hook'
[deploy]='VAR:IGconf_deploy_hook'
)


# Optional filesystem overlays applied at the appropriate phase
declare -A FS_OVERLAYS=(
[customize]="\
IMAGE_ASSET:device/rootfs-overlay \
DEVICE_ASSET:device/rootfs-overlay \
SRCROOT:rootfs-overlay"
)


# Execution wrapper
runhook() {
local path=$1; shift || true
local dir=$(dirname "$path")
local name=$(basename "$path")

if [[ -x $path ]]; then
msg "runner: $dir [run $name] [args $@]"
env -C "$dir" "./$name" "$@" # honour script shebang
local rc=$?
[[ $rc -eq 0 ]] || die "runner: $dir/$name ($rc)"
else
warn "runner: $dir [ignored, non-executable $name]"
fi
}


# Apply overlays
apply_overlays() {
local phase=$1 dest=$2
[[ -n ${FS_OVERLAYS[$phase]:-} && -d $dest ]] || return 0

local spec src
for spec in ${FS_OVERLAYS[$phase]}; do
if src=$(map_path "$spec") && [[ -d $src ]]; then
msg "runner: overlay [$src -> $dest]"
rsync -a -- "$src"/ "$dest"/
local rc=$?
[[ $rc -eq 0 ]] || die "runner: error applying overlay ($rc)"
fi
done
}


# Execute script bundles in all dirs for a given phase
run_phase_scripts() {
local phase=$1; shift || true
local dir script s err
for dir in "${dirs[@]}"; do
[[ -d $dir ]] || continue
if compgen -G "${dir}/${phase}"* >/dev/null; then
for script in "${dir}/${phase}"*; do
s=$(basename "$script")
if [[ $s =~ ^[a-zA-Z0-9-]+$ ]]; then
runhook "$script" "$@"
else
warn "runner: $dir [skipped $s]"
fi
done
fi
done
}


# Execute hooks for a given phase, support conditional groups
run_hook_phase() {
local phase=$1; shift || true
local specs=${HOOK_PHASES[$phase]:-}
[[ -n $specs ]] || return 0

# Latch phase args if there are any. This still allows positional
# args if runner gets them.
local args=()
while IFS= read -r -d '' arg ; do
args+=("$arg")
done < <(phase_args "$phase")
args+=("$@")

local group spec path
for group in $specs; do
IFS='|' read -r -a _choices <<< "$group"
for spec in "${_choices[@]}"; do
path=$(map_path "$spec")
if path=$(map_path "$spec" 2>/dev/null) && [[ -f $path ]]; then
runhook "$path" "${args[@]}"
break
fi
done
done
}


# per-phase positional arg tweaks
phase_args() {
local phase=$1
shift || true

case $phase in
post-build)
set -- "$@" "${IGconf_target_path:?"required for $phase"}"
;;
pre-image)
set -- "$@" "${IGconf_target_path:?"required for $phase"}"
set -- "$@" "${IGconf_image_outputdir:?"required for $phase"}"
;;
post-image)
set -- "$@" "${IGconf_image_outputdir:?"required for $phase"}"
;;
sbom)
set -- "$@" "${IGconf_target_path:?"required for $phase"}"
set -- "$@" "${IGconf_target_dir:?"required for $phase"}"
;;
deploy)
[[ -n ${IGconf_deploy_dir:-} ]] && set -- "$@" "$IGconf_deploy_dir"
;;
*)
esac
printf '%s\0' "$@"
}


msg "runner: in $phase"
case "$phase" in
setup)
run_hook_phase "$phase" "$@"
run_phase_scripts "$phase" "$@"
;;
customize)
run_phase_scripts "$phase" "$@"
apply_overlays "$phase" "$IGconf_target_path"
run_hook_phase "$phase" "$@"
;;
extract|essential|cleanup)
run_phase_scripts "$phase" "$@"
;;
post-build)
run_hook_phase "$phase" "$@"
;;
pre-image|post-image)
run_hook_phase "$phase" "$@"
;;
sbom|deploy)
run_hook_phase "$phase" "$@"
;;
*)
;;
esac
msg "runner: out $phase"
87 changes: 74 additions & 13 deletions docs/execution/index.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ link:../index.adoc[← Back to Main Documentation]

== Overview

The rpi-image-gen build process consists of five main stages executed sequentially:
The rpi-image-gen build process consists of six main stages executed sequentially:

. **Parameter Assembly** - Input validation and configuration parsing
. **Layer Processing** - Layer collection, validation, and dependency resolution
Expand Down Expand Up @@ -80,7 +80,7 @@ Set up the build environment and assemble bdebstrap parameters with the necessar
** Environment variables
** Target settings
** All compatible YAML layer files
** Installing core hook runners for setup, essential, customize, and cleanup phases
** Installing core hook runners for setup, extract, essential, customize, and cleanup phases

== Stage 4: Filesystem Generation

Expand All @@ -89,11 +89,11 @@ Create the filesystem and generate the Software Bill of Materials.

=== Activities

* *Hook*`[pre-build.sh]`: Execute from within image and device asset directory.
* *Hook*`[pre-build.sh]`
** _Example Use_ - Custom validation of pre-build settings
* *Filesystem Generation*: Execute bdebstrap
* *Overlays*: Apply static filesystem overlays from image and device `rootfs-overlay/` directories.
* *Hook*`[post-build.sh]`: Execute from within image and device asset directory.
* *Overlays*: Apply static filesystem overlays
* *Hook*`[post-build.sh]`
** _Example Use_ - Custom installation of image or device specific assets, eg boot configuration files.
* *SBOM*: Execute the Software Bill of Materials provider to create the SBOM file.

Expand All @@ -105,10 +105,10 @@ Create disk images from the prepared filesystem using the provider.

=== Activities

* *Hook*`[pre-image.sh]`: Execute from within device and image asset directory.
* *Hook*`[pre-image.sh]`
** _Example Use_ - Creating genimage templates, setting up image creation resources.
* *Image Generation*: Execute the image provider to create images.
* *Hook*`[post-image.sh]`: Execute from within device and image asset directory.
* *Hook*`[post-image.sh]`
** _Example Use_ - Custom packaging, signing with device keys, etc.

== Stage 6: Deployment
Expand All @@ -130,21 +130,82 @@ Install and compress:
Hooks are optional but if a hook is to be executed, it must have executable permissions for the user performing the build.
====

=== Core
=== Overview

`rpi-image-gen` supports different classifications of hook, and runs all hooks in _phases_ throughout stages 4, 5 and 6 via script `bin/runner`. Runner is responsible for invoking all hook scripts outside of layer declarations and is the single entry point every hook runs through. `bdebstrap` invokes it for each lifecycle phase (setup, customize, etc.) and the main script calls it for image, SBOM, and deploy phases. Runner:

* Resolves hook locations from tagged specs (`IGROOT:*`, `DEVICE_ASSET:*`, `VAR:IGconf_*`) so the same path resolves regardless of environment (e.g. host vs container).
* Executes phase-specific hooks in a deterministic order.
* Applies optional filesystem overlays during `customize`.
* Supports fallback hook groups (A|B) so that built-in hooks run when no custom is provided.
* Orchestrates deterministic hook execution with the correct environment and arguments, regardless of where the build runs.

=== Phases

[cols="1,1,1,4", options="header"]
|===
| Phase | Context | Classification | Execution
| setup | bdebstrap | bundle | After directory creation, before any packages are downloaded or installed
| pre-build | bdebstrap | single | Within setup phase
| extract | bdebstrap | bundle | After `Essential:yes` packages have been extracted, before installing them
| essential | bdebstrap | bundle | After `Essential:yes` packages have been installed, before installing remaining packages
| customize | bdebstrap | bundle | After all packages are installed, before cleanup commences
| cleanup | bdebstrap | bundle | After customize
| post-build | main | single | After bdebstrap, before sbom
| sbom | main | single | After post-build, before pre-image. Layer driven.
| pre-image | main | single | After sbom, before image generation
| post-image | main | single | After image generation
| deploy | main | single | Final stage, after post-image
|===

=== Hook Classification: bundle

Runner scans for `setup*`, `extract*`, `essential*`, `customize*`, and `cleanup*` hooks inside `bdebstrap/` subdirectories of:

* Device asset directory (`DEVICE_ASSET`)
* Image asset directory (`IMAGE_ASSET`)
* Source tree (`SRCROOT/bdebstrap`)
* Built-in (`IGROOT/scripts/bdebstrap`)

It executes matching hooks (alphanumeric basename) in that order for each phase. Subdirectories aren’t traversed; the file extension is ignored.

=== Hook Classification: single

With the exception of sbom and deploy, all `single` hooks are customisable and optional. Runner scans for `<phase>.sh` inside:

rpi-image-gen executes its own `pre/post build/image` hooks for device and image regardless of the configuration. These hooks perform generic tasks, such as installing the image provisioning map. Depending on requirements, there may not be a need to use custom hooks at these stages.
* Device asset directory (if set)
* Image asset directory (if set)
* Built-in (IGROOT)
* Source tree (SRCROOT)

Execution takes place from the parent directory of the hook.

=== Overlays

Filesystem overlays are applied during `customize`. The following overlay directories are supported:

* `DEVICE_ASSET:device/rootfs-overlay`
* `IMAGE_ASSET:device/rootfs-overlay`
* `SRCROOT:rootfs-overlay`

=== Core

rpi-image-gen ships its own hooks to perform generic tasks. Depending on requirements, there may not be a need to add custom hooks at these stages as the generic hooks execute before SRCROOT hooks.

=== Device and Image Asset Directory
A device or image layer may declare `IGconf_device_assetdir` or `IGconf_image_assetdir` respectively as a location of where to store device or image specific assets. These optional variables are referenced by rpi-image-gen. If the pre/post build and pre/post image hooks mentioned above are present in these locations, they will be executed. The hooks won't execute if the variables are not declared.

=== bdebstrap
If a layer sets `IGconf_device_assetdir` (`DEVICE_ASSET`) or `IGconf_image_assetdir` (`IMAGE_ASSET`), Runner automatically looks for optional hooks inside those directories:

[source,bash]
----
<assetdir>/<phase>.sh
----

rpi-image-gen extends the support of bdebstrap hooks to image, device and source directories. Hooks with filenames beginning with ```setup```, ```essential```, ```customize``` and ```cleanup```, and which only contain alphanumeric characters, are supported and must exist in a sub-directory named```bdebstrap``` within the directory in order for them to be run at the respective stage of chroot creation. Their file extension is ignored. Sub-directories are not traversed.
If the asset directory isn’t defined, its hooks are skipped.

=== initramfs

If ```initramfs-tools(7)``` is installed in the chroot, rpi-image-gen extends the support of initramfs scripts and hooks to image and device directories via their sub-directory ```device/initramfs-tools```. If present, the entire contents of this directory is recursively copied into the chroot. Mode and ownership attributes are preserved. Destination files will not be overwritten. rpi-image-gen performs this operation during the ```customize``` stage of chroot creation and guarantees it will take place after invocation of all image and device bdebstrap ```customize``` hooks.
If `initramfs-tools(7)` is installed in the chroot, rpi-image-gen extends the support of `initramfs-tools` compatible scripts and hooks to image and device directories via their sub-directory `device/initramfs-tools`. If present, the entire contents of this directory is recursively copied into the chroot. Mode and ownership attributes are preserved. Destination files will not be overwritten. This takes place during the `customize` stage of chroot creation and after execution of `DEVICE_ASSET` and `IMAGE_ASSET` hook bundles.

== Interactive Mode

Expand Down
Loading