Yocto: BitBake and Dependencies - e.g. One recipe to use output of another recipe
It feels weird to help some get their working build to fail and then for them to be happy about it... (RP)
Introduction
This work is sponsored by Reliable Embedded Systems. You can find more information about our training/consulting services here.
Objectives
The
goal of this blog post is to look into BitBake dependencies. As an example let's take recipe A (u-boot-phytex-imx
) which needs the output of recipe B (firmware-imx-8m
) for a successful compilation. (Please don't ask why.)
To be more precise here is an excerpt of recipe A (u-boot-phytex-imx
):
# --> this funny u-boot from Phytec needs the ddr_firmware binaries in ${S} # Note: here we copy deployed stuff from another recipe # to the source dir of this recipe! # # DEPENDS deals with build-time depenencies # Read my blog post to find out why DEPENDS does not work here # # DEPENDS += " \ # firmware-imx-8m \ # " # RDEPENDS is wrong anyhow for what I want it, # since it deals with runtime dependencies # # RDEPENDS_${PN} += "firmware-imx-8m" # # this works: # do_compile[depends] = "firmware-imx-8m:do_deploy" do_compile_prepend() { bbnote 8MQ/8MM/8MN firmware for ddr_firmware in ${DDR_FIRMWARE_NAME}; do bbnote "Copy ddr_firmware: ${ddr_firmware} from ${DEPLOY_DIR_IMAGE} -> ${S} " cp ${DEPLOY_DIR_IMAGE}/${ddr_firmware} ${S} done } # <-- this funny u-boot from Phytec needs the ddr_firmware binaries in ${S}
Dependencies
RDEPENDS
You could try something like that:
RDEPENDS_${PN} += "firmware-imx-8m"
but this is most definitely wrong, since RDEPENDS
deal with runtime dependencies. Like this the u-boot-phytex-imx
recipe would tell the package manager to install the firmware-imx-8m
package as well which is not what we want here and also it does not make much sense.
That's what the BitBake Manual says:
- RDEPENDS
Lists a package's runtime dependencies (i.e. other packages) that must be installed in order for the built package to run correctly. If a package in this list cannot be found during the build, you will get a build error.
Because the
RDEPENDS
variable applies to packages being built, you should always use the variable in a form with an attached package name. For example, suppose you are building a development package that depends on theperl
package. In this case, you would use the followingRDEPENDS
statement:RDEPENDS_${PN}-dev += "perl"
In the example, the development package depends on the
perl
package. Thus, theRDEPENDS
variable has the${PN}-dev
package name as part of the variable.BitBake supports specifying versioned dependencies. Although the syntax varies depending on the packaging format, BitBake hides these differences from you. Here is the general syntax to specify versions with the
RDEPENDS
variable:RDEPENDS_${PN} = "
package
(operator
version
)"For
operator
, you can specify the following:= < > <= >=
For example, the following sets up a dependency on version 1.2 or greater of the package
foo
:RDEPENDS_${PN} = "foo (>= 1.2)"
For information on build-time dependencies, see the
DEPENDS
variable.
DEPENDS
You could try something like that:
DEPENDS += " \
firmware-imx-8m \
"
That's most likely what most people would try and they would be surprised if it doesn't work.
That's what the BitBake Manual says:
- DEPENDS
Lists a recipe's build-time dependencies (i.e. other recipe files).
Consider this simple example for two recipes named "a" and "b" that produce similarly named packages. In this example, the
DEPENDS
statement appears in the "a" recipe:DEPENDS = "b"
Here, the dependency is such that the
do_configure
task for recipe "a" depends on thedo_populate_sysroot
task of recipe "b". This means anything that recipe "b" puts into sysroot is available when recipe "a" is configuring itself.For information on runtime dependencies, see the
RDEPENDS
variable.
Let's give it a try:
Despite the fact that my friend and our beloved chief system architect RP does not quite understand why it would make me happy to break my build he successfully helped me to break it, so I can show these things here:
pokyuser@862ca11939a7:/workdir/build$ bitbake firmware-imx-8m -c clean; bitbake u-boot-phytec-imx -c clean; bitbake u-boot-phytec-imx -c compile NOTE: Started PRServer with DBfile: /workdir/build/phyboard-polis-imx8mm-wic/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 40609, PID: 13422 Loading cache: 100% |#########################################################################################################################################| Time: 0:00:00 Loaded 3385 entries from dependency cache. Parsing recipes: 100% |#######################################################################################################################################| Time: 0:00:00 Parsing of 2275 .bb files complete (2274 cached, 1 parsed). 3386 targets, 210 skipped, 2 masked, 0 errors. NOTE: Resolving any missing task queue dependencies Build Configuration: BB_VERSION = "1.46.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "universal" TARGET_SYS = "aarch64-resy-linux" MACHINE = "phyboard-polis-imx8mm" DISTRO = "resy" DISTRO_VERSION = "3.1.2" TUNE_FEATURES = "aarch64 armv8a crc crypto" TARGET_FPU = "" meta meta-poky meta-yocto-bsp = "2020-08-28-dunfell-3.1.2+:d3d80fa6fbf15189f6183a33c95fa90053535ae2" meta-resy = "dunfell:2968f06c1e400131e15bd726808a43195eec8c16" meta-freescale = "2020-08-28-dunfell:3fc701d11b4b096ba016110c83fe1b48c5f3a256" meta-phyboard-polis-imx8mm-bsp = "v5.8.x-upstream:e1aaa104ab7879cd1b8f9272aafd06c24691ccb5" meta-oe meta-python meta-networking = "2020-04-30-dunfell-3.1:17fd382f3467e20308924499b7531ae8b789f056" Initialising tasks: 100% |####################################################################################################################################| Time: 0:00:00 Sstate summary: Wanted 0 Found 0 Missed 0 Current 0 (0% match, 0% complete) NOTE: No setscene tasks NOTE: Executing Tasks NOTE: Tasks Summary: Attempted 1 tasks of which 0 didn't need to be rerun and all succeeded. NOTE: Writing buildhistory NOTE: Writing buildhistory took: 0 seconds NOTE: Started PRServer with DBfile: /workdir/build/phyboard-polis-imx8mm-wic/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 43644, PID: 13587 Loading cache: 100% |#########################################################################################################################################| Time: 0:00:00 Loaded 3385 entries from dependency cache. Parsing recipes: 100% |#######################################################################################################################################| Time: 0:00:00 Parsing of 2275 .bb files complete (2274 cached, 1 parsed). 3386 targets, 210 skipped, 2 masked, 0 errors. NOTE: Resolving any missing task queue dependencies Build Configuration: BB_VERSION = "1.46.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "universal" TARGET_SYS = "aarch64-resy-linux" MACHINE = "phyboard-polis-imx8mm" DISTRO = "resy" DISTRO_VERSION = "3.1.2" TUNE_FEATURES = "aarch64 armv8a crc crypto" TARGET_FPU = "" meta meta-poky meta-yocto-bsp = "2020-08-28-dunfell-3.1.2+:d3d80fa6fbf15189f6183a33c95fa90053535ae2" meta-resy = "dunfell:2968f06c1e400131e15bd726808a43195eec8c16" meta-freescale = "2020-08-28-dunfell:3fc701d11b4b096ba016110c83fe1b48c5f3a256" meta-phyboard-polis-imx8mm-bsp = "v5.8.x-upstream:e1aaa104ab7879cd1b8f9272aafd06c24691ccb5" meta-oe meta-python meta-networking = "2020-04-30-dunfell-3.1:17fd382f3467e20308924499b7531ae8b789f056" Initialising tasks: 100% |####################################################################################################################################| Time: 0:00:00 Sstate summary: Wanted 0 Found 0 Missed 0 Current 0 (0% match, 0% complete) NOTE: No setscene tasks NOTE: Executing Tasks NOTE: Tasks Summary: Attempted 1 tasks of which 0 didn't need to be rerun and all succeeded. NOTE: Writing buildhistory NOTE: Writing buildhistory took: 0 seconds NOTE: Started PRServer with DBfile: /workdir/build/phyboard-polis-imx8mm-wic/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 33230, PID: 13752 Loading cache: 100% |#########################################################################################################################################| Time: 0:00:00 Loaded 3385 entries from dependency cache. Parsing recipes: 100% |#######################################################################################################################################| Time: 0:00:00 Parsing of 2275 .bb files complete (2274 cached, 1 parsed). 3386 targets, 210 skipped, 2 masked, 0 errors. NOTE: Resolving any missing task queue dependencies Build Configuration: BB_VERSION = "1.46.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "universal" TARGET_SYS = "aarch64-resy-linux" MACHINE = "phyboard-polis-imx8mm" DISTRO = "resy" DISTRO_VERSION = "3.1.2" TUNE_FEATURES = "aarch64 armv8a crc crypto" TARGET_FPU = "" meta meta-poky meta-yocto-bsp = "2020-08-28-dunfell-3.1.2+:d3d80fa6fbf15189f6183a33c95fa90053535ae2" meta-resy = "dunfell:2968f06c1e400131e15bd726808a43195eec8c16" meta-freescale = "2020-08-28-dunfell:3fc701d11b4b096ba016110c83fe1b48c5f3a256" meta-phyboard-polis-imx8mm-bsp = "v5.8.x-upstream:e1aaa104ab7879cd1b8f9272aafd06c24691ccb5" meta-oe meta-python meta-networking = "2020-04-30-dunfell-3.1:17fd382f3467e20308924499b7531ae8b789f056" Initialising tasks: 100% |####################################################################################################################################| Time: 0:00:00 Sstate summary: Wanted 3 Found 3 Missed 0 Current 100 (100% match, 100% complete) NOTE: Executing Tasks ERROR: u-boot-phytec-imx-v2019.04-1.1.0-phy5+gitAUTOINC+6567aa7d61-r0 do_compile: Execution of '/workdir/build/phyboard-polis-imx8mm-wic/tmp/work/phyboard_polis_imx8mm-resy-linux/u-boot-phytec-imx/v2019.04-1.1.0-phy5+gitAUTOINC+6567aa7d61-r0/temp/run.do_compile.14116' failed with exit code 1: cp: cannot stat '/workdir/build/phyboard-polis-imx8mm-wic/tmp/deploy/images/phyboard-polis-imx8mm/lpddr4_pmu_train_1d_imem.bin': No such file or directory WARNING: /workdir/build/phyboard-polis-imx8mm-wic/tmp/work/phyboard_polis_imx8mm-resy-linux/u-boot-phytec-imx/v2019.04-1.1.0-phy5+gitAUTOINC+6567aa7d61-r0/temp/run.do_compile.14116:1 exit 1 from 'cp /workdir/build/phyboard-polis-imx8mm-wic/tmp/deploy/images/phyboard-polis-imx8mm/${ddr_firmware} /workdir/build/phyboard-polis-imx8mm-wic/tmp/work/phyboard_polis_imx8mm-resy-linux/u-boot-phytec-imx/v2019.04-1.1.0-phy5+gitAUTOINC+6567aa7d61-r0/git' ERROR: Logfile of failure stored in: /workdir/build/phyboard-polis-imx8mm-wic/tmp/work/phyboard_polis_imx8mm-resy-linux/u-boot-phytec-imx/v2019.04-1.1.0-phy5+gitAUTOINC+6567aa7d61-r0/temp/log.do_compile.14116 Log data follows: | DEBUG: Executing shell function do_compile | NOTE: 8MQ/8MM/8MN firmware | NOTE: Copy ddr_firmware: lpddr4_pmu_train_1d_imem.bin from /workdir/build/phyboard-polis-imx8mm-wic/tmp/deploy/images/phyboard-polis-imx8mm -> /workdir/build/phyboard-polis-imx8mm-wic/tmp/work/phyboard_polis_imx8mm-resy-linux/u-boot-phytec-imx/v2019.04-1.1.0-phy5+gitAUTOINC+6567aa7d61-r0/git | cp: cannot stat '/workdir/build/phyboard-polis-imx8mm-wic/tmp/deploy/images/phyboard-polis-imx8mm/lpddr4_pmu_train_1d_imem.bin': No such file or directory | WARNING: /workdir/build/phyboard-polis-imx8mm-wic/tmp/work/phyboard_polis_imx8mm-resy-linux/u-boot-phytec-imx/v2019.04-1.1.0-phy5+gitAUTOINC+6567aa7d61-r0/temp/run.do_compile.14116:1 exit 1 from 'cp /workdir/build/phyboard-polis-imx8mm-wic/tmp/deploy/images/phyboard-polis-imx8mm/${ddr_firmware} /workdir/build/phyboard-polis-imx8mm-wic/tmp/work/phyboard_polis_imx8mm-resy-linux/u-boot-phytec-imx/v2019.04-1.1.0-phy5+gitAUTOINC+6567aa7d61-r0/git' | ERROR: Execution of '/workdir/build/phyboard-polis-imx8mm-wic/tmp/work/phyboard_polis_imx8mm-resy-linux/u-boot-phytec-imx/v2019.04-1.1.0-phy5+gitAUTOINC+6567aa7d61-r0/temp/run.do_compile.14116' failed with exit code 1: | cp: cannot stat '/workdir/build/phyboard-polis-imx8mm-wic/tmp/deploy/images/phyboard-polis-imx8mm/lpddr4_pmu_train_1d_imem.bin': No such file or directory | WARNING: /workdir/build/phyboard-polis-imx8mm-wic/tmp/work/phyboard_polis_imx8mm-resy-linux/u-boot-phytec-imx/v2019.04-1.1.0-phy5+gitAUTOINC+6567aa7d61-r0/temp/run.do_compile.14116:1 exit 1 from 'cp /workdir/build/phyboard-polis-imx8mm-wic/tmp/deploy/images/phyboard-polis-imx8mm/${ddr_firmware} /workdir/build/phyboard-polis-imx8mm-wic/tmp/work/phyboard_polis_imx8mm-resy-linux/u-boot-phytec-imx/v2019.04-1.1.0-phy5+gitAUTOINC+6567aa7d61-r0/git' | ERROR: Task (/workdir/sources/poky/../meta-phyboard-polis-imx8mm-bsp/recipes-bsp/u-boot/u-boot-phytec-imx_2019.04.bb:do_compile) failed with exit code '1' NOTE: Tasks Summary: Attempted 459 tasks of which 453 didn't need to be rerun and 1 failed. NOTE: The errors for this build are stored in /workdir/build/phyboard-polis-imx8mm-wic/tmp/log/error-report/error_report_20200907151617.txt You can send the errors to a reports server by running: send-error-report /workdir/build/phyboard-polis-imx8mm-wic/tmp/log/error-report/error_report_20200907151617.txt [-s server] NOTE: The contents of these logs will be posted in public if you use the above command with the default server. Please ensure you remove any identifying or proprietary information when prompted before sending. NOTE: Writing buildhistory NOTE: Writing buildhistory took: 0 seconds Summary: 1 task failed: /workdir/sources/poky/../meta-phyboard-polis-imx8mm-bsp/recipes-bsp/u-boot/u-boot-phytec-imx_2019.04.bb:do_compile Summary: There was 1 ERROR message shown, returning a non-zero exit code. pokyuser@862ca11939a7:/workdir/build$ bitbake firmware-imx-8m -e | grep ^T= T="/workdir/build/phyboard-polis-imx8mm-wic/tmp/work/aarch64-mx8mm-resy-linux/firmware-imx-8m/8.5-r0/temp" pokyuser@862ca11939a7:/workdir/build$ cat /workdir/build/phyboard-polis-imx8mm-wic/tmp/work/aarch64-mx8mm-resy-linux/firmware-imx-8m/8.5-r0/temp/log. log.do_deploy_source_date_epoch_setscene log.do_populate_sysroot_setscene log.task_order log.do_deploy_source_date_epoch_setscene.13814 log.do_populate_sysroot_setscene.13789
This build breaks because it cannot copy the ddr firmware from DEPLOY_DIR_IMAGE
to wherever u-boot
needs it.
Why? Because, as we can see at the end of the previous shell session, do_populate_sysroot_setscene
and do_deploy_source_date_epoch_setscene
were executed, but nothing like do_deploy_setscene
, which is what we need.
We can see that in the log files as well as the doc indicates something along those lines.
[depends] Inter-Task Dependencies
You could try something like that:
do_compile[depends] = "firmware-imx-8m:do_deploy"
That's what the BitBake Manual says:
Inter-Task Dependencies
BitBake uses the [depends]
flag in a more generic form
to manage inter-task dependencies.
This more generic form allows for inter-dependency
checks for specific tasks rather than checks for
the data in DEPENDS
.
Here is an example:
do_patch[depends] = "quilt-native:do_populate_sysroot"
In this example, the do_populate_sysroot
task of the target quilt-native
must have completed before the
do_patch
task can execute.
The [rdepends]
flag works in a similar
way but takes targets
in the runtime namespace instead of the build-time dependency
namespace.
Let's give it a try:
pokyuser@862ca11939a7:/workdir/build$ bitbake firmware-imx-8m -c clean; bitbake u-boot-phytec-imx -c clean; bitbake u-boot-phytec-imx -c compile NOTE: Started PRServer with DBfile: /workdir/build/phyboard-polis-imx8mm-wic/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 40834, PID: 23027 Loading cache: 100% |#########################################################################################################################################| Time: 0:00:00 Loaded 3385 entries from dependency cache. Parsing recipes: 100% |#######################################################################################################################################| Time: 0:00:00 Parsing of 2275 .bb files complete (2274 cached, 1 parsed). 3386 targets, 210 skipped, 2 masked, 0 errors. NOTE: Resolving any missing task queue dependencies Build Configuration: BB_VERSION = "1.46.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "universal" TARGET_SYS = "aarch64-resy-linux" MACHINE = "phyboard-polis-imx8mm" DISTRO = "resy" DISTRO_VERSION = "3.1.2" TUNE_FEATURES = "aarch64 armv8a crc crypto" TARGET_FPU = "" meta meta-poky meta-yocto-bsp = "2020-08-28-dunfell-3.1.2+:d3d80fa6fbf15189f6183a33c95fa90053535ae2" meta-resy = "dunfell:2968f06c1e400131e15bd726808a43195eec8c16" meta-freescale = "2020-08-28-dunfell:3fc701d11b4b096ba016110c83fe1b48c5f3a256" meta-phyboard-polis-imx8mm-bsp = "v5.8.x-upstream:e1aaa104ab7879cd1b8f9272aafd06c24691ccb5" meta-oe meta-python meta-networking = "2020-04-30-dunfell-3.1:17fd382f3467e20308924499b7531ae8b789f056" Initialising tasks: 100% |####################################################################################################################################| Time: 0:00:00 Sstate summary: Wanted 0 Found 0 Missed 0 Current 0 (0% match, 0% complete) NOTE: No setscene tasks NOTE: Executing Tasks NOTE: Tasks Summary: Attempted 1 tasks of which 0 didn't need to be rerun and all succeeded. NOTE: Writing buildhistory NOTE: Writing buildhistory took: 0 seconds NOTE: Started PRServer with DBfile: /workdir/build/phyboard-polis-imx8mm-wic/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 45439, PID: 23192 Loading cache: 100% |#########################################################################################################################################| Time: 0:00:00 Loaded 3385 entries from dependency cache. Parsing recipes: 100% |#######################################################################################################################################| Time: 0:00:00 Parsing of 2275 .bb files complete (2274 cached, 1 parsed). 3386 targets, 210 skipped, 2 masked, 0 errors. NOTE: Resolving any missing task queue dependencies Build Configuration: BB_VERSION = "1.46.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "universal" TARGET_SYS = "aarch64-resy-linux" MACHINE = "phyboard-polis-imx8mm" DISTRO = "resy" DISTRO_VERSION = "3.1.2" TUNE_FEATURES = "aarch64 armv8a crc crypto" TARGET_FPU = "" meta meta-poky meta-yocto-bsp = "2020-08-28-dunfell-3.1.2+:d3d80fa6fbf15189f6183a33c95fa90053535ae2" meta-resy = "dunfell:2968f06c1e400131e15bd726808a43195eec8c16" meta-freescale = "2020-08-28-dunfell:3fc701d11b4b096ba016110c83fe1b48c5f3a256" meta-phyboard-polis-imx8mm-bsp = "v5.8.x-upstream:e1aaa104ab7879cd1b8f9272aafd06c24691ccb5" meta-oe meta-python meta-networking = "2020-04-30-dunfell-3.1:17fd382f3467e20308924499b7531ae8b789f056" Initialising tasks: 100% |####################################################################################################################################| Time: 0:00:00 Sstate summary: Wanted 0 Found 0 Missed 0 Current 0 (0% match, 0% complete) NOTE: No setscene tasks NOTE: Executing Tasks NOTE: Tasks Summary: Attempted 1 tasks of which 0 didn't need to be rerun and all succeeded. NOTE: Writing buildhistory NOTE: Writing buildhistory took: 0 seconds NOTE: Started PRServer with DBfile: /workdir/build/phyboard-polis-imx8mm-wic/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 34744, PID: 23357 Loading cache: 100% |#########################################################################################################################################| Time: 0:00:00 Loaded 3385 entries from dependency cache. Parsing recipes: 100% |#######################################################################################################################################| Time: 0:00:00 Parsing of 2275 .bb files complete (2274 cached, 1 parsed). 3386 targets, 210 skipped, 2 masked, 0 errors. NOTE: Resolving any missing task queue dependencies Build Configuration: BB_VERSION = "1.46.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "universal" TARGET_SYS = "aarch64-resy-linux" MACHINE = "phyboard-polis-imx8mm" DISTRO = "resy" DISTRO_VERSION = "3.1.2" TUNE_FEATURES = "aarch64 armv8a crc crypto" TARGET_FPU = "" meta meta-poky meta-yocto-bsp = "2020-08-28-dunfell-3.1.2+:d3d80fa6fbf15189f6183a33c95fa90053535ae2" meta-resy = "dunfell:2968f06c1e400131e15bd726808a43195eec8c16" meta-freescale = "2020-08-28-dunfell:3fc701d11b4b096ba016110c83fe1b48c5f3a256" meta-phyboard-polis-imx8mm-bsp = "v5.8.x-upstream:e1aaa104ab7879cd1b8f9272aafd06c24691ccb5" meta-oe meta-python meta-networking = "2020-04-30-dunfell-3.1:17fd382f3467e20308924499b7531ae8b789f056" Initialising tasks: 100% |####################################################################################################################################| Time: 0:00:00 Sstate summary: Wanted 3 Found 3 Missed 0 Current 100 (100% match, 100% complete) NOTE: Executing Tasks NOTE: Tasks Summary: Attempted 459 tasks of which 453 didn't need to be rerun and all succeeded. NOTE: Writing buildhistory NOTE: Writing buildhistory took: 0 seconds pokyuser@862ca11939a7:/workdir/build$ cat /workdir/build/phyboard-polis-imx8mm-wic/tmp/work/aarch64-mx8mm-resy-linux/firmware-imx-8m/8.5-r0/temp/log.task_order do_deploy_setscene (23394): log.do_deploy_setscene.23394 do_deploy_source_date_epoch_setscene (23411): log.do_deploy_source_date_epoch_setscene.23411 pokyuser@862ca11939a7:/workdir/build$
Now we explicitly told our u-boot
recipe that u-boot/do_compile
depends on firmware-imx-8m/do_deploy
and this does now what we want.
The log files show that do_deploy_setscene
and do_deploy_source_date_epoch_setscene
were executed, meaning the build artifacts of firmware-imx-8m
ended up where u-boot-phytec-imx
can pick them up.
Conclusion
- Read the BitBake Manual - when done
- Read the Mega Manual - when done
- Ask and contribute to our community
- Continue with step 1
Comments
Post a Comment