1
0
mirror of https://git.yoctoproject.org/meta-arm synced 2026-07-17 16:17:09 +00:00

Compare commits

..

12 Commits

Author SHA1 Message Date
Abdellatif El Khlifi 2e2dbc2587 arm/trusted-firmware-m: setting the toolchain file path in PACKAGECONFIG
Toolchain files toolchain_GNUARM.cmake and toolchain_ARMCLANG.cmake are
located at trusted-firmware-m source directory.

This commit sets that.

Change-Id: If9c26f65b0c8111a6ff1f1a7d56610563efd501b
Signed-off-by: Abdellatif El Khlifi <abdellatif.elkhlifi@arm.com>
Signed-off-by: Jon Mason <jon.mason@arm.com>
2021-10-20 14:07:18 -04:00
Abdellatif El Khlifi ab574b084e arm/trusted-firmware-m: enabling PACKAGECONFIG when using CMake
In case of CMake, PACKAGECONFIG configs take effect when passing
PACKAGECONFIG_CONFARGS to the configure task.

Change-Id: I126ba089c9a5db8e895b8a9545e96ef9fa98ce0d
Signed-off-by: Abdellatif El Khlifi <abdellatif.elkhlifi@arm.com>
Signed-off-by: Jon Mason <jon.mason@arm.com>
2021-10-20 14:07:18 -04:00
Ross Burton b4f52f2642 arm/lib/oeqa: fix module lookup
As multiple paths can and do provide modules under oeqa.controllers, all
of these paths need to call pkgutil.extend_path() so the lookup works
correctly.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Jon Mason <jon.mason@arm.com>
2021-10-20 14:07:18 -04:00
Ross Burton 2d22a1f7ae CI: enable testimage for fvp-base
Inherit fvpboot so that the FVP binary is fetched and configuration
generated.

Configure the FVP to forward host port 8022 to port 22 locally, and
tell testimage to SSH to localhost:8022. This has the limitation that
only one testimage can run per machine, but this will be removed shortly.

Disable the parselogs test case, as there are some harmless warnings in
the dmesg which cause it to fail.  Currently meta-arm can't extend the
whitelist of ignorable warnings.

The FVP binaries are x86-64 only, so tag the job appropriately.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Jon Mason <jon.mason@arm.com>
2021-10-20 14:07:11 -04:00
Ross Burton ba86aeb31f CI: install telnet into the images
runfvp currently needs telnet to communicate with the guest, so install
telnet into the image.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Jon Mason <jon.mason@arm.com>
2021-10-20 14:07:11 -04:00
Ross Burton 48524c32d5 arm-bsp/fvp-common: set TEST_TARGET to OEFVPTarget
Set TEST_TARGET so that all FVP machines use the new FVP target out of
the box, instead of attempting to use qemu.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Jon Mason <jon.mason@arm.com>
2021-10-20 14:07:11 -04:00
Ross Burton 4b1a0c7484 arm-bsp/fvp: enable virtio drivers
All FVPs can use virtio networking devices, so enable virtio in all of
the FVP kernels.

Remove our fvp/fvp-virtio.cfg as there's cfg/virtio.scc we can use.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Jon Mason <jon.mason@arm.com>
2021-10-20 14:07:11 -04:00
Ross Burton 2824f6a58b arm/oeqa: add FVP testimage controller target
Add a new oeqa.core.target.OETarget subclass for testimage which starts
a FVP using runfvp. This uses --console to connect to the console so
telnet is needed on the host.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Jon Mason <jon.mason@arm.com>
2021-10-20 14:07:11 -04:00
Ross Burton 0a64644bc9 runfvp: reset the process group on startup
So that it is easy to kill runfvp and everything it starts (such as
telnet or the FVP itself), reset the process group on startup.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Jon Mason <jon.mason@arm.com>
2021-10-20 14:07:11 -04:00
Ross Burton 491cac17fe gem5/gem5-native: add missing m4-native DEPENDS
Latest oe-core has less implicit DEPENDS, so gem5 fails to configure:

Error: Can't find version of M4 macro processor.  Please install M4 and try again.

Explicitly add m4-native to DEPENDS.

Change-Id: I2e107ea6448eec399619e0d1922fd29930a95fd8
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Jon Mason <jon.mason@arm.com>
2021-10-20 14:07:11 -04:00
Ross Burton 328d95e27d CI: use http: to access sstate mirror
The sstate mirrors are not available over HTTPS currently due to a
certificate problem, so use plain HTTP instead.

Change-Id: I5b974d67bc13f7c7234927c6bc62a8c733e454c3
Signed-off-by: Ross Burton <ross.burton@arm.com>
2021-10-19 16:39:48 +01:00
Hugo L'Hostis 99b9ee4470 kas/fvp-baser-aemv8r64: use honister as default branch
Issue-Id: SCM-3532
Signed-off-by: Hugo L'Hostis <hugo.lhostis@arm.com>
Change-Id: I05b529da0f5bbdcee038476d07c42f1b6ac0526f
Signed-off-by: Jon Mason <jon.mason@arm.com>
2021-10-15 10:16:14 -04:00
934 changed files with 44755 additions and 28806 deletions
-1
View File
@@ -1,2 +1 @@
__pycache__
build
+159 -210
View File
@@ -1,34 +1,22 @@
image: ${MIRROR_GHCR}/siemens/kas/kas:4.0
variables:
CPU_REQUEST: ""
DEFAULT_TAG: ""
CACHE_DIR: $CI_BUILDS_DIR/persist
MIRROR_GHCR: ghcr.io
# These are needed as the k8s executor doesn't respect the container entrypoint
# by default
FF_KUBERNETES_HONOR_ENTRYPOINT: 1
FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY: 0
ACS_TEST: 0
ACS_TAG: ""
image: ghcr.io/siemens/kas/kas
# First do a common bootstrap, and then build all the targets
stages:
- prep
- bootstrap
- build
# Common job fragment to get a worker ready
.setup:
tags:
- $DEFAULT_TAG
stage: build
interruptible: true
variables:
KAS_WORK_DIR: $CI_PROJECT_DIR/work
KAS_REPO_REF_DIR: $CACHE_DIR/repos
SSTATE_DIR: $CACHE_DIR/sstate
DL_DIR: $CACHE_DIR/downloads
KAS_REPO_REF_DIR: $CI_BUILDS_DIR/persist/repos
SSTATE_DIR: $CI_BUILDS_DIR/persist/sstate
DL_DIR: $CI_BUILDS_DIR/persist/downloads
BB_LOGCONFIG: $CI_PROJECT_DIR/ci/logging.yml
TOOLCHAIN_DIR: $CACHE_DIR/toolchains
TOOLCHAIN_DIR: $CI_BUILDS_DIR/persist/toolchains
IMAGE_DIR: $CI_PROJECT_DIR/work/build/tmp/deploy/images
TOOLCHAIN_LINK_DIR: $CI_PROJECT_DIR/work/build/toolchains
before_script:
@@ -39,122 +27,125 @@ stages:
- mkdir --verbose --parents $KAS_WORK_DIR $KAS_REPO_REF_DIR $SSTATE_DIR $DL_DIR $TOOLCHAIN_DIR $TOOLCHAIN_LINK_DIR
# Must do this here, as it's the only way to make sure the toolchain is installed on the same builder
- ./ci/get-binary-toolchains $DL_DIR $TOOLCHAIN_DIR $TOOLCHAIN_LINK_DIR
- sudo apt update && sudo apt install telnet -y
# Generalised fragment to do a Kas build
.build:
extends: .setup
variables:
KUBERNETES_CPU_REQUEST: $CPU_REQUEST
rules:
# Don't run MR pipelines
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
when: never
# Don't run pipelines for tags
- if: $CI_COMMIT_TAG
when: never
# Don't run if BUILD_ENABLE_REGEX is set, but the job doesn't match the regex
- if: '$BUILD_ENABLE_REGEX != null && $CI_JOB_NAME !~ $BUILD_ENABLE_REGEX'
when: never
# Allow the dev kernels to fail and not fail the overall build
- if: '$KERNEL == "linux-yocto-dev"'
allow_failure: true
# Catch all for everything else
- if: '$KERNEL != "linux-yocto-dev"'
script:
- KASFILES=$(./ci/jobs-to-kas "$CI_JOB_NAME"):lockfile.yml
- kas dump --update --force-checkout --resolve-refs --resolve-env $KASFILES
- KASFILES=$(./ci/jobs-to-kas $CI_JOB_NAME)
- kas shell --update --force-checkout $KASFILES -c 'cat conf/*.conf'
- kas build $KASFILES
- ./ci/check-warnings $KAS_WORK_DIR/build/warnings.log
artifacts:
name: "logs"
when: always
paths:
- $CI_PROJECT_DIR/work/build/tmp*/work*/**/temp/log.do_*.*
- $CI_PROJECT_DIR/work/build/tmp*/work*/**/testimage/*
# Workaround for Zephyr not currectly handling TESTIMAGE_AUTO
.build_and_test:
extends: .setup
script:
- KASFILES=$(./ci/jobs-to-kas $CI_JOB_NAME)
- kas shell --update --force-checkout $KASFILES -c 'cat conf/*.conf'
- kas build $KASFILES
- kas build $KASFILES -c testimage
- ./ci/check-warnings $KAS_WORK_DIR/build/warnings.log
#
# Prep stage, update repositories once.
# Set the CI variable CI_CLEAN_REPOS=1 to refetch the respositories from scratch
# Prep stage, update repositories once
#
update-repos:
extends: .setup
stage: prep
allow_failure:
exit_codes: 128
script:
- |
flock --verbose --timeout 60 $KAS_REPO_REF_DIR ./ci/update-repos
# Only generate if doesn't already exist, to allow feature branches to drop one in.
if test -f lockfile.yml; then
echo Using existing lockfile.yml
else
# Be sure that this is the complete list of layers being fetched
kas dump --lock --update ci/qemuarm64.yml:ci/meta-openembedded.yml:ci/clang.yml:ci/meta-virtualization.yml | tee lockfile.yml
fi
artifacts:
name: "lockfile"
paths:
- lockfile.yml
- flock --verbose --timeout 60 $KAS_REPO_REF_DIR ./ci/update-repos
#
# Bootstrap stage, bootstrap and machine coverage
#
# Build a number of native tools first to ensure the other builders don't race
# over them
n1sdp/bootstrap:
extends: .build
stage: bootstrap
n1sdp/clang/clang-bootstrap:
extends: .build
stage: bootstrap
# What percentage of machines in the layer do we build
machine-coverage:
stage: bootstrap
interruptible: true
script:
- ./ci/check-machine-coverage
coverage: '/Coverage: \d+/'
#
# Build stage, the actual build jobs
#
# Available options for building are
# DISTRO: [poky, poky-tiny]
# KERNEL: [linux-yocto, linux-yocto-dev, linux-yocto-rt]
# TOOLCHAINS: [gcc, clang, external-gccarm]
# TCLIBC: [glibc, musl]
# FIRMWARE: [u-boot, edk2]
# TS: [none, trusted-services]
# VIRT: [none, xen]
# TESTING: testimage
corstone1000-fvp:
# Validate layers are Yocto Project Compatible
check-layers:
extends: .setup
coverage: '/Coverage: \d+/'
script:
- kas shell --update --force-checkout ci/base.yml:ci/meta-arm-autonomy.yml:ci/meta-openembedded.yml --command \
"$CI_PROJECT_DIR/ci/check-layers.py $CI_PROJECT_DIR/ci/check-layers.yml $CI_PROJECT_DIR $KAS_WORK_DIR"
pending-updates:
extends: .setup
artifacts:
paths:
- update-report.html
script:
- kas shell ci/qemuarm64.yml:ci/meta-openembedded.yml -c "$CI_PROJECT_DIR/scripts/machine-summary.py -t updates.html -o $CI_PROJECT_DIR/update-report.html $($CI_PROJECT_DIR/ci/listmachines.py meta-arm meta-arm-bsp)"
corstone500:
extends: .build
parallel:
matrix:
- TESTING: [testimage, tftf]
corstone1000-mps3:
corstone700-fvp:
extends: .build
parallel:
matrix:
- TESTING: [none, tftf]
fvp-base:
corstone700-mps3:
extends: .build
parallel:
matrix:
- TESTING: testimage
- FIRMWARE: edk2
- SYSTEMREADY_FIRMWARE: arm-systemready-firmware
arm-systemready-ir-acs:
fvp-base/testimage:
extends: .build
timeout: 12h
parallel:
matrix:
# arm-systemready-ir-acs must be specified after fvp-base for ordering
# purposes for the jobs-to-kas output. It is not enough to just have it
# in the job name because fvp-base.yml overwrites the target.
- PLATFORM: fvp-base
ARM_SYSTEMREADY_IR_ACS: arm-systemready-ir-acs
tags:
- ${ACS_TAG}
- x86_64
fvp-base-arm32:
extends: .build
fvp-base-arm32/external-gccarm:
extends: .build
fvp-baser-aemv8r64:
extends: .build
fvps:
extends: .build
gem5-arm64:
extends: .build
gem5-arm64/xen:
extends: .build
gem5-atp-arm64:
extends: .build
generic-arm64:
extends: .build
juno:
extends: .build
parallel:
matrix:
- TOOLCHAINS: [gcc, clang]
FIRMWARE: [u-boot, edk2]
juno/clang:
extends: .build
microbit-v1/testimage-zephyr:
extends: .build_and_test
musca-b1:
extends: .build
@@ -164,154 +155,112 @@ musca-s1:
n1sdp:
extends: .build
parallel:
matrix:
- TESTING: [none, n1sdp-ts, n1sdp-optee, tftf]
qemu-generic-arm64:
n1sdp/armgcc:
extends: .build
parallel:
matrix:
- KERNEL: [linux-yocto, linux-yocto-dev, linux-yocto-rt]
TOOLCHAINS: [gcc, clang]
TESTING: testimage
qemuarm64-secureboot:
qemu-cortex-a53:
extends: .build
parallel:
matrix:
- KERNEL: [linux-yocto, linux-yocto-dev, linux-yocto-rt]
TOOLCHAINS: [gcc, clang]
TCLIBC: [glibc, musl]
TS: [none, qemuarm64-secureboot-ts]
TESTING: testimage
qemuarm64:
extends: .build
parallel:
matrix:
- DISTRO: poky
KERNEL: [linux-yocto, linux-yocto-dev, linux-yocto-rt]
TOOLCHAINS: [gcc, clang]
FIRMWARE: [u-boot, edk2]
TESTING: testimage
- DISTRO: poky-tiny
TESTING: testimage
- VIRT: xen
qemu-cortex-m3/testimage-zephyr:
extends: .build_and_test
qemuarm-secureboot:
qemu-cortex-r5:
extends: .build
parallel:
matrix:
- KERNEL: [linux-yocto, linux-yocto-dev, linux-yocto-rt]
TOOLCHAINS: [gcc, clang]
TCLIBC: [glibc, musl]
TESTING: testimage
- TOOLCHAINS: external-gccarm
TESTING: testimage
qemuarm:
qemuarm64-sbsa:
extends: .build
parallel:
matrix:
- DISTRO: poky
KERNEL: [linux-yocto, linux-yocto-dev, linux-yocto-rt]
TOOLCHAINS: [gcc, clang]
FIRMWARE: [u-boot, edk2]
TESTING: testimage
- DISTRO: poky-tiny
TESTING: testimage
- VIRT: xen
qemuarmv5:
qemuarm64-secureboot/testimage:
extends: .build
qemuarm64-secureboot/clang/testimage:
extends: .build
qemuarm64-secureboot/clang/musl/testimage:
extends: .build
qemuarm64-secureboot/musl/testimage:
extends: .build
qemuarm64/xen:
extends: .build
qemuarm/clang/testimage:
extends: .build
qemuarm/testimage:
extends: .build
qemuarm/xen:
extends: .build
qemuarmv5/testimage:
extends: .build
parallel:
matrix:
- DISTRO: poky
KERNEL: [linux-yocto, linux-yocto-dev, linux-yocto-rt]
TESTING: testimage
- DISTRO: poky-tiny
TESTING: testimage
sgi575:
extends: .build
tc0:
extends: .build
tags:
- x86_64
tc1:
extends: .build
parallel:
matrix:
- TESTING: testimage
tags:
- x86_64
toolchains:
extends: .build
selftest:
#
# Utility tasks, not executed automatically
#
delete-dl-dir:
extends: .setup
stage: prep
when: manual
script:
- KASFILES=./ci/qemuarm64.yml:./ci/selftest.yml:lockfile.yml
- kas shell --update --force-checkout $KASFILES -c 'oe-selftest --num-processes 2 --select-tag meta-arm --run-all-tests'
- rm -rf $DL_DIR/*
# Validate layers are Yocto Project Compatible
check-layers:
delete-repo-dir:
extends: .setup
stage: prep
when: manual
script:
- kas shell --update --force-checkout ci/base.yml:ci/meta-openembedded.yml:lockfile.yml --command \
"yocto-check-layer-wrapper $CI_PROJECT_DIR/$LAYER --dependency $CI_PROJECT_DIR/meta-* $KAS_WORK_DIR/meta-openembedded/meta-oe --no-auto-dependency"
parallel:
matrix:
- LAYER: [meta-arm, meta-arm-bsp, meta-arm-toolchain]
- rm -rf $KAS_REPO_REF_DIR/*
pending-updates:
# Delete all sstate
delete-sstate:
extends: .setup
artifacts:
paths:
- update-report
stage: prep
when: manual
script:
- rm -fr update-report
# This configuration has all of the layers we need enabled
- kas shell --update --force-checkout ci/qemuarm64.yml:ci/meta-openembedded.yml:ci/meta-secure-core.yml:lockfile.yml --command \
"$CI_PROJECT_DIR/scripts/machine-summary.py -t report -o $CI_PROJECT_DIR/update-report $($CI_PROJECT_DIR/ci/listmachines.py meta-arm meta-arm-bsp)"
# Do this on x86 whilst the compilers are x86-only
tags:
- x86_64
- rm -rf $SSTATE_DIR/*
# What percentage of machines in the layer do we build
machine-coverage:
delete-toolchains:
extends: .setup
stage: prep
when: manual
script:
- ./ci/check-machine-coverage
coverage: '/Coverage: \d+/'
- rm -rf $TOOLCHAIN_DIR/*
metrics:
# Wipe out old sstate
prune-sstate:
extends: .setup
artifacts:
reports:
metrics: metrics.txt
stage: prep
when: manual
script:
- kas shell --update --force-checkout ci/base.yml --command \
"$CI_PROJECT_DIR/ci/patchreview $CI_PROJECT_DIR/meta-* --verbose --metrics $CI_PROJECT_DIR/metrics.txt"
- du -h -s $SSTATE_DIR
- find $SSTATE_DIR -type f -atime +30 -delete
- du -h -s $SSTATE_DIR
documentation:
# Report on disk usage
usage:
extends: .setup
stage: prep
when: manual
script:
- |
# This can be removed when the kas container has python3-venv installed
sudo apt-get update && sudo apt-get install --yes python3-venv
python3 -m venv venv
. ./venv/bin/activate
pip3 install -r meta-arm-bsp/documentation/requirements.txt
for CONF in meta-*/documentation/*/conf.py ; do
echo Building $CONF...
SOURCE_DIR=$(dirname $CONF)
MACHINE=$(basename $SOURCE_DIR)
sphinx-build -vW $SOURCE_DIR build-docs/$MACHINE
done
test -d build-docs/
artifacts:
paths:
- build-docs/
- du -h -s $DL_DIR $SSTATE_DIR $KAS_REPO_REF_DIR $TOOLCHAIN_DIR
+13
View File
@@ -6,6 +6,10 @@ This repository contains the Arm layers for OpenEmbedded.
This layer contains general recipes for the Arm architecture, such as firmware, FVPs, and Arm-specific integration.
* meta-arm-autonomy
This layer is the distribution for a reference stack for autonomous systems.
* meta-arm-bsp
This layer contains machines for Arm reference platforms, for example FVP Base, N1SDP, and Juno.
@@ -14,6 +18,15 @@ This repository contains the Arm layers for OpenEmbedded.
This layer contains recipes for Arm's binary toolchains (GCC and Clang for -A and -M), and a recipe to build Arm's GCC.
* meta-atp
This layer contains recipes for the Adaptive Traffic Generation integration into meta-gem5.
* meta-gem5
This layer contains recipes and machines for gem5, a system-level and processor simulator.
Other Directories
-----------------
-4
View File
@@ -1,4 +0,0 @@
header:
version: 11
includes:
- kas/arm-systemready-firmware.yml
-14
View File
@@ -1,14 +0,0 @@
header:
version: 11
includes:
- kas/arm-systemready-ir-acs.yml
env:
ACS_TEST: "0"
local_conf_header:
testimage: |
TESTIMAGE_AUTO = "${ACS_TEST}"
target:
- arm-systemready-ir-acs
+6
View File
@@ -0,0 +1,6 @@
header:
version: 9
local_conf_header:
cc: |
GCCVERSION = "arm-10.3"
+15 -8
View File
@@ -1,11 +1,11 @@
header:
version: 14
version: 9
distro: poky
defaults:
repos:
branch: nanbield
refspec: honister
repos:
meta-arm:
@@ -27,17 +27,24 @@ env:
local_conf_header:
base: |
CONF_VERSION = "2"
setup: |
PACKAGE_CLASSES = "package_ipk"
LICENSE_FLAGS_WHITELIST += "armcompiler Arm-FVP-EULA"
PACKAGECONFIG:remove:pn-qemu-system-native = "gtk+ sdl"
PACKAGECONFIG:append:pn-perf = " coresight"
EXTRA_IMAGE_FEATURES:append = " debug-tweaks"
BB_NUMBER_THREADS = "16"
PARALLEL_MAKE = "-j16"
INHERIT += "rm_work"
PACKAGECONFIG:append:pn-perf = " coresight"
noptest: |
DISTRO_FEATURES:remove = "ptest"
extrapackages: |
CORE_IMAGE_EXTRA_INSTALL += "perf opencsd"
CORE_IMAGE_EXTRA_INSTALL:append:aarch64 = " gator-daemon"
sstate_mirror: |
BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687"
BB_HASHSERVE = "auto"
BB_SIGNATURE_HANDLER = "OEEquivHash"
SSTATE_MIRRORS = "file://.* http://sstate.yoctoproject.org/dev/PATH;downloadfilename=PATH"
machine: unset
target:
- core-image-sato
- core-image-base
- perf
+9
View File
@@ -0,0 +1,9 @@
header:
version: 9
target:
- binutils-cross-aarch64
- gcc-cross-aarch64
- python3-native
- opkg-native
- rpm-native
+44
View File
@@ -0,0 +1,44 @@
#! /usr/bin/env python3
import argparse
import pathlib
import re
import subprocess
import sys
import yaml
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("config", type=argparse.FileType())
parser.add_argument("metaarm", type=pathlib.Path, help="Path to meta-arm")
parser.add_argument("others", type=pathlib.Path, help="Path to parent of dependencies")
args = parser.parse_args()
config = yaml.safe_load(args.config)
layers = config["layers"]
dependencies = config["dependencies"]
found_layers = [p for p in args.metaarm.glob("meta-*") if p.is_dir()]
print(f"Testing {len(layers)} layers: {', '.join(layers)}.")
print(f"Found {len(found_layers)} layers in meta-arm.")
print()
cli = ["yocto-check-layer-wrapper",]
cli.extend([args.metaarm / layer for layer in layers])
cli.append("--dependency")
cli.extend([args.others / layer for layer in dependencies])
cli.append("--no-auto-dependency")
passed = 0
process = subprocess.Popen(cli, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, universal_newlines=True)
while True:
line = process.stdout.readline()
if process.poll() is not None:
break
print(line.strip())
if re.search(r"meta-.+ PASS", line):
passed += 1
print(f"Coverage: {int(passed / len(found_layers) * 100)}%")
sys.exit(process.returncode)
+13
View File
@@ -0,0 +1,13 @@
layers:
- meta-arm
- meta-arm-bsp
- meta-arm-toolchain
- meta-gem5
- meta-arm-autonomy
dependencies:
- meta-openembedded/meta-oe
- meta-openembedded/meta-networking
- meta-openembedded/meta-python
- meta-openembedded/meta-filesystems
- poky/meta-poky
- meta-virtualization
+10
View File
@@ -0,0 +1,10 @@
header:
version: 9
target:
- binutils-cross-aarch64
- clang-cross-aarch64
- clang-native
- python3-native
- opkg-native
- rpm-native
+3 -11
View File
@@ -1,19 +1,11 @@
header:
version: 14
version: 9
repos:
meta-clang:
url: https://github.com/kraj/meta-clang
refspec: master
local_conf_header:
toolchain: |
clang: |
TOOLCHAIN = "clang"
PREFERRED_PROVIDER_llvm = "clang"
PREFERRED_PROVIDER_llvm-native = "clang-native"
PREFERRED_PROVIDER_nativesdk-llvm = "nativesdk-clang"
PROVIDES:pn-clang = "llvm"
PROVIDES:pn-clang-native = "llvm-native"
PROVIDES:pn-nativesdk-clang = "nativesdk-llvm"
# This is needed to stop bitbake getting confused about what clang/llvm is
# being used, see https://github.com/kraj/meta-clang/pull/766
BBMASK += "/meta/recipes-devtools/llvm/llvm.*\.bb"
-15
View File
@@ -1,15 +0,0 @@
header:
version: 14
includes:
- ci/base.yml
- ci/meta-openembedded.yml
- ci/poky-tiny.yml
- ci/meta-secure-core.yml
local_conf_header:
extrapackages: |
# Intentionally blank to prevent perf from being added to the image in base.yml
target:
- corstone1000-image
- perf
-12
View File
@@ -1,12 +0,0 @@
header:
version: 14
includes:
- ci/corstone1000-common.yml
- ci/fvp.yml
local_conf_header:
fvp-config: |
# Remove Dropbear SSH as it will not fit into the corstone1000 image.
IMAGE_FEATURES:remove = " ssh-server-dropbear"
machine: corstone1000-fvp
-6
View File
@@ -1,6 +0,0 @@
header:
version: 14
includes:
- ci/corstone1000-common.yml
machine: corstone1000-mps3
+6
View File
@@ -0,0 +1,6 @@
header:
version: 9
includes:
- base.yml
machine: corstone500
+10
View File
@@ -0,0 +1,10 @@
header:
version: 9
includes:
- base.yml
machine: corstone700-fvp
local_conf_header:
image: |
CORE_IMAGE_EXTRA_INSTALL = "corstone700-test-app"
+6
View File
@@ -0,0 +1,6 @@
header:
version: 9
includes:
- corstone700-fvp.yml
machine: corstone700-mps3
-7
View File
@@ -1,7 +0,0 @@
header:
version: 14
# Add universally helpful features when testing boards
local_conf_header:
debug: |
EXTRA_IMAGE_FEATURES:append = " debug-tweaks"
-31
View File
@@ -1,31 +0,0 @@
#! /usr/bin/env python3
"""
Download the lockfile.yml produced by a CI pipeline, specified by the GitLab
server, full name of the meta-arm project, and the refspec that was executed.
For example,
$ ./download-lockfile.py https://gitlab.com/ rossburton/meta-arm master
SPDX-FileCopyrightText: Copyright 2023 Arm Limited and Contributors
SPDX-License-Identifier: GPL-2.0-only
"""
import argparse
import gitlab
import io
import zipfile
parser = argparse.ArgumentParser()
parser.add_argument("server", help="GitLab server name")
parser.add_argument("project", help="meta-arm project name")
parser.add_argument("refspec", help="Branch/commit")
args = parser.parse_args()
gl = gitlab.Gitlab(args.server)
project = gl.projects.get(args.project)
artefact = project.artifacts.download(ref_name=args.refspec, job="update-repos")
z = zipfile.ZipFile(io.BytesIO(artefact))
z.extract("lockfile.yml")
print("Fetched lockfile.yml")
-17
View File
@@ -1,17 +0,0 @@
header:
version: 14
local_conf_header:
bootfirmware: |
PREFERRED_PROVIDER_virtual/bootloader = "edk2-firmware"
MACHINE_FEATURES += "efi"
TFA_UBOOT = "0"
TFA_UEFI = "1"
EXTRA_IMAGEDEPENDS += "edk2-firmware"
EFI_PROVIDER ?= "grub-efi"
QB_DEFAULT_BIOS = "QEMU_EFI.fd"
WKS_FILE ?= "efi-disk.wks.in"
failing_tests: |
TEST_SUITES:remove = "xorg"
+2 -5
View File
@@ -1,11 +1,8 @@
header:
version: 14
version: 9
local_conf_header:
cc: |
SKIP_RECIPE[gcc-cross-arm] = "Using external toolchain"
PNBLACKLIST[gcc-cross-arm] = "Using external toolchain"
TCMODE = "external-arm"
EXTERNAL_TOOLCHAIN = "${TOPDIR}/toolchains/${TARGET_ARCH}"
# Temporary workaround for a number binaries in the toolchains that are using 32bit timer API
# This must be done here instead of the recipe because of all the libraries in the toolchain have the issue
INSANE_SKIP:append = " 32bit-time"
+6
View File
@@ -0,0 +1,6 @@
header:
version: 9
includes:
- base.yml
machine: fvp-base-arm32
+12 -3
View File
@@ -1,7 +1,16 @@
header:
version: 14
version: 9
includes:
- ci/base.yml
- ci/fvp.yml
- base.yml
machine: fvp-base
local_conf_header:
testimagefvp: |
INHERIT = "fvpboot"
# This fails but we can't add to the ignorelist from meta-arm yet
# https://bugzilla.yoctoproject.org/show_bug.cgi?id=14604
TEST_SUITES:remove = "parselogs"
# Tell testimage to connect to localhost:8022, and forward that to SSH in the FVP.
TEST_TARGET_IP = "localhost:8022"
FVP_CONFIG[bp.virtio_net.hostbridge.userNetPorts] ?= "8022=22"
+7
View File
@@ -0,0 +1,7 @@
header:
version: 9
includes:
- base.yml
machine: fvp-baser-aemv8r64
-12
View File
@@ -1,12 +0,0 @@
header:
version: 14
local_conf_header:
testimagefvp: |
LICENSE_FLAGS_ACCEPTED += "Arm-FVP-EULA"
IMAGE_CLASSES += "fvpboot"
failing_tests: |
# This fails but we can't add to the ignorelist from meta-arm yet
# https://bugzilla.yoctoproject.org/show_bug.cgi?id=14604
TEST_SUITES:remove = "parselogs"
TEST_SUITES:remove = "xorg"
+5 -10
View File
@@ -1,25 +1,20 @@
# Simple target to build the FVPs that are publically available
header:
version: 14
version: 9
includes:
- ci/base.yml
- base.yml
machine: qemuarm64
local_conf_header:
license: |
LICENSE_FLAGS_ACCEPTED += "Arm-FVP-EULA"
sdk: |
SDKMACHINE = "x86_64"
target:
# Target packages to test aarch64
- fvp-base-a-aem
- fvp-corstone1000
# Nativesdk to test x86-64
- nativesdk-fvp-base-a-aem
- nativesdk-fvp-corstone1000
- nativesdk-fvp-n1-edge
- nativesdk-fvp-sgi575
- nativesdk-fvp-tc1
- nativesdk-fvp-corstone500
- nativesdk-fvp-corstone700
- nativesdk-fvp-tc0
-7
View File
@@ -1,7 +0,0 @@
header:
version: 14
#NOTE: This is the default for poky. This is only being added for completeness/clarity
local_conf_header:
toolchain: |
TOOLCHAIN = "gcc"
+17
View File
@@ -0,0 +1,17 @@
header:
version: 9
includes:
- base.yml
- meta-openembedded.yml
repos:
meta-arm:
layers:
meta-gem5:
machine: gem5-arm64
target:
- core-image-minimal
- perf
- gem5-aarch64-native
+15
View File
@@ -0,0 +1,15 @@
header:
version: 9
includes:
- gem5-arm64.yml
repos:
meta-arm:
layers:
meta-atp:
machine: gem5-atp-arm64
target:
- atp-native
- core-image-minimal
+2 -2
View File
@@ -1,6 +1,6 @@
header:
version: 14
version: 9
includes:
- ci/base.yml
- base.yml
machine: generic-arm64
+22 -27
View File
@@ -1,51 +1,46 @@
#!/bin/bash
set -u -e
set -u
BASENAME=arm-gnu-toolchain
VER=${VER:-12.2.rel1}
HOST_ARCH=${HOST_ARCH:-$(uname -m)}
HOST_ARCH=$(uname -m)
VER="10.3-2021.07"
# Use the standard kas container locations if nothing is passed into the script
DOWNLOAD_DIR="${1:-/builds/persist/downloads/}"
TOOLCHAIN_DIR="${2:-/builds/persist//toolchains/}"
TOOLCHAIN_LINK_DIR="${3:-build/toolchains/}"
DOWNLOAD_DIR=$1
TOOLCHAIN_DIR=$2
TOOLCHAIN_LINK_DIR=$3
# These should be already created by .gitlab-ci.yml, but do here if run outside of that env
mkdir -p $DOWNLOAD_DIR $TOOLCHAIN_DIR $TOOLCHAIN_LINK_DIR
download() {
TRIPLE=$1
URL=https://developer.arm.com/-/media/Files/downloads/gnu/$VER/binrel/$BASENAME-$VER-$HOST_ARCH-$TRIPLE.tar.xz
wget -P $DOWNLOAD_DIR -nc $URL
}
if [ $HOST_ARCH = "aarch64" ]; then
# AArch64 Linux hosted cross compilers
#AArch64 Linux hosted cross compilers
# AArch32 target with hard float
download arm-none-linux-gnueabihf
#AArch32 target with hard float (arm-none-linux-gnueabihf)
wget -P $DOWNLOAD_DIR -nc https://developer.arm.com/-/media/Files/downloads/gnu-a/$VER/binrel/gcc-arm-$VER-$HOST_ARCH-arm-none-linux-gnueabihf.tar.xz
elif [ $HOST_ARCH = "x86_64" ]; then
# x86_64 Linux hosted cross compilers
#x86_64 Linux hosted cross compilers
# AArch32 target with hard float
download arm-none-linux-gnueabihf
#AArch32 target with hard float (arm-linux-none-gnueabihf)
wget -P $DOWNLOAD_DIR -nc https://developer.arm.com/-/media/Files/downloads/gnu-a/$VER/binrel/gcc-arm-$VER-$HOST_ARCH-arm-none-linux-gnueabihf.tar.xz
# AArch64 GNU/Linux target
download aarch64-none-linux-gnu
#AArch64 GNU/Linux target (aarch64-none-linux-gnu)
wget -P $DOWNLOAD_DIR -nc https://developer.arm.com/-/media/Files/downloads/gnu-a/$VER/binrel/gcc-arm-$VER-$HOST_ARCH-aarch64-none-linux-gnu.tar.xz
#AArch64 GNU/Linux target (aarch64_be-none-linux-gnu)
wget -P $DOWNLOAD_DIR -nc https://developer.arm.com/-/media/Files/downloads/gnu-a/$VER/binrel/gcc-arm-$VER-$HOST_ARCH-aarch64_be-none-linux-gnu.tar.xz
else
echo "ERROR - Unknown build arch of $HOST_ARCH"
exit 1
fi
for i in arm aarch64; do
if [ ! -d $TOOLCHAIN_DIR/$BASENAME-$VER-$HOST_ARCH-$i-none-linux-gnu*/ ]; then
if [ ! -f $DOWNLOAD_DIR/$BASENAME-$VER-$HOST_ARCH-$i-none-linux-gnu*.tar.xz ]; then
for i in arm aarch64 aarch64_be; do
if [ ! -d $TOOLCHAIN_DIR/gcc-arm-$VER-$HOST_ARCH-$i-none-linux-gnu*/ ]; then
if [ ! -f $DOWNLOAD_DIR/gcc-arm-$VER-$HOST_ARCH-$i-none-linux-gnu*.tar.xz ]; then
continue
fi
tar -C $TOOLCHAIN_DIR -axvf $DOWNLOAD_DIR/$BASENAME-$VER-$HOST_ARCH-$i-none-linux-gnu*.tar.xz
tar -C $TOOLCHAIN_DIR -axvf $DOWNLOAD_DIR/gcc-arm-$VER-$HOST_ARCH-$i-none-linux-gnu*.tar.xz
fi
# Setup a link for the toolchain to use local to the building machine (e.g., not in a shared location)
ln -s $TOOLCHAIN_DIR/$BASENAME-$VER-$HOST_ARCH-$i-none-linux-gnu* $TOOLCHAIN_LINK_DIR/$i
ln -s $TOOLCHAIN_DIR/gcc-arm-$VER-$HOST_ARCH-$i-none-linux-gnu* $TOOLCHAIN_LINK_DIR/$i
done
-7
View File
@@ -1,7 +0,0 @@
header:
version: 14
#NOTE: This is the default for poky. This is only being added for completeness/clarity
local_conf_header:
libc: |
TCLIBC = "glibc"
+12 -20
View File
@@ -1,27 +1,19 @@
#! /bin/bash
# This script is expecting an input of machine name, optionally followed by a
# colon and a list of one or more parameters separated by commas between
# brackets. For example, the following are acceptable:
# corstone1000-mps3
# fvp-base: [testimage]
# qemuarm64-secureboot: [clang, glibc, testimage]
#
# Turn this list into a series of yml files separated by colons to pass to kas
# Read a GitLab CI job name on $1 and transform it to a
# list of Kas yaml files
set -e -u
FILES="ci/$(echo $1 | cut -d ':' -f 1).yml"
# Read Job namne from $1 and split on /
IFS=/ read -r -a PARTS<<<$1
for i in $(echo $1 | cut -s -d ':' -f 2 | sed 's/[][,]//g'); do
# Given that there are no yml files for gcc or glibc, as those are the
# defaults, we can simply ignore those parameters. They are necessary
# to pass in so that matrix can correctly setup all of the permutations
# of each individual run.
if [[ $i == 'none' ]]; then
continue
fi
FILES+=":ci/$i.yml"
done
# Prefix each part with ci/
PARTS=("${PARTS[@]/#/ci/}")
echo $FILES
# Suffix each part with .yml
PARTS=("${PARTS[@]/%/.yml}")
# Print colon-separated
IFS=":"
echo "${PARTS[*]}"
+2 -2
View File
@@ -1,6 +1,6 @@
header:
version: 14
version: 9
includes:
- ci/base.yml
- base.yml
machine: juno
-6
View File
@@ -1,6 +0,0 @@
header:
version: 14
local_conf_header:
kernel: |
PREFERRED_PROVIDER_virtual/kernel = "linux-yocto-dev"
-7
View File
@@ -1,7 +0,0 @@
header:
version: 14
#NOTE: This is the default for poky. This is only being added for completeness/clarity
local_conf_header:
kernel: |
PREFERRED_PROVIDER_virtual/kernel = "linux-yocto"
+10
View File
@@ -0,0 +1,10 @@
header:
version: 9
includes:
- meta-openembedded.yml
- meta-virtualization.yml
repos:
meta-arm:
layers:
meta-arm-autonomy:
+1 -2
View File
@@ -1,5 +1,5 @@
header:
version: 14
version: 9
repos:
meta-openembedded:
@@ -9,4 +9,3 @@ repos:
meta-networking:
meta-oe:
meta-python:
meta-perl:
-11
View File
@@ -1,11 +0,0 @@
header:
version: 14
repos:
meta-secure-core:
url: https://github.com/Wind-River/meta-secure-core.git
layers:
meta:
meta-signing-key:
meta-efi-secure-boot:
+3 -3
View File
@@ -1,8 +1,8 @@
header:
version: 14
version: 9
includes:
- ci/meta-openembedded.yml
- meta-openembedded.yml
repos:
meta-virtualization:
url: https://git.yoctoproject.org/meta-virtualization
url: git://git.yoctoproject.org/meta-virtualization
+9
View File
@@ -0,0 +1,9 @@
header:
version: 9
includes:
- meta-openembedded.yml
repos:
meta-zephyr:
url: https://git.yoctoproject.org/git/meta-zephyr
refspec: master
+16
View File
@@ -0,0 +1,16 @@
header:
version: 9
includes:
- base.yml
- meta-zephyr.yml
local_conf_header:
nonbuilding_tests: |
ZEPHYRTESTS:remove = "common"
failing_tests: |
ZEPHYRTESTS:remove = "context early_sleep sleep"
machine: microbit-v1
target:
- zephyr-kernel-test-all
+4 -3
View File
@@ -1,10 +1,11 @@
header:
version: 14
version: 9
includes:
- ci/base.yml
- ci/meta-openembedded.yml
- base.yml
- meta-zephyr.yml
machine: musca-b1
target:
- trusted-firmware-m
- zephyr-philosophers
+3 -3
View File
@@ -1,8 +1,8 @@
header:
version: 14
version: 9
includes:
- ci/base.yml
- ci/meta-openembedded.yml
- base.yml
- meta-openembedded.yml
machine: musca-s1
+1 -1
View File
@@ -1,5 +1,5 @@
header:
version: 14
version: 9
local_conf_header:
libc: |
-12
View File
@@ -1,12 +0,0 @@
header:
version: 14
# Config specific for the optee-xtests
local_conf_header:
optee-test: |
# Include ARM FFA
MACHINE_FEATURES:append = " arm-ffa"
# Include trusted services
TEST_SUITES:append = " trusted_services"
# Include Optee xtests
IMAGE_INSTALL:append = " optee-test"
-14
View File
@@ -1,14 +0,0 @@
header:
version: 14
includes:
- ci/meta-openembedded.yml
local_conf_header:
trusted_services: |
TEST_SUITES:append = " trusted_services"
# Include TS Crypto, TS Protected Storage, TS Internal and Trusted Storage SPs into optee-os image
MACHINE_FEATURES:append = " arm-ffa ts-crypto ts-storage ts-its"
# Include TS demo/test tools into image
IMAGE_INSTALL:append = " packagegroup-ts-tests"
# Include TS PSA Arch tests into image
IMAGE_INSTALL:append = " packagegroup-ts-tests-psa"
+2 -6
View File
@@ -1,10 +1,6 @@
header:
version: 14
version: 9
includes:
- ci/base.yml
- base.yml
machine: n1sdp
local_conf_header:
unsupported_trusted_services: |
MACHINE_FEATURES:remove = "ts-smm-gateway"
-286
View File
@@ -1,286 +0,0 @@
#! /usr/bin/env python3
#
# SPDX-License-Identifier: GPL-2.0-only
#
# TODO
# - option to just list all broken files
# - test suite
# - validate signed-off-by
import argparse
import collections
import json
import os
import re
import subprocess
status_values = (
"accepted",
"pending",
"inappropriate",
"backport",
"submitted",
"denied",
)
class PatchResult:
# Whether the patch has an Upstream-Status or not
missing_upstream_status = False
# If the Upstream-Status tag is malformed in some way (string for bad bit)
malformed_upstream_status = None
# If the Upstream-Status value is unknown (boolean)
unknown_upstream_status = False
# The upstream status value (Pending, etc)
upstream_status = None
# Whether the patch has a Signed-off-by or not
missing_sob = False
# Whether the Signed-off-by tag is malformed in some way
malformed_sob = False
# The Signed-off-by tag value
sob = None
# Whether a patch looks like a CVE but doesn't have a CVE tag
missing_cve = False
class Summary:
total = 0
cve_missing = 0
sob_missing = 0
sob_malformed = 0
status_missing = 0
status_malformed = 0
status_pending = 0
def blame_patch(patch):
"""
From a patch filename, return a list of "commit summary (author name <author
email>)" strings representing the history.
"""
return subprocess.check_output(("git", "log",
"--follow", "--find-renames", "--diff-filter=A",
"--format=%s (%aN <%aE>)",
"--", patch)).decode("utf-8").splitlines()
def patchreview(patches):
# General pattern: start of line, optional whitespace, tag with optional
# hyphen or spaces, maybe a colon, some whitespace, then the value, all case
# insensitive.
sob_re = re.compile(r"^[\t ]*(Signed[-_ ]off[-_ ]by:?)[\t ]*(.+)", re.IGNORECASE | re.MULTILINE)
status_re = re.compile(r"^[\t ]*(Upstream[-_ ]Status:?)[\t ]*(\w*)", re.IGNORECASE | re.MULTILINE)
cve_tag_re = re.compile(r"^[\t ]*(CVE:)[\t ]*(.*)", re.IGNORECASE | re.MULTILINE)
cve_re = re.compile(r"cve-[0-9]{4}-[0-9]{4,6}", re.IGNORECASE)
results = {}
for patch in patches:
result = PatchResult()
results[patch] = result
content = open(patch, encoding="ascii", errors="ignore").read()
# Find the Signed-off-by tag
match = sob_re.search(content)
if match:
value = match.group(1)
if value != "Signed-off-by:":
result.malformed_sob = value
result.sob = match.group(2)
else:
result.missing_sob = True
# Find the Upstream-Status tag
match = status_re.search(content)
if match:
value = match.group(1)
if value != "Upstream-Status:":
result.malformed_upstream_status = value
value = match.group(2).lower()
# TODO: check case
if value not in status_values:
result.unknown_upstream_status = True
result.upstream_status = value
else:
result.missing_upstream_status = True
# Check that patches which looks like CVEs have CVE tags
if cve_re.search(patch) or cve_re.search(content):
if not cve_tag_re.search(content):
result.missing_cve = True
# TODO: extract CVE list
return results
def analyse(results, want_blame=False, verbose=True):
"""
want_blame: display blame data for each malformed patch
verbose: display per-file results instead of just summary
"""
# want_blame requires verbose, so disable blame if we're not verbose
if want_blame and not verbose:
want_blame = False
summary = Summary()
for patch in sorted(results):
r = results[patch]
summary.total += 1
need_blame = False
# Build statistics
if r.missing_sob:
summary.sob_missing += 1
if r.malformed_sob:
summary.sob_malformed += 1
if r.missing_upstream_status:
summary.status_missing += 1
if r.malformed_upstream_status or r.unknown_upstream_status:
summary.status_malformed += 1
# Count patches with no status as pending
summary.status_pending += 1
if r.missing_cve:
summary.cve_missing += 1
if r.upstream_status == "pending":
summary.status_pending += 1
# Output warnings
if r.missing_sob:
need_blame = True
if verbose:
print("Missing Signed-off-by tag (%s)" % patch)
if r.malformed_sob:
need_blame = True
if verbose:
print("Malformed Signed-off-by '%s' (%s)" % (r.malformed_sob, patch))
if r.missing_cve:
need_blame = True
if verbose:
print("Missing CVE tag (%s)" % patch)
if r.missing_upstream_status:
need_blame = True
if verbose:
print("Missing Upstream-Status tag (%s)" % patch)
if r.malformed_upstream_status:
need_blame = True
if verbose:
print("Malformed Upstream-Status '%s' (%s)" % (r.malformed_upstream_status, patch))
if r.unknown_upstream_status:
need_blame = True
if verbose:
print("Unknown Upstream-Status value '%s' (%s)" % (r.upstream_status, patch))
if want_blame and need_blame:
print("\n".join(blame_patch(patch)) + "\n")
return summary
def display_summary(summary, verbose):
def percent(num):
try:
return "%d (%d%%)" % (num, round(num * 100.0 / summary.total))
except ZeroDivisionError:
return "N/A"
if verbose:
print()
print("""Total patches found: %d
Patches missing Signed-off-by: %s
Patches with malformed Signed-off-by: %s
Patches missing CVE: %s
Patches missing Upstream-Status: %s
Patches with malformed Upstream-Status: %s
Patches in Pending state: %s""" % (summary.total,
percent(summary.sob_missing),
percent(summary.sob_malformed),
percent(summary.cve_missing),
percent(summary.status_missing),
percent(summary.status_malformed),
percent(summary.status_pending)))
def generate_metrics(summary, output):
# https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md
# Summary attribute name, MetricPoint help
mapping = (
("total", "Total patches"),
("cve_missing", "Patches missing CVE tag"),
("sob_malformed", "Patches with malformed Signed-off-by"),
("sob_missing", "Patches with missing Signed-off-by"),
("status_malformed", "Patches with malformed Upstream-Status"),
("status_missing", "Patches with missing Upstream-Status"),
("status_pending", "Patches with Pending Upstream-Status")
)
for attr, help in mapping:
metric = f"patch_check_{attr}"
value = getattr(summary, attr)
output.write(f"""
# TYPE {metric} gauge
# HELP {help}
{metric} {value}
""")
output.write("\n# EOF\n")
def histogram(results):
import math
from toolz import dicttoolz, recipes
counts = recipes.countby(lambda r: r.upstream_status, results.values())
bars = dicttoolz.valmap(lambda v: "#" * int(math.ceil(float(v) / len(results) * 100)), counts)
for k in bars:
print("%-20s %s (%d)" % (k.capitalize() if k else "No status", bars[k], counts[k]))
def gather_patches(directories):
patches = []
for directory in directories:
filenames = subprocess.check_output(("git", "-C", directory, "ls-files", "recipes-*/**/*.patch", "recipes-*/**/*.diff")).decode("utf-8").split()
patches += [os.path.join(directory, f) for f in filenames]
return patches
if __name__ == "__main__":
args = argparse.ArgumentParser(description="Patch Review Tool")
args.add_argument("-b", "--blame", action="store_true", help="show blame for malformed patches")
args.add_argument("-v", "--verbose", action="store_true", help="show per-patch results")
args.add_argument("-g", "--histogram", action="store_true", help="show patch histogram")
args.add_argument("-j", "--json", help="update JSON")
args.add_argument("-m", "--metrics", type=argparse.FileType('w'), help="write OpenMetrics")
args.add_argument("dirs", metavar="DIRECTORY", nargs="+", help="directory to scan")
args = args.parse_args()
patches = gather_patches(args.dirs)
results = patchreview(patches)
summary = analyse(results, want_blame=args.blame, verbose=args.verbose)
display_summary(summary, verbose=args.verbose)
if args.json:
if os.path.isfile(args.json):
data = json.load(open(args.json))
else:
data = []
row = collections.Counter()
row["total"] = len(results)
row["date"] = subprocess.check_output(["git", "-C", args.dirs[0], "show", "-s", "--pretty=format:%cd", "--date=format:%s"]).decode("utf-8").strip()
for r in results.values():
if r.upstream_status in status_values:
row[r.upstream_status] += 1
if r.malformed_upstream_status or r.missing_upstream_status:
row["malformed-upstream-status"] += 1
if r.malformed_sob or r.missing_sob:
row["malformed-sob"] += 1
data.append(row)
json.dump(data, open(args.json, "w"))
if args.metrics:
generate_metrics(summary, args.metrics)
if args.histogram:
print()
histogram(results)
-14
View File
@@ -1,14 +0,0 @@
header:
version: 14
distro: poky-tiny
local_conf_header:
hacking: |
TEST_SUITES = "_qemutiny ping"
extrapackages: |
# Intentionally blank to prevent perf from being added to the image in base.yml
target:
- core-image-minimal
- perf
-4
View File
@@ -1,4 +0,0 @@
header:
version: 14
distro: poky
+14
View File
@@ -0,0 +1,14 @@
header:
version: 9
includes:
- base.yml
- meta-zephyr.yml
local_conf_header:
nonbuilding_tests: |
ZEPHYRTESTS:remove += "common device poll queue sleep"
machine: qemu-cortex-a53
target:
- zephyr-kernel-test-all
+16
View File
@@ -0,0 +1,16 @@
header:
version: 9
includes:
- base.yml
- meta-zephyr.yml
local_conf_header:
tclibc: |
TCLIBC = "newlib"
nonbuilding_tests: |
ZEPHYRTESTS:remove = "common context pending poll sleep"
machine: qemu-cortex-m3
target:
- zephyr-kernel-test-all
+10
View File
@@ -0,0 +1,10 @@
header:
version: 9
includes:
- base.yml
- meta-zephyr.yml
machine: qemu-cortex-r5
target:
- zephyr-philosophers
-14
View File
@@ -1,14 +0,0 @@
header:
version: 14
includes:
- ci/generic-arm64.yml
local_conf_header:
failing_tests: |
DEFAULT_TEST_SUITES:remove = "parselogs"
machine: qemu-generic-arm64
target:
- core-image-sato
- sbsa-acs
-12
View File
@@ -1,12 +0,0 @@
header:
version: 14
includes:
- ci/base.yml
machine: qemuarm-secureboot
target:
- core-image-base
- optee-examples
- optee-test
- optee-os-tadevkit
+2 -2
View File
@@ -1,6 +1,6 @@
header:
version: 14
version: 9
includes:
- ci/base.yml
- base.yml
machine: qemuarm
+11
View File
@@ -0,0 +1,11 @@
header:
version: 9
includes:
- base.yml
machine: qemuarm64-sbsa
target:
- core-image-base
- perf
- sbsa-acs
-14
View File
@@ -1,14 +0,0 @@
header:
version: 14
includes:
- ci/meta-openembedded.yml
local_conf_header:
trusted_services: |
TEST_SUITES:append = " trusted_services"
# Include TS Crypto, TS Protected Storage, TS Internal Trusted Storage and SMM-Gateway SPs into optee-os image
MACHINE_FEATURES:append = " arm-ffa ts-crypto ts-storage ts-its ts-smm-gateway"
# Include TS demo/test tools into image
IMAGE_INSTALL:append = " packagegroup-ts-tests"
# Include TS PSA Arch tests into image
IMAGE_INSTALL:append = " packagegroup-ts-tests-psa"
+8 -4
View File
@@ -1,12 +1,16 @@
header:
version: 14
version: 9
includes:
- ci/base.yml
- base.yml
machine: qemuarm64-secureboot
local_conf_header:
bugs: |
# Only ping until errors can be resolved
TEST_SUITES = "ping"
target:
- core-image-base
- perf
- optee-examples
- optee-test
- optee-os-tadevkit
+2 -2
View File
@@ -1,6 +1,6 @@
header:
version: 14
version: 9
includes:
- ci/base.yml
- base.yml
machine: qemuarm64
+2 -2
View File
@@ -1,6 +1,6 @@
header:
version: 14
version: 9
includes:
- ci/base.yml
- base.yml
machine: qemuarmv5
-7
View File
@@ -1,7 +0,0 @@
header:
version: 14
local_conf_header:
setup: |
BB_LOGCONFIG = ""
SANITY_TESTED_DISTROS = ""
+2 -2
View File
@@ -1,6 +1,6 @@
header:
version: 14
version: 9
includes:
- ci/base.yml
- base.yml
machine: sgi575
+10
View File
@@ -0,0 +1,10 @@
header:
version: 9
includes:
- base.yml
machine: tc0
target:
- tc-artifacts-image
- perf
+4 -5
View File
@@ -1,11 +1,10 @@
header:
version: 14
version: 9
includes:
- ci/base.yml
- ci/fvp.yml
- ci/meta-openembedded.yml
- base.yml
machine: tc1
target:
- core-image-minimal
- tc-artifacts-image
- perf
+8
View File
@@ -0,0 +1,8 @@
header:
version: 9
local_conf_header:
testimage: |
IMAGE_CLASSES += "testimage"
TEST_TARGET = "QemuTargetZephyr"
TEST_SUITES = "zephyr"
+8 -10
View File
@@ -1,19 +1,17 @@
header:
version: 14
includes:
- ci/debug.yml
version: 9
local_conf_header:
testimage: |
IMAGE_CLASSES += "testimage"
TESTIMAGE_AUTO = "1"
kvm: |
QEMU_USE_KVM = ""
slirp: |
TEST_RUNQEMUPARAMS = "slirp"
sshd: |
TEST_SERVER_IP = "127.0.0.1"
QEMU_USE_SLIRP = "1"
packages: |
IMAGE_FEATURES:append = " ssh-server-dropbear"
sshkeys: |
CORE_IMAGE_EXTRA_INSTALL += "ssh-pregen-hostkeys"
universally_failing_tests: |
TEST_SUITES:remove = "opkg"
# Multiple targets are available, put it down to just one
target:
- core-image-base
-8
View File
@@ -1,8 +0,0 @@
header:
version: 14
local_conf_header:
tftf: |
TFA_UBOOT = "0"
TFA_UEFI = "0"
TFTF_TESTS = "1"
+4 -4
View File
@@ -1,7 +1,7 @@
header:
version: 14
version: 9
includes:
- ci/base.yml
- base.yml
# Target is arm64 and SDK is x86-64 to ensure that we exercise both
# architectures
@@ -11,10 +11,10 @@ local_conf_header:
toolchains: |
SDKMACHINE = "x86_64"
# No target armcompiler as currently there is no arm64 build
target:
- nativesdk-armcompiler
- gcc-aarch64-none-elf
- nativesdk-gcc-aarch64-none-elf
- gcc-arm-none-eabi
- nativesdk-gcc-arm-none-eabi
- gcc-arm-none-eabi-11.2
- nativesdk-gcc-arm-none-eabi-11.2
-8
View File
@@ -1,8 +0,0 @@
header:
version: 14
local_conf_header:
bootfirmware: |
PREFERRED_PROVIDER_virtual/bootloader = "u-boot"
TFA_UBOOT = "1"
TFA_UEFI = "0"
+3 -16
View File
@@ -4,7 +4,6 @@
import sys
import os
import shutil
import subprocess
import pathlib
@@ -23,6 +22,7 @@ repositories = (
"https://git.yoctoproject.org/git/poky",
"https://git.openembedded.org/meta-openembedded",
"https://git.yoctoproject.org/git/meta-virtualization",
"https://git.yoctoproject.org/git/meta-zephyr",
"https://github.com/kraj/meta-clang",
)
@@ -32,25 +32,12 @@ if __name__ == "__main__":
sys.exit(1)
base_repodir = pathlib.Path(os.environ["KAS_REPO_REF_DIR"])
failed = False
for repo in repositories:
repodir = base_repodir / repo_shortname(repo)
if "CI_CLEAN_REPOS" in os.environ:
print("Cleaning %s..." % repo)
shutil.rmtree(repodir, ignore_errors=True)
if repodir.exists():
try:
print("Updating %s..." % repo)
subprocess.run(["git", "-C", repodir, "-c", "gc.autoDetach=false", "fetch"], check=True)
except subprocess.CalledProcessError as e:
print(e)
failed = True
print("Updating %s..." % repo)
subprocess.run(["git", "-C", repodir, "fetch"], check=True)
else:
print("Cloning %s..." % repo)
subprocess.run(["git", "clone", "--bare", repo, repodir], check=True)
if failed:
sys.exit(128)
+2 -2
View File
@@ -1,7 +1,7 @@
header:
version: 14
version: 9
includes:
- ci/meta-virtualization.yml
- meta-virtualization.yml
local_conf_header:
meta-virt: |
-49
View File
@@ -1,49 +0,0 @@
# OEQA on Arm FVPs
OE-Core's [oeqa][OEQA] framework provides a method of performing runtime tests on machines using the `testimage` Yocto task. meta-arm has good support for writing test cases against [Arm FVPs][FVP], meaning the [runfvp][RUNFVP] boot configuration can be re-used.
Tests can be configured to run automatically post-build by setting the variable `TESTIMAGE_AUTO="1"`, e.g. in your Kas file or local.conf.
meta-arm provides the OEFVPTarget which must be set up in the machine configuration:
```
TEST_TARGET = "OEFVPTarget"
TEST_SERVER_IP = "127.0.0.1"
TEST_TARGET_IP = "127.0.0.1:2222"
IMAGE_FEATURES:append = " ssh-server-dropbear"
FVP_CONFIG[bp.virtio_net.hostbridge.userNetPorts] ?= "2222=22"
FVP_CONSOLES[default] = "terminal_0"
FVP_CONSOLES[tf-a] = "s_terminal_0"
```
The test target also generates a log file with the prefix 'fvp_log' in the image recipe's `${WORKDIR}/testimage` containing the FVP's stdout.
OEFVPTarget supports two different test interfaces - SSH and pexpect.
## SSH
As in OEQA in OE-core, tests cases can run commands on the machine using SSH. It therefore requires that an SSH server is installed in the image.
This uses the `run` method on the target, e.g:
```
(status, output) = self.target.run('uname -a')
```
which executes a single command on the target (using `ssh -c`) and returns the status code and the output. It is therefore useful for running tests in a Linux environment.
For examples of test cases, see meta/lib/oeqa/runtime/cases in OE-Core. The majority of test cases depend on `ssh.SSHTest.test_ssh`, which first validates that the SSH connection is functioning.
## pexpect
To support firmware and baremetal testing, OEFVPTarget also allows test cases to make assertions against one or more consoles using the pexpect library.
Internally, this test target launches a [Pexpect][PEXPECT] instance for each entry in FVP_CONSOLES which can be used with the provided alias. The whole Pexpect API is exposed on the target, where the alias is always passed as the first argument, e.g.:
```
self.target.expect('default', r'root@.*\:~#', timeout=30)
self.assertNotIn(b'ERROR:', self.target.before('tf-a'))
```
For an example of a full test case, see meta-arm/lib/oeqa/runtime/cases/linuxboot.py This test case can be used to minimally verify that a machine boots to a Linux shell. The default timeout is 10 minutes, but this can be configured with the variable TEST_FVP_LINUX_BOOT_TIMEOUT, which expects a value in seconds.
[OEQA]: https://docs.yoctoproject.org/test-manual/intro.html
[FVP]: https://developer.arm.com/tools-and-software/simulation-models/fixed-virtual-platforms
[RUNFVP]: runfvp.md
[PEXPECT]: https://pexpect.readthedocs.io/en/stable/overview.html
-131
View File
@@ -1,131 +0,0 @@
# Running Images with a FVP
The `runfvp` tool in meta-arm makes it easy to run Yocto Project disk images inside a [Fixed Virtual Platform (FVP)][FVP]. Some FVPs, such as the [Arm Architecture Models][AEM], are available free to download, but others need registration or are only available commercially. The `fvp-base` machine in meta-arm-bsp uses one of these AEM models.
## Running images with `runfvp`
To build images with the FVP integration, the `fvpboot` class needs to be inherited. If the machine does not do this explicitly it can be done in `local.conf`:
```
INHERIT += "fvpboot"
```
The class will download the correct FVP and write a `.fvpconf` configuration file when an image is built.
To run an image in a FVP, pass either a machine name or a `.fvpconf` path to `runfvp`.
```
$ ./meta-arm/scripts/runfvp tmp/deploy/images/fvp-base/core-image-minimal-fvp-base.fvpconf
```
When a machine name is passed, `runfvp` will start the latest image that has been built for that machine. This requires that the BitBake environment has been initialized (using `oe-init-build-env` or similar) as it will start BitBake to determine where the images are.
```
$ ./meta-arm/scripts/runfvp fvp-base
```
Note that currently meta-arm's `scripts` directory isn't in `PATH`, so a full path needs to be used.
`runfvp` will automatically start terminals connected to each of the serial ports that the machine specifies. This can be controlled by using the `--terminals` option, for example `--terminals=none` will mean no terminals are started, and `--terminals=tmux` will start the terminals in [`tmux`][tmux] sessions. Alternatively, passing `--console` will connect the serial port directly to the current session, without needing to open further windows.
The default terminal can also be configured by writing a [INI-style][INI] configuration file to `~/.config/runfvp.conf`:
```
[RunFVP]
Terminal=tmux
```
Arbitrary options can be passed directly to the FVP by specifying them after a double dash, for example this will list all of the FVP parameters:
```
$ runfvp fvp-base -- --list-params
```
## Configuring machines with `fvpboot`
To configure a machine so that it can be ran inside `runfvp`, a number of variables need to be set in the machine configuration file (such as `meta-arm-bsp/conf/machine/fvp-base.conf`).
Note that at present these variables are not stable and their behaviour may be changed in the future.
### `FVP_EXE`
The name of the FVP binary itself, for example `fvp-base` uses `FVP_Base_RevC-2xAEMvA`.
### `FVP_PROVIDER`
The name of the recipe that provides the FVP executable set in `FVP_EXE`, for example `fvp-base` uses `fvp-base-a-aem-native`. This *must* be a `-native` recipe as the binary will be executed on the build host.
There are recipes for common FVPs in meta-arm already, and writing new recipes is trivial. For FVPs which are free to download `fvp-base-a-aem.bb` is a good example. Some FVPs must be downloaded separately as they need an account on Arm's website.
If `FVP_PROVIDER` is not set then it is assumed that `FVP_EXE` is installed on the host already.
### `FVP_CONFIG`
Parameters passed to the FVP with the `--parameter`/`-C` option. These are expressed as variable flags so individual parameters can be altered easily. For example:
```
FVP_CONFIG[bp.flashloader0.fname] = "fip-fvp.bin"
```
### `FVP_DATA`
Specify raw data to load at the specified address, passed to the FVP with the `--data` option. This is a space-separated list of parameters in the format `[INST=]FILE@[MEMSPACE:]ADDRESS`. For example:
```
FVP_DATA = "cluster0.cpu0=Image@0x80080000 \
cluster0.cpu0=fvp-base-revc.dtb@0x83000000"
```
### `FVP_APPLICATIONS`
Applications to load on the cores, passed to the FVP with the `--application` option. These are expressed as variable flags with the flag name being the instance and flag value the filename, for example:
```
FVP_APPLICATIONS[cluster0] = "linux-system.axf"
```
Note that symbols are not allowed in flag names, so if you need to use a wildcard in the instance then you'll need to use `FVP_EXTRA_ARGS` and `--application` directly.
### `FVP_TERMINALS`
Map hardware serial ports to abstract names. For example the `FVP_Base_RevC-2xAEMvA` FVP exposes four serial ports, `terminal_0` to `terminal_3`. Typically only `terminal_0` is used in the `fvp-base` machine so this can be named `"Console"` and the others `""`. When runfvp starts terminals it will only start named serial ports, so instead of opening four windows where only one is useful, it will only open one.
For example:
```
FVP_TERMINALS[bp.terminal_0] = "Console"
FVP_TERMINALS[bp.terminal_1] = ""
FVP_TERMINALS[bp.terminal_2] = ""
FVP_TERMINALS[bp.terminal_3] = ""
```
### `FVP_CONSOLES`
This specifies what serial ports can be used in oeqa tests, along with an alias to be used in the test cases. Note that the values have to be the FVP identifier but without the board prefix, for example:
```
FVP_CONSOLES[default] = "terminal_0"
FVP_CONSOLES[tf-a] = "s_terminal_0"
```
The 'default' console is also used when `--console` is passed to runfvp.
### `FVP_EXTRA_ARGS`
Arbitrary extra arguments that are passed directly to the FVP. For example:
```
FVP_EXTRA_ARGS = "--simlimit 60"
```
### `FVP_ENV_PASSTHROUGH`
The FVP is launched with an isolated set of environment variables. Add the name of a Bitbake variable to this list to pass it through to the FVP environment. For example:
```
FVP_ENV_PASSTHROUGH = "ARMLMD_LICENSE_FILE FM_TRACE_PLUGINS"
```
[AEM]: https://developer.arm.com/tools-and-software/simulation-models/fixed-virtual-platforms/arm-ecosystem-models
[FVP]: https://developer.arm.com/tools-and-software/simulation-models/fixed-virtual-platforms
[tmux]: https://tmux.github.io/
[INI]: https://docs.python.org/3/library/configparser.html
-65
View File
@@ -1,65 +0,0 @@
# The Trusted Services: framework for developing root-of-trust services
meta-arm layer includes recipes for [Trusted Services][^1] Secure Partitions and Normal World applications
in `meta-arm/recipes-security/trusted-services`
## Secure Partitions recipes
We define dedicated recipes for all supported Trusted Services (TS) Secure Partitions.
These recipes produce ELF and DTB files for SPs.
These files are automatically included into optee-os image accordingly to defined MACHINE_FEATURES.
### How to include TS SPs
To include TS SPs into optee-os image you need to add into MACHINE_FEATURES
features for each [Secure Partition][^2] you would like to include:
| Secure Partition | MACHINE_FEATURE |
| ----------------- | --------------- |
| Attestation | ts-attesation |
| Crypto | ts-crypto |
| Internal Storage | ts-its |
| Protected Storage | ts-storage |
| se-proxy | ts-se-proxy |
| smm-gateway | ts-smm-gateway |
| spm-test[1-3] | optee-spmc-test |
Other steps depend on your machine/platform definition:
1. For communications between Secure and Normal Words Linux kernel option `CONFIG_ARM_FFA_TRANSPORT=y`
is required. If your platform doesn't include it already you can add `arm-ffa` into MACHINE_FEATURES.
(Please see ` meta-arm/recipes-kernel/arm-ffa-tee`.)
For running the `uefi-test` or the `xtest -t ffa_spmc` tests under Linux the `arm-ffa-user` drivel is required. This is
enabled if the `ts-smm-gateway` and/or the `optee-spmc-test` machine features are enabled.
(Please see ` meta-arm/recipes-kernel/arm-ffa-user`.)
2. optee-os might require platform specific OP-TEE build parameters (for example what SEL the SPM Core is implemented at).
You can find examples in `meta-arm/recipes-security/optee/optee-os_%.bbappend` for qemuarm64-secureboot machine
and in `meta-arm-bsp/recipes-security/optee/optee-os-n1sdp.inc` and `meta-arm-bsp/recipes-security/optee/optee-os-corstone1000-common.inc`
for N1SDP and Corstone1000 platforms accordingly.
3. trusted-firmware-a might require platform specific TF-A build parameters (SPD and SPMC details on the platform).
See `meta-arm/recipes-bsp/trusted-firmware-a/trusted-firmware-a_%.bbappend` for qemuarm64-secureboot machine
and in `meta-arm-bsp/recipes-bsp/trusted-firmware-a/trusted-firmware-a-n1sdp.inc` and
`meta-arm-bsp/recipes-bsp/trusted-firmware-a/trusted-firmware-a-corstone1000.inc` for N1SDP and Corstone1000 platforms.
## Normal World applications
Optionally for testing purposes you can add `packagegroup-ts-tests` into your image. It includes
[Trusted Services test and demo tools][^3] and [xtest][^4] configured to include the `ffa_spmc` tests.
## OEQA Trusted Services tests
meta-arm also includes Trusted Service OEQA tests which can be used for automated testing.
See `ci/trusted-services.yml` for an example how to include them into an image.
------
[^1]: https://trusted-services.readthedocs.io/en/integration/overview/index.html
[^2]: https://trusted-services.readthedocs.io/en/integration/deployments/secure-partitions.html
[^3]: https://trusted-services.readthedocs.io/en/integration/deployments/test-executables.html
[^4]: https://optee.readthedocs.io/en/latest/building/gits/optee_test.html
-12
View File
@@ -1,12 +0,0 @@
header:
version: 13
repos:
meta-arm:
layers:
meta-arm-systemready:
distro: nodistro
target:
- arm-systemready-firmware
-12
View File
@@ -1,12 +0,0 @@
header:
version: 13
includes:
- kas/arm-systemready-firmware.yml
env:
TESTIMAGE_AUTO: "1"
# The full testimage run typically takes around 12-24h on fvp-base.
TEST_OVERALL_TIMEOUT: "${@ 24*60*60}"
target:
- arm-systemready-ir-acs
@@ -1,7 +0,0 @@
header:
version: 13
includes:
- kas/arm-systemready-firmware.yml
target:
- arm-systemready-linux-distros-debian
@@ -1,7 +0,0 @@
header:
version: 13
includes:
- kas/arm-systemready-firmware.yml
target:
- arm-systemready-linux-distros-opensuse
-54
View File
@@ -1,54 +0,0 @@
header:
version: 14
env:
DISPLAY: ""
distro: poky-tiny
defaults:
repos:
branch: nanbield
repos:
meta-arm:
layers:
meta-arm:
meta-arm-bsp:
meta-arm-toolchain:
poky:
url: https://git.yoctoproject.org/git/poky
commit: 2e9c2a2381105f1306bcbcb54816cbc5d8110eff
layers:
meta:
meta-poky:
meta-openembedded:
url: https://git.openembedded.org/meta-openembedded
commit: 1750c66ae8e4268c472c0b2b94748a59d6ef866d
layers:
meta-oe:
meta-python:
meta-perl:
meta-secure-core:
url: https://github.com/wind-river/meta-secure-core.git
commit: e29165a1031dcf601edbed1733cedd64826672a5
layers:
meta:
meta-signing-key:
meta-efi-secure-boot:
local_conf_header:
base: |
CONF_VERSION = "2"
PACKAGE_CLASSES = "package_ipk"
BB_NUMBER_THREADS ?= "16"
PARALLEL_MAKE ?= "-j16"
PACKAGECONFIG:append:pn-perf = " coresight"
machine: unset
target:
- corstone1000-image
-16
View File
@@ -1,16 +0,0 @@
header:
version: 14
includes:
- kas/corstone1000-base.yml
- kas/fvp-eula.yml
machine: corstone1000-fvp
local_conf_header:
fvp-config: |
# Remove Dropbear SSH as it will not fit into the corstone1000 image.
IMAGE_FEATURES:remove = " ssh-server-dropbear"
INHERIT += "fvpboot"
target:
- corstone1000-image
-6
View File
@@ -1,6 +0,0 @@
header:
version: 14
includes:
- kas/corstone1000-base.yml
machine: corstone1000-mps3
-43
View File
@@ -1,43 +0,0 @@
header:
version: 13
includes:
- kas/fvp-eula.yml
env:
DISPLAY:
WAYLAND_DISPLAY:
XAUTHORITY:
distro: poky
machine: fvp-base
defaults:
repos:
refspec: master
repos:
meta-arm:
layers:
meta-arm:
meta-arm-bsp:
meta-arm-toolchain:
poky:
url: https://git.yoctoproject.org/git/poky
path: layers/poky
layers:
meta:
meta-poky:
local_conf_header:
base: |
CONF_VERSION = "2"
PACKAGE_CLASSES = "package_ipk"
PACKAGECONFIG:remove:pn-qemu-system-native = "gtk+ sdl"
EXTRA_IMAGE_FEATURES:append = " debug-tweaks ssh-server-openssh"
CORE_IMAGE_EXTRA_INSTALL:append = " ssh-pregen-hostkeys"
IMAGE_CLASSES:append = " testimage fvpboot"
target:
- core-image-minimal
+41
View File
@@ -0,0 +1,41 @@
header:
version: 9
distro: poky
machine: fvp-baser-aemv8r64
defaults:
repos:
refspec: honister
repos:
meta-arm:
url: https://git.yoctoproject.org/git/meta-arm
path: layers/meta-arm
layers:
meta-arm:
meta-arm-bsp:
meta-arm-toolchain:
poky:
url: https://git.yoctoproject.org/git/poky
path: layers/poky
layers:
meta:
meta-poky:
env:
FVP_BASE_R_AEM_TARBALL_URI: ""
FVP_BASE_R_ARM_EULA_ACCEPT: "False"
local_conf_header:
base: |
CONF_VERSION = "2"
PACKAGE_CLASSES = "package_ipk"
PACKAGECONFIG:remove:pn-qemu-system-native = "gtk+ sdl"
EXTRA_IMAGE_FEATURES:append = " debug-tweaks"
IMAGE_CLASSES:append = " ${@oe.utils.ifelse(d.getVar('FVP_BASE_R_AEM_TARBALL_URI'), 'fvpboot', '')}"
LICENSE_FLAGS_WHITELIST:append = " ${@oe.utils.vartrue('FVP_BASE_R_ARM_EULA_ACCEPT', 'Arm-FVP-EULA', '', d)}"
target:
- core-image-minimal
@@ -1,6 +1,8 @@
header:
version: 14
version: 9
includes:
- fvp-baser-aemv8r64-bsp.yml
local_conf_header:
kernel: |
base-rt: |
PREFERRED_PROVIDER_virtual/kernel = "linux-yocto-rt"
-5
View File
@@ -1,5 +0,0 @@
header:
version: 13
env:
ARM_FVP_EULA_ACCEPT:
-7
View File
@@ -1,7 +0,0 @@
header:
version: 9
local_conf_header:
tftf: |
TFA_UBOOT = "0"
TFTF_TESTS = "1"
+93
View File
@@ -0,0 +1,93 @@
meta-arm-autonomy Yocto Layer
=============================
Introduction
------------
This layer provides an hypervisor based solution (currently based on Xen) for
autonomous system. It contains recipes and classes to build host and guest
images.
To start using this layer, please check the
[Quick Start Guide](documentation/arm-autonomy-quickstart.md).
Dependencies
------------
This layer depends on several other Yocto layers:
* meta-openembedded (https://git.openembedded.org/meta-openembedded)
* poky (https://git.yoctoproject.org/poky)
* meta-virtualization (https://git.yoctoproject.org/meta-virtualization)
* meta-networking (git://git.openembedded.org/meta-openembedded)
Distribution Features
---------------------
This layer adds the following Yocto DISTRO_FEATURES:
* arm-autonomy-host: this feature activates functionalities required to build
an autonomy host system. It has the following effects:
- add 'xen' and 'ipv4' to DISTRO_FEATURES.
- add xen backend drivers to linux kernel configuration.
- To reduce the root filesystem image size the kernel image is not installed.
* arm-autonomy-guest: this feature activates functionalities to run as guest
of an autonomy system. It is doing the following:
- add 'ipv4' to DISTRO_FEATURES.
- add xen frontend drivers to linux kernel configuration.
- add console on hvc0 during init.
Bitbake variables
-----------------
Some recipes and classes in this layer are introducing variables which can be
modified by the user in local.conf.
Each recipe introducing such variables has a chapter "Bitbake parameters" in
its documentation.
Those documentation files should be checked for variables:
- [xen-devicetree](documentation/xen-devicetree.md)
- [xenguest-manager](documentation/xenguest-manager.md)
- [xenguest-network](documentation/xenguest-network.md)
BSPs
----
This layer adds the following machine:
* arm64-autonomy-guest: This machine creates a minimal BSP suitable to be used
as an autonomy guest. It is in fact only activating ARM64 architecture and
SMP in the linux kernel and is enabling the DISTRO_FEATURE
'arm-autonomy-guest'.
Images
------
This layer is adding the following images:
* arm-autonomy-host-image-minimal: This image includes all elements required
to create a minimal arm-autonomy-host system. This includes xen, and tools to
manage xen guests and xenguest images. This image depends on
'arm-autonomy-host' distribution feature.
Recipes and classes
-------------------
This layer adds the following recipes and classes:
* [xen-devicetree](documentation/xen-devicetree.md): This is a recipe to modify
a device tree blob to add information required to boot xen and a Dom0 linux.
* [xenguest-mkimage](documentation/xenguest-mkimage.md): This is a tool used to
create and modify images to be used as Xen guests.
* [xenguest-manager](documentation/xenguest-manager.md): This is a tool used to
create/remove/start/stop xen guest generated using xenguest-mkimage.
* [xenguest-network](documentation/xenguest-network.md): This
recipe add tools and init scripts to create a bridge connected to the
external network on the host and allow guests to be connected to it.
Contributing
------------
This project has not put in place a process for contributions currently. If you
would like to contribute, please contact the maintainers
Maintainer(s)
-------------
* Diego Sueiro <diego.sueiro@arm.com>
* Bertrand Marquis <bertrand.marquis@arm.com>
@@ -0,0 +1,26 @@
# This class is to be inherited by image recipes that want to build and install
# an alternate kernel (set via PREFERRED_PROVIDER_alternate/kernel).
#
# It is mandatory to also set the KERNEL_PACKAGE_NAME for the alternate kernel
# recipe via KERNEL_PACKAGE_NAME:pn-${PREFERRED_PROVIDER_alternate/kernel} and
# its value needs to be different from "kernel" since this is the default set
# for PREFERRED_PROVIDER_virtual/kernel.
#
# An example of these settings can be found at meta-arm-autonomy/dynamic-layers/meta-arm-bsp/conf/machine/n1sdp-extra-settings.inc
#
# When building and installing an alternate kernel, the kernel-modules packages
# for both virtual/kernel and alternate/kernel will be installed.
PREFERRED_PROVIDER_alternate/kernel ??= ""
python () {
alternate_kernel = d.getVar('PREFERRED_PROVIDER_alternate/kernel')
if alternate_kernel:
alternate_kernel_pkg_name = d.getVar('KERNEL_PACKAGE_NAME:pn-%s' % alternate_kernel)
if alternate_kernel_pkg_name:
d.appendVar('EXTRA_IMAGEDEPENDS', ' ' + alternate_kernel)
d.appendVar('IMAGE_INSTALL', ' kernel-modules')
d.appendVar('IMAGE_INSTALL', ' ' + alternate_kernel_pkg_name + '-modules')
else:
raise bb.parse.SkipRecipe("No KERNEL_PACKAGE_NAME:pn-%s set" % alternate_kernel )
}
@@ -0,0 +1,5 @@
# Include arm-autonomy distro config files if the distro features are set
require ${@bb.utils.contains('DISTRO_FEATURES', 'arm-autonomy-host', '${ARM_AUTONOMY_DISTRO_CFGDIR}/arm-autonomy-host.inc', '', d)}
require ${@bb.utils.contains('DISTRO_FEATURES', 'arm-autonomy-guest', '${ARM_AUTONOMY_DISTRO_CFGDIR}/arm-autonomy-guest.inc', '', d)}
@@ -0,0 +1,115 @@
# Docker Extern Containers
#
# This class allows docker image tarballs to be installed in the rootfs
#
# The images can be selected using the variable CONTAINER_IMAGE_FILES which
# should contain a space seperated list of absolute paths or yocto urls for
# docker images that have been exported using docker export:
# - https://docs.docker.com/engine/reference/commandline/export/
#
# src_uri_parse_var.bbclass is used to parse CONTAINER_IMAGE_FILES
#
# There are 4 supported formats for CONTAINER_IMAGE_FILES entries:
#
# - http/https url
# - CONTAINER_IMAGE_FILES = "https://[url]:[port]/alpine.tar;md5sum=..."
#
# - file:// absolute local path from root
# - CONTAINER_IMAGE_FILES = "file:///containers/alpine2.tar"
#
# - file:// path relative to FILESEXTRAPATHS
# - CONTAINER_IMAGE_FILES = "file://foo/alpine3.tar"
# FILESEXTRAPATHS .= "/containers:"
#
# - plain absolute local path from root
# - CONTAINER_IMAGE_FILES = "/containers/foo/bar/alpine4.tar"
#
# It is not recommended to use other yocto URL types, as they may result in
# undefined behaviour.
#
# A semicolon seperated list of install arguments can follow each image path:
# - conname : the name that will be attached when the image is imported
# (default: [filename, without extension])
# - contag : the tag that will be attached when the image is imported
# (default: local)
# - conkeep : Flag for whether the exported container image file should be
# kept once the import has been completed
# (default: 0)
#
# Any other arguments, for example an md5sum, will be assumed to be fetch
# arguments, and will be kept when the path is added to the SRC_URI
#
# e.g. CONTAINER_IMAGE_FILES = "\
# https://[url]:[port]/alpine.tar;md5sum=[checksum];conkeep=1 \
# file:///containers/alpine2.tar;contag=latest;conname=docker2 \
# file://foo/alpine3.tar \
# /containers/foo/bar/alpine4.tar;contag=1.0;conkeep=1 "
#
# Resulting Manifest:
# ARCHIVE NAME TAG KEEP
# alpine.tar alpine local 1
# alpine2.tar docker2 latest 0
# alpine3.tar alpine3 local 0
# alpine4.tar alpine4 1.0 1
#
# Other configurable variables:
# CONTAINERS_INSTALL_DIR : The folder underneath ${WORKDIR} where the docker
# images will be stored
# (default: "/usr/share/docker/images")
# CONTAINERS_MANIFEST : The name of the manifest file containing image
# parameters, also stored in CONTAINERS_INSTALL_DIR
# (default: "containers.manifest")
# CONTAINERS_TAG_DEFAULT : Use this to change the value that will be used as
# contag if no value is provided
# (default: "local")
# CONTAINERS_KEEP_DEFAULT : Use this to change the value that will be used for
# conkeep if no value is provided
# (default: "0")
#
inherit features_check
REQUIRED_DISTRO_FEATURES = "docker"
RDEPENDS:${PN} = "packagegroup-docker-runtime-minimal"
CONTAINER_IMAGE_FILES ??= ""
CONTAINERS_INSTALL_DIR ??= "${datadir}/docker/images"
CONTAINERS_MANIFEST ??= "containers.manifest"
CONTAINERS_TAG_DEFAULT ??= "local"
CONTAINERS_KEEP_DEFAULT ??= "0"
inherit set_src_uri_from_var
SRC_URI_FROM_VAR_NAME = "CONTAINER_IMAGE_FILES"
# Define installation params
SRC_URI_FROM_VAR_MANIFEST_PARAMS = "conname=[basename] \
contag=${CONTAINERS_TAG_DEFAULT} conkeep=${CONTAINERS_KEEP_DEFAULT}"
SRC_URI_FROM_VAR_UNPACK_DIR = "containers"
# Read manifest and install container images
do_install() {
local archive name tag keep
if [ -f "${WORKDIR}/${SRC_URI_FROM_VAR_UNPACK_DIR}/manifest" ]; then
install -d "${D}${CONTAINERS_INSTALL_DIR}"
install -m 644 \
"${WORKDIR}/${SRC_URI_FROM_VAR_UNPACK_DIR}/manifest" \
"${D}${CONTAINERS_INSTALL_DIR}/${CONTAINERS_MANIFEST}"
while read -r archive name tag keep _; do
[ -f "${WORKDIR}/${SRC_URI_FROM_VAR_UNPACK_DIR}/${archive}" ] ||
bbfatal "${archive} does not exist"
install -m 644 \
"${WORKDIR}/${SRC_URI_FROM_VAR_UNPACK_DIR}/${archive}" \
"${D}${CONTAINERS_INSTALL_DIR}/${archive}"
done < "${D}${CONTAINERS_INSTALL_DIR}/${CONTAINERS_MANIFEST}"
fi
}
do_install[vardeps] += "CONTAINER_IMAGE_FILES"
FILES:${PN} += "${CONTAINERS_INSTALL_DIR}"
@@ -0,0 +1,169 @@
# Create a xenguest image with kernel and filesystem produced by Yocto
# This will create a .xenguest file that the xenguest-manager can use.
inherit xenguest_image
# We are creating our guest in a local subdirectory
# force the value so that we are not impacted if the user is changing it
XENGUEST_IMAGE_DEPLOY_DIR = "${WORKDIR}/tmp-xenguest"
# Name of deployed file (keep standard image name and add .xenguest)
XENGUEST_IMAGE_DEPLOY ??= "${IMAGE_NAME}"
# Add kernel XENGUEST_IMAGE_KERNEL from DEPLOY_DIR_IMAGE to image
xenguest_image_add_kernel() {
srcfile="${1:-}"
if [ -z "${srcfile}" ]; then
srcfile="${DEPLOY_DIR_IMAGE}/${XENGUEST_IMAGE_KERNEL}"
fi
call_xenguest_mkimage partial --xen-kernel=$srcfile
}
# Add rootfs file to the image
xenguest_image_add_rootfs() {
call_xenguest_mkimage partial \
--disk-add-file=${IMGDEPLOYDIR}/${IMAGE_LINK_NAME}.${IMAGE_TYPEDEP:xenguest}:rootfs.${IMAGE_TYPEDEP:xenguest}
}
# Pack xenguest image
xenguest_image_pack() {
mkdir -p ${IMGDEPLOYDIR}
rm -f ${IMGDEPLOYDIR}/${XENGUEST_IMAGE_DEPLOY}.xenguest
call_xenguest_mkimage pack \
${IMGDEPLOYDIR}/${XENGUEST_IMAGE_DEPLOY}.xenguest
}
#
# Task finishing the bootimg
# We need this task to actually create the symlinks
#
python do_bootimg_xenguest() {
subtasks = d.getVarFlag('do_bootimg_xenguest', 'subtasks')
bb.build.exec_func('xenguest_image_clone', d)
if subtasks:
for tk in subtasks.split():
bb.build.exec_func(tk, d)
bb.build.exec_func('xenguest_image_pack', d)
bb.build.exec_func('create_symlinks', d)
}
# This is used to add sub-tasks to do_bootimg_xenguest
do_bootimg_xenguest[subtasks] = ""
# Those are required by create_symlinks to find our image
do_bootimg_xenguest[subimages] = "xenguest"
do_bootimg_xenguest[imgsuffix] = "."
do_bootimg_xenguest[depends] += "xenguest-base-image:do_deploy"
# Need to have rootfs so all recipes have deployed their content
do_bootimg_xenguest[depends] += "${PN}:do_rootfs"
# This set in python anonymous after, just set a default value here
IMAGE_TYPEDEP:xenguest ?= "tar"
# We must not be built at rootfs build time because we need the kernel
IMAGE_TYPES_MASKED += "xenguest"
IMAGE_TYPES += "xenguest"
XENGUEST_IMAGE_RECIPE = "${PN}"
XENGUEST_IMAGE_VARS += "XENGUEST_IMAGE_RECIPE"
# Merge intermediate env files from all recipes into a single file
python do_merge_xenguestenv () {
import re
# Open final merged file in DEPLOY_DIR_IMAGE for writing, or create
outdir = d.getVar('DEPLOY_DIR_IMAGE')
with open(os.path.join(outdir,'xenguest.env'), 'w') as merged_file:
# Adds vars from xenguest_image to list
merged_env = []
xenguest_vars = d.getVar('XENGUEST_IMAGE_VARS')
for var in xenguest_vars.split():
value = d.getVar(var)
if value:
merged_env.append(var + "=" + " ".join(value.split()) + "\n")
# Resolve dependencies for this task to find names of intermediate
# .xenguestenv files
taskdepdata = d.getVar('BB_TASKDEPDATA')
task_mc = d.getVar('BB_CURRENT_MC')
task_file = d.getVar('FILE')
# See runqueue.py function build_taskdepdata
DEPS_INDEX = 3
depdata_key = task_file + ":do_merge_xenguestenv"
# If in a multiconfig, need to add that to the key
if task_mc != "default":
depdata_key = "mc:" + task_mc + ":" + depdata_key
# Retrieve filename using regex
get_filename = re.compile(r'/([^/]+\.bb):do_deploy_xenguestenv$')
env_dir = d.getVar('XENGUEST_ENV_STAGING_DIR')
for task_dep in taskdepdata[depdata_key][DEPS_INDEX]:
if task_dep.endswith(":do_deploy_xenguestenv"):
filename = re.search(get_filename, task_dep).group(1) + ".xenguestenv"
bb.note("Merging: " + filename)
try:
with open(env_dir + "/" + filename, 'r') as f:
# Eliminate duplicates
merged_env = list(set(merged_env + f.readlines()))
except (FileNotFoundError, IOError):
bb.note(" " + filename + " has no extra vars")
# Sort Alphabetically and write
merged_env.sort()
merged_file.write("".join(merged_env))
}
do_merge_xenguestenv[dirs] = "${DEPLOY_DIR_IMAGE}"
do_merge_xenguestenv[vardeps] += "${XENGUEST_IMAGE_VARS}"
do_merge_xenguestenv[vardepsexclude] += "BB_TASKDEPDATA"
do_merge_xenguestenv[recrdeptask] += "do_deploy_xenguestenv"
addtask merge_xenguestenv before do_populate_lic_deploy after do_image_complete
python __anonymous() {
# Do not do anything if we are not in the want FSTYPES
if bb.utils.contains_any('IMAGE_FSTYPES', 'xenguest', '1', '0', d):
# Check the coherency of the configuration
rootfs_needed = False
rootfs_file = ''
kernel_needed = False
rootfs_file = xenguest_image_rootfs_file(d)
if rootfs_file:
rootfs_needed = True
if d.getVar('XENGUEST_IMAGE_KERNEL') and not d.getVar('INITRAMFS_IMAGE'):
# If INITRAMFS_IMAGE is set, even if INITRAMFS_IMAGE_BUNDLE is not
# set to 1 to bundle the initramfs with the kernel, kernel.bbclass
# is setting a dependency on ${PN}:do_image_complete. We cannot
# in this case depend on do_deploy as it would create a circular
# dependency:
# do_image_complete would depend on kernel:do_deploy which would
# depend on ${PN}:do_image_complete
# In the case INITRAMFS_IMAGE_BUNDLE = 1, the kernel-xenguest class
# will handle the creation of a xenguest image with the kernel.
# In the other case the kernel can be added manually to the image.
kernel_needed = True
bb.build.addtask('do_bootimg_xenguest', 'do_image_complete', None, d)
if rootfs_needed:
# tell do_bootimg_xenguest to call add_rootfs
d.appendVarFlag('do_bootimg_xenguest', 'subtasks', ' xenguest_image_add_rootfs')
# do_bootimg_xenguest will need the tar file
d.appendVarFlag('do_bootimg_xenguest', 'depends', ' %s:do_image_tar' % (d.getVar('PN')))
# set our TYPEDEP to the proper compression
d.setVar('IMAGE_TYPEDEP:xenguest', 'tar' + (rootfs_file.split('.tar', 1)[1] or ''))
if kernel_needed:
# Tell do_bootimg_xenguest to call xenguest_image_add_kernel
d.appendVarFlag('do_bootimg_xenguest', 'subtasks', ' xenguest_image_add_kernel')
# we will need kernel do_deploy
d.appendVarFlag('do_bootimg_xenguest', 'depends', ' virtual/kernel:do_deploy')
}
@@ -0,0 +1,37 @@
# Create a xenguest image containing the kernel with initramfs when
# initramfs is activated
# This is done using kernel-fitimage as model
# To activate this, kernel-xenguest must be added to KERNEL_CLASSES
# Add a variable name to XENGUEST_IMAGE_VARS_EXTRA if you want it to
# appear in xenguest.env when the image is deployed
inherit xenguest_image
# use a local copy to pack all together
XENGUEST_IMAGE_DEPLOY_DIR = "${WORKDIR}/tmp-xenguest"
python __anonymous () {
# only if xenguest image type is present
if bb.utils.contains('IMAGE_FSTYPES', 'xenguest', '1', '0', d):
# only if initramfs bundle is activated
if d.getVar('INITRAMFS_IMAGE') and d.getVar('INITRAMFS_IMAGE_BUNDLE') == "1":
if not bb.utils.contains('KERNEL_IMAGETYPES', 'Image', '1', '0', d):
bb.fatal("xenguest image type with initramfs require Image kernel image type")
bb.build.addtask('do_assemble_xenguest_initramfs', 'do_deploy', 'do_bundle_initramfs', d)
}
do_assemble_xenguest_initramfs() {
xenguest_image_clone
call_xenguest_mkimage partial --xen-kernel=${B}/${KERNEL_OUTPUT_DIR}/Image.initramfs
rm -f ${B}/${KERNEL_OUTPUT_DIR}/Image-initramfs.xenguest
call_xenguest_mkimage pack ${B}/${KERNEL_OUTPUT_DIR}/Image-initramfs.xenguest
}
do_assemble_xenguest_initramfs[depends] += "${INITRAMFS_IMAGE}:do_merge_xenguestenv"
kernel_do_deploy:append() {
if [ -f "${B}/${KERNEL_OUTPUT_DIR}/Image-initramfs.xenguest" ]; then
install -m 0644 ${B}/${KERNEL_OUTPUT_DIR}/Image-initramfs.xenguest "$deployDir/Image-${INITRAMFS_NAME}.xenguest"
ln -snf Image-${INITRAMFS_NAME}.xenguest $deployDir/Image-${INITRAMFS_LINK_NAME}.xenguest
fi
}
@@ -0,0 +1,224 @@
# Set SRC_URI from Variable
# This class parses a variable named in SRC_URI_FROM_VAR_NAME for entries that
# should be added to the SRC_URI
#
# There are 4 supported formats for entries:
#
# - http/https url
# - Note that a checksum (md5sum or sha256sum) must be provided for http(s)
#
# - file:// absolute local path from root
# - In this case the filename will be added to SRC_URI and the path from '/'
# added to FILESEXTRAPATHS
#
# - file:// path relative to FILESEXTRAPATHS
# - In this case the filename will be added to SRC_URI and the preceding path
# added to FILESOVERRIDES, so that the full path to the file will
# be available in FILESPATH when it is generated by combining
# FILESEXTRAPATHS and FILESOVERRIDES.
#
# - plain absolute local path from root
# - This will be treated the same as an file:// path from root. Plain paths
# must be absolute, and cannot be relative to FILESEXTRAPATHS
#
# It is not recommended to use other yocto URL types, as they may result in
# undefined behaviour.
#
# These entries will be added to the SRC_URI so that the yocto fetcher can
# unpack a copy into ${WORKDIR}/${SRC_URI_FROM_VAR_UNPACK_DIR}
#
#
# A list of arguments can follow each entry in the input variable, seperated
# by semi-colons (;). Arguments may be FETCH arguments or MANIFEST arguments.
#
# FETCH arguments will be appended to the entry in SRC_URI, for example
# "downloadfilename" to specify the filename used when storing a
# downloaded file.
# Each SRC_URI entry will automatically have the arguments
# "unpack=0;subdir=${SRC_URI_FROM_VAR_UNPACK_DIR}" added to them, so do not
# attempt to set these options.
#
# MANIFEST arguments are defined in the variable
# SRC_URI_FROM_VAR_MANIFEST_PARAMS which should be a space seperated list of
# names, each optionally followed by an equals sign (=) and a default value.
#
# The values provided for the manifest arguments will be written to the manifest
# file in ${WORKDIR}/${SRC_URI_FROM_VAR_UNPACK_DIR} as columns, in the same
# order as they appear in SRC_URI_FROM_VAR_MANIFEST_PARAMS.
#
# For entries that do not provide a value for a manifest argument, the default
# value will be used if possible.
# If no default is availale, omitting the parameter on any item will cause
# an error.
#
# "[basename]" is a special case default that will set the value to
# the filename without the path or file extension.
#
# e.g.
# SRC_URI_FROM_VAR_MANIFEST_PARAMS="conname=[basename] contag=local conkeep"
#
# Any arguments that follow an entry in SRC_URI_FROM_VAR_NAME, that are not
# named in SRC_URI_FROM_VAR_MANIFEST_PARAMS are assumed to be FETCH arguments,
# so are added to the corresponding entry in the SRC_URI.
SRC_URI_FROM_VAR_NAME ??= ""
SRC_URI_FROM_VAR_MANIFEST_PARAMS ??= ""
SRC_URI_FROM_VAR_UNPACK_DIR ??= "items"
python __anonymous() {
parse_var = d.getVar('SRC_URI_FROM_VAR_NAME')
if not parse_var:
return
parse_var_items = d.getVar(parse_var)
if parse_var_items:
for item in parse_var_items.split(' '):
if not item:
continue
if item.startswith('/'):
# If not a Yocto URL, must be an absolute path
yocto_url = "file://" + item
else:
# Otherwise assume valid Yocto URL.
# Error case is caught later
yocto_url = item
fetcher = host = path = parm = None
try:
# Attempt to parse a Yocto URL
fetcher,host,path,_,_,parm = bb.fetch.decodeurl(yocto_url)
except:
# Something invalid is in the variable!
raise bb.parse.SkipRecipe(parse_var + \
" contains an invalid entry:\n'" + \
item + "'")
# This var is space seperated list of parameter names,
# with optional default value following an equals sign
# (name=default)
item_params_str = d.getVar('SRC_URI_FROM_VAR_MANIFEST_PARAMS')
# remove directories from path
filename = os.path.basename(path)
if "downloadfilename" in parm:
filename = parm["downloadfilename"]
item_manifest_args = {"filename": filename}
if item_params_str:
# required manifest arguments have been provided
# If no default is given add "=" for map parsing
item_params_list = [ arg + "=" if '=' not in arg
else arg
for arg in item_params_str.split(' ')
]
# Generate key value pairs of argument names and
# default values
item_params_map = dict( (name.strip(), val.strip())
for name, val in (arg.split('=')
for arg in item_params_list)
)
for argname in item_params_map:
# Iterate over required manifest arguments
argvalue = parm.pop(argname, None)
if argvalue:
# a value has been provided for this item
item_manifest_args[argname] = argvalue
else:
# No value provided, process default value
default = item_params_map[argname]
if default:
# A default value is provided
if default == "[basename]":
# use the filename without extension
default = os.path.splitext(filename)[0]
# store default value in dict
item_manifest_args[argname] = default
else:
# No default provided
raise bb.fatal(parse_var + \
" entry is missing a required parameter '" + \
argname + "':\n'" + item + "'")
# Write value to var flags to ensure data structure is preserved
# Each entry of parse_var will have a varflag where the value
# is a dictionary of argument names and values
d.setVarFlags(parse_var, {item: item_manifest_args})
src_uri_entry_suffix = ';'
# HTTP(S) fetcher must provide a checksum
if fetcher.startswith('http') and not \
( 'md5sum' in parm or 'sha256sum' in parm ):
# Ensure http/https fetchers get a checksum
raise bb.parse.SkipRecipe(parse_var + \
" entry is missing a checksum:\n'" + \
item + "'")
# add remaining fetch parameters including checksum
for arg in parm:
src_uri_entry_suffix += ";" + arg + "=" + parm[arg]
# Add default and extra parameters to SRC_URI entry
src_uri_entry_suffix += ';unpack=0;subdir=' + \
d.getVar('SRC_URI_FROM_VAR_UNPACK_DIR')
if fetcher == 'file':
# Prevent local fetcher from re-creating dir structure
filedir = os.path.split(path)[0]
if filedir.startswith('/'):
# Path is from the root
d.appendVar('FILESEXTRAPATHS', filedir + ':')
else:
# Path is relative to FILESEXTRAPATHS
d.appendVar('FILESOVERRIDES', ':' + filedir)
# Add filename without path to SRC_URI
d.appendVar('SRC_URI', ' file://' + \
filename + src_uri_entry_suffix)
else:
# Add full entry to SRC_URI
d.appendVar('SRC_URI', ' ' + fetcher + \
"://" + host + path + src_uri_entry_suffix)
}
python generate_manifest() {
parse_var = d.getVar('SRC_URI_FROM_VAR_NAME')
if not parse_var:
return
target_dir = os.path.join(d.getVar('WORKDIR'),
d.getVar('SRC_URI_FROM_VAR_UNPACK_DIR'))
# Write a manifest file containing the parameters so SRC_URI
# doesn't need to be parsed by do_install
with open (target_dir + "/manifest", 'w') as manifest_file:
manifest_args = d.getVarFlags(parse_var)
parse_var_items = d.getVar(parse_var)
if parse_var_items:
for item in parse_var_items.split():
manifest_file.write(" ".join(manifest_args[item].values())+"\n")
}
do_unpack[cleandirs] += "${WORKDIR}/${SRC_URI_FROM_VAR_UNPACK_DIR}"
do_unpack[postfuncs] += "generate_manifest"
do_unpack[vardeps] += "${SRC_URI_FROM_VAR_NAME}"
@@ -0,0 +1,251 @@
# This class must be used to create, extend or pack a xenguest image.
# It is using xenguest-mkimage tool to do operations
DEPENDS += "xenguest-mkimage-native"
#
# Xenguest image parameters
# All the following parameters can be modified in local.conf or on recipes
# inheriting this class
#
# Guest memory size in MB
XENGUEST_IMAGE_MEMORY_SIZE ??= "512"
# Guest number of vcpus
XENGUEST_IMAGE_NUM_VCPUS ??= "1"
# Guest auto boot during init, set to 1 to have guest started during init or
# to 0 if the guest should not be auto started
XENGUEST_IMAGE_AUTOBOOT ??= "1"
# Partition containing the root file system
# Xen will actually add root=${XENGUEST_IMAGE_ROOT} to your guest kernel
# command line
# You can let this empty if the root filesystem is specified in an other way
# and have root= option added to the command line for example or if you don't
# need a root filesystem mounted for your guest (initrd for example)
XENGUEST_IMAGE_ROOT ??= "/dev/xvda1"
# Guest kernel command line arguments
XENGUEST_IMAGE_CMDLINE ??= "earlyprintk=xenboot console=hvc0 rw"
# Extra commands to add to xenguest_image when creating the image
XENGUEST_IMAGE_EXTRA_CMD ??= ""
# Kernel binary
# This value is used by the xenguest image type but is declared here to have
# all variables in the same place
# If this value is empty no kernel will be added to the image
XENGUEST_IMAGE_KERNEL ??= "Image"
# Size of the disk to create (if 0 no disk will be created and rootfs will not
# be included in the xenguest image)
XENGUEST_IMAGE_DISK_SIZE ??= "${@ '4' if not d.getVar('INITRAMFS_IMAGE') else '0'}"
# set empty partition to be used by xenguest-manager for this image
XENGUEST_IMAGE_DISK_DEVICE ??= ""
#
# XENGUEST_IMAGE_DISK_PARTITIONS is used to describe the partitions to setup
# and their content.
# It must be set to a space separated list of entries with each entry having
# the format num:sz:[fs]:[file] where:
# - num is a partition number
# - sz is the partition size in MB or GB(default), e.g 1000M or 1[G]
# - fs is optional filesystem to use for the partition
# - file is optionally pointing to a file to use as content of the partition
# Please check image_types_xenguest.bbclass for rootfs handling of files
#
# Default value creates a partition 1 using the full disk, formated with ext4
# and containing the root filesystem produced by Yocto
XENGUEST_IMAGE_DISK_PARTITIONS ??= "1:${XENGUEST_IMAGE_DISK_SIZE}:ext4:rootfs.tar.gz"
# XENGUEST_IMAGE_NETWORK_TYPE can be set to "bridge", "nat" or "none".
# The "bridge" type will share the physical eth interface from dom0 with the
# domU. This will allow the domU to have access to the external network.
# The "nat" type will setup a virtual network between dom0 and domU and also
# configure and run the kea dhcp4 server on dom0 to serve the domU.
# The "none" type will not affect any networking setting between on dom0 and
# domU.
XENGUEST_IMAGE_NETWORK_TYPE ??= "bridge"
# Sub-directory in wich the guest is created. This is create in deploy as a
# subdirectory and must be coherent between all components using this class so
# it must only be modified from local.conf if needed
XENGUEST_IMAGE_DEPLOY_SUBDIR ?= "xenguest"
# Directory in which the xenguest should be deployed
# a sub-directory named ${XENGUEST_IMAGE_DEPLOY_SUBDIR} will be created there.
# This should be set to:
# - ${DEPLOYDIR} (default) if creating or extending the xenguest for a normal
# recipe.
# - something in ${WORKDIR} if you need to clone and manipulate an image
XENGUEST_IMAGE_DEPLOY_DIR ??= "${DEPLOYDIR}"
# These vars are used by image_types_xenguest.bbclass to generate the
# xenguest.env file. In a recipe that inherits this class and extra variables
# that should be included in xenguest.env need to be added to
# XENGUEST_IMAGE_VARS_EXTRA
XENGUEST_IMAGE_VARS ?= "\
MACHINE DISTRO DISTRO_VERSION DISTRO_FEATURES TUNE_FEATURES TARGET_FPU \
IMAGE_FEATURES INITRAMFS_IMAGE_BUNDLE INITRAMFS_IMAGE \
XENGUEST_IMAGE_MEMORY_SIZE XENGUEST_IMAGE_NUM_VCPUS XENGUEST_IMAGE_AUTOBOOT \
XENGUEST_IMAGE_ROOT XENGUEST_IMAGE_CMDLINE XENGUEST_IMAGE_EXTRA_CMD \
XENGUEST_IMAGE_KERNEL XENGUEST_IMAGE_DISK_SIZE XENGUEST_IMAGE_DISK_DEVICE \
XENGUEST_IMAGE_DISK_PARTITIONS XENGUEST_IMAGE_NETWORK_TYPE"
#
# Wrapper to call xenguest-mkimage
# It is using XENGUEST_IMAGE_DEPLOY_DIR and XENGUEST_IMAGE_DEPLOY_SUBDIR
# to find the image to operate on
#
# Usage: call_xenguest_mkimage [operation] [args]
call_xenguest_mkimage() {
local cmd="${1}"
local img="${XENGUEST_IMAGE_DEPLOY_DIR}/${XENGUEST_IMAGE_DEPLOY_SUBDIR}"
shift
echo "xenguest-mkimage $cmd $img $@"
xenguest-mkimage $cmd $img $@
}
#
# Create an initial xenguest image.
# This is a task which must be added in a recipe inheriting deploy
# It is using XENGUEST_IMAGE_MEMORY_SIZE, XENGUEST_IMAGE_NUM_VCPUS,
# XENGUEST_IMAGE_AUTOBOOT, XENGUEST_IMAGE_ROOT, XENGUEST_IMAGE_EXTRA_CMD,
# XENGUEST_IMAGE_CMDLINE, XENGUEST_IMAGE_DISK_SIZE and
# XENGUEST_IMAGE_DISK_PARTITIONS to customize the initial guest
#
xenguest_image_create() {
if [ -z "${XENGUEST_IMAGE_DEPLOY_DIR}" -o \
-z "${XENGUEST_IMAGE_DEPLOY_SUBDIR}" ]; then
die "Configuration error: XENGUEST_IMAGE_DEPLOY_DIR or XENGUEST_IMAGE_DEPLOY_SUBDIR is empty"
fi
rm -rf ${XENGUEST_IMAGE_DEPLOY_DIR}/${XENGUEST_IMAGE_DEPLOY_SUBDIR}
mkdir -p ${XENGUEST_IMAGE_DEPLOY_DIR}/${XENGUEST_IMAGE_DEPLOY_SUBDIR}
# Create the image
call_xenguest_mkimage create --xen-memory=${XENGUEST_IMAGE_MEMORY_SIZE} \
--xen-vcpus=${XENGUEST_IMAGE_NUM_VCPUS} \
--xen-root=${XENGUEST_IMAGE_ROOT} \
${XENGUEST_IMAGE_EXTRA_CMD}
# add command line
if [ -n "${XENGUEST_IMAGE_CMDLINE}" ]; then
call_xenguest_mkimage update --xen-clean-extra
for arg in ${XENGUEST_IMAGE_CMDLINE}; do
call_xenguest_mkimage update --xen-extra=$arg
done
fi
# create disk if needed
disksize="${XENGUEST_IMAGE_DISK_SIZE}"
case ${disksize:=0} in
0|0M|0G)
;;
*)
# setup disk size
call_xenguest_mkimage update --disk-reset-config --disk-size=$disksize
diskparts="${XENGUEST_IMAGE_DISK_PARTITIONS}"
if [ -n "$diskparts" ]; then
for arg in $diskparts; do
call_xenguest_mkimage update --disk-add-part=$arg
done
fi
diskdevice="${XENGUEST_IMAGE_DISK_DEVICE}"
if [ -n "$diskdevice" ]; then
call_xenguest_mkimage update --disk-device="${diskdevice}"
fi
;;
esac
if [ "${XENGUEST_IMAGE_AUTOBOOT}" = "1" ]; then
call_xenguest_mkimage update --set-param=GUEST_AUTOBOOT=1
else
call_xenguest_mkimage update --set-param=GUEST_AUTOBOOT=0
fi
if [ -n "${XENGUEST_IMAGE_NETWORK_TYPE}" ]; then
call_xenguest_mkimage update --set-param=XENGUEST_NETWORK_TYPE="${XENGUEST_IMAGE_NETWORK_TYPE}"
else
call_xenguest_mkimage update --set-param=XENGUEST_NETWORK_TYPE="none"
fi
}
XENGUEST_ENV_STAGING_DIR ??= "${STAGING_DIR}/${MACHINE}/xenguestenv"
# Create an intermediary file containing all variables used to by a
# particular recipe that inherits this class
# File will contain the values of all variables listed in:
# XENGUEST_IMAGE_VARS_EXTRA
python do_deploy_xenguestenv () {
xenguest_vars = d.getVar('XENGUEST_IMAGE_VARS_EXTRA')
if not xenguest_vars:
return
outdir = d.getVar('XENGUEST_ENV_STAGING_DIR')
# Writes file to tmp/sysroots/${MACHINE}/xenguestenv/ by default
filename = os.path.basename(d.getVar('FILE')) + '.xenguestenv'
with open(os.path.join(outdir, filename), 'w') as envf:
for var in xenguest_vars.split():
value = d.getVar(var)
if value:
# Write value only if set
envf.write('%s="%s"\n' % (var, " ".join(value.split())))
envf.close()
}
# Since the intermediary file is deleted by do_merge_xenguestenv it
# must be re-created every time
do_deploy_xenguestenv[vardeps] += "${XENGUEST_IMAGE_VARS_EXTRA}"
do_deploy_xenguestenv[dirs] = "${XENGUEST_ENV_STAGING_DIR}"
addtask deploy_xenguestenv before do_populate_sysroot
# Clone the current xenguest from deploy to manipulate it locally
# This is required if you need to change things before packing an image
# To set the local directory where to clone you must set
# XENGUEST_IMAGE_DEPLOY_DIR if you don't want to use do_deploy to modify the
# image
#
xenguest_image_clone() {
if [ -z "${XENGUEST_IMAGE_DEPLOY_DIR}" -o \
-z "${XENGUEST_IMAGE_DEPLOY_SUBDIR}" ]; then
die "Configuration error: XENGUEST_IMAGE_DEPLOY_DIR or XENGUEST_IMAGE_DEPLOY_SUBDIR is empty"
fi
if [ ! -f ${DEPLOY_DIR_IMAGE}/${XENGUEST_IMAGE_DEPLOY_SUBDIR}/guest.cfg ]; then
die "xenguest_image: ${DEPLOY_DIR_IMAGE}/${XENGUEST_IMAGE_DEPLOY_SUBDIR} does not contain a valid guest"
fi
rm -rf ${XENGUEST_IMAGE_DEPLOY_DIR}/${XENGUEST_IMAGE_DEPLOY_SUBDIR}
mkdir -p ${XENGUEST_IMAGE_DEPLOY_DIR}
cp -rf ${DEPLOY_DIR_IMAGE}/${XENGUEST_IMAGE_DEPLOY_SUBDIR} \
${XENGUEST_IMAGE_DEPLOY_DIR}/${XENGUEST_IMAGE_DEPLOY_SUBDIR}
}
# Helper function to retrieve rootfs file if present in one partition
# This can return an empty string or rootfs.tar[.COMP]
def xenguest_image_rootfs_file(d):
disksize = d.getVar('XENGUEST_IMAGE_DISK_SIZE')
# if disksize is 0, we don't create anything
if not disksize or disksize == '0':
return ""
# Find first partition with file=rootfs.tar*
partlist = d.getVar('XENGUEST_IMAGE_DISK_PARTITIONS')
if partlist:
for partdesc in partlist.split():
partelems = partdesc.split(':', 3)
if partelems[3]:
if partelems[3].startswith('rootfs.tar'):
return partelems[3]
return ""
@@ -0,0 +1,123 @@
# This class must be used to extend the xenguest image
# It provides variables to add init scripts, a dtb, xen files or disk files.
#
# The class is extending deploy function so you recipe must inherit deploy and
# have a do_deploy function (even if it is empty)
# Add a variable name to XENGUEST_IMAGE_VARS_EXTRA if you want it to
# appear in xenguest.env when the image is deployed
# Use standard xenguest_image
inherit xenguest_image
# Add a DTB file for the guest
# Only one file should be added, if this is set multiple times or in several
# recipes, the last recipe setting it will prevail.
XENGUEST_EXTRA_DTB ??= ""
# Add a ramdisk file for the guest
# Only one file should be added, if this is set multiple times or in several
# recipes, the last recipe setting it will prevail.
XENGUEST_EXTRA_RAMDISK ??= ""
# Append something to the guest xen configuration
# All files here will be merged together in the final xen configuration
# This can contain several files or be used in several recipes
XENGUEST_EXTRA_XENCONFIG ??= ""
# Add a xenguest init, init-pre or init-post script
XENGUEST_EXTRA_INIT_PRE ??= ""
XENGUEST_EXTRA_INIT ??= ""
XENGUEST_EXTRA_INIT_POST ??= ""
# Add xenguest files, (to be used in extra xen config for example)
# several files may be added, space separated, the path will be kept on the
# generated xenguest image (if dir1/file1 is added, it can be used as
# dir1/file1 file in the xen configuration).
XENGUEST_EXTRA_FILES ??= ""
# Add xenguest disk files (to be used as disk partition content)
# several files may be added, space separated, the path will be kept on the
# generated xenguest image (if dir1/file1 is added, it can be used as
# dir1/file1 file in the disk content parameters).
XENGUEST_EXTRA_DISK_FILES ??= ""
# Extra vars to be written to xenguest.env
XENGUEST_IMAGE_VARS_EXTRA += "\
XENGUEST_EXTRA_DTB XENGUEST_EXTRA_RAMDISK XENGUEST_EXTRA_XENCONFIG \
XENGUEST_EXTRA_INIT_PRE XENGUEST_EXTRA_INIT XENGUEST_EXTRA_INIT_POST \
XENGUEST_EXTRA_FILES XENGUEST_EXTRA_DISK_FILES"
do_deploy:append() {
if [ -z "${XENGUEST_IMAGE_DEPLOY_DIR}" -o \
-z "${XENGUEST_IMAGE_DEPLOY_SUBDIR}" ]; then
die "Configuration error: XENGUEST_IMAGE_DEPLOY_DIR or XENGUEST_IMAGE_DEPLOY_SUBDIR is empty"
fi
rm -rf ${XENGUEST_IMAGE_DEPLOY_DIR}/${XENGUEST_IMAGE_DEPLOY_SUBDIR}
mkdir -p ${XENGUEST_IMAGE_DEPLOY_DIR}/${XENGUEST_IMAGE_DEPLOY_SUBDIR}
if [ -n "${XENGUEST_EXTRA_DTB}" ]; then
if [ ! -f ${XENGUEST_EXTRA_DTB} ]; then
die "xenguest_image: DTB file ${XENGUEST_EXTRA_DTB} does not exist"
fi
call_xenguest_mkimage partial --xen-device-tree=${XENGUEST_EXTRA_DTB}
fi
if [ -n "${XENGUEST_EXTRA_RAMDISK}" ]; then
if [ ! -f ${XENGUEST_EXTRA_RAMDISK} ]; then
die "xenguest_image: DTB file ${XENGUEST_EXTRA_RAMDISK} does not exist"
fi
call_xenguest_mkimage partial --xen-ramdisk=${XENGUEST_EXTRA_RAMDISK}
fi
if [ -n "${XENGUEST_EXTRA_XENCONFIG}" ]; then
for f in ${XENGUEST_EXTRA_XENCONFIG}; do
if [ ! -f $f ]; then
die "xenguest_image: Xen config $f does not exist"
fi
call_xenguest_mkimage partial --xen-append=$f
done
fi
if [ -n "${XENGUEST_EXTRA_INIT_PRE}" ]; then
if [ ! -f ${XENGUEST_EXTRA_INIT_PRE} ]; then
die "xenguest_image: Init script ${XENGUEST_EXTRA_INIT_PRE} does not exist"
fi
call_xenguest_mkimage partial --init-pre=${XENGUEST_EXTRA_INIT_PRE}
fi
if [ -n "${XENGUEST_EXTRA_INIT}" ]; then
if [ ! -f ${XENGUEST_EXTRA_INIT} ]; then
die "xenguest_image: Init script ${XENGUEST_EXTRA_INIT} does not exist"
fi
call_xenguest_mkimage partial --init-script=${XENGUEST_EXTRA_INIT}
fi
if [ -n "${XENGUEST_EXTRA_INIT_POST}" ]; then
if [ ! -f ${XENGUEST_EXTRA_INIT_POST} ]; then
die "xenguest_image: Init script ${XENGUEST_EXTRA_INIT_POST} does not exist"
fi
call_xenguest_mkimage partial --init-post=${XENGUEST_EXTRA_INIT_POST}
fi
if [ -n "${XENGUEST_EXTRA_FILES}" ]; then
for f in ${XENGUEST_EXTRA_FILES}; do
if [ ! -f $f ]; then
die "xenguest_image: Xen file $f does not exist"
fi
call_xenguest_mkimage partial --xen-add-file=$f
done
fi
if [ -n "${XENGUEST_EXTRA_DISK_FILES}" ]; then
for f in ${XENGUEST_EXTRA_DISK_FILES}; do
if [ ! -f $f ]; then
die "xenguest_image: Disk file $f does not exist"
fi
call_xenguest_mkimage partial --disk-add-file=$f
done
fi
}
# Need to have xenguest_image tool
do_deploy[depends] += "xenguest-base-image:do_deploy"
@@ -0,0 +1,15 @@
# This files is added when DISTRO_FEATURES contains arm-autonomy-guest
# We need to have ipv4 activated
DISTRO_FEATURES:append = " ipv4"
# Build a xenguest type image
IMAGE_CLASSES += "image_types_xenguest"
IMAGE_FSTYPES += "xenguest"
# xenguest kernel extension to handle initramfs
KERNEL_CLASSES += "kernel-xenguest"
IMAGE_INSTALL:append = "${@bb.utils.contains('DISTRO_FEATURES', 'docker', \
' packagegroup-docker-runtime-minimal', \
'', d)}"
@@ -0,0 +1,23 @@
# This files is added when DISTRO_FEATURES contains arm-autonomy-host
ARM_AUTONOMY_HOST_OVERRIDES = ":autonomy-host"
# We need to have xen and ipv4 activated
DISTRO_FEATURES:append = " xen ipv4"
DISTRO_FEATURES_NATIVE:append = " arm-autonomy-host"
# Don't include kernels in standard images when building arm-autonomy-host
# If the kernel image is needed in the rootfs the following should be set from
# a bbappend: RDEPENDS:${KERNEL_PACKAGE_NAME}-base = "${KERNEL_PACKAGE_NAME}-image"
RDEPENDS:${KERNEL_PACKAGE_NAME}-base ?= ""
# Require extra machine specific settings from meta-arm-bsp dynamic-layers only
# if meta-arm-bsp is in the bblayers.conf
# Directory for meta-arm-autonomy/dynamic-layers/meta-arm-bsp machine extra settings
ARM_AUTONOMY_ARM_BSP_DYNAMIC_EXTRA_CFGDIR = "${ARM_AUTONOMY_ARM_BSP_DYNAMIC_DIR}/conf/machine"
ARM_AUTONOMY_MACHINE_EXTRA_REQUIRE ?= \
"${ARM_AUTONOMY_ARM_BSP_DYNAMIC_EXTRA_CFGDIR}/arm-autonomy-machine-extra-settings.inc"
require ${@bb.utils.contains('BBFILE_COLLECTIONS', 'meta-arm-bsp', \
'${ARM_AUTONOMY_MACHINE_EXTRA_REQUIRE}' , \
'', d)}

Some files were not shown because too many files have changed in this diff Show More