As the poky repository is no longer used, measurements are indexed using
the oe-core commit. But as bitbake, oe-core and meta-yocto are now
retrieved from separate gits, while measuring performances for a given branch
at some time interval, we can get the same commit for oe-core but
different ones for bitbake or meta-yocto. As a consequence, metadata
associated with the same index (oe-core commit) might differ.
To work around this, relax the equality checks for commit, commit_time
and commit_count since they might no longer match.
Ideally we'd group them into separate results but for now, treat them
as being the same.
[Based on work from Mathieu Dubois-Briand but fixed differently]
(From OE-Core rev: d9c30edf908c129a7540b23e920dd669d2a30657)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e7dc42e30c76bf0fbb4d3cc019bbec675bac55fa)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
When combining an SPDX document, the package list might be empty (e.g.
a baremetal image). Handle this case instead of erroring out
(From OE-Core rev: 1f7326799c33d2a734c58d360773b87d7b86b0ec)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
The base-files test for SPDX 2.2 did not give good coverage, since
base-files doesn't have any dependencies. Add building tar as another
test which more fully exercises the code
(From OE-Core rev: 2c299c17ef1a97505fd7de8d3ebc9de25fb838fc)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Changes to cve-check (see poky commit fb3f440b7d,
"cve-check: annotate CVEs during analysis") modified the
get_patched_cves() API to return a set of CVE IDs instead of a
dictionary of CVE metadata.
The SPDX 3 backport still expected a dictionary and attempted to call
.items(), leading to:
AttributeError: 'set' object has no attribute 'items'
This patch updates the SPDX3 code to iterate directly over the CVE IDs
and use `oe.cve_check.decode_cve_status()` to retrieve the mapping,
detail, and description for each CVE. This restores compatibility with
the updated CVE API and matches the behavior of SPDX3 handling on
Walnascar.
A warning is logged if a CVE has missing or unknown status.
(From OE-Core rev: 55fdeea44ffbecb705f7900bfa85ab88e1191878)
Signed-off-by: Kamel Bouhara (Schneider Electric) <kamel.bouhara@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
The SPDX code needs to be able to look up an Element by its SPDX ID,
locating the file that (should) contain the SPDX ID and opening it for
parsing. Previously, the code would do this be hashing each Element
SPDX ID and Alias, and the creating a symbolic link to the file that
contains the element with a name of the hash.
This worked well as it was possible to look up any arbitrary SPDX ID or
alias by simply hashing it and following the symbolic link to get the
file. However, the down side of this approach is that it creates a lot
of symbolic links, since it will make one or two per Element in the
document. This can be a problem when using SPDX_INCLUDE_SOURCES, for
example.
This change reworks this strategy so that the only Element that gets a
symbolic link based on the hash is the singular SpdxDocument that is
create for each file. All other Elements are assigned an alias with a
special prefix that encodes the hash of SpdxDocument alias. Thus, when
attempting to look up an arbitrary alias, the code sees the special
prefix, extract the hash, opens the file based on the symlink with that
hash name, then finds the matching Element in the file. This drastically
reduces the number of symbolic links by making only one per file.
This also means that the custom link extension can be removed since it
is now superfluous.
(From OE-Core rev: 551433c7a1eddf5090c87a243ea104bf091992b0)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 838d64c09657ac53175737fc4e7fd6f01f3dcf47)
Signed-off-by: Kamel Bouhara (Schneider Electric) <kamel.bouhara@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Adds a test for several of the extra options provided by the SPDX
classes. In particular, these are the options that can produce
non-reproducible results, so are not enabled by default in OE core. This
test takes care to configure the build so that the tests do run in a
reproducible manner so that pre-built test objects can be pulled from
sstate
(From OE-Core rev: 72ee311d4f74499674a29223fb02d4e774097a54)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 14f534f15f7fe6362723d7f064d39783c5bd758f)
Signed-off-by: Kamel Bouhara (Schneider Electric) <kamel.bouhara@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Adds test cases for SPDX 3.0. Reworks the SPDX 2.2 test setup so it can
also be run even if the default is SPDX 3.0
(From OE-Core rev: e182f76a866d4d750d2baf7b56ffebead5264de2)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b1d2309b3ab0fd8b0d8c4dfa59f50c85074bbd3b)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Upstream commit 544d46e4169a ("selftest/spdx: Fix for SPDX_VERSION addition")
updated the selftests to expect SPDX artifacts under:
${DEPLOY_DIR}/spdx/${SPDX_VERSION}/
However, in this branch the effective SPDX output was still being
written to:
${DEPLOY_DIR}/spdx/${PACKAGE_ARCH}/
without the version subdirectory. This caused SPDX selftests such as
test_spdx_tar to fail with missing file errors, e.g.:
AssertionError: .../deploy/spdx/SPDX-1.1/core2-64/packages/tar.spdx.json does not exist
Update create-spdx-2.2.bbclass so that DEPLOY_DIR_SPDX includes
${SPDX_VERSION}, matching the expected deploy structure and restoring
successful SPDX selftests.
(From OE-Core rev: 8996d0899df5316742ba5fd73c351e8ca67dc90b)
Signed-off-by: Kamel Bouhara (Schneider Electric) <kamel.bouhara@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Update the test for the addition of SPDX_VERSION to the deploy path.
(From OE-Core rev: d75bfbaf69292f80cacc5b8d6cbff03418a34ebc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 055e8c21908127722abad9e992d6408d8697a119)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Downstream tasks may want to know what image files were written so write
out a manifest in do_image_complete. The format of the manifest is the
same as the one in image.bbclass
(From OE-Core rev: e15a9934be84c59fc1bf957a60fa395e521abcfc)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Downstream tasks may want to know what image files were written by the
do_image family of tasks (e.g. SPDX) so have each task write out a
manifest file that describes the files it produced, then aggregate them
in do_image_complete
(From OE-Core rev: 5da5e2c528e8f4c78d389d60b03725323ff1527c)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5e55ed4c5b9d5af3c96b82805af34af1512fc3d1)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
The code in this file uses oe.qa, but it was not imported resulting an
an exception when a license error was detected
(From OE-Core rev: f8aa42faa3640d0414745ae32b00a37b2f5d638b)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e44cb7c5b7281d614ed51fdec06dad0a7211528a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Moves the code that skips packages with incompatible licenses to the
library code so that it can be called in other locations
(From OE-Core rev: 4f7a047c4a1e14bbb3bf593764aace1e25bcd4a4)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 86eb409e3c1b30110869ec5a0027ae2d48bbfe7f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
do_create_spdx is a outlier in that it doesn't need the RSS to be
extended just because it depends on do_populate_sysroot. In fact, it
only depends on do_populate_sysroot so it can see the actual recipes
sysroot, and attempting to extend the sysroot can cause problems for
some recipes (e.g. if a recipe does do_populate_sysroot[noexec] = "1")
As such, explicitly exclude do_create_spdx from extending the sysroot
just because it depends on do_populate_sysroot.
(From OE-Core rev: 572bf8fd0ba83c7174f706f17a589abbcdc54df5)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Moves several of the functions in license.bbclass to be library code
New function dependencies were manually verified using bitbake-dumpsigs
to ensure that bitbake identified the same dependencies even though they
are now in library code (although the new function names mean that the
task hashes still change)
(From OE-Core rev: 5e220e20833fd800687b05c8f5cef602dfc47202)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0333e04e353991260c5f67a72f80f3ab9dcf526a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
The add_download_files() function incorrectly accessed fd.name, which
does not exist on FetchData objects.
Change to use fd.names[0] to correctly retrieve the first filename.
This fixes AttributeError during SPDX document generation.
(From OE-Core rev: 17031d71cf4bc4fc19dd8a41c49b94e1f6a1edee)
Signed-off-by: Kamel Bouhara (Schneider Electric) <kamel.bouhara@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Backports the SPDX 3.0 support and fixes from upstream walnascar
commit 49f47169953b807d430461ca33f3a2b076119712 into upstream
scarthgap.
(From OE-Core rev: 9c9b9545049a2f6e5c99edcb079275d29a4d1ac6)
Signed-off-by: Kamel Bouhara (Schneider Electric) <kamel.bouhara@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
The upstream changed the branch name, moving to archive/ so we need to
update too. Take the opportunity to match the new location too to avoid
the redirect.
We could use a different branch but upstream would probably eventually
rename that too so this may last longer.
(From OE-Core rev: 1e0a64a1890a62e130595f46d93c8d08af9170f5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 478a645bad150f04dee1b0085c4542c2eefe7007)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
When the package is installed directly on the machine (instead of
installing it in the rootfs directly), the postinstall script fails with
the following error:
/usr/sbin/update-ca-certificates: line 75: shift: shift count out of range
The reason is that the "update-ca-certificates" script is executed with
the "--sysroot" argument, and as the sysroot $D is passed. However on the
target system this variable doesn't exist, so the argument is passed without
this mandatory value, and the execution fails.
To avoid this error, check if the $D variable exists, and pass the --sysroot
argument only when it does.
Reported-by: WXbet <Wxbet@proton.me>
(From OE-Core rev: 9a2bd3b6e2e53071a1463d2804d0d4fb17b1814f)
Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit cf39461e97098a1b28693299677888ba7e8bfccf)
Signed-off-by: Ankur Tyagi <ankur.tyagi85@gmail.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
0001-Revert-mozilla-certdata2pem.py-print-a-warning-for-e.patch
0001-update-ca-certificates-don-t-use-Debianisms-in-run-p.patch
refreshed for 20250419
0002-sbin-update-ca-certificates-add-a-sysroot-option.patch
removed since it's included in 20250419
(From OE-Core rev: dd05818a422c8c5be1aef06405d200280b382b91)
Signed-off-by: Wang Mingyu <wangmy@fujitsu.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e39cc1fb7234bf2b37856296d3c0d10ddf8cae64)
Signed-off-by: Ankur Tyagi <ankur.tyagi85@gmail.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
ca-certificates/0002-update-ca-certificates-use-SYSROOT.patch
was using a non-standard environment variable, and was replaced
with a patch that adds a command line option (and then this
was submitted upstream). ca-certificates recipe was tweaked accordingly,
and nothing else in core or meta-oe is using update-ca-certificates.
Drop default-sysroot.patch as the use case is unclear: sysroot
is explicitly specified in all known invocations of update-ca-certificate,
and if there's a place where it isn't, then update-ca-certificates
will error out trying to write to /etc, and should be fixed to
explicitly specify the sysroot.
(From OE-Core rev: a80185fd72a2be183783b0e464c07f1043d7dd37)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 90d9f0ba674d4fe8e9291f0513c13dff3775c545)
Signed-off-by: Ankur Tyagi <ankur.tyagi85@gmail.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
git repo no longer has tags for recent versions which means
we had missed several of them, and wouldn't be able to get
notifications about any future releases.
(From OE-Core rev: 44c113497c7e3f9f06604e892df1eb717bb3410d)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 81f013fd1312551628701bf36ac62746a2606dbd)
Signed-off-by: Ankur Tyagi <ankur.tyagi85@gmail.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Provide references for how the SRCREV was arrived at for the 20240203
release.
(From OE-Core rev: b1d86653f1485aa56fe8bf050931d5b8657ee499)
Signed-off-by: Theodore A. Roth <troth@openavr.org>
Signed-off-by: Theodore A. Roth <theodore_roth@trimble.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 6916cdb0f05f6644edb1e432a9421595abb9f0ca)
Signed-off-by: Ankur Tyagi <ankur.tyagi85@gmail.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
The 20240203 version is the same as used in Ubuntu >= 24.04 and Debian
Trixie (testing).
(From OE-Core rev: 63620f034019b3b3585e263bd26b3fadd9a1692e)
Signed-off-by: Theodore A. Roth <troth@openavr.org>
Signed-off-by: Theodore A. Roth <theodore_roth@trimble.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ce19168885a04b0d77e81c1fd1c4262b195a47d4)
Signed-off-by: Ankur Tyagi <ankur.tyagi85@gmail.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
The Reader.ReadResponse function constructs a response string through
repeated string concatenation of lines. When the number of lines in a
response is large, this can cause excessive CPU consumption.
(From OE-Core rev: 512c36af3b9d344606b2ebf54bc2f99b88dfea63)
Signed-off-by: Archana Polampalli <archana.polampalli@windriver.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
The processing time for parsing some invalid inputs scales non-linearly with
respect to the size of the input. This affects programs which parse untrusted PEM inputs.
(From OE-Core rev: 228e4aa70743b92eaf1abd5526827b34b33f3419)
Signed-off-by: Archana Polampalli <archana.polampalli@windriver.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
The Parse function permits values other than IPv6 addresses to be included
in square brackets within the host component of a URL. RFC 3986 permits
IPv6 addresses to be included within the host component, enclosed within
square brackets. For example: "http://[::1]/". IPv4 addresses and hostnames
must not appear within square brackets. Parse did not enforce this requirement.
(From OE-Core rev: c5fc59eb87d0f92ba8596b7848d16d59773582a0)
Signed-off-by: Archana Polampalli <archana.polampalli@windriver.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
When Conn.Handshake fails during ALPN negotiation the error contains attacker
controlled information (the ALPN protocols sent by the client) which is not escaped.
(From OE-Core rev: e734cf62f24640d116c901dd97e09ddbb1f0cc4f)
Signed-off-by: Archana Polampalli <archana.polampalli@windriver.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Validating certificate chains which contain DSA public keys can cause
programs to panic, due to a interface cast that assumes they implement
the Equal method. This affects programs which validate arbitrary certificate chains.
(From OE-Core rev: b532fa208d0b102326642a2fba8b17661a14307e)
Signed-off-by: Archana Polampalli <archana.polampalli@windriver.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Due to the design of the name constraint checking algorithm, the processing
time of some inputs scals non-linearly with respect to the size of the certificate.
This affects programs which validate arbitrary certificate chains.
(From OE-Core rev: ce1626d1f1e232bc6da81e89088d0c0f5f3c52b4)
Signed-off-by: Archana Polampalli <archana.polampalli@windriver.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Parsing a maliciously crafted DER payload could allocate large amounts of memory,
causing memory exhaustion.
(From OE-Core rev: f27acc863ee34b56e2c49dc96ad2b58fb35e2d46)
Signed-off-by: Archana Polampalli <archana.polampalli@windriver.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
When recreating the uri in wget's checkstatus method, we only use the
scheme, host and path. This completely strips the query parameters from
the final URI and potentially breaks the checking functionality for
URLs that require query parameters (such as the AZ fetcher with SAS
token).
This bug was resolved on master in
`096301250455e2a83bdd818a56317c62436c9981`.
This patch is adapted to the scarthgap branch.
CC: Steve Sakoman <steve@sakoman.com>
(Bitbake rev: 8dcf084522b9c66a6639b5f117f554fde9b6b45a)
Signed-off-by: Philippe-Alexandre Mathieu <pamathieu@poum.ca>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
There is the following warning when executing to bitbake linux-yocto:
bitbake/lib/bb/fetch2/__init__.py:464: DeprecationWarning: 'count' is passed as positional argument
This is because the 4th parameter of re.sub(pattern, repl, string, count=0, flags=0)
is a keyword parameter. We use keyword arguments for parameters that are not positional.
(Bitbake rev: c2a54aceab4c75cea6f8be16fe6d0caed12b32c4)
Signed-off-by: Bin Lan <bin.lan.cn@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
kernel commit bfb713ea53c7 ("perf tools: Fix arm64 build by generating unistd_64.h")
introduces a new dependency on source files for arm64, specifically
include/uapi/asm-generic.
Build fails with:
[..]/perf/1.0/perf-1.0/scripts/Makefile.asm-headers:33: [...]/perf/1.0/perf-1.0/include/uapi/asm-generic/Kbuild: No such file or directory
make[4]: *** No rule to make target '[...]/perf/1.0/perf-1.0/include/uapi/asm-generic/Kbuild'. Stop.
Add the directory to PERF_SRC.
Fix whitespace error while at it.
(From OE-Core rev: 06d4981313ce67a8d53b1c14be9845b4b5a9f4cf)
Signed-off-by: Yannic Moog <y.moog@phytec.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Passing params as numbers to hwclock is broken in util-linux 2.39.3 due
to wrong pointer handling. So backport the fix from upstream included
since util-linux 2.41.
(From OE-Core rev: 3d8f88906f5560286462eaf55226b872e2805df7)
Signed-off-by: Bastian Krause <bst@pengutronix.de>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
An error occurred in backport commit
649147913e89cd8f7390cb17cd0be94c9710ffa6. The test file
is empty and has no functionality at all.
(From OE-Core rev: 0539a7869c4a3e28b3e7d0ab93fe07bfb9462d13)
Signed-off-by: Michael Haener <michael.haener@siemens.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Pick patches as listed in NVD CVE report.
Note that Debian lists one of the patches as introducing the
vulnerability. This is against what the original report [1] says.
Also the commit messages provide hints that the first patch fixes this
issue and second is fixing problem with the first patch.
[1] https://jvn.jp/en/jp/JVN19358384/
(From OE-Core rev: a157719ab349d9393d5a640bb2e45fc2489d5338)
Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Added by commit e478550c8cd8 ("openssl/fontconfig/bzip2: Use relative
symlinks instead of absolute ones (using a new class)") in OE-Core.
(From yocto-docs rev: a8687e4bb2e822670b6ad110613a12fa02943d3d)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
(cherry picked from commit c0dc554eba7d421023ecc68a70b7a19df38628b0)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
