The tests were not cleaning up after themselves and making assumptions about
memory resident bitbake being stopped by the scripts.
Add cleanup logic to ensure the tests don't break other things and
clean up created files.
(From OE-Core rev: 81d7011a793de2548c573ebb84f60fb80a357bed)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 692dd762a0c817797c28381c6169205fbaeb2705)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Running `devtool search gcc` would result in errors for virtual recipes.
The reason was the newly introduced :layer-<layername> override.
When tinfoil parsed the recipes using the parseRecipeFile() function, it
failed to properly identify the layername for virtual recipes, which
resulted in the errors.
Add tests that exercise these code paths through parseRecipeFile().
(From OE-Core rev: f415c97ad538ce046c87af3fc9eeec6599313764)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
This test was occasionally failing for no obvious reason, so refactor
and improve:
- While waiting for the daemon, check that it is still running and
explicitly timeout after 10s when making the HTTP call.
- While waiting for the daemon to be ready, log the current state of the
daemon so we can tell if we're timing out as it is still scanning.
- This was in fact the cause of the intermittant failures, because the
TMPDIR is reused between tests and may contain a large number of
packages. Do the tests in an isolated TMPDIR to hopefully mitigate this
issue and increase the timeout to two minutes.
- Decorate the test using runqemu as such so that can be skipped in
environments without runqemu
- Add a second test that doesn't use runqemu or images, which is faster
but less realistic.
(From OE-Core rev: 99590fac1bfb5474f5bf0e02d3888b518af9fb3e)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 88b660aaae2527736b6eccec4c952eee969e20a2)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
if user run devtool selftests with a local workspacelayer
the tests fail with various error such as:
- devtool.DevtoolAddTests.test_devtool_add just hangs
- devtool.DevtoolModifyTests.* fail with the following error:
ERROR: Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check bblayers.conf or layer.conf to fix it.
Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check bblayers.conf or layer.conf to fix it.
Check if a workspacelayer exists, warn the user and abort the tests
(From OE-Core rev: b8756f6e20d15f1cc724784d8ceb45a969ec7f81)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit a74962cfb0485f6f2b9e2b751c33c8eafca8705a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Unfortunately there is still an issue with rustdoc not being reproducible
so part of the problem has been fixed but not everything.
Add the exclusion back until this has been addressed to avoid autobuilder
failures and long diffoscope analysis time.
(From OE-Core rev: ab03b3416dda65e39a756c02d87492e1fe5e0fb0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no need to skip this test anymore, as skipping on qemuarm64 was
a proxy for "uses SERIAL_CONSOLES_CHECK" which no longer exists.
[ YOCTO #14962 ]
(From OE-Core rev: 1b8b1cb74c8f5f387a14d86f03f37f68c07dd9aa)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The '--remap-path-prefix' option removes all references to build directory
structure in the debug information within the compiled output for Cargo
dependencies and the project's binary.
However, some references to build directories remains in the final binary
in .rustc section in the form of compressed metadata and this makes the
build output dependent on the folder structure of the computer it's compiled on.
So, for reproducible builds, use the configuration option
'remap-debuginfo = true' along with the '--remap-path-prefix'.
[YOCTO# 14875]
(From OE-Core rev: 6ae62259afbbe861ed74211dab18a27b8c8d8b7a)
Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
it implements various combination of locking single/multiple recipe(s)/task(s)
it also tests that locked sig are architecture dependant
(From OE-Core rev: 7047a067548acd592a834d4315579fb606b4b9d7)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2.13 may not be buildable with latest compilers without patching
(From OE-Core rev: 406a33f896accc35a9cb6ab156f1e0f42dda67d8)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using get_bb_var in the class setup leads to slow startup of oe-selftest.
Move the calls into setupClass instead to remove the overhead at the expense
of some code duplication.
(From OE-Core rev: 64786106746fbcc9d8a35eb6cfa82fd5c6bce7f8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"bitbake -e" executions from get_bb_var calls are slow and slow down oe-selftest
startup. Rationalise the code to avoid them and minimise the number of "parsing"
locations we use by caching key variables and passing them around more.
This was particularly problematic with oe-selftest -j usage since it would
have multiple bitbake -e executions per process making parallel usage
particularly slow.
(From OE-Core rev: 3689cadeb07d76e66f97d890e844f899f69666fe)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the bb_get_var calls trigger multiple "bitbake -e" executions
which slow the start of oe-selftest for any test. Rework the code to avoid
these delays.
(From OE-Core rev: c9ea3570f5dbdaf8c61e445ae9de759a0b71e21f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test above this removal correctly looks at symlinks however to
remove a symlink we should call unlink(), not remove(). This avoids
some build failures/tracebacks.
(From OE-Core rev: dbdb6e73b0f52bc5f9429aca47802d51edbbc834)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test no longer worked with fixes to avoid overlapping license files
in deploy. Fix the test but also improve the logging messages so we don't
get "False is not True" messages upon failure.
(From OE-Core rev: 2fc7714dab5cce585bce4d6c4e8ffbefce2a709b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
assertTrue is a problematic call use in test cases since when it fails,
you just get an unhelpful "False is not True" message.
Replace some uses with assertIn/assertNotIn which will give more helpful results
and for the rest, add msg entries which given more helpful debugging.
For example, this patch would help debugging of #15176.
(From OE-Core rev: 35d4c39e0df1a304f557471151a03d1e4b0f30c7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Don't ignore return values from the git command lines. If something goes
wrong, fail the test right away.
(From OE-Core rev: dfc178a70d6fa60e89d4716f05d68e2c72c6ecd3)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A simple += operation is fine for working with TEST_RUNQEMUPARAMS so remove
the heavier append override usage.
(From OE-Core rev: e1a12c1212013823b44aff88317e5b30df03a35c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Builds core-image-minimal with testimage.bbclass and an ssh server
and then runs tests with slirp networking. Default test suite must
pass.
With full caches on x86_64 with x86_64 target the test passes in 416 seconds:
2023-08-21 09:01:12,295 - oe-selftest - INFO - Ran 1 test in 464.955s
2023-08-21 09:01:12,295 - oe-selftest - INFO - OK
2023-08-21 09:01:19,648 - oe-selftest - INFO - RESULTS:
2023-08-21 09:01:19,648 - oe-selftest - INFO - RESULTS - runtime_test.TestImage.test_testimage_slirp: PASSED (416.62s)
2023-08-21 09:01:19,711 - oe-selftest - INFO - SUMMARY:
2023-08-21 09:01:19,711 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 464.956s
2023-08-21 09:01:19,711 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0)
(From OE-Core rev: a93147278ee92b6aba281ab2b4b7b9d79dc1c89f)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Don't overwrite TEST_RUNQEMUPARAMS since user may have added "slirp"
or similar things to it in site.conf, auto.conf or local.conf. Use
:append to add to it. Fixes failing tests when only "slirp" networking
works with qemu and tun based one is broken on the host machine.
(From OE-Core rev: 5c1d94aa207e5c9a884bf3104eb9e8d1f8b6e432)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have autobuilder workers which don't have the -b option unfortunately.
This reverts commit 5ec557467dda29309e25102b507bb919275bedbb.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Check return values of each git command and set initial branch name
to avoid this warning on every test run:
NOTE: Starting bitbake server...
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: git branch -m <name>
(From OE-Core rev: 5ec557467dda29309e25102b507bb919275bedbb)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's not a warning but a handled case and layer gets added
automatically. Very few build configs have this layer enabled
by default.
(From OE-Core rev: 9a2493ea83f0b30578a819de05108502aaadc7f5)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds support for creating FAT formatted file system images (useful for
boot partitions on some SoCs).
Note that FAT partitions are limited in what they can represent (no
symlinks or device files), so they can't really be used for general
purpose root file systems. As such, they are skipped when testing for
that purpose.
(From OE-Core rev: 440fa508d362b8a449beb1b82dd999e980b753b7)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that SRCPV isn't needed we can simplify things in a few places...
(From OE-Core rev: 843f82a246a535c353e08072f252d1dc78217872)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Source control information being present in PV used to be a hard requirement
for bitbake to operate correctly. Now that hashes are a required part of task
stamps, this requirement no longer exists.
This means we can defer the hash pieces to PKGV and simplify PV.
Use new bitbake fetcher API to inject the source revisions directly into the hash
allowing removal of some horrible code from base.bbclass and avoiding any hardcoding
about how SRCREV may or may not be used.
Use that API to object the string to append to PKGV and append that directly.
The user visible effect of this change is that PV will no longer have revision
information in it and this will now be appended to PV through PKGV when the
packages are written. Since PV is used in STAMP and WORKDIR, users will see
small directory naming and stamp naming changes.
This will mean that sstate reuse through hash equivalence where the source
revision changes but the output does not will become possible as the sstate
naming will become less specific and no longer contain the revision.
The SRCPV variable will no longer be needed in PV and is effectively now just
a null operation. Usage can be removed over time.
(From OE-Core rev: a8e7b0f932b9ea69b3a218fca18041676c65aba0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
e9cff55e73 has switched tag listing from bare "git tag" to "git
ls-remote" to make sure not to miss remote tags which are not fetched
locally. This mechanism first checks for configured remote repository, next
for possibly passed url, and then fails if none worked.
However there are still cases where no remote repository is
configured and no url is provided (for instance: buildperf tests use an
empty git directory to store tests). Fix those cases by putting back the
old behavior (local tags check) as last resort, with at least a warning for
future diagnostics if we still encounter tagging issues
Fixes: e9cff55e73 ("oeqa/utils/gitarchive: fix tag computation when creating archive")
(From OE-Core rev: 34e1f845687d2f7169f5d6c1bb54e1a7ab5412c4)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Propagate a "log" parameter to get_tags in order to know what method is
used to retrieve existing tags
(From OE-Core rev: f8212eda45444dfb330e4d930eeceefde936adeb)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add specific tests on gitarchive for when tag listing is required but no
remote is configured in target directory: it should either succeed if valid
url is provided, or fail is url is not provided or wrong
(From OE-Core rev: 5e9d84d82fde81d66550d8c694ea70f0911fa4f7)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a test suite for gitarchive.py. For now, only introduce tests on
methods which needs to read existing tags
The tests rely on tmpdirs to create local, "fake" results repository in
order to allow basic git commands
(From OE-Core rev: bc9b8ef4bc3e51ba7fca372ab7e532a80975d819)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
systemd now requires usrmerge, ensure this is always the case in
our test cases.
(From OE-Core rev: 465bf84c523403ccc7dc6ed8c2a9c32d85929e8c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We can't have systemd here any longer without usrmerge. We don't really
want to enable the latter since having separate usr will likely result in
a class of reproducibility and host contamination issues that enabling it
might hide.
Also drop INHIBIT_PACKAGE_STRIP since we generally don't build with that
and the debug binaries should be generated regardless. I suspect this
is legacy from older issues.
(From OE-Core rev: 7b7411788e805fa067dd672c9771dcaf2af918a0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously some dependencies couldn't be followed through their siginfo
files. This has been fixed, add a test to ensure this doesn't regress.
(From OE-Core rev: a59cd1502ff14c5d8ccb04385bf4a3ad338d998d)
Signed-off-by: Yang Xu <yang.xu@mediatek.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was done in a selftest, but that is too late and creates
friction in integration as errors are not seen until autobuilder fails.
Bonus fix: SUMMARY check wasn't even working, as in the absence
of one set in the recipe there is a default value set from bitbake.conf.
I left DESCRIPTION check out for now, as many recipes don't actually
have it, and it's set from SUMMARY (plus a dot) if absent.
(From OE-Core rev: 4144c2f43da39336b03cfd612cbe1694cbf8c7bd)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To match the other tests, round the test duration to an intger value
so the test reports are consistent.
(From OE-Core rev: c5be3b22a7d5b3f211080ecdf05a077f8b413ef2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Add the failing test cases in exclude_list to exclude them from testing during the Rust Oe-selftest.
* Drop meta/recipes-devtools/rust/files/rust-oe-selftest.patch file as the failing tests are
moved from being a patch to exclude_list in meta/lib/oeqa/selftest/cases/rust.py.
* When updating to a newer version of Rust, it is manually needed to update the exclude_list. The tests that fail are observed to
work on some versions and fail on others. These tests have been excluded in order to successfully complete testing of Rust Oe-selftest.
* The tests that are passed and skipped are as follows-
Target PASS SKIPPED
ARM 15507 428
ARM64 15535 400
MIPS64 15479 456
X86 15528 407
X86-64 15643 292
Also, time the test execution and pass the time taken for inclusion in the test report.
[RP: Note duration addiion in the commit log]
(From OE-Core rev: e81197c4d3b36e9ad52e56708c21987cacd13147)
Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This provides a more reliable test execution when running tests that
write a large buffer/file and significantly reduces the localedata test
failures.
(From OE-Core rev: 97a7612e3959bc9c75116a4e696f47cc31aea75d)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some of the tests trigger OOM and fail. Increase the amount of memory
available so we dont run into these issues.
(From OE-Core rev: 4d22dba482cb19ffcff5abee73f24526ea9d1c2a)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix the override so we actually pass the correct value to glibc.
(From OE-Core rev: 38fd2120f0f48512091ddad6205ce19839eaf589)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- After introducing the CVE_STATUS and CVE_CHECK_STATUSMAP flag
variables, CVEs could contain a more information for assigned statuses.
- Add an example conversion in logrotate recipe.
(From OE-Core rev: da1f34b58fd106a40c296b68daf8a7e5db354c9a)
Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Ensure the test/class naming doesn't allow the to be triggered
without the toolchain decorator
* Add the toolchain-user decorator so it runs on non-IA targets
* Strip the leading "[XX] " prefix from the test names
* Ensure skipped test counts are passed through correctly (as SKIPPED, not SKIP)
* Avoid duplicate test results and show a warning if any are found (duplicates
were from other sources in the end but the code remains sensible to have)
(From OE-Core rev: b1718ce5b2b2db35a8e1c88087deee41f99094b4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The patch implements Rust testing framework similar to other selftest,
specifically the gcc selftest in OE. It uses the client and server
based method to test the binaries for cross-target on the image.
The test framework is a wrapper around the Rust build system as ./x.py
test. It tests many functionalities of Rust distribution like tools,
documentation, libraries, packages, tools, Cargo, Crater etc.
Please refer the following link for detailed description of Rust
testing:-
https://rustc-dev-guide.rust-lang.org/tests/intro.html#tool-tests
To support the rust tests in oe-core, the following functions were
added:-
setup_cargo_environment(): Build bootstrap and some early stage tools.
do_rust_setup_snapshot(): Install the snapshot version of rust binaries.
do_configure(): To generate config.toml
do_compile(): To build "remote-test-server" for qemu target image.
Approximate Number of Tests Run in the Rust Testsuite :- 18000
Approximate Number of Tests that FAIL in bitbake environment :- 100-150
Normally majority of the testcases are present in major folder "test/"
It contributes to more than 80% of the testcases present in Rust test
framework. These tests pass as expected on any Rust versions without
much fuss. The tests that fail are of less important and contribute to
less than 2% of the total testcases. These minor tests are observed to
work on some versions and fail on others. They have to be added, ignored
or excluded for different versions as per the behavior.
These tests have been ignored or excluded in the Rust selftest
environment to generate success of completing the testsuite.
These tests work in parallel mode even in the skipped test mode as
expected. Although the patch to disable tests is large, it is very simple
in that it only disables tests. When updating to a newer version of Rust,
the patch can usually be ported in a day.
Tested for X86, X86-64, ARM, ARM64 and MIPS64 on Ubuntu 22.04.
(From OE-Core rev: 7c3346d8fbe85302b605bb3f772b029ea7bfaa6c)
Signed-off-by: pgowda <pgowda.cve@gmail.com>
Signed-off-by: Vinay Kumar <vinay.m.engg@gmail.com>
Signed-off-by: Yash Shinde <yashinde145@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie
Peter Kjellerstedt
Ross Burton
Julien Stephan
Sundeep KOKKONDA
Samantha Jalabert
Khem Raj
Mikko Rapeli
Adrian Freihofer
Alexander Kanavin
Joshua Watt
Markus Niebel
Alexis Lothoré
Yang Xu
Yash Shinde
Anuj Mittal
Andrej Valek
Yoann Congal