Fixes the following warning sometimes appearing during image builds:
WARNING: The license listed ABC was not in the licenses collected for recipe xyz
The files being looked for here, which runs during do_rootfs,
are written out by the do_populate_lic task for each recipe. However,
there was no explicit dependency between do_rootfs and all of the
do_populate_lic tasks to ensure they had run - only an implicit link via
do_build, so it is possible that sometimes they had not depending on how
the tasks were scheduled. Add an explicit set of dependencies to fix
this.
(From OE-Core master rev: ef7dc532e800d9b170246550cbc8703adf624beb)
(From OE-Core rev: f521d8d2d1ea495383f54e5e7c2754dde007f7eb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes the following error showing up for layers that aren't a git repo
(or aren't parented by one):
fatal: Not a git repository (or any of the parent directories): .git
This was because we weren't intercepting stderr. We might as well just
use bb.process.run() here which does that and returns stdout and stderr
separately.
(This was a regression that came in with OE-Core revision
3aac11076e).
Fixes [YOCTO #8661].
(From OE-Core master rev: f533c1bf4c6edbecc67f9e2c62fd475d64668e86)
(From OE-Core rev: 8968ede9c8cdcd2cbf13bd5bba95883082189908)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix test_layer_appends to work when build directory is not inside
COREBASE.
Fixes [YOCTO #8639].
(From OE-Core master rev: 0f146e77655d153d3f9a59e489265450f08c6ad7)
(From OE-Core rev: e353b303e271368426e71810bb75173ca6f53455)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix test_devtool_update_recipe_git to work when build directory is not
inside COREBASE.
Fixes [YOCTO #8639].
(From OE-Core master rev: 0225888207f82e5f1d9e3dffb7c342a10169aea3)
(From OE-Core rev: 16250994516ff907e18e71158aeb15e4d637de63)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We weren't splitting SRC_URI values containing multiple URIs here; this
didn't cause any errors except when a trailing ; was left on a URI, in
which case the next URI was considered part of the parameter, which
didn't contain a = and therefore was considered invalid.
We only care about the first URI in SRC_URI in this context (since
that's the upstream URI by convention) so split it as we should and take
the first item.
Fixes [YOCTO #8645].
(From OE-Core master rev: 8e75b7e7d54e5638b42b9e7f90f2c6c17e62033f)
(From OE-Core rev: a28eba9fb03720c805eae02c3d0aebf9294e300b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make the following improvements to edit_bblayers_conf():
* Support ~ in BBLAYERS entries
* Handle where BBLAYERS items are added over multiple lines with +=
instead of one single long item
Also add some comments documenting the function arguments and return
values as well as a set of bitbake-selftest tests.
(This function is used by the bitbake-layers add, remove and
layerindex-fetch subcommands, as well as devtool when adding the
workspace layer).
(Bitbake master rev: e9a0858023c7671e30cc8ebb08496304b7f26b31)
(Bitbake rev: fca41cf073469493e9dada377fc42d4b084c45c9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you tried to delete the variable on the first line passed to
edit_metadata() this failed because the logic for trimming extra blank
lines didn't expect the list to be empty at that point - fix that bad
assumption.
(Bitbake master rev: 8bce6fefdc5c046b916588962a2b429c0f648133)
(Bitbake rev: 3fbf3f8211183ecb18938f2fc9acaa400766d9f0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to handle keyword arguments here or sending a keyword argument
to a decorated function that accepts keyword arguments will trigger an
error. (This showed up when testcase decorators were added to the
recipetool.RecipetoolTests.test_recipetool_appendsrcfiles_basic_subdir
test).
(From OE-Core rev: f787b688f2884ce3fa888b4041030538c7d2bf55)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For modify / extract / upgrade, if the specified "recipe" is not
actually a recipe but a virtual target such as virtual/kernel, map it
correctly to the actual recipe and make sure we use that name within the
workspace. Thanks to Chris Larson for reminding me this was still broken
and for a hint on how to fix it.
(From OE-Core rev: 1f7752282ffb47d2621030ddb2fa42a5e491d6d2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Print a line that's the full length of the title that it's underneath.
(From OE-Core rev: 83729437c584fc74577a2b15dbb2ae094788339d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you run the extensible SDK environment setup script in a shell
session where oe-init-build-env has been run already, and attempt to use
the two together, strange things happen - you may not even be running
devtool from the extensible SDK, but the OE tree. This isn't a supported
use case anyway, so show a warning recommending starting a new shell
session.
(From OE-Core rev: 41afc48ab979dff6ebb3ea4003dd6baefff7f644)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Print a note at the end of the environment setup script pointing to
devtool.
(From OE-Core rev: ea1d566bc2ff61f2e086effb9ca6551b263eacbe)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to prepare the build system within the extensible SDK, we
actually go ahead and build the targets specified by SDK_TARGETS (by
default the image the SDK was built for). Assuming that's an image, we
don't actually need to build the image itself - we just need to have
everything done up to the point before building the image, so that we
have everything needed in the sysroot.
In order to do this, create temporary bbappends for each of the targets
in the workspace layer that stub out do_rootfs and related tasks if they
exist. This is a little bit of a hack but is the least intrusive fix at
this point. To make things a bit tidier, I have split out the
preparation commands into a separate script so we can run that in the
appropriate environment rather than all the commands separately.
Fixes [YOCTO #7590].
(From OE-Core rev: d2a2962897b89731a5705b0cbc7c6f36aa53dcc8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make indentation consistent here in preparation for the changes that
follow.
(From OE-Core rev: 59a3789c678bf58c0a04b8def416246654680841)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As part of populating the sstate-cache with an artifact (.tgz file) we
create a temp file and then atomically move it to the final name. Due to
the glob used in this script such temp files were being matched, and
between the time they were matched and the time the script started
copying files, the temp file may have vanished.
This fixes random "No such file or directory" failures building the
extensible SDK on build setups where the sstate-cache directory is shared
amongst multiple build machines.
(From OE-Core rev: 588de5198c641ea1cfc3e01a6d129296bd2f706b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If recipe parsing fails we need to exit out instead of attempting to use
the data object that's set to None, which resulted in a traceback.
(From OE-Core rev: a46fb1a06be2df00a0bceb4e60ebdecb55b5974c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
OE-Core commit 57af3fb9662106f0a65a1b4edf83e2398be0a8f1 upgraded tzdata
but also reverted a couple of changes to SUMMARY and LIC_FILES_CHKSUM.
Reinstate these (with an update to the README md5 value since that has
changed slightly, without any change to the licensing statements
within).
(From OE-Core rev: cea4f6b86129f84a99700207777929bf7e811ed6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't need PATCHTOOL to be set to git in this recipe, and setting it
that way requires that the running user has git user & email configured,
which on a build server it might well not be.
(From OE-Core rev: ae8dcdc3c8e090fe392de86dc59135a38f06e9ca)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the directory specified by --extract-to exists, because we were using
shutil.move() to move the temporary extracted directory to the specified
path, a subdirectory was being created under that directory instead of
moving the contents, which was a different result than if the directory
didn't previously exist. We could try to always move the contents but
that's complicated when any symlinks are involved; the simplest thing is
just to remove the directory (which should be empty anyway) before
moving the temporary directory across in its place.
(From OE-Core rev: 2880bd23b471c1966661b9f05726faf60f9c0e7e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use "git cherry" against the original tag that we made when we extracted
the source in order to find the revisions that are definitely new. This
allows you to modify a commit in the middle of the series and then run
devtool update-recipe and not have the subsequent patches unnecessarily
modified.
Fixes [YOCTO #8388].
(From OE-Core rev: 7baf57ad896112cf2258b3e2c2a1f8b756fb39bc)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We only need the base configuration to get the variable values we want
to get here, there's no need to parse recipes / load the cache.
(From OE-Core rev: 84172ff7b325f081dba6430fd27f12e3745838b5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds a subcommand to search to find the target recipe name providing
some file or capability. This is implemented by searching on recipe
name, package name, description, package contents (file names), and
runtime file provides. For example:
$ devtool search libGL
mesa
$ devtool search X11
xextproto
libxxf86vm
xf86driproto
xf86vidmodeproto
libxfixes
xproto
libx11
...
$ devtool search /bin/sed
busybox
sed
This is particularly useful within the extensible SDK but is also made
available in devtool alongside the build system.
Note of course that because this searches pkgdata, useful results depend
upon do_packagedata(_setscene) having executed for the recipe being
searched for.
(From OE-Core rev: 48cbde0ea77ed20126eceba5feb37c42a9229500)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We ship the runqemu script and if we build QEMU itself within the
extensible SDK, then it would be nice to be able to run it. This is a
very thin wrapper around runqemu, supplying the machine and image name
so the user doesn't need to. (This subcommand is only available within
the extensible SDK since it only really makes sense there where it is
otherwise hard to run runqemu directly.)
Implements [YOCTO #6657].
(From OE-Core rev: abca7a0cac7068ffe6a6b873d0842f804388b621)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a means of creating recipes for package files or archives that
contain a directory structure to be installed verbatim, for example an
rpm file. (We mostly just re-use bin_package here and skip some of the
normal build system checks.) This support is available in "recipetool
create" and "devtool add" which wraps the former.
(From OE-Core rev: 9e0a6b2e6f16185f8032d36b77d40802bc388987)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Upon further reflection, it seems to me that this bbappend ought to just
be deleted at the end of the build. This keeps things simple; you never
have to remember to delete any files to get back to where you were
before with the image. This means we can also drop the slightly awkward
message reminding the user how to do that. I've also updated the test to
look at the image manifest to determine if the command has worked
instead of looking for the (now deleted) bbappend.
(From OE-Core rev: f6b90bceaedf9bad3d111e6ca1fa79e59f472c73)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Make image optional for the extensible SDK (auto-determine it based on
the targets the SDK was built for)
* Check that specified recipe is in fact an image
(From OE-Core rev: 8884875aacfedc69cc72898684e391e69fea00ba)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the user is running "devtool build-image" within the extensible SDK
then they probably won't know where to find the resulting output files,
so we should tell them explicitly.
(From OE-Core rev: 9baf9fd7a53142a98c7f1cd49c7b001760af51f9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We build recipes and include packages into the image, adjust the
terminology used in code and messages accordingly. Also fix a few typos.
(From OE-Core rev: 866f6e2de20b7022803e53c4de3ff341521b4db5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we end up printing a message about the build directory being the same
as the source, we should print that first and then print the message
about the recipe file possibly needing to be edited to the end so that
it has slightly more impact.
(From OE-Core rev: 97398d14c444fe2408dd6101ef46a0a406924bb5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the fetched source isn't already a git repository, initialise it as
one and then branch and tag, just as we do with "devtool modify". This
makes it easier to make changes, commit them and then use the
"devtool update-recipe" command to turn those commits into patches
on the recipe.
(From OE-Core rev: 2dd865086c37c9eff63c6d0bbfa9f2e909f9fffe)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some recipes don't extract any source (for example, opkg-keyrings). We
were producing a traceback in this case because we weren't checking if
the directory existed.
(From OE-Core rev: 087ca0c644aad19e6e7d3921f99f3056471deb65)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Recipes that use work-shared (such as libgcc) are capable of unpacking
the source, but it doesn't necessarily unpack to ${WORKDIR}/${BP}. Use
the last part of the actual S value instead which is more likely to
work.
(From OE-Core rev: 0bab8086fdef93b88d5ec6b88b07ccc921dd786f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you try to use "devtool modify -x" or "devtool extract" on a recipe
where do_unpack has been set as noexec (e.g. glibc-locale), then we get
an error because the expected source wasn't ever unpacked. Do a check up
front for noexec being set on do_unpack and error out with a reasonable
message if that's the case.
(From OE-Core rev: 39a93c85e798e72edd6ab3c05f8adbb1a97e893e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The # character in a URI denotes a fragment; we don't care about this
since it is never supposed to be sent to the server, so remove it from
the URI before actually trying to fetch it or use it in SRC_URI within
the recipe.
(This has come up because download links on pypi.python.org seem to have
a fragment containing the md5sum of the download; without stripping this
off the fetcher will choke on it.)
(From OE-Core rev: 58dc726808817210764eab963a4d453dc4ff49aa)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The shell considers empty functions to be a syntax error, so for
template shell functions that contain only comments (or no lines at all)
then add a : to act as a no-op which avoids the syntax error.
(From OE-Core rev: ff14d9e5b935b99b2efde479515e54c02ba58f6e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we were adding a recipe for software that would typically be built
in the same directory as the source, we were always using a separate
build directory unless the user explicitly specified not to, leading to
errors for software that doesn't expect to be built that way (such as
Python modules using distutils). Split out the code that makes this
determination automatically from the "devtool modify" and "devtool
upgrade" code and re-use that here so the behaviour is consistent.
(From OE-Core rev: 320585b7ff6340df0b0dbc63f95ed3ca8fc3a993)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These functions ostensibly allowed parsing a recipe without bbappends
but this clearly hadn't been tested because a variable was unassigned in
both of them in that case.
(From OE-Core rev: d2bb9f08303bb120e811c03af2f5339e8f262cfa)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes, particularly if you extracted the source to /tmp which is on
tmpfs, the external source tree that is being pointed to may no longer
exist when you come to run "devtool build" or "devtool update-recipe"
etc. Make all of the commands that need to check for a recipe being in
the workspace call a single function and have that function additionally
check the source tree still exists where appropriate.
(From OE-Core rev: 0c3f289576a2ab35b1d1d8854d6763553cc3bf09)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a script to run "devtool modify" followed by a build on every target
recipe in the environment (with the option to skip/resume from/only
include specific recipes). This takes far too long to run as an
oe-selftest test but is still something that is useful to be able to
run. There's also a slightly quicker mode that just runs "devtool
extract" on each recipe.
(From OE-Core rev: 278f40cce14af430ac1743436132584eedfe792e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unfortunately it appears that under certain circumstances, a failed
git am followed by git am --abort won't clean up any changes the patch
might have made - this was seen when running "devtool extract" on the
unzip recipe; unzip-6.0_overflow3.diff has a malformed date as far as
git am is concerned but it triggers this condition. Add a
git reset --hard HEAD followed by git clean -f in order to recover from
this scenario.
(From OE-Core rev: 21fdbd76f458b70a6646dd6d0749e3a465ebd320)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The value of the deps varflag for tasks is (unusually) expected to be a
list object rather than a space-separated string, hence we cannot use
appendVarFlag() here. This fixes a traceback when parsing the gcc recipe
with externalsrc enabled, for example.
(From OE-Core rev: 47c1ff11a9b8b1e9f40ffb2b3d387252200cf0ec)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It turns out that there are folks out there who use externalsrc in
normal builds and don't really need to be warned; additionally within
the extensible SDK or when using devtool, it shouldn't be a warning
situation. Thus, scale it back to a note (we can't use bb.note() here
since that wouldn't actually be piped through to the bitbake UI). Also
touch up the message a little bit.
(From OE-Core rev: 87bedd137a77c5ac0033a1de4587c40dbe93e81d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tar has supported a --checkpoint option since version 1.15.91, so it
should be safe to use here to print dots showing that it's still doing
something (technically it's not really progress unless you know how many
dots it's going to print, which even it doesn't know at the start, but
it's better than nothing).
(From OE-Core rev: ee4aadd179be8141536f2fae16482683855a9a37)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is not actually used for anything - I thought that we would need to
use it within devtool to set global configuration, but we're able to do
everything we need within the bbappends it creates, which also saves on
parse time. If we're not going to use work-config.inc let's just drop it
completely.
(From OE-Core rev: 84a1f82acd3b6ebb3c073aae6b996d2203dad2ce)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Provide the ability to define a function containing extra configuration
values to be added to the local.conf file that goes into the SDK. For
example, this could be used to set up SSTATE_MIRRORS within the SDK.
(From OE-Core rev: ef7001232ac2da8ee63ec568d1abba13e4fd4382)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you ran the extensible SDK installer file with sh (instead of bash),
then the additional call to buildtools environment setup, extension of
PATH to support running devtool, and setting of OE_SKIP_SDK_CHECK
weren't being added to the end of the script. This is because apparently
bash is happy to expand wildcards in the target of a redirection, but
bash running in POSIX sh mode won't (although it apparently does work on
the sh command line rather than within a script run as an argument to
sh). In any case using a wildcard here is a bit of a crutch which we
don't need, so replace it with the proper path to the environment setup
script.
(From OE-Core rev: ba0f6b6ec32275329ebbb7c27f661c027b7a921d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Variables such as SRC_URI which are space-separated may also contain
Python expressions (${@...}) which themselves contain spaces that
shouldn't be split when splitting the value into items. In order to
ensure this we need to use a custom splitting function instead of just
string.split().
This issue could be seen when doing "devtool modify sudo", adding a
commit to the resulting source repository then "devtool update-recipe" -
the Python expression in SRC_URI was being unnecessarily broken onto
multiple lines.
Fixes [YOCTO #8046].
(From OE-Core rev: bbec2ee98a08270c681189a6ba26beb1034d3e2f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch was merged into the 4.8.7 release. This was missed on
upgrade, and we didn't notice because our quilt-based patch handling
code ignores already-applied patches.
(From OE-Core rev: 598c9660f26018a748a4749377389ced7eab2229)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is just rudimentary support at the moment as we'd potentially want
to compare the control files a bit more specifically than this does, but
it's a start.
(From OE-Core rev: 60564c6d6b8c1a3813baa04fb0d5597cf63f2a9f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is useful in a few different contexts to see which files have been
written out by an sstate task; break out a function that lets us get the
path to the manifest file easily.
(From OE-Core rev: 090196dd2d8f4306b34b239e78c39d37cc86034c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes the following error when running devtool from a directory other
than the build directory (or the SDK base path when using within the
extensible SDK):
The BBPATH variable is not set and bitbake did not find a
conf/bblayers.conf file in the expected location.
Maybe you accidentally invoked bitbake from the wrong directory?
(From OE-Core rev: 1687ec04bfee327fe24fdfecb67db689835769f2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a recipe has been added to the workspace via "devtool add"
or "devtool upgrade" then the recipe file itself will be in the
workspace layer; if you run "devtool update-recipe" particularly in the
upgrade case you might think it would apply the upgrade to the original
recipe, but it will not - in order to remain consistent it has to update
the recipe that's in use which is the one in the workspace. Warn the
user in this situation so that they know what they need to do when they
are finished.
(From OE-Core rev: 4801b64243e57e554a593f0857dd53621d8f52e5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow the _add_md5() function to be called with a directory in order to
recursively add the files under it. Additionally, we need to skip
preserving empty directories (since directories aren't listed in the md5
file).
(From OE-Core rev: 9383af78adc854a6f6de8b1520edf3cea0c477a6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Only poky sets SDK_NAME to include ${IMAGE_BASENAME} (i.e. ${PN}), so we
can't assume the buildtools filename will include it here. Change it to
look for a file with "buildtools-nativesdk-standalone" in the name
(the buildtools-tarball recipe itself sets TOOLCHAIN_OUTPUTNAME to
include this.)
(From OE-Core rev: 78ea4fcdea468888c0faef22a95dea7015a91df2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have the command in the commit message, we might as well have the
build result as well (succeeded/failed and whether or not it was
interrupted by the user). The interrupted part relies upon a change to
BitBake to extend the BuildCompleted event to include an attribute for
that, but will not fail if the attribute is not present.
(From OE-Core rev: 6ca7f5c0d0f024ae5d21368188e3428534cab2a4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We do already commit these into the repository itself, but have them in
the commit message as well as a reference. As part of this, refactor out
running "git commit" into a separate function so we don't have to
duplicate the code in the two places we call it.
Implements [YOCTO #7966].
(From OE-Core rev: c77a068e534b274dee7fb8a2399ecafd33da0587)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Avoid an error when attempting to remove previous data if it's not a
subdirectory - we were assuming that anything that wasn't named "latest"
or "latest_srcrev" had to be a directory. This makes it possible to have
a buildhistory_emit_pkghistory_append which writes additional files at
the recipe level.
(From OE-Core rev: b018a046a578e41b105cf72b9fdeed0220ae3046)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we want the correct file permissions to show up here as they would on
the target, we need to run the command under pseudo. Normally we'd set
the fakeroot varflag on the function and that would be enough, but it
turns out that setting fakeroot on a non-task function that you execute
using bb.build.exec_func() isn't working at the moment. Work around this
by simply using FAKEROOTENV and FAKEROOTCMD. Unfortunately that means we
have to duplicate the command for the two cases but I couldn't find a
better means of doing that that actually works.
(From OE-Core rev: 1380aa333ed90559f4a24d52aefc52cadb60646c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Avoid using ${...} for shell variables (since they could be
expanded as bitbake variables if present)
* Use files-in-package.txt rather than files-in-<packagename>.txt; the
file is already in a subdirectory named with the package name and this
naming is consistent with that of files-in-image.txt.
(From OE-Core rev: 6f3992728613c39403ef59bbcf1cb67d9e8c526b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't care about the permissions on the top-level directory in which
the files are contained, just everything under it; this also avoids
lists with just this entry in it for empty packages. Affects file
listings for both images and packages.
(From OE-Core rev: d503122646b1b148113a4be31b7e88723459dae6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If there aren't any changes, we still make a commit to the buildhistory
repo, but this wasn't being pushed if BUILDHISTORY_PUSH_REPO is set.
Move the push to the end to make it unconditional.
(From OE-Core rev: af2fc914975d06fdd75a65d4b00cf9f5c1610528)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running "git add" on a directory with older git versions, deleted
files aren't added to what is to be committed unless you explicitly
specify the -A option. The result of this is that when applying a patch
from a recipe which doesn't apply with "git am" (and we fall back to
applying the patch through other means then "git add" following by
a "git commit") these deletes weren't committed with the patch, leaving
them sitting deleted but not committed at the end.
This should fix test_devtool_modify_native (which unwittingly exercises
this scenario by attempting to run "devtool modify" on apt-native) on
machines with older git versions.
(From OE-Core rev: 367ffba394bb815d776f48a367d5d7e5ea9b3bba)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow any listeners for this event (such as buildhistory.bbclass in
OpenEmbedded) to find out if the build was interrupted rather than
completing normally. The value will be 0 if not interrupted, 1 if
interrupted waiting for remaining tasks to complete, or 2 if force
interrupted (stopping any running tasks immediately).
(Bitbake rev: df2b778efd2ecc48f6c5a3ed446f6459f2250035)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're importing the server and UI modules in order to check they are
valid, but it turns out that that has some nasty side-effects. We don't
actually need to do this except when --help is passed or the module
doesn't exist, so rearrange the code so that we only do the module
listing in those two cases.
Additionally, let's just go ahead and catch all errors on import; we
really don't care to have them cause a failure now.
(Bitbake rev: c9dc6d9c86e8b887821a6d00346bd0b09e1da97c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having disabled phonon by default in Qt4 we need to ensure this
packagegroup is still buildable when it's not available.
(From OE-Core rev: 9f1ce43fa1bdd3952ccf510cb648ebe9e0e8aac2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* PR = "r0" is the default, no need to set it (and most people won't
need to set it anyway with the PR service)
* PV = "0.1" is already set by virtue of 0.1 being in the filename
(From OE-Core rev: 91b323526427f67a8982c7b090e505aaf810d4e7)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I'd somehow missed the existence of fail() when I wrote these. It's
preferable here so you don't get the somewhat useless "false is not
true" message in the case of failure.
(From OE-Core rev: 173a5896fff57136e1f15e15f90961416aadde94)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have "recipetool create" (or "devtool add" which wraps it) instead
which works pretty well now, is written in Python and thus can call
into BitBake/OE code easily, and can do things that create-recipe can't;
any future improvements should go into recipetool and therefore I think
we'd rather people used it instead.
(From OE-Core rev: df0ef260f36df3993377196962eb6d52ab36b38c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a -i/--inherits option to filter the output to include only recipes
inheriting the specified class(es).
Implements [YOCTO #7475].
(Bitbake rev: e4ec622d9dc5f882049f5e8027331ec1e17681de)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake-layers needs to map recipe and class files to the layer they
came from within the show-recipes and show-overlayed commands. However,
it turns out that mapping a file to the layer it came from is not as
trivial as it might seem. To do it properly we need to match the path to
an entry in BBFILES then map that to the collection name using
BBFILE_PATTERN, then map that to the actual layer using variable
history. If it doesn't match any entry in BBFILES, then we can fall back
to BBFILE_PATTERN (to handle classes and conf files).
This fixes the layer name not showing up properly in the output of the
show-recipes and show-overlayed commands for recipes in layers such as
meta-intel that have subdirectories in BBFILE_PATTERN. It also fixes the
priority not showing up in show-layers for such layers.
As part of this I've added a function to VariableHistory which for a
space-separated list variable gives you a dict mapping the items added
to the files in which they were added. I've also fixed
bb.utils.get_file_layer() and reduced some of the duplication by using
this function in bitbake-layers. Also fixes the priority not showing up
for layers such as meta-intel
Fixes [YOCTO #8160].
(Bitbake rev: e852f6cabd7489585477ab567a1afeb2252377ac)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the specified layer isn't a path, then just match on the directory.
Fixes [YOCTO #7839].
(Bitbake rev: b4c45bf6c42b4d319ba868f4ce77e86c8b585818)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Provide us with a means of showing the list of UIs / server choices for
the command line help, and do the processing in one place for both.
(Bitbake rev: 24035c1daad5a904c3372d21d44191ee8182338f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a file no longer exists, drop it from the cache silently instead of
generating a traceback. This was visible in some cases when a recipe was
deleted when bitbake was resident in memory.
(Bitbake rev: fe105b9042bdac4afd9f38fcf92bfdc2c04ec23f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's best practice to use "with open..." rather than open & close where
practical.
(Bitbake rev: 1ef38549cae5639f2c8bcc2b270c5c82a5e29e3c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now we have the bbappends list and all users have been converted over to
use it, we don't need this anymore.
(Bitbake rev: 279770c42d4c63aa2cebce331b55a92a564b50ac)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whilst collecting this list on the fly may be quicker, doing so within a
function that's meant to *query* the list of bbappends is poor practice.
(Bitbake rev: 5c12aa3b0010d7d1733e54a0ca7d0af465454210)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we can't connect to the server we should error out, because it might
not be that the server is actually dead - it might just be unable to
execute commands.
(Bitbake rev: d4b921676859d6ba4e1922fa4682ee941652f483)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The prefile/postfile options weren't working in memory resident mode
because they weren't being passed through to the server, so ensure that
they do get passed through and that the server is reset when the values
come through.
(Bitbake rev: a3f7dc042fc7b1c308bfd248431930eb8ba50326)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unfortunately we were acting on inotify notifications about any files
changing within the watched directories, not just the ones we actually
care about. In OE the build directory is in BBPATH and hence it gets
watched, and we write things like bitbake.lock and
bitbake-cookerdaemon.log to that directory, hence effectively
notifications were being tripped on every bitbake invocation. To avoid
this, record which file/subdirectory we're interested in against each
watched directory so we can ignore any events for files/subdirectories
we don't care about.
Additionally, if we move up to the parent dir, ensure we haven't already
seen it before adding a watch on it (we were previously calling
watcher.add_watch() on the same directory multiple times before in a
typical OE configuration).
(Bitbake rev: bc39b8da34c046b629c43fd0a8cac2efbf1c060f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If sys.exit() is called within a command run over XMLRPC, the XMLRPC
server is effectively trashed (apparently listening but no longer able
to respond to commands). We need to intercept the SystemExit exception
and deal with it as we would any other exception.
(Bitbake rev: 95e391acbc3b4efd6c77637a1ce815012ae0f09b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Let's have the caller do this and then the function is a bit more
flexible (e.g. we can choose to parse with bbappends or not); fix up
calls to this function appropriately (of which there are only two, both
within devtool).
(From OE-Core rev: 6ba969f1ac5a1a0e277a21287fc5ae1622a6b14e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a recipe is BBCLASSEXTENDed (e.g. to -native), its PN value and the
name of the bbappend will be different; we were assuming them to be the
same when reading in the workspace, leading to us seeing the base recipe
name everywhere afterwards.
Also add a test so we ensure this doesn't regress in future.
Fixes [YOCTO #8157].
(From OE-Core rev: b63fca00c2e24ad0c8b8b3c492d93ee4372fa92d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure that the "patches" subdirectory is removed from the right
location when S == WORKDIR (e.g. devtool extract makedevs).
(From OE-Core rev: 2062c88726400e09599aff51af95799a866b90c9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In case the user has set up externalsrc outside of devtool, force
EXTERNALSRC to blank for the recipe when extracting so that the original
source URI is still in SRC_URI and we're still able to extract it. (This
isn't a problem with devtool itself because the bbappends within the
workspace layer that apply externalsrc are explicitly filtered out when
devtool parses a recipe).
(From OE-Core rev: 5be16d639d1b78d114755bfd552ac901d0fdf4a9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't need to run bitbake -c cleansstate on this because it's being
cleaned out as part of "devtool reset".
Addresses [YOCTO #8031].
(From OE-Core rev: fb0e372bed321998d12460282ad8b30203a871a9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tinfoil now has its own shutdown method, use it instead of calling into
cooker - not only is it the right thing to do from an API perspective,
it also ensures proper lock handling.
(From OE-Core rev: c1f58caad4ff45efd213bf1412304a72903abe7e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These are the gummiboot tests recently removed from OE-Core, with the
following changes:
* Make them use MACHINE = "genericx86-64" instead of "nuc" and remove
all usage of meta-intel
* Use DEPLOY_DIR instead of assuming build directory will be in the
default path relative to COREBASE
* Fix build test to actually check if the file was created as part of
the test instead of possibly being around from a previous build
* Fix wic test to actually check the output to find the name of the
.direct file that wic has created, instead of passing if any .direct
file happens to exist in /var/tmp/wic
(We can now add these in this layer courtesy of Chris Larson's OE-Core
commit 182b8ed9f26ed8b6a8eb6dcaec82db9aca14b010).
(From meta-yocto rev: 8e945e39327869c6b53959c5eb324ba3e4cb0b20)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton