This script expected the port number to be specified first, which is
somewhat counter-intuitive especially if you're used to
oe-init-build-env; besides, in local usage you are unlikely to need to
specify a custom port. Given that few people are using this yet (based
on the issues I have recently fixed), switch the arguments around so
that the two scripts behave consistently.
(From OE-Core rev: 5d8f4c46329fbd38ebd034c1e32aa49dfec5f529)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's possible for tasks to stage symlinks that point to non-existent
files; an example is ncurses-native.do_populate_sysroot. There wasn't
any error checking here so this broke the build when "task" was included
in BUILDHISTORY_FEATURES. In any case we shouldn't be following symlinks
and getting the sha256sum of the link target - we need concern ourselves
only with the target path, so check if the file is a link and sha256 the
target path instead if it is. If it's neither a regular file nor a
symlink (perhaps a pipe or a device), just skip it.
(From OE-Core rev: f60520d97f53dafe783f61eb58fe249798a1e1be)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These were all used by support code for the Hob UI which has been
removed - nothing will currently fire or respond to these events.
(Bitbake rev: 03ba8db0595723286ad462fa197f862e1efdf0de)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If an event handler for bb.event.BuildCompleted fails, we still need to
call finishAsyncCommand() or else BitBake will just exit immediately
without showing any error summary, or worse in the case of memory
resident mode BitBake will hang and if you Ctrl+C to break out, the
command won't be marked as finished which means that no further commands
will be able to be executed until the server is manually restarted.
(Bitbake rev: 5639faa3eef55cc476a82e810e61ca228cbdf221)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The main point of memory resident bitbake is to avoid loading data
unnecessarily on every bitbake invocation. Unfortunately the code that
updated options from the UI was simply treating the fact that either
of the "prefile" or "postfile" options were in the list of options
passed in as an indication that the configuration was invalid, which was
bad because these are always passed in. We only need to mark the
configuration as invalid and thus reload it (and thus reload the cache)
if the option value has actually changed.
At the same time, the recently handled "tracking" option needs to be
treated in a similar manner since the configuration needs to be reparsed
if that has changed. Also, add a few extra debug messages to aid
debugging this code in future.
(Bitbake rev: 18dfd144d0da6ff662308ce4f9d135dc11412133)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We put the bitbake command line in the buildhistory commit message for
informational purposes, but calling sys.argv to get that as we were
previously doesn't really work in memory resident mode - that gives you
the command used to start the bitbake server which is much less
interesting. Use the just-introduced BB_CMDLINE variable instead.
Part of the fix for [YOCTO #11634].
(From OE-Core rev: 1a6424ee4c865601ff324e9599a2f48c9e6723ee)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
tinfoil sets up its own logger by default, but if and when we initialise
the UI (by default knotty) will also set one up, leading to duplicated
messages specifically from tasks. To avoid this, rather than adding some
kind of parameter, just check if there is already a logger outputting to
stdout/stderr and if so, skip adding our own.
Part of the fix for [YOCTO #11275].
(Bitbake rev: 66d866745f35468d1540a793d07e3a401298b84b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding an additional logger in setup_bitbake() interacts poorly with the
logger we have added by default in tinfoil's constructor, with the
result that messages may be doubled or even tripled in tinfoil-using
scripts. Disable adding this one when calling setup_bitbake() from
tinfoil to avoid this problem.
Part of the fix for [YOCTO #11275].
(Bitbake rev: 8a5bae76f91f2411187c638a42fa3c762052cf11)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop a number of the commands and support functions that were created
solely to support functionality in the now-removed Hob UI. In most cases
we now have this functionality elsewhere e.g. to modify config files and
recipes we have bb.utils.edit_metadata_file() and friends, and in OE we
have oe.recipeutils.patch_recipe_file() and friends which build on top of
the former. Additionally, some of it represented pretty egregious
incursion of OE metadata-specific references into BitBake code.
For now I have left in the find*File functions and commands as they are
relatively generic and possibly still useful, but they might be removed
in future.
(Bitbake rev: a322f13183c66a28d93cc4bc3d839d95fc1d90f4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In OpenEmbedded's buildhistory class we want access to the bitbake
command line that launched the build, and up to now we were simply using
sys.argv from within the event handler to get that. Unfortunately that
doesn't work in memory resident mode, since the event handler is
naturally executing within the server and thus will give you the command
that launched the bitbake server which is much less interesting. Add a
dynamic variable BB_CMDLINE to provide access to this, set from sys.argv
within the UI process in updateToServer().
(Note that BB_CMDLINE isn't currently passed through to the worker, so
this is only really readable from event handlers plus any explicit
getVariable calls - in theory an observe-only UI could read it for
example.)
Part of the fix for [YOCTO #11634].
(Bitbake rev: 8ad31a3eff5cfcb7b1c462578a582dafcbc426d4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we set these flags here then we can not only tidy up some of the code
that calls initConfigurationData() (and reset() which simply calls the
former), we also avoid calling initConfigurationData() an extra time
unnecessarily during startup (since setting baseconfig_valid = False in
updateConfigOpts() also resulted in initConfigurationData() being
called from updateCacheSync() at the end of the command.)
(Bitbake rev: 74d2ef99a920aa9e70470ba8e600ea973a3c672a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you build a recipe, modify one of the files referred to in SRC_URI,
then build it again, it should re-run do_fetch and the tasks that depend
upon it to incorporate the modified file. With memory resident mode this
was not working unless you restarted the server, because the mtime of
the file was cached and thus we never checked the actual file. Clear out
the mtime caches in buildTargets() to ensure this isn't an issue. I did
a minor refactoring to break this out to its own function since we were
already clearing out the build mtime cache here.
(The test I used for this was . ./oe-init-build-env-memres,
bitbake -c fetch mdadm, modify the comments at the top of one of the
patches referred to in the recipe, and then bitbake -c fetch mdadm again
and it should re-execute the fetch task ("... 0 didn't need to be
re-run".)
Fixes [YOCTO #10732].
(Bitbake rev: fe4f3487368bc35043a1ba464e1c69d2c9cbefc6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
CookerConfiguration sets the "tracking" option to True when -e is
specified in order to have history tracking enabled in the datastore
so that we can show variable history (which isn't enabled by default for
performance reasons), however in memory resident mode this wasn't doing
anything because it was happening on the UI side only. We do have a
mechanism for updating the cooker configuration in the server, but the
tracking option wasn't being included in the list of options we updated,
so we just need to add this option to fix the issue.
Fixes [YOCTO #10730].
(Bitbake rev: 57d4977555cf892b15dd0302dfe261fe37d49327)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Way back in OE-Core commit fba198ac7efe476a25c5761878ef2fcee97bf9f1 in
2012 we split committing to the repository, making a commit per
top-level directory. However, as we add more information it becomes
harder to see which commits belong to which build. Switch back to a
single commit per build to keep the history tidier.
To address the original concern, if you do want to see just the changes
for a particular subdirectory, git can filter that for you - just
specify that subdirectory as the last parameter on the git show / git
diff command line and that's all you will see.
(From OE-Core rev: b49a4a47783609fe9161fbc11cc7c7ff3ff4b6bb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since recipe-specific sysroots were implemented, devtool add and devtool
upgrade operations that fetch from a URL that requires native sysroot
dependencies will fail to work as there is no recipe-specific sysroot
set up for them during fetching. An example was any URL pointing to a
tarball compressed with xz, e.g. devtool upgrade on gnutls.
The most expedient way to fix this is to set up a dummy recipe-specific
sysroot to use for the fetch/unpack operations. We do this in the same
manner as bitbake -b does, so we're just taking all of the sysroot
components available and creating a sysroot from those rather than
ensuring the correct dependencies are there - this means that we're
still going to have problems if e.g. xz-native hasn't been built yet,
but that issue will be trickier to solve and is tracked separately.
Fixes [YOCTO #11474].
(From OE-Core rev: 559151e783759af78b5cdd76cdbb9ce325a391e6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For git repositories in the absence of any other indicator, it's not an
unreasonable assumption that the name of the repository is the name of
the software package it contains, so use that as PN if we don't have
anything else.
(From OE-Core rev: ef73fa70f0955912b0da140922465a3c817424e9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a value we extract from a spec file contains an unexpanded macro
(e.g. %{macroname}) then we should discard it since we're not seeing the
actual value and we don't have an easy way of expanding it at the
moment.
This fixes for example getting %{name} as the recipe name when running
the following:
recipetool create https://github.com/gavincarr/mod_auth_tkt.git
(From OE-Core rev: eee56a19cda051da6267f808cd3a04a4c644acb3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you have, for example, no value set for a variable VARIABLE and a
then VARIABLE_pn-something = "value" and then you parse something.bb,
you expect getVar('VARIABLE') on the resulting datastore to return
"value", but the code here assumed that if the variable wasn't set
without overrides then we didn't need to return the overridedata and
thus we didn't get the overridden value.
In OE this affected the ability to get RECIPE_MAINTAINER for a recipe
in a script using tinfoil (since this is only set from an inc file with
_pn- overrides for each recipe, and no default is set).
(Bitbake rev: b3d2c9917c5fd8278878328794daa107ddf79b64)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to have a shared sysroot usable within the eSDK after recipe
specific sysroots were implemented, we need to run
bitbake build-sysroots as a separate call. However, unlike the first
call, --quiet wasn't being specified and that somewhat undermined the
earlier effort to clean up the eSDK installation output. Make this
second call quiet as well so that the output is tidier.
(From OE-Core rev: 56b73788edaa0796e53f1a30e9ebdb2ae85b1646)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, in order to figure out variable values when run within the
eSDK, runqemu does not use the standard SDK method nor is it able to run
bitbake (since the eSDK environment isn't initialised like the normal
OE build environment). runqemu really ought to be fixed, but the quick
workaround is to set DEPLOY_DIR_IMAGE in the environment so that runqemu
can find image files.
Fixes [YOCTO #10447].
(From OE-Core rev: 1ef833b6393366a10f4bb65df89725ad65761386)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were specifying a default parameter; the get() function defined here
does not take such a parameter. I appears this code had not been tested.
This fixes runqemu erroring out immediately when used within the eSDK.
(From OE-Core rev: e4548531112c824653ae42b9bcc335a7ca8588e0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a non-existent event - we already have the actual
bb.build.TaskSucceeded further down in the list hence why it wasn't
noticed earlier.
(From OE-Core rev: 4e059a5ceb6f44401154e89e37f56de1d664a7cb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a task such as do_fetch fails when we're extracting source for a
recipe (within devtool modify / upgrade / extract / sync) then we should
naturally stop processing instead of blundering on; in order to do that
we need to be listening for the TaskFailed event. Thanks to Richard
Purdie for noticing and fixing this.
(From OE-Core rev: 9174b845bf6a6be7753bf6b921959b1f3f2dcbc0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Most of the other extract-based commands have this option but oddly I
left it out for modify - I guess because if I was debugging an issue here
I just used devtool extract to do so, but there's no reason why we can't
have it here and it is useful.
(From OE-Core rev: 98fbc46e1a51237213bd7825a922389d3ab2ad9b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If devtool is called with a URL to a source repository containing a
node.js module, we don't know that until recipetool has fetched it, and
due to the structure of the code we have to exit with a special code in
order to let devtool know it needs to build nodejs-native. We also want
to suppress the error message that recipetool would normally print under
these circumstances; there is already a mechanism for this but it wasn't
operative in the case where we're pointed to a source repository rather
than an npm:// URL, so create some plumbing so that we know to hide the
message.
(From OE-Core rev: 0c2d0fbb1c6c5b82183799eb7ef80074f86bcfc4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If recipetool returns with exit code 14 this means devtool needs to
build nodejs-native and then call it again. If recipetool returns exit
code 14 again then clearly something has gone wrong and we should just
quit with an error.
(From OE-Core rev: 8d7cced6e06d7c2037f5ab75ac859f501129532e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The change over to recipe specific sysroots means that we can no longer
get a known location simply from configuration for the npm binary - we
need to get the recipe sysroot for nodejs-native, look there for npm if
we need to check it's present, and add that to PATH when calling out to
npm. Unfortunately this means anywhere we need to get that path we have
to have parsed all recipes, otherwise we have no reliable way of
resolving nodejs-native. Thus we have to change recipetool create to
always parse all recipes (the structure of the code does not allow us to
do this conditionally).
In the worst case, if npm hasn't already been added to its own sysroot
and we are fetching from a source repository rather than an npm
registry, this gets a bit ugly because we end up parsing recipes three
times:
1) recipetool startup, which then fetches the code and determines it's
a node.js module, finds that npm isn't available and then exits with
a specific error to tell devtool it needs to build npm
2) when we invoke bitbake -c addto_recipe_sysroot nodejs-native
3) when we re-invoke recipetool
This code is badly in need of refactoring, but now is unfortunately not
the time to do that, so we're going to have to live with this ugliness
for now.
Fixes [YOCTO #10992].
(From OE-Core rev: acfdbd796c99882b8586023c8c6b848716105c8d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
OE-Core commit c0cfd9b1d54b05ad048f444d6fe248aa0500159e added handling
for AND / OR in license strings coming from npm, but made the assumption
that an & would always be present in the license value. Check if it's
there first so we don't fail if it isn't.
(From OE-Core rev: abe2955df2dc558de6068d9373dfcb47d690704b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Save a file per task listing sha256sums for each file staged, i.e.
the output of the task. Some caveats:
1) This only covers sstate tasks since it uses SSTATEPOSTUNPACKFUNCS,
however those are generally the most interesting in terms of output
anyway.
2) The signature is taken before applying any relocations, so any
relocated files will actually have different signatures, but that's
churn that you probably won't want to see here.
3) At the moment if you run the same build twice without sstate you will
very likely see changes in the output for certain tasks due to things
like timestamps being present in the binary output. Fixing that is
a general Linux ecosystem problem - see this page for our efforts to
resolve it on our side:
https://wiki.yoctoproject.org/wiki/Reproducible_Builds
NOTE: you need to set your BUILDHISTORY_FEATURES value to include
"task" to enable collection of these signatures as it is is disabled by
default.
(From OE-Core rev: ca5d1273432e20059ab66d721a9eb314a54e81e7)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Append to the value with appendVarFlag() instead of setting it outright,
so that we can also append to it in other places. Accordingly, this
varflag is pipe-separated (since we want to be able to exclude any
string fragment, in this case including the leading space), thus put a
leading pipe character to play nicely with any existing value.
(From OE-Core rev: a147838c3dfd4c53084a19b052b8d4e183293412)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the code underpinning bitbake-diffsigs to add an option to
buildhistory-diff to determine and display the differences between the
actual signature inputs, with a twist - we collapse identical
changes across different tasks, showing only the most recent task to
have that difference, meaning that there's less noise to wade through
when you just want to know what changed in order to cause some
rebuilding you're seeing.
(From OE-Core rev: 86cb4b01f2020553902554e512c02147eb4e0f51)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having added writing out of the task signature list to buildhistory
(when BUILDHISTORY_FEATURES includes "task"), we now need a way to
compare the list. This just shows which tasks have been added / changed
signature / removed.
(From OE-Core rev: 63bd7e9f780a98dda458d612877495756bcc5463)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we want to determine what changed since the last build, one angle
from which to look at it is to check the signatures. However, if we
don't actually have the signatures from the last build we don't have
anywhere to start. Save the signatures on each build in order to give us
the starting point.
NOTE: you need to set your BUILDHISTORY_FEATURES value to include
"task" to enable collection of these signatures as it is is disabled by
default.
(From OE-Core rev: 11f68f65c46c5bc26ddeeade3021e83b3a7f895a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the cwd is named "buildhistory" and the user hasn't specified an
alternative path on the command line, then assume that the current
directory is the buildhistory directory. This makes it easier to run
buildhistory-diff and also interact with the buildhistory git repository
as you no longer have to jump into the buildhistory directory and up to
the parent again when doing so.
(From OE-Core rev: e4ccec2e4c2f521a2bb473083b42aefd494eea23)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the move over to argparse we've made the two signature file arguments
optional and thus if -t is not in use we need to explicitly check if at
least one signature file has been specified - and if not, show an error
and the command-line help.
(Bitbake rev: 9011366689b26305281fcb2d412dcacece917e18)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In find_siginfo(), which is used by bitbake-diffsigs among other things,
avoid adding a siginfo file from the sstate-cache where we've already
collected a sigdata file from the stamps directory with the same hash.
This avoids the possibility that the top two files (as picked by default
using the bitbake-diffsigs -t option) are for the same signature and
thus the tool would report no differences. In order to do that, just use
the hashfiles dict that we already have - we just need to change the
code to populate that even if we're collecting matching files without
looking for a fixed set of hashes (i.e. taskhashlist isn't set).
This replaces previous code in bitbake-diffsigs that attempted to filter
these out with limited success.
(From OE-Core rev: 5b69eef40868180c59400624096d7ebbbbea446b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When comparing signatures with bitbake-diffsigs -t or bitbake -S
printdiff, we use this find_siginfo() function implemented in this
module to find the siginfo/sigdata files corresponding to the tasks
we're looking for. However, native sstate files go into a
NATIVELSBSTRING subdirectory and there was no handling for this when
asking about native recipes.
I'm not even sure why we were walking SSTATE_DIR in order to find
this - we don't need to, we just need to run glob.glob() on the filespec
we calculate, which should be a little bit more efficient.
(From OE-Core rev: 8cb472e4ed25e56ec0d9cf6d8d101d1ab6687a5b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the output is a TTY, add colour to the output in order to make it
easier to read. At the moment this is fairly basic, just add colour to
the "titles" of each change and to the diff output.
I tried to introduce this without changing the code too much - rather
than moving everything over to the new python formatting style, I've
introduced a color_format() function which takes care of the colour
formatting, either accepting additional format arguments or
alternatively leaving the caller to use the old-style formatting (%) to
insert values.
(Bitbake rev: 04a023c8fdea1e1812fcdcaf00345aab59f9abe1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a variable value has changed and either the new or old value contains
spaces, a word diff should be appropriate and may be a bit more readable.
Import the "simplediff" module and use it to show a word diff (in the
style of GNU wdiff and git diff --word-diff).
Also use a similar style diff to show changes in the runtaskhashes list.
I didn't use an actual word-diff here since it's a little different - we
can be sure that the list is a list and not simply a free-format string.
(Bitbake rev: 20db6b6553c80e18afc4f43dc2495435f7477822)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we just want to drill down to the actual differences then we don't
need to see certain things in the output, e.g. basehash changing or the
signature of dependent tasks. This will be used for comparing signatures
within buildhistory-diff in OE-Core; the default mode as used by
bitbake-diffsigs and bitbake -S printdiff remains unchanged for the
moment.
(Bitbake rev: 6543a59b1ebd3194a7c6421cffc66ebe31a67c62)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the -t option which recurses to find the ultimate cause of a
signature change, it was hardcoded to take the last two executions of
the specified task. On the other hand, if you have two specific task
hashes (say from bitbake output, or some other tool) then you'll want to
pick those, so provide an option to specify those as well. (Note, the
new -s option needs to be specified alongside -t rather than instead of
it.)
(Bitbake rev: d9813b1a4223cf8dc80cab90e467ddf4bf8d1078)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Argparse is a bit easier to deal with than optparse, and since we're
about to add some options, migrate this script over.
(Bitbake rev: 7f130e0b5ce6cfc6b35176465f260092cd3b3d64)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the runtaskdeps list hasn't actually changed (but the signatures of
some of the tasks did) then it doesn't make sense to print out the old
and new lists as they are both the same and may be very long, e.g. for
do_rootfs in OE.
(Bitbake rev: cb170543605288b3e8badfac3a54c588f4c95413)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When dumping changes to signatures e.g. output of bitbake -s printdiff,
if for example a function has changed, it's much more readable to see a
unified diff of the changes rather than just printing the old function
followed by the new function, so use difflib to do that.
Note: I elected to keep to one item in the returned list per change,
rather than one line per line of output, so that the caller can still
look at changes individually if needed. Thus I've added some handling to
bitbake-diffsigs to split the change into lines so that each line is
displayed indented.
(Bitbake rev: 4d254ae63a35231c98e3f73f669b040ed1144042)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This logic doesn't work in practice, certainly not with current versions
where sigdata files are preserved in the stamps directory and therefore
there will often be multiple sigdata files - you can now easily get
files for the same signature from sstate and the stamps directory with the
result that bitbake-diffsigs reports nothing has changed. Instead, let's
change the find_siginfo function in OE-Core to simply not return
duplicates so we don't have to filter them out here.
(Bitbake rev: f0d7ab259d8ef95643e7229474b7850608aa4426)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If just one of the two signatures we want to compare aren't available,
report that one rather than misleadingly claiming both are missing.
(Bitbake rev: c87764b9147792a10efad3ed5378f36f0a055bc6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Printing "pbzip2pbzip2_1.1.13.bb" is ugly, we need to add a separating
slash so that we get "pbzip2/pbzip2_1.1.13.bb" instead.
(Bitbake rev: 55cd4045a37afc954f4d5091f524756b266064fc)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We weren't picking the right files to compare here - according to the
order in which the list is sorted (by mtime), we need to be taking the
last two items and not the first two.
(Bitbake rev: 99f49b56115b1f8d1a0a0b911da62ffd1f997b5f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python-pyqt in meta-qt4 is about to require sip 4.19 that only meta-oe
provides; as we only bring in meta-qt4 for LSB, there's no need to be
building this as part of world and it would fail without meta-oe anyway.
(From meta-yocto rev: 202598d670a1d335d2a8de7685244fa7f480bd26)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a few comments at the top of the file explaining what it's for, and
a comment pointing out that you need to increment the cache version when
changing any code that changes the output.
(Bitbake rev: c1780bc1872bb35bc28c76eff9110717f02d9db2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add some tests to verify that we are extracting "contains" information
from python expressions in the code in the bb.data and bb.codeparser
modules.
(Bitbake rev: 88fda492df875dd79b7aecf1f34b38517fc1eb33)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure we handle bb.utils.contains_any() as separate items, rather than
how we handle contains() where every item must be in the list.
Additionally, enable handling bb.utils.filter() which for the purposes
of looking at dependencies is the same as contains_any().
Additionally bump the codeparser cache and recipe cache versions to
invalidate the user's existing caches (ensuring that the changes take
effect and avoiding "taskhash mismatch" errors respectively).
(Bitbake rev: 496e3c84820a2a889d99d3604659e47a550941d5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code that determines variable dependencies uses the codeparser to
find references to "contains" type operations e.g. bb.utils.contains().
That function can take multiple items to check, and all specified items
have to be present. However this code didn't handle that - it assumed
that only one item would be specified and thus it was treating the
multiple items as a single item with spaces in between. Split the value
and check if all words are present in order to determine whether the
check is "set" or "unset".
(Bitbake rev: 272f1245acdd4fb85cb78612aa03627a9c246d8f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Turn reasons from a list into a string (usually there will be only one
reason, but the interface provides for more than one) and state up
front that the recipe is unavailable for clarity
* Use quotes around invalid recipe name
(Bitbake rev: 8922f1d23400049d2894a97915a533769a24ca07)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This function calls cooker.findBestProvider() but didn't handle the fact
that that function returns a tuple (None, None, None, None) when there
is no matching recipe. (This fixes devtool in OpenEmbedded showing a
traceback when an invalid recipe is specified instead of a proper error
message.)
(Bitbake rev: 54a4757ca706afc6e98c7692f960592e80cab12b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
BufferedLogger was sending log records to the target logger without
checking if the logger is enabled for the level of the record - and
handle() doesn't check this either (it's normally checked earlier when
the relevant log function is called e.g. logger.debug()), leading for
example to debug messages from codeparser getting printed when the log
level for the main BitBake logger was set to logging.WARNING.
(Bitbake rev: 968a77388dd1a24c1dadec6ce49bf53ebb5b643f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you had a script that uses tinfoil and it failed to connect to the
BitBake server, you did't see any of the expected messages - this was
because client-side logging wasn't being handled at all. Since you'll
almost always want this when using tinfoil, have it use the new
bb.msg.logger_create() function to enable client-side logging by
default.
Relates to [YOCTO #11185].
(Bitbake rev: 824e73e0f3eaa96b4d84da7e31f9a17ce5c5d7ee)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We use this code to set up a logger with colour in a number of different
places, so create one function that does this and make some of bitbake's
utility scripts use it.
(Bitbake rev: b1ba7d1cc8ec33e2d081230287abd07f52136097)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We call git ls-remote to get the latest revision from a git repository,
however by calling runfetchcmd() we can end up recursively running
git ls-remote a number of times with OE e.g. if ${SRCPV} is in PV, ${PV}
is in WORKDIR, and ${WORKDIR} is in PATH (as a result of recipe-specific
sysroots), our call to runfetchcmd() exports PATH so _lsremote() will
get called again - with the end result that we run git ls-remote 30
times in quick succession (!). Prevent that from happening by using a
guard variable and returning a dummy value if it's called recursively.
Fixes [YOCTO #11185].
(Bitbake rev: ff1ccd1db5d70b3fc9ad0d3e8f3d7b804c22bf36)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The data we read from an HTTPConnection comes in the form of bytes, but
we need it as a string, so in Python 3 we need to decode it (missed in
the Python 3 migration).
(Bitbake rev: 7e6a3cd1472e1a1c8304b46611e1676914a68b59)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enable the following operations from a remote datastore to affect the
other end:
* setVarFlag()
* delVar()
* delVarFlag()
* renameVar()
In practice I don't expect these to be used much, but they should be
present so that the implementation is at least reasonably filled out
and that the tests pass.
Also add tests for the interface, mostly by subclassing the existing
local test classes so that they are using a remote datastore. (These
don't actually test remote usage via tinfoil, just that the
datastore's interface can be used.)
(Bitbake rev: 282dc0719d22a39df746eea762ebe05c66aa8f8a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There was a huge gap in the remote datastore code introduced in the
tinfoil2 rework - we weren't handling overrides at all, since these are
stored separately from the actual data in the DataSmart object. Thus,
when a datastore actually represents a remote datastore we need to go
back to that remote datastore to get the override data as well, so
introduce code to do that.
To avoid a second round-trip I had to modify the _findVar() function to
return the override data as well. This will increase the overhead a
little when that data is superfluous, but without making the function
even uglier I don't think there's a way to avoid that.
(Bitbake rev: 4f9d6f060ed247fb6fa2f45668a892a1788d3f91)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the tinfoil2 refactoring, if an error occurred during parsing, we
were showing a traceback and not correctly exiting (since we weren't
calling shutdown()). Fix both of these issues.
(From OE-Core rev: 18304036e1b513fd12c049dbf549ba75c503ed84)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A recent patch to bitbake fixes these datastore operations so that they
actually affect the server end, so we should test that they work.
(For full disclosure, some of these tests would probably pass without
those fixes, since the operation would be done on the client side
instead - but we are at least exercising the code paths.)
(From OE-Core rev: 4eb3c705b9cadccacdb191ae89f5242a00f397f3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's regrettable that code in here is referring to BBFILE_COLLECTIONS,
but it is, and the result is that this packagegroup will rebuild simply
by adding or removing a layer which may be completely unrelated to
whether you've got meta-qt4 or not. Add BBFILE_COLLECTIONS to
vardepsexclude for the function to avoid this. (This was flagged up when
testing the new yocto-compat-layer script - it understandably but
undesirably reports it as a change caused by a layer).
(From OE-Core rev: 10dc69467677a4ce2a26aa670bbb9c1d7d783a0e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adjust the message slightly to make it clearer, in particular mentioning
the HOSTTOOLS variable.
(From OE-Core rev: e022b27d189c1dcc7b3baea2b99dba8d724e6e2b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The runqueue expects to be able to run 'bitbake-worker' from PATH, but
for example in the OE extensible SDK, tinfoil is used within devtool
where this isn't the case (we're not exposing bitbake to the user, thus
bitbake's bin directory isn't in PATH).
This fixes devtool modify usage within the extensible SDK which has
apparently been broken since the tinfoil2 changes went in.
Fixes [YOCTO #11034].
(Bitbake rev: 52d87e4a58a52174e8e0b6297abaac8fd93ffcc3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we're including all GPL licenses then we should include AGPL in
the default COPYLEFT_LICENSE_INCLUDE since it has similar terms (and
currently we do have one AGPL-licensed recipe in OE-Core, namely db).
(Note that the default list isn't meant to be a guaranteed complete
safe list - seek legal advice based on what you're distributing if
unsure.)
(From OE-Core rev: a16fbf6970056a79cac8b70ce54e55e61ef63a8d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the move to tinfoil2, the behaviour when parsing failed has changed
a bit - exceptions are now raised, so handle these appropriately.
Specifically when if parsing the recipe created when running devtool add
fails, rename it to .bb.parsefailed so that the user can run bitbake
afterwards without parsing being interrupted.
(From OE-Core rev: b9592bd3c1ab2155b9f62d18ce593b9c88eff279)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Numbers within SCM (e.g. git) URLs are extremely unlikely to be valid
version numbers - more likely they are just part of the name, thus don't
try to extract them and use them as the version - doing so causes pretty
bad behaviour within devtool:
--------- snip ---------
$ devtool add https://github.com/inhedron/libtr50
NOTE: Fetching git://github.com/inhedron/libtr50;protocol=https...
...
NOTE: Using default source tree path .../build/workspace/sources/libtr
...
RecursionError: maximum recursion depth exceeded while calling a Python object
--------- snip ---------
(This was because ${PV} was being substituted into the URL, but PV's
value was being set to include ${SRCPV}, so there was a circular
reference.)
(From OE-Core rev: 3427508b6ce865654f8bf01a6fc04b83c70315d3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
npm's package.json supports two types of dependencies -
optionalDependencies and dependencies; in the code for creating a recipe
from a non-npm source (e.g. a git repository) we were not handling
optionalDependencies and thus when pointed at a node.js application
outside of npm we weren't taking care of all dependencies.
(From OE-Core rev: 2b66cb9982d10ce1744d430858eaef3e5a72c8c0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In do_compile we set HOME so that ~/.npm* only get created in the work
directory; we need to do the same in do_install as well or they'll go
into the user's home directory which we do not want.
(From OE-Core rev: a667cb9ed2dceb804b85d1a024e2619bc34d2681)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Any custom tasks that were added on the image between do_image_complete
and do_build were not being taken into account. Use the newly added
bb.build.tasksbetween() function to take care of that.
(From OE-Core rev: c966dee6208d3b7a9fa3c1b8aa1266058dcb65ff)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A generic version of the code to work out the tasks between two
tasks (based on the code here) has been added to bb.build, so use that
instead.
(From OE-Core rev: 1226e2eb2678407c315befd5700e653602e616a0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Up until recently it was possible to set SDK_TARGETS to include a native
recipe you wanted installed into the sysroot when installing the eSDK.
I'm not sure what happened but now when you try to add a native recipe
to SDK_TARGETS you get a missing task error because this recipe has no
do_package_write_* task. Of course such a task dependency is erroneous
and is apparently caused by setting SDK_RDEPENDS. I've checked and it
turns out that we no longer need to set SDK_RDEPENDS anyway (probably
because we explicitly set up task dependencies further down in the
class, which I don't think we were fully doing in early versions of the
eSDK). Thus, drop setting this variable to restore the functionality.
(From OE-Core rev: fe8268ede33a8c76492b85da0f41062f47a5279c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When I originally added this check I didn't quite understand how the
values in this field should be expressed - it seems from reading the
documentation if there is an entry starting with '!' then the list is
a blacklist and we shouldn't expect "linux" to be in the list, or we'll
end up skipping important dependencies.
This fixes fetching the "statsd" npm package.
Fixes [YOCTO #10760].
(Bitbake rev: 7aa6d1586417e0e7d9925917a82caee5884957db)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
An npm package.json file has two dependency fields: dependencies and
optionalDependencies. An item in optionalDependencies *may* also be
listed in dependencies, but this is not required (and not necessary
since if it's in optionalDependencies it will be optional, adding it to
dependencies won't do anything). The code here was assuming that an
optional dependency would always be in both, that's probably because
that was true of the examples I was looking at at the time. To fix it,
just add the optional ones to the list we're iterating over.
(Bitbake rev: c0c50d43266150a80be31ae2c6fcaf37f5ba231d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a utility function that gives the list of dependent tasks between
two specified tasks (just within the same recipe). This is intended to
be able to be executed from recipe context so it uses the datastore
rather than having access to the runqueue.
This will be used in OpenEmbedded-Core's populate_sdk_ext.bbclass to
get the list of tasks between do_image_complete and do_build.
(Bitbake rev: 433379bf12cf31fdf46defdf66695cf8be9994b1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The /cgit.cgi/ part is no longer valid in cgit URLs on
git.openembedded.org as of recent infrastructure changes.
(From OE-Core rev: a17ac420290c7e0debddec78b9540ae8726720dc)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to ensure the locale is UTF-8 or otherwise strange errors will
occur later on during execution - the same reason we check this in
BitBake itself. Unfortunately this check has to be before command line
parsing and therefore showing the help text in response to --help, since
that relies upon parsing bitbake's configuration (as we need to load
plugins in other layers).
Fixes [YOCTO #10908].
(From OE-Core rev: 370c6ba16c72bb52e80da098a5812ed1e09ac659)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The -C option isn't available in versions of git older than 1.8.5,
and officially we only require git 1.8.3.1 or newer (and the latter is
the version you'll find on CentOS 7, so the test fails there). In any
case we can simply specify the working directory to runCmd() so just
do that instead.
(From OE-Core rev: ce5c7075d530c0950f2feed35f95fbcd9f50721f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It turns out that this check cannot work. We don't have the information
to know whether an sstate package is really needed at this point in the
execution, so we check the availability for things that we won't
actually end up needing later on. Thus we can't fail if some of these
aren't found or we'll get needless failures.
This check was intended to give earlier more accurate errors when sstate
artifacts failed to download, but that's not practical so we'll rely
solely on the task execution check that was added within the runqueue.
This reverts most of commit 9e711b54487c3141d7264b8cf0d74f9465020190 (we
still need to allow BB_SETSCENE_ENFORCE through from the external
environment since the eSDK relies upon that.)
(From OE-Core rev: ff29ac6901d04487312f554d9e62250a18729c6c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If TMPDIR is configured to be somewhere outside of TOPDIR (a not
uncommon configuration where you have multiple disks and space on /home
is at a premium) then our attempt to find out the location of paths
under TMPDIR by using a relative path led to horribly broken paths
ending up in the eSDK. To save pain, just force a known value for TMPDIR
(i.e. ${TOPDIR}/tmp) and then we can assume that everywhere else.
Fixes [YOCTO #10797].
(From OE-Core rev: 4fe60d526a94f30b201c434994e80fef0f7392f0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When extracting source for a recipe within devtool (for extract, modify
or upgrade) We need to redirect WORKDIR, STAMPS_DIR etc. under a
temporary directory so that:
(a) we pick up all files that get unpacked to the WORKDIR, and
(b) we don't disturb the existing build
However, with recipe-specific sysroots the sysroots for the recipe will
be prepared under WORKDIR, and if we used the system temporary directory
i.e. usually /tmp) as used by mkdtemp by default, then our attempts to
hardlink files into the recipe-specific sysroots will fail on systems
where /tmp is a different filesystem, and we'd have to fall back to
copying the files which is a waste of time. Put the temp directory under
the WORKDIR to prevent that from being a problem.
(From OE-Core rev: e10a973cd9390eacb13bdb99693a0622bd3695f5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
[RP: Add needed mkdirhier call]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the file is expected to exist, then we should always be using require
so that if it doesn't we get an error rather than some other more
obscure failure later on.
(From OE-Core rev: 603ae6eb487489e65da69c68e532cb767ccc1fc2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you delete the log file that the oe-selftest.log symlink points to
but not the symlink itself, because we were using os.path.exists() here
the code assumed that the symlink didn't exist when in fact it still
did. Use os.path.lexists() instead.
(From OE-Core rev: 263af91a0efd21e041ecdb0c40f9b2d4e735f67d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're placing an object into the datastore - it's very definitely not
something we want to be expanding.
(From OE-Core rev: adfee41d2b1edbd2abd09f9101c359919e21e5d6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Fix not decoding output from grep ("Matched in b'manifest...')
* Fix showing "Matched in b''" if no match (show "not matched to any
task" instead)
* Drop the filtering out of .populate-sysroot from matched manifest
names - it should have been .populate_sysroot so it doesn't work, and
in any case the value of removing the task name is questionable given
that we aren't removing it for any other task, and that the rest of
the filename isn't only the task name, we might as well have the whole
thing. At least then you can do a find on that exact name without
wildcards and find it.
* Fix indenting of file list entries and indent "matched in" further
underneath
* Minor punctuation fixes
(From OE-Core rev: 4675ce43496898fccbac738835d7e92b1cca648e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a recipe installs a file or directory whose name contains square
brackets [ ] that form a valid glob expression and that file then they
won't be correctly removed from the sysroot, because we pass each path
in the sstate manifest to our oe.path.remove() function which calls
glob.glob() on the path passed into it and the expression won't
actually match the original filename. Since we don't expect to put any
wildcarded expressions in the sstate manifests, and we already have a
try...except around this, we can actually use os.remove() here instead.
Similarly, when we pass existing file paths to "grep" looking through
the manifests, we don't want those paths to be treated as regexes - so
use grep's -F command line switch.
Fixes [YOCTO #10836].
(From OE-Core rev: fd8a57861024fc82e15a2a4ec8c20ed0ebb242f6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When constructing a spec file we list files and directory paths in the
%files section. If ] or [ characters are in a file or directory name,
rpm treats them as wildcards which will mean it won't properly match the
filename. Instead, transform these into an ? wildcard so they don't
cause a problem.
(This fixes packaging the npm package "file-set" and anything that
happens to depend upon it, since it includes tests with files that
contain unusual characters including ] and [).
(From OE-Core rev: f95adb749619e70920c6cc6cd01c6d02cd348fd8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a warning in the doc comment for oe.path.remove() about using that
function on paths that may contain wildcards in the actual
file/directory names.
(From OE-Core rev: 18cc0965741102bccc62dfb32ed7753cdacbadc7)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you want to be able to use -fstack-protector then you need the
runtime support - you can either write this yourself or use libssp
supplied with GCC. If you're using GCC then it seems likely that you'd
just be using libssp, so include in the SDK by default; however use
RRECOMMENDS just in case it's been disabled or you aren't using GCC.
(From OE-Core rev: 6c990655e35bb3a14d59555662ec5802c9980028)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using devtool modify on the kernel, we have to do a bit of a dance
with tinfoil instances because we only find out that we're working on a
kernel recipe after tinfoil is initialised, but then we need to build
kern-tools-native which we're doing just by running bitbake directly.
With the tinfoil2 changes, a datastore for the recipe that we were
keeping around across the opening and closing of tinfoil is no longer
able to be used. Re-parse the recipe to avoid this problem.
(In future this whole thing will be able to be done in the same tinfoil
instance thanks to tinfoil2, but that refactoring is yet to be done.)
(From OE-Core rev: 06127d0115ba449bf04e2579cd1010065e0ed6e3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For the purposes BB_SETSCENE_ENFORCE is designed for (in OE, it is used
by the installation process for the extensible SDK), we don't actually
need the whitelisted real tasks to execute - we just need to have them
in the dependency tree so that we get all of the setscene tasks they
depend on to run. Therefore we can actually dry-run those real tasks
i.e. they won't be run (and thus we won't waste a significant amount of
time doing so) and won't be stamped as having run either. We do already
have a dry-run mode in BitBake (activated by the -n or --dry-run command
line option), but it dry-runs the setscene tasks as well which we don't
want here.
Note that this has no effect on the checking we are doing with
BB_SETSCENE_ENFORCE to ensure that only whitelisted real tasks are
scheduled to run - that's handled separately.
Fixes [YOCTO #10369].
(Bitbake rev: 58f084291beb3a87d8d9fdb36dfe7eff911fa36b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Check that man .in file actually gets modified, since sed -i doesn't
fail if it it doesn't
* Use a variable for man file path
(From OE-Core rev: 9ad36e945fa5b03726f78ba99e823eade8daa710)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we don't catch this then attempting to run devtool in non-memres mode
when bitbake is already running will produce a traceback instead of just
an error message.
(From OE-Core rev: e01b75dff599ffa2b66e6608b28bbb3564365eee)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton