1
0
mirror of https://git.yoctoproject.org/poky synced 2026-05-09 17:39:31 +00:00
Files
poky/documentation/ref-manual/technical-details.xml
T
Scott Rifenbark a60d9e2917 overview-manual, ref-manual: Moved auto added runtime deps section
Fixes [YOCTO #12370]

The "Automatically Added Runtime Dependencies" section moved from the
ref-manual to the overview-manual.  This topic is concepts and needs
to live in the new overview-manual's concepts chapter.  Fixed some
links in the ref-manual and one in the dev-manual.

(From yocto-docs rev: fa3e12030ce867cb81feed453d35c3a3643decd2)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-14 15:25:28 +00:00

581 lines
27 KiB
XML

<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
<chapter id='technical-details'>
<title>Technical Details</title>
<para>
This chapter provides technical details for various parts of the
Yocto Project.
Currently, topics include Yocto Project components,
cross-toolchain generation, shared state (sstate) cache,
x32, Wayland support, and Licenses.
</para>
<section id='fakeroot-and-pseudo'>
<title>Fakeroot and Pseudo</title>
<para>
Some tasks are easier to implement when allowed to perform certain
operations that are normally reserved for the root user.
For example, the
<link linkend='ref-tasks-install'><filename>do_install</filename></link>
task benefits from being able to set the UID and GID of installed files
to arbitrary values.
</para>
<para>
One approach to allowing tasks to perform root-only operations
would be to require BitBake to run as root.
However, this method is cumbersome and has security issues.
The approach that is actually used is to run tasks that benefit from
root privileges in a "fake" root environment.
Within this environment, the task and its child processes believe that
they are running as the root user, and see an internally consistent
view of the filesystem.
As long as generating the final output (e.g. a package or an image)
does not require root privileges, the fact that some earlier steps ran
in a fake root environment does not cause problems.
</para>
<para>
The capability to run tasks in a fake root environment is known as
"fakeroot", which is derived from the BitBake keyword/variable
flag that requests a fake root environment for a task.
In current versions of the OpenEmbedded build system,
the program that implements fakeroot is known as Pseudo.
</para>
<para>
Pseudo overrides system calls through the
<filename>LD_PRELOAD</filename> mechanism to give the
illusion of running as root.
To keep track of "fake" file ownership and permissions resulting from
operations that require root permissions, an sqlite3
database is used.
This database is stored in
<filename>${</filename><link linkend='var-WORKDIR'><filename>WORKDIR</filename></link><filename>}/pseudo/files.db</filename>
for individual recipes.
Storing the database in a file as opposed to in memory
gives persistence between tasks, and even between builds.
<note><title>Caution</title>
If you add your own task that manipulates the same files or
directories as a fakeroot task, then that task should also run
under fakeroot.
Otherwise, the task will not be able to run root-only operations,
and will not see the fake file ownership and permissions set by the
other task.
You should also add a dependency on
<filename>virtual/fakeroot-native:do_populate_sysroot</filename>,
giving the following:
<literallayout class='monospaced'>
fakeroot do_mytask () {
...
}
do_mytask[depends] += "virtual/fakeroot-native:do_populate_sysroot"
</literallayout>
</note>
For more information, see the
<ulink url='&YOCTO_DOCS_BB_URL;#var-FAKEROOT'><filename>FAKEROOT*</filename></ulink>
variables in the BitBake User Manual.
You can also reference this
<ulink url='http://www.ibm.com/developerworks/opensource/library/os-aapseudo1/index.html'>Pseudo</ulink>
article.
</para>
</section>
<section id="wayland">
<title>Wayland</title>
<para>
<ulink url='http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)'>Wayland</ulink>
is a computer display server protocol that
provides a method for compositing window managers to communicate
directly with applications and video hardware and expects them to
communicate with input hardware using other libraries.
Using Wayland with supporting targets can result in better control
over graphics frame rendering than an application might otherwise
achieve.
</para>
<para>
The Yocto Project provides the Wayland protocol libraries and the
reference
<ulink url='http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)#Weston'>Weston</ulink>
compositor as part of its release.
This section describes what you need to do to implement Wayland and
use the compositor when building an image for a supporting target.
</para>
<section id="wayland-support">
<title>Support</title>
<para>
The Wayland protocol libraries and the reference Weston compositor
ship as integrated packages in the <filename>meta</filename> layer
of the
<link linkend='source-directory'>Source Directory</link>.
Specifically, you can find the recipes that build both Wayland
and Weston at <filename>meta/recipes-graphics/wayland</filename>.
</para>
<para>
You can build both the Wayland and Weston packages for use only
with targets that accept the
<ulink url='http://dri.freedesktop.org/wiki/'>Mesa 3D and Direct Rendering Infrastructure</ulink>,
which is also known as Mesa DRI.
This implies that you cannot build and use the packages if your
target uses, for example, the
<trademark class='registered'>Intel</trademark> Embedded Media and
Graphics Driver (<trademark class='registered'>Intel</trademark>
EMGD) that overrides Mesa DRI.
</para>
<note>
Due to lack of EGL support, Weston 1.0.3 will not run directly on
the emulated QEMU hardware.
However, this version of Weston will run under X emulation without
issues.
</note>
</section>
<section id="enabling-wayland-in-an-image">
<title>Enabling Wayland in an Image</title>
<para>
To enable Wayland, you need to enable it to be built and enable
it to be included in the image.
</para>
<section id="enable-building">
<title>Building</title>
<para>
To cause Mesa to build the <filename>wayland-egl</filename>
platform and Weston to build Wayland with Kernel Mode
Setting
(<ulink url='https://wiki.archlinux.org/index.php/Kernel_Mode_Setting'>KMS</ulink>)
support, include the "wayland" flag in the
<link linkend="var-DISTRO_FEATURES"><filename>DISTRO_FEATURES</filename></link>
statement in your <filename>local.conf</filename> file:
<literallayout class='monospaced'>
DISTRO_FEATURES_append = " wayland"
</literallayout>
</para>
<note>
If X11 has been enabled elsewhere, Weston will build Wayland
with X11 support
</note>
</section>
<section id="enable-installation-in-an-image">
<title>Installing</title>
<para>
To install the Wayland feature into an image, you must
include the following
<link linkend='var-CORE_IMAGE_EXTRA_INSTALL'><filename>CORE_IMAGE_EXTRA_INSTALL</filename></link>
statement in your <filename>local.conf</filename> file:
<literallayout class='monospaced'>
CORE_IMAGE_EXTRA_INSTALL += "wayland weston"
</literallayout>
</para>
</section>
</section>
<section id="running-weston">
<title>Running Weston</title>
<para>
To run Weston inside X11, enabling it as described earlier and
building a Sato image is sufficient.
If you are running your image under Sato, a Weston Launcher appears
in the "Utility" category.
</para>
<para>
Alternatively, you can run Weston through the command-line
interpretor (CLI), which is better suited for development work.
To run Weston under the CLI, you need to do the following after
your image is built:
<orderedlist>
<listitem><para>Run these commands to export
<filename>XDG_RUNTIME_DIR</filename>:
<literallayout class='monospaced'>
mkdir -p /tmp/$USER-weston
chmod 0700 /tmp/$USER-weston
export XDG_RUNTIME_DIR=/tmp/$USER-weston
</literallayout></para></listitem>
<listitem><para>Launch Weston in the shell:
<literallayout class='monospaced'>
weston
</literallayout></para></listitem>
</orderedlist>
</para>
</section>
</section>
<section id="licenses">
<title>Licenses</title>
<para>
This section describes the mechanism by which the OpenEmbedded build system
tracks changes to licensing text.
The section also describes how to enable commercially licensed recipes,
which by default are disabled.
</para>
<para>
For information that can help you maintain compliance with various open
source licensing during the lifecycle of the product, see the
"<ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Project's Lifecycle</ulink>"
section in the Yocto Project Development Tasks Manual.
</para>
<section id="usingpoky-configuring-LIC_FILES_CHKSUM">
<title>Tracking License Changes</title>
<para>
The license of an upstream project might change in the future.
In order to prevent these changes going unnoticed, the
<filename><link linkend='var-LIC_FILES_CHKSUM'>LIC_FILES_CHKSUM</link></filename>
variable tracks changes to the license text. The checksums are validated at the end of the
configure step, and if the checksums do not match, the build will fail.
</para>
<section id="usingpoky-specifying-LIC_FILES_CHKSUM">
<title>Specifying the <filename>LIC_FILES_CHKSUM</filename> Variable</title>
<para>
The <filename>LIC_FILES_CHKSUM</filename>
variable contains checksums of the license text in the source
code for the recipe.
Following is an example of how to specify
<filename>LIC_FILES_CHKSUM</filename>:
<literallayout class='monospaced'>
LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \
file://licfile1.txt;beginline=5;endline=29;md5=yyyy \
file://licfile2.txt;endline=50;md5=zzzz \
..."
</literallayout>
<note><title>Notes</title>
<itemizedlist>
<listitem><para>
When using "beginline" and "endline", realize that
line numbering begins with one and not zero.
Also, the included lines are inclusive (i.e. lines
five through and including 29 in the previous
example for <filename>licfile1.txt</filename>).
</para></listitem>
<listitem><para>
When a license check fails, the selected license
text is included as part of the QA message.
Using this output, you can determine the exact
start and finish for the needed license text.
</para></listitem>
</itemizedlist>
</note>
</para>
<para>
The build system uses the
<filename><link linkend='var-S'>S</link></filename> variable as
the default directory when searching files listed in
<filename>LIC_FILES_CHKSUM</filename>.
The previous example employs the default directory.
</para>
<para>
Consider this next example:
<literallayout class='monospaced'>
LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\
md5=bb14ed3c4cda583abc85401304b5cd4e"
LIC_FILES_CHKSUM = "file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6"
</literallayout>
</para>
<para>
The first line locates a file in
<filename>${S}/src/ls.c</filename> and isolates lines five
through 16 as license text.
The second line refers to a file in
<filename><link linkend='var-WORKDIR'>WORKDIR</link></filename>.
</para>
<para>
Note that <filename>LIC_FILES_CHKSUM</filename> variable is
mandatory for all recipes, unless the
<filename>LICENSE</filename> variable is set to "CLOSED".
</para>
</section>
<section id="usingpoky-LIC_FILES_CHKSUM-explanation-of-syntax">
<title>Explanation of Syntax</title>
<para>
As mentioned in the previous section, the
<filename>LIC_FILES_CHKSUM</filename> variable lists all the
important files that contain the license text for the source code.
It is possible to specify a checksum for an entire file, or a specific section of a
file (specified by beginning and ending line numbers with the "beginline" and "endline"
parameters, respectively).
The latter is useful for source files with a license notice header,
README documents, and so forth.
If you do not use the "beginline" parameter, then it is assumed that the text begins on the
first line of the file.
Similarly, if you do not use the "endline" parameter, it is assumed that the license text
ends with the last line of the file.
</para>
<para>
The "md5" parameter stores the md5 checksum of the license text.
If the license text changes in any way as compared to this parameter
then a mismatch occurs.
This mismatch triggers a build failure and notifies the developer.
Notification allows the developer to review and address the license text changes.
Also note that if a mismatch occurs during the build, the correct md5
checksum is placed in the build log and can be easily copied to the recipe.
</para>
<para>
There is no limit to how many files you can specify using the
<filename>LIC_FILES_CHKSUM</filename> variable.
Generally, however, every project requires a few specifications for license tracking.
Many projects have a "COPYING" file that stores the license information for all the source
code files.
This practice allows you to just track the "COPYING" file as long as it is kept up to date.
</para>
<tip>
If you specify an empty or invalid "md5" parameter, BitBake returns an md5 mis-match
error and displays the correct "md5" parameter value during the build.
The correct parameter is also captured in the build log.
</tip>
<tip>
If the whole file contains only license text, you do not need to use the "beginline" and
"endline" parameters.
</tip>
</section>
</section>
<section id="enabling-commercially-licensed-recipes">
<title>Enabling Commercially Licensed Recipes</title>
<para>
By default, the OpenEmbedded build system disables
components that have commercial or other special licensing
requirements.
Such requirements are defined on a
recipe-by-recipe basis through the
<link linkend='var-LICENSE_FLAGS'><filename>LICENSE_FLAGS</filename></link>
variable definition in the affected recipe.
For instance, the
<filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
recipe contains the following statement:
<literallayout class='monospaced'>
LICENSE_FLAGS = "commercial"
</literallayout>
Here is a slightly more complicated example that contains both an
explicit recipe name and version (after variable expansion):
<literallayout class='monospaced'>
LICENSE_FLAGS = "license_${PN}_${PV}"
</literallayout>
In order for a component restricted by a <filename>LICENSE_FLAGS</filename>
definition to be enabled and included in an image, it
needs to have a matching entry in the global
<link linkend='var-LICENSE_FLAGS_WHITELIST'><filename>LICENSE_FLAGS_WHITELIST</filename></link>
variable, which is a variable
typically defined in your <filename>local.conf</filename> file.
For example, to enable
the <filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
package, you could add either the string
"commercial_gst-plugins-ugly" or the more general string
"commercial" to <filename>LICENSE_FLAGS_WHITELIST</filename>.
See the
"<link linkend='license-flag-matching'>License Flag Matching</link>" section
for a full explanation of how <filename>LICENSE_FLAGS</filename> matching works.
Here is the example:
<literallayout class='monospaced'>
LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly"
</literallayout>
Likewise, to additionally enable the package built from the recipe containing
<filename>LICENSE_FLAGS = "license_${PN}_${PV}"</filename>, and assuming
that the actual recipe name was <filename>emgd_1.10.bb</filename>,
the following string would enable that package as well as
the original <filename>gst-plugins-ugly</filename> package:
<literallayout class='monospaced'>
LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly license_emgd_1.10"
</literallayout>
As a convenience, you do not need to specify the complete license string
in the whitelist for every package.
You can use an abbreviated form, which consists
of just the first portion or portions of the license string before
the initial underscore character or characters.
A partial string will match
any license that contains the given string as the first
portion of its license.
For example, the following
whitelist string will also match both of the packages
previously mentioned as well as any other packages that have
licenses starting with "commercial" or "license".
<literallayout class='monospaced'>
LICENSE_FLAGS_WHITELIST = "commercial license"
</literallayout>
</para>
<section id="license-flag-matching">
<title>License Flag Matching</title>
<para>
License flag matching allows you to control what recipes the
OpenEmbedded build system includes in the build.
Fundamentally, the build system attempts to match
<link linkend='var-LICENSE_FLAGS'><filename>LICENSE_FLAGS</filename></link>
strings found in recipes against
<link linkend='var-LICENSE_FLAGS_WHITELIST'><filename>LICENSE_FLAGS_WHITELIST</filename></link>
strings found in the whitelist.
A match causes the build system to include a recipe in the
build, while failure to find a match causes the build system to
exclude a recipe.
</para>
<para>
In general, license flag matching is simple.
However, understanding some concepts will help you
correctly and effectively use matching.
</para>
<para>
Before a flag
defined by a particular recipe is tested against the
contents of the whitelist, the expanded string
<filename>_${PN}</filename> is appended to the flag.
This expansion makes each <filename>LICENSE_FLAGS</filename>
value recipe-specific.
After expansion, the string is then matched against the
whitelist.
Thus, specifying
<filename>LICENSE_FLAGS = "commercial"</filename>
in recipe "foo", for example, results in the string
<filename>"commercial_foo"</filename>.
And, to create a match, that string must appear in the
whitelist.
</para>
<para>
Judicious use of the <filename>LICENSE_FLAGS</filename>
strings and the contents of the
<filename>LICENSE_FLAGS_WHITELIST</filename> variable
allows you a lot of flexibility for including or excluding
recipes based on licensing.
For example, you can broaden the matching capabilities by
using license flags string subsets in the whitelist.
<note>When using a string subset, be sure to use the part of
the expanded string that precedes the appended underscore
character (e.g. <filename>usethispart_1.3</filename>,
<filename>usethispart_1.4</filename>, and so forth).
</note>
For example, simply specifying the string "commercial" in
the whitelist matches any expanded
<filename>LICENSE_FLAGS</filename> definition that starts with
the string "commercial" such as "commercial_foo" and
"commercial_bar", which are the strings the build system
automatically generates for hypothetical recipes named
"foo" and "bar" assuming those recipes simply specify the
following:
<literallayout class='monospaced'>
LICENSE_FLAGS = "commercial"
</literallayout>
Thus, you can choose to exhaustively
enumerate each license flag in the whitelist and
allow only specific recipes into the image, or
you can use a string subset that causes a broader range of
matches to allow a range of recipes into the image.
</para>
<para>
This scheme works even if the
<filename>LICENSE_FLAGS</filename> string already
has <filename>_${PN}</filename> appended.
For example, the build system turns the license flag
"commercial_1.2_foo" into "commercial_1.2_foo_foo" and would
match both the general "commercial" and the specific
"commercial_1.2_foo" strings found in the whitelist, as
expected.
</para>
<para>
Here are some other scenarios:
<itemizedlist>
<listitem><para>You can specify a versioned string in the
recipe such as "commercial_foo_1.2" in a "foo" recipe.
The build system expands this string to
"commercial_foo_1.2_foo".
Combine this license flag with a whitelist that has
the string "commercial" and you match the flag along
with any other flag that starts with the string
"commercial".</para></listitem>
<listitem><para>Under the same circumstances, you can
use "commercial_foo" in the whitelist and the
build system not only matches "commercial_foo_1.2" but
also matches any license flag with the string
"commercial_foo", regardless of the version.
</para></listitem>
<listitem><para>You can be very specific and use both the
package and version parts in the whitelist (e.g.
"commercial_foo_1.2") to specifically match a
versioned recipe.</para></listitem>
</itemizedlist>
</para>
</section>
<section id="other-variables-related-to-commercial-licenses">
<title>Other Variables Related to Commercial Licenses</title>
<para>
Other helpful variables related to commercial
license handling exist and are defined in the
<filename>poky/meta/conf/distro/include/default-distrovars.inc</filename> file:
<literallayout class='monospaced'>
COMMERCIAL_AUDIO_PLUGINS ?= ""
COMMERCIAL_VIDEO_PLUGINS ?= ""
</literallayout>
If you want to enable these components, you can do so by making sure you have
statements similar to the following
in your <filename>local.conf</filename> configuration file:
<literallayout class='monospaced'>
COMMERCIAL_AUDIO_PLUGINS = "gst-plugins-ugly-mad \
gst-plugins-ugly-mpegaudioparse"
COMMERCIAL_VIDEO_PLUGINS = "gst-plugins-ugly-mpeg2dec \
gst-plugins-ugly-mpegstream gst-plugins-bad-mpegvideoparse"
LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly commercial_gst-plugins-bad commercial_qmmp"
</literallayout>
Of course, you could also create a matching whitelist
for those components using the more general "commercial"
in the whitelist, but that would also enable all the
other packages with
<link linkend='var-LICENSE_FLAGS'><filename>LICENSE_FLAGS</filename></link>
containing "commercial", which you may or may not want:
<literallayout class='monospaced'>
LICENSE_FLAGS_WHITELIST = "commercial"
</literallayout>
</para>
<para>
Specifying audio and video plug-ins as part of the
<filename>COMMERCIAL_AUDIO_PLUGINS</filename> and
<filename>COMMERCIAL_VIDEO_PLUGINS</filename> statements
(along with the enabling
<filename>LICENSE_FLAGS_WHITELIST</filename>) includes the
plug-ins or components into built images, thus adding
support for media formats or components.
</para>
</section>
</section>
</section>
</chapter>
<!--
vim: expandtab tw=80 ts=4
-->