diff --git a/documentation/concepts-manual/concepts-manual-concepts.xml b/documentation/concepts-manual/concepts-manual-concepts.xml
index 010eda6191..e59ec6c4b8 100644
--- a/documentation/concepts-manual/concepts-manual-concepts.xml
+++ b/documentation/concepts-manual/concepts-manual-concepts.xml
@@ -52,8 +52,8 @@
Following are some brief details on these core components.
For additional information on how these components interact during
a build, see the
- "Development Concepts"
- section in the Getting Started With Yocto Project Manual.
+ "Development Concepts"
+ section.
@@ -248,6 +248,1798 @@
+
+
+
+ Development Concepts
+
+
+ This section takes a more detailed look inside the development
+ process.
+ The following diagram represents development at a high level.
+ The remainder of this section expands on the fundamental input,
+ output, process, and
+ Metadata) blocks
+ that make up development in the Yocto Project environment.
+
+
+
+
+
+
+
+ In general, development consists of several functional areas:
+
+
+ User Configuration:
+ Metadata you can use to control the build process.
+
+
+ Metadata Layers:
+ Various layers that provide software, machine, and
+ distro Metadata.
+
+
+ Source Files:
+ Upstream releases, local projects, and SCMs.
+
+
+ Build System:
+ Processes under the control of
+ BitBake.
+ This block expands on how BitBake fetches source, applies
+ patches, completes compilation, analyzes output for package
+ generation, creates and tests packages, generates images,
+ and generates cross-development tools.
+
+
+ Package Feeds:
+ Directories containing output packages (RPM, DEB or IPK),
+ which are subsequently used in the construction of an
+ image or SDK, produced by the build system.
+ These feeds can also be copied and shared using a web
+ server or other means to facilitate extending or updating
+ existing images on devices at runtime if runtime package
+ management is enabled.
+
+
+ Images:
+ Images produced by the development process.
+
+
+ Application Development SDK:
+ Cross-development tools that are produced along with
+ an image or separately with BitBake.
+
+
+
+
+
+ User Configuration
+
+
+ User configuration helps define the build.
+ Through user configuration, you can tell BitBake the
+ target architecture for which you are building the image,
+ where to store downloaded source, and other build properties.
+
+
+
+ The following figure shows an expanded representation of the
+ "User Configuration" box of the
+ general Yocto Project Development Environment figure:
+
+
+
+
+
+
+
+ BitBake needs some basic configuration files in order to
+ complete a build.
+ These files are *.conf files.
+ The minimally necessary ones reside as example files in the
+ Source Directory.
+ For simplicity, this section refers to the Source Directory as
+ the "Poky Directory."
+
+
+
+ When you clone the poky Git repository
+ or you download and unpack a Yocto Project release, you
+ can set up the Source Directory to be named anything you want.
+ For this discussion, the cloned repository uses the default
+ name poky.
+
+ The
+ Poky
+ repository is primarily an aggregation of existing
+ repositories.
+ It is not a canonical upstream source.
+
+
+
+
+ The meta-poky layer inside Poky contains
+ a conf directory that has example
+ configuration files.
+ These example files are used as a basis for creating actual
+ configuration files when you source the build environment
+ script
+ (i.e.
+ &OE_INIT_FILE;).
+
+
+
+ Sourcing the build environment script creates a
+ Build Directory
+ if one does not already exist.
+ BitBake uses the Build Directory for all its work during
+ builds.
+ The Build Directory has a conf directory
+ that contains default versions of your
+ local.conf and
+ bblayers.conf configuration files.
+ These default configuration files are created only if versions
+ do not already exist in the Build Directory at the time you
+ source the build environment setup script.
+
+
+
+ Because the Poky repository is fundamentally an aggregation of
+ existing repositories, some users might be familiar with
+ running the &OE_INIT_FILE; script
+ in the context of separate
+ OpenEmbedded-Core
+ and BitBake repositories rather than a single Poky repository.
+ This discussion assumes the script is executed from
+ within a cloned or unpacked version of Poky.
+
+
+
+ Depending on where the script is sourced, different
+ sub-scripts are called to set up the Build Directory
+ (Yocto or OpenEmbedded).
+ Specifically, the script
+ scripts/oe-setup-builddir inside the
+ poky directory sets up the Build Directory and seeds the
+ directory (if necessary) with configuration files appropriate
+ for the Yocto Project development environment.
+
+ The scripts/oe-setup-builddir script
+ uses the $TEMPLATECONF variable to
+ determine which sample configuration files to locate.
+
+
+
+
+ The local.conf file provides many
+ basic variables that define a build environment.
+ Here is a list of a few.
+ To see the default configurations in a
+ local.conf file created by the build
+ environment script, see the
+ local.conf.sample in the
+ meta-poky layer:
+
+
+ Parallelism Options:
+ Controlled by the
+ BB_NUMBER_THREADS,
+ PARALLEL_MAKE,
+ and
+ BB_NUMBER_PARSE_THREADS
+ variables.
+
+
+ Target Machine Selection:
+ Controlled by the
+ MACHINE
+ variable.
+
+
+ Download Directory:
+ Controlled by the
+ DL_DIR
+ variable.
+
+
+ Shared State Directory:
+ Controlled by the
+ SSTATE_DIR
+ variable.
+
+
+ Build Output:
+ Controlled by the
+ TMPDIR
+ variable.
+
+
+
+ Configurations set in the
+ conf/local.conf file can also be set
+ in the conf/site.conf and
+ conf/auto.conf configuration files.
+
+
+
+
+ The bblayers.conf file tells BitBake what
+ layers you want considered during the build.
+ By default, the layers listed in this file include layers
+ minimally needed by the build system.
+ However, you must manually add any custom layers you have
+ created.
+ You can find more information on working with the
+ bblayers.conf file in the
+ "Enabling Your Layer"
+ section in the Yocto Project Development Tasks Manual.
+
+
+
+ The files site.conf and
+ auto.conf are not created by the
+ environment initialization script.
+ If you want the site.conf file, you
+ need to create that yourself.
+ The auto.conf file is typically created by
+ an autobuilder:
+
+
+ site.conf:
+ You can use the conf/site.conf
+ configuration file to configure multiple
+ build directories.
+ For example, suppose you had several build environments
+ and they shared some common features.
+ You can set these default build properties here.
+ A good example is perhaps the packaging format to use
+ through the
+ PACKAGE_CLASSES
+ variable.
+
+ One useful scenario for using the
+ conf/site.conf file is to extend
+ your
+ BBPATH
+ variable to include the path to a
+ conf/site.conf.
+ Then, when BitBake looks for Metadata using
+ BBPATH, it finds the
+ conf/site.conf file and applies
+ your common configurations found in the file.
+ To override configurations in a particular build
+ directory, alter the similar configurations within
+ that build directory's
+ conf/local.conf file.
+
+
+ auto.conf:
+ The file is usually created and written to by
+ an autobuilder.
+ The settings put into the file are typically the
+ same as you would find in the
+ conf/local.conf or the
+ conf/site.conf files.
+
+
+
+
+
+ You can edit all configuration files to further define
+ any particular build environment.
+ This process is represented by the "User Configuration Edits"
+ box in the figure.
+
+
+
+ When you launch your build with the
+ bitbake target
+ command, BitBake sorts out the configurations to ultimately
+ define your build environment.
+ It is important to understand that the
+ OpenEmbedded build system
+ reads the configuration files in a specific order:
+ site.conf, auto.conf,
+ and local.conf.
+ And, the build system applies the normal assignment statement
+ rules.
+ Because the files are parsed in a specific order, variable
+ assignments for the same variable could be affected.
+ For example, if the auto.conf file and
+ the local.conf set
+ variable1 to different values,
+ because the build system parses local.conf
+ after auto.conf,
+ variable1 is assigned the value from
+ the local.conf file.
+
+
+
+
+ Metadata, Machine Configuration, and Policy Configuration
+
+
+ The previous section described the user configurations that
+ define BitBake's global behavior.
+ This section takes a closer look at the layers the build system
+ uses to further control the build.
+ These layers provide Metadata for the software, machine, and
+ policy.
+
+
+
+ In general, three types of layer input exist:
+
+
+ Policy Configuration:
+ Distribution Layers provide top-level or general
+ policies for the image or SDK being built.
+ For example, this layer would dictate whether BitBake
+ produces RPM or IPK packages.
+
+
+ Machine Configuration:
+ Board Support Package (BSP) layers provide machine
+ configurations.
+ This type of information is specific to a particular
+ target architecture.
+
+
+ Metadata:
+ Software layers contain user-supplied recipe files,
+ patches, and append files.
+
+
+
+
+
+ The following figure shows an expanded representation of the
+ Metadata, Machine Configuration, and Policy Configuration input
+ (layers) boxes of the
+ general Yocto Project Development Environment figure:
+
+
+
+
+
+
+
+ In general, all layers have a similar structure.
+ They all contain a licensing file
+ (e.g. COPYING) if the layer is to be
+ distributed, a README file as good
+ practice and especially if the layer is to be distributed, a
+ configuration directory, and recipe directories.
+
+
+
+ The Yocto Project has many layers that can be used.
+ You can see a web-interface listing of them on the
+ Source Repositories
+ page.
+ The layers appear at the bottom categorized under
+ "Yocto Metadata Layers."
+ These layers are fundamentally a subset of the
+ OpenEmbedded Metadata Index,
+ which lists all layers provided by the OpenEmbedded community.
+
+ Layers exist in the Yocto Project Source Repositories that
+ cannot be found in the OpenEmbedded Metadata Index.
+ These layers are either deprecated or experimental
+ in nature.
+
+
+
+
+ BitBake uses the conf/bblayers.conf file,
+ which is part of the user configuration, to find what layers it
+ should be using as part of the build.
+
+
+
+ For more information on layers, see the
+ "Understanding and Creating Layers"
+ section in the Yocto Project Development Tasks Manual.
+
+
+
+ Distro Layer
+
+
+ The distribution layer provides policy configurations for
+ your distribution.
+ Best practices dictate that you isolate these types of
+ configurations into their own layer.
+ Settings you provide in
+ conf/distro/distro.conf override
+ similar settings that BitBake finds in your
+ conf/local.conf file in the Build
+ Directory.
+
+
+
+ The following list provides some explanation and references
+ for what you typically find in the distribution layer:
+
+
+ classes:
+ Class files (.bbclass) hold
+ common functionality that can be shared among
+ recipes in the distribution.
+ When your recipes inherit a class, they take on the
+ settings and functions for that class.
+ You can read more about class files in the
+ "Classes"
+ chapter of the Yocto Reference Manual.
+
+
+ conf:
+ This area holds configuration files for the
+ layer (conf/layer.conf),
+ the distribution
+ (conf/distro/distro.conf),
+ and any distribution-wide include files.
+
+
+ recipes-*:
+ Recipes and append files that affect common
+ functionality across the distribution.
+ This area could include recipes and append files
+ to add distribution-specific configuration,
+ initialization scripts, custom image recipes,
+ and so forth.
+
+
+
+
+
+
+ BSP Layer
+
+
+ The BSP Layer provides machine configurations.
+ Everything in this layer is specific to the machine for
+ which you are building the image or the SDK.
+ A common structure or form is defined for BSP layers.
+ You can learn more about this structure in the
+ Yocto Project Board Support Package (BSP) Developer's Guide.
+
+ In order for a BSP layer to be considered compliant
+ with the Yocto Project, it must meet some structural
+ requirements.
+
+
+
+
+ The BSP Layer's configuration directory contains
+ configuration files for the machine
+ (conf/machine/machine.conf)
+ and, of course, the layer
+ (conf/layer.conf).
+
+
+
+ The remainder of the layer is dedicated to specific recipes
+ by function: recipes-bsp,
+ recipes-core,
+ recipes-graphics, and
+ recipes-kernel.
+ Metadata can exist for multiple formfactors, graphics
+ support systems, and so forth.
+
+ While the figure shows several
+ recipes-* directories, not all
+ these directories appear in all BSP layers.
+
+
+
+
+
+ Software Layer
+
+
+ The software layer provides the Metadata for additional
+ software packages used during the build.
+ This layer does not include Metadata that is specific to
+ the distribution or the machine, which are found in their
+ respective layers.
+
+
+
+ This layer contains any new recipes that your project
+ needs in the form of recipe files.
+
+
+
+
+
+ Sources
+
+
+ In order for the OpenEmbedded build system to create an
+ image or any target, it must be able to access source files.
+ The
+ general Yocto Project Development Environment figure
+ represents source files using the "Upstream Project Releases",
+ "Local Projects", and "SCMs (optional)" boxes.
+ The figure represents mirrors, which also play a role in
+ locating source files, with the "Source Mirror(s)" box.
+
+
+
+ The method by which source files are ultimately organized is
+ a function of the project.
+ For example, for released software, projects tend to use
+ tarballs or other archived files that can capture the
+ state of a release guaranteeing that it is statically
+ represented.
+ On the other hand, for a project that is more dynamic or
+ experimental in nature, a project might keep source files in a
+ repository controlled by a Source Control Manager (SCM) such as
+ Git.
+ Pulling source from a repository allows you to control
+ the point in the repository (the revision) from which you
+ want to build software.
+ Finally, a combination of the two might exist, which would
+ give the consumer a choice when deciding where to get
+ source files.
+
+
+
+ BitBake uses the
+ SRC_URI
+ variable to point to source files regardless of their location.
+ Each recipe must have a SRC_URI variable
+ that points to the source.
+
+
+
+ Another area that plays a significant role in where source
+ files come from is pointed to by the
+ DL_DIR
+ variable.
+ This area is a cache that can hold previously downloaded
+ source.
+ You can also instruct the OpenEmbedded build system to create
+ tarballs from Git repositories, which is not the default
+ behavior, and store them in the DL_DIR
+ by using the
+ BB_GENERATE_MIRROR_TARBALLS
+ variable.
+
+
+
+ Judicious use of a DL_DIR directory can
+ save the build system a trip across the Internet when looking
+ for files.
+ A good method for using a download directory is to have
+ DL_DIR point to an area outside of your
+ Build Directory.
+ Doing so allows you to safely delete the Build Directory
+ if needed without fear of removing any downloaded source file.
+
+
+
+ The remainder of this section provides a deeper look into the
+ source files and the mirrors.
+ Here is a more detailed look at the source file area of the
+ base figure:
+
+
+
+
+ Upstream Project Releases
+
+
+ Upstream project releases exist anywhere in the form of an
+ archived file (e.g. tarball or zip file).
+ These files correspond to individual recipes.
+ For example, the figure uses specific releases each for
+ BusyBox, Qt, and Dbus.
+ An archive file can be for any released product that can be
+ built using a recipe.
+
+
+
+
+ Local Projects
+
+
+ Local projects are custom bits of software the user
+ provides.
+ These bits reside somewhere local to a project - perhaps
+ a directory into which the user checks in items (e.g.
+ a local directory containing a development source tree
+ used by the group).
+
+
+
+ The canonical method through which to include a local
+ project is to use the
+ externalsrc
+ class to include that local project.
+ You use either the local.conf or a
+ recipe's append file to override or set the
+ recipe to point to the local directory on your disk to pull
+ in the whole source tree.
+
+
+
+ For information on how to use the
+ externalsrc class, see the
+ "externalsrc.bbclass"
+ section.
+
+
+
+
+ Source Control Managers (Optional)
+
+
+ Another place the build system can get source files from is
+ through an SCM such as Git or Subversion.
+ In this case, a repository is cloned or checked out.
+ The
+ do_fetch
+ task inside BitBake uses
+ the SRC_URI
+ variable and the argument's prefix to determine the correct
+ fetcher module.
+
+ For information on how to have the OpenEmbedded build
+ system generate tarballs for Git repositories and place
+ them in the
+ DL_DIR
+ directory, see the
+ BB_GENERATE_MIRROR_TARBALLS
+ variable.
+
+
+
+
+ When fetching a repository, BitBake uses the
+ SRCREV
+ variable to determine the specific revision from which to
+ build.
+
+
+
+
+ Source Mirror(s)
+
+
+ Two kinds of mirrors exist: pre-mirrors and regular
+ mirrors.
+ The
+ PREMIRRORS
+ and
+ MIRRORS
+ variables point to these, respectively.
+ BitBake checks pre-mirrors before looking upstream for any
+ source files.
+ Pre-mirrors are appropriate when you have a shared
+ directory that is not a directory defined by the
+ DL_DIR
+ variable.
+ A Pre-mirror typically points to a shared directory that is
+ local to your organization.
+
+
+
+ Regular mirrors can be any site across the Internet
+ that is used as an alternative location for source
+ code should the primary site not be functioning for
+ some reason or another.
+
+
+
+
+
+ Package Feeds
+
+
+ When the OpenEmbedded build system generates an image or an
+ SDK, it gets the packages from a package feed area located
+ in the
+ Build Directory.
+ The
+ general Yocto Project Development Environment figure
+ shows this package feeds area in the upper-right corner.
+
+
+
+ This section looks a little closer into the package feeds
+ area used by the build system.
+ Here is a more detailed look at the area:
+
+
+
+
+ Package feeds are an intermediary step in the build process.
+ The OpenEmbedded build system provides classes to generate
+ different package types, and you specify which classes to
+ enable through the
+ PACKAGE_CLASSES
+ variable.
+ Before placing the packages into package feeds,
+ the build process validates them with generated output quality
+ assurance checks through the
+ insane
+ class.
+
+
+
+ The package feed area resides in the Build Directory.
+ The directory the build system uses to temporarily store
+ packages is determined by a combination of variables and the
+ particular package manager in use.
+ See the "Package Feeds" box in the illustration and note the
+ information to the right of that area.
+ In particular, the following defines where package files are
+ kept:
+
+
+ DEPLOY_DIR:
+ Defined as tmp/deploy in the Build
+ Directory.
+
+
+ DEPLOY_DIR_*:
+ Depending on the package manager used, the package type
+ sub-folder.
+ Given RPM, IPK, or DEB packaging and tarball creation,
+ the
+ DEPLOY_DIR_RPM,
+ DEPLOY_DIR_IPK,
+ DEPLOY_DIR_DEB,
+ or
+ DEPLOY_DIR_TAR,
+ variables are used, respectively.
+
+
+ PACKAGE_ARCH:
+ Defines architecture-specific sub-folders.
+ For example, packages could exist for the i586 or
+ qemux86 architectures.
+
+
+
+
+
+ BitBake uses the
+ do_package_write_*
+ tasks to generate packages and place them into the package
+ holding area (e.g. do_package_write_ipk
+ for IPK packages).
+ See the
+ "do_package_write_deb",
+ "do_package_write_ipk",
+ "do_package_write_rpm",
+ and
+ "do_package_write_tar"
+ sections in the Yocto Project Reference Manual
+ for additional information.
+ As an example, consider a scenario where an IPK packaging
+ manager is being used and package architecture support for
+ both i586 and qemux86 exist.
+ Packages for the i586 architecture are placed in
+ build/tmp/deploy/ipk/i586, while packages
+ for the qemux86 architecture are placed in
+ build/tmp/deploy/ipk/qemux86.
+
+
+
+
+ BitBake
+
+
+ The OpenEmbedded build system uses
+ BitBake
+ to produce images.
+ You can see from the
+ general Yocto Project Development Environment figure,
+ the BitBake area consists of several functional areas.
+ This section takes a closer look at each of those areas.
+
+
+
+ Separate documentation exists for the BitBake tool.
+ See the
+ BitBake User Manual
+ for reference material on BitBake.
+
+
+
+ Source Fetching
+
+
+ The first stages of building a recipe are to fetch and
+ unpack the source code:
+
+
+
+
+ The
+ do_fetch
+ and
+ do_unpack
+ tasks fetch the source files and unpack them into the work
+ directory.
+
+ For every local file (e.g. file://)
+ that is part of a recipe's
+ SRC_URI
+ statement, the OpenEmbedded build system takes a
+ checksum of the file for the recipe and inserts the
+ checksum into the signature for the
+ do_fetch.
+ If any local file has been modified, the
+ do_fetch task and all tasks that
+ depend on it are re-executed.
+
+ By default, everything is accomplished in the
+ Build Directory,
+ which has a defined structure.
+ For additional general information on the Build Directory,
+ see the
+ "build/"
+ section in the Yocto Project Reference Manual.
+
+
+
+ Unpacked source files are pointed to by the
+ S
+ variable.
+ Each recipe has an area in the Build Directory where the
+ unpacked source code resides.
+ The name of that directory for any given recipe is defined
+ from several different variables.
+ You can see the variables that define these directories
+ by looking at the figure:
+
+
+ TMPDIR:
+ The base directory where the OpenEmbedded build
+ system performs all its work during the build.
+
+
+ PACKAGE_ARCH:
+ The architecture of the built package or packages.
+
+
+ TARGET_OS:
+ The operating system of the target device.
+
+
+ PN:
+ The name of the built package.
+
+
+ PV:
+ The version of the recipe used to build the
+ package.
+
+
+ PR:
+ The revision of the recipe used to build the
+ package.
+
+
+ WORKDIR:
+ The location within TMPDIR
+ where a specific package is built.
+
+
+ S:
+ Contains the unpacked source files for a given
+ recipe.
+
+
+
+
+
+
+ Patching
+
+
+ Once source code is fetched and unpacked, BitBake locates
+ patch files and applies them to the source files:
+
+
+
+
+ The
+ do_patch
+ task processes recipes by using the
+ SRC_URI
+ variable to locate applicable patch files, which by default
+ are *.patch or
+ *.diff files, or any file if
+ "apply=yes" is specified for the file in
+ SRC_URI.
+
+
+
+ BitBake finds and applies multiple patches for a single
+ recipe in the order in which it finds the patches.
+ Patches are applied to the recipe's source files located
+ in the
+ S
+ directory.
+
+
+
+ For more information on how the source directories are
+ created, see the
+ "Source Fetching"
+ section.
+
+
+
+
+ Configuration and Compilation
+
+
+ After source code is patched, BitBake executes tasks that
+ configure and compile the source code:
+
+
+
+
+ This step in the build process consists of three tasks:
+
+
+ do_prepare_recipe_sysroot:
+ This task sets up the two sysroots in
+ ${WORKDIR}
+ (i.e. recipe-sysroot and
+ recipe-sysroot-native) so that
+ the sysroots contain the contents of the
+ do_populate_sysroot
+ tasks of the recipes on which the recipe
+ containing the tasks depends.
+ A sysroot exists for both the target and for the
+ native binaries, which run on the host system.
+
+
+ do_configure:
+ This task configures the source by enabling and
+ disabling any build-time and configuration options
+ for the software being built.
+ Configurations can come from the recipe itself as
+ well as from an inherited class.
+ Additionally, the software itself might configure
+ itself depending on the target for which it is
+ being built.
+
+ The configurations handled by the
+ do_configure
+ task are specific to source code configuration for
+ the source code being built by the recipe.
+
+ If you are using the
+ autotools
+ class, you can add additional configuration options
+ by using the
+ EXTRA_OECONF
+ or
+ PACKAGECONFIG_CONFARGS
+ variables.
+ For information on how this variable works within
+ that class, see the
+ meta/classes/autotools.bbclass
+ file.
+
+
+ do_compile:
+ Once a configuration task has been satisfied, BitBake
+ compiles the source using the
+ do_compile
+ task.
+ Compilation occurs in the directory pointed to by
+ the
+ B
+ variable.
+ Realize that the B directory
+ is, by default, the same as the
+ S
+ directory.
+
+
+ do_install:
+ Once compilation is done, BitBake executes the
+ do_install
+ task.
+ This task copies files from the
+ B directory and places them
+ in a holding area pointed to by the
+ D
+ variable.
+
+
+
+
+
+
+ Package Splitting
+
+
+ After source code is configured and compiled, the
+ OpenEmbedded build system analyzes
+ the results and splits the output into packages:
+
+
+
+
+ The
+ do_package
+ and
+ do_packagedata
+ tasks combine to analyze the files found in the
+ D
+ directory and split them into subsets based on available
+ packages and files.
+ The analyzing process involves the following as well as
+ other items: splitting out debugging symbols, looking at
+ shared library dependencies between packages, and looking
+ at package relationships.
+ The do_packagedata task creates
+ package metadata based on the analysis such that the
+ OpenEmbedded build system can generate the final packages.
+ Working, staged, and intermediate results of the analysis
+ and package splitting process use these areas:
+
+
+ PKGD:
+ The destination directory for packages before they
+ are split.
+
+
+ PKGDATA_DIR:
+ A shared, global-state directory that holds data
+ generated during the packaging process.
+
+
+ PKGDESTWORK:
+ A temporary work area used by the
+ do_package task.
+
+
+ PKGDEST:
+ The parent directory for packages after they have
+ been split.
+
+
+ The
+ FILES
+ variable defines the files that go into each package in
+ PACKAGES.
+ If you want details on how this is accomplished, you can
+ look at the
+ package
+ class.
+
+
+
+ Depending on the type of packages being created (RPM, DEB,
+ or IPK), the
+ do_package_write_*
+ task creates the actual packages and places them in the
+ Package Feed area, which is
+ ${TMPDIR}/deploy.
+ You can see the
+ "Package Feeds"
+ section for more detail on that part of the build process.
+
+ Support for creating feeds directly from the
+ deploy/* directories does not
+ exist.
+ Creating such feeds usually requires some kind of feed
+ maintenance mechanism that would upload the new
+ packages into an official package feed (e.g. the
+ Ångström distribution).
+ This functionality is highly distribution-specific
+ and thus is not provided out of the box.
+
+
+
+
+
+ Image Generation
+
+
+ Once packages are split and stored in the Package Feeds
+ area, the OpenEmbedded build system uses BitBake to
+ generate the root filesystem image:
+
+
+
+
+ The image generation process consists of several stages and
+ depends on several tasks and variables.
+ The
+ do_rootfs
+ task creates the root filesystem (file and directory
+ structure) for an image.
+ This task uses several key variables to help create the
+ list of packages to actually install:
+
+
+ IMAGE_INSTALL:
+ Lists out the base set of packages to install from
+ the Package Feeds area.
+
+
+ PACKAGE_EXCLUDE:
+ Specifies packages that should not be installed.
+
+
+ IMAGE_FEATURES:
+ Specifies features to include in the image.
+ Most of these features map to additional packages
+ for installation.
+
+
+ PACKAGE_CLASSES:
+ Specifies the package backend to use and
+ consequently helps determine where to locate
+ packages within the Package Feeds area.
+
+
+ IMAGE_LINGUAS:
+ Determines the language(s) for which additional
+ language support packages are installed.
+
+
+ PACKAGE_INSTALL:
+ The final list of packages passed to the package manager
+ for installation into the image.
+
+
+
+
+
+ With
+ IMAGE_ROOTFS
+ pointing to the location of the filesystem under
+ construction and the PACKAGE_INSTALL
+ variable providing the final list of packages to install,
+ the root file system is created.
+
+
+
+ Package installation is under control of the package
+ manager (e.g. dnf/rpm, opkg, or apt/dpkg) regardless of
+ whether or not package management is enabled for the
+ target.
+ At the end of the process, if package management is not
+ enabled for the target, the package manager's data files
+ are deleted from the root filesystem.
+ As part of the final stage of package installation,
+ postinstall scripts that are part of the packages are run.
+ Any scripts that fail to run
+ on the build host are run on the target when the target
+ system is first booted.
+ If you are using a
+ read-only root filesystem,
+ all the post installation scripts must succeed during the
+ package installation phase since the root filesystem is
+ read-only.
+
+
+
+ The final stages of the do_rootfs task
+ handle post processing.
+ Post processing includes creation of a manifest file and
+ optimizations.
+
+
+
+ The manifest file (.manifest) resides
+ in the same directory as the root filesystem image.
+ This file lists out, line-by-line, the installed packages.
+ The manifest file is useful for the
+ testimage
+ class, for example, to determine whether or not to run
+ specific tests.
+ See the
+ IMAGE_MANIFEST
+ variable for additional information.
+
+
+
+ Optimizing processes run across the image include
+ mklibs, prelink,
+ and any other post-processing commands as defined by the
+ ROOTFS_POSTPROCESS_COMMAND
+ variable.
+ The mklibs process optimizes the size
+ of the libraries, while the prelink
+ process optimizes the dynamic linking of shared libraries
+ to reduce start up time of executables.
+
+
+
+ After the root filesystem is built, processing begins on
+ the image through the
+ do_image
+ task.
+ The build system runs any pre-processing commands as
+ defined by the
+ IMAGE_PREPROCESS_COMMAND
+ variable.
+ This variable specifies a list of functions to call before
+ the OpenEmbedded build system creates the final image
+ output files.
+
+
+
+ The OpenEmbedded build system dynamically creates
+ do_image_* tasks as needed, based
+ on the image types specified in the
+ IMAGE_FSTYPES
+ variable.
+ The process turns everything into an image file or a set of
+ image files and compresses the root filesystem image to
+ reduce the overall size of the image.
+ The formats used for the root filesystem depend on the
+ IMAGE_FSTYPES variable.
+
+
+
+ As an example, a dynamically created task when creating a
+ particular image type would
+ take the following form:
+
+ do_image_type[depends]
+
+ So, if the type as specified by
+ the IMAGE_FSTYPES were
+ ext4, the dynamically generated task
+ would be as follows:
+
+ do_image_ext4[depends]
+
+
+
+
+ The final task involved in image creation is the
+ do_image_complete
+ task.
+ This task completes the image by applying any image
+ post processing as defined through the
+ IMAGE_POSTPROCESS_COMMAND
+ variable.
+ The variable specifies a list of functions to call once the
+ OpenEmbedded build system has created the final image
+ output files.
+
+ The entire image generation process is run under
+ Pseudo.
+ Running under Pseudo ensures that the files in the
+ root filesystem have correct ownership.
+
+
+
+
+
+ SDK Generation
+
+
+ The OpenEmbedded build system uses BitBake to generate the
+ Software Development Kit (SDK) installer script for both
+ the standard and extensible SDKs:
+
+
+ For more information on the cross-development toolchain
+ generation, see the
+ "Cross-Development Toolchain Generation"
+ section.
+ For information on advantages gained when building a
+ cross-development toolchain using the
+ do_populate_sdk
+ task, see the
+ "Building an SDK Installer"
+ section in the Yocto Project Application Development
+ and the Extensible Software Development Kit (SDK)
+ manual.
+
+
+
+
+ Like image generation, the SDK script process consists of
+ several stages and depends on many variables.
+ The do_populate_sdk and
+ do_populate_sdk_ext tasks use these
+ key variables to help create the list of packages to
+ actually install.
+ For information on the variables listed in the figure,
+ see the
+ "Application Development SDK"
+ section.
+
+
+
+ The do_populate_sdk task helps create
+ the standard SDK and handles two parts: a target part and a
+ host part.
+ The target part is the part built for the target hardware
+ and includes libraries and headers.
+ The host part is the part of the SDK that runs on the
+ SDKMACHINE.
+
+
+
+ The do_populate_sdk_ext task helps
+ create the extensible SDK and handles host and target parts
+ differently than its counter part does for the standard SDK.
+ For the extensible SDK, the task encapsulates the build
+ system, which includes everything needed (host and target)
+ for the SDK.
+
+
+
+ Regardless of the type of SDK being constructed, the
+ tasks perform some cleanup after which a cross-development
+ environment setup script and any needed configuration files
+ are created.
+ The final output is the Cross-development
+ toolchain installation script (.sh
+ file), which includes the environment setup script.
+
+
+
+
+ Stamp Files and the Rerunning of Tasks
+
+
+ For each task that completes successfully, BitBake writes a
+ stamp file into the
+ STAMPS_DIR
+ directory.
+ The beginning of the stamp file's filename is determined
+ by the
+ STAMP
+ variable, and the end of the name consists of the task's
+ name and current
+ input checksum.
+
+ This naming scheme assumes that
+ BB_SIGNATURE_HANDLER
+ is "OEBasicHash", which is almost always the case in
+ current OpenEmbedded.
+
+ To determine if a task needs to be rerun, BitBake checks
+ if a stamp file with a matching input checksum exists
+ for the task.
+ If such a stamp file exists, the task's output is
+ assumed to exist and still be valid.
+ If the file does not exist, the task is rerun.
+
+ The stamp mechanism is more general than the
+ shared state (sstate) cache mechanism described in the
+ "Setscene Tasks and Shared State"
+ section.
+ BitBake avoids rerunning any task that has a valid
+ stamp file, not just tasks that can be accelerated
+ through the sstate cache.
+
+ However, you should realize that stamp files only
+ serve as a marker that some work has been done and that
+ these files do not record task output.
+ The actual task output would usually be somewhere in
+ TMPDIR
+ (e.g. in some recipe's
+ WORKDIR.)
+ What the sstate cache mechanism adds is a way to cache
+ task output that can then be shared between build
+ machines.
+
+ Since STAMPS_DIR is usually a
+ subdirectory of TMPDIR, removing
+ TMPDIR will also remove
+ STAMPS_DIR, which means tasks will
+ properly be rerun to repopulate
+ TMPDIR.
+
+
+
+ If you want some task to always be considered "out of
+ date", you can mark it with the
+ nostamp
+ varflag.
+ If some other task depends on such a task, then that
+ task will also always be considered out of date, which
+ might not be what you want.
+
+
+
+ For details on how to view information about a task's
+ signature, see the
+ "Viewing Task Variable Dependencies"
+ section in the Yocto Project Development Tasks Manual.
+
+
+
+
+ Setscene Tasks and Shared State
+
+
+ The description of tasks so far assumes that BitBake needs
+ to build everything and there are no prebuilt objects
+ available.
+ BitBake does support skipping tasks if prebuilt objects are
+ available.
+ These objects are usually made available in the form of a
+ shared state (sstate) cache.
+
+ For information on variables affecting sstate, see the
+ SSTATE_DIR
+ and
+ SSTATE_MIRRORS
+ variables.
+
+
+
+
+ The idea of a setscene task (i.e
+ do_taskname_setscene)
+ is a version of the task where
+ instead of building something, BitBake can skip to the end
+ result and simply place a set of files into specific
+ locations as needed.
+ In some cases, it makes sense to have a setscene task
+ variant (e.g. generating package files in the
+ do_package_write_*
+ task).
+ In other cases, it does not make sense, (e.g. a
+ do_patch
+ task or
+ do_unpack
+ task) since the work involved would be equal to or greater
+ than the underlying task.
+
+
+
+ In the OpenEmbedded build system, the common tasks that
+ have setscene variants are
+ do_package,
+ do_package_write_*,
+ do_deploy,
+ do_packagedata,
+ and
+ do_populate_sysroot.
+ Notice that these are most of the tasks whose output is an
+ end result.
+
+
+
+ The OpenEmbedded build system has knowledge of the
+ relationship between these tasks and other tasks that
+ precede them.
+ For example, if BitBake runs
+ do_populate_sysroot_setscene for
+ something, there is little point in running any of the
+ do_fetch,
+ do_unpack,
+ do_patch,
+ do_configure,
+ do_compile, and
+ do_install tasks.
+ However, if do_package needs to be
+ run, BitBake would need to run those other tasks.
+
+
+
+ It becomes more complicated if everything can come
+ from an sstate cache because some objects are simply
+ not required at all.
+ For example, you do not need a compiler or native tools,
+ such as quilt, if there is nothing to compile or patch.
+ If the do_package_write_* packages
+ are available from sstate, BitBake does not need the
+ do_package task data.
+
+
+
+ To handle all these complexities, BitBake runs in two
+ phases.
+ The first is the "setscene" stage.
+ During this stage, BitBake first checks the sstate cache
+ for any targets it is planning to build.
+ BitBake does a fast check to see if the object exists
+ rather than a complete download.
+ If nothing exists, the second phase, which is the setscene
+ stage, completes and the main build proceeds.
+
+
+
+ If objects are found in the sstate cache, the OpenEmbedded
+ build system works backwards from the end targets specified
+ by the user.
+ For example, if an image is being built, the OpenEmbedded
+ build system first looks for the packages needed for
+ that image and the tools needed to construct an image.
+ If those are available, the compiler is not needed.
+ Thus, the compiler is not even downloaded.
+ If something was found to be unavailable, or the
+ download or setscene task fails, the OpenEmbedded build
+ system then tries to install dependencies, such as the
+ compiler, from the cache.
+
+
+
+ The availability of objects in the sstate cache is
+ handled by the function specified by the
+ BB_HASHCHECK_FUNCTION
+ variable and returns a list of the objects that are
+ available.
+ The function specified by the
+ BB_SETSCENE_DEPVALID
+ variable is the function that determines whether a given
+ dependency needs to be followed, and whether for any given
+ relationship the function needs to be passed.
+ The function returns a True or False value.
+
+
+
+
+
+ Images
+
+
+ The images produced by the OpenEmbedded build system
+ are compressed forms of the
+ root filesystem that are ready to boot on a target device.
+ You can see from the
+ general Yocto Project Development Environment figure
+ that BitBake output, in part, consists of images.
+ This section is going to look more closely at this output:
+
+
+
+
+ For a list of example images that the Yocto Project provides,
+ see the
+ "Images"
+ chapter in the Yocto Project Reference Manual.
+
+
+
+ Images are written out to the
+ Build Directory
+ inside the
+ tmp/deploy/images/machine/
+ folder as shown in the figure.
+ This folder contains any files expected to be loaded on the
+ target device.
+ The
+ DEPLOY_DIR
+ variable points to the deploy directory,
+ while the
+ DEPLOY_DIR_IMAGE
+ variable points to the appropriate directory containing images
+ for the current configuration.
+
+
+ kernel-image:
+ A kernel binary file.
+ The
+ KERNEL_IMAGETYPE
+ variable setting determines the naming scheme for the
+ kernel image file.
+ Depending on that variable, the file could begin with
+ a variety of naming strings.
+ The
+ deploy/images/machine
+ directory can contain multiple image files for the
+ machine.
+
+
+ root-filesystem-image:
+ Root filesystems for the target device (e.g.
+ *.ext3 or
+ *.bz2 files).
+ The
+ IMAGE_FSTYPES
+ variable setting determines the root filesystem image
+ type.
+ The
+ deploy/images/machine
+ directory can contain multiple root filesystems for the
+ machine.
+
+
+ kernel-modules:
+ Tarballs that contain all the modules built for the
+ kernel.
+ Kernel module tarballs exist for legacy purposes and
+ can be suppressed by setting the
+ MODULE_TARBALL_DEPLOY
+ variable to "0".
+ The
+ deploy/images/machine
+ directory can contain multiple kernel module tarballs
+ for the machine.
+
+
+ bootloaders:
+ Bootloaders supporting the image, if applicable to the
+ target machine.
+ The deploy/images/machine
+ directory can contain multiple bootloaders for the
+ machine.
+
+
+ symlinks:
+ The
+ deploy/images/machine
+ folder contains a symbolic link that points to the
+ most recently built file for each machine.
+ These links might be useful for external scripts that
+ need to obtain the latest version of each file.
+
+
+
+
+
+
+ Application Development SDK
+
+
+ In the
+ general Yocto Project Development Environment figure,
+ the output labeled "Application Development SDK" represents an
+ SDK.
+ The SDK generation process differs depending on whether you
+ build a standard SDK (e.g.
+ bitbake -c populate_sdkimagename)
+ or an extensible SDK (e.g.
+ bitbake -c populate_sdk_extimagename).
+ This section is going to take a closer look at this output:
+
+
+
+
+ The specific form of this output is a self-extracting
+ SDK installer (*.sh) that, when run,
+ installs the SDK, which consists of a cross-development
+ toolchain, a set of libraries and headers, and an SDK
+ environment setup script.
+ Running this installer essentially sets up your
+ cross-development environment.
+ You can think of the cross-toolchain as the "host"
+ part because it runs on the SDK machine.
+ You can think of the libraries and headers as the "target"
+ part because they are built for the target hardware.
+ The environment setup script is added so that you can
+ initialize the environment before using the tools.
+
+
+ Notes
+
+
+ The Yocto Project supports several methods by which
+ you can set up this cross-development environment.
+ These methods include downloading pre-built SDK
+ installers or building and installing your own SDK
+ installer.
+
+
+ For background information on cross-development
+ toolchains in the Yocto Project development
+ environment, see the
+ "Cross-Development Toolchain Generation"
+ section.
+
+
+ For information on setting up a cross-development
+ environment, see the
+ Yocto Project Application Development and the Extensible Software Development Kit (eSDK)
+ manual.
+
+
+
+
+
+ Once built, the SDK installers are written out to the
+ deploy/sdk folder inside the
+ Build Directory
+ as shown in the figure at the beginning of this section.
+ Depending on the type of SDK, several variables exist that help
+ configure these files.
+ The following list shows the variables associated with
+ a standard SDK:
+
+
+ DEPLOY_DIR:
+ Points to the deploy
+ directory.
+
+
+ SDKMACHINE:
+ Specifies the architecture of the machine
+ on which the cross-development tools are run to
+ create packages for the target hardware.
+
+
+ SDKIMAGE_FEATURES:
+ Lists the features to include in the "target" part
+ of the SDK.
+
+
+ TOOLCHAIN_HOST_TASK:
+ Lists packages that make up the host
+ part of the SDK (i.e. the part that runs on
+ the SDKMACHINE).
+ When you use
+ bitbake -c populate_sdk imagename
+ to create the SDK, a set of default packages
+ apply.
+ This variable allows you to add more packages.
+
+
+ TOOLCHAIN_TARGET_TASK:
+ Lists packages that make up the target part
+ of the SDK (i.e. the part built for the
+ target hardware).
+
+
+ SDKPATH:
+ Defines the default SDK installation path offered
+ by the installation script.
+
+
+ This next list, shows the variables associated with an
+ extensible SDK:
+
+
+ DEPLOY_DIR:
+ Points to the deploy directory.
+
+
+ SDK_EXT_TYPE:
+ Controls whether or not shared state artifacts are
+ copied into the extensible SDK.
+ By default, all required shared state artifacts are
+ copied into the SDK.
+
+
+ SDK_INCLUDE_PKGDATA:
+ Specifies whether or not packagedata will be
+ included in the extensible SDK for all recipes in
+ the "world" target.
+
+
+ SDK_INCLUDE_TOOLCHAIN:
+ Specifies whether or not the toolchain will be included
+ when building the extensible SDK.
+
+
+ SDK_LOCAL_CONF_WHITELIST:
+ A list of variables allowed through from the build
+ system configuration into the extensible SDK
+ configuration.
+
+
+ SDK_LOCAL_CONF_BLACKLIST:
+ A list of variables not allowed through from the build
+ system configuration into the extensible SDK
+ configuration.
+
+
+ SDK_INHERIT_BLACKLIST:
+ A list of classes to remove from the
+ INHERIT
+ value globally within the extensible SDK configuration.
+
+
+
+
+
+
Cross-Development Toolchain Generation
diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml
index dbbe67657d..8e8b06fba1 100644
--- a/documentation/dev-manual/dev-manual-common-tasks.xml
+++ b/documentation/dev-manual/dev-manual-common-tasks.xml
@@ -1815,8 +1815,8 @@
Your recipe must have a SRC_URI variable
that points to where the source is located.
For a graphical representation of source locations, see the
- "Sources"
- section in the Getting Started With Yocto Project Manual.
+ "Sources"
+ section in the Yocto Project Concepts Manual.
@@ -11457,8 +11457,8 @@ Some notes from Cal:
-c, BitBake will only run the task if it
considers it "out of date".
See the
- "Stamp Files and the Rerunning of Tasks"
- section in the Getting Started With Yocto Project Manual for
+ "Stamp Files and the Rerunning of Tasks"
+ section in the Yocto Project Concepts Manual for
how BitBake determines whether a task is "out of date".
diff --git a/documentation/dev-manual/dev-manual-start.xml b/documentation/dev-manual/dev-manual-start.xml
index f14a91f94b..39d36fa0f1 100644
--- a/documentation/dev-manual/dev-manual-start.xml
+++ b/documentation/dev-manual/dev-manual-start.xml
@@ -741,8 +741,8 @@
Build Directory
under tmp/deploy/images.
For detailed information on the build process using BitBake, see the
- "Images"
- section in the Getting Started With Yocto Project Manual.
+ "Images"
+ section in the Yocto Project Concepts Manual.
diff --git a/documentation/getting-started/getting-started-development-environment.xml b/documentation/getting-started/getting-started-development-environment.xml
index 096517b729..5d6491c51b 100644
--- a/documentation/getting-started/getting-started-development-environment.xml
+++ b/documentation/getting-started/getting-started-development-environment.xml
@@ -937,1656 +937,6 @@
section in the Yocto Project Development Tasks Manual.
-
-
- Development Concepts
-
-
- This section takes a more detailed look inside the development
- process.
- The following diagram represents development at a high level.
- The remainder of this chapter expands on the fundamental input, output,
- process, and
- Metadata) blocks
- that make up development in the Yocto Project environment.
-
-
-
-
-
-
-
- In general, development consists of several functional areas:
-
- User Configuration:
- Metadata you can use to control the build process.
-
- Metadata Layers:
- Various layers that provide software, machine, and
- distro Metadata.
- Source Files:
- Upstream releases, local projects, and SCMs.
- Build System:
- Processes under the control of
- BitBake.
- This block expands on how BitBake fetches source, applies
- patches, completes compilation, analyzes output for package
- generation, creates and tests packages, generates images, and
- generates cross-development tools.
- Package Feeds:
- Directories containing output packages (RPM, DEB or IPK),
- which are subsequently used in the construction of an image or
- SDK, produced by the build system.
- These feeds can also be copied and shared using a web server or
- other means to facilitate extending or updating existing
- images on devices at runtime if runtime package management is
- enabled.
- Images:
- Images produced by the development process.
-
- Application Development SDK:
- Cross-development tools that are produced along with an image
- or separately with BitBake.
-
-
-
-
- User Configuration
-
-
- User configuration helps define the build.
- Through user configuration, you can tell BitBake the
- target architecture for which you are building the image,
- where to store downloaded source, and other build properties.
-
-
-
- The following figure shows an expanded representation of the
- "User Configuration" box of the
- general Yocto Project Development Environment figure:
-
-
-
-
-
-
-
- BitBake needs some basic configuration files in order to complete
- a build.
- These files are *.conf files.
- The minimally necessary ones reside as example files in the
- Source Directory.
- For simplicity, this section refers to the Source Directory as
- the "Poky Directory."
-
-
-
- When you clone the poky Git repository or you
- download and unpack a Yocto Project release, you can set up the
- Source Directory to be named anything you want.
- For this discussion, the cloned repository uses the default
- name poky.
-
- The Poky repository is primarily an aggregation of existing
- repositories.
- It is not a canonical upstream source.
-
-
-
-
- The meta-poky layer inside Poky contains
- a conf directory that has example
- configuration files.
- These example files are used as a basis for creating actual
- configuration files when you source the build environment
- script
- (i.e.
- &OE_INIT_FILE;).
-
-
-
- Sourcing the build environment script creates a
- Build Directory
- if one does not already exist.
- BitBake uses the Build Directory for all its work during builds.
- The Build Directory has a conf directory that
- contains default versions of your local.conf
- and bblayers.conf configuration files.
- These default configuration files are created only if versions
- do not already exist in the Build Directory at the time you
- source the build environment setup script.
-
-
-
- Because the Poky repository is fundamentally an aggregation of
- existing repositories, some users might be familiar with running
- the &OE_INIT_FILE; script in the context
- of separate OpenEmbedded-Core and BitBake repositories rather than a
- single Poky repository.
- This discussion assumes the script is executed from within a cloned
- or unpacked version of Poky.
-
-
-
- Depending on where the script is sourced, different sub-scripts
- are called to set up the Build Directory (Yocto or OpenEmbedded).
- Specifically, the script
- scripts/oe-setup-builddir inside the
- poky directory sets up the Build Directory and seeds the directory
- (if necessary) with configuration files appropriate for the
- Yocto Project development environment.
-
- The scripts/oe-setup-builddir script
- uses the $TEMPLATECONF variable to
- determine which sample configuration files to locate.
-
-
-
-
- The local.conf file provides many
- basic variables that define a build environment.
- Here is a list of a few.
- To see the default configurations in a local.conf
- file created by the build environment script, see the
- local.conf.sample in the
- meta-poky layer:
-
- Parallelism Options:
- Controlled by the
- BB_NUMBER_THREADS,
- PARALLEL_MAKE,
- and
- BB_NUMBER_PARSE_THREADS
- variables.
- Target Machine Selection:
- Controlled by the
- MACHINE
- variable.
- Download Directory:
- Controlled by the
- DL_DIR
- variable.
- Shared State Directory:
- Controlled by the
- SSTATE_DIR
- variable.
- Build Output:
- Controlled by the
- TMPDIR
- variable.
-
-
- Configurations set in the conf/local.conf
- file can also be set in the
- conf/site.conf and
- conf/auto.conf configuration files.
-
-
-
-
- The bblayers.conf file tells BitBake what
- layers you want considered during the build.
- By default, the layers listed in this file include layers
- minimally needed by the build system.
- However, you must manually add any custom layers you have created.
- You can find more information on working with the
- bblayers.conf file in the
- "Enabling Your Layer"
- section in the Yocto Project Development Tasks Manual.
-
-
-
- The files site.conf and
- auto.conf are not created by the environment
- initialization script.
- If you want the site.conf file, you need to
- create that yourself.
- The auto.conf file is typically created by
- an autobuilder:
-
- site.conf:
- You can use the conf/site.conf
- configuration file to configure multiple build directories.
- For example, suppose you had several build environments and
- they shared some common features.
- You can set these default build properties here.
- A good example is perhaps the packaging format to use
- through the
- PACKAGE_CLASSES
- variable.
- One useful scenario for using the
- conf/site.conf file is to extend your
- BBPATH
- variable to include the path to a
- conf/site.conf.
- Then, when BitBake looks for Metadata using
- BBPATH, it finds the
- conf/site.conf file and applies your
- common configurations found in the file.
- To override configurations in a particular build directory,
- alter the similar configurations within that build
- directory's conf/local.conf file.
-
- auto.conf:
- The file is usually created and written to by
- an autobuilder.
- The settings put into the file are typically the same as
- you would find in the conf/local.conf
- or the conf/site.conf files.
-
-
-
-
-
- You can edit all configuration files to further define
- any particular build environment.
- This process is represented by the "User Configuration Edits"
- box in the figure.
-
-
-
- When you launch your build with the
- bitbake target
- command, BitBake sorts out the configurations to ultimately
- define your build environment.
- It is important to understand that the OpenEmbedded build system
- reads the configuration files in a specific order:
- site.conf, auto.conf,
- and local.conf.
- And, the build system applies the normal assignment statement
- rules.
- Because the files are parsed in a specific order, variable
- assignments for the same variable could be affected.
- For example, if the auto.conf file and
- the local.conf set
- variable1 to different values, because
- the build system parses local.conf after
- auto.conf,
- variable1 is assigned the value from
- the local.conf file.
-
-
-
-
- Metadata, Machine Configuration, and Policy Configuration
-
-
- The previous section described the user configurations that
- define BitBake's global behavior.
- This section takes a closer look at the layers the build system
- uses to further control the build.
- These layers provide Metadata for the software, machine, and
- policy.
-
-
-
- In general, three types of layer input exist:
-
- Policy Configuration:
- Distribution Layers provide top-level or general
- policies for the image or SDK being built.
- For example, this layer would dictate whether BitBake
- produces RPM or IPK packages.
- Machine Configuration:
- Board Support Package (BSP) layers provide machine
- configurations.
- This type of information is specific to a particular
- target architecture.
- Metadata:
- Software layers contain user-supplied recipe files,
- patches, and append files.
-
-
-
-
-
- The following figure shows an expanded representation of the
- Metadata, Machine Configuration, and Policy Configuration input
- (layers) boxes of the
- general Yocto Project Development Environment figure:
-
-
-
-
-
-
-
- In general, all layers have a similar structure.
- They all contain a licensing file
- (e.g. COPYING) if the layer is to be
- distributed, a README file as good practice
- and especially if the layer is to be distributed, a
- configuration directory, and recipe directories.
-
-
-
- The Yocto Project has many layers that can be used.
- You can see a web-interface listing of them on the
- Source Repositories
- page.
- The layers are shown at the bottom categorized under
- "Yocto Metadata Layers."
- These layers are fundamentally a subset of the
- OpenEmbedded Metadata Index,
- which lists all layers provided by the OpenEmbedded community.
-
- Layers exist in the Yocto Project Source Repositories that
- cannot be found in the OpenEmbedded Metadata Index.
- These layers are either deprecated or experimental in nature.
-
-
-
-
- BitBake uses the conf/bblayers.conf file,
- which is part of the user configuration, to find what layers it
- should be using as part of the build.
-
-
-
- For more information on layers, see the
- "Understanding and Creating Layers"
- section in the Yocto Project Development Tasks Manual.
-
-
-
- Distro Layer
-
-
- The distribution layer provides policy configurations for your
- distribution.
- Best practices dictate that you isolate these types of
- configurations into their own layer.
- Settings you provide in
- conf/distro/distro.conf override
- similar
- settings that BitBake finds in your
- conf/local.conf file in the Build
- Directory.
-
-
-
- The following list provides some explanation and references
- for what you typically find in the distribution layer:
-
- classes:
- Class files (.bbclass) hold
- common functionality that can be shared among
- recipes in the distribution.
- When your recipes inherit a class, they take on the
- settings and functions for that class.
- You can read more about class files in the
- "Classes"
- section of the Yocto Reference Manual.
-
- conf:
- This area holds configuration files for the
- layer (conf/layer.conf),
- the distribution
- (conf/distro/distro.conf),
- and any distribution-wide include files.
-
- recipes-*:
- Recipes and append files that affect common
- functionality across the distribution.
- This area could include recipes and append files
- to add distribution-specific configuration,
- initialization scripts, custom image recipes,
- and so forth.
-
-
-
-
-
- BSP Layer
-
-
- The BSP Layer provides machine configurations.
- Everything in this layer is specific to the machine for which
- you are building the image or the SDK.
- A common structure or form is defined for BSP layers.
- You can learn more about this structure in the
- Yocto Project Board Support Package (BSP) Developer's Guide.
-
- In order for a BSP layer to be considered compliant with the
- Yocto Project, it must meet some structural requirements.
-
-
-
-
- The BSP Layer's configuration directory contains
- configuration files for the machine
- (conf/machine/machine.conf) and,
- of course, the layer (conf/layer.conf).
-
-
-
- The remainder of the layer is dedicated to specific recipes
- by function: recipes-bsp,
- recipes-core,
- recipes-graphics, and
- recipes-kernel.
- Metadata can exist for multiple formfactors, graphics
- support systems, and so forth.
-
- While the figure shows several recipes-*
- directories, not all these directories appear in all
- BSP layers.
-
-
-
-
-
- Software Layer
-
-
- The software layer provides the Metadata for additional
- software packages used during the build.
- This layer does not include Metadata that is specific to the
- distribution or the machine, which are found in their
- respective layers.
-
-
-
- This layer contains any new recipes that your project needs
- in the form of recipe files.
-
-
-
-
-
- Sources
-
-
- In order for the OpenEmbedded build system to create an image or
- any target, it must be able to access source files.
- The
- general Yocto Project Development Environment figure
- represents source files using the "Upstream Project Releases",
- "Local Projects", and "SCMs (optional)" boxes.
- The figure represents mirrors, which also play a role in locating
- source files, with the "Source Mirror(s)" box.
-
-
-
- The method by which source files are ultimately organized is
- a function of the project.
- For example, for released software, projects tend to use tarballs
- or other archived files that can capture the state of a release
- guaranteeing that it is statically represented.
- On the other hand, for a project that is more dynamic or
- experimental in nature, a project might keep source files in a
- repository controlled by a Source Control Manager (SCM) such as
- Git.
- Pulling source from a repository allows you to control
- the point in the repository (the revision) from which you want to
- build software.
- Finally, a combination of the two might exist, which would give the
- consumer a choice when deciding where to get source files.
-
-
-
- BitBake uses the
- SRC_URI
- variable to point to source files regardless of their location.
- Each recipe must have a SRC_URI variable
- that points to the source.
-
-
-
- Another area that plays a significant role in where source files
- come from is pointed to by the
- DL_DIR
- variable.
- This area is a cache that can hold previously downloaded source.
- You can also instruct the OpenEmbedded build system to create
- tarballs from Git repositories, which is not the default behavior,
- and store them in the DL_DIR by using the
- BB_GENERATE_MIRROR_TARBALLS
- variable.
-
-
-
- Judicious use of a DL_DIR directory can
- save the build system a trip across the Internet when looking
- for files.
- A good method for using a download directory is to have
- DL_DIR point to an area outside of your
- Build Directory.
- Doing so allows you to safely delete the Build Directory
- if needed without fear of removing any downloaded source file.
-
-
-
- The remainder of this section provides a deeper look into the
- source files and the mirrors.
- Here is a more detailed look at the source file area of the
- base figure:
-
-
-
-
- Upstream Project Releases
-
-
- Upstream project releases exist anywhere in the form of an
- archived file (e.g. tarball or zip file).
- These files correspond to individual recipes.
- For example, the figure uses specific releases each for
- BusyBox, Qt, and Dbus.
- An archive file can be for any released product that can be
- built using a recipe.
-
-
-
-
- Local Projects
-
-
- Local projects are custom bits of software the user provides.
- These bits reside somewhere local to a project - perhaps
- a directory into which the user checks in items (e.g.
- a local directory containing a development source tree
- used by the group).
-
-
-
- The canonical method through which to include a local project
- is to use the
- externalsrc
- class to include that local project.
- You use either the local.conf or a
- recipe's append file to override or set the
- recipe to point to the local directory on your disk to pull
- in the whole source tree.
-
-
-
- For information on how to use the
- externalsrc class, see the
- "externalsrc.bbclass"
- section.
-
-
-
-
- Source Control Managers (Optional)
-
-
- Another place the build system can get source files from is
- through an SCM such as Git or Subversion.
- In this case, a repository is cloned or checked out.
- The
- do_fetch
- task inside BitBake uses
- the SRC_URI
- variable and the argument's prefix to determine the correct
- fetcher module.
-
-
-
- For information on how to have the OpenEmbedded build system
- generate tarballs for Git repositories and place them in the
- DL_DIR
- directory, see the
- BB_GENERATE_MIRROR_TARBALLS
- variable.
-
-
-
- When fetching a repository, BitBake uses the
- SRCREV
- variable to determine the specific revision from which to
- build.
-
-
-
-
- Source Mirror(s)
-
-
- Two kinds of mirrors exist: pre-mirrors and regular mirrors.
- The
- PREMIRRORS
- and
- MIRRORS
- variables point to these, respectively.
- BitBake checks pre-mirrors before looking upstream for any
- source files.
- Pre-mirrors are appropriate when you have a shared directory
- that is not a directory defined by the
- DL_DIR
- variable.
- A Pre-mirror typically points to a shared directory that is
- local to your organization.
-
-
-
- Regular mirrors can be any site across the Internet that is
- used as an alternative location for source code should the
- primary site not be functioning for some reason or another.
-
-
-
-
-
- Package Feeds
-
-
- When the OpenEmbedded build system generates an image or an SDK,
- it gets the packages from a package feed area located in the
- Build Directory.
- The
- general Yocto Project Development Environment figure
- shows this package feeds area in the upper-right corner.
-
-
-
- This section looks a little closer into the package feeds area used
- by the build system.
- Here is a more detailed look at the area:
-
-
-
-
- Package feeds are an intermediary step in the build process.
- The OpenEmbedded build system provides classes to generate
- different package types, and you specify which classes to enable
- through the
- PACKAGE_CLASSES
- variable.
- Before placing the packages into package feeds,
- the build process validates them with generated output quality
- assurance checks through the
- insane
- class.
-
-
-
- The package feed area resides in the Build Directory.
- The directory the build system uses to temporarily store packages
- is determined by a combination of variables and the particular
- package manager in use.
- See the "Package Feeds" box in the illustration and note the
- information to the right of that area.
- In particular, the following defines where package files are
- kept:
-
- DEPLOY_DIR:
- Defined as tmp/deploy in the Build
- Directory.
-
- DEPLOY_DIR_*:
- Depending on the package manager used, the package type
- sub-folder.
- Given RPM, IPK, or DEB packaging and tarball creation, the
- DEPLOY_DIR_RPM,
- DEPLOY_DIR_IPK,
- DEPLOY_DIR_DEB,
- or
- DEPLOY_DIR_TAR,
- variables are used, respectively.
-
- PACKAGE_ARCH:
- Defines architecture-specific sub-folders.
- For example, packages could exist for the i586 or qemux86
- architectures.
-
-
-
-
-
- BitBake uses the do_package_write_* tasks to
- generate packages and place them into the package holding area (e.g.
- do_package_write_ipk for IPK packages).
- See the
- "do_package_write_deb",
- "do_package_write_ipk",
- "do_package_write_rpm",
- and
- "do_package_write_tar"
- sections for additional information.
- As an example, consider a scenario where an IPK packaging manager
- is being used and package architecture support for both i586
- and qemux86 exist.
- Packages for the i586 architecture are placed in
- build/tmp/deploy/ipk/i586, while packages for
- the qemux86 architecture are placed in
- build/tmp/deploy/ipk/qemux86.
-
-
-
-
- BitBake
-
-
- The OpenEmbedded build system uses
- BitBake
- to produce images.
- You can see from the
- general Yocto Project Development Environment figure,
- the BitBake area consists of several functional areas.
- This section takes a closer look at each of those areas.
-
-
-
- Separate documentation exists for the BitBake tool.
- See the
- BitBake User Manual
- for reference material on BitBake.
-
-
-
- Source Fetching
-
-
- The first stages of building a recipe are to fetch and unpack
- the source code:
-
-
-
-
- The
- do_fetch
- and
- do_unpack
- tasks fetch the source files and unpack them into the work
- directory.
-
- For every local file (e.g. file://)
- that is part of a recipe's
- SRC_URI
- statement, the OpenEmbedded build system takes a checksum
- of the file for the recipe and inserts the checksum into
- the signature for the do_fetch.
- If any local file has been modified, the
- do_fetch task and all tasks that
- depend on it are re-executed.
-
- By default, everything is accomplished in the
- Build Directory,
- which has a defined structure.
- For additional general information on the Build Directory,
- see the
- "build/"
- section in the Yocto Project Reference Manual.
-
-
-
- Unpacked source files are pointed to by the
- S
- variable.
- Each recipe has an area in the Build Directory where the
- unpacked source code resides.
- The name of that directory for any given recipe is defined from
- several different variables.
- You can see the variables that define these directories
- by looking at the figure:
-
- TMPDIR -
- The base directory where the OpenEmbedded build system
- performs all its work during the build.
-
- PACKAGE_ARCH -
- The architecture of the built package or packages.
-
- TARGET_OS -
- The operating system of the target device.
-
- PN -
- The name of the built package.
-
- PV -
- The version of the recipe used to build the package.
-
- PR -
- The revision of the recipe used to build the package.
-
- WORKDIR -
- The location within TMPDIR where
- a specific package is built.
-
- S -
- Contains the unpacked source files for a given recipe.
-
-
-
-
-
-
- Patching
-
-
- Once source code is fetched and unpacked, BitBake locates
- patch files and applies them to the source files:
-
-
-
-
- The
- do_patch
- task processes recipes by
- using the
- SRC_URI
- variable to locate applicable patch files, which by default
- are *.patch or
- *.diff files, or any file if
- "apply=yes" is specified for the file in
- SRC_URI.
-
-
-
- BitBake finds and applies multiple patches for a single recipe
- in the order in which it finds the patches.
- Patches are applied to the recipe's source files located in the
- S
- directory.
-
-
-
- For more information on how the source directories are
- created, see the
- "Source Fetching"
- section.
-
-
-
-
- Configuration and Compilation
-
-
- After source code is patched, BitBake executes tasks that
- configure and compile the source code:
-
-
-
-
- This step in the build process consists of three tasks:
-
-
- do_prepare_recipe_sysroot:
- This task sets up the two sysroots in
- ${WORKDIR}
- (i.e. recipe-sysroot and
- recipe-sysroot-native) so that
- the sysroots contain the contents of the
- do_populate_sysroot
- tasks of the recipes on which the recipe
- containing the tasks depends.
- A sysroot exists for both the target and for the native
- binaries, which run on the host system.
-
- do_configure:
- This task configures the source by enabling and
- disabling any build-time and configuration options for
- the software being built.
- Configurations can come from the recipe itself as well
- as from an inherited class.
- Additionally, the software itself might configure itself
- depending on the target for which it is being built.
-
-
- The configurations handled by the
- do_configure
- task are specific
- to source code configuration for the source code
- being built by the recipe.
-
- If you are using the
- autotools
- class,
- you can add additional configuration options by using
- the
- EXTRA_OECONF
- or
- PACKAGECONFIG_CONFARGS
- variables.
- For information on how this variable works within
- that class, see the
- meta/classes/autotools.bbclass file.
-
- do_compile:
- Once a configuration task has been satisfied, BitBake
- compiles the source using the
- do_compile
- task.
- Compilation occurs in the directory pointed to by the
- B
- variable.
- Realize that the B directory is, by
- default, the same as the
- S
- directory.
- do_install:
- Once compilation is done, BitBake executes the
- do_install
- task.
- This task copies files from the B
- directory and places them in a holding area pointed to
- by the
- D
- variable.
-
-
-
-
-
- Package Splitting
-
-
- After source code is configured and compiled, the
- OpenEmbedded build system analyzes
- the results and splits the output into packages:
-
-
-
-
- The
- do_package
- and
- do_packagedata
- tasks combine to analyze
- the files found in the
- D directory
- and split them into subsets based on available packages and
- files.
- The analyzing process involves the following as well as other
- items: splitting out debugging symbols,
- looking at shared library dependencies between packages,
- and looking at package relationships.
- The do_packagedata task creates package
- metadata based on the analysis such that the
- OpenEmbedded build system can generate the final packages.
- Working, staged, and intermediate results of the analysis
- and package splitting process use these areas:
-
- PKGD -
- The destination directory for packages before they are
- split.
-
- PKGDATA_DIR -
- A shared, global-state directory that holds data
- generated during the packaging process.
-
- PKGDESTWORK -
- A temporary work area used by the
- do_package task.
-
- PKGDEST -
- The parent directory for packages after they have
- been split.
-
-
- The FILES
- variable defines the files that go into each package in
- PACKAGES.
- If you want details on how this is accomplished, you can
- look at the
- package
- class.
-
-
-
- Depending on the type of packages being created (RPM, DEB, or
- IPK), the do_package_write_* task
- creates the actual packages and places them in the
- Package Feed area, which is
- ${TMPDIR}/deploy.
- You can see the
- "Package Feeds"
- section for more detail on that part of the build process.
-
- Support for creating feeds directly from the
- deploy/* directories does not exist.
- Creating such feeds usually requires some kind of feed
- maintenance mechanism that would upload the new packages
- into an official package feed (e.g. the
- Ångström distribution).
- This functionality is highly distribution-specific
- and thus is not provided out of the box.
-
-
-
-
-
- Image Generation
-
-
- Once packages are split and stored in the Package Feeds area,
- the OpenEmbedded build system uses BitBake to generate the
- root filesystem image:
-
-
-
-
- The image generation process consists of several stages and
- depends on several tasks and variables.
- The
- do_rootfs
- task creates the root filesystem (file and directory structure)
- for an image.
- This task uses several key variables to help create the list
- of packages to actually install:
-
- IMAGE_INSTALL:
- Lists out the base set of packages to install from
- the Package Feeds area.
- PACKAGE_EXCLUDE:
- Specifies packages that should not be installed.
-
- IMAGE_FEATURES:
- Specifies features to include in the image.
- Most of these features map to additional packages for
- installation.
- PACKAGE_CLASSES:
- Specifies the package backend to use and consequently
- helps determine where to locate packages within the
- Package Feeds area.
- IMAGE_LINGUAS:
- Determines the language(s) for which additional
- language support packages are installed.
-
- PACKAGE_INSTALL:
- The final list of packages passed to the package manager
- for installation into the image.
-
-
-
-
-
- With
- IMAGE_ROOTFS
- pointing to the location of the filesystem under construction and
- the PACKAGE_INSTALL variable providing the
- final list of packages to install, the root file system is
- created.
-
-
-
- Package installation is under control of the package manager
- (e.g. dnf/rpm, opkg, or apt/dpkg) regardless of whether or
- not package management is enabled for the target.
- At the end of the process, if package management is not
- enabled for the target, the package manager's data files
- are deleted from the root filesystem.
- As part of the final stage of package installation, postinstall
- scripts that are part of the packages are run.
- Any scripts that fail to run
- on the build host are run on the target when the target system
- is first booted.
- If you are using a
- read-only root filesystem,
- all the post installation scripts must succeed during the
- package installation phase since the root filesystem is
- read-only.
-
-
-
- The final stages of the do_rootfs task
- handle post processing.
- Post processing includes creation of a manifest file and
- optimizations.
-
-
-
- The manifest file (.manifest) resides
- in the same directory as the root filesystem image.
- This file lists out, line-by-line, the installed packages.
- The manifest file is useful for the
- testimage
- class, for example, to determine whether or not to run
- specific tests.
- See the
- IMAGE_MANIFEST
- variable for additional information.
-
-
-
- Optimizing processes run across the image include
- mklibs, prelink,
- and any other post-processing commands as defined by the
- ROOTFS_POSTPROCESS_COMMAND
- variable.
- The mklibs process optimizes the size
- of the libraries, while the
- prelink process optimizes the dynamic
- linking of shared libraries to reduce start up time of
- executables.
-
-
-
- After the root filesystem is built, processing begins on
- the image through the
- do_image
- task.
- The build system runs any pre-processing commands as defined
- by the
- IMAGE_PREPROCESS_COMMAND
- variable.
- This variable specifies a list of functions to call before
- the OpenEmbedded build system creates the final image output
- files.
-
-
-
- The OpenEmbedded build system dynamically creates
- do_image_* tasks as needed, based
- on the image types specified in the
- IMAGE_FSTYPES
- variable.
- The process turns everything into an image file or a set of
- image files and compresses the root filesystem image to reduce
- the overall size of the image.
- The formats used for the root filesystem depend on the
- IMAGE_FSTYPES variable.
-
-
-
- As an example, a dynamically created task when creating a
- particular image type would take the
- following form:
-
- do_image_type[depends]
-
- So, if the type as specified by the
- IMAGE_FSTYPES were
- ext4, the dynamically generated task
- would be as follows:
-
- do_image_ext4[depends]
-
-
-
-
- The final task involved in image creation is the
- do_image_complete
- task.
- This task completes the image by applying any image
- post processing as defined through the
- IMAGE_POSTPROCESS_COMMAND
- variable.
- The variable specifies a list of functions to call once the
- OpenEmbedded build system has created the final image output
- files.
-
-
-
- The entire image generation process is run under Pseudo.
- Running under Pseudo ensures that the files in the root
- filesystem have correct ownership.
-
-
-
-
- SDK Generation
-
-
- The OpenEmbedded build system uses BitBake to generate the
- Software Development Kit (SDK) installer script for both the
- standard and extensible SDKs:
-
-
-
-
- For more information on the cross-development toolchain
- generation, see the
- "Cross-Development Toolchain Generation"
- section in the Yocto Project Concepts Manual.
- For information on advantages gained when building a
- cross-development toolchain using the
- do_populate_sdk
- task, see the
- "Building an SDK Installer"
- section in the Yocto Project Application Development and the
- Extensible Software Development Kit (SDK) manual.
-
-
-
- Like image generation, the SDK script process consists of
- several stages and depends on many variables.
- The do_populate_sdk and
- do_populate_sdk_ext tasks use these
- key variables to help create the list of packages to actually
- install.
- For information on the variables listed in the figure, see the
- "Application Development SDK"
- section.
-
-
-
- The do_populate_sdk task helps create
- the standard SDK and handles two parts: a target part and a
- host part.
- The target part is the part built for the target hardware and
- includes libraries and headers.
- The host part is the part of the SDK that runs on the
- SDKMACHINE.
-
-
-
- The do_populate_sdk_ext task helps create
- the extensible SDK and handles host and target parts
- differently than its counter part does for the standard SDK.
- For the extensible SDK, the task encapsulates the build system,
- which includes everything needed (host and target) for the SDK.
-
-
-
- Regardless of the type of SDK being constructed, the
- tasks perform some cleanup after which a cross-development
- environment setup script and any needed configuration files
- are created.
- The final output is the Cross-development
- toolchain installation script (.sh file),
- which includes the environment setup script.
-
-
-
-
- Stamp Files and the Rerunning of Tasks
-
-
- For each task that completes successfully, BitBake writes a
- stamp file into the
- STAMPS_DIR
- directory.
- The beginning of the stamp file's filename is determined by the
- STAMP
- variable, and the end of the name consists of the task's name
- and current
- input checksum.
-
- This naming scheme assumes that
- BB_SIGNATURE_HANDLER
- is "OEBasicHash", which is almost always the case in
- current OpenEmbedded.
-
- To determine if a task needs to be rerun, BitBake checks if a
- stamp file with a matching input checksum exists for the task.
- If such a stamp file exists, the task's output is assumed to
- exist and still be valid.
- If the file does not exist, the task is rerun.
-
- The stamp mechanism is more general than the shared
- state (sstate) cache mechanism described in the
- "Setscene Tasks and Shared State"
- section.
- BitBake avoids rerunning any task that has a valid
- stamp file, not just tasks that can be accelerated through
- the sstate cache.
- However, you should realize that stamp files only
- serve as a marker that some work has been done and that
- these files do not record task output.
- The actual task output would usually be somewhere in
- TMPDIR
- (e.g. in some recipe's
- WORKDIR.)
- What the sstate cache mechanism adds is a way to cache task
- output that can then be shared between build machines.
-
-
- Since STAMPS_DIR is usually a subdirectory
- of TMPDIR, removing
- TMPDIR will also remove
- STAMPS_DIR, which means tasks will
- properly be rerun to repopulate TMPDIR.
-
-
-
- If you want some task to always be considered "out of date",
- you can mark it with the
- nostamp
- varflag.
- If some other task depends on such a task, then that task will
- also always be considered out of date, which might not be what
- you want.
-
-
-
- For details on how to view information about a task's
- signature, see the
- "Viewing Task Variable Dependencies"
- section in the Yocto Project Development Tasks Manual.
-
-
-
-
- Setscene Tasks and Shared State
-
-
- The description of tasks so far assumes that BitBake needs to
- build everything and there are no prebuilt objects available.
- BitBake does support skipping tasks if prebuilt objects are
- available.
- These objects are usually made available in the form of a
- shared state (sstate) cache.
-
- For information on variables affecting sstate, see the
- SSTATE_DIR
- and
- SSTATE_MIRRORS
- variables.
-
-
-
-
- The idea of a setscene task (i.e
- do_taskname_setscene)
- is a version of the task where
- instead of building something, BitBake can skip to the end
- result and simply place a set of files into specific locations
- as needed.
- In some cases, it makes sense to have a setscene task variant
- (e.g. generating package files in the
- do_package_write_* task).
- In other cases, it does not make sense, (e.g. a
- do_patch
- task or
- do_unpack
- task) since the work involved would be equal to or greater than
- the underlying task.
-
-
-
- In the OpenEmbedded build system, the common tasks that have
- setscene variants are
- do_package,
- do_package_write_*,
- do_deploy,
- do_packagedata,
- and
- do_populate_sysroot.
- Notice that these are most of the tasks whose output is an
- end result.
-
-
-
- The OpenEmbedded build system has knowledge of the relationship
- between these tasks and other tasks that precede them.
- For example, if BitBake runs
- do_populate_sysroot_setscene for
- something, there is little point in running any of the
- do_fetch, do_unpack,
- do_patch,
- do_configure,
- do_compile, and
- do_install tasks.
- However, if do_package needs to be run,
- BitBake would need to run those other tasks.
-
-
-
- It becomes more complicated if everything can come from an
- sstate cache because some objects are simply not required at
- all.
- For example, you do not need a compiler or native tools, such
- as quilt, if there is nothing to compile or patch.
- If the do_package_write_* packages are
- available from sstate, BitBake does not need the
- do_package task data.
-
-
-
- To handle all these complexities, BitBake runs in two phases.
- The first is the "setscene" stage.
- During this stage, BitBake first checks the sstate cache for
- any targets it is planning to build.
- BitBake does a fast check to see if the object exists rather
- than a complete download.
- If nothing exists, the second phase, which is the setscene
- stage, completes and the main build proceeds.
-
-
-
- If objects are found in the sstate cache, the OpenEmbedded
- build system works backwards from the end targets specified
- by the user.
- For example, if an image is being built, the OpenEmbedded build
- system first looks for the packages needed for that image and
- the tools needed to construct an image.
- If those are available, the compiler is not needed.
- Thus, the compiler is not even downloaded.
- If something was found to be unavailable, or the download or
- setscene task fails, the OpenEmbedded build system then tries
- to install dependencies, such as the compiler, from the cache.
-
-
-
- The availability of objects in the sstate cache is handled by
- the function specified by the
- BB_HASHCHECK_FUNCTION
- variable and returns a list of the objects that are available.
- The function specified by the
- BB_SETSCENE_DEPVALID
- variable is the function that determines whether a given
- dependency needs to be followed, and whether for any given
- relationship the function needs to be passed.
- The function returns a True or False value.
-
-
-
-
-
- Images
-
-
- The images produced by the OpenEmbedded build system
- are compressed forms of the
- root filesystem that are ready to boot on a target device.
- You can see from the
- general Yocto Project Development Environment figure
- that BitBake output, in part, consists of images.
- This section is going to look more closely at this output:
-
-
-
-
- For a list of example images that the Yocto Project provides,
- see the
- "Images"
- chapter in the Yocto Project Reference Manual.
-
-
-
- Images are written out to the
- Build Directory
- inside the
- tmp/deploy/images/machine/
- folder as shown in the figure.
- This folder contains any files expected to be loaded on the
- target device.
- The
- DEPLOY_DIR
- variable points to the deploy directory,
- while the
- DEPLOY_DIR_IMAGE
- variable points to the appropriate directory containing images for
- the current configuration.
-
- kernel-image:
- A kernel binary file.
- The
- KERNEL_IMAGETYPE
- variable setting determines the naming scheme for the
- kernel image file.
- Depending on that variable, the file could begin with
- a variety of naming strings.
- The deploy/images/machine
- directory can contain multiple image files for the
- machine.
- root-filesystem-image:
- Root filesystems for the target device (e.g.
- *.ext3 or *.bz2
- files).
- The
- IMAGE_FSTYPES
- variable setting determines the root filesystem image
- type.
- The deploy/images/machine
- directory can contain multiple root filesystems for the
- machine.
- kernel-modules:
- Tarballs that contain all the modules built for the kernel.
- Kernel module tarballs exist for legacy purposes and
- can be suppressed by setting the
- MODULE_TARBALL_DEPLOY
- variable to "0".
- The deploy/images/machine
- directory can contain multiple kernel module tarballs
- for the machine.
- bootloaders:
- Bootloaders supporting the image, if applicable to the
- target machine.
- The deploy/images/machine
- directory can contain multiple bootloaders for the
- machine.
- symlinks:
- The deploy/images/machine
- folder contains
- a symbolic link that points to the most recently built file
- for each machine.
- These links might be useful for external scripts that
- need to obtain the latest version of each file.
-
-
-
-
-
-
- Application Development SDK
-
-
- In the
- general Yocto Project Development Environment figure,
- the output labeled "Application Development SDK" represents an
- SDK.
- The SDK generation process differs depending on whether you build
- a standard SDK
- (e.g. bitbake -c populate_sdkimagename)
- or an extensible SDK
- (e.g. bitbake -c populate_sdk_extimagename).
- This section is going to take a closer look at this output:
-
-
-
-
- The specific form of this output is a self-extracting
- SDK installer (*.sh) that, when run,
- installs the SDK, which consists of a cross-development
- toolchain, a set of libraries and headers, and an SDK
- environment setup script.
- Running this installer essentially sets up your
- cross-development environment.
- You can think of the cross-toolchain as the "host"
- part because it runs on the SDK machine.
- You can think of the libraries and headers as the "target"
- part because they are built for the target hardware.
- The environment setup script is added so that you can initialize
- the environment before using the tools.
-
-
- Notes
-
-
- The Yocto Project supports several methods by which you can
- set up this cross-development environment.
- These methods include downloading pre-built SDK installers
- or building and installing your own SDK installer.
-
-
- For background information on cross-development toolchains
- in the Yocto Project development environment, see the
- "Cross-Development Toolchain Generation"
- section in the Yocto Projects Concepts Manual.
-
-
- For information on setting up a cross-development
- environment, see the
- Yocto Project Application Development and the Extensible Software Development Kit (eSDK)
- manual.
-
-
-
-
-
- Once built, the SDK installers are written out to the
- deploy/sdk folder inside the
- Build Directory
- as shown in the figure at the beginning of this section.
- Depending on the type of SDK, several variables exist that help
- configure these files.
- The following list shows the variables associated with a standard
- SDK:
-
- DEPLOY_DIR:
- Points to the deploy
- directory.
- SDKMACHINE:
- Specifies the architecture of the machine
- on which the cross-development tools are run to
- create packages for the target hardware.
-
- SDKIMAGE_FEATURES:
- Lists the features to include in the "target" part
- of the SDK.
-
- TOOLCHAIN_HOST_TASK:
- Lists packages that make up the host
- part of the SDK (i.e. the part that runs on
- the SDKMACHINE).
- When you use
- bitbake -c populate_sdk imagename
- to create the SDK, a set of default packages
- apply.
- This variable allows you to add more packages.
-
- TOOLCHAIN_TARGET_TASK:
- Lists packages that make up the target part
- of the SDK (i.e. the part built for the
- target hardware).
-
- SDKPATH:
- Defines the default SDK installation path offered by the
- installation script.
-
-
- This next list, shows the variables associated with an extensible
- SDK:
-
- DEPLOY_DIR:
- Points to the deploy directory.
-
- SDK_EXT_TYPE:
- Controls whether or not shared state artifacts are copied
- into the extensible SDK.
- By default, all required shared state artifacts are copied
- into the SDK.
-
- SDK_INCLUDE_PKGDATA:
- Specifies whether or not packagedata will be included in
- the extensible SDK for all recipes in the "world" target.
-
- SDK_INCLUDE_TOOLCHAIN:
- Specifies whether or not the toolchain will be included
- when building the extensible SDK.
-
- SDK_LOCAL_CONF_WHITELIST:
- A list of variables allowed through from the build system
- configuration into the extensible SDK configuration.
-
- SDK_LOCAL_CONF_BLACKLIST:
- A list of variables not allowed through from the build
- system configuration into the extensible SDK configuration.
-
- SDK_INHERIT_BLACKLIST:
- A list of classes to remove from the
- INHERIT
- value globally within the extensible SDK configuration.
-
-
-
-
-
-