diff --git a/documentation/dev-manual/dev-manual-newbie.xml b/documentation/dev-manual/dev-manual-newbie.xml
index 1bb82b711d..1d9926c6cd 100644
--- a/documentation/dev-manual/dev-manual-newbie.xml
+++ b/documentation/dev-manual/dev-manual-newbie.xml
@@ -7,12 +7,12 @@
The Yocto Project Open Source Development Environment
- Using the Yocto Project in a Team Environment
+ Setting Up a Team Yocto Project Development Environment
It might not be immediately clear how you can use the Yocto
- Project in a team environment, or scale it for a large team of
- developers.
+ Project in a team development environment, or scale it for a large
+ team of developers.
One of the strengths of the Yocto Project is that it is extremely
flexible.
Thus, you can adapt it to many different use cases and scenarios.
@@ -21,337 +21,350 @@
- To help with these types of situations, this section presents
- some of the project's most successful experiences,
- practices, solutions, and available technologies that work well.
- Keep in mind, the information here is a starting point.
+ To help you understand how to set up this type of environment,
+ this section presents a procedure that gives you the information
+ to learn how to get the results you want.
+ The procedure is high-level and presents some of the project's most
+ successful experiences, practices, solutions, and available
+ technologies that work well.
+ Keep in mind, the procedure here is a starting point.
You can build off it and customize it to fit any
particular working environment and set of practices.
-
+
+
+ Determine Who is Going to be Developing:
+ You need to understand who is going to be doing anything
+ related to the Yocto Project and what their roles would be.
+ Making this determination is essential to completing the
+ steps two and three, which are to get your equipment together
+ and set up your development environment's hardware topology.
+
-
- System Configurations
-
-
- Systems across a large team should meet the needs of
- two types of developers: those working on the contents of the
- operating system image itself and those developing applications.
- Regardless of the type of developer, their workstations must
- be both reasonably powerful and run Linux.
-
-
-
- Application Development
-
-
- For developers who mainly do application level work
- on top of an existing software stack,
- the following list shows practices that work best.
- For information on using a Software Development Kit (SDK), see
- the
- Yocto Project Software Development Kit (SDK) Developer's Guide:
+ The following roles exist:
- Use a pre-built toolchain that
+
+ Application Development:
+ These types of developers do application level work
+ on top of an existing software stack.
+
+
+ Core System Development:
+ These types of developers work on the contents of the
+ operating system image itself.
+
+
+ Build Engineer:
+ This type of developer manages Autobuilders and
+ releases.
+ Not all environments need a Build Engineer.
+
+
+ Test Engineer:
+ This type of developer creates and manages automated
+ tests needed to ensure all application and core
+ system development meets desired quality standards.
+
+
+
+
+ Gather the Hardware:
+ Based on the size and make-up of the team, get the hardware
+ together.
+ Any development, build, or test engineer should be using
+ a system that is running a supported Linux distribution.
+ Systems, in general, should be high performance (e.g. dual,
+ six-core Xeons with 24 Gbytes of RAM and plenty of disk space).
+ You can help ensure efficiency by having any machines used
+ for testing or that run Autobuilders be as high performance
+ as possible.
+
+
+ Understand the Hardware Topology of the Environment:
+ Now that you know how many developers and support engineers
+ are required, you can understand the topology of the
+ hardware environment.
+ The following figure shows a moderately sized Yocto Project
+ development environment.
+
+
+ Need figure.
+
+
+
+ Use Git as Your Source Control Manager (SCM):
+ Keeping your
+ Metadata
+ and any software you are developing under the
+ control of an SCM system that is compatible
+ with the OpenEmbedded build system is advisable.
+ Of the SCMs BitBake supports, the
+ Yocto Project team strongly recommends using
+ Git.
+ Git is a distributed system that is easy to backup,
+ allows you to work remotely, and then connects back to the
+ infrastructure.
+
+ For information about BitBake, see the
+ BitBake User Manual.
+
+
+ It is relatively easy to set up Git services and create
+ infrastructure like
+ http://git.yoctoproject.org,
+ which is based on server software called
+ gitolite with cgit
+ being used to generate the web interface that lets you view the
+ repositories.
+ The gitolite software identifies users
+ using SSH keys and allows branch-based
+ access controls to repositories that you can control as little
+ or as much as necessary.
+
+
+ The setup of these services is beyond the scope of this
+ manual.
+ However, sites such as these exist that describe how to
+ perform setup:
+
+
+ Git documentation:
+ Describes how to install gitolite
+ on the server.
+
+
+ The gitolite master index:
+ All topics for gitolite.
+
+
+ Interfaces, frontends, and tools:
+ Documentation on how to create interfaces and frontends
+ for Git.
+
+
+
+
+
+ Set up the Application Development Machines:
+ As mentioned earlier, application developers are creating
+ applications on top of existing software stacks.
+ Following are some best practices for setting up machines
+ that do application development:
+
+
+ Use a pre-built toolchain that
contains the software stack itself.
Then, develop the application code on top of the
stack.
This method works well for small numbers of relatively
- isolated applications.
- When possible, use the Yocto Project
- plug-in for the Eclipse IDE
+ isolated applications.
+
+
+ When possible, use the Yocto Project
+ plug-in for the
+ Eclipse IDE
and SDK development practices.
For more information, see the
"Yocto Project Software Development Kit (SDK) Developer's Guide".
- Keep your cross-development toolchains
- updated.
+
+ Keep your cross-development toolchains updated.
You can do this through provisioning either as new
toolchain downloads or as updates through a package
update mechanism using opkg
to provide updates to an existing toolchain.
The exact mechanics of how and when to do this are a
- question for local policy.
- Use multiple toolchains installed locally
+ question for local policy.
+
+
+ Use multiple toolchains installed locally
into different locations to allow development across
- versions.
+ versions.
+
-
-
-
-
- Core System Development
-
-
- For core system development, it is often best to have the
- build system itself available on the developer workstations
- so developers can run their own builds and directly
- rebuild the software stack.
- You should keep the core system unchanged as much as
- possible and do your work in layers on top of the core system.
- Doing so gives you a greater level of portability when
- upgrading to new versions of the core system or Board
- Support Packages (BSPs).
- You can share layers amongst the developers of a particular
- project and contain the policy configuration that defines
- the project.
-
-
-
- Aside from the previous best practices, there exists a number
- of tips and tricks that can help speed up core development
- projects:
+
+
+ Set up the Core Development Machines:
+ As mentioned earlier, these types of developers work on the
+ contents of the operating system itself.
+ Following are some best practices for setting up machines
+ used for developing images:
- Use a
- Shared State Cache
- (sstate) among groups of developers who are on a
- fast network.
- The best way to share sstate is through a
- Network File System (NFS) share.
- The first user to build a given component for the
- first time contributes that object to the sstate,
- while subsequent builds from other developers then
- reuse the object rather than rebuild it themselves.
-
- Although it is possible to use other protocols for the
- sstate such as HTTP and FTP, you should avoid these.
- Using HTTP limits the sstate to read-only and
- FTP provides poor performance.
+
+ Have the Yocto Project build system itself available on
+ the developer workstations so developers can run their own
+ builds and directly rebuild the software stack.
- Have autobuilders contribute to the sstate
- pool similarly to how the developer workstations
- contribute.
- For information, see the
- "Autobuilders"
- section.
- Build stand-alone tarballs that contain
- "missing" system requirements if for some reason
- developer workstations do not meet minimum system
- requirements such as latest Python versions,
- chrpath, or other tools.
- You can install and relocate the tarball exactly as you
- would the usual cross-development toolchain so that
- all developers can meet minimum version requirements
- on most distributions.
- Use a small number of shared,
- high performance systems for testing purposes
- (e.g. dual, six-core Xeons with 24 Gbytes of RAM
- and plenty of disk space).
- Developers can use these systems for wider, more
- extensive testing while they continue to develop
- locally using their primary development system.
+
+ Keep the core system unchanged as much as
+ possible and do your work in layers on top of the
+ core system.
+ Doing so gives you a greater level of portability when
+ upgrading to new versions of the core system or Board
+ Support Packages (BSPs).
- Enable the PR Service when package feeds
- need to be incremental with continually increasing
- PR
- values.
- Typically, this situation occurs when you use or
- publish package feeds and use a shared state.
- You should enable the PR Service for all users who
- use the shared state pool.
- For more information on the PR Service, see the
- "Working With a PR Service".
+
+ Share layers amongst the developers of a
+ particular project and contain the policy configuration
+ that defines the project.
-
-
-
+
+
+ Set up an Autobuilder:
+ Autobuilders are often the core of the development
+ environment.
+ It is here that changes from individual developers are brought
+ together and centrally tested and subsequent decisions about
+ releases can be made.
+ Autobuilders also allow for "continuous integration" style
+ testing of software components and regression identification
+ and tracking.
-
- Source Control Management (SCM)
+ See "Yocto Project Autobuilder"
+ for more information and links to buildbot.
+ The Yocto Project team has found this implementation
+ works well in this role.
+ A public example of this is the Yocto Project
+ Autobuilders, which we use to test the overall health of the
+ project.
-
- Keeping your
- Metadata
- and any software you are developing under the
- control of an SCM system that is compatible
- with the OpenEmbedded build system is advisable.
- Of the SCMs BitBake supports, the
- Yocto Project team strongly recommends using
- Git.
- Git is a distributed system that is easy to backup,
- allows you to work remotely, and then connects back to the
- infrastructure.
-
- For information about BitBake, see the
- BitBake User Manual.
-
-
+ The features of this system are:
+
+
+ Highlights when commits break the build.
+
+
+ Populates an sstate cache from which
+ developers can pull rather than requiring local
+ builds.
+
+
+ Allows commit hook triggers,
+ which trigger builds when commits are made.
+
+
+ Allows triggering of automated image booting
+ and testing under the QuickEMUlator (QEMU).
+
+
+ Supports incremental build testing and
+ from-scratch builds.
+
+
+ Shares output that allows developer
+ testing and historical regression investigation.
+
+
+ Creates output that can be used for releases.
+
+
+ Allows scheduling of builds so that resources
+ can be used efficiently.
+
+
+
+
+ Set up Test Machines:
+ Use a small number of shared, high performance systems
+ for testing purposes.
+ Developers can use these systems for wider, more
+ extensive testing while they continue to develop
+ locally using their primary development system.
+
+
+ Document Policies and Change Flow:
+ The Yocto Project itself uses a hierarchical structure and a
+ pull model.
+ Scripts exist to create and send pull requests
+ (i.e. create-pull-request and
+ send-pull-request).
+ This model is in line with other open source projects where
+ maintainers are responsible for specific areas of the project
+ and a single maintainer handles the final "top-of-tree" merges.
+
+ You can also use a more collective push model.
+ The gitolite software supports both the
+ push and pull models quite easily.
+
-
- It is relatively easy to set up Git services and create
- infrastructure like
- http://git.yoctoproject.org,
- which is based on server software called
- gitolite with cgit
- being used to generate the web interface that lets you view the
- repositories.
- The gitolite software identifies users
- using SSH keys and allows branch-based
- access controls to repositories that you can control as little
- or as much as necessary.
-
+ As with any development environment, it is important
+ to document the policy used as well as any main project
+ guidelines so they are understood by everyone.
+ It is also a good idea to have well structured
+ commit messages, which are usually a part of a project's
+ guidelines.
+ Good commit messages are essential when looking back in time and
+ trying to understand why changes were made.
-
- The setup of these services is beyond the scope of this manual.
- However, sites such as these exist that describe how to perform
- setup:
-
- Git documentation:
- Describes how to install gitolite
- on the server.
- The gitolite master index:
- All topics for gitolite.
-
- Interfaces, frontends, and tools:
- Documentation on how to create interfaces and frontends
- for Git.
-
-
-
-
-
- Autobuilders
-
-
- Autobuilders are often the core of a development project.
- It is here that changes from individual developers are brought
- together and centrally tested and subsequent decisions about
- releases can be made.
- Autobuilders also allow for "continuous integration" style
- testing of software components and regression identification
- and tracking.
-
-
-
- See "Yocto Project Autobuilder"
- for more information and links to buildbot.
- The Yocto Project team has found this implementation
- works well in this role.
- A public example of this is the Yocto Project
- Autobuilders, which we use to test the overall health of the
- project.
-
-
-
- The features of this system are:
-
- Highlights when commits break the build.
-
- Populates an sstate cache from which
- developers can pull rather than requiring local
- builds.
- Allows commit hook triggers,
- which trigger builds when commits are made.
-
- Allows triggering of automated image booting
- and testing under the QuickEMUlator (QEMU).
-
- Supports incremental build testing and
- from-scratch builds.
- Shares output that allows developer
- testing and historical regression investigation.
-
- Creates output that can be used for releases.
-
- Allows scheduling of builds so that resources
- can be used efficiently.
-
-
-
-
-
- Policies and Change Flow
-
-
- The Yocto Project itself uses a hierarchical structure and a
- pull model.
- Scripts exist to create and send pull requests
- (i.e. create-pull-request and
- send-pull-request).
- This model is in line with other open source projects where
- maintainers are responsible for specific areas of the project
- and a single maintainer handles the final "top-of-tree" merges.
-
-
-
- You can also use a more collective push model.
- The gitolite software supports both the
- push and pull models quite easily.
-
-
-
- As with any development environment, it is important
- to document the policy used as well as any main project
- guidelines so they are understood by everyone.
- It is also a good idea to have well structured
- commit messages, which are usually a part of a project's
- guidelines.
- Good commit messages are essential when looking back in time and
- trying to understand why changes were made.
-
-
-
- If you discover that changes are needed to the core layer of the
- project, it is worth sharing those with the community as soon
- as possible.
- Chances are if you have discovered the need for changes, someone
- else in the community needs them also.
-
-
-
-
- Summary
-
-
- This section summarizes the key recommendations described in the
- previous sections:
-
- Use Git
- as the source control system.
- Maintain your Metadata in layers that make sense
- for your situation.
- See the "Understanding
- and Creating Layers" section for more information on
- layers.
-
- Separate the project's Metadata and code by using
- separate Git repositories.
- See the
- "Yocto Project Source Repositories"
- section for information on these repositories.
- See the
- "Getting Set Up"
- section for information on how to set up local Git
- repositories for related upstream Yocto Project
- Git repositories.
-
- Set up the directory for the shared state cache
- (SSTATE_DIR)
- where it makes sense.
- For example, set up the sstate cache on a system used
- by developers in the same organization and share the
- same source directories on their machines.
-
- Set up an Autobuilder and have it populate the
- sstate cache and source directories.
- The Yocto Project community encourages you
- to send patches to the project to fix bugs or add features.
- If you do submit patches, follow the project commit
- guidelines for writing good commit messages.
- See the "How to Submit a Change"
- section.
- Send changes to the core sooner than later
- as others are likely to run into the same issues.
- For some guidance on mailing lists to use, see the list in the
- "How to Submit a Change"
- section.
- For a description of the available mailing lists, see the
- "Mailing Lists"
- section in the Yocto Project Reference Manual.
-
-
-
-
+ If you discover that changes are needed to the core
+ layer of the project, it is worth sharing those with the
+ community as soon as possible.
+ Chances are if you have discovered the need for changes,
+ someone else in the community needs them also.
+
+
+ Development Environment Summary:
+ Aside from the previous steps, some best practices exist
+ within the Yocto Project development environment.
+ Consider the following:
+
+
+ Use Git
+ as the source control system.
+
+
+ Maintain your Metadata in layers that make sense
+ for your situation.
+ See the "Understanding
+ and Creating Layers" section for more information on
+ layers.
+
+
+ Separate the project's Metadata and code by using
+ separate Git repositories.
+ See the
+ "Yocto Project Source Repositories"
+ section for information on these repositories.
+ See the
+ "Getting Set Up"
+ section for information on how to set up local Git
+ repositories for related upstream Yocto Project
+ Git repositories.
+
+
+ Set up the directory for the shared state cache
+ (SSTATE_DIR)
+ where it makes sense.
+ For example, set up the sstate cache on a system used
+ by developers in the same organization and share the
+ same source directories on their machines.
+
+
+ Set up an Autobuilder and have it populate the
+ sstate cache and source directories.
+
+
+ The Yocto Project community encourages you
+ to send patches to the project to fix bugs or add features.
+ If you do submit patches, follow the project commit
+ guidelines for writing good commit messages.
+ See the "How to Submit a Change"
+ section.
+
+
+ Send changes to the core sooner than later
+ as others are likely to run into the same issues.
+ For some guidance on mailing lists to use, see the list in the
+ "How to Submit a Change"
+ section.
+ For a description of the available mailing lists, see the
+ "Mailing Lists"
+ section in the Yocto Project Reference Manual.
+
+
+
+
+