Getting Started with the Yocto Project - New Developer Screencast TutorialMar 27, 2020
hello and welcome to the
screencastis intended to provide a smooth introduction to the octo
developerperspective you don't need to have any previous experience with the
yoctoproject to follow along with the screencast but it will be helpful to have some experience previous working with Linux systems and have a basic understanding of how software is configured and built under Linux. My name is Scott Carmen and for the next half hour we're going to walk you through a couple of builds of a complete Linux operating system and show you some of what goes on behind the scenes, while there's no way we can cover everything you need.
Know With this screencast, the goal is to familiarize you enough with the build system so that you can get the most out of our documentation and start your exploration of the octo project with a solid foundation of knowledge, so we'll start by spending a couple of minutes looking at goals and benefits. ts from the octo project and then we'll give a high-level overview of the Yoko build system. We'll take a closer look at the recipes that are an important foundational part of the build system, and then build and run a Linux image under emulation. After that we'll discuss Yoko's layering system and how board support packages are used as layers and finally we'll see how easy it is to rebuild a Linux image for a different board and I'll point you to some useful resources from the project. for more information. research, so the purpose of the Yocto project is to help you create a custom Linux operating system for almost any type of embedded computing device. the industry and the open source community are coming together to collaborate on a set of tools and best practices for developing custom Linux images.
More Interesting Facts About,
getting started with the yocto project new developer screencast tutorial...
The idea is not that this is a Linux distribution that sets software policies. Instead, Yocto focuses on allowing a high degree of flexibility and customization, which is critical when you are developing Linux-based embedded products and need to have a great amount of control over the software that runs. on your device, whether you need to focus on making your device boot up as fast as possible or enabling certain optimizations in your software, Yocto allows you to do this while adhering to practices that allow you to reuse your software stack across a wide range of applications. hardware platforms that save your organization time and money.
The Yocto project is hosted by the Linux Foundation, making it an independent project of any particular company or vendor, but as an open source project, many companies and individuals are shaping the future of the project through their participation and code contributions. As you can see there are quite a few organizations supporting the Octo project ranging from Silicon Vendo RS to OS vs2 Consulting Group's and if you would like to become an official participating organization we encourage you to contact the Linux Foundation for more details. Many benefits can be gained by adopting the Yocto project, one of the most important. is to be able to reuse your development efforts regardless of the hardware device you're working with, no longer will you have to reorganize your development processes every time you start using a new integrated board with the often octo board support package, all you need to do to rebuild your software for a new platform is to change one line in a configuration file, we'll demonstrate this later in the screencast, opto allows you to start with a validated set of applications and software libraries, and since we are based on open embedded applications, there are thousands of additional recipes that you can easily transfer to Yocto octo offers integration with a great set of development tools, such as an eclipse plugin for elopment application development, as well as tools for debugging and generate profiles, analyze energy consumption and more finally, the fact that Yocto is a code project. igo open allows you to start working with it right away, you can use it to develop prototypes or fully finished products if you decide you want commercial support any of the OSVs listed on the previous slide can provide without forcing you to adopt a completely different build system, so So now that you've learned a few things about the Yocto project, let's turn our attention to the build system.
The name of the build system software Yocto uses is called Pocky, and Pocky consists of two main components: bitbake, which runs and manages all the steps. build, and a set of metadata that describes what gets built and how to build it; metadata is divided into three main types we have configuration files that define global defaults for things such as standard filesystem paths where build output will be placed compiler flags machine architectures etc. there are bitbake classes that they end up in the dot DB class filename extension and these are the ones that really do most of the heavy lifting in the build system, for example we have bitbake classes that define how a linux kernel is built or how a piece of software is created based on automatic tools, we have some on how to generate an RPM package, how to create a root file system image, etc.
Finally, there are recipes and recipes that define an individual piece of software to be built, so we have a recipe for the guitar command, a recipe for the gtk library, and so on. The recipes are also used to define which packages will be included in the final filesystem image, so one of the key things about Yocto is that it provides a very flexible set of tools for creating highly customized Linux images that are adapt to the embedded device you're using give you a lot of control over what software and features you want to include in your image, but at the same time offer a sensible set of defaults to start with when you start developing your embedded product, the building blocks of these Images are recipes and we'll be looking at some recipes very soon, but I want to make it clear up front that a recipe is not a complete script or build file that creates software as it is in other embedded systems.
Instead, the recipes include bits of metadata about the software, such as where you download the upstream sources for build or runtime dependencies and what features you want to enable. in your application you can also split the final output of a recipe into multiple binary packages, for example you probably want to separate the documentation files into another package so you can choose whether or not to install that package in your image in most bundled packages . applications you would probably leave out to save disk space this workflow diagram provides a high level overview of what happens during a pocky build the first step is to download the sources of the software you are going to build, we mean this step how to get the fonts.
Of course, we support standard methods, such as downloading a tarball from an HTTP or FTP site, and we also allow you to check Sources from several different source code repository systems, such as get or subversion. Once the sources have been downloaded, they will be extracted to a local workspace where the necessary patches will be applied. At that point, the common steps for configuring and building the software will be followed. will run, the software will be installed in a temporary staging area, at which time the binary package format you selected will be used to summarize the software into one or more output packages. run, such as checking that newly built software is not linked to software libraries located outside the build system, it's actually not true to say that a All of these tests run in one step.
There are many different checks that are run throughout the entire build process. Once all the necessary binary packages have been created, a binary package feed is generated which is then used to create the final root filesystem image. The fact that a packaged font is used in this process becomes very useful if you are developing an embedded product with network capabilities that may need to be upgraded in the field. What this allows you to do is copy that package source to a network server, and then design your online update system to pull your updates from that package. in the right order, you will finally notice that the file system image and the app development SDK are built in parallel, it is true that there is no single S DK that you can use to develop for any Yocto based embedded device because Yocto doesn't it is a linux distribution and instead allows you to decide what software in the libraries is included in your linux image so when you build your custom linux image you can also build a nest that is customized for the software you have chosen for your Linux image, so in a nutshell, this is what happens during a Pocky build, since we're building everything from source.
A side effect of this process is that the first build to run can take a significant amount of time. The good news is that we have developed several caching mechanisms to speed up successive builds. Fonts you have downloaded, for example, are kept in a separate area and will not be re-downloaded each time. keep a cache or the binary build output that is generated so that you don't recompile the same software over and over again when running successive builds only if you make changes to the recipes those parts of th The system rebuilds now, there was a lot to cover on the last slide, but in contrast to the amount of work that Pocky goes through to build an image, we've tried to make it extremely easy to start building your first Linux image with the octave involving literally just a handful of commands and a little of patience during your first build while it completes, follow our quick start guide in the documentation section of the Yocto project organization website to install the necessary build system prerequisites on your Linux system, then you should be able to follow along with the live examples I'll show you for the rest of the screencast, so now that you have some basic information a about the Yocto project and an abstract understanding of the build systems workflow, let's take a look at some more specific things For this exercise, we're going to take a look at the source tree you get when you download Pocky and I'll show you where They store different types of metadata.
The Linux system I'm using for my build host is based on fedora 16. I generally recommend using a recent version of Fedora Ubuntu or opening up SUSE for your build hosts, since those three Linux distributions are what we make most of our builds on. Test now in theory. You can run builds from a virtual machine, but if you can avoid it, I don't recommend using one for performance reasons. If you end up using a virtual machine for your builds, keep in mind that builds will take much longer, typically three times as long as if you were running Linux natively, so I'm going to launch the graphical file manager here to explore the pocky fonts.
You can see that the pocky 6.0 tarball has already been downloaded and extracted into this directory. I will find several subdirectories. The bitbake directory is where the bitbake command and fonts are stored. I'll be back in the build directory in just a second. The documentation directory is where we store the sources of our document. entation and you will find them easier to read by simply going to the documentation section of our website at Yocto project org and you will notice that we have a number of meta directories, these are the main build system metadata files where they are located and the reason why The reason we have multiple directories is that each of these directories represents a layer, which is a concept we'll dive into later in the screencast within the metadirectory.
You'll notice we have recipe subdirectories where I store the recipe files this is where you'll find our recipe files for the busybox environment for example in the classes directory you'll find all our bitbake classes you can see we have classes for auto tools for the sea build system and the one here for sea pan would be used to build pearl modules and so on in the conf directory you will find different types of configuration files for the build system one of the top level configuration files is bitbake kam pf which as you can see here defines the default filesystem paths and sets architecture specific build variables and a bit more in the machine subdirectory we have machine configuration files that define the features of the hardware of our QEMU targets. file contentadditional ones going back to the top level directory the scripts directory contains a number of supporting scripts that are used by the build system, for example we have some scripts here to help simplify running the QEMU emulator now the build directory is something that gets dynamically generated during builds and this is where all the generated output is stored.
The download directory is where all upstream sources are downloaded into the state cache where we store compiled output that can be reused during successive builds. When the builds are done, you'll find the working directories under here, as well as the sister directories, etc. in the conf directory is very important as it stores the build files of the local kampf and BB layers which set critical variables you'll want to see in the local build, for example it allows you to configure things like how many threads you want to bitbakeuse the target machine which you are building for the packaging format you want to use, and therefore the baby layer build file specifies which metadata layer we want to use, since a board support package is distributed as a Yocto layer.
I doubt you'd want to enable that within your Bebe Layers dot-com file, so it's just a quick tour of the Pocky directory tree to give you a bit of the lay of the land. Why don't we build our first image? Open a terminal and a CD. in the pocky directory now we are going to get a script called oh we and weave build env this script just sets some basic environment variables and automatically changes your working directory to the build directory before starting the build for the first time which you always want to open your local build file and make sure to specify the desired target machine as well as set performance related variables to get the most out of your build hardware; in this case i set the target machine to QEMU x86 and paragraph llelism options 2 2 and 4 which is appropriate for my dual core laptop for a quad core system i would set them to 4 and 8 now that I we've done we want to make sure we're in the build subdirectory and then run bitbake core image minimal this starts the build and now it should just be a wait set to give you an idea of what to expect at this point if you have a processor modern i7 quad core and 8 gigabytes of ram this build should take about an hour and a half if you are running it on native hardware, keep in mind that you are building a complete linux operating system from scratch and if you are running it in a virtual machine, it will take even longer than that with our build running in the background, let's take a closer look at the recipe files.
This slide shows a sample recipe for the kill tool utility. As you can see, we specify some metadata about the purpose of the software that this recipe creates in the summary description and home page fields, we also specify the l The license or licenses under which the software is distributed most importantly, we use the source viewer variable I to tell bitbake where to download upstream sources from. In this case, we are downloading them from SourceForge. The pv variable in the RI source view represents the version of the package. y corresponds to the portion of the recipe file that is named after the underscore, in this case it expands to 2.6 36 doing this makes it easier to update this recipe when a new version of the tool is released eath allows us to simply change the name of the recipe file and minify a set of changes we need to make to the recipe content you will finally notice the inherited autotools line this line tells bitbake to use the autotools BB class logic to build the software making use of With these bitbake class files you can keep a lot of complexity out of the individual recipe files.
This is a design goal of the Pocky build system, as we have so many individual recipe files that we try to keep them as simple as possible. possible and take care of the build logic within the bitbake class files, instead the process of building a piece of software, no matter what kind of build system you use, is reduced to a set of standard steps that we run with bitbake. These steps are listed here and include retrieving the sources unpacking the sources into the working directory patching the sources configuring the sources compiling them installing the compiled output in a staging area and packaging the output into one or more binary package formats what happens behind The scene when you inherit something like the Auto Tools BB class is that you are inheriting a version of these build steps that work to successfully compile the software using the desired build system; in addition to this, you can override these build steps by defining them inside your recipe file; we'll look at some examples of this shortly, let's take a closer look at some specific recipes, the first one we'll look at is mini-com, which is found in the mehta s extended mini-com recipe and will recognize the same types of metadata we saw in ether recipe example swipe a new field to note that it is the depends variable this variable is used to declare any compile time dependencies so in this case we are saying that Minicon depends on the N library curses, so bitbake will make sure to create n curses first.
We also have a license file checksum field that we use to track possible changes to license information from upstream sources. This works by specifying one or more files. including license information and taking the md5 sum of that file in case the file changes in a future version of the software bitbake will warn us about a checksum mismatch and we can stop to inspect what the changes actually are to check if this license has changed we can do this with entire files like with the copy file here or just a section of the file we are doing here with a mini university file just going through the first twelve lines of the file which presumably has license information for the source URI, we can see that we are downloading a Debian tar ball or a website, but we also included an extra line here that references a patch file, you might be wondering where these patches are kept so if we go back to our file explorer, we are in the extended mini-com directory of recipes and, in addition to the file of recipes itself, you will notice that there is another directory with the name of the recipes file and if we look here, you will find the patch file which can store patches in this directory as well as other files that your recipe might need like startup scripts personalized recipes that go back to the recipe.
We have some additional fields associated with the source URI, these are md5 and sha- 1 or star ball, we do this to make sure that upstream sources haven't been tampered with or corrupted due to a bad download, as those hashes shouldn't change for this particular version of mini-com, you'll see here that we're inheriting not in l and auto tools, but also the get text bate get text class is used to manage localization support during the software build process and eventually we see that a function do install has been defined and so this means that instead of using the function do install defined in autotools class bitbake we are executing the contents of this function, instead this is an important feature that we provide because we don't all software follows the exact same build pattern, even if you're using automated tools, so this gives us the control to override the behavior of any r build step when needed another recipe we can see is the redlining recipe which is under the basic redlining of recipes of course this is for the new redlining library so we start with boilerplate metadata typical at the top and you'll notice the red line also depends at the end of the curses library to compile we also have a couple of patches we do to read the build system lines you'll notice we've defined a couple of functions but in this case the do configure function has an underscore at the end and this is a hook function that allows us to execute the contents of this function before the new configure step is executed instead of overriding it entirely.
Also, we have an underscore link function to install, so the content of this function will be executed after the install step. Finally, you'll notice that the BB class extension variable allows us to create virtual versions of this recipe so that they exist for the native cases and native SDKs. Native recipes generate software that is not installed on the target machine, but is used to boot the build system so that the target software can be built. This is typically done for utilities that need to be run on the build system, such as quilt native, which we use to apply patches for our final build. recipe example let's take a look at the urine splash recipe P splash is a system for displaying a graphical logo on the splash screen when a system is booting up it stays in the centerpiece of flash recipes and you will notice that this recipe, instead of a version number at the end of the file name, it has a get word instead, as you might guess, this means that the recipe pulls its sources from a git repository, the git repository is specified in the URI of source as you can see here and by default this will pull the latest version from the git repository whenever P splash is created to specify a particular commit to build from you would set the commit hash in the source Rev variable as is done here, the variable S here refers to where the fonts are stored within the working directory; you usually have to set it manually when your source code doesn't extract like a tarball normally does to a directory whose name matches your recipe name and version number we're inheriting Auto Tools again by far it's the system most common build configuration for linux programs but we also inherit the package configuration and update our cd package configuration is an interface to locate shared libraries that the build system if p splash uses an update our cd means the software it's a startup daemon and therefore needs some logic enabled to support files underscore variable pn allows you to control which files go in which binary packages pn corresponds to the recipe name without the version number, so here we are manually adding the slash mount dot P splash file to the package called P splash this recipe also uses the a hooks Overrides for the setup and installation steps and finally the annette script variables here need to be set when you inherit the update from our CD class, they specify the name of the startup script file as well as options to set the runlevels for when the startup script runs.
One thing I would like to mention is if you ever come across a variable within a recipe that you don't recognize the first place to look for it is in the Yocto project reference manual which you can find on our website at Yocto project org age with the QEMU emulator , the image files are generated under build temporary deployment images and as you can see here there is a kernel and root filesystem in ext3 format as well as a root filesystem tarball pakhi includes a script called run QEMU which automatically passes the appropriate options to the qemu command, all you have to do is run it from the build directory and pass it the name of the QEMU architecture you've built, so we'll just call run q mu q mu x86 and as you can see our Linux system is booting for the first time when you get to the login prompt, the root account by default doesn't use a password, so just Just hit enter and you'll be logged in, as you can see the root filesystem for this image is about twelve megabytes, you can actually create a much smaller linux image than that if you make use of our small meta layer which is currently in development and is expected to be included with version 1.2 of Yoko to log out of q mu simply issue the shutdown command let's take a few minutes to understand layers in the Yocto project layers are a powerful layout mechanism that allows you to switch different stacks of software in a modular way a layer includes a set of bitbake metadata so you can include configuration files that fit the bake classes and recipes in the octo project a board support package for a hardware device inThis particular is distributed as a layer you enable in your BB layer draft file this allows you to develop the software platform for your embedded application and then deploy this application on any hardware platform you can get a Yocto board support package because it also You can add support for certain classes of software, such as a particular library environment or software toolkit by
gettingor creating a Yocto layer for your software.
This diagram shows how layers can be used to develop an embedded product with all the features. functions, each layer has a priority associated with it, so you can control which levels have higher priority over others on the basis of this example, we have the open built-in core and Yocto metadata layers that are included in our ball pocket which download with our releases, on top of that I would add a BSP layer for hardware support and possibly include a layer consisting of software for a user interface or application library; commercial Linux vendors also have the opportunity to customize and add value to Yocto-based embedded environments by maintaining their own layers. create a layer that enables these features that you can develop with this layer enabled and then disable the layer when it's time to move your app to production as I mentioned advanced support packages are layers whose focus is to enable hardware features and specify the drivers used with a particular hardware device, a typical bsp layer will include a linux kernel configuration, specify w which graphics drivers to use if any, and include any other recipes, including userspace ones needed to enable features custom hardware.
Let's look at an example of using a layer by getting and enabling a board support package. Let's say we want to build a kernel. minimal image to get an Intel Cedar Trail embedded board
started, we would go to the board support packages page on the Yocto project or website, it can be found under the download tab, make sure the BSP you have selected be compatible with the pocky release you're using to build it since we're using pocky 6.0 this should work for us the board support package is distributed as a tarball we download inside the starball you'll find documentation files that they tell you things eg how to install the image in the onboard release notes and who is the maintainer zuv the bsp is the base directory of the layer is the one with the name of subdirectory brown with a file wit of layer point Tell him that in a minute we're going to add the cedar trail meta directory to our BB layer build file, as you can see this layer includes a configuration file.
Since it is called cedar trail kampf will set the machine variable to cedar trail within our local build file, this BSP also includes custom recipes that set the proper configuration for the kernel and X server. The binary directory you see here is optional for BSP and can include pre-built reference images or kernels. Now let's enable this layer in our Pocky Build System First we edit our database layers build file and include the metacedron trace directory. Note that this must be an absolute path now our build system has access to all the configuration files and additional recipes that the BSP offers.
Next, we set the Machine variable. to Cedar Trail in our local communication archive is now as simple as baking our desired image. The resulting image files will be generated under temporary deployment images and can be copied to the Cedar Trail dashboard as per the documentation included with the BSP. you can see the layering system that the Yocto project uses allows you to see hardware support as a simple matter of plugging in the desired layer or layers that you want to us As the Yocto project community continues to grow, the layering ecosystem allows Embedded application
developers dramatically improve time to market and reuse code more easily than ever before.
Here are some final notes on the Yocto project that you should keep in mind. origin project with lots of integrated community and industry participation and you are welcome to jump in and contribute anytime our website at Yocto project org will help you get started with documentation and other resources we have mailing lists we encourage you to join Join us and our source code repository is getting bug reports from the Yocto project organization and feature requests are normally submitted via our Bugzilla site, so I hope this screencast has given you a solid introduction to the Yocto project. and now that you have built and booted your first Yocto image based linux you can feel more comfortable diving into our documentation and code base thanks for looking
If you have any copyright issue, please Contact