The Linux Standard Base

The GNU/Linux operating system (hereafter Linux ) is an aggregation of the Linux kernel, packages from the GNU project, and many other open source software packages. The aggregations are usually called distributions . According to the Linux Weekly News distribution list (http://lwn.net/Distributions) there are 375 different Linux distributions. Each serves particular purposes and targets, and the diversity in and of itself does not indicate a weakness. But it is also clear that there is ample potential for incompatibility between distributions.

Free Software and other open source software carry a certain promise of compatibility: with the source code available users have the freedom to change the code or compile an older version if a new version has introduced incompatibilities. But there is a limit to how far many users are willing to go in such pursuits; for them the value of free software is that it gives them control of their own computing environments, but they almost certainly don't want to spend their time trying to build/keep compatibility between different distributions. And not all software can end up being open source.

The extent to which compatibility has actually been a problem varies depending on the viewpoint of the observer. On the one hand, the core of Linux is much the same - the distribution companies and projects have tended to choose largely the same software as the foundation for their distributions, and this brings a level of compatibility. But the open source projects that are aggregated into Linux evolve over time, and often introduce intended or unintended incompatibilities as part of that evolution, the aggregators make different decisions on when to take up these new versions, and additionally make changes to suit their purposes. Distributions have generally been pretty good about carrying old and new versions together when that is necessary for compatibility, although the promises are only in the forward direction, and are of course limited to that distribution.

Other compatibility concerns have come from differing packaging schemes and different file system layout policies.

Despite the above claim of reasonable compatibility, it is almost certain that every veteran of using Linux has a few war stories where a program fails after an OS upgrade, or a program pulled off the net won't work, etc. It is common to see precompiled software come with a precise description of the distribution and version for which it was built, with few expectations that it will work on any other combination. Commercial software may even come with statements that the product is unsupported except on the very limited list printed on the box. Clearly, this is not a desirable situation if Linux is to become more widely used among those who see their computer more as a tool than as a development platform.

A binary standard helps mitigate these issues.

For application developers, having a standard application binary interface for Linux means:

These points apply to open source developers as well as commercial, although the benefits may be realized a bit differently. For example, the process of preparing a package to build to a binary standard will help identify potential code portability problems and unexpected dependencies, and the code becomes more widely usable with less support burden on the project team, which can then spend more time innovating.

For users, a binary standard enables availability of a standardized binary application platform for Linux from multiple suppliers. Users have freedom of choice rather than being locked in to a single supplier. The presence of such a standard also offers the prospect of a supply of ready-to-install applications that can run across a range of systems from multiple suppliers.

To attain the benefits of a binary standard and forestall the possibility of any real divergence between Linux versions, the LSB project was founded in 1998, with the support of many key figures including Linus Torvalds, Linux International, leaders of the distribution vendors, and even the FreeBSD project. The mission statement of the LSB Project is:

Originally, the LSB intended to create a common reference implementation for the base of a Linux system, backed by a specification. A reference platform model helps develop a specification more quickly as it adjudicates disputes over unclear aspects: if in doubt, the reference platform wins. But in the Linux space, this model was not practical. In an environment of many peer implementations, a reference implementation must either be chosen from among them, or developed independently of them; in either case the non-chosen must then commit to being bug-compatible with the reference. Choosing one would have alienated the other distributions, while having the LSB Project develop their own required resources which were not available, and would have added no real value to the market situation. In reality, a reference platform would have diminished the value of distributors' core software and overly constrain new development, and without the key element of a supportive community, it could not succeed.

Consensus was eventually achieved around a different approach: a written behavioral specification of the services available to LSB programs instead of an implementation defined by package/version pairs. This new focus was realized as a three pronged approach that included a Written Specification which defines the behavior of the system, a formal Test Suite which measures an implementation against the specification, and a Sample Implementation which provides an example instantiation of the specification. In this model the sample implementation has no legal standing, the final decision on interpretations is always done against the written specification.

The LSB project was founded as an open source project, with open participation, merit-based selection, etc. While retaining this structure, the LSB is now a workgroup of the Free Standards Group ( FSG ). The FSG is a not-for-profit organization dedicated to accelerating the use and acceptance of open source technologies through the development, application and promotion of standards. The FSG supports the LSB project's efforts, and operates a certification program that allows conforming systems and applications to use a special logo immediately identifying them as LSB Compliant. The FSG, as a company, can receive funds through memberships and donations, and this is beneficial to the LSB project as a means for some financial support.

To ensure progress on the various tasks the LSB undertakes, the project operates as several related subprojects; the subproject leaders make up an LSB steering committee and operations are coordinated by an elected chairperson.

The LSB is a behavioral specification, which means it describes how things behave, not how they are implemented. For a given interface, the correct calling parameters and possible returns and error outcomes are defined. From these definitions, behavioral tests are developed to measure whether an implementation follows the specification. An important detail is that the LSB only talks about services which must be available to LSB programs; an implementation is allowed to provide other services, and it is allowed to provide the same services in a different way to non-LSB programs. This is handled through versioning: if the LSB version exists and works as specified, there is no problem with providing an interface of the same name with different behavior, as long as the symbol version is different. Thus, existing implementations are able to continue to provide their value add.

Continued innovation and deployment of new software is possible; this activity simply happens outside the LSB space without disturbing it. When new features have become compelling and widely accepted, they can be brought in to the LSB as part of the specification.

This model also makes it possible to provide the LSB as a compatibility layer on top of an operating system, even a non-Linux operating system. The LSB Sample Implementation, one of the three prongs in the LSB strategy described above, is delivered in this layered manner. The Sample can be run as a chroot session, or inside a virtual machine (UML, VMware, etc), providing an LSB runtime environment on a system that may not itself be a conforming system. Thus, as a matter of terminology, a conforming system is referred to as an LSB Runtime instead of as an LSB Linux Distribution.

It is perhaps easiest to consider the LSB concept as a contract between systems and applications: a conforming runtime promises to provide a set of services, that work in the specified manner, to conforming applications; and conforming applications promise to use only those services that are required of the runtime, and only in the way specified.

Figure 1. The LSB Concept

The LSB Specification is actually made up of a set of specifications for a binary portability environment. As it describes a binary environment, some details, such as the sizes of fundamental data types or structure layouts, and symbol versions in libraries, differ between architectures. For example the size of a C language pointer is different between 32-bit and 64-bit architectures. To avoid numerous lists of if arch A then X else if arch B then Y , the specification is split into a generic piece that is common to all architectures ( generic LSB ) and a set of architecture-specific pieces. Most of the information appears in the generic LSB document; the set of architecture-specific variances has turned out to be quite small.

To the extent possible, the LSB builds on existing standards rather than duplicating effort and writing its own specification for each interface. Two such standards are the Single Unix Specification ( SUS ) which has evolved from POSIX and the SVID, and the System V Application Binary Interface (ABI). The LSB uses the ELF object format that has evolved from the System V ABI and the interface behaviors from the SUS, and adds the formal listing of interfaces available in each library and the data structures and constants associated with them.

As a codification of existing practice, the LSB specification references a base standard for a feature and describes variances, if any from that base standard. These are kept to a minimum to leave it possible to implement an LSB runtime without constraining the implementation to any specific package. For example, the cp command may most often come from the coreutils package, but by avoiding requiring all of the extra command options found in that version, some other version of cp, such as perhaps the one from Busybox, might also meet the requirement.

Figure 2. Specification Hierarchy

The LSB is made up of a few distinct pieces and a fair bit of glue.

Prior to LSB 2.0 all of these were presented as a single document with matching supplements for each architecture. With LSB 2.0, the specification is modularized into component pieces as shown in Figure 2 above: ELF, LSB, Packaging (including Filesystem layout) and Graphics (split off from the core LSB). Each component has a set of architecture-specific supplements.

ELF and the libraries make up the ABI, which will be described in more detail shortly.

As of LSB 2.0, the following shared libraries are required by the LSB. Applications using these libraries must use only the interfaces specified for these libraries. Any other libraries used by the application must either be provided with the application and built LSB conforming, be provided LSB conforming by another LSB package, or statically linked into the application. The list of libraries is an area for future growth.

In addition to the ABI portion of the LSB, the specification also describes a set of commands that may be used in scripts associated with the application. It is important to note that this is not intended to be a full user environment; commands are only required by the LSB when they have a specific administrative purpose such as supporting portable installation and startup sequences.

Another key component of the LSB is a program interpreter. The program interpreter (sometimes called runtime linker) is the first thing that is executed when an application is started, and is responsible for identifying required shared libraries, performing runtime relocation and other setup tasks. The LSB requires that each architecture define a unique program loader name that can be distinguished from the regular program loader for that architecture. This provides the operating system a hook early in the process execution in case something special needs to be done to provide the correct runtime environment to the application. In practice, most LSB runtime environments have provided LSB conforming behavior from their regular system libraries, and just made the LSB program interpreter a symbolic link to the regular system program interpreter, but this requirement allows something different to be done if necessary.

Table 2.  Figure 3. Purpose of LSB Linker

To facilitate building not just portable binaries, but also portable packages to install those binaries, the LSB specifies a packaging format. This is a subset of the RPM file format. The LSB does NOT specify that the distribution has to be based on RPM, only that it has some way of correctly processing a file in the RPM format.

The final major component of the LSB is the inclusion of most of the Filesystem Hierarchy Standard ( FHS ). The FHS describes the location of system components, and describes the part of the filesystem space that is reserved to add-on applications. The rules for using this space are designed to avoid naming conflicts amongst different applications and different vendors. A portion of this is a registry of provider and package names which is administered through the Linux Assigned Names and Numbers Authority ( LANANA ).

An Application Binary Interface is a set of rules and definitions that describe how source code is to be transformed into a binary format which can be processed by an operating system, and executed on a particular processor architecture. Normally, this is all handled by the compiler and toolchain (assembler, linker, etc.), so application developers don't have to worry about the details. The purpose of an ABI it to allow for the correct interoperability of object files, no matter how they were produced, including by different compilers.

An ABI has two major parts. The first is the set of rules and definitions that describe the object file format. These are the things typically handled by the compiler and toolchain. For Linux, the ELF file format is used. This is the same format that is used by other operating systems such as Solaris and FreeBSD, but there are specifics particular to Linux, and to each processor architecture supported by the LSB.

As an example of the kind of issues that must be specified, if a compiler emits a special section type which must be understood by the program loader in order to correctly run the program, then that section must be specified in the LSB, or it becomes impossible for implementors to write a program loader that can correctly run that program.

The second major part of an ABI is the definition of which shared libraries will be present, and which interfaces are required in each library. This part of the ABI is always unique to the operating system.

Let's take a look at the classic example:

  main()
  {
     printf("%s %s\n","hello","world");
  }
        

When compiled for ia32, the following assembly code is produced, which shows some of the interesting details of the object file.

.file	"hellow.c"
.section	.rodata
.LC0:
	.string	"world"
.LC1:
	.string	"hello"
.LC2:
	.string	"%s %s\n"
.text
	.align 4
.globl main
	.type	 main,@function
main:
	pushl %ebp
	movl %esp,%ebp
	subl $8,%esp
	addl $-4,%esp
	pushl $.LC0
	pushl $.LC1
	pushl $.LC2
	call printf
	addl $16,%esp
	leave
	ret
        

The ABI specifies how the arguments to any function are passed (on the stack), and the size and ordering of the arguments to printf . The assembly shows the stack being prepared, a call being placed to the routine, then the stack being cleaned up on return. This sequence matches that described by the ABI for this processor. Also note the mapping of function names in C is simple, the name is the same in the object as it is in the C source code.

To illustrate some of the details from the library part of the ABI, let us examine the resulting executable file from our example.

       # ldd hellow
       libc.so.6 => /lib/libc.so.6 (0x40028000)
       /lib/ld-lsb.so.1 => /lib/ld-lsb.so.1 (0x40000000)
        

The program is dynamically linked, and the libraries with which it can be linked (including library version number if required) is defined by the ABI. The LSB ABI defines the program interpreter for an ia32 system to be ld-lsb.so.1 , and the C library to be libc.so.6 .

       # nm hellow
	 .
	 .
08049538 D _DYNAMIC
08049610 D _GLOBAL_OFFSET_TABLE_
08049524 D __data_start
08049528 D __dso_handle
         U __libc_start_main@@GLIBC_2.0
080483d4 T main
         U printf@@GLIBC_2.0
	 .
	 .
	 .
        

On Linux systems, symbols in the C library are versioned, so the ABI portion of the LSB must also define the required versions. Versioning allows an interface to migrate forward by changing the version number, and leaving another instance of the interface in the library with the old version number. In our example, the symbol printf is linked against version GLIBC_2.0 , and will use this version at runtime, even if there are other versions of printf present in the library. Also note that there's a reference to __libc_start_main although we didn't include that in our source program. That was added by the toolchain; such hidden symbols must also be accounted for by the ABI, since a conforming library needs to supply those routines in order to execute the conforming binary.

The ABI defines how aggregate types (structures and unions), are laid out in memory with respect to the order, offset and alignment of member fields, as well as tail padding. The ABI also defines constant values which are passed to and from functions found in shared libraries.

With LSB 2.0, there is also a C++ ABI. The language mapping for C++ is much more complicated than the simple mapping which is used for C. Fortunately, an industry group led by Mark Mitchell at Code Sourcery did a lot of this work for the Intel Itanium architecture. This work has been extended to other architectures, and is included in the current GNU Compiler Collection project.

With C++, what looks like a simple class instantiation can actually result in several additional pieces of data being generated by the toolchain. An example is the easiest way to describe this:

       #include <iostream>

using std::cout;

class foo {
protected:
	int _bar;
public:
	virtual void baz()=0;
};

class bar : foo {
public:
	inline bar() {
		_bar=0;
	};
	inline void baz() {
		cout << "bar " << _bar << " times\n";
		_bar++;
	}
};

class bar baz;

main()
{
	baz.baz();
	baz.baz();
}
        

The instantiation of the class bar has resulted in the following symbols being generated by the compiler.

       # nm hw | c++filt
         .
         .
         .
08048754 T main
0804880e W bar::bar()
08048834 W bar::baz()
0804887a W foo::foo()
080489c0 V vtable for bar
080489e0 V typeinfo for bar
08048a00 V typeinfo name for bar
08048a08 V vtable for foo
08048a14 V typeinfo for foo
08048a1c V typeinfo name for foo
08049d2c B baz
         .
         .
        

For the class bar , three additional data structures are added, a Virtual Table, a Run-Time Type Info table, and a string containing the name of the class. Because class bar is derived from class foo , three data structures are added for it also. Under the right conditions, other data structures may also be produced.

In the C++ language mapping, symbols names are mangled from the name used in the source code. This mangling creates a name that uniquely describes the entire function signature, including return type and parameter types. This results in ordinary looking source names like bar::baz() becoming _ZN3bar3bazEv in the object file.

A subtle point caused by the C++ Standards requirement for identifying function signatures is that types are reduced to underlying types before mangling. On architectures where the type size_t is implemented as an int , the function signature func1(char *, size_t) is reduced to func1(char *, int) before being mangled. While the C++ standard has good reasons for this requirement, it has two undesirable side-effects.

First, name mangling is not fully reversible. Once a function name is mangled, the unmangling of that name can go as far as the reduced signature. At first glance, unmangled names may not look quite like the original, if any reduction took place.

Second, name mangling results in different symbol in the object files on different architectures. On a 64-bit architecture, our previous example is reduced to func1(char *, long) instead because size_t is implemented as a long instead of an int .

Because of these complications, the second part of the ABI, the definition of libstdc++.so , proved to be quite a challenging task. For the C library, there is a lot of consistency across the architectures. With few exceptions, the symbol names are the same for every architecture. For libstdc++.so a substantial number of the symbols are mangled differently on different architectures. This causes the database schema and code for the testing tools to be more complicated than was necessary for the C libraries.

The LSB has been successfully grown from a single architecture, Intel IA32, to include Intel Itanium, IBM PowerPC 32-bit and 64-bit, IBM 390 (31-bit) and 390x (64-bit), and AMD64.

There are now many LSB conforming runtimes. Most of those have become LSB certified: there were almost two dozen for either LSB 1.2 or LSB 1.3 at the time of writing, including nearly all of what would be considered the major distributions. Runtime providers have seen a benefit; their customers have asked for LSB compliance, and recently, several governmental agencies around the world have begun requiring LSB on Linux systems they purchase.

The process of putting in place a test suite to measure behavior has helped both smooth out unintentional incompatibilities between distributions and differences between architectures: the Linux kernel and certain libraries did not always do things quite the same way on all architectures. Some participants feel that this has been LSB's largest contribution to date.

While conforming applications had to wait for there to be a base of conforming systems in place first, the lack of applications has been the biggest disappointment. The LSB project has, as part of its proof-of-concept implementations, built a number of open source programs LSB conforming. Some of the projects producing those programs, such as Samba and Apache, now look likely to being providing an LSB package as one of their standard builds. At this writing, a small number of commercial applications were completing the certification process for LSB 1.3.

The LSB project continues to study the relative dearth of conforming applications to see what can and should be done. It is telling that commercial developers who understand the LSB uniformly praise it as valuable, but may nonetheless not use it. One preliminary conclusion is that those commercial applications who have been committed to a Linux product for a while have already addressed possible incompatibilities in their own way: usually by making partnerships with a few distributions that are key to their business. They may be reluctant to impact those agreements and so LSB becomes, rather than a way to consolidate ports, simply an additional Linux port to be developed and maintained. These same vendors probably also have product commitments to Linux versions that are too old to be LSB conforming and so again, an LSB port looks less attractive. The solution to this appears to be a combination of education and time. As it becomes more likely that every target system can be LSB conforming the value of an LSB port will rise. Applications not already on Linux may not see these issues as problems for them.

Another possible issue is that the LSB specification is too restrictive - key features are not available, making it technically impossible to complete an LSB port. The largest such feature has clearly been the lack of a C++ ABI, which is addressed with the forthcoming LSB 2.0 specification.

There are now three avenues for promoting a feature into the LSB specification family. One is to have the feature added to an existing module. The second is to build the feature as a new LSB module. The third is to build the feature as module which requires an LSB runtime, but which is not managed by the Free Standards Group.

The LSB Futures subproject, which has been charged with evaluating new-feature requests, has developed a checklist against which to measure new features. Following the checklist helps to ensure that features are not prematurely or incorrectly promoted as a standard. Key items on the checklist include:

Other considerations would be technology unencumbered by intellectual property constraints and dependencies being met (that is, dependencies are either in the LSB or progressing in parallel to be added to the same version of the LSB).

Figure 4 shows the results of an analysis tool the LSB project uses to help identify unmet dependencies in programs.

Figure 4. mysql Dependency

The way to add a new feature is to insure an affirmative answer to all of the checklist questions. This applies whichever of the three avenues described above will be followed. The checklist will normally serve as a roadmap for the process by highlighting what remains to be resolved. Each of these points has a key reason for being considered.

An example of why the new-feature checklist needs to be rigorously applied: the LSB approved a 1.3 version of a new architecture supplement based on data gathered from the single existing implementation. Later it evolved that new runtimes were going to disagree with the supplement on glibc symbol versions, with the original having been built off unofficial patches applied to an earlier version of glibc than the one in which official support for that architecture was provided. This problem could not be compatibly resolved, and the supplement had to be withdrawn. The de-facto standard checklist item would have helped prevent this situation.

With LSB 2.0, two new concepts are introduced with the purpose of making it easier for groups other than the LSB to promote features for standardization. This enables a process where interested expert groups can build a specification that plugs into or is built on top of the LSB.