Chapter 8 Thread-Local Storage

The compilation environment supports the declaration of thread-local data. This data is sometimes referred to as thread-specific, or thread-private data, but more typically by the acronym TLS. By declaring variables to be thread-local, the compiler automatically arranges for these variables to be allocated on a per-thread basis.

The built-in support for this feature serves three purposes.

• A foundation is provided upon which the POSIX interfaces for allocating thread specific data are built.

• A convenient, and efficient mechanism for direct use of thread local variables by applications and libraries is provided.

• Compilers can allocate TLS as necessary when performing loop-parallelizing optimizations.

C/C++ Programming Interface

Variables are declared thread-local using the __thread keyword, as in the following examples.

__thread int i;
__thread char *p;
__thread struct state s;

During loop optimizations, the compiler can choose to create thread-local temporaries as needed.

Applicability

The __thread keyword can be applied to any global, file-scoped static, or function-scoped static variable. It has no effect on automatic variables, which are always thread-local.

Initialization

In C++, a thread-local variable can not be initialized if the initialization requires a static constructor. Otherwise, a thread-local variable can be initialized to any value that would be legal for an ordinary static variable.

No variable, thread-local or otherwise, can be statically initialized to the address of a thread-local variable.

Binding

Thread-local variables can be declared externally and referenced externally. Thread-local variables are subject to the same interposition rules as normal symbols.

Dynamic loading restrictions

Various TLS access models are available. See Thread-Local Storage Access Models. Shared object developers should be aware of the restrictions imposed by some of these access models in relation to object loading. A shared object can be dynamically loaded during process startup, or after process startup by means of lazy loading, filters, or dlopen(3C). At the completion of process startup, the thread pointer for the main thread is established. All static TLS storage requirements are calculated before the thread pointer is established.

Shared objects that reference thread-local variables, should insure that every translation unit containing the reference is compiled with a dynamic TLS model. This model of access provides the greatest flexibility for loading shared objects. However, static TLS models can generate faster code. Shared objects that use a static TLS model can be loaded as part of process initialization. However, after process initialization, shared objects that use a static TLS model can only be loaded if sufficient backup TLS storage is available. See Program Startup.

Address-of operator

The address-of operator, &, can be applied to a thread-local variable. This operator is evaluated at runtime, and returns the address of the variable within the current thread. The address obtained by this operator can be used freely by any thread in the process as long as the thread that evaluated the address remains in existence. When a thread terminates, any pointers to thread-local variables in that thread become invalid.

When dlsym(3C) is used to obtain the address of a thread-local variable, the address that is returned is the address of the instance of that variable in the thread that called dlsym().

Thread-Local Storage Section

Separate copies of thread-local data that have been allocated at compile-time, must be associated with individual threads of execution. To provide this data, TLS sections are used to specify the size and initial contents. The compilation environment allocates TLS in sections that are identified with the SHF_TLS flag. These sections provide initialized TLS and uninitialized TLS based on how the storage is declared.

• An initialized thread-local variable is allocated in a .tdata, or .tdata1 section. This initialization can require relocation.

• An uninitialized thread-local variable is defined as a COMMON symbol. The resulting allocation is made in a .tbss section.

The uninitialized section is allocated immediately following any initialized sections, subject to padding for proper alignment. Together, the combined sections form a TLS template that is used to allocate TLS whenever a new thread is created. The initialized portion of this template is called the TLS initialization image. All relocations that are generated as a result of initialized thread-local variables are applied to this template. The relocated values are used when a new thread requires the initial values.

TLS symbols have the symbol type STT_TLS. These symbols are assigned offsets relative to the beginning of the TLS template. The actual virtual address that is associated with these symbols is irrelevant. The address refers only to the template, and not to the per-thread copy of each data item. In dynamic executables and shared objects, the st_value field of a STT_TLS symbol contains the assigned TLS offset for defined symbols. This field contains zero for undefined symbols.

Several relocations are defined to support access to TLS. See SPARC: Thread-Local Storage Relocation Types, 32-bit x86: Thread-Local Storage Relocation Types and x64: Thread-Local Storage Relocation Types. TLS relocations typically reference symbols of type STT_TLS. TLS relocations can also reference local section symbols in association with a GOT entry. In this case, the assigned TLS offset is stored in the associated GOT entry.

In dynamic executables and shared objects, a PT_TLS program entry describes a TLS template. This template has the following members.

Runtime Allocation of Thread-Local Storage

TLS is created at three occasions during the lifetime of a program.

• At program startup.

• When a new thread is created.

• When a thread references a TLS block for the first time after a shared object is loaded following program startup.

Thread-local data storage is laid out at runtime as illustrated in Figure 8–1.

Figure 8–1 Runtime Storage Layout of Thread-Local Storage

Program Startup

At program startup, the runtime system creates TLS for the main thread.

First, the runtime linker logically combines the TLS templates for all loaded dynamic objects, including the dynamic executable, into a single static template. Each dynamic objects's TLS template is assigned an offset within the combined template, tlsoffset_m, as follows.

• tlsoffset₁ = round(tlssize₁, align₁ )

• tlsoffset_m+1 = round(tlsoffset_m + tlssize_m+1, align_m+1)

tlssize_m+1 and align_m+1 are the size and alignment, respectively, for the allocation template for dynamic object m. Where 1 <= m <= M, and M is the total number of loaded dynamic objects. The round(offset, align) function returns an offset rounded up to the next multiple of align.

Next, the runtime linker computes the allocation size that is required for the startup TLS, tlssize_S. This size is equal to tlsoffset_M, plus an additional 512 bytes. This addition provides a backup reservation for static TLS references. Shared objects that make static TLS references, and are loaded after process initialization, are assigned to this backup reservation. However, this reservation is a fixed, limited size. In addition, this reservation is only capable of providing storage for uninitialized TLS data items. For maximum flexibility, shared objects should reference thread-local variables using a dynamic TLS model.

The static TLS arena associated with the calculated TLS size tlssize_S, is placed immediately preceding the thread pointer tp_t. Accesses to this TLS data is based off of subtractions from tp_t.

The static TLS arena is associated with a linked list of initialization records. Each record in this list describes the TLS initialization image for one loaded dynamic object. Each record contains the following fields.

• A pointer to the TLS initialization image.

• The size of the TLS initialization image.

• The tlsoffset_m of the object.

• A flag indicating whether the object uses a static TLS model.

The thread library uses this information to allocate storage for the initial thread. This storage is initialized, and a dynamic TLS vector for the initial thread is created.

Thread Creation

For the initial thread, and for each new thread created, the thread library allocates a new TLS block for each loaded dynamic object. Blocks can be allocated separately, or as a single contiguous block.

Each thread t, has an associated thread pointer tp_t, which points to the thread control block, TCB. The thread pointer, tp, always contains the value of tp_t for the current running thread.

The thread library then creates a vector of pointers, dtv_t, for the current thread t. The first element of each vector contains a generation number gen_t, which is used to determine when the vector needs to be extended. See Deferred Allocation of Thread-Local Storage Blocks.

Each element remaining in the vector dtv_t,m, is a pointer to the block that is reserved for the TLS belonging to the dynamic object m.

For dynamically loaded, post-startup objects, the thread library defers the allocation of TLS blocks. Allocation occurs when the first reference is made to a TLS variable within the loaded object. For blocks whose allocation has been deferred, the pointer dtv_t,m is set to an implementation-defined special value.

The runtime linker can group TLS templates for all startup objects so as to share a single element in the vector, dtv _t,1. This grouping does not affect the offset calculations described previously or the creation of the list of initialization records. For the following sections, however, the value of M, the total number of objects, start with the value of 1.

The thread library then copies the initialization images to the corresponding locations within the new block of storage.

Post-Startup Dynamic Loading

A shared object containing only dynamic TLS can be loaded following process startup without limitations. The runtime linker extends the list of initialization records to include the initialization template of the new object. The new object is given an index of m = M + 1. The counter M is incremented by 1. However, the allocation of new TLS blocks is deferred until the blocks are actually referenced.

When a shared object that contains only dynamic TLS is unloaded, the TLS blocks used by that shared object are freed.

A shared object containing static TLS can be loaded following process startup with limitations. Static TLS references can only be satisfied from any remaining backup TLS reservation. See Program Startup. This reservation is limited in size. In addition, this reservation can only provide storage for uninitialized TLS data items.

A shared object that contains static TLS is never unloaded. The shared object is tagged as non-deletable as a consequence of processing the static TLS.

Deferred Allocation of Thread-Local Storage Blocks

In a dynamic TLS model, when a thread t needs to access a TLS block for object m, the code updates the dtv_t and performs the initial allocation of the TLS block. The thread library provides the following interface to provide for dynamic TLS allocation.

typedef struct {
    unsigned long ti_moduleid;
    unsigned long ti_tlsoffset;
} TLS_index;

extern void * __tls_get_addr(TLS_index * ti);     (SPARC and x64)
extern void * ___tls_get_addr(TLS_index * ti);    (32–bit x86)

The SPARC and 64–bit x86 definitions of this function have the same function signature. However, the 32–bit x86 version does not use the default calling convention of passing arguments on the stack. Instead, the 32–bit x86 version passes its arguments by means of the %eax register which is more efficient. To denote that this alternate calling method is used, the 32–bit x86 function name has three leading underscores in its name.

Both versions of tls_get_addr() check the per-thread generation counter, gen_t, to determine whether the vector needs to be updated. If the vector dtv_t is out of date, the routine updates the vector, possibly reallocating the vector to make room for more entries. The routine then checks to see if the TLS block corresponding to dtv_t,m has been allocated. If the vector has not been allocated, the routine allocates and initializes the block. The routine uses the information in the list of initialization records provided by the runtime linker. The pointer dtv _t,m is set to point to the allocated block. The routine returns a pointer to the given offset within the block.

Thread-Local Storage Access Models

Each TLS reference follows one of the following access models. These models are listed from the most general, but least optimized, to the fastest, but most restrictive.

General Dynamic (GD) - dynamic TLS

This model allows reference of all TLS variables, from either a shared object or a dynamic executable. This model also supports the deferred allocation of a TLS block when the block is first referenced from a specific thread.

Local Dynamic (LD) - dynamic TLS of local symbols

This model is a optimization of the GD model. The compiler might determine that a variable is bound locally, or protected, within the object being built. In this case, the compiler instructs the link-editor to statically bind the dynamic tlsoffset and use this model. This model provides a performance benefit over the GD model. Only one call to tls_get_addr() is required per function, to determine the address of dtv_0,m. The dynamic TLS offset, bound at link-edit time, is added to the dtv_0,m address for each reference.

Initial Executable (IE) - static TLS with assigned offsets

This model can only reference TLS variables which are available as part of the initial static TLS template. This template is composed of all TLS blocks that are available at process startup, plus a small backup reservation. See Program Startup. In this model, the thread pointer-relative offset for a given variable x is stored in the GOT entry for x.

This model can reference a limited number of TLS variables from shared libraries loaded after initial process startup, such as by means of lazy loading, filters, or dlopen(3C). This access is satisfied from a fixed backup reservation. This reservation can only provide storage for uninitialized TLS data items. For maximum flexibility, shared objects should reference thread-local variables using a dynamic TLS model.

---

Note –

Filters can be employed to dynamically select the use of static TLS. A shared object can be built to use dynamic TLS, and act as an auxiliary filter upon a counterpart built to use static TLS. If resourses allow the static TLS object to be loaded, the object is used. Otherwise, a fall back to the dynamic TLS object insures that the functionality provided by the shared object is always available. For more information on filters see Shared Objects as Filters.

---

Local Executable (LE) - static TLS

This model can only reference TLS variables which are part of the TLS block of the dynamic executable. The link-editor calculates the thread pointer-relative offsets statically, without the need for dynamic relocations, or the extra reference to the GOT. This model can not be used to reference variables outside of the dynamic executable.

The link-editor can transition code from the more general access models to the more optimized models, if the transition is determined appropriate. This transitioning is achievable through the use of unique TLS relocations. These relocations, not only request updates be performed, but identify which TLS access model is being used.

Knowledge of the TLS access model, together with the type of object being created, allows the link-editor to perform translations. An example is if a relocatable object using the GD access model is being linked into a dynamic executable. In this case, the link-editor can transition the references using the IE or LE access models, as appropriate. The relocations that are required for the model are then performed.

The following diagram illustrates the different access models, together with the transition of one model to another model.

Figure 8–2 Thread-Local Storage Access Models and Transitions

SPARC: Thread-Local Variable Access

On SPARC, the following code sequence models are available for accessing thread-local variables.

SPARC: General Dynamic (GD)

This code sequence implements the GD model described in Thread-Local Storage Access Models.

# %l7 - initialized to GOT pointer

0x00 sethi %hi(@dtlndx(x)), %o0
0x04 add   %o0, %lo(@dtlndx(x)), %o0
0x08 add   %l7, %o0, %o0
0x0c call  x@TLSPLT

# %o0 - contains address of TLS variable

 
 
R_SPARC_TLS_GD_HI22
R_SPARC_TLS_GD_LO10
R_SPARC_TLS_GD_ADD
R_SPARC_TLS_GD_CALL

 
x
x
x
x

GOT[n]
GOT[n + 1]

R_SPARC_TLS_DTPMOD32
R_SPARC_TLS_DTPOFF32

x
x

GOT[n]
GOT[n + 1]

R_SPARC_TLS_DTPMOD64
R_SPARC_TLS_DTPOFF64

x
x

The sethi, and add instructions generate R_SPARC_TLS_GD_HI22 and R_SPARC_TLS_GD_LO10 relocations respectively. These relocations instruct the link-editor to allocate space in the GOT to hold a TLS_index structure for variable x. The link-editor processes this relocation by substituting the GOT-relative offset for the new GOT entry.

The load object index and TLS block index for x are not known until runtime. Therefore, the link-editor places the R_SPARC_TLS_DTPMOD32 and R_SPARC_TLS_DPTOFF32 relocations against the GOT for processing by the runtime linker.

The second add instruction causes the generation of the R_SPARC_TLS_GD_ADD relocation. This relocation is used only if the GD code sequence is changed to another sequence by the link-editor.

The call instruction uses the special syntax, x@TLSPLT. This call references the TLS variable and generates the R_SPARC_TLS_GD_CALL relocation. This relocation instructs the link-editor to bind the call to the __tls_get_addr() function, and associates the call instruction with the GD code sequence.

The add instruction must appear before the call instruction. The add instruction can not be placed into the delay slot for the call. This requirement is necessary as the code-transformations that can occur later require a known order.

The register used as the GOT-pointer for the add instruction tagged by the R_SPARC_TLS_GD_ADD relocation, must be the first register in the add instruction. This requirement permits the link-editor to identify the GOT-pointer register during a code transformation.

SPARC: Local Dynamic (LD)

This code sequence implements the LD model described in Thread-Local Storage Access Models.

# %l7 - initialized to GOT pointer

0x00 sethi %hi(@tmndx(x1)), %o0
0x04 add   %o0, %lo(@tmndx(x1)), %o0
0x08 add   %l7, %o0, %o0
0x0c call  x@TLSPLT

# %o0 - contains address of TLS block of current object

0x10 sethi %hi(@dtpoff(x1)), %l1
0x14 xor   %l1, %lo(@dtpoff(x1)), %l1
0x18 add   %o0, %l1, %l1

# %l1 - contains address of local TLS variable x1

0x20 sethi %hi(@dtpoff(x2)), %l2
0x24 xor   %l2, %lo(@dtpoff(x2)), %l2
0x28 add   %o0, %l2, %l2

# %l2 - contains address of local TLS variable x2

 
 
R_SPARC_TLS_LDM_HI22
R_SPARC_TLS_LDM_LO10
R_SPARC_TLS_LDM_ADD
R_SPARC_TLS_LDM_CALL
 
 
 
R_SPARC_TLS_LDO_HIX22
R_SPARC_TLS_LDO_LOX10
R_SPARC_TLS_LDO_ADD
 
 
 
R_SPARC_TLS_LDO_HIX22
R_SPARC_TLS_LDO_LOX10
R_SPARC_TLS_LDO_ADD

 
 
x1
x1
x1
x1
 
 
 
x1
x1
x1
 
 
 
x2
x2
x2

GOT[n]
GOT[n + 1]

R_SPARC_TLS_DTPMOD32
<none>

x1

GOT[n]
GOT[n + 1]

R_SPARC_TLS_DTPMOD64
<none>

x1

The first sethi instruction and add instruction generate R_SPARC_TLS_LDM_HI22 and R_SPARC_TLS_LDM_LO10 relocations respectively. These relocations instruct the link-editor to allocate space in the GOT to hold a TLS_index structure for the current object. The link-editor processes this relocation by substituting the GOT -relative offset for the new GOT entry.

The load object index is not known until runtime. Therefore, a R_SPARC_TLS_DTPMOD32 relocation is created, and the ti_tlsoffset field of the TLS_index structure is zero filled.

The second add and the call instruction are tagged with the R_SPARC_TLS_LDM_ADD and R_SPARC_TLS_LDM_CALL relocations respectively.

The following sethi instruction and xor instruction generate the R_SPARC_LDO_HIX22 and R_SPARC_TLS_LDO_LOX10 relocations, respectively. The TLS offset for each local symbol is known at link-edit time, therefore these values are filled in directly. The add instruction is tagged with the R_SPARC_TLS_LDO_ADD relocation.

When a procedure references more than one local symbol, the compiler generates code to obtain the base address of the TLS block once. This base address is then used to calculate the address of each symbol without a separate library call.

The register containing the TLS object address in the add instruction tagged by the R_SPARC_TLS_LDO_ADD must be the first register in the instruction sequence. This requirement permits the link-editor to identify the register during a code transformation.

32-bit SPARC: Initial Executable (IE)

This code sequence implements the IE model described in Thread-Local Storage Access Models.

# %l7 - initialized to GOT pointer, %g7 - thread pointer

0x00 sethi %hi(@tpoff(x)), %o0
0x04 or    %o0, %lo(@tpoff(x)), %o0
0x08 ld    [%l7 + %o0], %o0
0x0c add   %g7, %o0, %o0
 
# %o0 - contains address of TLS variable

 
 
R_SPARC_TLS_IE_HI22
R_SPARC_TLS_IE_LO10
R_SPARC_TLS_IE_LD
R_SPARC_TLS_IE_ADD

 
 
x
x
x
x

GOT[n]

R_SPARC_TLS_TPOFF32

x

The sethi instruction and or instruction generate R_SPARC_TLS_IE_HI22 and R_SPARC_TLS_IE_LO10 relocations, respectively. These relocations instruct the link-editor to create space in the GOT to store the static TLS offset for symbol x. An R_SPARC_TLS_TPOFF32 relocation is left outstanding against the GOT for the runtime linker to fill in with the negative static TLS offset for symbol x. The ld and the add instructions are tagged with the R_SPARC_TLS_IE_LD and R_SPARC_TLS_IE_ADD relocations respectively.

The register used as the GOT-pointer for the add instruction tagged by the R_SPARC_TLS_IE_ADD relocation must be the first register in the instruction. This requirement permits the link-editor to identify the GOT-pointer register during a code transformation.

64-bit SPARC: Initial Executable (IE)

This code sequence implements the IE model described in Thread-Local Storage Access Models.

# %l7 - initialized to GOT pointer, %g7 - thread pointer

0x00 sethi %hi(@tpoff(x)), %o0
0x04 or    %o0, %lo(@tpoff(x)), %o0
0x08 ldx   [%l7 + %o0], %o0
0x0c add   %g7, %o0, %o0
 
# %o0 - contains address of TLS variable

 
 
R_SPARC_TLS_IE_HI22
R_SPARC_TLS_IE_LO10
R_SPARC_TLS_IE_LD
R_SPARC_TLS_IE_ADD

 
 
x
x
x
x

GOT[n]

R_SPARC_TLS_TPOFF64

x

SPARC: Local Executable (LE)

This code sequence implements the LE model described in Thread-Local Storage Access Models.

# %g7 - thread pointer

0x00 sethi %hix(@tpoff(x)), %o0
0x04 xor   %o0,%lo(@tpoff(x)),%o0
0x08 add   %g7, %o0, %o0
 
# %o0 - contains address of TLS variable

 
 
R_SPARC_TLS_LE_HIX22
R_SPARC_TLS_LE_LOX10
<none>

 
 
x
x

The sethi and xor instructions generate R_SPARC_TLS_LE_HIX22 and R_SPARC_TLS_LE_LOX10 relocations respectively. The link-editor binds these relocations directly to the static TLS offset for the symbol defined in the executable. No relocation processing is required at runtime.

SPARC: Thread-Local Storage Relocation Types

The TLS relocations that are listed in the following table are defined for SPARC. Descriptions in the table use the following notation.

@dtlndx(x)

Allocates two contiguous entries in the GOT to hold a TLS_index structure. This information is passed to __tls_get_addr(). The instruction referencing this entry is bound to the address of the first of the two GOT entries.

@tmndx(x)

Allocates two contiguous entries in the GOT to hold a TLS_index structure. This information is passed to __tls_get_addr(). The ti_tlsoffset field of this structure is set to 0, and the ti_moduleid is filled in at runtime. The call to __tls_get_addr() returns the starting offset of the dynamic TLS block.

@dtpoff(x)

Calculates the tlsoffset relative to the TLS block.

@tpoff(x)

Calculates the negative tlsoffset relative to the static TLS block. This value is added to the thread-pointer to calculate the TLS address.

@dtpmod(x)

Calculates the object identifier of the object containing a TLS symbol.

Some relocation types have semantics beyond simple calculations.

R_SPARC_TLS_GD_ADD

This relocation tags the add instruction of a GD code sequence. The register used for the GOT-pointer is the first register in the sequence. The instruction tagged by this relocation comes before the call instruction tagged by the R_SPARC_TLS_GD_CALL relocation. This relocation is used to transition between TLS models at link-edit time.

R_SPARC_TLS_GD_CALL

This relocation is handled as if it were a R_SPARC_WPLT30 relocation referencing the __tls_get_addr() function. This relocation is part of a GD code sequence.

R_SPARC_LDM_ADD

This relocation tags the first add instruction of a LD code sequence. The register used for the GOT-pointer is the first register in the sequence. The instruction tagged by this relocation comes before the call instruction tagged by the R_SPARC_TLS_GD_CALL relocation. This relocation is used to transition between TLS models at link-edit time.

R_SPARC_LDM_CALL

This relocation is handled as if it were a R_SPARC_WPLT30 relocation referencing the __tls_get_addr() function. This relocation is part of a LD code sequence.

R_SPARC_LDO_ADD

This relocation tags the final add instruction in a LD code sequence. The register which contains the object address that is computed in the initial part of the code sequence is the first register in this instruction. This relocation permits the link-editor to identify this register for code transformations.

R_SPARC_TLS_IE_LD

This relocation tags the ld instruction in the 32–bit IE code sequence. This relocation is used to transition between TLS models at link-edit time.

R_SPARC_TLS_IE_LDX

This relocation tags the ldx instruction in the 64–bit IE code sequence. This relocation is used to transition between TLS models at link-edit time.

R_SPARC_TLS_IE_ADD

This relocation tags the add instruction in the IE code sequence. The register that is used for the GOT-pointer is the first register in the sequence.

32-bit x86: Thread-Local Variable Access

On x86, the following code sequence models are available for accessing TLS.

32-bit x86: General Dynamic (GD)

This code sequence implements the GD model described in Thread-Local Storage Access Models.

0x00 leal  x@tlsgd(,%ebx,1), %eax
0x07 call  x@tlsgdplt

# %eax - contains address of TLS variable

R_386_TLS_GD
R_386_TLS_GD_PLT

x
x

GOT[n]
GOT[n + 1]

R_386_TLS_DTPMOD32
R_386_TLS_DTPOFF32

x

The leal instruction generates a R_386_TLS_GD relocation which instructs the link-editor to allocate space in the GOT to hold a TLS_index structure for variable x. The link-editor processes this relocation by substituting the GOT-relative offset for the new GOT entry.

Since the load object index and TLS block index for x are not known until runtime, the link-editor places the R_386_TLS_DTPMOD32 and R_386_TLS_DTPOFF32 relocations against the GOT for processing by the runtime linker. The address of the generated GOT entry is loaded into register %eax for the call to ___tls_get_addr().

The call instruction causes the generation of the R_386_TLS_GD_PLT relocation. This instructs the link-editor to bind the call to the ___tls_get_addr() function and associates the call instruction with the GD code sequence.

The call instruction must immediately follow the leal instruction. This requirement is necessary to permit the code transformations.

x86: Local Dynamic (LD)

This code sequence implements the LD model described in Thread-Local Storage Access Models.

0x00 leal  x1@tlsldm(%ebx), %eax
0x06 call  x1@tlsldmplt

# %eax - contains address of TLS block of current object

0x10 leal  x1@dtpoff(%eax), %edx

# %edx - contains address of local TLS variable x1

0x20 leal  x2@dtpoff(%eax), %edx

# %edx - contains address of local TLS variable x2

R_386_TLS_LDM
R_386_TLS_LDM_PLT
 
 
 
R_386_TLS_LDO_32
 
 
 
R_386_TLS_LDO_32

x1
x1
 
 
 
x1
 
 
 
x2

GOT[n]
GOT[n + 1]

R_386_TLS_DTPMOD32
<none>

x

The first leal instruction generates a R_386_TLS_LDM relocation. This relocation instructs the link-editor to allocate space in the GOT to hold a TLS_index structure for the current object. The link-editor process this relocation by substituting the GOT -relative offset for the new linkage table entry.

The load object index is not known until runtime. Therefore, a R_386_TLS_DTPMOD32 relocation is created, and the ti_tlsoffset field of the structure is zero filled. The call instruction is tagged with the R_386_TLS_LDM_PLT relocation.

The TLS offset for each local symbol is known at link-edit time so the link-editor fills these values in directly.

When a procedure references more than one local symbol, the compiler generates code to obtain the base address of the TLS block once. This base address is then used to calculate the address of each symbol without a separate library call.

32-bit x86: Initial Executable (IE)

This code sequence implements the IE model described in Thread-Local Storage Access Models.

Two code-sequences for the IE model exist. One sequence is for position independent code which uses a GOT-pointer. The other sequence is for position dependent code which does not use a GOT-pointer.

0x00 movl  %gs:0, %eax
0x06 addl  x@gotntpoff(%ebx), %eax

# %eax - contains address of TLS variable

<none>
R_386_TLS_GOTIE

 
x

GOT[n]

R_386_TLS_TPOFF

x

The addl instruction generates a R_386_TLS_GOTIE relocation. This relocation instructs the link–editor to create space in the GOT to store the static TLS offset for symbol x. A R_386_TLS_TPOFF relocation is left outstanding against the GOT table for the runtime linker to fill in with the static TLS offset for symbol x.

0x00 movl  %gs:0, %eax
0x06 addl  x@indntpoff, %eax

# %eax - contains address of TLS variable

<none>
R_386_TLS_IE

 
x

GOT[n]

R_386_TLS_TPOFF

x

The addl instruction generates a R_386_TLS_IE relocation. This relocation instructs the link-editor to create space in the GOT to store the static TLS offset for symbol x. The main difference between this sequence and the position independent form, is that the instruction is bound directly to the GOT entry created, instead of using an offset off of the GOT-pointer register. A R_386_TLS_TPOFF relocation is left outstanding against the GOT for the runtime linker to fill in with the static TLS offset for symbol x.

The contents of variable x, rather than the address, can be loaded by embedding the offset directly into the memory reference as shown in the next two sequences.

0x00 movl  x@gotntpoff(%ebx), %eax
0x06 movl  %gs:(%eax), %eax

# %eax - contains address of TLS variable

R_386_TLS_GOTIE
<none>

x

GOT[n]

R_386_TLS_TPOFF

x

0x00 movl  x@indntpoff, %ecx
0x06 movl  %gs:(%ecx), %eax

# %eax - contains address of TLS variable

R_386_TLS_IE
<none>

x

GOT[n]

R_386_TLS_TPOFF

x

In the last sequence, if the %eax register is used instead of the %ecx register, the first instruction can be either 5 or 6 bytes long.

32-bit x86: Local Executable (LE)

This code sequence implements the LE model described in Thread-Local Storage Access Models.

0x00 movl  %gs:0, %eax
0x06 leal  x@ntpoff(%eax), %eax

# %eax - contains address of TLS variable

<none>
R_386_TLS_LE

 
x

The movl instruction generates a R_386_TLS_LE_32 relocation. The link-editor binds this relocation directly to the static TLS offset for the symbol defined in the executable. No processing is required at runtime.

The contents of variable x, rather then the address, can be accessed with the same relocation by using the following instruction sequence.

0x00 movl  %gs:0, %eax
0x06 movl  x@ntpoff(%eax), %eax

# %eax - contains address of TLS variable

<none>
R_386_TLS_LE

 
x

Rather than computing the address of the variable, a load from the variable or store to the variable can be accomplished using the following sequence. Note, the x@ntpoff expression is not used as an immediate value, but as an absolute address.

0x00 movl %gs:x@ntpoff, %eax

# %eax - contains address of TLS variable

R_386_TLS_LE

x

32-bit x86: Thread-Local Storage Relocation Types

The TLS relocations that are listed in the following table are defined for x86. Descriptions in the table use the following notation.

@tlsgd(x)

Allocates two contiguous entries in the GOT to hold a TLS_index structure. This structure is passed to ___tls_get_addr(). The instruction referencing this entry will be bound to the first of the two GOT entries.

@tlsgdplt(x)

This relocation is handled as if it were a R_386_PLT32 relocation referencing the ___tls_get_addr() function.

@tlsldm(x)

Allocates two contiguous entries in the GOT to hold a TLS_index structure. This structure is passed to the ___tls_get_addr(). The ti_tlsoffset field of the TLS_index is set to 0, and the ti_moduleid is filled in at runtime. The call to ___tls_get_addr() returns the starting offset of the dynamic TLS block.

@gotntpoff(x)

Allocates a entry in the GOT, and initializes the entry with the negative tlsoffset relative to the static TLS block. This sequence is performed at runtime using the R_386_TLS_TPOFF relocation.

@indntpoff(x)

This expression is similar to @gotntpoff, but is used in position dependent code. @gotntpoff resolves to a GOT slot address relative to the start of the GOT in the movl or addl instructions. @indntpoff resolves to the absolute GOT slot address.

@ntpoff(x)

Calculates the negative tlsoffset relative to the static TLS block.

@dtpoff(x)

Calculates the tlsoffset relative to the TLS block. The value is used as an immediate value of an addend and is not associated with a specific register.

@dtpmod(x)

Calculates the object identifier of the object containing a TLS symbol.

x64: Thread-Local Variable Access

On x64, the following code sequence models are available for accessing TLS

x64: General Dynamic (GD)

This code sequence implements the GD model described in Thread-Local Storage Access Models.

0x00 .byte 0x66
0x01 leaq  x@tlsgd(%rip), %rdi
0x08 .word 0x666
0x0a rex64
0x0b call  __tls_get_addr@plt

# %iax - contains address of TLS variable

<none>
R_AMD64_TLSGD
<none>
<none>
R_AMD64_PLT32

 
x


__tls_get_addr

GOT[n]
GOT[n + 1]

R_AMD64_DTPMOD64
R_AMD64_DTPOFF64

x
x

The __tls_get_addr() function takes a single parameter, the address of the tls_index structure. The R_AMD64_TLSGD relocation that is associated with the x@tlsgd(%rip) expression, instructs the link-editor to allocate a tls_index structure within the GOT. The two elements required for the tls_index structure are maintained in consecutive GOT entries, GOT[n] and GOT[n+1]. These GOT entries are associated to the R_AMD64_DTPMOD64 and R_AMD64_DTPOFF64 relocations.

The instruction at address 0x00 computes the address of the first GOT entry. This computation adds the PC relative address of the beginning of the GOT, which is known at link-edit time, to the current instruction pointer. The result is passed using the %rdi register to the __tls_get_addr() function.

The leaq instruction computes the address of the first GOT entry. This computation is carried out by adding the PC-relative address of the GOT, which was determined at link-edit time, to the current instruction pointer. The .byte, .word, and .rex64 prefixes insure that the whole instruction sequence occupies 16 bytes. Prefixes are employed, as prefixes have no negative inpact on the code.

x64: Local Dynamic (LD)

This code sequence implements the LD model described in Thread-Local Storage Access Models.

0x00 leaq  x1@tlsld(%rip), %rdi
0x07 call  __tls_get_addr@plt

# %rax - contains address of TLS block

0x10 leaq  x1@dtpoff(%rax), %rcx

# %rcx - contains address of TLS variable x1

0x20 leaq  x2@dtpoff(%rax), %r9

# %rcx - contains address of TLS variable x2

R_AMD64_TLSLD
R_AMD64_PLT32
 


R_AMD64_DTOFF32
 


R_AMD64_DTOFF32

x1
__tls_get_addr



 
x1


 
x2

GOT[n]

R_AMD64_DTMOD64

x1

The first two instructions are equivalent to the code sequence used for the general dynamic model, although without any padding. The two instructions must be consecutive. The x1@tlsld(%rip) sequence generates a the tls_index entry for symbol x1. This index refers to the current module that contains x1 with an offset of zero. The link-editor creates one relocation for the object, R_AMD64_DTMOD64.

The R_AMD64_DTOFF32 relocation is unnecessary, because offsets are loaded separately. The x1@dtpoff expression is used to access the offset of the symbol x1. Using the instruction as address 0x10, the complete offset is loaded and added to the result of the __tls_get_addr() call in %rax to produce the result in %rcx. The x1@dtpoff expression creates the R_AMD64_DTPOFF32 relocation.

Instead of computing the address of the variable, the value of the variable can be loaded using the following instruction. This instruction creates the same relocation as the original leaq instruction.

movq  x1@dtpoff(%rax), %r11

Provided the base address of a TLS block is maintained within a register, loading, storing or computing the address of a protected thread-local variable requires one instruction.

Benefits exist in using the local dynamic model over the general dynamic model. Every additional thread-local variable access only requires three new instructions. In addition, no additional GOT entries, or runtime relocations are required.

x64: Initial Executable (IE)

This code sequence implements the IE model described in Thread-Local Storage Access Models.

0x00 movq  %fs:0, %rax
0x09 addq  x@gottpoff(%rip), %rax

# %rax - contains address of TLS variable

<none>
R_AMD64_GOTTPOFF

 
x

GOT[n]

R_AMD64_TPOFF64

x

The R_AMD64_GOTTPOFF relocation for the symbol x requests the link-editor to generate a GOT entry and an associated R_AMD64_TPOFF64 relocation. The offset of the GOT entry relative to the end of the x@gottpoff(%rip) instruction, is then used by the instruction. The R_AMD64_TPOFF64 relocation uses the value of the symbol x that is determined from the presently loaded modules. The offset is written in the GOT entry and later loaded by the addq instruction.

To load the contents of x, rather than the address of x, the following sequence is available.

0x00 movq  x@gottpoff(%rip), %rax
0x06 movq  %fs:(%rax), %rax

# %rax - contains contents of TLS variable

R_AMD64_GOTTPOFF
<none>

x

GOT[n]

R_AMD64_TPOFF64

x

x64: Local Executable (LE)

This code sequence implements the LE model described in Thread-Local Storage Access Models.

0x00 movq %fs:0, %rax
0x06 leaq  x@tpoff(%rax), %rax

# %rax - contains address of TLS variable

<none>
R_AMD64_TPOFF32

x

To load the contents of a TLS variable instead of the address of a TLS variable, the following sequence can be used.

0x00 movq %fs:0, %rax
0x06 movq  x@tpoff(%rax), %rax

# %rax - contains contents of TLS variable

<none>
R_AMD64_TPOFF32

x

The following sequence is even shorter.

0x00 movq  %fs:x@tpoff, %rax

# %rax - contains contents of TLS variable

R_AMD64_TPOFF32

x

x64: Thread-Local Storage Relocation Types

The TLS relocations that are listed in the following table are defined for x64. Descriptions in the table use the following notation.

@tlsgd(%rip)

Allocates two contiguous entries in the GOT to hold a TLS_index structure. This structure is passed to __tls_get_addr(). This instruction can only be used in the exact general dynamic code sequence.

@tlsld(%rip)

Allocates two contiguous entries in the GOT to hold a TLS_index structure. This structure is passed to __tls_get_addr(). At runtime, the ti_offset offset field of the object is set to zero, and the ti_module offset is initialized. A call to the __tls_get_addr() function returns the starting offset if the dynamic TLS block. This instruction can be used in the exact code sequence.

@dtpoff

Calculates the offset of the variable relative to the start of the TLS block which contains the variable. The computed value is used as an immediate value of an addend, and is not associated with a specific register.

@dtpmod(x)

Calculates the object identifier of the object containing a TLS symbol.

@gottpoff(%rip)

Allocates a entry in the GOT, to hold a variable offset in the initial TLS block. This offset is relative to the TLS blocks end, %fs:0. The operator can only be used with a movq or addq instruction.

@tpoff(x)

Calculates the offset of a variable relative to the TLS block end, %fs:0. No GOT entry is created.