From dan at codesourcery.com Thu Nov 4 17:02:34 2004 From: dan at codesourcery.com (Daniel Jacobowitz) Date: Thu, 4 Nov 2004 12:02:34 -0500 Subject: [mips-tls] Revised versions of MIPS TLS ABI specification In-Reply-To: <41814CE8.8060601@codesourcery.com> References: <41814CE8.8060601@codesourcery.com> Message-ID: <20041104170233.GB10079@nevyn.them.org> On Thu, Oct 28, 2004 at 12:47:52PM -0700, Mark Mitchell wrote: > I've attached a revised version of the specification. > > As you will see from the revision history at the top, the only changes > I've made are the incorrect uses of "$(3)" found by Nigel. > > I've also added an Open Issues section, where I've attempted to record > the issues raised thus far. My priority (well, MontaVista's priority) > is to get the basics nailed down so that we can start implementing them > ASAP. That means that I would prefer to get consensus about the basics > of the specification, even if there are some bits that we want to add in > the future. > > I would like to get the GD/LD bits specified first, so we can start > implementing; then we can come back and LE/IE. Thus, I think the > following issues are all things that we can decide later, given that at > the moment we're talking only about General Dynamic and Local Dynamic. > > 1. Add |-mxgot| support. > > We want to do this, for sure, but it does not affect General Dynamic, > and it is clear that this can be added to Local Dynamic without > impacting the currently specified functionality. Yes, it does affect GD. This instruction: 0x08 addiu $4, $28, %tlsgd(x) R_MIPS_TLS_GD x assumes that the GD entry in the GOT is within 64k of $28; this assumption prohibits the use of xgot. However, like the rest of xgot, this can wait a little longer. > 3. Should we use Model I or Model II from Drepper's paper? > > This issue does not arise for Global/Local Dynamic. Well, not for the binutils and GCC portions of the implementation, anyway. > 4. Does __tls_get_addr need a special calling convention? > > This issue does not matter until we start doing linker optimizations. > Until that time, the compiler can be conservative, and assume this is an > ordinary call. > > The following issue does need resolution: > > 1. Should we extend the 32K Local Dynamic Model to 64K by using a biased > offset? > > Daniel and I thought this would be more trouble than it's worth; Thiemo > thinks otherwise. Are there any other opinions? I have since learned that PowerPC already does this: | The PowerPC32 TLS ABI is similar to the PowerPC64 model. The | thread-local storage data structures follow variant I. The TCB is 8 | bytes, with the first 4 bytes containing the pointer to the dynamic | thread vector. tlsoffset calculations and definition of __tls_get_addr | are identical to PowerPC64. r2 is the thread pointer, and points | 0x7000 past the end of the thread control block. Dynamic thread vector | pointers point 0x8000 past the start of each TLS block. (*) This | allows the first 64K of each block to be addressed from a dtv pointer | using fewer machine instructions. The tp offset allows for efficient | addressing of the TCB and up to 4K-8 of other thread library | information. | | (*) For implementation reasons the actual value stored in dtv may point | to the start of a block, however values returned by accessor functions | will be offset by 0x8000. So for local dynamic, offsets would be biased by 0x8000 instead of 0; for initial exec, offsets would be biased by -0x7000 instead of +8. This leaves room for the pthread descriptor to be accessed in a single instruction as long as it is no more than 4k-8 bytes. Shall we use this model? -- Daniel Jacobowitz From mark at codesourcery.com Thu Nov 4 17:08:37 2004 From: mark at codesourcery.com (Mark Mitchell) Date: Thu, 04 Nov 2004 09:08:37 -0800 Subject: MIPS TLS ABI In-Reply-To: <20041103161325.GA5208@linux-mips.org> References: <200410152028.i9FKStNt000557@sirius.codesourcery.com> <20041103161325.GA5208@linux-mips.org> Message-ID: <418A6215.1090104@codesourcery.com> Ralf Baechle wrote: >On Fri, Oct 15, 2004 at 01:28:55PM -0700, Mark Mitchell wrote: > >Mark, > >I'm working on setting up a Linux/MIPS Wiki in the hope it's going to end >the age of terribly incomplete and outdated documentation. As part of >that I would like to put your NTPL draft into the wiki at > > http://wiki.linux-mips.org/index.php/NPTL > >This is currently only a stub page with a link at Ulrich's PDF document. >Is it ok to put your NPTL document there? > > Yes, that's fine. It would be nice if you'd mention CodeSourcery as the initial author, somewhere -- but I'm sure other people will make changes going forward, and they should get credit too! Thanks! -- Mark Mitchell CodeSourcery, LLC (916) 791-8304 mark at codesourcery.com From mark at codesourcery.com Thu Nov 4 17:13:06 2004 From: mark at codesourcery.com (Mark Mitchell) Date: Thu, 04 Nov 2004 09:13:06 -0800 Subject: [mips-tls] Revised versions of MIPS TLS ABI specification In-Reply-To: <20041104170233.GB10079@nevyn.them.org> References: <41814CE8.8060601@codesourcery.com> <20041104170233.GB10079@nevyn.them.org> Message-ID: <418A6322.5060302@codesourcery.com> Daniel Jacobowitz wrote: >So for local dynamic, offsets would be biased by 0x8000 instead of 0; >for initial exec, offsets would be biased by -0x7000 instead of +8. >This leaves room for the pthread descriptor to be accessed in a single >instruction as long as it is no more than 4k-8 bytes. > >Shall we use this model? > > It's OK by me. Are there any special values (0x7ff0 was mentioned?) that make more sense than just 0x8000 and -0x7000, due to the specific way in which MIPS hardware works? Or, which would be easier to implement because the linker, etc., are already set up to deal with them? Ralf, I guess we need to work out who has the authority to change the public Wiki page you're going to create. We certainly want this to be a consensus effort. Is it OK for Daniel to make updates? Also, since the text I've been sending around came from our Wiki (which is ZWiki) would it help to have the Wiki-formatted version to start with? Thanks, -- Mark Mitchell CodeSourcery, LLC (916) 791-8304 mark at codesourcery.com From nigel at mips.com Thu Nov 4 18:00:53 2004 From: nigel at mips.com (Nigel Stephens) Date: Thu, 04 Nov 2004 18:00:53 +0000 Subject: [mips-tls] Revised versions of MIPS TLS ABI specification In-Reply-To: <20041104170233.GB10079@nevyn.them.org> References: <41814CE8.8060601@codesourcery.com> <20041104170233.GB10079@nevyn.them.org> Message-ID: <418A6E55.9030101@mips.com> Daniel Jacobowitz wrote: >>The following issue does need resolution: >> >>1. Should we extend the 32K Local Dynamic Model to 64K by using a biased >>offset? >> >>Daniel and I thought this would be more trouble than it's worth; Thiemo >>thinks otherwise. Are there any other opinions? >> >> > >I have since learned that PowerPC already does this: > >| The PowerPC32 TLS ABI is similar to the PowerPC64 model. The >| thread-local storage data structures follow variant I. The TCB is 8 >| bytes, with the first 4 bytes containing the pointer to the dynamic >| thread vector. tlsoffset calculations and definition of __tls_get_addr >| are identical to PowerPC64. r2 is the thread pointer, and points >| 0x7000 past the end of the thread control block. Dynamic thread vector >| pointers point 0x8000 past the start of each TLS block. (*) This >| allows the first 64K of each block to be addressed from a dtv pointer >| using fewer machine instructions. The tp offset allows for efficient >| addressing of the TCB and up to 4K-8 of other thread library >| information. >| >| (*) For implementation reasons the actual value stored in dtv may point >| to the start of a block, however values returned by accessor functions >| will be offset by 0x8000. > > Hi Daniel I'd be interested to know where that quoted passage comes from - is it a public document, I can't find it with Google? > Shall we use this model? It does sounds like we could use the same trick. But I'd like to understand it better. I thought that the problem was that in variant I the size of the TCB was indeterminate, so the static linker couldn't insert optimised references to initial-exec TLS data as fixed offsets from the thread pointer. But the implication of the above is that the TCB does have a known, fixed size, so the a fixed offset can now be inserted by the linker, as for variant II. Have I got that right? Nigel From nigel at mips.com Thu Nov 4 18:30:20 2004 From: nigel at mips.com (Nigel Stephens) Date: Thu, 04 Nov 2004 18:30:20 +0000 Subject: [mips-tls] Revised versions of MIPS TLS ABI specification In-Reply-To: <418A6322.5060302@codesourcery.com> References: <41814CE8.8060601@codesourcery.com> <20041104170233.GB10079@nevyn.them.org> <418A6322.5060302@codesourcery.com> Message-ID: <418A753C.2010309@mips.com> Mark Mitchell wrote: > Daniel Jacobowitz wrote: > >> So for local dynamic, offsets would be biased by 0x8000 instead of 0; >> for initial exec, offsets would be biased by -0x7000 instead of +8. >> This leaves room for the pthread descriptor to be accessed in a single >> instruction as long as it is no more than 4k-8 bytes. >> >> Shall we use this model? >> >> > It's OK by me. Are there any special values (0x7ff0 was mentioned?) > that make more sense than just 0x8000 and -0x7000, due to the specific > way in which MIPS hardware works? Or, which would be easier to > implement because the linker, etc., are already set up to deal with them? I think this was suggested because 0x7ff0 is currently used by the linker as the offset of the $gp register from the start of the small data region. It's probably this value to ensure that a reference to the sub-words of a larger variable is safe to break down into gp-relative offsets, without causing an underflow below -0x8000, for the largest possible directly loadable scalar (a "long double"). Thiemo may know a better reason :-). But I don't think that causes us a problem with the suggested 0x7000 offset: the resulting range of offsets from -0x7000 up to 0x7fff will work fine on MIPS, but care will be needed when accessing the "private" thread data stored in the 4KB region below the TCB, to make sure that it cannot underflow. Nigel From dan at codesourcery.com Thu Nov 4 21:41:06 2004 From: dan at codesourcery.com (Daniel Jacobowitz) Date: Thu, 4 Nov 2004 16:41:06 -0500 Subject: [mips-tls] Revised versions of MIPS TLS ABI specification In-Reply-To: <418A6E55.9030101@mips.com> References: <41814CE8.8060601@codesourcery.com> <20041104170233.GB10079@nevyn.them.org> <418A6E55.9030101@mips.com> Message-ID: <20041104214105.GB8696@nevyn.them.org> On Thu, Nov 04, 2004 at 06:00:53PM +0000, Nigel Stephens wrote: > Daniel Jacobowitz wrote: > > >>The following issue does need resolution: > >> > >>1. Should we extend the 32K Local Dynamic Model to 64K by using a biased > >>offset? > >> > >>Daniel and I thought this would be more trouble than it's worth; Thiemo > >>thinks otherwise. Are there any other opinions? > >> > >> > > > >I have since learned that PowerPC already does this: > > > >| The PowerPC32 TLS ABI is similar to the PowerPC64 model. The > >| thread-local storage data structures follow variant I. The TCB is 8 > >| bytes, with the first 4 bytes containing the pointer to the dynamic > >| thread vector. tlsoffset calculations and definition of __tls_get_addr > >| are identical to PowerPC64. r2 is the thread pointer, and points > >| 0x7000 past the end of the thread control block. Dynamic thread vector > >| pointers point 0x8000 past the start of each TLS block. (*) This > >| allows the first 64K of each block to be addressed from a dtv pointer > >| using fewer machine instructions. The tp offset allows for efficient > >| addressing of the TCB and up to 4K-8 of other thread library > >| information. > >| > >| (*) For implementation reasons the actual value stored in dtv may point > >| to the start of a block, however values returned by accessor functions > >| will be offset by 0x8000. > > > > > > Hi Daniel > > I'd be interested to know where that quoted passage comes from - is it a > public document, I can't find it with Google? Take a look at ftp://ftp.linuxppc64.org/pub/people/amodra (from memory; I hope I got the path right.) > >Shall we use this model? > > > It does sounds like we could use the same trick. But I'd like to > understand it better. I thought that the problem was that in variant I > the size of the TCB was indeterminate, so the static linker couldn't > insert optimised references to initial-exec TLS data as fixed offsets > from the thread pointer. But the implication of the above is that the > TCB does have a known, fixed size, so the a fixed offset can now be > inserted by the linker, as for variant II. Have I got that right? No. This is a variant I model, which means that the size and layout of the TCB are known; it's in variant II that the TCB is indeterminate. In variant I the TLS areas have a positive offset from the TCB, which starts with a pointer to the DTV; in variant II they have a negative offset from the TCB, which has indeterminate contents. In either case the static linker can resolve IE/LE references that bind to the local executable. Normally in variant I the thread pointer points to the beginning of the TCB, i.e. a known distance before the start of the first TLS block. In this case the thread pointer points to 0x7000 past the _end_ of the first TLS block - but that's just an implementation detail. -- Daniel Jacobowitz From ralf at linux-mips.org Fri Nov 5 05:54:04 2004 From: ralf at linux-mips.org (Ralf Baechle) Date: Fri, 5 Nov 2004 06:54:04 +0100 Subject: MIPS TLS ABI In-Reply-To: <418A6215.1090104@codesourcery.com> References: <200410152028.i9FKStNt000557@sirius.codesourcery.com> <20041103161325.GA5208@linux-mips.org> <418A6215.1090104@codesourcery.com> Message-ID: <20041105055404.GA26110@linux-mips.org> On Thu, Nov 04, 2004 at 09:08:37AM -0800, Mark Mitchell wrote: > Yes, that's fine. It would be nice if you'd mention CodeSourcery as the > initial author, somewhere -- but I'm sure other people will make > changes going forward, and they should get credit too! Yes, definately. Honor who deserves honor but in case of this case also any modification also needs to be discussed with others, so I've added an authors section to the bottom. Ralf From ica2_ts at csv.ica.uni-stuttgart.de Mon Nov 8 16:53:52 2004 From: ica2_ts at csv.ica.uni-stuttgart.de (Thiemo Seufer) Date: Mon, 8 Nov 2004 17:53:52 +0100 Subject: [mips-tls] Revised versions of MIPS TLS ABI specification In-Reply-To: <418A753C.2010309@mips.com> References: <41814CE8.8060601@codesourcery.com> <20041104170233.GB10079@nevyn.them.org> <418A6322.5060302@codesourcery.com> <418A753C.2010309@mips.com> Message-ID: <20041108165352.GB22577@rembrandt.csv.ica.uni-stuttgart.de> Nigel Stephens wrote: [snip] > I think this was suggested because 0x7ff0 is currently used by the > linker as the offset of the $gp register from the start of the small > data region. It's probably this value to ensure that a reference to the > sub-words of a larger variable is safe to break down into gp-relative > offsets, without causing an underflow below -0x8000, for the largest > possible directly loadable scalar (a "long double"). Thiemo may know a > better reason :-). I don't. :-) I simply used the gp offset as example. I guess it has a value of 0x7ff0 instead of 0x8000 for historical reasons (probably some implementation limitation in very early MIPS toolchains). > But I don't think that causes us a problem with the suggested 0x7000 > offset: the resulting range of offsets from -0x7000 up to 0x7fff will > work fine on MIPS, but care will be needed when accessing the "private" > thread data stored in the 4KB region below the TCB, to make sure that it > cannot underflow. Yes, the biases suggested by Daniel (0x8000 and -0x7000) are ok. Thiemo From nigel at mips.com Tue Nov 9 23:38:20 2004 From: nigel at mips.com (Nigel Stephens) Date: Tue, 09 Nov 2004 23:38:20 +0000 Subject: [mips-tls] Revised versions of MIPS TLS ABI specification In-Reply-To: <20041104214105.GB8696@nevyn.them.org> References: <41814CE8.8060601@codesourcery.com> <20041104170233.GB10079@nevyn.them.org> <418A6E55.9030101@mips.com> <20041104214105.GB8696@nevyn.them.org> Message-ID: <419154EC.1050206@mips.com> Daniel Jacobowitz wrote: >Take a look at > ftp://ftp.linuxppc64.org/pub/people/amodra > >(from memory; I hope I got the path right.) > > > Thanks. >No. This is a variant I model, which means that the size and layout of >the TCB are known; it's in variant II that the TCB is indeterminate. In >variant I the TLS areas have a positive offset from the TCB, which >starts with a pointer to the DTV; in variant II they have a negative >offset from the TCB, which has indeterminate contents. In either case >the static linker can resolve IE/LE references that bind to the local >executable. > > Ah, yes. So variant I works because we are defining the TCB structure and we don't require compatibility with any pre-existing TLS library. Right? Nigel From dan at codesourcery.com Wed Nov 10 00:09:06 2004 From: dan at codesourcery.com (Daniel Jacobowitz) Date: Tue, 9 Nov 2004 19:09:06 -0500 Subject: [mips-tls] Revised versions of MIPS TLS ABI specification In-Reply-To: <419154EC.1050206@mips.com> References: <41814CE8.8060601@codesourcery.com> <20041104170233.GB10079@nevyn.them.org> <418A6E55.9030101@mips.com> <20041104214105.GB8696@nevyn.them.org> <419154EC.1050206@mips.com> Message-ID: <20041110000904.GB4425@nevyn.them.org> On Tue, Nov 09, 2004 at 11:38:20PM +0000, Nigel Stephens wrote: > >No. This is a variant I model, which means that the size and layout of > >the TCB are known; it's in variant II that the TCB is indeterminate. In > >variant I the TLS areas have a positive offset from the TCB, which > >starts with a pointer to the DTV; in variant II they have a negative > >offset from the TCB, which has indeterminate contents. In either case > >the static linker can resolve IE/LE references that bind to the local > >executable. > > > > > > Ah, yes. So variant I works because we are defining the TCB structure > and we don't require compatibility with any pre-existing TLS library. Right? That's correct. We can define the size of the TCB as fixed, so we can place TLS blocks after it. We could even allow application access to the DTV... if there was benefit shown. -- Daniel Jacobowitz