From roland.schwarz at chello.at Sun Oct 8 10:44:31 2006 From: roland.schwarz at chello.at (Roland Schwarz) Date: Sun, 08 Oct 2006 12:44:31 +0200 Subject: Initialization of local static mutex Message-ID: <4528D68F.1070303@chello.at> I am unsure if it is thread safe to use a local static mutex in the following manner: void foo() { static pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER; pthread_mutex_loc(&mutex); ... pthread_mutex_unlock(&mutex); } I am unsure because the standard says: 6.7/4: "The zero-initialization of all local objects with static storage duration is performed before any other initialization takes place. A local object of POD type with static storage duration initialized with constant-expressions is initialized before its block is first entered. ..." Does this mean, the compiler is allowed to emit code to initialize the mutex variable at runtime, after "any other initializations have taken place" and before "block is entered"? If this is true, there would be a race condition for the code that performs this runtime assignment. I am hoping that I misunderstand the standard here, and that the above is indeed thread-safe. Regards, Roland From roland.schwarz at chello.at Sun Oct 8 15:04:01 2006 From: roland.schwarz at chello.at (Roland Schwarz) Date: Sun, 08 Oct 2006 17:04:01 +0200 Subject: [c++-pthreads] Initialization of local static mutex In-Reply-To: <52CA6BC0D0E9EE4E93208DB39A3218FE029DB3BA@EXCHANGE.gp.cv.commvault.com> References: <52CA6BC0D0E9EE4E93208DB39A3218FE029DB3BA@EXCHANGE.gp.cv.commvault.com> Message-ID: <45291361.8020605@chello.at> Your answer does not yet silence my doubts: George Shimanovich wrote: > There will be no race condition between two static initializations and > any static and non-static initialization: > 1) The zero-initialization of static mutex is performed before its > initialization with constant-expression PTHREAD_MUTEX_INITIALIZER Ok, this is said by the first sentence of 6.7/4. > 2) The initialization with that constant-expression occurs before any > non-static initialization in foo Ok, this is covered by the next sentence > 3) Compiler makes sure that each static initialization is done only once The 3) still puzzles me. If the compiler makes this sure, he has to remember this fact somewhere, i.e. in a (global, hidden) state variable. Access to this variable poses a race condition, and since the PTHREAD_MUTEX_INITIALIZER might not be copied in an atomic manner, the assignment itself is not thread safe. Roland From mark at codesourcery.com Sun Oct 8 16:25:16 2006 From: mark at codesourcery.com (Mark Mitchell) Date: Sun, 08 Oct 2006 09:25:16 -0700 Subject: [c++-pthreads] Initialization of local static mutex In-Reply-To: <4528D68F.1070303@chello.at> References: <4528D68F.1070303@chello.at> Message-ID: <4529266C.7090404@codesourcery.com> Roland Schwarz wrote: > I am unsure if it is thread safe to use a local > static mutex in the following manner: > > void foo() > { > static pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER; > pthread_mutex_loc(&mutex); > > ... > > pthread_mutex_unlock(&mutex); > } > > I am unsure because the standard says: > > 6.7/4: > "The zero-initialization of all local objects with static > storage duration is performed before any other initialization > takes place. A local object of POD type with static storage > duration initialized with constant-expressions is initialized > before its block is first entered. ..." PTHREAD_MUTEX_INITIALIZER is supposed to be a constant-expression (like "-1" or "{ 1, 0 }"), so the initialization will happen before foo is called, typically at program start-up. > I am hoping that I misunderstand the standard here, and that > the above is indeed thread-safe. I believe your code is thread-safe in practice for the particular case of PTHREAD_MUTEX_INITIALIZER. -- Mark Mitchell CodeSourcery mark at codesourcery.com (650) 331-3385 x713 From roland.schwarz at chello.at Sun Oct 8 16:55:54 2006 From: roland.schwarz at chello.at (Roland Schwarz) Date: Sun, 08 Oct 2006 18:55:54 +0200 Subject: [c++-pthreads] Initialization of local static mutex In-Reply-To: <4529266C.7090404@codesourcery.com> References: <4528D68F.1070303@chello.at> <4529266C.7090404@codesourcery.com> Message-ID: <45292D9A.3060407@chello.at> Mark Mitchell wrote: > PTHREAD_MUTEX_INITIALIZER is supposed to be a constant-expression (like > "-1" or "{ 1, 0 }"), so the initialization will happen before foo is > called, typically at program start-up. This was not my question, or concern. I am not asking what is typical, but if it is possible that a standard conforming compiler can suffer from such a race condition. > I believe your code is thread-safe in practice for the particular case > of PTHREAD_MUTEX_INITIALIZER. I am still afraid, that behavior is implementation dependent, and would be glad if someone could _prove_ me wrong. (I don't think believe is enough.) Roland From mark at codesourcery.com Sun Oct 8 17:00:49 2006 From: mark at codesourcery.com (Mark Mitchell) Date: Sun, 08 Oct 2006 10:00:49 -0700 Subject: [c++-pthreads] Initialization of local static mutex In-Reply-To: <45292D9A.3060407@chello.at> References: <4528D68F.1070303@chello.at> <4529266C.7090404@codesourcery.com> <45292D9A.3060407@chello.at> Message-ID: <45292EC1.2010504@codesourcery.com> Roland Schwarz wrote: > Mark Mitchell wrote: >> PTHREAD_MUTEX_INITIALIZER is supposed to be a constant-expression >> (like "-1" or "{ 1, 0 }"), so the initialization will happen before >> foo is called, typically at program start-up. > > This was not my question, or concern. I am not asking what is typical, > but if it is possible that a standard conforming compiler can suffer > from such a race condition. A standard-conforming compiler, used with a standard-conforming POSIX library, cannot suffer from a race condition in the code that you posted because PTHREAD_MUTEX_INITIALIZER must be a constant-expression. -- Mark Mitchell CodeSourcery mark at codesourcery.com (650) 331-3385 x713 From roland.schwarz at chello.at Sun Oct 8 17:19:36 2006 From: roland.schwarz at chello.at (Roland Schwarz) Date: Sun, 08 Oct 2006 19:19:36 +0200 Subject: [c++-pthreads] Initialization of local static mutex In-Reply-To: <45292EC1.2010504@codesourcery.com> References: <4528D68F.1070303@chello.at> <4529266C.7090404@codesourcery.com> <45292D9A.3060407@chello.at> <45292EC1.2010504@codesourcery.com> Message-ID: <45293328.5040303@chello.at> Mark Mitchell wrote: > A standard-conforming compiler, used with a standard-conforming POSIX > library, cannot suffer from a race condition in the code that you posted > because PTHREAD_MUTEX_INITIALIZER must be a constant-expression. > Sorry for keep on bugging, but did you read my citation from the standard: 6.7/4: "The zero-initialization of all local objects with static storage duration is performed before any other initialization takes place. A local object of POD type with static storage duration initialized with constant-expressions is initialized before its block is first entered. ..." Does this mean, the compiler is allowed to emit code to initialize the mutex variable at runtime, after "any other initializations have taken place" and before "block is entered"? A compiler is still conforming in this case, while at the same time susceptible to the race. We do not have the zero initialization case here, which would be fine, but the initialization by constant-expression. What am I missing here? Roland From mark at codesourcery.com Sun Oct 8 17:25:43 2006 From: mark at codesourcery.com (Mark Mitchell) Date: Sun, 08 Oct 2006 10:25:43 -0700 Subject: [c++-pthreads] Initialization of local static mutex In-Reply-To: <45293328.5040303@chello.at> References: <4528D68F.1070303@chello.at> <4529266C.7090404@codesourcery.com> <45292D9A.3060407@chello.at> <45292EC1.2010504@codesourcery.com> <45293328.5040303@chello.at> Message-ID: <45293497.2000502@codesourcery.com> Roland Schwarz wrote: > "The zero-initialization of all local objects with static > storage duration is performed before any other initialization > takes place. A local object of POD type with static storage > duration initialized with constant-expressions is initialized > before its block is first entered. ..." Yes, I've read this. (I've spent more than a decade working on C++ compilers, the C++ standard, and C++ ABIs.) > Does this mean, the compiler is allowed to emit code to initialize > the mutex variable at runtime, after "any other initializations > have taken place" and before "block is entered"? Yes. > A compiler is still conforming in this case, while at the same > time susceptible to the race. No. If the initialization occurs before the block is entered, there is no race, independent of when and how that initialization occurs. The race can only occur inside the block. In some sense, the question you're asking has no well-formed answer. The C++ standard doesn't mention threads; therefore, it can't talk about thread-safety. Therefore, if you need to formally validate of your program that it is thread-safe, you're going to need look beyond the C++ stnadard, or even the POSIX standard; you're going to need to define your own model. However, in practice, your code is safe. There are other issues around thread-safe initialization of statics with a constant initializers which are addressed by some C++ ABIs, but you have the simpler case of a constant initializer. -- Mark Mitchell CodeSourcery mark at codesourcery.com (650) 331-3385 x713 From roland.schwarz at chello.at Mon Oct 9 06:15:33 2006 From: roland.schwarz at chello.at (Roland Schwarz) Date: Mon, 09 Oct 2006 08:15:33 +0200 Subject: [c++-pthreads] Initialization of local static mutex In-Reply-To: <45293497.2000502@codesourcery.com> References: <4528D68F.1070303@chello.at> <4529266C.7090404@codesourcery.com> <45292D9A.3060407@chello.at> <45292EC1.2010504@codesourcery.com> <45293328.5040303@chello.at> <45293497.2000502@codesourcery.com> Message-ID: <4529E905.3070001@chello.at> Mark Mitchell wrote: > Yes, I've read this. (I've spent more than a decade working on C++ > compilers, the C++ standard, and C++ ABIs.) Sorry, I did not doubt your reputation, this is why I went here to ask this question. > However, in practice, your code is safe. This is what I was suspecting. But you will admit, that this is a different thing than prove. But if the initialization would have been done as foo() { static int bar; } bar would happen to be zero initialized on any conforming compiler. (before any other initializations take place) So zero initialization is more conservative with respect to the standard. Isn't it? > There are other issues around > thread-safe initialization of statics with a constant initializers which > are addressed by some C++ ABIs, but you have the simpler case of a > constant initializer. Now that you have brought up this issue I am curious which "other issues" you are refering to. Roland From mark at codesourcery.com Mon Oct 9 22:09:51 2006 From: mark at codesourcery.com (Mark Mitchell) Date: Mon, 09 Oct 2006 15:09:51 -0700 Subject: [c++-pthreads] Initialization of local static mutex In-Reply-To: <4529E905.3070001@chello.at> References: <4528D68F.1070303@chello.at> <4529266C.7090404@codesourcery.com> <45292D9A.3060407@chello.at> <45292EC1.2010504@codesourcery.com> <45293328.5040303@chello.at> <45293497.2000502@codesourcery.com> <4529E905.3070001@chello.at> Message-ID: <452AC8AF.6010908@codesourcery.com> Roland Schwarz wrote: >> However, in practice, your code is safe. > > This is what I was suspecting. But you will admit, that this is a > different thing than prove. Yes. For the reasons I stated, I don't think this is a provable thing; the ISO C++ standard doesn't contemplate threads, so it's hard to reason about them. The multi-platform C++ ABI used by G++ (and many other compliers) does contemplate threads, but doesn't specifically address the particular case that you posted. > But if the initialization would have been done as > > foo() > { > static int bar; > } > > bar would happen to be zero initialized on any conforming compiler. > (before any other initializations take place) > So zero initialization is more conservative with respect to the > standard. Isn't it? Yes. It would certainly be nice if PTHREAD_MUTEX_INIT were uniformly zero, but on some hardware that's not feasible. >> There are other issues around thread-safe initialization of statics >> with a constant initializers which are addressed by some C++ ABIs, but >> you have the simpler case of a constant initializer. > > Now that you have brought up this issue I am curious which "other > issues" you are refering to. void f() { static int i = g(); } Here, you cannot do the initialization statically; it must be done the first time that f() is called. So, the key question is whether the initialization is thread-safe. Does the programmer have to change that initialization to be thread-safe, using locks in f()? Or does the compiler take care of it? The multi-platform C++ ABI defines library functions that compilers may use to ensure thread-safe initialization of local statics. However, using these functions is optional. So, some compilers do it, some don't, and some do it only in certain modes. -- Mark Mitchell CodeSourcery mark at codesourcery.com (650) 331-3385 x713 From jm.bourguet at gmail.com Tue Oct 10 06:05:16 2006 From: jm.bourguet at gmail.com (Jean-Marc Bourguet) Date: Tue, 10 Oct 2006 08:05:16 +0200 Subject: [c++-pthreads] Initialization of local static mutex In-Reply-To: <452AC8AF.6010908@codesourcery.com> References: <4528D68F.1070303@chello.at> <4529266C.7090404@codesourcery.com> <45292D9A.3060407@chello.at> <45292EC1.2010504@codesourcery.com> <45293328.5040303@chello.at> <45293497.2000502@codesourcery.com> <4529E905.3070001@chello.at> <452AC8AF.6010908@codesourcery.com> Message-ID: <452B381C.1010104@bourguet.org> Mark Mitchell wrote: > Roland Schwarz wrote: >> Now that you have brought up this issue I am curious which "other >> issues" you are refering to. > > void f() { > static int i = g(); > } > > Here, you cannot do the initialization statically; it must be done the > first time that f() is called. So, the key question is whether the > initialization is thread-safe. Does the programmer have to change that > initialization to be thread-safe, using locks in f()? Or does the > compiler take care of it? I fail to see how the programmer can make the initialization thread-safe if the compiler doesn't take care of it? Lack of memory barriers could make the bool indicating that the initialization is done visible, but not the written value. -- Jean-Marc From mark at codesourcery.com Tue Oct 10 16:49:44 2006 From: mark at codesourcery.com (Mark Mitchell) Date: Tue, 10 Oct 2006 09:49:44 -0700 Subject: [c++-pthreads] Initialization of local static mutex In-Reply-To: <452B381C.1010104@bourguet.org> References: <4528D68F.1070303@chello.at> <4529266C.7090404@codesourcery.com> <45292D9A.3060407@chello.at> <45292EC1.2010504@codesourcery.com> <45293328.5040303@chello.at> <45293497.2000502@codesourcery.com> <4529E905.3070001@chello.at> <452AC8AF.6010908@codesourcery.com> <452B381C.1010104@bourguet.org> Message-ID: <452BCF28.3080101@codesourcery.com> Jean-Marc Bourguet wrote: > Mark Mitchell wrote: >> Roland Schwarz wrote: > >>> Now that you have brought up this issue I am curious which "other >>> issues" you are refering to. >> >> void f() { >> static int i = g(); >> } >> >> Here, you cannot do the initialization statically; it must be done the >> first time that f() is called. So, the key question is whether the >> initialization is thread-safe. Does the programmer have to change >> that initialization to be thread-safe, using locks in f()? Or does >> the compiler take care of it? > > I fail to see how the programmer can make the initialization thread-safe > if the compiler doesn't take care of it? Lack of memory barriers could > make the bool indicating that the initialization is done visible, but > not the written value. The programmer can't take care of it easily. If the compiler doesn't do it for you, have to do things like: static int i; static bool initialized; lock_mutex(); if (!initialized) { i = g(); initialized = true; } unlock_mutex(); i.e., you have to manage the initialization manually. And, yes, there are lots of problems with that, including exceptions being thrown by g(). So, it's a good feature for compilers to have. Of course, since it makes initialization slower (and code bigger) in the single-threaded case, some programmers want to be able to turn off the compiler support. -- Mark Mitchell CodeSourcery mark at codesourcery.com (650) 331-3385 x713