From gdr at integrable-solutions.net Thu Jan 1 00:48:19 2004 From: gdr at integrable-solutions.net (Gabriel Dos Reis) Date: 01 Jan 2004 01:48:19 +0100 Subject: [c++-pthreads] Re: C++ and posix threads In-Reply-To: <20031231220611.GI30780@tofu.dreamhost.com> References: <1072131497.7562.22.camel@doubledemon.codesourcery.com> <20031223160314.GA9780@diablo.name> <20031224044622.GG27836@tofu.dreamhost.com> <20031225001117.GA13447@redhat.com> <20031227232444.GA14692@redhat.com> <20031228045406.GI2774@tofu.dreamhost.com> <20031229232817.GA21305@redhat.com> <20031230015808.GM2774@tofu.dreamhost.com> <20031230050755.GB21387@redhat.com> <20031231220611.GI30780@tofu.dreamhost.com> Message-ID: Nathan Myers writes: | Richard, please suspend judgment and read carefully. | | On Mon, Dec 29, 2003 at 09:07:55PM -0800, Richard Henderson wrote: | > On Mon, Dec 29, 2003 at 05:58:08PM -0800, Nathan Myers wrote: | > > | > > Richard, all these "ZERO uptake" remarks make it very hard to discuss | > > design decisions calmly. [...] | > | > If you think I mean just Uli, then you're wrong. I mean any vendor. | | Have you actually asked all all these vendors? I suspect you'd find | that most would have preferred the option to offer _only_ error-return | semantics, and not be obliged to implement unwinding at all. On one hand, what it is important here is what is of interest to users. The idea of checking return value for error codes might work in small, but it does not work properly in larger systems/libraries. I do not believe we should be conducting this discussion by focusing on specific individual and I can't see anything from RTH's previous message leading to believe that he was meaning Uli. On the other hand, the idea of having printf (or assimilated) work differently depending on whether it is part of C or C++ program is a good recipe for confusion and disaster. We have something to learn from past experience, messing with C functions. -- Gaby From TEREKHOV at de.ibm.com Fri Jan 2 12:30:12 2004 From: TEREKHOV at de.ibm.com (Alexander Terekhov) Date: Fri, 2 Jan 2004 13:30:12 +0100 Subject: [c++-pthreads] Re: C++ and posix threads In-Reply-To: <20031231220611.GI30780@tofu.dreamhost.com> Message-ID: Nathan Myers wrote: [...] > Nothing in the goal of supporting thread cancellation seems to > require violating ISO-standard language semantics. Exception based thread cancellation comes from the DCE (CMA aside for a moment) which *by far* predates ISO C++. http://archive.adaic.com/tools/bindings/DCE/THREADS/pthread_exc.h regards, alexander. From alexvn at connect.to Fri Jan 2 12:14:08 2004 From: alexvn at connect.to (Alex Vinokur) Date: Fri, 2 Jan 2004 14:14:08 +0200 Subject: pthreads, printf and cout Message-ID: Hi, Behavior of printf and cout in a program below is different : cout prints nothing. What is wrong? =================================== Windows 2000 Professional CYGWIN_NT-5.0 1.5.4(0.94/3/2) gcc version 3.3.1 (cygming special) =================================== ====== C++ code : foo.cpp : BEGIN ====== #include #include #include #include using namespace std; extern "C" void* Run(void *) { printf ("printf : Run Start\n"); cout << "cout : Run Start" << endl; sleep (3); printf ("printf : Run Finish\n"); cout << "cout : Run Finish" << endl; } int main () { pthread_t t1; pthread_create(&t1, NULL, Run, NULL); int errNum = pthread_join(t1, NULL); printf ("printf : errNum = %d\n", errNum); cout << "cout : errNum = " << errNum << endl; } ====== C++ code : foo.cpp : END ======== ====== Compilation & Run : BEGIN ====== $ g++ foo.cpp $ a printf : Run Start printf : Run Finish printf : errNum = 0 // cout prints nothing ====== Compilation & Run : END ======== -- ===================================== Alex Vinokur mailto:alexvn at connect.to http://mathforum.org/library/view/10978.html news://news.gmane.org/gmane.comp.lang.c++.perfometer ===================================== From David.Butenhof at hp.com Mon Jan 5 16:57:32 2004 From: David.Butenhof at hp.com (Dave Butenhof) Date: Mon, 05 Jan 2004 11:57:32 -0500 Subject: [c++-pthreads] concrete library-code example In-Reply-To: <20031224170517.GG30780@tofu.dreamhost.com> References: <3FE70542.5060104@bourguet.org> <1072111706.3474.17.camel@minax.codesourcery.com> <20031222215712.GB1528@diablo.name> <1072131497.7562.22.camel@doubledemon.codesourcery.com> <20031223160314.GA9780@diablo.name> <20031224170517.GG30780@tofu.dreamhost.com> Message-ID: <3FF9977C.7040003@hp.com> Nathan Myers wrote: >On Wed, Dec 24, 2003 at 08:09:57AM -0500, Jason Merrill wrote: > > >>On Tue, 23 Dec 2003 11:03:14 -0500, Ted Baker wrote: >> >> >>>How do you propose to modify read() to throw an exception and >>>still have backwards compatability with applications that expect >>>read() to always return (more specifically, to return -1 if it >>>fails)? >>> >>> >>read() already doesn't return if it's acting on a cancellation request. >>Throwing an exception is just a different way of not returning. >> >> First, let me say, "ouch, my brain hurts". I've been on vacation for 2 weeks, and this mailing list had just started when I left, and the traffic during that time is a bit overwhelming! I haven't finished yet, and I don't expect to today. Nevertheless, I'm going to offer some comments while I'm "inspired". I'll also comment (with some degree of surprise and perhaps even consternation) that so far I've seen nothing to suggest that an eventual more detailed response will differ in any great detail from the comments made by Alexander Terekhov. But then, as I said, I haven't finished catching up yet. ;-) >Enlarging on this question... > >Here is a more-or-less concrete example, for discussion purposes. >It's meant as a generic example of code written according to the >existing contract offered by C libraries. > > Correction: "... offered by C libraries that support POSIX 1003.1b-1993 or earlier." > int affect_world(struct state* s) > { > int result; > violate_invariants_or_claim_resources(s); > result = c_function_or_system_call(s->member); > if (result < 0) { > clean_up(s, result); > return result; > } > act_on_result(s, result); > restore_invariants_and_release_resources(s); > return 0; > } > >This pattern is extremely common in both C and C++ libraries. If >read() were to throw (or to "just ... not return"), the program state >would be corrupted. A redefinition of c_function_or_system_call >semantics that breaks this code breaks many thousands of existing >thread-safe C and C++ libraries. > > If this code exists in a pure ANSI C/POSIX application using threads, and if the thread running this code can be cancelled, then the implementation of this function is broken because IT (not the implementation, nor the cancellation) corrupts program state. While I'm not at all trying to argue that the issue is at all as simple as this, that's the facts all the same. Depending on propagation of error statuses is a really bad way to implement cancellation. At least, given the primitive and limited concept of ANSI/POSIX error codes. Too much code ignores statuses in the first place, which is bad enough. But, worse, there are many legitimate reasons for library code to CONVERT return status values; e.g., I called read() and it returned some error but MY function only implicitly involves a read() and it simply wouldn't be useful or meaningful to return that error to my caller. Instead, I want to indicate that my function (say, synchronizing a database) failed, and so any (or at least most) failures of my "support calls" will result in my returning 'unable to synchronize database' (which often isn't an ANSI/POSIX error number in the first place, but even if it is, it's unlikely to be the value returned by read). The ECANCELLED some have proposed would be lost, and that's unacceptable. This is why we settled on exceptions to represent cancellation. And because POSIX and ANSI C don't have exceptions, we devised the simple "cleanup handler" mechanism that allowed a clean and transparent implementation on top of exceptions, or a "hack" implementation private to the thread library where exceptions weren't available. >(The cancellation model described in > http://www.codesourcery.com/archives/c++-pthreads/msg00021.html >is designed to preserve libraries that contain code that follows >this pattern.) > >Jason, do you not consider those libraries worth preserving? > > If you're talking about a currently non-threaded library to which you'd like to transparently add thread support; well, I doubt that's possible, and this particular proposal isn't going to help. When they're redesigned and recoded to be thread-safe, they can also be made cancel-safe. If you're talking about adding cancel support transparently to an existing C++ library, I doubt this is sufficient unless there's some standard requirement that all C++ libraries must pass through the system failure code to the caller. (There isn't, can't be, and shouldn't be.) And it also presupposes that the C++ library isn't exception-safe; because if it is, then delivering cancellation as an exception would seem "obviously" to be the most compatible and complete solution. And I'm deliberately discounting the mention I've seen several times in this list of "thread-safe" libraries that aren't "cancel-safe". Such libraries are simply broken, from basic design. Cancellation is a basic and important part of the POSIX thread model, and if you're not safe you're not safe. The only viable exclusion (there, I avoided using the word "exception", though it took me a few moments of thought) is if you can be guaranteed to be running only in threads that can never be cancelled... and in that case the whole issue is irrelevant! -- /--------------------[ David.Butenhof at hp.com ]--------------------\ | Hewlett-Packard Company Tru64 UNIX & VMS Thread Architect | | My book: http://www.awl.com/cseng/titles/0-201-63392-2/ | \----[ http://homepage.mac.com/dbutenhof/Threads/Threads.html ]---/ From David.Butenhof at hp.com Mon Jan 5 17:39:09 2004 From: David.Butenhof at hp.com (Dave Butenhof) Date: Mon, 05 Jan 2004 12:39:09 -0500 Subject: [c++-pthreads] Re: cancellation points report failure In-Reply-To: <3FF18485.93F2D588@terekhov.de> References:

<3FF18485.93F2D588@terekhov.de> Message-ID: <3FF9A13D.9030105@hp.com> Alexander Terekhov wrote: >David Abrahams wrote: >[...] > > >>Of course it's not safe. That's my point, sort of: if you use >>synchronous cancellation, you have to give up on any *guarantee* that >>the thread will be cancelled, so we shouldn't be considering measures >>that take heroic steps to try to ensure it. If you want to guarantee >>that cancellation happens, you have to do something unsafe. >> >> >Note that the use of POSIX asynchronous cancellation (I mean the >presence of async-cancel{-safe} regions on the execution path) does >NOT guarantee thread termination (cancel request delivery) at all. >Conforming implementations are free to ignore it completely, so to >speak. I wish the standard would define pthread_testcancel() "in >terms" of an empty async-cancel region (and it would also provide >async-cancel-safety for pthread_testcancel() itself): > >void pthread_testcancel() { /* mandatory shall occur semantics */ > int oldtype; > pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, &oldtype); > pthread_setcanceltype(oldtype, &oldtype); >} > > This is all way off topic, but what's your point here, Alexander? Are you intending to suggest that pthread_setcanceltype() be made a guaranteed cancellation point, so your substitution would have the same effect as the current pthread_testcancel()... or that the definition of pthread_testcancel() (using the current semantics of pthread_setcanceltype()) should be allowed to ignore a pending cancel? ;-) -- /--------------------[ David.Butenhof at hp.com ]--------------------\ | Hewlett-Packard Company Tru64 UNIX & VMS Thread Architect | | My book: http://www.awl.com/cseng/titles/0-201-63392-2/ | \----[ http://homepage.mac.com/dbutenhof/Threads/Threads.html ]---/ From boo at terekhov.de Mon Jan 5 18:53:33 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Mon, 05 Jan 2004 19:53:33 +0100 Subject: [c++-pthreads] Re: cancellation points report failure References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> Message-ID: <3FF9B2AD.D7E6C4DD@terekhov.de> Dave Butenhof wrote: [...] > >Note that the use of POSIX asynchronous cancellation (I mean the > >presence of async-cancel{-safe} regions on the execution path) does > >NOT guarantee thread termination (cancel request delivery) at all. > >Conforming implementations are free to ignore it completely, so to > >speak. I wish the standard would define pthread_testcancel() "in > >terms" of an empty async-cancel region (and it would also provide > >async-cancel-safety for pthread_testcancel() itself): > > > >void pthread_testcancel() { /* mandatory shall occur semantics */ > > int oldtype; > > pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, &oldtype); > > pthread_setcanceltype(oldtype, &oldtype); > >} > > > > > This is all way off topic, but what's your point here, Alexander? > > Are you intending to suggest that pthread_setcanceltype() be made a > guaranteed cancellation point, ... I'm (once again) intending to suggest that "an act" of transition from synchronous to asynchronous cancellation be made a guaranteed cancellation point so that "passing through" any async-cancel-safe region with cancel request pending on its entry (or by the time of pthread_testcancel() invocation inside async-cancel-safe region [I also want pthread_testcancel() be added to the list of async.cancel safe functions]) shall result in raising std::thread_cancel_request exception (subject to PTHREAD_CANCEL_ENABLE cancellation state, of course). Well, to you, this is/was just a matter of control (apart from your async-cancel "phobia"***, so to say): google.com/groups?threadm=ki1M8.3%24VD1.187532%40news.cpqcorp.net (Subject: Re: cancelling one thread from inside another one) I disagree. regards, alexander. ***) "Personally, if I were to spend any serious time considering the future of async cancelability in the standard, I'd rather argue for removing it entirely." (quote taken from the other message of yours in the same thread referenced above) From boo at terekhov.de Mon Jan 5 19:11:23 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Mon, 05 Jan 2004 20:11:23 +0100 Subject: [c++-pthreads] concrete library-code example References: <3FE70542.5060104@bourguet.org> <1072111706.3474.17.camel@minax.codesourcery.com> <20031222215712.GB1528@diablo.name> <1072131497.7562.22.camel@doubledemon.codesourcery.com> <20031223160314.GA9780@diablo.name> <20031224170517.GG30780@tofu.dreamhost.com> <3FF9977C.7040003@hp.com> Message-ID: <3FF9B6DB.1C0EA581@terekhov.de> Dave Butenhof wrote: [...] > returned by read). The ECANCELLED some have proposed would be lost, and > that's unacceptable. This is why we settled on exceptions to represent > cancellation. And asynchronous cancelability was another reason, I guess. Oder? ;-) regards, alexander. From David.Butenhof at hp.com Tue Jan 6 11:58:57 2004 From: David.Butenhof at hp.com (Dave Butenhof) Date: Tue, 06 Jan 2004 06:58:57 -0500 Subject: [c++-pthreads] Re: cancellation points report failure In-Reply-To: <3FF9B2AD.D7E6C4DD@terekhov.de> References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> Message-ID: <3FFAA301.6080301@hp.com> Alexander Terekhov wrote: >Dave Butenhof wrote: >[...] > > >>Are you intending to suggest that pthread_setcanceltype() be made a >>guaranteed cancellation point, ... >> >> >I'm (once again) intending to suggest that "an act" of transition >from synchronous to asynchronous cancellation be made a guaranteed >cancellation point so that "passing through" any async-cancel-safe >region with cancel request pending on its entry (or by the time of >pthread_testcancel() invocation inside async-cancel-safe region [I >also want pthread_testcancel() be added to the list of async.cancel >safe functions]) shall result in raising std::thread_cancel_request >exception (subject to PTHREAD_CANCEL_ENABLE cancellation state, of >course). > Very simply, Alexander, your proposal COMBINES two separate primitives: disabling async cancel, and testing for a pending cancel. Keeping those primitives separate allows applications to choose the behavior appropriate to their needs. Combining them removes that flexibility. We could have created two variants of "disable async cancel", one that tests for pending cancel and one that doesn't, but there'd be no real advantage over disabling and then testing separately. >Well, to you, this is/was just a matter of control (apart >from your async-cancel "phobia"***, so to say): > >google.com/groups?threadm=ki1M8.3%24VD1.187532%40news.cpqcorp.net >(Subject: Re: cancelling one thread from inside another one) > > Since a "phobia" is an "abnormal or illogical fear", I'd have to say you're wrong on two counts. First, I don't fear it; I merely argue that it's rarely usable and far too easy to misuse, and that the cost of the feature outweighs its value. And none of that is "illogical". Furthermore, since some of the C++ people here seem to have enough trouble with the idea of cancel as an exception in the first place, (one of those rare goals in which you and I are united), nevermind introducing the concept of an ASYNCHRONOUS exception, let's just leave this alone, eh? Personally, I would be happy to accept a C++ binding with no way to enable asynchronous cancelability, and to avoid defining any C++ code as "async cancel-safe". -- /--------------------[ David.Butenhof at hp.com ]--------------------\ | Hewlett-Packard Company Tru64 UNIX & VMS Thread Architect | | My book: http://www.awl.com/cseng/titles/0-201-63392-2/ | \----[ http://homepage.mac.com/dbutenhof/Threads/Threads.html ]---/ From dave at boost-consulting.com Tue Jan 6 13:51:19 2004 From: dave at boost-consulting.com (David Abrahams) Date: Tue, 06 Jan 2004 08:51:19 -0500 Subject: cancellation points report failure References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> Message-ID: Dave Butenhof writes: > Since a "phobia" is an "abnormal or illogical fear", I'd have to say > you're wrong on two counts. First, I don't fear it; I merely argue > that it's rarely usable and far too easy to misuse, and that the cost > of the feature outweighs its value. And none of that is > "illogical". Furthermore, since some of the C++ people here seem to > have enough trouble with the idea of cancel as an exception in the > first place, (one of those rare goals in which you and I are united), > nevermind introducing the concept of an ASYNCHRONOUS exception, let's > just leave this alone, eh? I, at least, have no trouble with the idea of using C++ exceptions to report cancellation. In fact, I think it's a great idea in general. It's just a really bad idea for functions which can normally be expected never to throw exceptions to start throwing cancellations when used in a POSIX threads environment. IIUC, other "C++ people" who don't like the idea of printf throwing an exception feel the same way. -- Dave Abrahams Boost Consulting www.boost-consulting.com From dave at boost-consulting.com Tue Jan 6 14:03:09 2004 From: dave at boost-consulting.com (David Abrahams) Date: Tue, 06 Jan 2004 09:03:09 -0500 Subject: concrete library-code example References: <3FE70542.5060104@bourguet.org> <1072111706.3474.17.camel@minax.codesourcery.com> <20031222215712.GB1528@diablo.name> <1072131497.7562.22.camel@doubledemon.codesourcery.com> <20031223160314.GA9780@diablo.name> <20031224170517.GG30780@tofu.dreamhost.com> <3FF9977C.7040003@hp.com> Message-ID: Dave Butenhof writes: >>Here is a more-or-less concrete example, for discussion purposes. >> It's meant as a generic example of code written according to the >> existing contract offered by C libraries. >> >> > Correction: "... offered by C libraries that support POSIX > 1003.1b-1993 or earlier." I think Nathan meant what he said. You seem to be viewing this in terms of some POSIX standard, but I'm pretty sure even most good C programmers write code without giving POSIX a second thought... even thread-safe code. >> int affect_world(struct state* s) >> { >> int result; >> violate_invariants_or_claim_resources(s); >> result = c_function_or_system_call(s->member); >> if (result < 0) { >> clean_up(s, result); >> return result; >> } >> act_on_result(s, result); >> restore_invariants_and_release_resources(s); >> return 0; >> } >> >> This pattern is extremely common in both C and C++ libraries. If >> read() were to throw (or to "just ... not return"), the program >> state would be corrupted. A redefinition of >> c_function_or_system_call semantics that breaks this code breaks >> many thousands of existing thread-safe C and C++ libraries. >> >> > If this code exists in a pure ANSI C/POSIX application using threads, > and if the thread running this code can be cancelled, then the > implementation of this function is broken because IT (not the > implementation, nor the cancellation) corrupts program state. Again, you're missing our point. We'd like to keep working (non-POSIX) thread-safe code working when used in a POSIX environment, without getting into a detailed review of every line of code. Adding exception-safety to an existing codebase is a nontrivial exercise, and that's essentially what the current binding requires of people who want to use code not written with POSIX in mind. -- Dave Abrahams Boost Consulting www.boost-consulting.com From dave at boost-consulting.com Tue Jan 6 14:04:54 2004 From: dave at boost-consulting.com (David Abrahams) Date: Tue, 06 Jan 2004 09:04:54 -0500 Subject: concrete library-code example References: <3FE70542.5060104@bourguet.org> <1072111706.3474.17.camel@minax.codesourcery.com> <20031222215712.GB1528@diablo.name> <1072131497.7562.22.camel@doubledemon.codesourcery.com> <20031223160314.GA9780@diablo.name> <20031224170517.GG30780@tofu.dreamhost.com> <3FF9977C.7040003@hp.com> Message-ID: Dave Butenhof writes: > And I'm deliberately discounting the mention I've seen several times > in this list of "thread-safe" libraries that aren't > "cancel-safe". Such libraries are simply broken, from basic > design. Cancellation is a basic and important part of the POSIX thread > model, and if you're not safe you're not safe. Do any other threading models count? -- Dave Abrahams Boost Consulting www.boost-consulting.com From David.Butenhof at hp.com Tue Jan 6 14:24:22 2004 From: David.Butenhof at hp.com (Dave Butenhof) Date: Tue, 06 Jan 2004 09:24:22 -0500 Subject: [c++-pthreads] Re: cancellation points report failure In-Reply-To: References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> Message-ID: <3FFAC516.2010803@hp.com> David Abrahams wrote: >Dave Butenhof writes: > > >>Since a "phobia" is an "abnormal or illogical fear", I'd have to say >>you're wrong on two counts. First, I don't fear it; I merely argue >>that it's rarely usable and far too easy to misuse, and that the cost >>of the feature outweighs its value. And none of that is >>"illogical". Furthermore, since some of the C++ people here seem to >>have enough trouble with the idea of cancel as an exception in the >>first place, (one of those rare goals in which you and I are united), >>nevermind introducing the concept of an ASYNCHRONOUS exception, let's >>just leave this alone, eh? >> >> >I, at least, have no trouble with the idea of using C++ exceptions to >report cancellation. In fact, I think it's a great idea in general. >It's just a really bad idea for functions which can normally be >expected never to throw exceptions to start throwing cancellations >when used in a POSIX threads environment. > >IIUC, other "C++ people" who don't like the idea of printf throwing >an exception feel the same way. > > Although I wasn't counting you in particular as an objector, I may have overgeneralized the objections of others. The whole matter of interaction with throw specs in particular is complicated, and I certainly don't claim to have any answers -- easy or otherwise. I've considered making a couple of rambling conceptual statements, but I'm way out of my league here on C++ philosophy and history, and I wouldn't want anyone to think I'm doing more than idle speculation. One of those idle thoughts was that the concept of "cancel state" (enable or disable) might even be meaningless (or at least irrelevant) for a pure C++ binding. This might simply be a dynamic property of the current call tree; depending on the nested throw specs. That is, if the cancel exception can be thrown, cancel is "enabled"; and if it cannot then cancel is "disabled". This might well make it too hard to ever allow cancellation, since "most" throw specs would probably need to include cancellation. (That it might also make everything without a throw spec cancellable I discount as irrelevant: that's only an issue if the code runs in a thread that can be cancelled, in which case the code is so far out of any supported boundaries now that I don't care. This of course doesn't disallow or invalidate the possibility that YOU care. ;-) ) That also doesn't help in dealing with "printf throwing an exception" in the most generic sense, if that's taken as a placeholder for "library routines that don't currently raise exceptions but ought to be cancellation points, including read and write. In particular, though, POSIX designates printf as an OPTIONAL cancellation point. Most current implementations do NOT support cancellation of printf because it'd mean putting handlers into the C runtime to clean up stdio state. (On Tru64 UNIX, for example, handling exceptions requires linking libexc, on which libc has never depended, so adding cancellation support to libc would be "a big deal".) If printf isn't a cancellation point, then it shouldn't normally be expected to raise the cancel exception. ;-) -- /--------------------[ David.Butenhof at hp.com ]--------------------\ | Hewlett-Packard Company Tru64 UNIX & VMS Thread Architect | | My book: http://www.awl.com/cseng/titles/0-201-63392-2/ | \----[ http://homepage.mac.com/dbutenhof/Threads/Threads.html ]---/ From David.Butenhof at hp.com Tue Jan 6 14:31:45 2004 From: David.Butenhof at hp.com (Dave Butenhof) Date: Tue, 06 Jan 2004 09:31:45 -0500 Subject: [c++-pthreads] Re: concrete library-code example In-Reply-To: References: <3FE70542.5060104@bourguet.org> <1072111706.3474.17.camel@minax.codesourcery.com> <20031222215712.GB1528@diablo.name> <1072131497.7562.22.camel@doubledemon.codesourcery.com> <20031223160314.GA9780@diablo.name> <20031224170517.GG30780@tofu.dreamhost.com> <3FF9977C.7040003@hp.com> Message-ID: <3FFAC6D1.2090005@hp.com> David Abrahams wrote: >Dave Butenhof writes: > > >>And I'm deliberately discounting the mention I've seen several times >>in this list of "thread-safe" libraries that aren't >>"cancel-safe". Such libraries are simply broken, from basic >>design. Cancellation is a basic and important part of the POSIX thread >>model, and if you're not safe you're not safe. >> >> >Do any other threading models count? > > On a discussion list titled explicitly "c++-pthreads", I automatically presumed not, without really giving it much thought. But, as I said, I was completely overwhelmed with the flood of messages when I returned from vacation yesterday, and I SHOULD have recalled previous discussions on this subject that pointed out you really didn't intend to target "pthreads" specifically. (I.e., this mailing list is inappropriately named.) You're basically talking about Win32 or UI threads (which essentially is legacy Solaris code), being converted trivially to "native C++ threading" environment. Is that a realistic limitation on your model? If you say so, hey, I'm not in a position to argue about your goals. -- /--------------------[ David.Butenhof at hp.com ]--------------------\ | Hewlett-Packard Company Tru64 UNIX & VMS Thread Architect | | My book: http://www.awl.com/cseng/titles/0-201-63392-2/ | \----[ http://homepage.mac.com/dbutenhof/Threads/Threads.html ]---/ From dave at boost-consulting.com Tue Jan 6 18:37:24 2004 From: dave at boost-consulting.com (David Abrahams) Date: Tue, 06 Jan 2004 13:37:24 -0500 Subject: concrete library-code example References: <3FE70542.5060104@bourguet.org> <1072111706.3474.17.camel@minax.codesourcery.com> <20031222215712.GB1528@diablo.name> <1072131497.7562.22.camel@doubledemon.codesourcery.com> <20031223160314.GA9780@diablo.name> <20031224170517.GG30780@tofu.dreamhost.com> <3FF9977C.7040003@hp.com> <3FFAC6D1.2090005@hp.com> Message-ID: Dave Butenhof writes: >>Do any other threading models count? >> >> > On a discussion list titled explicitly "c++-pthreads", I automatically > presumed not, without really giving it much thought. > > But, as I said, I was completely overwhelmed with the flood of > messages when I returned from vacation yesterday, and I SHOULD have > recalled previous discussions on this subject that pointed out you > really didn't intend to target "pthreads" specifically. (I.e., this > mailing list is inappropriately named.) > > You're basically talking about Win32 or UI threads (which essentially > is legacy Solaris code) Aren't there also various "lightweight thread" models used in embedded systems? I think non-threaded library code which doesn't use globals alos counts. > being converted trivially to "native C++ threading" environment. Well, not neccessarily trivially, but the idea is that it should be tractable to do it. > Is that a realistic limitation on your model? If you say so, hey, > I'm not in a position to argue about your goals. I'm not really in a position to say how important or realistic the scenario is. It does seem reasonable to want to pick up a C/C++ library that's written with the general priniciples of thread-safety in mind but without specific knowledge of POSIX, and use it in some other code that's launched in a cancellable POSIX thread. -- Dave Abrahams Boost Consulting www.boost-consulting.com From dave at boost-consulting.com Tue Jan 6 18:41:25 2004 From: dave at boost-consulting.com (David Abrahams) Date: Tue, 06 Jan 2004 13:41:25 -0500 Subject: cancellation points report failure References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <3FFAC516.2010803@hp.com> Message-ID: Dave Butenhof writes: > David Abrahams wrote: > >>Dave Butenhof writes: >> >> >>>Since a "phobia" is an "abnormal or illogical fear", I'd have to say >>>you're wrong on two counts. First, I don't fear it; I merely argue >>>that it's rarely usable and far too easy to misuse, and that the cost >>>of the feature outweighs its value. And none of that is >>>"illogical". Furthermore, since some of the C++ people here seem to >>>have enough trouble with the idea of cancel as an exception in the >>>first place, (one of those rare goals in which you and I are united), >>>nevermind introducing the concept of an ASYNCHRONOUS exception, let's >>>just leave this alone, eh? >>> >>> >>I, at least, have no trouble with the idea of using C++ exceptions to >>report cancellation. In fact, I think it's a great idea in general. >>It's just a really bad idea for functions which can normally be >>expected never to throw exceptions to start throwing cancellations >>when used in a POSIX threads environment. >> >>IIUC, other "C++ people" who don't like the idea of printf throwing >>an exception feel the same way. >> >> > Although I wasn't counting you in particular as an objector, I may > have overgeneralized the objections of others. The whole matter of > interaction with throw specs in particular is complicated, and I > certainly don't claim to have any answers -- easy or otherwise. I've > considered making a couple of rambling conceptual statements, but I'm > way out of my league here on C++ philosophy and history, and I > wouldn't want anyone to think I'm doing more than idle speculation. > > One of those idle thoughts was that the concept of "cancel state" > (enable or disable) might even be meaningless (or at least > irrelevant) for a pure C++ binding. This might simply be a dynamic > property of the current call tree; depending on the nested throw > specs. That is, if the cancel exception can be thrown cancel is > "enabled"; and if it cannot then cancel is "disabled". What do you mean by "can be", and what does it have to do with exception-specifications? Are you suggesting that cancellation is disabled by writing an exception-specification that doesn't include it? Interesting thought, if so, but how do you disable just cancellation without masking other exceptions? -- Dave Abrahams Boost Consulting www.boost-consulting.com From jason at redhat.com Tue Jan 6 18:59:20 2004 From: jason at redhat.com (Jason Merrill) Date: Tue, 06 Jan 2004 13:59:20 -0500 Subject: [c++-pthreads] Re: cancellation points report failure In-Reply-To: (David Abrahams's message of "Wed, 24 Dec 2003 12:38:46 -0500") References: <2A8DB02E3018D411901B009027FD3A3F01CAAF6A@mchp905a.mch.sbs.de> <2E98D8A4-3251-11D8-B7E5-00039390D9E0@apple.com> <20031219194503.GA31795@tofu.dreamhost.com> <20031219205746.GA12322@diablo.name> <20031219225650.GA13735@diablo.name> <20031220045618.GA27836@tofu.dreamhost.com> <1071946483.25325.69.camel@doubledemon.codesourcery.com>

Message-ID: On Wed, 24 Dec 2003 12:38:46 -0500, David Abrahams wrote: > Jason Merrill writes: > >> * C++ I/O functions can throw a cancellation exception. >> * C++ catch blocks work normally. >> * If a cancellation exception is destroyed, the cancellation request >> is re-entered, and acted on again at the next cancellation point. > > I appreciate that your motivation is to ensure that synchronous > cancellation requests aren't ignored, but that doesn't really seem to > accomplish the goal. After all, a thread never has to reach a > cancellation point, and even if it does, it can keep catching and > discarding the exceptions indefinitely. That's fine with me. I'm not trying to assure that cancellation always succeeds; as you say, a thread never has to reach a cancellation point. My goal is to make sure that the cancellation request is never lost, that it remains either active or pending until the thread exits. Jason From austern at apple.com Tue Jan 6 19:33:47 2004 From: austern at apple.com (Matt Austern) Date: Tue, 6 Jan 2004 11:33:47 -0800 Subject: What are the real issues? In-Reply-To: <3FFAC516.2010803@hp.com> References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <3FFAC516.2010803@hp.com> Message-ID: <3F190B5E-407F-11D8-B918-00039390D9E0@apple.com> I'm just trying to trace back why people see C++ thread cancellation as a problem. First: the usual assumption has been that thread cancellation is most naturally represented in C++ as an exception. That's because the POSIX model for thread cancellation is that if you cancel a particular thread, then, the next time the thread reaches one of a set of well-defined cancellation points, it'll perform some cleanup and then stop. In C the cleanup is via handlers that you register. We could have that model in C++ as well, but most C++ programmers would think it's very strange that you would stop execution at some point without cleaning up the stack. And once you're talking about stack cleanup, i.e. invoking destructors, you're talking about something that's close enough to an exception that it makes no difference. The Itanium C++ ABI, which gcc adopted, made cancellation a special kind of exception, "forced unwinding", so that a thread can't just catch the cancellation exception and swallow it. The idea was that a thread may disable cancellation tempoarily, but that once a cancellation has begun it can't be thrown away. This was a slightly controversial decision at the time, and maybe it's at the heart of the real issue. Second: the immediate issue was that some people saw a contradiction between this model and the C++ library specification. For example, POSIX says that read() is a cancellation point. But it's reasonable to assume that std::istream:;read() invokes POSIX's read() system call, and the C++ standard says that (under ordinary circumstances) std::istream doesn't throw. If the cancellation exception were an ordinary exception instead of special forced unwinding, what would happen would be that istream would invoke its streambuf, the streambuf would call read(), read would throw a cancellation exception, then istream would catch the exception, swallow it, and set and error flag. Forced unwinding means that the cancellation exception will propagate even though istream tries to swallow it. This means that an exception will propagate from istream when the C++ standard says it's not supposed to. Third, some people simply object to having cancellation be an exception (or anything that looks like an exception): it means there are some functions that can throw exceptions in the presence of threads that wouldn't throw in a single-threaded program. It's actually pretty hard for me to imagine any model for thread cancellation that's very different from the POSIX. Anything that doesn't involve thread execution stopping sounds more like a communication mechanism than a cancellation mechanism. Seems to me that the only real issues for debate are which functions are cancellation points, how a thread can enable and disable cancellation, whether a thread should be allowed to disregard a cancellation request once it has been received, and what kind of cleanup a thread performs before it stops. My feeling: it's just plain inevitable that a multithreaded program has more functions that might throw than a single-thread program. Dealing with this is part of what it means to make a program thread-safe. We might argue that POSIX defines too many cancellation points, and that cutting it down to a smaller number (and, especially, getting rid of the functions that POSIX says may or may not be cancellation points) would make it easier to write correct code. But as I said, I think the really fundamental issue is whether a thread should be allowed to receive a cancellation request, start to do some work as a result of the request, and then decide that it doesn't want to be cancelled. If we think that's reasonable then I think what we should probably do is abandon the notion of forced unwinding and make cancellation into an ordinary exception. If we do that then the std::istream contradiction goes away. The cost, of course, is that it might become surprisingly hard to cancel some threads. --Matt From ncm at cantrip.org Tue Jan 6 21:12:36 2004 From: ncm at cantrip.org (Nathan Myers) Date: Tue, 6 Jan 2004 13:12:36 -0800 Subject: [c++-pthreads] What are the real issues? In-Reply-To: <3F190B5E-407F-11D8-B918-00039390D9E0@apple.com> References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <3FFAC516.2010803@hp.com> <3F190B5E-407F-11D8-B918-00039390D9E0@apple.com> Message-ID: <20040106211236.GX30780@tofu.dreamhost.com> I object to Matt's characterization of the issues on several levels. It misstates expressed concerns, omits others, and assumes its own conclusions. I hope to post in more detail, but I would like to address the most glaring problems immediately. On Tue, Jan 06, 2004 at 11:33:47AM -0800, Matt Austern wrote: > It's actually pretty hard for me to imagine any model for thread > cancellation that's very different from the POSIX. ... I presume you mean "different from the POSIX C binding". One such was already detailed here, so you don't need to imagine it, you can just go back and read it. (Since it's a proper subset of the POSIX C binding, it remains compatible, and could even be back-ported, as an optional mode, to the C binding.) > Seems to me that the only real issues for debate are: > which functions are cancellation points, Any function may be a cancellation point. If cancellation does not cause it to violate its long-documented interface, nothing is broken. If it does, then it doesn't matter much which set you choose, you've broken most libraries anyway. If you're starting with zero existing code, all that new code can be written to any interface you make up. > how a thread can enable and disable cancellation, POSIX already defines functions to block and unblock cancellation. (They might be called automatically on entry and certain exits from catch blocks, though. That's an interesting discussion.) > whether a thread should be allowed to disregard > a cancellation request once it has been received, and Nobody has proposed allowing a thread to "disregard" cancellation. That's a strawman. > what kind of cleanup a thread performs before it stops. This is just where we started. Really at issue are (1) what cancellation is to mean in a C++ context, (2) how to arrange that cancellation does not corrupt the process state to the point where it would be pointless not to terminate anyway, and (3) how to preserve the frankly enormous body of thread-safe code already written, deployed, and running for years. Most presentations here that argue that the POSIX C++ binding cannot deviate from the POSIX C binding ignore, or carefully avoid, point (2), and sidestop point (3) with disingenuous historical revisionism. They also, as in Matt's, start out assuming much that is as issue. > My feeling: it's just plain inevitable that a multithreaded program has > more functions that might throw than a single-thread program. Dealing > with this is part of what it means to make a program thread-safe. This is far from clear, and makes a very poor starting assumption. We have had thread-safe libraries for a long time. Under one proposed model most can simply be declared already "cancellation-safe", without changes. The POSIX C binding people have already abandoned their constituents' libraries, but we need not, and indeed we could rescue many of those that they have abandoned. > ... I think the really fundamental issue is whether a thread > should be allowed to receive a cancellation request, start to do some > work as a result of the request, and then decide that it doesn't want > to be cancelled. Nobody has proposed any such thing. Matt, have you confused the cancellation-exception object, which would be destroyed by running off the end of a catch block, with the thread's cancellation state? Nathan Myers ncm at cantrip.org From austern at apple.com Tue Jan 6 21:29:22 2004 From: austern at apple.com (Matt Austern) Date: Tue, 6 Jan 2004 13:29:22 -0800 Subject: [c++-pthreads] What are the real issues? In-Reply-To: <20040106211236.GX30780@tofu.dreamhost.com> References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <3FFAC516.2010803@hp.com> <3F190B5E-407F-11D8-B918-00039390D9E0@apple.com> Message-ID: <20040106221551.GA18646@redhat.com> On Tue, Jan 06, 2004 at 11:33:47AM -0800, Matt Austern wrote: > The Itanium C++ ABI, which gcc adopted, made cancellation a special > kind of exception, "forced unwinding", so that a thread can't just > catch the cancellation exception and swallow it. This is not correct. The IA-64 ABI describes "forced unwinding" but does not describe its semantics at all. Which is of course completely unhelpful. My initial implementation of forced unwinding skipped catch-all, and ran destructors. This was vetoed by G++ folks. The current implementation of forced unwinding is to treat it just like any other kind of exception. Almost useless, IMO, since there's now a high likelyhood that longjmp_unwind will not arrive at its intended destination. The current protection against swallowing thread cancellation (and longjmp_unwind) is done by having the destructor for the exception object call abort. > ... we should probably do is abandon the notion of forced unwinding > and make cancellation into an ordinary exception. We already have, modulo the fact that the current implementation has no typename. r~ From austern at apple.com Tue Jan 6 22:22:13 2004 From: austern at apple.com (Matt Austern) Date: Tue, 6 Jan 2004 14:22:13 -0800 Subject: [c++-pthreads] What are the real issues? In-Reply-To: <20040106221551.GA18646@redhat.com> References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <3FFAC516.2010803@hp.com> <3F190B5E-407F-11D8-B918-00039390D9E0@apple.com> <20040106221551.GA18646@redhat.com> Message-ID: On Jan 6, 2004, at 2:15 PM, Richard Henderson wrote: > On Tue, Jan 06, 2004 at 11:33:47AM -0800, Matt Austern wrote: >> The Itanium C++ ABI, which gcc adopted, made cancellation a special >> kind of exception, "forced unwinding", so that a thread can't just >> catch the cancellation exception and swallow it. > > This is not correct. The IA-64 ABI describes "forced unwinding" > but does not describe its semantics at all. Which is of course > completely unhelpful. That's interesting. Sorry for the goof! That sounds like a pretty serious problem that we ought to fix no matter what else we do. --Matt From ncm at cantrip.org Tue Jan 6 22:33:16 2004 From: ncm at cantrip.org (Nathan Myers) Date: Tue, 6 Jan 2004 14:33:16 -0800 Subject: [c++-pthreads] What are the real issues? In-Reply-To: <64F121A7-408F-11D8-B918-00039390D9E0@apple.com> References: <3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <3FFAC516.2010803@hp.com> <3F190B5E-407F-11D8-B918-00039390D9E0@apple.com> <20040106211236.GX30780@tofu.dreamhost.com> <64F121A7-408F-11D8-B918-00039390D9E0@apple.com> Message-ID: <20040106223316.GY30780@tofu.dreamhost.com> On Tue, Jan 06, 2004 at 01:29:22PM -0800, Matt Austern wrote: > On Jan 6, 2004, at 1:12 PM, Nathan Myers wrote: > > >>... I think the really fundamental issue is whether a thread > >>should be allowed to receive a cancellation request, start to do some > >>work as a result of the request, and then decide that it doesn't want > >>to be cancelled. > > > >Nobody has proposed any such thing. > > Really? I think that you and I both did. I didn't. Jason didn't. Those were the only models posted. If you want to post a model or a family of models, we can discuss them too. Did you have some reason for suggesting that cancellations might be readily discarded, or did you think somebody else had asked for it? > (2) Your proposal is that the POSIX functions that are currently > described as cancellation points shouldn't raise any kind of > exception and shouldn't cause the thread to stop executing, > but should instead return an error code. Again, this would allow > a thread to note the error return, check errno to see if it's > ETHREADCANCELED, set a flag as a result of having received > the cancelation request, and continue executing. Again, your > proposal would allow a thread to receive and act on a > cancellation request without being canceled. No, the thread remains cancelled. The difference between that and the thread actually discarding the cancellation request is that the next time a function identified as a cancellation point is called, it fails, too. The cancellation is not lost, or discarded, or ignored; it surfaces again and again, indefinitely, as long as the thread fails to terminate. Furthermore, a subsequent (e.g.) filebuf operation would actually throw. If that exception were caught and discarded, the next would throw again. Somebody described this as "sticky cancellation". Under POSIX C, a thread cancellation handler can continue to run indefinitely, so you're not describing something fundamentally different from the C model. Likewise, if unwinding were to run catch clauses, code in a catch clause could continue to run indefinitely. Nathan Myers ncm at cantrip.org From ncm at cantrip.org Tue Jan 6 22:48:13 2004 From: ncm at cantrip.org (Nathan Myers) Date: Tue, 6 Jan 2004 14:48:13 -0800 Subject: [c++-pthreads] What are the real issues? In-Reply-To: <20040106221551.GA18646@redhat.com> References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <3FFAC516.2010803@hp.com> <3F190B5E-407F-11D8-B918-00039390D9E0@apple.com> <20040106221551.GA18646@redhat.com> Message-ID: <20040106224813.GZ30780@tofu.dreamhost.com> On Tue, Jan 06, 2004 at 02:15:51PM -0800, Richard Henderson wrote: > On Tue, Jan 06, 2004 at 11:33:47AM -0800, Matt Austern wrote: > My initial implementation of forced unwinding skipped catch-all, > and ran destructors. This was vetoed by G++ folks. This is an example of proposed semantics (fortunately vetoed) that would have corrupted the process state, making the unwind pointless. > The current implementation of forced unwinding is to treat it > just like any other kind of exception. Almost useless, IMO, > since there's now a high likelyhood that longjmp_unwind will > not arrive at its intended destination. If that's part of its definition, then new code can be written with that in mind. Vanishingly little existing code makes longjmp_unwind calls, so there need be no concern (in particular) about breaking third-party libraries. Nathan Myers ncm at cantrip.org From dave at boost-consulting.com Tue Jan 6 23:31:23 2004 From: dave at boost-consulting.com (David Abrahams) Date: Tue, 06 Jan 2004 18:31:23 -0500 Subject: cancellation points report failure References: <2A8DB02E3018D411901B009027FD3A3F01CAAF6A@mchp905a.mch.sbs.de> <2E98D8A4-3251-11D8-B7E5-00039390D9E0@apple.com> <20031219194503.GA31795@tofu.dreamhost.com> <20031219205746.GA12322@diablo.name> <20031219225650.GA13735@diablo.name> <20031220045618.GA27836@tofu.dreamhost.com> <1071946483.25325.69.camel@doubledemon.codesourcery.com>

Message-ID: Jason Merrill writes: > On Wed, 24 Dec 2003 12:38:46 -0500, David Abrahams wrote: > >> Jason Merrill writes: >> >>> * C++ I/O functions can throw a cancellation exception. >>> * C++ catch blocks work normally. >>> * If a cancellation exception is destroyed, the cancellation request >>> is re-entered, and acted on again at the next cancellation point. >> >> I appreciate that your motivation is to ensure that synchronous >> cancellation requests aren't ignored, but that doesn't really seem to >> accomplish the goal. After all, a thread never has to reach a >> cancellation point, and even if it does, it can keep catching and >> discarding the exceptions indefinitely. > > That's fine with me. I'm not trying to assure that cancellation always > succeeds; as you say, a thread never has to reach a cancellation point. > My goal is to make sure that the cancellation request is never lost, > that it remains either active or pending until the thread exits. OK. I don't have enough of an understanding of the reasons for cancellation in the first place to know whether it's a good goal or not, but I don't see any major problems with it. AFAICT, the difference between your proposal and Nathan's, once cancellation is thrown, is that your proposal can prevent cancellation from being thrown unexpectedly in a catch block (**) that's part of an unwind sequence? (**) From a destructor during unwinding also, but that could be prevented using std::unhandled_exception(). -- Dave Abrahams Boost Consulting www.boost-consulting.com From dave at boost-consulting.com Tue Jan 6 23:33:51 2004 From: dave at boost-consulting.com (David Abrahams) Date: Tue, 06 Jan 2004 18:33:51 -0500 Subject: What are the real issues? References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <3FFAC516.2010803@hp.com> <3F190B5E-407F-11D8-B918-00039390D9E0@apple.com> Message-ID: Matt Austern writes: > Third, some people simply object to having cancellation be an > exception (or anything that looks like an exception): Who? > it means there are some functions that can throw exceptions in the > presence of threads that wouldn't throw in a single-threaded > program. I don't think it means that. We might choose to throw cancellation exceptions only from functions that could throw in a single-threaded app or (threading-specific) functions that can't appear in a single-threaded app. -- Dave Abrahams Boost Consulting www.boost-consulting.com From hinnant at twcny.rr.com Wed Jan 7 02:50:09 2004 From: hinnant at twcny.rr.com (Howard Hinnant) Date: Tue, 6 Jan 2004 21:50:09 -0500 Subject: [c++-pthreads] Re: What are the real issues? In-Reply-To: References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <3FFAC516.2010803@hp.com> <3F190B5E-407F-11D8-B918-00039390D9E0@apple.com> Message-ID: <3503666E-40BC-11D8-A006-003065D18932@twcny.rr.com> I've been debating whether to stay silent and merely suspect I'm an idiot, or to open my mouth and remove all doubt... Basic question: Are we trying to solve a problem that really exists? I'm putting aside for the moment asynchronous cancelation as once you get into that you give up all hope of releasing resources. For now I'm simply discussing deferred cancelation. In C you need an organized system in which to register cleanup handlers to release resources in case a thread is canceled. You also need a way to temporarily disable cancelations. And finally you need a mechanism by which a thread can be notified that someone has requested cancelation. The POSIX lib neatly supplies all of these requirements. In C++ we have such a different environment than in C that I am questioning whether there is a need for a library supported cancelation mechanism in the first place. One thread can easily request another thread to cancel itself via a reference to a shared variable (which need not be global). As long as the "cancelation mechanism" is via a thrown exception, there is no need for cleanup handlers. And since a thread can manually check whether its cancel variable has been set or not, there is no need to disable cancelations (the thread simply doesn't check whether it has been canceled if it isn't willing to cancel itself). In other words, the C++ language makes it so easy to write "cancelable threads" and "cancel-safe code", are we making an easy problem hard by trying to code a solution into a library? For example below is a simple multi-thread helloworld using Metrowerks::threads (think boost::threads) where the main thread creates and then cancels a couple of worker threads. Resources are acquired and released properly, not due to the Metrowerks::threads lib, but due to the C++ language itself (RAII and basic exception safety). One might point out that the demo below relies on polling to detect thread cancelation, and this is true. But pthread cancellation also seems to rely on polling, so . The demo also avoids global variables to communicate cancelation to the threads by using function objects instead of functions (a technique unavailable to the C programmer). And of course, the use of exceptions and (basic) exception-safe code eliminates the need for cleanup handlers (also unavailable to the C programmer). So is the solution we're designing here somehow superior to what the C++ programmer can easily do for himself already? I'm merely asking, I'm not claiming to know the answer to that question. #include #include Metrowerks::mutex cout_mutex; // A is a generic resource holder class class A { public: explicit A(int id) : id_(id) { Metrowerks::mutex::scoped_lock lock(cout_mutex); std::cout << "A(" << id_ << ") constructed\n"; } ~A() { Metrowerks::mutex::scoped_lock lock(cout_mutex); std::cout << "A(" << id_ << ") destructed\n"; } private: A(const A&); A& operator=(const A&); int id_; }; struct cancel {}; // func is a generic (cancelable) thread-worker function object class func { public: func(int id, volatile bool& b) : id_(id), cancel_(b) {} void operator()() const; private: void job1() const; void sub_job1() const; void job2() const; int id_; volatile bool& cancel_; }; void func::operator()() const { unsigned i = 0; A a(id_); while (true) { try { job1(); job2(); Metrowerks::mutex::scoped_lock lock(cout_mutex); std::cout << i++ << std::endl; } catch (std::exception& e) { // fix and swallow std::exceptions, but not a cancel Metrowerks::mutex::scoped_lock lock(cout_mutex); std::cout << "Fixing " << e.what() << '\n'; } catch (cancel& e) { // Note (but don't swallow) cancel Metrowerks::mutex::scoped_lock lock(cout_mutex); std::cout << "Canceling " << id_ << '\n'; throw; } catch (...) { // Note (but don't swallow) non-std exceptions Metrowerks::mutex::scoped_lock lock(cout_mutex); std::cout << "Unknown exception - rethrowing\n"; throw; } } } void func::job1() const { A a(id_); sub_job1(); } void func::sub_job1() const { A a(id_); if (cancel_) throw cancel(); } void func::job2() const { A a(id_); if (cancel_) throw cancel(); } int main() { try { // main can create and cancel worker threads volatile bool cancel1 = false; volatile bool cancel2 = false; Metrowerks::thread t1(func(1, cancel1)); Metrowerks::thread t2(func(2, cancel2)); Metrowerks::thread::sleep(Metrowerks::elapsed_time(1)); cancel2 = true; Metrowerks::thread::sleep(Metrowerks::elapsed_time(1)); cancel1 = true; t2.join(); t1.join(); std::cout << "Done\n"; } catch (std::exception& e) { // Note std::exceptions Metrowerks::mutex::scoped_lock lock(cout_mutex); std::cout << e.what() << " - Abnormal termination\n"; } catch (...) { // Note non-std exceptions Metrowerks::mutex::scoped_lock lock(cout_mutex); std::cout << "Unknown exception - Abnormal termination\n"; } } A(1) constructed A(1) constructed A(1) constructed A(1) destructed A(1) destructed A(1) constructed A(1) destructed 0 ... 3093 A(1) constructed A(1) constructed A(1) destructed A(1) destructed A(1) constructed A(1) destructed Canceling 1 A(1) destructed Done -Howard From fjh at cs.mu.OZ.AU Wed Jan 7 03:06:40 2004 From: fjh at cs.mu.OZ.AU (Fergus Henderson) Date: Wed, 7 Jan 2004 14:06:40 +1100 Subject: [c++-pthreads] Re: What are the real issues? In-Reply-To: <3503666E-40BC-11D8-A006-003065D18932@twcny.rr.com> References: <3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <3FFAC516.2010803@hp.com> <3F190B5E-407F-11D8-B918-00039390D9E0@apple.com> <3503666E-40BC-11D8-A006-003065D18932@twcny.rr.com> Message-ID: <20040107030640.GA2175@ceres.cs.mu.oz.au> On 06-Jan-2004, Howard Hinnant wrote: > In other words, the C++ language makes it so easy to write "cancelable > threads" and "cancel-safe code", are we making an easy problem hard by > trying to code a solution into a library? Sure you can write your own cancellation framework. That can be done quite easily in C too. The point of standardizing a particular cancellation framework is to enable interoperability between code written by different parties. For example, this is needed to allow application code to cancel threads which happen to be running third-party library code, or even standard library code such as getchar() or system calls such as read(). -- Fergus Henderson | "I have always known that the pursuit The University of Melbourne | of excellence is a lethal habit" WWW: | -- the last words of T. S. Garp. From fjh at cs.mu.OZ.AU Wed Jan 7 03:24:38 2004 From: fjh at cs.mu.OZ.AU (Fergus Henderson) Date: Wed, 7 Jan 2004 14:24:38 +1100 Subject: [c++-pthreads] What are the real issues? In-Reply-To: <20040106211236.GX30780@tofu.dreamhost.com> References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> Message-ID: <20040107052644.GO32272@tofu.dreamhost.com> On Tue, Jan 06, 2004 at 06:58:57AM -0500, Dave Butenhof wrote: > Personally, I would be happy to accept a C++ binding with no way to > enable asynchronous cancelability, and to avoid defining any C++ code > as "async cancel-safe". I think we're all agreed on that. This list is about semantics of a C++ binding for synchronous cancellation. Nathan Myers ncm at cantrip.org From ncm at cantrip.org Wed Jan 7 07:17:50 2004 From: ncm at cantrip.org (Nathan Myers) Date: Tue, 6 Jan 2004 23:17:50 -0800 Subject: [c++-pthreads] concrete library-code example In-Reply-To: <3FF9977C.7040003@hp.com> References: <3FE70542.5060104@bourguet.org> <1072111706.3474.17.camel@minax.codesourcery.com> <20031222215712.GB1528@diablo.name> <1072131497.7562.22.camel@doubledemon.codesourcery.com> <20031223160314.GA9780@diablo.name> <20031224170517.GG30780@tofu.dreamhost.com> <3FF9977C.7040003@hp.com> Message-ID: <20040107071750.GP32272@tofu.dreamhost.com> A quick reply... On Mon, Jan 05, 2004 at 11:57:32AM -0500, Dave Butenhof wrote: > Nathan Myers wrote: > > > >Here is a more-or-less concrete example, for discussion purposes. > >It's meant as a generic example of code written according to the > >existing contract offered by C libraries. > > Correction: "... offered by C libraries that support POSIX 1003.1b-1993 > or earlier." Very few programmers can identify any POSIX definition by number. They write, and have long written, exception-safe library code that, at most, uses mutexes (wrapped carefully for portability!) to guard global state. Few have even heard of cancellation. Many millions of lines of such code have been running for years on millions of installations, worldwide. It's good code. To pretend that it's all suddenly worthless because it doesn't take into account new (or newly-deployed) standard revision 7834-stroke-"b"-slash- 667-stroke-"a" would simply make _us_ irrelevant. > > int affect_world(struct state* s) > > { > > int result; > > violate_invariants_or_claim_resources(s); > > result = c_function_or_system_call(s->member); > > if (result < 0) { > > clean_up(s, result); > > return result; > > } > > act_on_result(s, result); > > restore_invariants_and_release_resources(s); > > return 0; > > } > > > >This pattern is extremely common in both C and C++ libraries. If > >read() were to throw (or to "just ... not return"), the program state > >would be corrupted. A redefinition of c_function_or_system_call > >semantics that breaks this code breaks many thousands of existing > >thread-safe C and C++ libraries. > > If this code exists in a pure ANSI C/POSIX application using threads, > and if the thread running this code can be cancelled, then the > implementation of this function is broken because IT (not the > implementation, nor the cancellation) corrupts program state. No. The code was written to a documented interface. Whoever changes the interface semantics without changing the interface name is responsible for corrupting the program state. > While I'm not at all trying to argue that the issue is at all as simple > as this, that's the facts all the same. Sorry, that's simply disingenuous. To argue that everybody should have coded to an interface that you only just got around to documenting, implementing, and deploying, many years after the code was written, borders on arrogant contempt. Such an attitude may be fine for the POSIX C committee, but I see no reason to match it here. In any case, we have a great deal more already-thread-safe code to preserve, because thread-safety (by the common definition) is the norm, in C++. > Depending on propagation of error statuses is a really bad way to > implement cancellation. At least, given the primitive and limited > concept of ANSI/POSIX error codes. Too much code ignores statuses in > the first place, which is bad enough. But, worse, there are many > legitimate reasons for library code to CONVERT return status values; > e.g., I called read() and it returned some error but MY function only > implicitly involves a read() and it simply wouldn't be useful or > meaningful to return that error to my caller. Instead, I want to > indicate that my function (say, synchronizing a database) failed, and > so any (or at least most) failures of my "support calls" will result > in my returning 'unable to synchronize database' (which often isn't an > ANSI/POSIX error number in the first place, but even if it is, it's > unlikely to be the value returned by read). The ECANCELLED some have > proposed would be lost, and that's unacceptable. This is why we > settled on exceptions to represent cancellation. And because POSIX and > ANSI C don't have exceptions, we devised the simple "cleanup handler" > mechanism that allowed a clean and transparent implementation on top > of exceptions, or a "hack" implementation private to the thread > library where exceptions weren't available. Again, that reasoning may be fine for C (did you really ask all those C programmers?), but we need not be bound by it here. Since a cancellation error return swallowed up in library code must surface again at the next cancellation point, eventually (given a well-written library) the failure must propagate upward to the point where it may be turned into an exception. (A library that never propagates system-call failures to its caller isn't anything-safe, and needn't concern us.) > >(The cancellation model described in > > http://www.codesourcery.com/archives/c++-pthreads/msg00021.html > >is designed to preserve libraries that contain code that follows > >this pattern.) > > > >Jason, do you not consider those libraries worth preserving? > > If you're talking about a currently non-threaded library to which > you'd like to transparently add thread support; well, I doubt that's > possible, and this particular proposal isn't going to help. When > they're redesigned and recoded to be thread-safe, they can also be > made cancel-safe. No. I'm talking about the many millions of lines of existing thread-safe library code. Ordinary thread-safety is the norm in C++ libraries, because it's the natural way to code, in C++. > If you're talking about adding cancel support transparently > to an existing C++ library, I doubt this is sufficient unless there's > some standard requirement that all C++ libraries must pass through the > system failure code to the caller. (There isn't, can't be, and shouldn't > be.) And it also presupposes that the C++ library isn't exception-safe; > because if it is, then delivering cancellation as an exception would > seem "obviously" to be the most compatible and complete solution. Exception-safety depends on identifying and guarding against documented sources of possible exceptions. System calls and C library functions are not among those. Also, C++ libraries very frequently rely on underlying C libraries, and are written to depend on their documented behavior. (None of my man(2) or man(3) pages mention unwinding, never mind throwing.) Even if you claim that the threat of "unwinding" from system calls is ancient, and that everything should have been written to assume it, a change to make them throw would be completely new. > And I'm deliberately discounting the mention I've seen several times in > this list of "thread-safe" libraries that aren't "cancel-safe". Such > libraries are simply broken, from basic design. Cancellation is a basic > and important part of the POSIX thread model, and if you're not safe > you're not safe. The only viable exclusion (there, I avoided using the > word "exception", though it took me a few moments of thought) is if you > can be guaranteed to be running only in threads that can never be > cancelled... and in that case the whole issue is irrelevant! No offense intended, but disingenuity makes a poor substitute for responsible design. We can afford to be more responsible here, because we have stronger language semantics to work with, and well-worked-out exception-safety standards. Nathan Myers ncm at cantrip.org From rth at redhat.com Wed Jan 7 09:57:58 2004 From: rth at redhat.com (Richard Henderson) Date: Wed, 7 Jan 2004 01:57:58 -0800 Subject: [c++-pthreads] What are the real issues? In-Reply-To: <20040106224813.GZ30780@tofu.dreamhost.com> References: <3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <3FFAC516.2010803@hp.com> <3F190B5E-407F-11D8-B918-00039390D9E0@apple.com> <20040106221551.GA18646@redhat.com> <20040106224813.GZ30780@tofu.dreamhost.com> Message-ID: <20040107095758.GB21007@redhat.com> On Tue, Jan 06, 2004 at 02:48:13PM -0800, Nathan Myers wrote: > This is an example of proposed semantics (fortunately vetoed) that > would have corrupted the process state, making the unwind pointless. Only with stupid C++ users using catch-all when they should have been using destructors in the first place. But that's another debate. r~ From wil at bogo.xs4all.nl Wed Jan 7 11:06:45 2004 From: wil at bogo.xs4all.nl (Wil Evers) Date: Wed, 07 Jan 2004 12:06:45 +0100 Subject: [c++-pthreads] What are the real issues? References: <3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <3FFAC516.2010803@hp.com> <3F190B5E-407F-11D8-B918-00039390D9E0@apple.com> <20040106211236.GX30780@tofu.dreamhost.com> <64F121A7-408F-11D8-B918-00039390D9E0@apple.com> <20040106223316.GY30780@tofu.dreamhost.com> Message-ID: <3FFBE845.9040707@bogo.xs4all.nl> Nathan Myers wrote: > On Tue, Jan 06, 2004 at 01:29:22PM -0800, Matt Austern wrote: >> (2) Your proposal is that the POSIX functions that are currently >> described as cancellation points shouldn't raise any kind of >> exception and shouldn't cause the thread to stop executing, >> but should instead return an error code. Again, this would allow >> a thread to note the error return, check errno to see if it's >> ETHREADCANCELED, set a flag as a result of having received >> the cancelation request, and continue executing. Again, your >> proposal would allow a thread to receive and act on a >> cancellation request without being canceled. > > > No, the thread remains cancelled. The difference between that and > the thread actually discarding the cancellation request is that the > next time a function identified as a cancellation point is called, it > fails, too. The cancellation is not lost, or discarded, or ignored; > it surfaces again and again, indefinitely, as long as the thread fails > to terminate. Furthermore, a subsequent (e.g.) filebuf operation > would actually throw. If that exception were caught and discarded, > the next would throw again. Somebody described this as "sticky > cancellation". OK, that sounds reasonable - the 'sticky cancellation' concept provides some guarantee that a thread that ignores or discards a cancellation request will be reminded later. However, it seems to me that the idea of simply having system calls defined as cancellation points fail until the cancelled thread has finished its job is just too crude. In particular, it would prevent the thread being cancelled from doing any I/O at all, even when that I/O is part of the cleanup the thread is supposed to perform in handling the cancellation request. Your ETHREADCANCELLED proposal relies on higher-level C++ code to turn this errno value into some kind of thread_cancelled exception at a point where such an exception can be reasonably expected. This seems to suggest that the underlying machinery in the system call interface library will be unaware of the actual steps the thread in question takes in honoring the cancellation request: it would simply report that cancellation has been requested, leaving it to higher-level code to take appropriate action. If that is the case, then how do we stop the ETHREADCANCELLED error from resurfacing - possibly triggering a C++ exception or even a call to terminate() - while the thread is busy cleaning up? - Wil From boo at terekhov.de Wed Jan 7 11:47:34 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Wed, 07 Jan 2004 12:47:34 +0100 Subject: [c++-pthreads] Re: cancellation points report failure References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <20040107052644.GO32272@tofu.dreamhost.com> Message-ID: <3FFBF1D6.56ABAC@terekhov.de> Nathan Myers wrote: > > On Tue, Jan 06, 2004 at 06:58:57AM -0500, Dave Butenhof wrote: > > Personally, I would be happy to accept a C++ binding with no way to > > enable asynchronous cancelability, and to avoid defining any C++ code > > as "async cancel-safe". > > I think we're all agreed on that. This list is about semantics of > a C++ binding for synchronous cancellation. Objection. To begin with, asynchronous cancelability IS part of pthreads. Well, void operation(); blabla another_operation( ...blablabla... ) async_cancel_safe; void operation() { // cancel points DO throw here (if expected) async_cancel { // cancel type = ASYNC // only async-cancel-safe operations // are allowed here. Error/ill-formed // otherwise. for example: another_operation(); // OK sync_cancel { // cancel type = SYNC/DEFERRED // Back to usual no_cancel { // cancel points do NOT throw here } // Back to usual } // only async-cancel-safe operations // are allowed here. Error/ill-formed // otherwise. no_cancel { // cancel *TYPE* = SYNC/DEFERRED // cancel points do NOT throw here } // only async-cancel-safe operations // are allowed here. Error/ill-formed otherwise. } // cancel points DO throw here no_cancel { // cancel points do NOT throw here } // cancel points DO throw here } blabla another_operation( ...blablabla... ) async_cancel_safe { // ^^^^^^^^^^^^^^^^^ // async_cancel_safe implies "async_cancel {" scope // restrictions/safety for body (on entry) AND // "blablabla" parameters PLUS "blabla-return-value" too! // only async-cancel-safe operations // are allowed here. Error/ill-formed // otherwise. sync_cancel { // cancel type = SYNC/DEFERRED // Back to usual no_cancel { // cancel points do not throw here } // Back to usual } // only async-cancel-safe operations // are allowed here. Error/ill-formed // otherwise. no_cancel { // cancel *TYPE* = SYNC/DEFERRED // cancel points do NOT throw here } // only async-cancel-safe operations // are allowed here. Error/ill-formed // otherwise. return ...; } regards, alexander. From boo at terekhov.de Wed Jan 7 11:54:51 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Wed, 07 Jan 2004 12:54:51 +0100 Subject: [c++-pthreads] Re: cancellation points report failure References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <3FFAC516.2010803@hp.com> Message-ID: <3FFBF38B.1E19A2F3@terekhov.de> David Abrahams wrote: [...] > it? Interesting thought, if so, but how do you disable just > cancellation without masking other exceptions? no_cancel { } Of course. regards, alexander. From boo at terekhov.de Wed Jan 7 12:16:05 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Wed, 07 Jan 2004 13:16:05 +0100 Subject: [c++-pthreads] What are the real issues? References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <3FFAC516.2010803@hp.com> <3F190B5E-407F-11D8-B918-00039390D9E0@apple.com> <20040106221551.GA18646@redhat.com> Message-ID: <3FFBF885.4B89C2E0@terekhov.de> Matt Austern wrote: > > On Jan 6, 2004, at 2:15 PM, Richard Henderson wrote: > > > On Tue, Jan 06, 2004 at 11:33:47AM -0800, Matt Austern wrote: > >> The Itanium C++ ABI, which gcc adopted, made cancellation a special > >> kind of exception, "forced unwinding", so that a thread can't just > >> catch the cancellation exception and swallow it. > > > > This is not correct. The IA-64 ABI describes "forced unwinding" > > but does not describe its semantics at all. Which is of course > > completely unhelpful. > > That's interesting. Sorry for the goof! > > That sounds like a pretty serious problem that we ought to fix > no matter what else we do. Get rid of it entirely. setjmp()/longjmp() can (and shall) be implemented without "the stop and stop_parameter parameters" which "control the termination of the unwind process, instead of the usual personality routine query", I believe. Ignoring the fact that jumping is severely restricted in C++, compilers are just ought to sort of "recognize" setjmp() and allow rather simple implementation of longjmp() -- "throw std::longjmp_request(val);" (or something like that), oder? regards, alexander. From boo at terekhov.de Wed Jan 7 12:33:26 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Wed, 07 Jan 2004 13:33:26 +0100 Subject: [c++-pthreads] What are the real issues? References: <3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <3FFAC516.2010803@hp.com> <3F190B5E-407F-11D8-B918-00039390D9E0@apple.com> <20040106211236.GX30780@tofu.dreamhost.com> <20040107032438.GB2175@ceres.cs.mu.oz.au> <20040107051511.GM32272@tofu.dreamhost.com> Message-ID: <3FFBFC96.F419A280@terekhov.de> Nathan Myers wrote: [...] > We have better prospects in C++, if we can, as it were, hitch the > cancellation-safety wagon to the exception-safety horse. Well-written > exception handling code (i.e. implemented mainly in destructors) is > much less subject to rot, being exercised on each block exit. Many > more people know how to write exception-safe C++ code than C > cancellation-safe code. (I'm not sure I would be able to write the > latter, maintainably. C simply lacks the tools for it.) P in "pthreads" stands for POSIX, not C, I guess. #define pthread_cleanup_push(routine, arg) \ { \ ScopeGuard _guard = MakeObjGuard(arg, routine); #define pthread_cleanup_pop(execute) \ if (!execute) _guard.Dismiss(); \ } http://www.cuj.com/documents/s=8000/cujcexp1812alexandr (Simplify Your Exception-Safe Code) regards, alexander. From boo at terekhov.de Wed Jan 7 13:25:29 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Wed, 07 Jan 2004 14:25:29 +0100 Subject: [c++-pthreads] What are the real issues? References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <3FFAC516.2010803@hp.com> <3F190B5E-407F-11D8-B918-00039390D9E0@apple.com> <20040106211236.GX30780@tofu.dreamhost.com> <20040107032438.GB2175@ceres.cs.mu.oz.au> Message-ID: <3FFC08C9.F2A93D4A@terekhov.de> Fergus Henderson wrote: [...] > Thread-safe code which may perform unbounded computation without encountering > a cancellation-point is not what I would call cancellation-safe. Yep. > > So even if code has been written to be thread-safe, you need to analyze it > in detail in order to be sure that it will be properly cancellation-safe. http://google.com/groups?selm=WJUO8.15690%24qL5.827507%40news4.srv.hcvlny.cv.net http://google.com/groups?selm=3D0EE2D6.34EE9712%40web.de regards, alexander. From boo at terekhov.de Wed Jan 7 13:29:14 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Wed, 07 Jan 2004 14:29:14 +0100 Subject: [c++-pthreads] What are the real issues? References: <3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <3FFAC516.2010803@hp.com> <3F190B5E-407F-11D8-B918-00039390D9E0@apple.com> <20040106211236.GX30780@tofu.dreamhost.com> <20040107032438.GB2175@ceres.cs.mu.oz.au> <20040107051511.GM32272@tofu.dreamhost.com> <3FFBFC96.F419A280@terekhov.de> <1073479235.28355.97.camel@felix.inria.fr> Message-ID: <3FFC09AA.99BF37A7@terekhov.de> Mathieu Lacage wrote: [...] > > http://www.cuj.com/documents/s=8000/cujcexp1812alexandr > > (Simplify Your Exception-Safe Code) > > File Not Found > cujcexp1812alexandr Try http://www.cuj.com/documents/s=8000/cujcexp1812alexandr/ regards, alexander. From David.Butenhof at hp.com Wed Jan 7 15:41:38 2004 From: David.Butenhof at hp.com (Dave Butenhof) Date: Wed, 07 Jan 2004 10:41:38 -0500 Subject: [c++-pthreads] concrete library-code example In-Reply-To: <20040107071750.GP32272@tofu.dreamhost.com> References: <3FE70542.5060104@bourguet.org> <1072111706.3474.17.camel@minax.codesourcery.com> <20031222215712.GB1528@diablo.name> <1072131497.7562.22.camel@doubledemon.codesourcery.com> <20031223160314.GA9780@diablo.name> <20031224170517.GG30780@tofu.dreamhost.com> <3FF9977C.7040003@hp.com> <20040107071750.GP32272@tofu.dreamhost.com> Message-ID: <3FFC28B2.707@hp.com> Nathan Myers wrote: >On Mon, Jan 05, 2004 at 11:57:32AM -0500, Dave Butenhof wrote: > > >>Nathan Myers wrote: >> >> >>>Here is a more-or-less concrete example, for discussion purposes. >>>It's meant as a generic example of code written according to the >>>existing contract offered by C libraries. >>> >>> >>Correction: "... offered by C libraries that support POSIX 1003.1b-1993 >>or earlier." >> >> >Very few programmers can identify any POSIX definition by number. >They write, and have long written, exception-safe library code that, >at most, uses mutexes (wrapped carefully for portability!) to guard >global state. Few have even heard of cancellation. > >Many millions of lines of such code have been running for years on >millions of installations, worldwide. It's good code. To pretend >that it's all suddenly worthless because it doesn't take into account >new (or newly-deployed) standard revision 7834-stroke-"b"-slash- >667-stroke-"a" would simply make _us_ irrelevant. > > You're right -- most people don't understand the distinctions between POSIX revisions, so perhaps I've presumed too much. So I'll make this clear. Once you're talking about "POSIX" and "threads", there's no such ambiguity. There were no concept at all of "threads" in POSIX before there was cancellation. Cancellation is a required base feature of "pthreads", and has been since well before the first (semi-)public draft of the specification. There is no such thing as "pthreads" without cancellation, and never has been. A library coded to "1003.lb-1993 or before" can't use cancellation... but it can't use (or work with) threads, either. (Among other details, until 1003.1c-1995 [threads], POSIX overruled ANSI C and unconditionally required a single global errno variable; which makes simultaneous or even multiplexed thread execution, er, "interesting".) The only rational or reasonable way to label code "correct" that uses POSIX thread interfaces (e.g., mutexes), and does not address cancellation, is if that code was designed exclusively for a SPECIALIZED (non-general) purpose where it could be known that it would run only in threads that the application WILL NOT cancel. There is no such thing as a CORRECT, general purpose, "POSIX" library recognizing threads that doesn't address cancellation. Period. If the code in question CORRECTLY uses non-cancel-safe threading, then it will continue to be safe no matter what choices C++ makes for cancellation support... because threads running the code will never be cancelled. If this is NOT true, then it was never correct in the first place. Again, as I've said, if you're talking about non-POSIX threaded code, the story may be different. Yes, Win32 doesn't have cancellation. Solaris "UI Threads" (deprecated since Solaris 2.5, which is a long way back) didn't have cancellation. There are undoubtedly embedded system threading systems that don't have cancellation. This goes back to my question "what are the real goals here"? Doing cancellation as error returns instead of exceptions doesn't make anything cancel-safe, either; not unless you analyze every place that calls a cancellable function to be sure it's doing something reasonable with this new return value that means something critically different from other errors. Many presume that the distinction between errors isn't really important, because they all simply mean "it didn't work". Many more simply ignore the error. Even those that try to "decode" the error and do something reasonable don't pass on the same error code to its own caller, for all sorts of reasons. Error returns aren't modular; particularly in complicated layered systems. Exceptions are. That's why exceptions are better, and why cancellation was always intended to be modelled as an exception. >>>int affect_world(struct state* s) >>>{ >>> int result; >>> violate_invariants_or_claim_resources(s); >>> result = c_function_or_system_call(s->member); >>> if (result < 0) { >>> clean_up(s, result); >>> return result; >>> } >>> act_on_result(s, result); >>> restore_invariants_and_release_resources(s); >>> return 0; >>>} >>> >>>This pattern is extremely common in both C and C++ libraries. If >>>read() were to throw (or to "just ... not return"), the program state >>>would be corrupted. A redefinition of c_function_or_system_call >>>semantics that breaks this code breaks many thousands of existing >>>thread-safe C and C++ libraries. >>> >>> >>If this code exists in a pure ANSI C/POSIX application using threads, >>and if the thread running this code can be cancelled, then the >>implementation of this function is broken because IT (not the >>implementation, nor the cancellation) corrupts program state. >> >> >No. The code was written to a documented interface. Whoever changes >the interface semantics without changing the interface name is >responsible for corrupting the program state. > > But the code is NOT written to the documented interface unambiguously applicable to ALL POSIX code that uses threads. If it doesn't use threads, there's no problem -- either in POSIX or in this hypothetical "threaded C++". The only complication is if it was written for some OTHER threading package without cancellation, and if it's really more critical to the C++ goals to protect it from cancellation than to allow the rest of the application to depend on timely and safe cancellation. (And if so, I'd suggest the best way to be safe is to simply omit any support at all for cancellation. ) Once more, as explanation and apology, I wrote my original reply on this list blindly believing the list's name: "c++-pthreads". Not "C++-threads". "pthreads" means "POSIX threads", and "POSIX threads" means modular cancellation cleanup deliberately modelled on exceptions. I ought to have already known from previous discussions that the list name was inaccurate. The question is... how inaccurate, and in what way? If cancellation is important, then you'll be compatible with all CORRECT general POSIX threaded code by closely following the POSIX cancellation semantics. Nothing will magically make "all arbitrary threading package code, or incorrect POSIX threading code" cancel-safe, so I personally think it's a waste of time to worry about that too much. Either way, someone's going to need to analyze the library code and make sure it's doing the right thing when it's cancelled. It's easier to do that with isolated modular cleanup mechanisms like cleanup handlers and destructors than to follow the often twisty passages by which typical code deals with and propagates error codes (if it does at all). >>While I'm not at all trying to argue that the issue is at all as simple >>as this, that's the facts all the same. >> >> >Sorry, that's simply disingenuous. To argue that everybody should have >coded to an interface that you only just got around to documenting, >implementing, and deploying, many years after the code was written, >borders on arrogant contempt. > >Such an attitude may be fine for the POSIX C committee, but I see no >reason to match it here. In any case, we have a great deal more >already-thread-safe code to preserve, because thread-safety (by the >common definition) is the norm, in C++. > > Again, either the code is "POSIX threads" or not. If it is, then either it deals with cancellation, knows a-priori that the application will never CHOOSE to cancel the thread that runs it, or it's broken. There are no other options. (Yes, broken code is common. Some code broken in this particular way may have no other problems, though it's not generally the way I'd prefer to bet. Designing a solution around support for broken code, though, without any guarantee that it'll fare any better under the less-modular alternative, doesn't seem to me the best way to start.) If it's NOT "POSIX threads", then it probably has no concept of cancellation in the first place, and cannot be made magically safe for cancellation by any reasonable set of definitions. If it's C then it PROBABLY doesn't handle hierarchical cleanup. But if it's C++ (and at least THAT part of the list's name is presumably accurate), then it's probably as likely to do the right cleanup when an arbitrary new exception propagates as any C or C++ code is to do the right thing on an arbitrary new error status. (Which may well be just a way of saying that neither will work very often; but the real point is that I believe using error returns will make things worse, not better.) >>Depending on propagation of error statuses is a really bad way to >>implement cancellation. At least, given the primitive and limited >>concept of ANSI/POSIX error codes. Too much code ignores statuses in >>the first place, which is bad enough. But, worse, there are many >>legitimate reasons for library code to CONVERT return status values; >>e.g., I called read() and it returned some error but MY function only >>implicitly involves a read() and it simply wouldn't be useful or >>meaningful to return that error to my caller. Instead, I want to >>indicate that my function (say, synchronizing a database) failed, and >>so any (or at least most) failures of my "support calls" will result >>in my returning 'unable to synchronize database' (which often isn't an >>ANSI/POSIX error number in the first place, but even if it is, it's >>unlikely to be the value returned by read). The ECANCELLED some have >>proposed would be lost, and that's unacceptable. This is why we >>settled on exceptions to represent cancellation. And because POSIX and >>ANSI C don't have exceptions, we devised the simple "cleanup handler" >>mechanism that allowed a clean and transparent implementation on top >>of exceptions, or a "hack" implementation private to the thread >>library where exceptions weren't available. >> >> >Again, that reasoning may be fine for C (did you really ask all those >C programmers?), but we need not be bound by it here. > >Since a cancellation error return swallowed up in library code must >surface again at the next cancellation point, eventually (given a >well-written library) the failure must propagate upward to the point >where it may be turned into an exception. (A library that never >propagates system-call failures to its caller isn't anything-safe, >and needn't concern us.) > > I really don't understand how this is supposed to work. "Pending cancel" remains until the thread terminates. There was some mention somewhere that implied it would be cleared on delivery but "re-set" when the exception object was destroyed -- but I don't see how you'd manage that in any model where the "real" cancellation is a status return that MIGHT sometime later be "converted" to an exception. That means every cancellable call made during cleanup will fail with your ECANCELLED. Which means most things simply cannot be cleaned up. OK, so maybe you defer all attempts to clean up until the exception is first raised, at which point you "unpend" the cancel until and unless the exception object is destroyed. That still breaks your example if the clean_up() routine makes any cancellable calls. (So much for preserving the old code.) Yet if returning the first ECANCEL "unpends", you'll lose the cancel if the code instead does something like: if (result < 0) { clean_up(s, result); return EINVAL; } ... and I thought the whole point of the "sticky cancel" was precisely to avoid that risk? I haven't seen anything so far that seems to offer a way out of this maze of messes. And that's precisely why we made cancellation an exception in the first place. >>>(The cancellation model described in >>>http://www.codesourcery.com/archives/c++-pthreads/msg00021.html >>>is designed to preserve libraries that contain code that follows >>>this pattern.) >>> >>>Jason, do you not consider those libraries worth preserving? >>> >>> >>If you're talking about a currently non-threaded library to which >>you'd like to transparently add thread support; well, I doubt that's >>possible, and this particular proposal isn't going to help. When >>they're redesigned and recoded to be thread-safe, they can also be >>made cancel-safe. >> >> >No. I'm talking about the many millions of lines of existing >thread-safe library code. Ordinary thread-safety is the norm in C++ >libraries, because it's the natural way to code, in C++. > > "Natural"? No, it's not, unless you presume that all objects and static data are private. And you know that's not true. C++ libraries, like STL, have gone through enormous pains to try to be "basically thread-safe", and there are still conceptual problems, particularly in areas like the interdependencies around iostreams. It's not easy to get any complicated system "right". Threading adds complication. Cancellation and/or exceptions adds complication. Nobody's saying it doesn't. But to be GENERAL PURPOSE C++ code, you need to be exception-safe; and to be GENERAL PURPOSE POSIX thread code, you need to be cancel-safe. To be GENERAL PURPOSE POSIX C++ thread code, you need to be both. And since there's not much difference between POSIX cancel-safe and C++ exception-safe, the combination shouldn't be so difficult, technically. The problems, as always, are more political. What SORTS of changes are required of existing code? (You may like to think it's "WHETHER" changes are required -- but you'll never convince me there's any way to avoid requiring change or at least careful analysis of every affected code path.) WHICH existing code patterns to prefer over others? (And much of this pivots on which patterns each person thinks are common or important, and that's a matter of seriously subjective opinion.) Whether it's OK to lose a cancel? What level of reliability and latency a cancellable application ought to be able to expect? >>If you're talking about adding cancel support transparently >>to an existing C++ library, I doubt this is sufficient unless there's >>some standard requirement that all C++ libraries must pass through the >>system failure code to the caller. (There isn't, can't be, and shouldn't >>be.) And it also presupposes that the C++ library isn't exception-safe; >>because if it is, then delivering cancellation as an exception would >>seem "obviously" to be the most compatible and complete solution. >> >> >Exception-safety depends on identifying and guarding against documented >sources of possible exceptions. System calls and C library functions >are not among those. Also, C++ libraries very frequently rely on >underlying C libraries, and are written to depend on their documented >behavior. (None of my man(2) or man(3) pages mention unwinding, never >mind throwing.) > > None mention ECANCELLED, either. So if we can't add an undocumented throw, how can we add an undocumented error code? While you might like to think that "just adding a new error code" is "nearly transparent" to "some existing code", you're correct only in some narrow boundary of "nearly" and "some". There'll inevitably be enormous volumes of code that DOESN'T fit that preconceived pattern, and I'm quite sure we've all seen plenty of examples. The only thing that's FREE is NOTHING. So either we do nothing, or we accept a cost. I don't see a whole lot of argument for simply ignoring cancellation entirely, which is nothing, and free. So you simply can't argue against some strategy because there's a cost. There's always a cost. The interesting questions revolve around how much cost is acceptable, and the benefits of each strategy relative to its cost. Currently, ANSI C++ doesn't "support" threads at all. So any use of threads is beyond the boundaries of the standard, and therefore nonportable and subject to the whims of various implementations. On Tru64 UNIX and OpenVMS, cancellation is and always has been an exception. In POSIX cancellation has always worked exactly like a special exception (within the constraints of POSIX and ANSI C, of course, which doesn't allow actually using the word "exception" except in non-binding explanations). Nowhere has it ever been represented as a special error return from general pre-existing system functions. So which is more compatible, and with what? >Even if you claim that the threat of "unwinding" from system calls is >ancient, and that everything should have been written to assume it, >a change to make them throw would be completely new. > > Replace "unwinding" and "throw" by "cancel", or "threads", and the statement still stands. If you've already decided to add threads and cancel, I don't see how this argument is relevant. Yes, things will change. Old assumptions will be broken. Code will need to adapt. If you really CAN change the world so fundamentally without changing the code that runs in it, that'd be great; but I don't believe it. >>And I'm deliberately discounting the mention I've seen several times in >>this list of "thread-safe" libraries that aren't "cancel-safe". Such >>libraries are simply broken, from basic design. Cancellation is a basic >>and important part of the POSIX thread model, and if you're not safe >>you're not safe. The only viable exclusion (there, I avoided using the >>word "exception", though it took me a few moments of thought) is if you >>can be guaranteed to be running only in threads that can never be >>cancelled... and in that case the whole issue is irrelevant! >> >> >No offense intended, but disingenuity makes a poor substitute for >responsible design. > You clearly like the word "disingenuous". You've used it several times. That's at best a subjective slur that doesn't belong in this discussion. I may disagree with you, but I'm trying hard not to accuse you of being deliberately "not-straightforward", despite the enormous temptation of turning the accusation back at you. Or I could be disingenuous and simply say that I agree completely with your statement... but why should I take offense when it clearly has nothing at all to do with me? >We can afford to be more responsible here, because we have stronger language semantics to work with, and well-worked-out exception-safety standards. > > Yes, let's be responsible. Perhaps we should first define precisely what "responsible" is intended to mean in this context, before we start arguing over which low-level details or discussions might fall under that banner. And I'm not being disingenuous here -- I absolutely mean it. Perhaps the C++ committee people already know exactly the full range of constraints and requirements on this effort, but I, and presumably others involved in this wider discussion group, cannot. If those constraints and requirements aren't to be explicitly and fully shared with us, then the discussion never should have been opened up in the first place... and I might as well just go away. -- /--------------------[ David.Butenhof at hp.com ]--------------------\ | Hewlett-Packard Company Tru64 UNIX & VMS Thread Architect | | My book: http://www.awl.com/cseng/titles/0-201-63392-2/ | \----[ http://homepage.mac.com/dbutenhof/Threads/Threads.html ]---/ From gdr at integrable-solutions.net Wed Jan 7 16:12:43 2004 From: gdr at integrable-solutions.net (Gabriel Dos Reis) Date: 07 Jan 2004 17:12:43 +0100 Subject: [c++-pthreads] concrete library-code example In-Reply-To: <3FFC28B2.707@hp.com> References: <3FE70542.5060104@bourguet.org> <1072111706.3474.17.camel@minax.codesourcery.com> <20031222215712.GB1528@diablo.name> <1072131497.7562.22.camel@doubledemon.codesourcery.com> <20031223160314.GA9780@diablo.name> <20031224170517.GG30780@tofu.dreamhost.com> <3FF9977C.7040003@hp.com> <20040107071750.GP32272@tofu.dreamhost.com> <3FFC28B2.707@hp.com> Message-ID: Dave Butenhof writes: | mean it. Perhaps the C++ committee people already know exactly the | full range of constraints and requirements on this effort, but I, and | presumably others involved in this wider discussion group, cannot. If | those constraints and requirements aren't to be explicitly and fully | shared with us, then the discussion never should have been opened up | in the first place... and I might as well just go away. Well, I would not say that the C++ committee people already know exactly the full range of constraints and requirements. I believe some people have firm opinions on what they would like to have, but those vary from individuals to individual -- you most certainly saw disagreements between C++ committee members on this list. At any rate, I've found your contributions to this discussion highly instructive. -- Gaby From David.Butenhof at hp.com Wed Jan 7 16:44:46 2004 From: David.Butenhof at hp.com (Dave Butenhof) Date: Wed, 07 Jan 2004 11:44:46 -0500 Subject: [c++-pthreads] concrete library-code example In-Reply-To: References: <3FE70542.5060104@bourguet.org> <1072111706.3474.17.camel@minax.codesourcery.com> <20031222215712.GB1528@diablo.name> <1072131497.7562.22.camel@doubledemon.codesourcery.com> <20031223160314.GA9780@diablo.name> <20031224170517.GG30780@tofu.dreamhost.com> <3FF9977C.7040003@hp.com> <20040107071750.GP32272@tofu.dreamhost.com> <3FFC28B2.707@hp.com> Message-ID: <3FFC377E.60800@hp.com> Gabriel Dos Reis wrote: >Dave Butenhof writes: > >| mean it. Perhaps the C++ committee people already know exactly the >| full range of constraints and requirements on this effort, but I, and >| presumably others involved in this wider discussion group, cannot. If >| those constraints and requirements aren't to be explicitly and fully >| shared with us, then the discussion never should have been opened up >| in the first place... and I might as well just go away. > >Well, I would not say that the C++ committee people already know >exactly the full range of constraints and requirements. I believe >some people have firm opinions on what they would like to have, but >those vary from individuals to individual -- you most certainly saw >disagreements between C++ committee members on this list. > > Well, perhaps there might have been just a tiny element of disingenuity in my paragraph; though I'd prefer to call it "tact" in this instance, that distinction may not really be justifiable. I really do think that IF there are any predefined requirements and constraints, either they need to be explicitly layed out for us "outsiders", or they need to be set aside entirely for these discussions, because we can't be expected to know them. On the other hand, I had already assumed everything you've stated above, and to some extent I suppose I was just teasing Nathan. I really don't like being called, even by implication, "irresponsible", (or "disingenuous") simply on the basis that I happen to disagree with one individual's opinion. For giving into that base temptation at retribution, I must sincerely apologize to all. Even to Nathan. ;-) >At any rate, I've found your contributions to this discussion highly >instructive. > > Thank you. -- /--------------------[ David.Butenhof at hp.com ]--------------------\ | Hewlett-Packard Company Tru64 UNIX & VMS Thread Architect | | My book: http://www.awl.com/cseng/titles/0-201-63392-2/ | \----[ http://homepage.mac.com/dbutenhof/Threads/Threads.html ]---/ From austern at apple.com Wed Jan 7 17:11:36 2004 From: austern at apple.com (Matt Austern) Date: Wed, 7 Jan 2004 09:11:36 -0800 Subject: [c++-pthreads] concrete library-code example In-Reply-To: <3FFC377E.60800@hp.com> References: <3FE70542.5060104@bourguet.org> <1072111706.3474.17.camel@minax.codesourcery.com> <20031222215712.GB1528@diablo.name> <1072131497.7562.22.camel@doubledemon.codesourcery.com> <20031223160314.GA9780@diablo.name> <20031224170517.GG30780@tofu.dreamhost.com> <3FF9977C.7040003@hp.com> <20040107071750.GP32272@tofu.dreamhost.com> <3FFC28B2.707@hp.com> <3FFC377E.60800@hp.com> Message-ID: <8CC625F5-4134-11D8-8CAE-000393B2ABA2@apple.com> On Jan 7, 2004, at 8:44 AM, Dave Butenhof wrote: > Gabriel Dos Reis wrote: > >> Dave Butenhof writes: >> >> | mean it. Perhaps the C++ committee people already know exactly the >> | full range of constraints and requirements on this effort, but I, >> and >> | presumably others involved in this wider discussion group, cannot. >> If >> | those constraints and requirements aren't to be explicitly and fully >> | shared with us, then the discussion never should have been opened up >> | in the first place... and I might as well just go away. >> >> Well, I would not say that the C++ committee people already know >> exactly the full range of constraints and requirements. I believe >> some people have firm opinions on what they would like to have, but >> those vary from individuals to individual -- you most certainly saw >> disagreements between C++ committee members on this list. >> > Well, perhaps there might have been just a tiny element of > disingenuity in my paragraph; though I'd prefer to call it "tact" in > this instance, that distinction may not really be justifiable. I > really do think that IF there are any predefined requirements and > constraints, either they need to be explicitly layed out for us > "outsiders", or they need to be set aside entirely for these > discussions, because we can't be expected to know them. To some extent I don't think anyone knows them. This is a group that consists of people from a lot of different places; some of the people on this list, by no means all, attend C++ committee meetings. The origin of this discussion was on the GCC mailing list, when people were trying to figure out how gcc, glibc, and libstdc++ should handle the intersection of C++ and pthreads. People in that discussion realized that this was a discussion that extended beyond the GNU community and that a solution adopted by gcc/linux wouldn't be nearly as interesting as a solution that was generally recognized as right. In the end, "generally recognized as right" probably means getting approval from the C++ committee and/or The Austin Group. But that's for another day. My goals are (I believe) very similar to yours: figure out what the POSIX C binding should mean for C++. This might mean something as ambitious as coming up with a separate C++ binding, or it might mean making some minor tweaks and clarifications in the existing C binding and/or the existing C++ language specification. In principle I'm agnostic between the two. In practice I suspect we don't have the resources or the vendor buy-in to do anything extremely ambitious. --Matt From David.Butenhof at hp.com Wed Jan 7 18:10:04 2004 From: David.Butenhof at hp.com (Dave Butenhof) Date: Wed, 07 Jan 2004 13:10:04 -0500 Subject: [c++-pthreads] concrete library-code example In-Reply-To: <8CC625F5-4134-11D8-8CAE-000393B2ABA2@apple.com> References: <3FE70542.5060104@bourguet.org> <1072111706.3474.17.camel@minax.codesourcery.com> <20031222215712.GB1528@diablo.name> <1072131497.7562.22.camel@doubledemon.codesourcery.com> <20031223160314.GA9780@diablo.name> <20031224170517.GG30780@tofu.dreamhost.com> <3FF9977C.7040003@hp.com> <20040107071750.GP32272@tofu.dreamhost.com> <3FFC28B2.707@hp.com> <3FFC377E.60800@hp.com> <8CC625F5-4134-11D8-8CAE-000393B2ABA2@apple.com> Message-ID: <3FFC4B7C.9070507@hp.com> Matt Austern wrote: > On Jan 7, 2004, at 8:44 AM, Dave Butenhof wrote: > >> Gabriel Dos Reis wrote: >> >>> Dave Butenhof writes: >>> >>> | mean it. Perhaps the C++ committee people already know exactly the >>> | full range of constraints and requirements on this effort, but I, and >>> | presumably others involved in this wider discussion group, cannot. If >>> | those constraints and requirements aren't to be explicitly and fully >>> | shared with us, then the discussion never should have been opened up >>> | in the first place... and I might as well just go away. >>> >>> Well, I would not say that the C++ committee people already know >>> exactly the full range of constraints and requirements. I believe >>> some people have firm opinions on what they would like to have, but >>> those vary from individuals to individual -- you most certainly saw >>> disagreements between C++ committee members on this list. >> >> Well, perhaps there might have been just a tiny element of >> disingenuity in my paragraph; though I'd prefer to call it "tact" in >> this instance, that distinction may not really be justifiable. I >> really do think that IF there are any predefined requirements and >> constraints, either they need to be explicitly layed out for us >> "outsiders", or they need to be set aside entirely for these >> discussions, because we can't be expected to know them. > > To some extent I don't think anyone knows them. This is a group that > consists of people from a lot of different places; some of the people > on this list, by no means all, attend C++ committee meetings. About what I'd expected. > The origin of this discussion was on the GCC mailing list, when people > were trying to figure out how gcc, glibc, and libstdc++ should handle > the intersection of C++ and pthreads. People in that discussion > realized that this was a discussion that extended beyond the GNU > community and that a solution adopted by gcc/linux wouldn't be nearly > as interesting as a solution that was generally recognized as right. That's encouraging. I guess it also explains the group name, since "gcc/linux" would likely be focused on C++ and POSIX threads, regardless of any skew between that and the long term goals of the C++ committee. So... "Ah". > In the end, "generally recognized as right" probably means getting > approval from the C++ committee and/or The Austin Group. But that's > for another day. > > My goals are (I believe) very similar to yours: figure out what the > POSIX C binding should mean for C++. This might mean something as > ambitious as coming up with a separate C++ binding, or it might mean > making some minor tweaks and clarifications in the existing C binding > and/or the existing C++ language specification. In principle I'm > agnostic between the two. In practice I suspect we don't have the > resources or the vendor buy-in to do anything extremely ambitious. I would like to think that it means a unique C++ binding, because this would present far too many unique and useful opportunities to ignore. However I also think there ought to be strong compatibility, at least "philosophically" with POSIX; especially in areas like cancellation where there's substantial POSIX existing practice and virtually no existing practice outside POSIX. And most especially considering that the kernel and C runtime will need to coexist with both models. While some changes to the existing C language POSIX specification might be necessary, inevitable, or even desirable, the sort of fundamental change in cancellation model some have proposed here is a non-starter. C++ may be worried about trying to support non-POSIX threaded code (though I've expressed several doubts about precisely what that's supposed to mean and whether it's relevant), but keep in mind that such a fundamental change in the existing POSIX specification would break ALL (correct) existing general-purpose POSIX-based threaded libraries, and many existing applications. To say such a change would be "contentious" would be an absurd understatement. In fact, I would argue that for the Austin group to accept such a change would be truly irresponsible. And also that for any OS to continue to support POSIX threads while also supporting a radically distinct C++ threading model would be onerous and impractical. None of this need necessarily serve as a constraint on the C++ committee. But then, it doesn't hurt to think about things like "vendor buy-in", and that's a lot easier when you're not making radical and contradictory requirements. Of course that's a simplification, because some of the C++ rules and idiom around exceptions presents a different sort of contradiction with "POSIX compatible" model, and that's not trivial to resolve either. I don't mean to discount that, but obviously my focus is more on POSIX than on C++, and I can only contribute what I have to contribute. -- /--------------------[ David.Butenhof at hp.com ]--------------------\ | Hewlett-Packard Company Tru64 UNIX & VMS Thread Architect | | My book: http://www.awl.com/cseng/titles/0-201-63392-2/ | \----[ http://homepage.mac.com/dbutenhof/Threads/Threads.html ]---/ From baker at cs.fsu.edu Wed Jan 7 19:32:23 2004 From: baker at cs.fsu.edu (Ted Baker) Date: Wed, 7 Jan 2004 14:32:23 -0500 Subject: [c++-pthreads] What are the real issues? In-Reply-To: <3F190B5E-407F-11D8-B918-00039390D9E0@apple.com> References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <3FFAC516.2010803@hp.com> <3F190B5E-407F-11D8-B918-00039390D9E0@apple.com> Message-ID: <20040107193223.GA2884@diablo.name> > The Itanium C++ ABI, which gcc adopted, made cancellation a special > kind of exception, "forced unwinding", so that a thread can't just > catch the cancellation exception and swallow it. The idea was that a This "forced unwinding" sounds different from normal exception processing, since you cannot catch the exception. How deeply is this embedded in the gcc C++ implementation? e.g., is the unwinding done by different code (maybe even by a proxy thread?) for cancellation than for normal exception propagation? For example, with GNAT, the task abort exception is processed exactly like other exceptions. The difference is pretty superficial in that tha parser does not allow handlers for abort in user code. This is easily overridden (a GNAT-specific language extension) when one is writing system support code, so it is not a problem for critical libraries. > thread may disable cancellation tempoarily, but that once a > cancellation has begun it can't be thrown away. This was a slightly > controversial decision at the time, and maybe it's at the heart of the > real issue. > Second: the immediate issue was that some people saw a contradiction > between this model and the C++ library specification. For example, > POSIX says that read() is a cancellation point. But it's reasonable to > assume that std::istream:;read() invokes POSIX's read() system call, > and the C++ standard says that (under ordinary circumstances) > std::istream doesn't throw. If the cancellation exception were an > ordinary exception instead of special forced unwinding, what would > happen would be that istream would invoke its streambuf, the streambuf > would call read(), read would throw a cancellation exception, then > istream would catch the exception, swallow it, and set and error flag. > Forced unwinding means that the cancellation exception will propagate > even though istream tries to swallow it. This means that an exception > will propagate from istream when the C++ standard says it's not > supposed to. See question above. Is this normal propagation? If so, then shouldn't it be possible to catch it? > Third, some people simply object to having cancellation be an exception > (or anything that looks like an exception): it means there are some > functions that can throw exceptions in the presence of threads that > wouldn't throw in a single-threaded program. There seems to be no way around this, short of requiring that cancellation be disabled over all suspect library calls. As Dave explained so well, if a system supports thread cancellation it can happen and it behaves virtually like an exception, regardless of what you call it. To ask the pthread library implementors to change this would be a fundamental change to the pthread semantics. > cancellation that's very different from the POSIX. Anything that > doesn't involve thread execution stopping sounds more like a > communication mechanism than a cancellation mechanism. Seems to me > that the only real issues for debate are which functions are > cancellation points, how a thread can enable and disable cancellation, > whether a thread should be allowed to disregard a cancellation request > once it has been received, and what kind of cleanup a thread performs > before it stops. POSIX and the Single Unix Specification cover these. In the SUS, much of the relevant text is in the Base Definitions part, which are intended to be language-independent. ---> whether a thread should be allowed to disregard a cancellation request once it has been received SUS'98, which subsumes the POSIX standards, says: "Whenever a thread has cancelability enabled and a cancellation request has been made with that thread as the target, and the thread then calls any function that is a cancellation point (such as pthread_testcancel() or read()), the cancellation request shall be acted upon before the function returns." [The "shall" means cancellation cannot be ignored at this point.] "... If cancelability is disabled ... all cancellation requests are held pending...". [So, cancellation must be ignored at this point.] This seems to be language-independent semantics. ---> how a thread can enable and disable cancellation The C API defines pthread_setcanceltype() and pthread_setcancelstate(). A C++ API could define additional functions or points at which the above functions are implicitly (virtually) called. ---> what kind of cleanup a thread performs before it stops: SUS'98 says: "When a cancellation request is acted upon, the routines in the list are invoked one by one in LIFO sequence; that is, the last routine pushed onto the list (Last In) is the first to be invoked (First Out). The thread invokes the cancellation cleanup handler with cancellation disabled until the last cancellation cleanup handler returns. When the cancellation cleanup handler for a scope is invoked, the storage for that scope remains valid. If the last cancellation cleanup handler returns, thread execution is terminated and a status of PTHREAD_CANCELED is made available to any threads joining with the target. The symbolic constant PTHREAD_CANCELED expands to a constant expression of type ( void *) whose value matches no pointer to an object in memory nor the value NULL. "The cancellation cleanup handlers are also invoked when the thread calls pthread_exit()." "...When the cancellation is acted on, the cancellation cleanup handlers for thread shall be called. When the last cancellation cleanup handler returns, the thread-specific data destructor functions shall be called for thread. When the last destructor function returns, thread shall be terminated. ..." ---> which functions are cancellation points: SUS'98 (and POSIX) define a raft of functions for which cancellation points "shall" occur, and some more for which cancellation points "may" occur. A C++ binding could define other names for functions with similar effects that are shall *not* be cancellation points. > My feeling: it's just plain inevitable that a multithreaded program has > more functions that might throw than a single-thread program. Dealing > with this is part of what it means to make a program thread-safe. We > might argue that POSIX defines too many cancellation points, and that > cutting it down to a smaller number (and, especially, getting rid of > the functions that POSIX says may or may not be cancellation points) > would make it easier to write correct code. Yes. > But as I said, I think the really fundamental issue is whether a thread > should be allowed to receive a cancellation request, start to do some > work as a result of the request, and then decide that it doesn't want > to be cancelled. If we think that's reasonable then I think what we POSIX and the SUS do not define this. I believe it would be an upward-compatible extension to define it. That is, no existing UNIX/POSIX conformant application could be broken by an extension that extends the API to allow a thread to regain control and "revoke" a cancellation, since no conformant application could presently include such an API call However, the debate will be over one *wants* to export such functionality. > should probably do is abandon the notion of forced unwinding and make > cancellation into an ordinary exception. If we do that then the > std::istream contradiction goes away. The cost, of course, is that it > might become surprisingly hard to cancel some threads. From what I've read here, and my own experience, it seems this is the right thing to do, provided: 1) The only way to handle this new cancellation exception is via a new syntax. 2) You can agree on new C++ API function names for whichever POSIX/Unix cancellation-point system calls you don't think should ever propagate a cancellation exception. To do the latter, you have several choices: (a) Have the wrapper (e.g., for read()) just disable cancellation (and not reenable it). (b) Work with your friendly pthread implementor (the one who manages the code of the library functions that are cancellation points) to develop alternate entry points (e.g., read_nocancel()) or a settable flag (e.g., that a C++ thread or process can call to make read() *not* serve as a cancellation point) for all the cancellation-point system calls that you don't think should really be cancellation points. --Ted From baker at cs.fsu.edu Wed Jan 7 21:26:04 2004 From: baker at cs.fsu.edu (Ted Baker) Date: Wed, 7 Jan 2004 16:26:04 -0500 Subject: [c++-pthreads] What are the real issues? In-Reply-To: <20040106211236.GX30780@tofu.dreamhost.com> References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <20040107052644.GO32272@tofu.dreamhost.com> <3FFBF1D6.56ABAC@terekhov.de> Message-ID: <20040108040112.GB9432@jupiter.cs.mu.oz.au> On 07-Jan-2004, Alexander Terekhov wrote: > Nathan Myers wrote: > > > > On Tue, Jan 06, 2004 at 06:58:57AM -0500, Dave Butenhof wrote: > > > Personally, I would be happy to accept a C++ binding with no way to > > > enable asynchronous cancelability, and to avoid defining any C++ code > > > as "async cancel-safe". > > > > I think we're all agreed on that. This list is about semantics of > > a C++ binding for synchronous cancellation. > > Objection. To begin with, asynchronous cancelability IS part of > pthreads. Asynchronous cancelability only works if you can be sure that there are no implicit calls to async-cancel-unsafe functions going on. That's not too hard to ensure in C, but it's a lot more difficult in C++. -- Fergus Henderson | "I have always known that the pursuit The University of Melbourne | of excellence is a lethal habit" WWW: | -- the last words of T. S. Garp. From Mathieu.Lacage at sophia.inria.fr Thu Jan 8 10:00:51 2004 From: Mathieu.Lacage at sophia.inria.fr (Mathieu Lacage) Date: Thu, 08 Jan 2004 11:00:51 +0100 Subject: thread-safety definition Message-ID: <1073556051.28360.195.camel@felix.inria.fr> hi all, It looks like what everyone is trying to achieve here is a way for C++ authors to write thread-safe libraries. The major problem which seems to plague this discussion is that different people assume different definitions for thread-safety. Here is my take. Hopefully, without too many factual errors... Definition "thread safety" -------------------------- A thread-safe library is a library which uses locks to protect data which can be potentially accessed from different threads. This kind of code is typically written with a trivial small OS-abstraction layer which implements basic semaphore semantics (or another synchronization primitive). This kind of library was never written to deal with "cancelation" (not the POSIX cancelation primitive but, more generally, externally-triggered death of the thread) and is not safe with regard to either "inside" or any form of "outside" cancelation as defined below. Definition "cancelation" ------------------------ All OSes provide a mean to stop the execution flow of a thread either from the inside of the thread or from the outside. 1) "inside cancelation": This is basically ExitThread (win32 name). It exists on all the platforms which support a form of threads or another I know of. It semantics vary a lot from one platform to the other unfortunatly. On win32, it will not invoke any thread-specific cleanup handlers (neither C++ exceptions nor SEH are involved). On BeOS (exit_thread), it will behave just like on windows. On POSIX (pthread_exit) systems, it will invoke the thread-specific cancelation handlers. 2) "outside cancelation": There are two kinds of "outside cancelation": 2.1) "async cancelation": The OS removes the thread from its list of tasks to schedule and does nothing to cleanup the thread ressources. This is the most extreme useless feature of a thread library. BeOS and win32 provide it. POSIX does not provide it. 2.2) "defered cancelation": I know of only POSIX to implement this. The canceled flag for the target thread is set and the thread cancelation handlers are invoked whenever the thread reaches a cancelation point (that is, it calls one of a set of specific library functions). Definition "Posix thread-safety": --------------------------------- A library is "posix thread-safe" if it is thread-safe and defered-cancelation-safe. I hope the above definitions will help. regards, Mathieu -- Mathieu Lacage From David.Butenhof at hp.com Thu Jan 8 11:34:12 2004 From: David.Butenhof at hp.com (Dave Butenhof) Date: Thu, 08 Jan 2004 06:34:12 -0500 Subject: [c++-pthreads] thread-safety definition In-Reply-To: <1073556051.28360.195.camel@felix.inria.fr> References: <1073556051.28360.195.camel@felix.inria.fr> Message-ID: <3FFD4034.7050903@hp.com> Mathieu Lacage wrote: >Definition "cancelation" >------------------------ >All OSes provide a mean to stop the execution flow of a thread either >from the inside of the thread or from the outside. > >1) "inside cancelation": This is basically ExitThread (win32 name). It >exists on all the platforms which support a form of threads or another I >know of. It semantics vary a lot from one platform to the other >unfortunatly. On win32, it will not invoke any thread-specific cleanup >handlers (neither C++ exceptions nor SEH are involved). On BeOS >(exit_thread), it will behave just like on windows. On POSIX >(pthread_exit) systems, it will invoke the thread-specific cancelation >handlers. > > The term "cancellation" seems heavy here. This is just a voluntary termination. But, yes, there are similar properties -- certainly from the point of view of the rest of the frames on the call stack at the time. >2) "outside cancelation": There are two kinds of "outside cancelation": > > 2.1) "async cancelation": The OS removes the thread from its list of >tasks to schedule and does nothing to cleanup the thread ressources. >This is the most extreme useless feature of a thread library. BeOS and >win32 provide it. POSIX does not provide it. > > I'd call this "abort", largely because there was at one time a proposal for a POSIX function to achieve this, which would have been called pthread_abort(). POSIX already defines "async cancel", as a mode where posting a cancel to a thread will cause the cancellation to be delivered at any arbitrary time supported by the OS and hardware. (Usually on the next clock tick, though that's a "common implementation" rather than any rule or even recommendation.) > 2.2) "defered cancelation": I know of only POSIX to implement this. The >canceled flag for the target thread is set and the thread cancelation >handlers are invoked whenever the thread reaches a cancelation point >(that is, it calls one of a set of specific library functions). > > "Cancellation" (both deferred and async) come from the Digital "CMA" architecture (where it was called "alert"). The CMA concept derives from a less structured (but fundamentally similar) capability in the SRC research labs' Topaz thread package. >Definition "Posix thread-safety": >--------------------------------- >A library is "posix thread-safe" if it is thread-safe and >defered-cancelation-safe. > > I wouldn't tack cancel-safety onto thread-safety so intimately, although I would agree that it's pointless to declare a general library function "thread safe under POSIX" unless it also supports deferred cancellation. (Async cancel is an oddity; there are, and should be, very few async-cancel-safe functions. Async-cancel regions of code cannot accomodate resource acquisition or release of any sort, as the recovery code is generally unable to determine the state of the resource.) Nevertheless, it's quite reasonable to write a "thread-safe" special purpose application routine that doesn't deal with cancellation simply because the designer KNOWS that a thread running that code cannot be cancelled. One might even make this choice within in a general purpose library in some cases -- say, for a daemon thread that could never run application code nor be identified to the application, and that therefore cannot be cancelled. -- /--------------------[ David.Butenhof at hp.com ]--------------------\ | Hewlett-Packard Company Tru64 UNIX & VMS Thread Architect | | My book: http://www.awl.com/cseng/titles/0-201-63392-2/ | \----[ http://homepage.mac.com/dbutenhof/Threads/Threads.html ]---/ From David.Butenhof at hp.com Thu Jan 8 11:51:47 2004 From: David.Butenhof at hp.com (Dave Butenhof) Date: Thu, 08 Jan 2004 06:51:47 -0500 Subject: [c++-pthreads] What are the real issues? In-Reply-To: <20040107193223.GA2884@diablo.name> References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <3FFAC516.2010803@hp.com> <3F190B5E-407F-11D8-B918-00039390D9E0@apple.com> <20040107193223.GA2884@diablo.name> Message-ID: <3FFD4453.9010204@hp.com> Ted Baker wrote: >>But as I said, I think the really fundamental issue is whether a thread >>should be allowed to receive a cancellation request, start to do some >>work as a result of the request, and then decide that it doesn't want >>to be cancelled. If we think that's reasonable then I think what we >> >> >POSIX and the SUS do not define this. I believe it would be an >upward-compatible extension to define it. That is, no existing >UNIX/POSIX conformant application could be broken by an extension >that extends the API to allow a thread to regain control and >"revoke" a cancellation, since no conformant application could >presently include such an API call > > The POSIX cancellation cleanup mechanism was designed to be implemented easily on top of a native exception mechanism. With the full expectation that these exceptions could ALSO be handled (and finalized, if desired) by native exception syntax -- such as C++ catch, or the C SEH "except" extension. While such extensions are beyond the scope of POSIX, there's no contradiction. (Any application using this is at best "conforming with extensions" rather than "strictly conforming"; but that distinction is of little interest to most developers. Since no "strictly conforming" application can exercise these extensions, or be affected by them, the extension doesn't affect the conformance of the IMPLEMENTATION.) This was in fact a requirement placed on our initial DCE thread implementation by the OSF DCE team. They wanted to be able to write server management threads that would call the application's RPC server code and be able to trap APPLICATION-level cancellation of the server activity. Such a cancel (generally reflected from the client process) would terminate the APPLICATION server code exactly like a normal cancel (because it is), but wouldn't affect the "management infrastructure" because the routine that called the application server would have something like a FINALLY or CATCH_ALL clause (like a C++ "catch (...)"). There would have been other design alternatives, and there's certainly room to question whether this example validates the concept. Nevertheless, there are potential justifications for this sort of thing, and I've never seen any reason to disallow it. >However, the debate will be over one *wants* to export such >functionality. > > My take on that can probably be inferred from my comments above. The only reasonable implementation of cancellation is as an exception -- it was, after all, designed and intended to be implemented that way. Once that's done, I see no reason to artificially prevent specialize applications from handling it as any other exception. There are, however, certain arguments in favor of Ted's description of gnat behavior; that the code must do something special to declare that it really wants to handle this class of exception. The capability might be generally packaged as something like a subclass hierarchy designating "exceptions that should always terminate", which can only be finalized by naming the sub-hierarchy or some particular exception within (e.g., catch(...) might implicitly re-throw). -- /--------------------[ David.Butenhof at hp.com ]--------------------\ | Hewlett-Packard Company Tru64 UNIX & VMS Thread Architect | | My book: http://www.awl.com/cseng/titles/0-201-63392-2/ | \----[ http://homepage.mac.com/dbutenhof/Threads/Threads.html ]---/ From boo at terekhov.de Thu Jan 8 12:05:54 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Thu, 08 Jan 2004 13:05:54 +0100 Subject: [c++-pthreads] Re: cancellation points report failure References:

<3FF18485.93F2D588@terekhov.de> <3FF9A13D.9030105@hp.com> <3FF9B2AD.D7E6C4DD@terekhov.de> <3FFAA301.6080301@hp.com> <20040107052644.GO32272@tofu.dreamhost.com> <3FFBF1D6.56ABAC@terekhov.de> <20040108040112.GB9432@jupiter.cs.mu.oz.au> <3FFD47A2.3147DE97@terekhov.de> <20040108142149.GA20652@ceres.cs.mu.oz.au> Message-ID: <3FFD83B4.2D45ABBD@terekhov.de> Fergus Henderson wrote: [...] > > > Asynchronous cancelability only works if you can be sure that there > > > are no implicit calls to async-cancel-unsafe functions going on. > > > > Yep (and probably also C++ things like throw-expressions and "try > > {"). > > "probably" is the word. I said "probably" because apart from inherently async-cancel-unsafe operations/functions like operator new()/operator delete() (and alike), a smart compiler understanding the meaning of async_cancel{} and sync_cancel{} (and also no_cancel{}) "probably" can translate things like void f() async_cancel_safe { /*...*/ sync_static T t(/*...*/); // synchronizised local static t.async_cancel_safe_operation(); } to void f() async_cancel_safe { /*...*/ sync_cancel { sync_static T t(/*...*/); // synchronizised local static async_cancel { t.async_cancel_safe_operation(); } } } > That's what I mean -- in C++, you can't even > tell which language constructs might be implemented with code which > under-the-hood is going to be doing dynamic memory allocation or other > async-cancel-unsafe operations. So, you really want to me stick to POF*** not only for signal handlers, but also for async-cancel-safe regions? Well, thankyou. regards, alexander. ***) http://groups.google.com/groups?selm=3F169C13.D83AA51E%40web.de (Subject: Re: Thread function and C++) From boo at terekhov.de Thu Jan 8 16:25:48 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Thu, 08 Jan 2004 17:25:48 +0100 Subject: [c++-pthreads] thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD4034.7050903@hp.com> <1073567261.28360.258.camel@felix.inria.fr> <3FFD6355.80200@hp.com> <3FFD6CA8.B5CEF27C@terekhov.de> <3FFD7213.1050903@hp.com> Message-ID: <3FFD848C.25CAC0E8@terekhov.de> Dave Butenhof wrote: [...] > >Could it be that the intent was to make pthread_testcancel(), not > >pthread_cancel(), async-cancel-safe? ;-) > > > > > No. First off, the only reason to use async-cancel, with the associated > complications and risks, is that you're unwilling to pay the cost of > calling pthread_testcancel() -- e.g., in a tight computational loop. If > you are willing to call pthread_testcancel() anywhere you "really want a > cancel" it'd be just silly to bother with async cancel at all. Not at all. I want to call pthread_testcancel() at "some intervals" (within computational loop) and, at the same time, I want async cancel delivery and do NOT want to bother myself turning async cancel mode off/on just to test for pending cancellation request and trigger guaranteed delivery. > Furthermore, despite the looseness in the standard wording that allows > implementations to ignore async cancel entirely or for long periods, > this was never the intent. In any implementation of any approximation to > what the working group would have accepted as "quality", there'd be no > point at all in making the call when async cancel is enabled. What if I want to have it "react a bit faster" than a "clock tick"? What if all of my async-cancel-regions can be passed within "clock tick"? To me, "may occur" delivery with respect to async-cancel- regions is as bad (sort of problematic) as it is with respect to optional cancellation points. > > >>but in the end we decided not to risk changing it.) Really, in terms of > >>POSIX standard APIs, all you can do with async cancel enabled is to > >>DISABLE async-cancel. I like it that way. There's no reason at all that > >>ANY of the standard C++ runtime should be designated (or coded) to be > >>async-cancel safe. > >> > >> > >C++ aside for a moment, I have yet to see async-cancel-UNsafe > >implementation of, say, strlen(). Care to inspire me? TIA. > > > > > Perhaps, but that's because you've looked. The point is that there's no > way to know without reading the source. And even if you do, it may > change on the next edit. You cannot safely PRESUME that anything at all > is async cancel safe unless that's a documented characteristic of the > function. Or, of course, if you wrote it yourself and are willing to bet > that you didn't make any mistakes. ;-) Heck, let's mandate to have async-cancel-safe strlen()-and-alike stuff "in-addition-to OR instead-of" async-cancel-unsafe stuff and simply use C++ function overloading to pick the right stuff (async_cancel_safe inside async_cancel{}), okay? regards, alexander. From baker at cs.fsu.edu Thu Jan 8 16:32:15 2004 From: baker at cs.fsu.edu (Ted Baker) Date: Thu, 8 Jan 2004 11:32:15 -0500 Subject: [c++-pthreads] thread-safety definition In-Reply-To: <3FFD848C.25CAC0E8@terekhov.de> References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD4034.7050903@hp.com> <1073567261.28360.258.camel@felix.inria.fr> <3FFD6355.80200@hp.com> <3FFD6CA8.B5CEF27C@terekhov.de> <3FFD7213.1050903@hp.com> <3FFD848C.25CAC0E8@terekhov.de> Message-ID: <20040108163215.GI23908@diablo.name> > Heck, let's mandate to have async-cancel-safe strlen()-and-alike > stuff "in-addition-to OR instead-of" async-cancel-unsafe stuff > and simply use C++ function overloading to pick the right stuff > (async_cancel_safe inside async_cancel{}), okay? A joke? (With nested calls and separate compilation you obviously cannot tell what call is "inside" an async_cancel{}.) --Ted From David.Butenhof at hp.com Thu Jan 8 16:36:04 2004 From: David.Butenhof at hp.com (Dave Butenhof) Date: Thu, 08 Jan 2004 11:36:04 -0500 Subject: [c++-pthreads] thread-safety definition In-Reply-To: <3FFD848C.25CAC0E8@terekhov.de> References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD4034.7050903@hp.com> <1073567261.28360.258.camel@felix.inria.fr> <3FFD6355.80200@hp.com> <3FFD6CA8.B5CEF27C@terekhov.de> <3FFD7213.1050903@hp.com> <3FFD848C.25CAC0E8@terekhov.de> Message-ID: <3FFD86F4.7020003@hp.com> Alexander Terekhov wrote: >Heck, let's mandate to have async-cancel-safe strlen()-and-alike >stuff "in-addition-to OR instead-of" async-cancel-unsafe stuff >and simply use C++ function overloading to pick the right stuff >(async_cancel_safe inside async_cancel{}), okay? > > Let's not. -- /--------------------[ David.Butenhof at hp.com ]--------------------\ | Hewlett-Packard Company Tru64 UNIX & VMS Thread Architect | | My book: http://www.awl.com/cseng/titles/0-201-63392-2/ | \----[ http://homepage.mac.com/dbutenhof/Threads/Threads.html ]---/ From boo at terekhov.de Thu Jan 8 16:52:31 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Thu, 08 Jan 2004 17:52:31 +0100 Subject: [c++-pthreads] thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD4034.7050903@hp.com> <1073567261.28360.258.camel@felix.inria.fr> <3FFD6355.80200@hp.com> <3FFD6CA8.B5CEF27C@terekhov.de> <3FFD7213.1050903@hp.com> <3FFD848C.25CAC0E8@terekhov.de> <20040108163215.GI23908@diablo.name> Message-ID: <3FFD8ACF.583E5AC3@terekhov.de> Ted Baker wrote: > > > Heck, let's mandate to have async-cancel-safe strlen()-and-alike > > stuff "in-addition-to OR instead-of" async-cancel-unsafe stuff > > and simply use C++ function overloading to pick the right stuff > > (async_cancel_safe inside async_cancel{}), okay? > > A joke? (With nested calls and separate compilation you obviously > cannot tell what call is "inside" an async_cancel{}.) It's pretty much like "const correctness" for member functions but with "execution context" instead of "type of this" (and with "async_cancel_safe"/"async_cancel {}" instead of "const"). regards, alexander. From dave at boost-consulting.com Thu Jan 8 16:52:34 2004 From: dave at boost-consulting.com (David Abrahams) Date: Thu, 08 Jan 2004 11:52:34 -0500 Subject: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> Message-ID: Dave Butenhof writes: > David Abrahams wrote: > >>Dave Butenhof writes: >> >> >>>David Abrahams wrote: >>> >>Please don't shout, especially to emphasize something I disagree with >>so fundamentally. ;-) >> > Ah. I see the problem. I believe that you neglected to file in > triplicate the official "List of all matters with which I disagree so > please don't shout when discussing them" form. Didn't you? ;-) I filed. It was *you* who neglected to check with the department of archival non-shouting topic storage, wasn't it? >>If you make cancellations behave sufficiently like an ordinary C++ >>exception (either of Nathan's or Jason's models would do that I think) >>then it's neither "IMPOSSIBLE" nor even difficult. A great deal of >>thread-safe exception-safe C++ library code would behave perfectly >>well under those conditions. >> >> > Well, it should be no surprise that this discussion has fragmented > into too many different little pathways for anyone to keep track of > them. > > If cancel is implemented completely as a "full stature" C++ exception, > and none of the "exception safe" code does anything silly like > "catch(...)" without a re-throw, then, yes, it's likely to be > reasonably cancel-safe. Even if only *some* of it does that, Nathan and Jason's models are both likely to result in an eventual cancellation. > (Though some code might need to be aware that an uncaught cancel > will terminate the thread rather than the process!) Remember, we're talking about library code. Usually it's only used "on the inside" of threads so it doesn't care whether when it's terminated, it's the process or only a thread. > But I was really responding to the ideas about turning cancel into a > return status ...only for 'C' functions which ordinarily have a return status and don't throw... did you miss that? -- Dave Abrahams Boost Consulting www.boost-consulting.com From baker at cs.fsu.edu Thu Jan 8 17:40:11 2004 From: baker at cs.fsu.edu (Ted Baker) Date: Thu, 8 Jan 2004 12:40:11 -0500 Subject: [c++-pthreads] thread-safety definition In-Reply-To: <3FFD8ACF.583E5AC3@terekhov.de> References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD4034.7050903@hp.com> <1073567261.28360.258.camel@felix.inria.fr> <3FFD6355.80200@hp.com> <3FFD6CA8.B5CEF27C@terekhov.de> <3FFD7213.1050903@hp.com> <3FFD848C.25CAC0E8@terekhov.de> <20040108163215.GI23908@diablo.name> <3FFD8ACF.583E5AC3@terekhov.de> Message-ID: <20040108174011.GA28721@diablo.name> On Thu, Jan 08, 2004 at 05:52:31PM +0100, Alexander Terekhov wrote: > Ted Baker wrote: > > > > > Heck, let's mandate to have async-cancel-safe strlen()-and-alike > > > stuff "in-addition-to OR instead-of" async-cancel-unsafe stuff > > > and simply use C++ function overloading to pick the right stuff > > > (async_cancel_safe inside async_cancel{}), okay? > > > > A joke? (With nested calls and separate compilation you obviously > > cannot tell what call is "inside" an async_cancel{}.) > > It's pretty much like "const correctness" for member functions > but with "execution context" instead of "type of this" (and with > "async_cancel_safe"/"async_cancel {}" instead of "const"). I thought you meant *compile* resolution of the overloading, so as to generate different call contexts of the same function name. Since your async_cancel{} code can have nested calls to arbitrary (separately compiled) code, you can't do that without interprocedural control flow analysis (across separately compiled modules), which I don't believe gcc has any plans to do. If you are talking about dynamic (runtime) dispatching, then you are adding runtime overhead to all calls, and could just as well *not* use overloading; instead, you could just as well have one wrapper function that is the C++ binding, and this wrapper function dynamically decides whether to do the extra work of protecting itself against async cancel, depending on the context implicit parameter. --Ted From austern at apple.com Thu Jan 8 17:45:34 2004 From: austern at apple.com (Matt Austern) Date: Thu, 8 Jan 2004 09:45:34 -0800 Subject: [c++-pthreads] thread-safety definition In-Reply-To: <3FFD7213.1050903@hp.com> References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD4034.7050903@hp.com> <1073567261.28360.258.camel@felix.inria.fr> <3FFD6355.80200@hp.com> <3FFD6CA8.B5CEF27C@terekhov.de> <3FFD7213.1050903@hp.com> Message-ID: <761D3FB4-4202-11D8-AD9A-00039390D9E0@apple.com> On Jan 8, 2004, at 7:06 AM, Dave Butenhof wrote: > Alexander Terekhov wrote: > >> Dave Butenhof wrote: >> [...] >> >>> Nevertheless, async cancel CAN be used safely if you're careful, >>> without >>> disrupting ... >>> ^^^^^^^ >>> >> ^^^^^^^^^^ >> >> Right. >> >>> Introducing asynchronous exceptions into C++ would be pointlessly >>> disruptive ... ^^^^^^^^^^^ >>> >> ^^^^^^^^^^ >> >> And that's the phobia hits DRB again. >> > Several insults come to mind, but I'll bite my tongue. Alexander, > you're either deliberately misusing, or just misunderstand, that word. > Try looking it up in an English dictionary. Then try using it in a > sentence where it actually belongs. After that, perhaps we can get > back to talking about cancel and exceptions. I'm almost willing to accept the word "phobia" to describe my own feelings about asynchronous cancellation in C++. Both as a user and as a compiler implementer, I'm very scared of asynchronous reporting mechanisms in C++. (This is one reason that I've always resisted any attempts to allow throwing an exception from a signal handler, for example.) I have some idea of what kind of review you'd need to make sure that user code is safe in the presence of asynchronous abnormal code paths, and what kinds of things you'd need to do in the compiler to make it reliable, and I have severe doubts that I could do either of those things correctly except in really rare cases. I think we're going to have enough trouble on this list coming to consensus about how C++ should handle POSIX deferred cancellation. Asynchronous cancellation is even harder. Let's leave it for another day. --Matt From terekhov at web.de Thu Jan 8 18:12:19 2004 From: terekhov at web.de (Alexander Terekhov) Date: Thu, 08 Jan 2004 19:12:19 +0100 Subject: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD4034.7050903@hp.com> <1073567261.28360.258.camel@felix.inria.fr> <3FFD6355.80200@hp.com> <3FFD6CA8.B5CEF27C@terekhov.de> <3FFD7213.1050903@hp.com> <761D3FB4-4202-11D8-AD9A-00039390D9E0@apple.com> Message-ID: <3FFD9D83.DE4D5260@web.de> Matt Austern wrote: [...] > I'm almost willing to accept the word "phobia" to describe my own > feelings about asynchronous cancellation in C++. Why. I know. http://google.com/groups?selm=3C83C428.1B1D36C7%40web.de regards, alexander. From terekhov at web.de Thu Jan 8 18:11:55 2004 From: terekhov at web.de (Alexander Terekhov) Date: Thu, 08 Jan 2004 19:11:55 +0100 Subject: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD4034.7050903@hp.com> <1073567261.28360.258.camel@felix.inria.fr> <3FFD6355.80200@hp.com> <3FFD6CA8.B5CEF27C@terekhov.de> <3FFD7213.1050903@hp.com> <3FFD848C.25CAC0E8@terekhov.de> <20040108163215.GI23908@diablo.name> <3FFD8ACF.583E5AC3@terekhov.de> <20040108174011.GA28721@diablo.name> Message-ID: <3FFD9D6B.52E97B36@web.de> Ted Baker wrote: [...] > I thought you meant *compile* resolution of the overloading, so as > to generate different call contexts of the same function name. > Since your async_cancel{} code can have nested calls to arbitrary > (separately compiled) code, you can't do that without ^ | async-cancel-safe -----+ (nested sync_cancel {} and no_cancel {} aside for a moment). > interprocedural control flow analysis (across separately compiled > modules), which I don't believe gcc has any plans to do. If you > are talking about dynamic (runtime) dispatching, then you are ... I'm NOT talking about dynamic (runtime) dispatching. Think of const correctness. regards, alexander. From jason at redhat.com Thu Jan 8 18:49:28 2004 From: jason at redhat.com (Jason Merrill) Date: Thu, 08 Jan 2004 13:49:28 -0500 Subject: [c++-pthreads] thread-safety definition In-Reply-To: <1073556051.28360.195.camel@felix.inria.fr> (Mathieu Lacage's message of "Thu, 08 Jan 2004 11:00:51 +0100") References: <1073556051.28360.195.camel@felix.inria.fr> Message-ID: On Thu, 08 Jan 2004 11:00:51 +0100, Mathieu Lacage wrote: > 2.2) "defered cancelation": I know of only POSIX to implement this. The > canceled flag for the target thread is set and the thread cancelation > handlers are invoked whenever the thread reaches a cancelation point > (that is, it calls one of a set of specific library functions). While we're enumerating existing practice... Deferred cancellation is also a feature of the Java (Thread.interrupt) and Ada (task abort) threading models. In Java, cancellation is implemented by throwing an InterruptedException. The cancellation points are Object.wait, Thread.join and Thread.sleep. In particular, an interrupt does not wake up a thread blocked on a lock or I/O. In Ada, cancellation runs cleanups, but is not defined to be an exception, though it is often implemented as one. In particular, it cannot be caught by exception handlers in user code. Jason From baker at cs.fsu.edu Thu Jan 8 21:22:03 2004 From: baker at cs.fsu.edu (Ted Baker) Date: Thu, 8 Jan 2004 16:22:03 -0500 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: <3FFD9D6B.52E97B36@web.de> References: <3FFD4034.7050903@hp.com> <1073567261.28360.258.camel@felix.inria.fr> <3FFD6355.80200@hp.com> <3FFD6CA8.B5CEF27C@terekhov.de> <3FFD7213.1050903@hp.com> <3FFD848C.25CAC0E8@terekhov.de> <20040108163215.GI23908@diablo.name> <3FFD8ACF.583E5AC3@terekhov.de> <20040108174011.GA28721@diablo.name> <3FFD9D6B.52E97B36@web.de> Message-ID: <20040108212203.GC372@diablo.name> OK. I now see from your example that intervened that you seem to intend that functions that are asyn-cancel safe/unsafe have some syntactic tag (like "const") so that the compiler can check them. I was assuming that you would rely on the compiler to infer which regions of code might be subject to asynch cancellation from the embedded async_cancel{} and then work back the call trees to infer which other functions might be unsafe. It did not occur to me that you would be willing to require users to go so far in marking up their code. --Ted On Thu, Jan 08, 2004 at 07:11:55PM +0100, Alexander Terekhov wrote: > > Ted Baker wrote: > [...] > > I thought you meant *compile* resolution of the overloading, so as > > to generate different call contexts of the same function name. > > Since your async_cancel{} code can have nested calls to arbitrary > > (separately compiled) code, you can't do that without > ^ > | > async-cancel-safe -----+ (nested sync_cancel {} and no_cancel {} > aside for a moment). > > > interprocedural control flow analysis (across separately compiled > > modules), which I don't believe gcc has any plans to do. If you > > are talking about dynamic (runtime) dispatching, then you are ... > > I'm NOT talking about dynamic (runtime) dispatching. Think of > const correctness. From rth at redhat.com Thu Jan 8 21:36:55 2004 From: rth at redhat.com (Richard Henderson) Date: Thu, 8 Jan 2004 13:36:55 -0800 Subject: [c++-pthreads] thread-safety definition In-Reply-To: <3FFD6CA8.B5CEF27C@terekhov.de> References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD4034.7050903@hp.com> <1073567261.28360.258.camel@felix.inria.fr> <3FFD6355.80200@hp.com> <3FFD6CA8.B5CEF27C@terekhov.de> Message-ID: <20040108213655.GB31384@redhat.com> On Thu, Jan 08, 2004 at 03:43:52PM +0100, Alexander Terekhov wrote: > C++ aside for a moment, I have yet to see async-cancel-UNsafe > implementation of, say, strlen(). Care to inspire me? TIA. I have yet to see an async-safe exception runtime. It's certainly possible, but it requires the addition of lots of memory barriers to the code. Which no one adds because all C++ exceptions are synchronous [1]. Even Java exceptions are synchronous; the times that they begin propagation from signal handlers are all synchronous signals like SIGSEGV or SIGFPE, and the compiler knows which instructions might produce them. Thus I believe you will find that async cancelation does *not* actually invoke cleanup handlers in existing implementations. r~ [1] Ada exceptions are async, but that language explicitly denys some of the data availability guarantees that other languages make, which allows you to avoid the memory barriers. From baker at cs.fsu.edu Thu Jan 8 21:53:10 2004 From: baker at cs.fsu.edu (Ted Baker) Date: Thu, 8 Jan 2004 16:53:10 -0500 Subject: [c++-pthreads] thread-safety definition In-Reply-To: <20040108213655.GB31384@redhat.com> References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD4034.7050903@hp.com> <1073567261.28360.258.camel@felix.inria.fr> <3FFD6355.80200@hp.com> <3FFD6CA8.B5CEF27C@terekhov.de> <20040108213655.GB31384@redhat.com> Message-ID: <20040108215310.GE372@diablo.name> > [1] Ada exceptions are async, but that language explicitly denys some > of the data availability guarantees that other languages make, which > allows you to avoid the memory barriers. Right. Both in Ada 83 and Ada 95 there was a lot of time and intelletual effort devoted to the rules regarding what one can assume has been done/undone when an exception is raised. We implemented asynchronous transfers of control in GNARL, and they tested out OK to the extent the GNAT regression tests and Ada validation tests exercise them. On the other hand, knowing what goes on underneath (essentially an longjmp, with unwinding, from a signal handler to the recovery point) and how much difficulty we had making the implementation work, I have cooled considerably on the idea of *using* asynchronous exceptions (and, to a lesser extent, exceptions in general). The opportunities for surpises are just too many. --Ted From wil at bogo.xs4all.nl Fri Jan 9 06:01:31 2004 From: wil at bogo.xs4all.nl (Wil Evers) Date: Fri, 09 Jan 2004 07:01:31 +0100 Subject: [c++-pthreads] Re: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFD7DD5.9090801@bogo.xs4all.nl> <3FFD8046.3000702@hp.com> Message-ID: <3FFE43BB.6040809@bogo.xs4all.nl> Dave Butenhof wrote: > Wil Evers wrote: > >> Dave Butenhof wrote: >> >>> If cancel is implemented completely as a "full stature" C++ >>> exception, and none of the "exception safe" code does anything silly >>> like "catch(...)" without a re-throw, then, yes, it's likely to be >>> reasonably cancel-safe. >> >> A catch(...) without a re-throw isn't necessarily a silly thing to do, >> especially in a destructor. Sometimes, the only alternative is to >> risk a call to terminate(). > > But it still breaks cancellation. Again, ignoring cancel entirely may be > "cancel safe" in some trivial sense, but it's not useful or reasonable > under nearly all circumstances. I'm not suggesting that catching a cancellation request and not rethrowing it is a good thing. I'm trying to point out that - sometimes - the C++ language rules leave with me no other choice. A second exception escaping from a destructor called while unwinding the stack because of some earlier exception will result in program termination. If catching (and not rethrowing) this second exception breaks the cancellation machinery, then it is the cancellation machinery - and not the program in question - that is broken. In other words: we need a way to prevent this scenario. - Wil From rth at redhat.com Fri Jan 9 08:20:36 2004 From: rth at redhat.com (Richard Henderson) Date: Fri, 9 Jan 2004 00:20:36 -0800 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: <3FFE43BB.6040809@bogo.xs4all.nl> References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFD7DD5.9090801@bogo.xs4all.nl> <3FFD8046.3000702@hp.com> <3FFE43BB.6040809@bogo.xs4all.nl> Message-ID: <20040109082036.GA31026@redhat.com> On Fri, Jan 09, 2004 at 07:01:31AM +0100, Wil Evers wrote: > A second exception escaping from a destructor called while unwinding > the stack because of some earlier exception will result in program > termination. > > If catching (and not rethrowing) this second exception breaks the > cancellation machinery, then it is the cancellation machinery - and not > the program in question - that is broken. In other words: we need a way > to prevent this scenario. It would be trivial to adjust the c++ runtime to disable cancelation while exceptions are in flight. That avoids that scenario entirely. r~ From baker at cs.fsu.edu Fri Jan 9 11:20:28 2004 From: baker at cs.fsu.edu (Ted Baker) Date: Fri, 9 Jan 2004 06:20:28 -0500 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: <20040109082036.GA31026@redhat.com> References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFD7DD5.9090801@bogo.xs4all.nl> <3FFD8046.3000702@hp.com> <3FFE43BB.6040809@bogo.xs4all.nl> <20040109082036.GA31026@redhat.com> Message-ID: <20040109112028.GA12343@diablo.name> > It would be trivial to adjust the c++ runtime to disable cancelation > while exceptions are in flight. That avoids that scenario entirely. That is what Gnat does. --Ted From David.Butenhof at hp.com Fri Jan 9 12:20:18 2004 From: David.Butenhof at hp.com (Dave Butenhof) Date: Fri, 09 Jan 2004 07:20:18 -0500 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> Message-ID: <3FFE9C82.3060500@hp.com> David Abrahams wrote: >>Ah. I see the problem. I believe that you neglected to file in >>triplicate the official "List of all matters with which I disagree so >>please don't shout when discussing them" form. Didn't you? ;-) >> >> >I filed. It was *you* who neglected to check with the department of >archival non-shouting topic storage, wasn't it? > > Aw, shoot. I forgot to pay the bill on my automatic update service, and it expired! Sorry 'bout that. >>>If you make cancellations behave sufficiently like an ordinary C++ >>>exception (either of Nathan's or Jason's models would do that I think) >>>then it's neither "IMPOSSIBLE" nor even difficult. A great deal of >>>thread-safe exception-safe C++ library code would behave perfectly >>>well under those conditions. >>> >>Well, it should be no surprise that this discussion has fragmented >>into too many different little pathways for anyone to keep track of >>them. >> >>If cancel is implemented completely as a "full stature" C++ exception, >>and none of the "exception safe" code does anything silly like >>"catch(...)" without a re-throw, then, yes, it's likely to be >>reasonably cancel-safe. >> >> >Even if only *some* of it does that, Nathan and Jason's models are >both likely to result in an eventual cancellation. > > I need to start keeping a scorecard. Lessee here, Nathan's model I believe could be trivially characterized as "return failure status on any cancellation point until you hit a C++ function allowed to throw". This would apparently disable any cleanup operations involving cancellation points (they'd fail) at least prior to the first throw point -- and it's not precisely clear how the "pending cancel" state is to be managed even beyond that point. While it might eventually lead to cancellation, it won't reliably perform cleanup. This seems to me to be more complicated, less predictable, less compatible, and less reliable than simply using exceptions integrated with the existing POSIX cleanup mechanism. Jason's model, putting together the puzzle pieces from various messages here, appears to be basing cancel on an ordinary ("finalizable") C++ exception, which would be raised by any and all cancellation points; but if the exception object were to be destroyed (by finalization), the "pending cancel" would (might?) be reasserted. I personally feel that it ought to be reasonable to finalize the cancel and continue operation. If that wasn't really what the application intended then it's an application error; but there are cases where it's arguably reasonable, and the capability falls naturally out of the model. Besides, "reasserting cancel" in all the right places and no other places sounds like one of those projects that ends up being a lot more subtle and error-prone than anyone ever expects. ;-) >>(Though some code might need to be aware that an uncaught cancel >>will terminate the thread rather than the process!) >> >> >Remember, we're talking about library code. Usually it's only used >"on the inside" of threads so it doesn't care whether when it's >terminated, it's the process or only a thread. > >>But I was really responding to the ideas about turning cancel into a >>return status >> >> >...only for 'C' functions which ordinarily have a return status and don't >throw... did you miss that? > It doesn't matter. There are many problems lurking here, and only one of the more obvious is that this attempt to avoid breaking C code that depends on 'if (error) {cleanup(); return error;}' will still break if 'cleanup()' depends on any cancellation points... which in fact is quite likely if 'error' originates from a cancellation point (e.g., I/O). I also don't believe that a sufficient body of library code really follows the simple and direct 'if (error) return error;' pattern to make this strategy particularly useful even without the cleanup issues -- but that's a different matter. (It might actually be better if the pending cancel wasn't "sticky", since then the cleanup at least might succeed; but then you'll lose cancels all over the place, and that's definitely not acceptable. It's fine for a thread to CHOOSE not to accept a cancel, either by keeping cancellation disabled or by not calling a cancellation point; but it's not OK to lose a delivered cancel. You can declare this to be the application/library developer's job rather than the implementation's job -- but that just means all this "broken" C code that fails to propagate errors needs to be redesigned; and you haven't solved any real problem. That's the basic point: it needs to be analyzed and likely redesigned to be cancel-safe. You simply can't avoid that requirement.) A) You cannot transparently make non cancel-safe C code cancel-safe, no matter how you try. B) Delivering cancellation as a C++ exception will make much (though not all) exception-safe C++ "translucently" cancel-safe. C) POSIX C cancel-safe code is already cancel-safe, but for an application containing mixed call stacks to remain cancel-safe, POSIX cleanup and C++ exceptions must be integrated into a single unwind; i.e., (by whatever name) basing both on a common exception library. D) There are no useful or interestingly large bodies of rigorously "cancel-safe" C code that are not POSIX cancel-safe. In other words, cancel-as-exception, and pthread_cleanup_push as a C-based "try" extension, gives you all the cancel-safety you can reasonably expect without recoding (and likely redesign). That's all that's needed. No less will suffice, and any more is needless complication. Non cancel-safe code will continue to work just fine as long as it's NOT cancelled -- which ought to be no terrible restriction since it was clearly written in and for an environment where cancellation either did not exist or wasn't needed. -- /--------------------[ David.Butenhof at hp.com ]--------------------\ | Hewlett-Packard Company Tru64 UNIX & VMS Thread Architect | | My book: http://www.awl.com/cseng/titles/0-201-63392-2/ | \----[ http://homepage.mac.com/dbutenhof/Threads/Threads.html ]---/ From boo at terekhov.de Fri Jan 9 13:20:12 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Fri, 09 Jan 2004 14:20:12 +0100 Subject: [c++-pthreads] thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD4034.7050903@hp.com> <1073567261.28360.258.camel@felix.inria.fr> <3FFD6355.80200@hp.com> <3FFD6CA8.B5CEF27C@terekhov.de> <20040108213655.GB31384@redhat.com> Message-ID: <3FFEAA8C.7085265B@terekhov.de> Richard Henderson wrote: > > On Thu, Jan 08, 2004 at 03:43:52PM +0100, Alexander Terekhov wrote: > > C++ aside for a moment, I have yet to see async-cancel-UNsafe > > implementation of, say, strlen(). Care to inspire me? TIA. > > I have yet to see an async-safe exception runtime. It's certainly > possible, but it requires the addition of lots of memory barriers > to the code. Huh? Let's talk about memory barriers next week, okay? http://www.terekhov.de/pthread_refcount_t/draft-edits.txt http://groups.google.com/groups?selm=3EEF38E8.311EEC77%40web.de http://www.terekhov.de/pthread_refcount_t/experimental/refcount.cpp > Which no one adds because all C++ exceptions are > synchronous [1]. Even Java exceptions are synchronous; the times Well, http://www.rtj.org/rtsj-V1.0.pdf "When a method is declared with AsynchronouslyInterruptedException in its throws clause the platform is expected to asynchronously throw this exception if RealtimeThread.interrupt() is called while the method is executing, or if such an interrupt is pending any time control returns to the method. The interrupt is not thrown while any methods it invokes are executing, unless they are, in turn, declared to throw the exception. This is intended to allow long-running computations to be terminated without the overhead or latency of polling with java.lang.Thread.interrupted(). The throws AsynchronouslyInterruptedException clause is a marker on a stack frame which allows a method to be statically marked as asynchronously interruptible." regards, alexander. P.S. http://tinyurl.com/2lfoz From boo at terekhov.de Fri Jan 9 14:40:59 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Fri, 09 Jan 2004 15:40:59 +0100 Subject: [c++-pthreads] Re: thread-safety definition References: <3FFD4034.7050903@hp.com> <1073567261.28360.258.camel@felix.inria.fr> <3FFD6355.80200@hp.com> <3FFD6CA8.B5CEF27C@terekhov.de> <3FFD7213.1050903@hp.com> <3FFD848C.25CAC0E8@terekhov.de> <20040108163215.GI23908@diablo.name> <3FFD8ACF.583E5AC3@terekhov.de> <20040108174011.GA28721@diablo.name> <3FFD9D6B.52E97B36@web.de> <20040108212203.GC372@diablo.name> Message-ID: <3FFEBD7B.FCCAD883@terekhov.de> Ted Baker wrote: > > OK. I now see from your example that intervened that you seem to > intend that functions that are asyn-cancel safe/unsafe have some > syntactic tag (like "const") so that the compiler can check them. > I was assuming that you would rely on the compiler to infer which > regions of code might be subject to asynch cancellation from the > embedded async_cancel{} and then work back the call trees to > infer which other functions might be unsafe. It did not occur > to me that you would be willing to require users to go so far > in marking up their code. http://google.com/groups?threadm=3D293F54.8FFA810%40web.de > How much knowledge about the standard libraries are you > willing to bring into the language? As much as given in the function prototype - no more, no less. The problem is, however, there is no way to give this information in function prototype in current language. regards, alexander. From boo at terekhov.de Fri Jan 9 14:42:35 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Fri, 09 Jan 2004 15:42:35 +0100 Subject: [c++-pthreads] Re: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFD7DD5.9090801@bogo.xs4all.nl> <3FFD8046.3000702@hp.com> <3FFE43BB.6040809@bogo.xs4all.nl> Message-ID: <3FFEBDDB.673AE752@terekhov.de> Wil Evers wrote: [...] > If catching (and not rethrowing) this second exception breaks the > cancellation machinery, then it is the cancellation machinery - and not > the program in question - that is broken. In other words: we need a way > to prevent this scenario. Here's a sort of "current way" to prevent this scenario: http://www.terekhov.de/DESIGN-futex-CV.cpp class cancel_off_guard { //*** unimplemented since it's non-copyable/non-copy-constructible cancel_off_guard(const cancel_off_guard &); cancel_off_guard & operator=(const cancel_off_guard &); int m_old_cancel_state; public: cancel_off_guard() { int status = pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, &m_old_cancel_state); assert(!status); } ~cancel_off_guard() { int status = pthread_setcancelstate(m_old_cancel_state, &m_old_cancel_state); assert(!status); } }; //*** class cancel_off_guard ~futex_condvar() { mutex::guard guard(m_mutex); assert(m_waiters[0] == m_wakeups); while (m_waiters[0]) { int ftx = m_futex = EOC(); mutex::release_guard release_guard(guard); cancel_off_guard no_cancel; m_futex.wait(ftx); } } However, the standard mandated mandatory 2-phase EH (with fixed exception specs/implicit throw() ES for dtors) and "intelligent" cancellation points (and async-cancel-regions, of course) is the way to go, I believe strongly. regards, alexander. From jason at redhat.com Sat Jan 10 02:41:14 2004 From: jason at redhat.com (Jason Merrill) Date: Fri, 09 Jan 2004 21:41:14 -0500 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: <3FFE9C82.3060500@hp.com> (Dave Butenhof's message of "Fri, 09 Jan 2004 07:20:18 -0500") References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> Message-ID: On Fri, 09 Jan 2004 07:20:18 -0500, Dave Butenhof wrote: > Jason's model, putting together the puzzle pieces from various messages > here My original formulation is at http://www.codesourcery.com/archives/c++-pthreads/msg00059.html > , appears to be basing cancel on an ordinary ("finalizable") C++ > exception, which would be raised by any and all cancellation points; but if > the exception object were to be destroyed (by finalization), the "pending > cancel" would (might?) be reasserted. I personally feel that it ought to be > reasonable to finalize the cancel and continue operation. If that wasn't > really what the application intended then it's an application error; but > there are cases where it's arguably reasonable, and the capability falls > naturally out of the model. Too naturally, IMO. There seems to be a substantial amount of C++ code out there which traps and finalizes all exceptions, some of it in the C++ standard library I/O classes. I disagree with this design, and propose to change it in the message cited above, but it's just one example. I don't think it's reasonable for a cancellation request to be silently discarded just because the thread was in the middle of doing some formatted output when the cancel was received. I wouldn't object to a facility for explicitly abandoning the cancel, but I think that allowing it to fall out of the model is far too fragile. > Besides, "reasserting cancel" in all the right places and no other places > sounds like one of those projects that ends up being a lot more subtle > and error-prone than anyone ever expects. ;-) Assuming support for abanoning the cancel, why would it be more complex than pthread_cancel (pthread_self ()); ? Jason From dave at boost-consulting.com Sat Jan 10 04:01:15 2004 From: dave at boost-consulting.com (David Abrahams) Date: Fri, 09 Jan 2004 23:01:15 -0500 Subject: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> Message-ID: Dave Butenhof writes: >>>>If you make cancellations behave sufficiently like an ordinary C++ >>>>exception (either of Nathan's or Jason's models would do that I think) >>>>then it's neither "IMPOSSIBLE" nor even difficult. A great deal of >>>>thread-safe exception-safe C++ library code would behave perfectly >>>>well under those conditions. >>>> >>>Well, it should be no surprise that this discussion has fragmented >>>into too many different little pathways for anyone to keep track of >>>them. >>> >>>If cancel is implemented completely as a "full stature" C++ exception, >>>and none of the "exception safe" code does anything silly like >>>"catch(...)" without a re-throw, then, yes, it's likely to be >>> reasonably cancel-safe. >>> >>Even if only *some* of it does that, Nathan and Jason's models are >>both likely to result in an eventual cancellation. >> > I need to start keeping a scorecard. Lessee here, Nathan's model I > believe could be trivially characterized as "return failure status on > any cancellation point until you hit a C++ function allowed to > throw". This would apparently disable any cleanup operations involving > cancellation points (they'd fail) at least prior to the first throw > point -- and it's not precisely clear how the "pending cancel" state > is to be managed even beyond that point. While it might eventually > lead to cancellation, it won't reliably perform cleanup. I'm having trouble understanding. How does a "pending cancel state" perform cleanup? And what do "reliably" and "cleanup" mean to you? > This seems to me to be more complicated, less predictable, less > compatible, and less reliable than simply using exceptions > integrated with the existing POSIX cleanup mechanism. That's a rather broad sweep of the judgement brush, accompanied by little to lend any insight into why you think it's so. > Jason's model, putting together the puzzle pieces from various > messages here, appears to be basing cancel on an ordinary > ("finalizable") C++ exception, which would be raised by any and all > cancellation points; but if the exception object were to be destroyed > (by finalization), the "pending cancel" would (might?) be > reasserted. I personally feel that it ought to be reasonable to > finalize the cancel and continue operation. If that wasn't really what > the application intended then it's an application error; but there are > cases where it's arguably reasonable, and the capability falls > naturally out of the model. I don't have a strong opinion about that, but I'm sympathetic to your POV. > Besides, "reasserting cancel" in all the right places and no other > places sounds like one of those projects that ends up being a lot > more subtle and error-prone than anyone ever expects. ;-) I don't really see what you think is complicated about this. In Jason's model cancellation gets reasserted as soon as a caught cancellation exception is destroyed by falling off the end of the catch block without rethrowing. >>>But I was really responding to the ideas about turning cancel into a >>>return status >>> >>> >>...only for 'C' functions which ordinarily have a return status and don't >>throw... did you miss that? >> > It doesn't matter. There are many problems lurking here, and only one > of the more obvious is that this attempt to avoid breaking C code that > depends on 'if (error) {cleanup(); return error;}' will still break if > 'cleanup()' depends on any cancellation points... which in fact is > quite likely if 'error' originates from a cancellation point (e.g., > I/O). In what sense will it break? > I also don't believe that a sufficient body of library code > really follows the simple and direct 'if (error) return error;' > pattern to make this strategy particularly useful even without the > cleanup issues -- but that's a different matter. Before I used exceptions I certainly wrote reams and reams of code just like that. > (It might actually be > better if the pending cancel wasn't "sticky", You lost me. What's "sticky"? > since then the cleanup at least might succeed; but then you'll lose > cancels all over the place, and that's definitely not > acceptable. AFAICT you haven't understood Jason's proposal at all, though I may be misinterpreting your response. > It's fine for a thread to CHOOSE not to accept a cancel, > either by keeping cancellation disabled or by not calling a > cancellation point; but it's not OK to lose a delivered cancel. You > can declare this to be the application/library developer's job > rather than the implementation's job -- but that just means all this > "broken" C code that fails to propagate errors needs to be > redesigned; and you haven't solved any real problem. That's the > basic point: it needs to be analyzed and likely redesigned to be > cancel-safe. You simply can't avoid that requirement.) > > A) You cannot transparently make non cancel-safe C code cancel-safe, > no matter how you try. It depends how "cancel" is implemented I guess. > B) Delivering cancellation as a C++ exception will make much (though > not all) exception-safe C++ "translucently" cancel-safe. Sure, provided that you only deliver it from functions which could otherwise throw. > C) POSIX C cancel-safe code is already cancel-safe, but for an > application containing mixed call stacks to remain cancel-safe, POSIX > cleanup and C++ exceptions must be integrated into a single unwind; > i.e., (by whatever name) basing both on a common exception library. > D) There are no useful or interestingly large bodies of rigorously > "cancel-safe" C code that are not POSIX cancel-safe. > > In other words, cancel-as-exception, and pthread_cleanup_push as a > C-based "try" extension, gives you all the cancel-safety you can > reasonably expect without recoding (and likely redesign). That's all > that's needed. No less will suffice, and any more is needless > complication. Except for the fact that we have normally-non-throwing functions that have started to throw, I don't think anyone's disagreeing with you. -- Dave Abrahams Boost Consulting www.boost-consulting.com From wil at bogo.xs4all.nl Sat Jan 10 08:19:43 2004 From: wil at bogo.xs4all.nl (Wil Evers) Date: Sat, 10 Jan 2004 09:19:43 +0100 Subject: [c++-pthreads] Re: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> Message-ID: <3FFFB59F.60002@bogo.xs4all.nl> David Abrahams wrote: > Dave Butenhof writes: > >>Besides, "reasserting cancel" in all the right places and no other >>places sounds like one of those projects that ends up being a lot >>more subtle and error-prone than anyone ever expects. ;-) > > I don't really see what you think is complicated about this. In > Jason's model cancellation gets reasserted as soon as a caught > cancellation exception is destroyed by falling off the end of the > catch block without rethrowing. What if, at that point, the thread has disabled cancellation? - Wil From dave at boost-consulting.com Sat Jan 10 13:47:18 2004 From: dave at boost-consulting.com (David Abrahams) Date: Sat, 10 Jan 2004 08:47:18 -0500 Subject: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> <3FFFB59F.60002@bogo.xs4all.nl> Message-ID: Wil Evers writes: > David Abrahams wrote: > >> Dave Butenhof writes: >> >>>Besides, "reasserting cancel" in all the right places and no other >>>places sounds like one of those projects that ends up being a lot >>>more subtle and error-prone than anyone ever expects. ;-) >> I don't really see what you think is complicated about this. In >> Jason's model cancellation gets reasserted as soon as a caught >> cancellation exception is destroyed by falling off the end of the >> catch block without rethrowing. > > What if, at that point, the thread has disabled cancellation? Naturally, no cancellation exceptions are thrown until cancellation is re-enabled (I think). -- Dave Abrahams Boost Consulting www.boost-consulting.com From baker at cs.fsu.edu Sat Jan 10 15:10:40 2004 From: baker at cs.fsu.edu (Ted Baker) Date: Sat, 10 Jan 2004 10:10:40 -0500 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: <3FFFB59F.60002@bogo.xs4all.nl> References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> <3FFFB59F.60002@bogo.xs4all.nl> Message-ID: <20040110151040.GB13609@diablo.name> > >I don't really see what you think is complicated about this. In > >Jason's model cancellation gets reasserted as soon as a caught > >cancellation exception is destroyed by falling off the end of the > >catch block without rethrowing. > > What if, at that point, the thread has disabled cancellation? First, if the thread had disabled cancellation before the the cancellation exception was raised, you could not get to this point. Therefore, you are only asking about the case where a thread disables cancellation in a cleanup routine or destructor and does not reenable it. In this case you still want to "reassert" the cancellation at the next point control leaves a handler/catch block without rethrowing. --Ted From baker at cs.fsu.edu Sat Jan 10 15:20:08 2004 From: baker at cs.fsu.edu (Ted Baker) Date: Sat, 10 Jan 2004 10:20:08 -0500 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> <3FFFB59F.60002@bogo.xs4all.nl> Message-ID: <20040110152008.GC13609@diablo.name> > >> Jason's model cancellation gets reasserted as soon as a caught > >> cancellation exception is destroyed by falling off the end of the > >> catch block without rethrowing. > > > > What if, at that point, the thread has disabled cancellation? > > Naturally, no cancellation exceptions are thrown until cancellation is > re-enabled (I think). BTW, it is important to distinguish between user-level enabling/disabling of cancellation and what may go on in the language implementation. You do not want some erroneous user code (e.g., a cleanup routine that disables cancellation) to break the language implementation of exception propagation. Therefore, the implementor may provide a lower-level mechanism, that supersedes or ignores whatever cancellation state the user set. The user-level cancellation enable/disable bit essentially just tells whether the various cancellation point routines should initiate cancellation processing if there is a pending cancellation, e.g., if (self->cancellation_enabled && self->cancellation_state == pending) { self->cancellation_state = cancelling; throw_exception (cancellation); // a runtime system call } This does not prevent the language runtime from *re*initiating cancellation processing at other points where it makes sense, e.g., if (self->cancellation_state == cancelling) throw_exception (cancellation); --Ted From baker at cs.fsu.edu Sat Jan 10 15:39:50 2004 From: baker at cs.fsu.edu (Ted Baker) Date: Sat, 10 Jan 2004 10:39:50 -0500 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> Message-ID: <20040110153950.GD13609@diablo.name> > Except for the fact that we have normally-non-throwing functions that > have started to throw, I don't think anyone's disagreeing with you. This still perplexes me. How does a "normally-non-throwing function" "start to throw"? The examples I recall seeing here are of C/C++ library functions that make system calls via C library routines that are cancellation points. 1) It is normal for these C library routines to be cancellation points, in any POSIX/Unix/DCE multithreaded program. Cancellation is semantically equivalent to an exception, so these are not normally-non-throwing functions. 2) It may be that the existing implementations of some other C++ library functions call such potentially-throwing C library functions, and these C++ library functions are not documented as potentially throwing exceptions. These functions do start to throw exceptions if you move from a multithreading environment that does not support cancellation to one that does. If that is the only problem, it seems you have a few possible alternative solutions: A) Modify the C++ library function implementations so that they call "safe" versions of the C library functions, that are not cancellation points. This could be done by either changing the sources of the C++ libraries to call the safe versions by name, or by using macro or linker-level renaming to do the substitution. B) Change the documentations (maybe standards) for the affected C++ library functions, to specify that they may throw thread cancellation when cancellation is enabled. C) Disable cancellation by default in all C++ threads, and warn users that enabling cancellation may cause some functions, not documented as throwing, to throw cancellation. Beyond that, I get the impression this group is asking for something that is impossible because it is self-contradictory, i.e., for a way to (1) allow cancellation, (2) ensure (1) cancellation cannot be ignored for long or lost, (3) not allow throwing of cancellation from certain library calls, (4) not make any change to existing code for those library functions (which were written without cancellation in mind). --Ted From dave at boost-consulting.com Sat Jan 10 16:15:45 2004 From: dave at boost-consulting.com (David Abrahams) Date: Sat, 10 Jan 2004 11:15:45 -0500 Subject: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> <20040110153950.GD13609@diablo.name> Message-ID: Ted Baker writes: >> Except for the fact that we have normally-non-throwing functions that >> have started to throw, I don't think anyone's disagreeing with you. > > This still perplexes me. How does a "normally-non-throwing function" > "start to throw"? You port the code which calls the function into an environment which changes the rules about what can throw exceptions. -- Dave Abrahams Boost Consulting www.boost-consulting.com From wil at bogo.xs4all.nl Sat Jan 10 21:19:41 2004 From: wil at bogo.xs4all.nl (Wil Evers) Date: Sat, 10 Jan 2004 22:19:41 +0100 Subject: [c++-pthreads] Re: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFD7DD5.9090801@bogo.xs4all.nl> <3FFD8046.3000702@hp.com> <3FFE43BB.6040809@bogo.xs4all.nl> <20040109082036.GA31026@redhat.com> Message-ID: <40006C6D.5090707@bogo.xs4all.nl> Richard Henderson wrote: > On Fri, Jan 09, 2004 at 07:01:31AM +0100, Wil Evers wrote: > >>A second exception escaping from a destructor called while unwinding >>the stack because of some earlier exception will result in program >>termination. >> >>If catching (and not rethrowing) this second exception breaks the >>cancellation machinery, then it is the cancellation machinery - and not >>the program in question - that is broken. In other words: we need a way >>to prevent this scenario. > > It would be trivial to adjust the c++ runtime to disable cancelation > while exceptions are in flight. That avoids that scenario entirely. It obviously does. But does that imply we don't need 'catch-and-dont-rethrow' blocks in destructors anymore? Please consider this example: remote_resource_holder::~remote_resource_holder() { remote_resource->release(); } Disabling cancellation while exceptions are 'in flight' means we can't run into a double fault because of a cancellation request detected in remote_resource->release() while unwinding on behalf of some previously raised exception. However, we may also be unwinding on behalf of some previously detected cancellation request, and it that case, a networking exception thrown from within remote_resource->release() *will* cause the program to terminate. Please note that this problem is not caused by the use of exceptions for handling cancellation; any other previously raised exception could lead to the same disaster scenario - which is why thinking about cancellation as some special kind of exception is unsufficient. Instead, this example serves to show how dangerous it is to let exceptions escape from destructors. Especially when we don't now which exceptions may be thrown from remote_resource->release(), about the only reasonable implementation of this destructor is: remote_resource_holder::~remote_resource_holder() { try { remote_resource->release(); } catch (...) { // We may try to log what happened, // but can't let the exception escape! } } I've noticed that some people on this list object to this design, but so far, I haven't seen an alternative. In other words, I think we should be prepared for threads that discard cancellation exceptions - which is why Nathan's 'sticky cancellation' makes sense to me. - Wil From boo at terekhov.de Sat Jan 10 22:27:29 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Sat, 10 Jan 2004 23:27:29 +0100 Subject: [c++-pthreads] Re: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFD7DD5.9090801@bogo.xs4all.nl> <3FFD8046.3000702@hp.com> <3FFE43BB.6040809@bogo.xs4all.nl> <20040109082036.GA31026@redhat.com> <40006C6D.5090707@bogo.xs4all.nl> Message-ID: <40007C51.FF744536@terekhov.de> Wil Evers wrote: [...] > remote_resource_holder::~remote_resource_holder() > { > try { > remote_resource->release(); > } catch (...) { > // We may try to log what happened, > // but can't let the exception escape! > } > } > > I've noticed that some people on this list object to this design, but so > far, I haven't seen an alternative. http://google.com/groups?selm=3E734A5F.8A7F5B56%40web.de http://google.com/groups?selm=3E76FB8A.A9C88386%40web.de regards, alexander. From baker at cs.fsu.edu Sat Jan 10 22:44:22 2004 From: baker at cs.fsu.edu (Ted Baker) Date: Sat, 10 Jan 2004 17:44:22 -0500 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: <40006C6D.5090707@bogo.xs4all.nl> References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFD7DD5.9090801@bogo.xs4all.nl> <3FFD8046.3000702@hp.com> <3FFE43BB.6040809@bogo.xs4all.nl> <20040109082036.GA31026@redhat.com> <40006C6D.5090707@bogo.xs4all.nl> Message-ID: <20040110224422.GA19106@diablo.name> > Instead, this example serves to show how dangerous it is to let > exceptions escape from destructors. Especially when we don't now which > exceptions may be thrown from remote_resource->release(), about the only > reasonable implementation of this destructor is: > > remote_resource_holder::~remote_resource_holder() > { > try { > remote_resource->release(); > } catch (...) { > // We may try to log what happened, > // but can't let the exception escape! > } > } > > I've noticed that some people on this list object to this design, but so > far, I haven't seen an alternative. In other words, I think we should > be prepared for threads that discard cancellation exceptions - which is > why Nathan's 'sticky cancellation' makes sense to me. There is no real problem here. 1) Yes, you want to be able to catch all *local* exceptions, and not rethrow them, in destructors and cancellation cleanup routines. If you disable cancellation during the execution of such routines you will never have a cancellation exception that would be caught by the "catch" of your destructor example above. 2) This does not mean you can't disable the *generation* of a (new) cancellation exception during execution of such a destructor or cleanup routine. 3) This does not mean you will lose an old cancellation exception, if that is what caused you to execute the destructor or cleanup routine. On return from this routine, execution will continue along the path that took you into this routine, i.e., the processing of the old cancellation exception. 4) This does not mean you will lose a new cancellation exception, if you were executing the destructor or cleanup routine for some other reason. Whenever you next get to a point where cancellation is enabled again, the cancellation processing will begin. It does mean that you need to be able to keep track of nested cancellation-disabled sections, so that you can disable cancellation during the entire exception processing sequence, up until you are in the "catch", and you can *also* (nestedly) disable cancellation during destructors. --Ted From wil at bogo.xs4all.nl Sun Jan 11 08:20:37 2004 From: wil at bogo.xs4all.nl (Wil Evers) Date: Sun, 11 Jan 2004 09:20:37 +0100 Subject: [c++-pthreads] Re: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFD7DD5.9090801@bogo.xs4all.nl> <3FFD8046.3000702@hp.com> <3FFE43BB.6040809@bogo.xs4all.nl> <20040109082036.GA31026@redhat.com> <40006C6D.5090707@bogo.xs4all.nl> <20040110224422.GA19106@diablo.name> Message-ID: <40010755.30702@bogo.xs4all.nl> Ted Baker wrote: >>Instead, this example serves to show how dangerous it is to let >>exceptions escape from destructors. Especially when we don't now which >>exceptions may be thrown from remote_resource->release(), about the only >>reasonable implementation of this destructor is: >> >>remote_resource_holder::~remote_resource_holder() >>{ >> try { >> remote_resource->release(); >> } catch (...) { >> // We may try to log what happened, >> // but can't let the exception escape! >> } >>} >> >>I've noticed that some people on this list object to this design, but so >>far, I haven't seen an alternative. In other words, I think we should >>be prepared for threads that discard cancellation exceptions - which is >>why Nathan's 'sticky cancellation' makes sense to me. > > > There is no real problem here. > > 1) Yes, you want to be able to catch all *local* exceptions, and > not rethrow them, in destructors and cancellation cleanup routines. > > If you disable cancellation during the execution of such routines > you will never have a cancellation exception that would be caught > by the "catch" of your destructor example above. > > 2) This does not mean you can't disable the *generation* of a (new) > cancellation exception during execution of such a destructor or > cleanup routine. > > 3) This does not mean you will lose an old cancellation exception, > if that is what caused you to execute the destructor or cleanup > routine. On return from this routine, execution will continue > along the path that took you into this routine, i.e., the > processing of the old cancellation exception. > > 4) This does not mean you will lose a new cancellation exception, if > you were executing the destructor or cleanup routine for some other > reason. Whenever you next get to a point where cancellation is > enabled again, the cancellation processing will begin. > > It does mean that you need to be able to keep track of nested > cancellation-disabled sections, so that you can disable > cancellation during the entire exception processing sequence, up > until you are in the "catch", and you can *also* (nestedly) > disable cancellation during destructors. Please note that the example destructor above is also called on a regular (non-exceptional) exit from the block in which the remote_resource_holder lives. Since, in that case, there is no exception 'in flight', a cancellation exception thrown from remote_source->release() *will* be discarded, unless - in *all* such destructors - the code is changed to explictly disable cancellation. In other words, non-sticky cancellation will break existing code, and the replacement code will not be pretty. IMHO, this is highly undesirable - writing robust destructors is hard enough without these complications. - Wil From jason at redhat.com Sun Jan 11 19:05:01 2004 From: jason at redhat.com (Jason Merrill) Date: Sun, 11 Jan 2004 14:05:01 -0500 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: <40006C6D.5090707@bogo.xs4all.nl> (Wil Evers's message of "Sat, 10 Jan 2004 22:19:41 +0100") References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFD7DD5.9090801@bogo.xs4all.nl> <3FFD8046.3000702@hp.com> <3FFE43BB.6040809@bogo.xs4all.nl> <20040109082036.GA31026@redhat.com> <40006C6D.5090707@bogo.xs4all.nl> Message-ID: On Sat, 10 Jan 2004 22:19:41 +0100, Wil Evers wrote: > Richard Henderson wrote: > >> On Fri, Jan 09, 2004 at 07:01:31AM +0100, Wil Evers wrote: >> >>>A second exception escaping from a destructor called while unwinding >>>the stack because of some earlier exception will result in program >>>termination. >>> >>> If catching (and not rethrowing) this second exception breaks the >>> cancellation machinery, then it is the cancellation machinery - and not >>> the program in question - that is broken. In other words: we need a way >>> to prevent this scenario. >> It would be trivial to adjust the c++ runtime to disable cancelation >> while exceptions are in flight. That avoids that scenario entirely. > > It obviously does. But does that imply we don't need > 'catch-and-dont-rethrow' blocks in destructors anymore? No, of course not; we can get double faults in non-threaded programs. It does, however, mean that a cancellation exception will never be caught by such a block. >[...] > I've noticed that some people on this list object to this design, but so > far, I haven't seen an alternative. In other words, I think we should be > prepared for threads that discard cancellation exceptions - which is why > Nathan's 'sticky cancellation' makes sense to me. I agree that cancellation should be sticky, and it is in my model as well as Nathan's. But I don't think that this particular example would discard cancellation exceptions. Jason From wil at bogo.xs4all.nl Sun Jan 11 21:03:58 2004 From: wil at bogo.xs4all.nl (Wil Evers) Date: Sun, 11 Jan 2004 22:03:58 +0100 Subject: [c++-pthreads] Re: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFD7DD5.9090801@bogo.xs4all.nl> <3FFD8046.3000702@hp.com> <3FFE43BB.6040809@bogo.xs4all.nl> <20040109082036.GA31026@redhat.com> <40006C6D.5090707@bogo.xs4all.nl> Message-ID: <4001BA3E.7010100@bogo.xs4all.nl> Jason Merrill wrote: > On Sat, 10 Jan 2004 22:19:41 +0100, Wil Evers wrote: > >>Richard Henderson wrote: >> >>>On Fri, Jan 09, 2004 at 07:01:31AM +0100, Wil Evers wrote: >>> >>>>A second exception escaping from a destructor called while unwinding >>>>the stack because of some earlier exception will result in program >>>>termination. >>>> >>>>If catching (and not rethrowing) this second exception breaks the >>>>cancellation machinery, then it is the cancellation machinery - and not >>>>the program in question - that is broken. In other words: we need a way >>>>to prevent this scenario. >>> >>> It would be trivial to adjust the c++ runtime to disable cancelation >>>while exceptions are in flight. That avoids that scenario entirely. >> >>It obviously does. But does that imply we don't need >>'catch-and-dont-rethrow' blocks in destructors anymore? > > No, of course not; we can get double faults in non-threaded programs. It > does, however, mean that a cancellation exception will never be caught by > such a block. > >>[...] >>I've noticed that some people on this list object to this design, but so >>far, I haven't seen an alternative. In other words, I think we should be >>prepared for threads that discard cancellation exceptions - which is why >>Nathan's 'sticky cancellation' makes sense to me. > > I agree that cancellation should be sticky, and it is in my model as well > as Nathan's. But I don't think that this particular example would discard > cancellation exceptions. I think it will. Even if cancellation is disabled while exceptions are in flight, a 'catch-everything-and-never-rethrow' block in a destructor will discard a cancellation exception when that destructor is triggered by a regular (non-exceptional, non-unwinding) exit from a block scope. Call me paranoid, but as a C++ programmer, the thing that scares me most in the discussions on this list is a certain (perceived?) pressure to open the door for letting exceptions escape from destructors. Speaking for myself, all I can say is that such a license would break nearly every piece of non-trivial code I've written over the last eight years or so. - Wil From fjh at cs.mu.oz.au Mon Jan 12 01:02:13 2004 From: fjh at cs.mu.oz.au (Fergus Henderson) Date: Mon, 12 Jan 2004 12:02:13 +1100 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> Message-ID: <20040112010213.GB14371@jupiter.cs.mu.oz.au> On 09-Jan-2004, David Abrahams wrote: > Dave Butenhof writes: > > > Lessee here, Nathan's model I > > believe could be trivially characterized as "return failure status on > > any cancellation point until you hit a C++ function allowed to > > throw". This would apparently disable any cleanup operations involving > > cancellation points (they'd fail) at least prior to the first throw > > point -- and it's not precisely clear how the "pending cancel" state > > is to be managed even beyond that point. While it might eventually > > lead to cancellation, it won't reliably perform cleanup. > > I'm having trouble understanding. How does a "pending cancel state" > perform cleanup? And what do "reliably" and "cleanup" mean to you? I think "it" in the last sentence quote above refers to Nathan's model, not to the pending cancel state. The problem is that cleanup -- e.g. execution of C++ destructors on stack unwinding -- may involve calls to I/O routines that are themselves cancellation points. If these routines fail, then the cleanup functions will not achieve their intended purpose; they will not be able to release the resources that the thread holds. Nathan's model implies that they will fail. > > It doesn't matter. There are many problems lurking here, and only one > > of the more obvious is that this attempt to avoid breaking C code that > > depends on 'if (error) {cleanup(); return error;}' will still break if > > 'cleanup()' depends on any cancellation points... which in fact is > > quite likely if 'error' originates from a cancellation point (e.g., > > I/O). > > In what sense will it break? In this example, the function "cleanup()" may need to do I/O to properly release the resources that the thread holds, but this I/O will fail. > > (It might actually be > > better if the pending cancel wasn't "sticky", > > You lost me. What's "sticky"? When a thread gets a cancellation request, the first cancellation point encountered thereafter will act on that request (by unwinding the stack, or in Nathan's model by returning an error return status and setting errno = ECANCELLED). If another cancellation point is encountered after that, and there has not been another cancellation request, then the cancellation point will normally not have any effect. With "sticky" cancels, after a cancellation request, *every* subsequent cancellation point will act on the cancel (by unwinding the stack, or failing with ECANCELLED). "stick" cancels make it difficult/impossible to clean up properly, since cleaning up may involve doing I/O, and in particular by calling functions that Posix specifies are cancellation points. These functions will fail to do the I/O, because they will instead attempt to act on the cancellation again (i.e. re-raising the cancellation exception or failing with ECANCELLED). -- Fergus Henderson | "I have always known that the pursuit The University of Melbourne | of excellence is a lethal habit" WWW: | -- the last words of T. S. Garp. From dave at boost-consulting.com Mon Jan 12 01:46:24 2004 From: dave at boost-consulting.com (David Abrahams) Date: Sun, 11 Jan 2004 20:46:24 -0500 Subject: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> <20040112010213.GB14371@jupiter.cs.mu.oz.au> Message-ID: Fergus Henderson writes: > When a thread gets a cancellation request, the first > cancellation point encountered thereafter will act on that request > (by unwinding the stack, or in Nathan's model by returning an error > return status and setting errno = ECANCELLED). If another > cancellation point is encountered after that, and there has not > been another cancellation request, then the cancellation point > will normally not have any effect. > > With "sticky" cancels, after a cancellation request, > *every* subsequent cancellation point will act on the cancel > (by unwinding the stack, or failing with ECANCELLED). > > "stick" cancels make it difficult/impossible to clean up properly, > since cleaning up may involve doing I/O, and in particular by calling > functions that Posix specifies are cancellation points. These functions > will fail to do the I/O, because they will instead attempt to act on > the cancellation again (i.e. re-raising the cancellation exception > or failing with ECANCELLED). OK, thanks for the explanation. Sounds like an argument for Jason's exceptions that re-assert cancel when they're destroyed. -- Dave Abrahams Boost Consulting www.boost-consulting.com From boo at terekhov.de Mon Jan 12 10:34:30 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Mon, 12 Jan 2004 11:34:30 +0100 Subject: [c++-pthreads] Re: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> <20040112010213.GB14371@jupiter.cs.mu.oz.au> Message-ID: <40027836.7B9D03BC@terekhov.de> David Abrahams wrote: > > Fergus Henderson writes: > > > When a thread gets a cancellation request, the first > > cancellation point encountered thereafter will act on that request > > (by unwinding the stack, or in Nathan's model by returning an error > > return status and setting errno = ECANCELLED). If another > > cancellation point is encountered after that, and there has not > > been another cancellation request, then the cancellation point > > will normally not have any effect. > > > > With "sticky" cancels, after a cancellation request, > > *every* subsequent cancellation point will act on the cancel > > (by unwinding the stack, or failing with ECANCELLED). > > > > "stick" cancels make it difficult/impossible to clean up properly, In the current POSIX/C++ environment one must disable/restore-old cancellation state in the destructors that call POSIX cancellation points anyway (I mean "general cancel-safe library" stuff). That does ensure proper cleanup. > > since cleaning up may involve doing I/O, and in particular by calling > > functions that Posix specifies are cancellation points. These functions > > will fail to do the I/O, because they will instead attempt to act on > > the cancellation again (i.e. re-raising the cancellation exception > > or failing with ECANCELLED). > > OK, thanks for the explanation. Sounds like an argument for Jason's > exceptions that re-assert cancel when they're destroyed. Making broken code a bit less broken is hardly the right design rationale. Presuming that we'll get a standard named thread cancel request exception, his "fix" (with respect to broken C++ libraries) is nothing but catch(...) { /* ... no re-throw here ... */ try { throw; } catch(std::thread_cancel_request const &) { std::thread_enable_cancel(); // re-enable cancel std::thread_self().cancel(); // re-inject cancel request } catch(...) { } } (unless I'm just missing and/or misunderstanding something). I don't think that the standard should mandate that. The standard should mandate mandatory 2-phase ES and intelligent cancellation points plus async-cancel regions, of course. regards, alexander. From fjh at cs.mu.oz.au Mon Jan 12 11:28:21 2004 From: fjh at cs.mu.oz.au (Fergus Henderson) Date: Mon, 12 Jan 2004 22:28:21 +1100 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: <40027836.7B9D03BC@terekhov.de> References: <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> <20040112010213.GB14371@jupiter.cs.mu.oz.au> <40027836.7B9D03BC@terekhov.de> Message-ID: <20040112112821.GD27442@jupiter.cs.mu.oz.au> On 12-Jan-2004, Alexander Terekhov wrote: > > Fergus Henderson writes: > > > > > When a thread gets a cancellation request, the first > > > cancellation point encountered thereafter will act on that request > > > (by unwinding the stack, or in Nathan's model by returning an error > > > return status and setting errno = ECANCELLED). If another > > > cancellation point is encountered after that, and there has not > > > been another cancellation request, then the cancellation point > > > will normally not have any effect. > > > > > > With "sticky" cancels, after a cancellation request, > > > *every* subsequent cancellation point will act on the cancel > > > (by unwinding the stack, or failing with ECANCELLED). > > > > > > "sticky" cancels make it difficult/impossible to clean up properly, > > In the current POSIX/C++ environment one must disable/restore-old > cancellation state in the destructors that call POSIX cancellation > points anyway (I mean "general cancel-safe library" stuff). That > does ensure proper cleanup. Yes, to the extent that there _is_ a de facto standard current POSIX/C++ environment, it does not use sticky cancels, and so it is possible to do proper cleanup. > David Abrahams wrote: > > OK, thanks for the explanation. Sounds like an argument for Jason's > > exceptions that re-assert cancel when they're destroyed. > > Making broken code a bit less broken is hardly the right design > rationale. Presuming that we'll get a standard named thread cancel > request exception, I hope that we get a standard base class for thread cancel request exceptions, and that we provide a way for code which requests a thread cancellation to specify an object of any class derived from this standard base class, and have that object (or a copy of it) be the object thrown in the cancelled thread. > his "fix" (with respect to broken C++ libraries) > is nothing but > > catch(...) { > > /* ... no re-throw here ... */ > > try { > throw; > } > catch(std::thread_cancel_request const &) { > std::thread_enable_cancel(); // re-enable cancel > std::thread_self().cancel(); // re-inject cancel request > } > catch(...) { } > > } There's a big difference between doing that automatically and doing it manually. If it is done automatically, you can be sure that the programmer won't accidentally forget to do it. -- Fergus Henderson | "I have always known that the pursuit The University of Melbourne | of excellence is a lethal habit" WWW: | -- the last words of T. S. Garp. From baker at cs.fsu.edu Mon Jan 12 11:35:34 2004 From: baker at cs.fsu.edu (Ted Baker) Date: Mon, 12 Jan 2004 06:35:34 -0500 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: <40010755.30702@bogo.xs4all.nl> References: <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFD7DD5.9090801@bogo.xs4all.nl> <3FFD8046.3000702@hp.com> <3FFE43BB.6040809@bogo.xs4all.nl> <20040109082036.GA31026@redhat.com> <40006C6D.5090707@bogo.xs4all.nl> <20040110224422.GA19106@diablo.name> <40010755.30702@bogo.xs4all.nl> Message-ID: <20040112113534.GA5193@diablo.name> > Please note that the example destructor above is also called on a > regular (non-exceptional) exit from the block in which the > remote_resource_holder lives. > > Since, in that case, there is no exception 'in flight', a cancellation > exception thrown from remote_source->release() *will* be discarded, There can be no cancellation exception thrown in this case, because cancellation is disabled during the destructor execution. (This what is required in Ada, by the way.) > unless - in *all* such destructors - the code is changed to explictly > disable cancellation. Yes. You must diable cancellation in *all* destructors. > In other words, non-sticky cancellation will break existing code, and > the replacement code will not be pretty. IMHO, this is highly > undesirable - writing robust destructors is hard enough without these > complications. Disabling cancellation in all destructors can be done automagically by the compiler, so it will not break existing code. The code will only need to be recompiled. --Ted From baker at cs.fsu.edu Mon Jan 12 11:37:39 2004 From: baker at cs.fsu.edu (Ted Baker) Date: Mon, 12 Jan 2004 06:37:39 -0500 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: <4001BA3E.7010100@bogo.xs4all.nl> References: <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFD7DD5.9090801@bogo.xs4all.nl> <3FFD8046.3000702@hp.com> <3FFE43BB.6040809@bogo.xs4all.nl> <20040109082036.GA31026@redhat.com> <40006C6D.5090707@bogo.xs4all.nl> <4001BA3E.7010100@bogo.xs4all.nl> Message-ID: <20040112113739.GB5193@diablo.name> > I think it will. Even if cancellation is disabled while exceptions are > in flight, a 'catch-everything-and-never-rethrow' block in a destructor > will discard a cancellation exception when that destructor is triggered > by a regular (non-exceptional, non-unwinding) exit from a block scope. That is why you need to disable cancellation during the execution of destructors (and C cancellation cleanup handlers). > Call me paranoid, but as a C++ programmer, the thing that scares me most > in the discussions on this list is a certain (perceived?) pressure to > open the door for letting exceptions escape from destructors. Speaking > for myself, all I can say is that such a license would break nearly > every piece of non-trivial code I've written over the last eight years > or so. There is no need to let exceptions escape from destructors if you disable cancellation during desctructor execution. > - Wil --Ted From baker at cs.fsu.edu Mon Jan 12 11:40:14 2004 From: baker at cs.fsu.edu (Ted Baker) Date: Mon, 12 Jan 2004 06:40:14 -0500 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: <20040112010213.GB14371@jupiter.cs.mu.oz.au> References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> <20040112010213.GB14371@jupiter.cs.mu.oz.au> Message-ID: <20040112114014.GC5193@diablo.name> > The problem is that cleanup -- e.g. execution of C++ destructors on > stack unwinding -- may involve calls to I/O routines that are themselves > cancellation points. If these routines fail, then the cleanup functions > will not achieve their intended purpose; they will not be able to release > the resources that the thread holds. Nathan's model implies that they > will fail. This cannot happen if cancellation is automagically disabled during execution of (all) destructors, along with the stack unwinding code that is performed during exception processing. --Ted From boo at terekhov.de Mon Jan 12 11:56:20 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Mon, 12 Jan 2004 12:56:20 +0100 Subject: [c++-pthreads] Re: thread-safety definition References: <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> <20040112010213.GB14371@jupiter.cs.mu.oz.au> <40027836.7B9D03BC@terekhov.de> <20040112112821.GD27442@jupiter.cs.mu.oz.au> Message-ID: <40028B64.E906234B@terekhov.de> Fergus Henderson wrote: [...] > > In the current POSIX/C++ environment one must disable/restore-old > > cancellation state in the destructors that call POSIX cancellation > > points anyway (I mean "general cancel-safe library" stuff). That > > does ensure proper cleanup. > > Yes, to the extent that there _is_ a de facto standard current POSIX/C++ > environment, it does not use sticky cancels, and so it is possible to > do proper cleanup. I suppose that even under the Nathan's "sticky cancels model", cancellation points are meant to be idle (with respect to cancellation) when cancellation state is set to PTHREAD_CANCEL_DISABLE. > > > David Abrahams wrote: > > > OK, thanks for the explanation. Sounds like an argument for Jason's > > > exceptions that re-assert cancel when they're destroyed. > > > > Making broken code a bit less broken is hardly the right design > > rationale. Presuming that we'll get a standard named thread cancel > > request exception, > > I hope that we get a standard base class for thread cancel request > exceptions, and that we provide a way for code which requests a thread > cancellation to specify an object of any class derived from this standard > base class, and have that object (or a copy of it) be the object thrown > in the cancelled thread. This would extend the current POSIX model and would also mean that "C++ version" of pthread_cancel() and cancel request delivery "in C++" would need to have "release" and "acquire" memory synchronization semantics respectively. Well, I guess that I should now better send my forthcoming reply to your recent private message to the list. Oder? ;-) > > > his "fix" (with respect to broken C++ libraries) > > is nothing but > > > > catch(...) { > > > > /* ... no re-throw here ... */ > > > > try { > > throw; > > } > > catch(std::thread_cancel_request const &) { > > std::thread_enable_cancel(); // re-enable cancel > > std::thread_self().cancel(); // re-inject cancel request > > } > > catch(...) { } > > > > } > > There's a big difference between doing that automatically > and doing it manually. If it is done automatically, you > can be sure that the programmer won't accidentally forget > to do it. I don't wont it to be done automatically because there is "just a few" valid scenarios where it would make sense to do it... and broken C++ libraries ala iostream-stuff is not one of them, IMO. regards, alexander. From boo at terekhov.de Mon Jan 12 12:05:47 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Mon, 12 Jan 2004 13:05:47 +0100 Subject: [c++-pthreads] Re: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> <20040112010213.GB14371@jupiter.cs.mu.oz.au> <20040112114014.GC5193@diablo.name> Message-ID: <40028D9B.988E8C81@terekhov.de> Ted Baker wrote: > > > The problem is that cleanup -- e.g. execution of C++ destructors on > > stack unwinding -- may involve calls to I/O routines that are themselves > > cancellation points. If these routines fail, then the cleanup functions > > will not achieve their intended purpose; they will not be able to release > > the resources that the thread holds. Nathan's model implies that they > > will fail. > > This cannot happen if cancellation is automagically disabled during > execution of (all) destructors, along with the stack unwinding code > that is performed during exception processing. Except that it would pretty much preclude SAFE exploitation of cancellation in the destructors (stuff ala "~temp_dataset()" which can simply catch cancel request exception and "re-enable/re-inject"). Burning processing cycles on cancel enable/disable is another issue here, BTW. Intelligent cancellation that would check the dynamic context (2-pase ES with exception specs acting like "fences" NOT causing unnecessary unwinding) would work much better, I guess. regards, alexander. From boo at terekhov.de Mon Jan 12 12:56:58 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Mon, 12 Jan 2004 13:56:58 +0100 Subject: [c++-pthreads] thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD4034.7050903@hp.com> <1073567261.28360.258.camel@felix.inria.fr> <3FFD6355.80200@hp.com> <3FFD6CA8.B5CEF27C@terekhov.de> <20040108213655.GB31384@redhat.com> <3FFEAA8C.7085265B@terekhov.de> <20040111211904.GA13141@jupiter.cs.mu.oz.au> <400278B8.112140FB@terekhov.de> <20040112111321.GA4300@jupiter.cs.mu.oz.au> Message-ID: <4002999A.485DF416@terekhov.de> Fergus Henderson wrote: > > On 12-Jan-2004, Alexander Terekhov wrote: > > Fergus Henderson wrote: > > > > > > On 09-Jan-2004, Alexander Terekhov wrote: > > > > Richard Henderson wrote: > > > > > > > > > > On Thu, Jan 08, 2004 at 03:43:52PM +0100, Alexander Terekhov wrote: > > > > > > C++ aside for a moment, I have yet to see async-cancel-UNsafe > > > > > > implementation of, say, strlen(). Care to inspire me? TIA. > > > > > > > > > > I have yet to see an async-safe exception runtime. It's certainly > > > > > possible, but it requires the addition of lots of memory barriers > > > > > to the code. > > > > > > > > Huh? Let's talk about memory barriers next week, okay? > > > > > > > > http://www.terekhov.de/pthread_refcount_t/draft-edits.txt > > > > http://groups.google.com/groups?selm=3EEF38E8.311EEC77%40web.de > > > > http://www.terekhov.de/pthread_refcount_t/experimental/refcount.cpp > > > > > > What is the relevance of those URLs? > > > > This stuff is about memory barriers aka memory synchronization. > > Well, I know what memory barriers are. I doubt you know what "hoist" and "sink" memory barriers are. ;-) > Is there any reason why I should > bother to read the stuff at those URLs? In particular, do those URLs > have any relevance to Richard Henderson's comment, In POSIX, cancellation runs in the context of canceled thread. There's a context switch, so to speak. And, more importantly, cancellation (just like pthread_kill() signaling) does NOT have any memory synchronization effects at all. > or to how to combine > Posix threads and C++? Your latest comment implied to me that you want pthread_cancel() (and/or "the C++ version" of it) and cancellation request delivery be added to the "XBD 4.10" (in "POSIX.C++", at least). regards, alexander. From fjh at cs.mu.oz.au Mon Jan 12 13:05:46 2004 From: fjh at cs.mu.oz.au (Fergus Henderson) Date: Tue, 13 Jan 2004 00:05:46 +1100 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: <20040112114014.GC5193@diablo.name> References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> <20040112010213.GB14371@jupiter.cs.mu.oz.au> <20040112114014.GC5193@diablo.name> Message-ID: <20040112130546.GA29001@jupiter.cs.mu.oz.au> On 12-Jan-2004, Ted Baker wrote: > > The problem is that cleanup -- e.g. execution of C++ destructors on > > stack unwinding -- may involve calls to I/O routines that are themselves > > cancellation points. If these routines fail, then the cleanup functions > > will not achieve their intended purpose; they will not be able to release > > the resources that the thread holds. Nathan's model implies that they > > will fail. > > This cannot happen if cancellation is automagically disabled during > execution of (all) destructors, along with the stack unwinding code > that is performed during exception processing. If all cleanup is done via destructors (or pthread_cleanup_push, which can be handled similarly), you are right. But recall that in Nathan's model, Posix functions that return status values will indicate cancellation by returning a failure status and setting errno to ECANCELLED. In this situation, cleanup actions will often be performed directly, rather than via destructors. Once some code has called a cancellation point that fails with ECANCELLED, there is no way for the implementation to tell whether any further code which is executed is cleanup code or just a resumption of the thread's main loop. For cleanup code, cancellation must be disabled, so that the cleanup code can perform I/O to release resources. But for any long-running non-cleanup code, cancellation must be reenabled; otherwise, we would be failing to provide the limited assurance that cancellation will not be accidentally ignored, which stickiness of cancellation was intended to ensure. Since the implementation can't tell, the code must be modified to explicitly enable and/or disable cancellation at appropriate points. Nathan's model was intended to preserve existing code that propagated error returns, and that did not expect functions like read() or printf() to throw exceptions. However, because of the problems described above, such existing code would still need to be changed to make cleanup work properly. So I don't think Nathan's model achieves his intended aim. -- Fergus Henderson | "I have always known that the pursuit The University of Melbourne | of excellence is a lethal habit" WWW: | -- the last words of T. S. Garp. From boo at terekhov.de Mon Jan 12 13:20:42 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Mon, 12 Jan 2004 14:20:42 +0100 Subject: [c++-pthreads] Re: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> <20040112010213.GB14371@jupiter.cs.mu.oz.au> <20040112114014.GC5193@diablo.name> <20040112130546.GA29001@jupiter.cs.mu.oz.au> Message-ID: <40029F2A.7C13BD76@terekhov.de> Fergus Henderson wrote: [...] > > This cannot happen if cancellation is automagically disabled during > > execution of (all) destructors, along with the stack unwinding code > > that is performed during exception processing. > > If all cleanup is done via destructors (or pthread_cleanup_push, > which can be handled similarly), you are right. AFAIK, POSIX doesn't disable cancellation when thread cleanup handler is invoked by pthread_cleanup_pop() call with a non-zero execute argument (cancellation is disabled when the thread exits [that is, calls pthread_exit()] or acts upon a cancellation request delivery). I do it "manually". regards, alexander. From fjh at cs.mu.oz.au Mon Jan 12 13:36:56 2004 From: fjh at cs.mu.oz.au (Fergus Henderson) Date: Tue, 13 Jan 2004 00:36:56 +1100 Subject: [c++-pthreads] thread-safety definition In-Reply-To: <4002999A.485DF416@terekhov.de> References: <3FFD4034.7050903@hp.com> <1073567261.28360.258.camel@felix.inria.fr> <3FFD6355.80200@hp.com> <3FFD6CA8.B5CEF27C@terekhov.de> <20040108213655.GB31384@redhat.com> <3FFEAA8C.7085265B@terekhov.de> <20040111211904.GA13141@jupiter.cs.mu.oz.au> <400278B8.112140FB@terekhov.de> <20040112111321.GA4300@jupiter.cs.mu.oz.au> <4002999A.485DF416@terekhov.de> Message-ID: <20040112133656.GB29001@jupiter.cs.mu.oz.au> On 12-Jan-2004, Alexander Terekhov wrote: > Fergus Henderson wrote: > > > > On 12-Jan-2004, Alexander Terekhov wrote: > > > Fergus Henderson wrote: > > > > > > > On 09-Jan-2004, Alexander Terekhov wrote: > > > > > Richard Henderson wrote: > > > > > > > > > > > > On Thu, Jan 08, 2004 at 03:43:52PM +0100, Alexander Terekhov wrote: > > > > > > > C++ aside for a moment, I have yet to see async-cancel-UNsafe > > > > > > > implementation of, say, strlen(). Care to inspire me? TIA. > > > > > > > > > > > > I have yet to see an async-safe exception runtime. It's certainly > > > > > > possible, but it requires the addition of lots of memory barriers > > > > > > to the code. > > > > > > > > > > Huh? Let's talk about memory barriers next week, okay? > > > > > > > > > > http://www.terekhov.de/pthread_refcount_t/draft-edits.txt > > > > > http://groups.google.com/groups?selm=3EEF38E8.311EEC77%40web.de > > > > > http://www.terekhov.de/pthread_refcount_t/experimental/refcount.cpp > > > > > > > > What is the relevance of those URLs? > > > > > > This stuff is about memory barriers aka memory synchronization. The first URL was a 23-page slab of standardese, apparently "additions to the XBD 4.10 Memory Synchronization" and maybe some other stuff. There was no explanatory text to give any hint of the rationale for these changes. I see no point reading 23 pages of standardese if you can't be bothered telling us what you are trying to achieve with those 23 pages of standardese. > > or to how to combine > > Posix threads and C++? > > Your latest comment implied to me that you want pthread_cancel() (and/or > "the C++ version" of it) and cancellation request delivery be added to > the "XBD 4.10" (in "POSIX.C++", at least). If thread cancellation involves passing data between the canceler and the cancellee, as I have suggested would make sense for C++, then clearly some memory synchronization will be required. I haven't analyzed it in great detail, but at first glance I don't see that being a problem. After all, cancellation should be a very rare event. -- Fergus Henderson | "I have always known that the pursuit The University of Melbourne | of excellence is a lethal habit" WWW: | -- the last words of T. S. Garp. From David.Butenhof at hp.com Mon Jan 12 13:41:39 2004 From: David.Butenhof at hp.com (Dave Butenhof) Date: Mon, 12 Jan 2004 08:41:39 -0500 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> Message-ID: <4002A413.4090908@hp.com> Jason Merrill wrote: >On Fri, 09 Jan 2004 07:20:18 -0500, Dave Butenhof wrote: > > >>Jason's model, putting together the puzzle pieces from various messages >>here >> >> >My original formulation is at > > http://www.codesourcery.com/archives/c++-pthreads/msg00059.html > > >>, appears to be basing cancel on an ordinary ("finalizable") C++ >>exception, which would be raised by any and all cancellation points; but if >>the exception object were to be destroyed (by finalization), the "pending >>cancel" would (might?) be reasserted. I personally feel that it ought to be >>reasonable to finalize the cancel and continue operation. If that wasn't >>really what the application intended then it's an application error; but >>there are cases where it's arguably reasonable, and the capability falls >>naturally out of the model. >> >> >Too naturally, IMO. There seems to be a substantial amount of C++ code out >there which traps and finalizes all exceptions, some of it in the C++ >standard library I/O classes. I disagree with this design, and propose to >change it in the message cited above, but it's just one example. I don't >think it's reasonable for a cancellation request to be silently discarded >just because the thread was in the middle of doing some formatted output >when the cancel was received. > >I wouldn't object to a facility for explicitly abandoning the cancel, but I >think that allowing it to fall out of the model is far too fragile. > I think I disagree, at least philsophically, with the characterization of the model as "fragile". But I think I also understand what you mean; and the problem isn't with the model, but rather with the effect of that model on existing code that all-too-casually and agressively eats exceptions it doesn't understand. I think there are vanishingly few circumstances where a blind catch(...) without an unconditional re-throw should be considered "legitimate". If you don't completely understand what an exception means, you cannot claim to have completely recovered, and therefore cannot reasonably finalize propagation. (And when you catch anonymously, you can't possibly understand what they mean since you can't even identify them.) On the other hand, regardless of whether the semantics are meaningful or reasonable, the syntax is legal and apparently (unfortunately) in common use, so I can't dispute that you need to consider that. >>Besides, "reasserting cancel" in all the right places and no other places >>sounds like one of those projects that ends up being a lot more subtle >>and error-prone than anyone ever expects. ;-) >> >> >Assuming support for abanoning the cancel, why would it be more complex than > > pthread_cancel (pthread_self ()); > > While I didn't have anything specific in mind, just a general concern, one example that occurs to me is "catch(...) {...; throw;}". One must be careful about specifying the "cancelled" state of the thread here; it shouldn't be possible to consider the exception destroyed during the body of the catch prior to the re-throw. I don't know if the current wording in the standard would allow this interpretation: but for "traditional" exceptions that have no "state" it shouldn't really matter, and it seems plausible that the standard doesn't tie this down explicitly. (It's enough in practice that the 'throw' continues propagation "as if". Now, though, the distinction might become critical.) -- /--------------------[ David.Butenhof at hp.com ]--------------------\ | Hewlett-Packard Company Tru64 UNIX & VMS Thread Architect | | My book: http://www.awl.com/cseng/titles/0-201-63392-2/ | \----[ http://homepage.mac.com/dbutenhof/Threads/Threads.html ]---/ From boo at terekhov.de Mon Jan 12 14:30:10 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Mon, 12 Jan 2004 15:30:10 +0100 Subject: [c++-pthreads] thread-safety definition References: <3FFD4034.7050903@hp.com> <1073567261.28360.258.camel@felix.inria.fr> <3FFD6355.80200@hp.com> <3FFD6CA8.B5CEF27C@terekhov.de> <20040108213655.GB31384@redhat.com> <3FFEAA8C.7085265B@terekhov.de> <20040111211904.GA13141@jupiter.cs.mu.oz.au> <400278B8.112140FB@terekhov.de> <20040112111321.GA4300@jupiter.cs.mu.oz.au> <4002999A.485DF416@terekhov.de> <20040112133656.GB29001@jupiter.cs.mu.oz.au> Message-ID: <4002AF72.C1CD6F98@terekhov.de> Fergus Henderson wrote: [...] > > > > > > http://www.terekhov.de/pthread_refcount_t/draft-edits.txt > > > > > > http://groups.google.com/groups?selm=3EEF38E8.311EEC77%40web.de > > > > > > http://www.terekhov.de/pthread_refcount_t/experimental/refcount.cpp > > > > > > > > > > What is the relevance of those URLs? > > > > > > > > This stuff is about memory barriers aka memory synchronization. > > The first URL was a 23-page slab of standardese, apparently "additions > to the XBD 4.10 Memory Synchronization" and maybe some other stuff. > There was no explanatory text to give any hint of the rationale for these > changes. I see no point reading 23 pages of standardese if you can't be > bothered telling us what you are trying to achieve with those 23 pages of > standardese. Well, thread-safe reference counting (hello tr1::shared_ptr and tr1::weak_ptr) aside for a moment, see http://groups.google.com/groups?selm=3EE75F28.76EC29B4%40web.de http://groups.google.com/groups?selm=3FAA2DC0.42F602DB%40web.de http://groups.google.com/groups?selm=3FD09072.BB1BEAF1%40web.de > > > > or to how to combine > > > Posix threads and C++? > > > > Your latest comment implied to me that you want pthread_cancel() (and/or > > "the C++ version" of it) and cancellation request delivery be added to > > the "XBD 4.10" (in "POSIX.C++", at least). > > If thread cancellation involves passing data between the canceler and the > cancellee, as I have suggested would make sense for C++, then clearly > some memory synchronization will be required. I haven't analyzed it > in great detail, but at first glance I don't see that being a problem. > After all, cancellation should be a very rare event. Agreed. Oh, BTW, with respect to pthread_kill()... perhaps you might want (reason: sort of "similar issue" with respect to msync plus cancellation/errno silliness in the context of async handlers) to take a look at the following thread: http://groups.google.com/groups?threadm=3F60A3B0.7E035CBD%40web.de What do you think? regards, alexander. From boo at terekhov.de Mon Jan 12 15:28:28 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Mon, 12 Jan 2004 16:28:28 +0100 Subject: [c++-pthreads] Re: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> <4002A413.4090908@hp.com> Message-ID: <4002BD1C.A034BAD0@terekhov.de> Dave Butenhof wrote: [...] > While I didn't have anything specific in mind, just a general concern, > one example that occurs to me is "catch(...) {...; throw;}". One must be > careful about specifying the "cancelled" state of the thread here; it > shouldn't be possible to consider the exception destroyed during the > body of the catch prior to the re-throw. I don't know if the current > wording in the standard would allow this interpretation: .... That's 15.1/6 and 15.1/7 (no changes in TC1-2003 edition, AFAIK). 6 A throw-expression with no operand rethrows the exception being handled. The exception is reactivated with the existing temporary; no new temporary exception object is created. The exception is no longer considered to be caught; therefore, the value of uncaught_exception() will again be true. [Example: code that must be executed because of an exception yet cannot completely handle the exception can be written like this: try { // ... } catch (...) { // catch all exceptions // respond (partially) to exception throw; // pass the exception to some // other handler } ?end example] 7 The exception thrown is the one most recently caught and not finished. An exception is considered caught when initialization is complete for the formal parameter of the corresponding catch clause, or when terminate() or unexpected() is entered due to a throw. An exception is considered finished when the corresponding catch clause exits or when unexpected() exits after being entered due to a throw. regards, alexander. From boo at terekhov.de Mon Jan 12 15:43:43 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Mon, 12 Jan 2004 16:43:43 +0100 Subject: [c++-pthreads] Re: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> <4002A413.4090908@hp.com> <4002BD1C.A034BAD0@terekhov.de> Message-ID: <4002C0AF.266D6744@terekhov.de> Alexander Terekhov wrote: > > Dave Butenhof wrote: > [...] > > While I didn't have anything specific in mind, just a general concern, > > one example that occurs to me is "catch(...) {...; throw;}". One must be > > careful about specifying the "cancelled" state of the thread here; it > > shouldn't be possible to consider the exception destroyed during the > > body of the catch prior to the re-throw. I don't know if the current > > wording in the standard would allow this interpretation: .... > > That's 15.1/6 and 15.1/7 (no changes in TC1-2003 edition, AFAIK). ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > 6 A throw-expression with no operand rethrows the exception being > handled. The exception is reactivated with the existing temporary; > no new temporary exception object is created. The exception is no > longer considered to be caught; therefore, the value of > uncaught_exception() will again be true. [Example: code that must ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ That's not necessarily true (it depends), BTW. > be executed because of an exception yet cannot completely handle > the exception can be written like this: > > try { > // ... > } > catch (...) { // catch all exceptions > > // respond (partially) to exception > > throw; // pass the exception to some > // other handler > } > > ?end example] http://groups.google.com/groups?selm=3D75E040.469A103E%40web.de regards, alexander. From r-smith at ihug.co.nz Mon Jan 12 15:52:19 2004 From: r-smith at ihug.co.nz (Ross Smith) Date: Tue, 13 Jan 2004 04:52:19 +1300 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: <4002A413.4090908@hp.com> References: <1073556051.28360.195.camel@felix.inria.fr> <4002A413.4090908@hp.com> Message-ID: <200401130452.19605.r-smith@ihug.co.nz> On Tuesday 13 January 2004 02:41, Dave Butenhof wrote: > > I think I disagree, at least philsophically, with the > characterization of the model as "fragile". But I think I also > understand what you mean; and the problem isn't with the model, but > rather with the effect of that model on existing code that > all-too-casually and agressively eats exceptions it doesn't > understand. I think there are vanishingly few circumstances where a > blind catch(...) without an unconditional re-throw should be > considered "legitimate". I'm beginning to get the same worried feeling Wil Evers expressed a few messages back. It sounds as though a lot of people are seriously considering the idea of allowing exceptions to escape from destructors. Dave, while I have the greatest respect for your knowledge and understanding of threads, I think the above shows that you don't really understand C++. Please try to understand that it is _absolutely vital_ that destructors never be allowed to throw under any circumstances. If a destructor calls a function that may throw some unknown exception (a very common case, especially in template classes whose destructors will often call member functions of some arbitrary user-supplied type), the call _must_ be wrapped in a catch-and-discard-all block. At most you can log the error somewhere. Any attempt to standardise a solution that doesn't take "Destructors Must Not Throw" as axiomatic is simply going to be a non-starter as far as C++ programmers are concerned. I'd really like to see this list explicitly address this issue rather than continuing to casually talk about uncatchable exceptions and similar horrors. -- Ross Smith ......... r-smith at ihug.co.nz ......... Auckland, New Zealand "This world is like a burnt steak: small, tough, and the chips are always stacked against you." -- Mike From wil at bogo.xs4all.nl Mon Jan 12 16:04:14 2004 From: wil at bogo.xs4all.nl (Wil Evers) Date: Mon, 12 Jan 2004 17:04:14 +0100 Subject: [c++-pthreads] Re: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com> <4002A413.4090908@hp.com> Message-ID: <4002C57E.8030805@bogo.xs4all.nl> Dave Butenhof wrote: > But I think I also understand what you mean; > and the problem isn't with the model, but rather with the effect of that > model on existing code that all-too-casually and agressively eats > exceptions it doesn't understand. I think there are vanishingly few > circumstances where a blind catch(...) without an unconditional re-throw > should be considered "legitimate". If you don't completely understand > what an exception means, you cannot claim to have completely recovered, > and therefore cannot reasonably finalize propagation. (And when you > catch anonymously, you can't possibly understand what they mean since > you can't even identify them.) Let me try again. Doing a blind catch(...) without an unconditional rethrow is not always a matter of sloppiness. Sometimes, the only alternative is a crash - the C++ standard clearly says that a second exception escaping from a destructor invoked by the stack unwinding machinery will result in program termination. There is nothing the poor programmer can do about this. To reiterate: a few days ago, I posted the following example - remote_resource_holder::~remote_resource_holder() { remote_resource->release(); } Now what do you suggest I should do when an exception (say, a networking error, but it might just as well be a cancellation request) is thrown from somewhere within remote_resource->release()? The way I see it, either the original exception or the one that was just thrown at me will have to be finalized. - Wil From boo at terekhov.de Mon Jan 12 16:27:41 2004 From: boo at terekhov.de (Alexander Terekhov) Date: Mon, 12 Jan 2004 17:27:41 +0100 Subject: [c++-pthreads] Re: thread-safety definition References: <1073556051.28360.195.camel@felix.inria.fr> <4002A413.4090908@hp.com> <200401130452.19605.r-smith@ihug.co.nz> Message-ID: <4002CAFD.706264D8@terekhov.de> Ross Smith wrote: [...] > I'm beginning to get the same worried feeling Wil Evers expressed a few > messages back. It sounds as though a lot of people are seriously > considering the idea of allowing exceptions to escape from destructors. > > Dave, while I have the greatest respect for your knowledge and Hey Ross, Dave wrote: "One of those idle thoughts was that the concept of "cancel state" (enable or disable) might even be meaningless (or at least irrelevant) for a pure C++ binding. This might simply be a dynamic property of the current call tree; depending on the nested throw specs. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > understanding of threads, I think the above shows that you don't really > understand C++. Please try to understand that it is _absolutely vital_ > that destructors never be allowed to throw under any circumstances. If Then why don't you complain that destructors are allowed to throw anything by default instead of having implicit throw() ("by default") ES/throw spec. imposed on them? > a destructor calls a function that may throw some unknown exception (a > very common case, especially in template classes whose destructors will > often call member functions of some arbitrary user-supplied type), the > call _must_ be wrapped in a catch-and-discard-all block. Not really. regards, alexander. From David.Butenhof at hp.com Mon Jan 12 16:31:09 2004 From: David.Butenhof at hp.com (Dave Butenhof) Date: Mon, 12 Jan 2004 11:31:09 -0500 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: <200401130452.19605.r-smith@ihug.co.nz> References: <1073556051.28360.195.camel@felix.inria.fr> <4002A413.4090908@hp.com> <200401130452.19605.r-smith@ihug.co.nz> Message-ID: <4002CBCD.4040701@hp.com> Ross Smith wrote: >On Tuesday 13 January 2004 02:41, Dave Butenhof wrote: > > >>I think I disagree, at least philsophically, with the >>characterization of the model as "fragile". But I think I also >>understand what you mean; and the problem isn't with the model, but >>rather with the effect of that model on existing code that >>all-too-casually and agressively eats exceptions it doesn't >>understand. I think there are vanishingly few circumstances where a >>blind catch(...) without an unconditional re-throw should be >>considered "legitimate". >> >> >I'm beginning to get the same worried feeling Wil Evers expressed a few >messages back. It sounds as though a lot of people are seriously >considering the idea of allowing exceptions to escape from destructors. > >Dave, while I have the greatest respect for your knowledge and >understanding of threads, I think the above shows that you don't really >understand C++. > While I wouldn't dispute the statement that "I don't really understand C++", I absolutely do understand objects, encapsulation, modularity, exceptions, cancellation, and resource ownership; and what you're saying seems essentially "obvious and self-evident". I never argued, or suggested, or assumed, that exceptions would propagate out of destructors. But I'm talking about cancel, not exceptions. >Please try to understand that it is _absolutely vital_ >that destructors never be allowed to throw under any circumstances. If >a destructor calls a function that may throw some unknown exception (a >very common case, especially in template classes whose destructors will >often call member functions of some arbitrary user-supplied type), the >call _must_ be wrapped in a catch-and-discard-all block. At most you >can log the error somewhere. > >Any attempt to standardise a solution that doesn't take "Destructors >Must Not Throw" as axiomatic is simply going to be a non-starter as far >as C++ programmers are concerned. I'd really like to see this list >explicitly address this issue rather than continuing to casually talk >about uncatchable exceptions and similar horrors. > > This discussion is a swirling mix of different issues. They're often related, but please don't assume relations where they're not stated! Generally, any code that cannot raise (or propagate) an exception SHOULD NOT BE A CANCELLATION POINT. (Note to Dave Abrahams: Sorry, was that shout approved? ;-) ) In the context of cancellation, I don't care about whether exceptions can propagate out of destructors. As has already been said, while a destructor might be activated as part of handling and cleaning up from a cancel, that's not the same as propagating a local exception out of the destructor. If destructors are not cancellable, then no cancel exception can BEGIN within the destructor and the question of whether it could propagate out of the destructor is moot. (For what it's worth, I don't believe that they can or should.) Furthermore, a no-cancel destructor, regardless of whether it has a catch(...) with or without a re-throw, cannot possibily "eat" a cancel, because it can never catch a cancel that's not thrown within the dynamic scope of its try block. I suspect that destructors should be implicitly "no cancel zones". (Someone expressed a concern about runtime overhead; but the cost of enabling and disabling cancel CAN be made extremely low by collusion between C++ and the thread library. That's an implementation detail the standard doesn't necessarily need to explicitly address.) There might be cases where someone really wants an indefinite blocking operation in a destructor (perhaps to disconnect from a remote server), and might want that to be cancellable. I think that's a rare special case, and could probably be handled by manually enabling cancel over that range of code, catching/finalizing the cancel if it occurs, cleaning up, restoring the original "cancel disabled" state, and then calling pthread_cancel(pthread_self()) to re-pend the cancel. Perhaps (though it seems doubtful) this might be common enough to justify some special syntax to encapsulate the mechanism and avoid errors. -- /--------------------[ David.Butenhof at hp.com ]--------------------\ | Hewlett-Packard Company Tru64 UNIX & VMS Thread Architect | | My book: http://www.awl.com/cseng/titles/0-201-63392-2/ | \----[ http://homepage.mac.com/dbutenhof/Threads/Threads.html ]---/ From baker at cs.fsu.edu Mon Jan 12 16:45:33 2004 From: baker at cs.fsu.edu (Ted Baker) Date: Mon, 12 Jan 2004 11:45:33 -0500 Subject: [c++-pthreads] Re: thread-safety definition In-Reply-To: <40028D9B.988E8C81@terekhov.de> References: <3FFD7487.1000404@hp.com> <3FFD7B71.1010708@hp.com> <3FFE9C82.3060500@hp.com>