[c++-pthreads] Re: FW: RE: Re: I'm Lost

[Dave Butenhof]

David Abrahams wrote:
> Dave Butenhof <david.butenhof at hp.com> writes:
>   
>> David Abrahams wrote:
>>     
>>> Dave Butenhof <david.butenhof at hp.com> writes:  
>>>       
>>>> However when one writes a robust general-purpose facility (library)
>>>> that will be used in an environment supporting cancellation, that
>>>> library ought to be written to support cancellation (whether or not
>>>> it actually uses cancellation on its own behalf). Such libraries
>>>> are not generally tasks taken on by "casual users"; and even so
>>>> while hardly ideal it's perfectly adequate to simply say "this
>>>> facility isn't cancel-safe; tough luck".
>>>>
>>>> "Industrial strength" libraries in the environment, for example the
>>>> "language runtime" itself, whether libc or STL, ought to be
>>>> cancel-safe certainly. Even at that, however, because the task can
>>>> be monumental, POSIX provided "cheats" -- the list of "optional"
>>>> cancellation points allow a libc developer to omit all but the most
>>>> critical. A C++ standard for STL *could* provide similar "cheats"
>>>> to avoid requiring full implementation of cancel-safety. And again,
>>>> if the user of the library (whether the main application or another
>>>> library) doesn't choose to use cancellation the point is moot.
>>>>         
>>> Picking this thread up from long ago, lete me say that I'm sort-of in
>>> agreement with the above.  I say "sort of" because Dave B's statement
>>> fails to address the following point (hereafter known as "the
>>> statement"), and I can't tell what side of it he'd come down on:
>>>
>>>    Any code that is already exception-safe could be automatically
>>>    cancel-safe depending on our definition of "cancel-safe" and the
>>>    semantics we assign to cancellation exceptions.
>>>
>>> In the definition of "cancel-safe" that allows the statement to be
>>> true, cancellation is a request, and doesn't absolutely force
>>> _anything_ to happen.  IIUC, that is the status quo anyway (nobody is
>>> even forced to invoke a cancellation point).
>>>
>>> The cancellation exception semantics that allow the statement to be
>>> true are that they act like any other exception, and are not
>>> automatically rethrown at the end of catch blocks.  This is the
>>> question primarily in dispute, IIUC. 
>>>       
>> This has been THE most contentious issue in every C++/threads discussion 
>> I've encountered since the beginning of (pthread) time.
>>
>> My preference has always been that cancellation is an exception. Period. 
>> In our initial CMA architecture, and in our exception mapping of 
>> cancellation/thread-exit onto C language exceptions in Tru64 UNIX and 
>> OpenVMS, it's possible and reasonable to finalize propagation of a 
>> cancel/exit exception. That was critical for DCE, for example, so that 
>> it could trap cancellation of an RPC server thread, bring the thread 
>> back into the server's work pool, and propagate the exception across the 
>> wire to the client.
>>     
> That sounds highly analogous to my case with Python.
>   
Sure; and there are other examples. I've just found that the "inverted 
call stack" of this sort of server setup seems to make sense to a lot of 
people.
>> To finalize a cancel/exit under almost any normal circumstance is
>> simply an application error. 
>>     
> The key word being "almost."  In some situations, like those we've
> both cited, it's absolutely necessary, to even get cancellation to
> work.
>   
Absolutely. There was a strong sub-faction in POSIX that can be loosely 
characterized as "academics" who were determined to try to prevent 
constructs that might be misused. It's why the realtime people didn't 
get pthread_abort() to force termination without cleanup, why you can't 
suspend a thread, or "force unlock" a mutex that might have been 
abandoned, and so forth. If cancellation can't be finalized, nobody can 
accidentally finalize it; and that's great if you don't trust anyone to 
know when it SHOULD be finalized. I started out as something of an 
"academic" in this sense and evolved into a pramatist... if someone 
thinks they need it, and they're right, don't keep them from doing it. 
And if they're WRONG, it's their problem, not yours. ;-) Portable Java 
GC would have been a lot easier had POSIX included suspend/resume; and 
does it really matter that nearly anyone else who used it would be 
breaking their application? Well, it all depends on your point of view...
>> There are many worse application errors, like infinite loops, that
>> we can't legislate around anyway.  Worrying too much that someone
>> might finalize the exception unintentionally just seemed like wasted
>> effort. However it's also important to keep in mind that my
>> preferences were formed with POSIX cancellation and C language (or
>> cross-language OS) exceptions. C++ adds a lot of exception semantics
>> and patterns on top of that.
>>
>> There have been plenty of people who argue that cancel "can't" be 
>> caught; and some of these arguments trace back to the ubiquity of 
>> catch(...), especially in constructors; 
>>     
> A ctor that does catch(...) without rethrow is almost always badly
> designed at best.  There was unfortunate advice going around for many
> years that you shouldn't throw from ctors, but that's exactly wrong:
> ctors that throw allow the establishment of strong invariants, and
> programming without them is much harder.
>
> On the other hand, stopping exceptions in dtors is absolutely the
> right thing to do.
>   
Yes, I may have said that backwards. As I've said before, although I use 
C++, it's not a "native language" for me, and a lot of this is based on 
opinions others have strongly stated rather than my own knowledge or 
experience. Another reason, incidentally, for not trying to come down 
too hard on one side or the other where language concerns might trump 
"threading" concerns.
>> and they have some legitimate concerns about common C++ language
>> patterns that might pretty much prevent a cancel from ever doing
>> what a cancel should do.
>>     
> Really, legitimate concerns? I can't think of any recommended patterns that would act that way.
>   
Hmm. OK. That's interesting. Well, you're the C++ expert here. ;-)
>> The catch lies in whether (and how far) you'll trust application
>> developers to do the re-throw properly. If we don't clean up all
>> frames and eventually re-throw the cancel/exit to the runtime's base
>> frame to terminate the thread, then we don't have cancellation.
>>     
> I'm of the school that says it's futile and even dangerous to try to
> operate correctly in an environment where you have to assume other
> code is incorrect.  My library will probably be equally broken if the
> user decides to throw out my bad_alloc exception without a proper
> response.
>   
True enough. "Over-engineering" is a persistent danger. Sometimes we 
just need to accept that we can't solve all problems, and that 
"idiot-proofing" is a losing concept, and move on. ;-)
>> On the other hand, if we prevent a catch or force a re-throw, we
>> lose a lot of C++ (particularly in constructors).
>>     
> I don't think that should be your concern.  The correct and
> well-written C++ we lose is generally sitting at module and language
> boundaries.  And then there are dtors.
>   
>> Part of the reason that you "can't tell what side of it [I'd] come
>> down on" is that I've long recognized this as an essentially
>> religious rather than technical argument.  You'll come down on the
>> side of the semantics toward which you feel the strongest emotional
>> attachment.
>>     
> You don't find the idea that exception-safe code implies cancel-safe
> code technically compelling?  I don't think that's an emotional issue.
>   
Well, yes, I do, because cancel was always intended to be "just" an 
exception that happened to be thrown from another thread. But then, 
nothing is ever that simple; the asynchronous nature required controls 
like cancelability type and state. C++ exceptions are synchronous and 
non-interrupting. (The latter a consequence of the former, really.) One 
of the main advantages of cancellation is that it can break through an 
extended blocking operation; but that's unavoidably an extra condition 
over "exception-safety". Cancel-safe has to mean something more unless 
we drop interruptibility. If we drop it, then cancel-safety is just 
exception-safety but loses much of its value in controlling application 
responsiveness.

In any case, though, I wasn't suggesting that you need to convince me. 
I'm saying there are diverse and strongly held positions that somehow 
need to be unified in order to get consensus on any proposal. I think 
that I'm the least of your worries. ;-)
>> While I'm happy to express my experience and even preferences, I
>> also recognize that "the other side" has some equally strong
>> arguments and expectations, and they (well, most of them!)  are not
>> "wrong".
>>     
> I don't think either side is "wrong," either.
>   
>> Someone needs to propose and champion "the great exception
>> compromise"; but if that's to be me I don't yet have the faintest
>> germ of a notion what it might be. So I sure hope it's going to be
>> someone else. ;-)
>>     
> If "finalized cancellation exceptions result in a new throw at the
> next cancellation point" isn't enough of a compromise, it isn't going
> to be me either, because I'm out of new ideas.
>
> Okay, how about this one: we count the number of times the
> cancellation is discarded.  The cancelling thread can specify the
> number of discards to tolerate, where the default is infinite.  After
> that, at the next cancellation point all pthread cancellation handlers
> (but not dtors or catch blocks) are run and the thread is terminated.
> Heck, at that point I don't care what happens; you're gambling anyway.
> Run all the dtors and catch blocks for all I care.
>   
I do NOT favor any model where "dtor/catch" and "cancellation handler" 
don't mean the same thing.

I don't think the count is tenable either because although it always 
feels tempting to add a control dial, it doesn't solve any actual 
problem if there's nobody who can know to what value the dial should be 
set. In this case, I can't see how either the canceler OR any modular 
call stack could possibly provide any useful data much less a single 
numeric value.

If "canceled" state persists when the exception is discarded, then 
cancel is something different from just "an exception"; which is too 
bad, but perhaps inevitable. You can't just catch it and continue -- you 
need to somehow also reset that state to recover your workgroup thread 
that's serially running RPC requests (or Python code, whatever). A lot 
of people have suggested various ways of making cancel-pending persist 
after the exception is launched; that's not necessarily "wrong", but it 
isn't "simple" either and somehow it doesn't feel right to me.
> A simpler approach might be to have two kinds of exception: "forced"
> and finalizable.  At least then we can say that exception-safe code
> implies finalizable cancellation safety.  Then "forced" synchronous
> cancellation can do whatever people desire.  I personally think it
> will become a useless appendage sort of like C++ exception
> specifications, but at least evolution will take care of it.  And if
> I'm wrong, evolution will wilt my finalizable cancellations.
>   
Is this the "unwind" vs "exception" idea? (Where "unwind" is like a new 
sort of 'throw' that triggers dtors but can't be caught/finalized.) Or 
something different...?
> I guess I'm not totally out of ideas ;-)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: david.butenhof.vcf
Type: text/x-vcard
Size: 476 bytes
Desc: not available
URL: <http://sourcerytools.com/pipermail/c++-pthreads/attachments/20060306/9c30e76f/attachment.vcf>