[c++-pthreads] Re: FW: RE: Re: I'm Lost

[Dave Butenhof]

David Abrahams wrote:
> Dave Butenhof <david.butenhof at hp.com> writes:
>   
>> Absolutely. There was a strong sub-faction in POSIX that can be
>> loosely characterized as "academics" who were determined to try to
>> prevent constructs that might be misused. 
>>     
> You mean, like, computers?
>   
OK, that's the laugh for today. Yeah, well, that's certainly what you 
get if you take it too far to the extreme.
>> It's why the realtime people didn't get pthread_abort() to force
>> termination without cleanup, why you can't suspend a thread, or
>> "force unlock" a mutex that might have been abandoned, and so
>> forth. 
>>     
> Oh, too bad.
>   
Yes and no. There's a line, somewhere, between something that's 
reasonably usable by some set of people to solve real problems, even if 
it can be easily and destructively misused by others; and something that 
makes a fun toy for a few researchers but is virtually impossible to use 
correctly or safely in real code. A lot of the argument hinges on where 
to draw that line; it's almost never "cut and dried".
>> If cancellation can't be finalized, nobody can accidentally
>> finalize it; and that's great if you don't trust anyone to know when
>> it SHOULD be finalized. I started out as something of an "academic"
>> in this sense and evolved into a pramatist... if someone thinks they
>> need it, and they're right, don't keep them from doing it.  And if
>> they're WRONG, it's their problem, not yours. ;-) Portable Java GC
>> would have been a lot easier had POSIX included suspend/resume; and
>> does it really matter that nearly anyone else who used it would be
>> breaking their application? Well, it all depends on your point of
>> view...
>>     
> I don't know about that suspend/resume, but I don't think legitimate
> cases where finalization is needed are nearly so rare as you describe
> cases where suspend/resume is needed to be.
>   
Probably. I probably also got carried away with the analogy, because the 
amount of detail was largely irrelevant in this forum.
>>> A ctor that does catch(...) without rethrow is almost always badly
>>> designed at best.  There was unfortunate advice going around for many
>>> years that you shouldn't throw from ctors, but that's exactly wrong:
>>> ctors that throw allow the establishment of strong invariants, and
>>> programming without them is much harder.
>>>
>>> On the other hand, stopping exceptions in dtors is absolutely the
>>> right thing to do.
>>>       
>> Yes, I may have said that backwards. As I've said before, although I use 
>> C++, it's not a "native language" for me, and a lot of this is based on 
>> opinions others have strongly stated 
>>     
> It wouldn't surprise me a bit if others had strongly stated ctors
> shouldn't throw. It used to be the advice in Stroustrup's books.
>   
And often I think it makes sense to construct a viable object even in 
the face of errors, and restrict the behavior of the object later.
>> rather than my own knowledge or experience. Another reason,
>> incidentally, for not trying to come down too hard on one side or
>> the other where language concerns might trump "threading" concerns.
>>     
>>>> and they have some legitimate concerns about common C++ language
>>>> patterns that might pretty much prevent a cancel from ever doing
>>>> what a cancel should do.
>>>>         
>>> Really, legitimate concerns? I can't think of any recommended
>>> patterns that would act that way.
>>>       
>> Hmm. OK. That's interesting. Well, you're the C++ expert here. ;-)
>>     
> None other than, "prevent exceptions from leaking across language
> boundaries."  But if you don't do that, your program is broken anyway.
>   
Well, non-exception-savvy languages, sure. But (OK, perhaps a VMS bias 
here) I think all languages on a platform should have a common exception 
model that interoperates cleanly when stack frames are interleaved. 
Certainly if you're interleaved for Java, or Ada, you should be able to 
have an exception propagate through and unwind, run destructors (or Ada 
finally clauses), etc., with no problems. And where ISO C is extended 
for exception semantics (e.g., VMS, Tru64, Windows...), C can join the club.

Sigh. The best thing about the abandoned effort to build an Itanium ABI 
for the Single UNIX Specification was that we'd succeeded in extracting 
the specification of a C++ exception runtime as the beginning of a 
standard cross platform and cross language common exception runtime.
>>> You don't find the idea that exception-safe code implies cancel-safe
>>> code technically compelling?  I don't think that's an emotional issue.
>>>       
>> Well, yes, I do, 
>>     
> find it technically compelling, or think it's an emotional issue
>   
What I meant was the former. However the fact that I, or you, find it 
technically compelling doesn't make it right, and certainly doesn't mean 
everyone will agree. And historically the disagreement has indeed been 
emotional. So, both are accurate.
>> because cancel was always intended to be "just" an exception that
>> happened to be thrown from another thread. But then, nothing is ever
>> that simple; the asynchronous nature required controls like
>> cancelability type and state. 
>>     
> "Asynchronous nature?"
>
> I haven't even been considering asynchronous cancellation as it's
> completely untenable to write anything but the most restricted code
> that could work in the face of async cancellation.
>   
Ah, but cancellation is basically asynchronous with respect to the 
receiving thread. Even though we deliver the exception only at defined 
synchronous points, the cancellation request can arrive at any instant. 
This is mostly relevant when you talk about blocking behavior -- that a 
blocking operation can be interrupted anywhere in the middle IS 
asynchronous.

"Deferred" cancelability converts that asynchronous interrupt into a 
synchronous exception, though the definition of the blocking operation 
as a cancellation point. So the cancelled thread doesn't necessarily see 
the exception as "asynchronous" (it called a function, and got back an 
exception); but that doesn't change the fact that it really was 
asynchronous all the same.

But I don't mean it in anything like the sense of "asynchronous 
cancelability mode", where the exception can be raised (cancel 
delivered) at any arbitrary point. Asynchronous cancelability was 
invented for use in tight compute-bound loops, and intended to be 
unusable anywhere else. It's one of the things we thought reasonable at 
the time but that I've since become convinced should have been on the 
other side of the "too unsafe and rarely useful to be standardized" line.
>> C++ exceptions are synchronous and non-interrupting. (The latter a
>> consequence of the former, really.)  One of the main advantages of
>> cancellation is that it can break through an extended blocking
>> operation; 
>>     
> If you make all blocking operations cancellation points you can do
> that anyway.  No?
>   
That's the intent...
>> but that's unavoidably an extra condition over "exception-safety".
>> Cancel-safe has to mean something more unless we drop
>> interruptibility. If we drop it, then cancel-safety is just
>> exception-safety but loses much of its value in controlling
>> application responsiveness.
>>     
> You lost me.  I think async cancel safety should be thought of as a
> separate level of design.
>   
Um, OK; I'm not sure where I lost you. I'm not talking about 
asynchronous cancelability. I personally don't think C++ should even 
briefly entertain the notion of any support for that.

However, when cancellation is enabled, any blocking call (or any 
method/operator that makes or might make a blocking call, like "cout<<", 
might raise an exception. Not all code will be prepared to handle that, 
and much shouldn't be; it's important to be able to disable 
cancelability dynamically over critical scopes. It's not like most 
exceptions where the conditions for an exception are generally static; 
it could happen at any time for reasons the current thread cannot 
possibly anticipate. It's asynchronous simply because it's external and 
independent. Also, where a normal exception means "something's wrong and 
I can't continue on this code path", cancellation means sometime subtly 
different -- "I've been asked not to" rather than "I can't"; but if you 
must, you may. ;-)

Therefore we have cancelability state to managed scoped local control 
over when the thread can respond to cancellation requests. (An obvious 
candidate, in C++, for guard objects.)
>> In any case, though, I wasn't suggesting that you need to convince
>> me.  I'm saying there are diverse and strongly held positions that
>> somehow need to be unified in order to get consensus on any
>> proposal. I think that I'm the least of your worries. ;-)
>>     
> Not that you have any obligation to do so, but it might be easier if
> you would recognize the weight your opinion carries.  That might mean
> learning enough about C++ to form a definite opinion.  That's, at
> least, what I've tried to do with threading.
>   
I'm not ignorant of C++, and I'm much less ignorant than I was 2 years 
ago when I started working with C++ and STL on a regular basis. Still, I 
am not steeped in the history and tradition of C++ as I am in threads, 
and probably never will be. More than that, while I have an 
authoritative voice on the POSIX working group and in the community, I'm 
not involved with the C++ committee and have no time or management 
support to get involved; and I won't put myself in the position of being 
an outside expert in some other area pretending to tell the C++ 
committee what it must (or even should) do. I will happily say that as a 
thread expert and C++ dabbler, this is what seems to make sense to me; 
but I reject any aura of authority in the C++ side of semantics and syntax.

However my statement above wasn't in any way related to my tradition of 
C++ deference. I was merely stating that I've seen many opinions (other 
than mine) that will need to be resolved or accommodated to make a standard.
>>>> Someone needs to propose and champion "the great exception
>>>> compromise"; but if that's to be me I don't yet have the faintest
>>>> germ of a notion what it might be. So I sure hope it's going to be
>>>> someone else. ;-)
>>>>         
>>> If "finalized cancellation exceptions result in a new throw at the
>>> next cancellation point" isn't enough of a compromise, it isn't going
>>> to be me either, because I'm out of new ideas.
>>>
>>> Okay, how about this one: we count the number of times the
>>> cancellation is discarded.  The cancelling thread can specify the
>>> number of discards to tolerate, where the default is infinite.  After
>>> that, at the next cancellation point all pthread cancellation handlers
>>> (but not dtors or catch blocks) are run and the thread is terminated.
>>> Heck, at that point I don't care what happens; you're gambling anyway.
>>> Run all the dtors and catch blocks for all I care.
>>>       
>> I do NOT favor any model where "dtor/catch" and "cancellation handler" 
>> don't mean the same thing.
>>     
> Like I said, I don't care at that point.  A forced cancellation is a
> big gamble.  If "you" want to roll the dice, it's your funeral.
>   
>> I don't think the count is tenable either because although it always 
>> feels tempting to add a control dial, it doesn't solve any actual 
>> problem if there's nobody who can know to what value the dial should be 
>> set. 
>>     
> Which is why I backed off to the simpler model below.
>   
>> If "canceled" state persists when the exception is discarded, then 
>> cancel is something different from just "an exception"; 
>>     
> It already was something different.  The state needed to be stored
> somewhere until the next cancellation point.  This just says that the
> state persists until otherwise specified.
>   
>> which is too bad, but perhaps inevitable. You can't just catch it
>> and continue -- you need to somehow also reset that state to recover
>> your workgroup thread that's serially running RPC requests (or
>> Python code, whatever). 
>>     
>
> We could do something awful, like have catch-cancellation-by-value
> cause the state to be reset, while catch(...) and
> catch-cancellation-by-reference don't.  That would preserve the
> convenience, at least.
>   
Um, yeah; I suppose it would. But I agree more strongly about the 
"awful". ;-)
>> A lot of people have suggested various ways of making cancel-pending
>> persist after the exception is launched; that's not necessarily
>> "wrong", but it isn't "simple" either 
>> and somehow it doesn't feel
>> right to me.
>>     
> It's simpler, by most measures I can think of, than resetting the
> state upon throwing.
>   
>>> A simpler approach might be to have two kinds of exception: "forced"
>>> and finalizable.  At least then we can say that exception-safe code
>>> implies finalizable cancellation safety.  Then "forced" synchronous
>>> cancellation can do whatever people desire.  I personally think it
>>> will become a useless appendage sort of like C++ exception
>>> specifications, but at least evolution will take care of it.  And if
>>> I'm wrong, evolution will wilt my finalizable cancellations.
>>>       
>> Is this the "unwind" vs "exception" idea? (Where "unwind" is like a new 
>> sort of 'throw' that triggers dtors but can't be caught/finalized.) Or 
>> something different...?
>>     
> No, I wasn't suggesting anything that couldn't be caught.  I was just
> suggesting an exception that couldn't be stopped.  It could throw
> itself in its dtor (not that I'm advocating it, but it might satisfy
> the "other side"), for example.
>   
The POSIX model where cancel propagates inexorably to thread termination 
is an inherently flawed compromise; but simply the best we could do 
within the context of ISO C and POSIX APIs. OUR implementation always 
allowed finalization, via C++ catch(...), our ISO C "CATCH_ALL" 
extensions, or whatever other language syntax might fit.

I really wouldn't want to propagate this restriction to C++.
> In fact, a general mechanism like:
>
>    cancel( thread_id, exception_object );
>
> is possible, where "cancel" really means throw a copy of the given
> exception object when the specified thread reaches the next
> cancellation point.
>
> We could call it 
>
>    throw_synchronously( thread_id, exception_object );
>    
> instead, if "cancel" really means forced execution to too many people.
>   
That's actually where we started out in CMA. Resolving down to a single 
pre-defined exception was partly a matter of simplicity, but also 
represented a basic thread model that "a thread is simple; it's an 
asynchronous procedure call within the context of an application, not an 
independent application". In any case where you would need to 
distinguish between two separate interrupt conditions, the functions 
stimulated by those separate interrupts should have been assigned to 
separate threads and therefore only one exception is needed.

While this made a lot of sense at the time, we were in a very academic 
and theoretical phase, and there was not that great a body of threaded 
code in 1987 -- and none using anything closely resembling the thread 
model we were inventing and that became the principal influence for 
POSIX. SRC's Firefly/Modula-3 had "alert", but it was so much simpler as 
to be a distinct variety of beast.

One advantage, though, of the single cancel exception, is that it's 
universal. When you asynchronously issue a cancel request for a thread, 
you can't really know what code is executing: your's, STL, some other 
shared library, etc. Cancel means the same to all of them, and either is 
supported with commonly agreed semantics or will be ignored (by 
disabling cancellation in critical scopes). Once you start firing off 
your own arbitrary exceptions, though, anything might happen because 
half the time the exceptions won't belong anywhere in the call tree 
that's active at the time they arrive.

Which brings us back to the "academic" resolution: if an exception means 
distinct things in different call trees, those call trees should be 
distinct threads and only one universal exception is necessary. ;-)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: david.butenhof.vcf
Type: text/x-vcard
Size: 476 bytes
Desc: not available
URL: <http://sourcerytools.com/pipermail/c++-pthreads/attachments/20060306/dd909ceb/attachment.vcf>