www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Nothrow, pure in druntime

reply dsimcha <dsimcha yahoo.com> writes:
Regarding ticket 14 in druntime:

http://www.dsource.org/projects/druntime/ticket/14

I've started looking at the best way to annotate all the pure/nothrow
functions in druntime.  A lot of druntime functionality calls functions from
the C standard library or the Windows/POSIX API.  I figure the best way to get
started is to deal w/ these first, and then worry about the higher level stuff
that's built on to of it.  These functions are obviously nothrow, since C
doesn't even support exceptions.  First of all, is there a decent automated
way to fix all of the declarations for these functions so that they're nothrow?

Secondly, since we don't have the sources to the C libraries for which the
function prototypes are declared, we can't be absolutely sure about their
purity.  A good portion of the ones that seem like they could be pure, mostly
math stuff like sqrt, log, etc., are probably messing w/ errno, and
theoretically they could be memoizing stuff, etc.  What's a reasonable course
of action for these?  It seems like pure is most useful for mathematical
functions, , so it would really be a shame not to use it here.
Jan 25 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
dsimcha wrote:

 I've started looking at the best way to annotate all the pure/nothrow
 functions in druntime.  A lot of druntime functionality calls functions from
 the C standard library or the Windows/POSIX API.  I figure the best way to get
 started is to deal w/ these first, and then worry about the higher level stuff
 that's built on to of it.  These functions are obviously nothrow, since C
 doesn't even support exceptions.  First of all, is there a decent automated
 way to fix all of the declarations for these functions so that they're nothrow?


I did think of making all functions that are extern(C) automatically 
nothrow, but was concerned that it would result in a lot of bugs and 
broken code from ones that did throw.


 Secondly, since we don't have the sources to the C libraries for which the
 function prototypes are declared, we can't be absolutely sure about their
 purity.  A good portion of the ones that seem like they could be pure, mostly
 math stuff like sqrt, log, etc., are probably messing w/ errno, and
 theoretically they could be memoizing stuff, etc.  What's a reasonable course
 of action for these?  It seems like pure is most useful for mathematical
 functions, , so it would really be a shame not to use it here.


Functions that modify errno cannot be pure. What needs to be done is go 
through the definition of each. For example, strlen, strcmp, etc. can be 
marked as pure. printf and strtok cannot.

The math functions can be a problem since earlier incarnations often 
messed with global state, but with the advent of thread safe C runtime 
libraries, this shouldn't be an issue anymore.
Jan 25 2009
next sibling parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Walter Bright (newshound1 digitalmars.com)'s article

 dsimcha wrote:

 I've started looking at the best way to annotate all the pure/nothrow
 functions in druntime.  A lot of druntime functionality calls functions from
 the C standard library or the Windows/POSIX API.  I figure the best way to get
 started is to deal w/ these first, and then worry about the higher level stuff
 that's built on to of it.  These functions are obviously nothrow, since C
 doesn't even support exceptions.  First of all, is there a decent automated
 way to fix all of the declarations for these functions so that they're nothrow?


 I did think of making all functions that are extern(C) automatically
 nothrow, but was concerned that it would result in a lot of bugs and
 broken code from ones that did throw.


I assume, when referring to the ones that do throw, you mean functions written
in
C++ or D, but declared w/ C linkage.  If so, you could make this a per-module
setting that defaults to not assuming nothrow.  For example, let's say you made
this pragma(Linkage, nothrow).  Then, if this statement is seen at the top of a
module, everything declared with extern(Linkage) in that module is assumed to be
nothrow.  For standard C, Windows and POSIX API functions and for any library
written in pure C, I believe (correct me if I'm wrong) this would be a safe
assumption.  At any rate, it would make nothrow a heck of a lot more usable.



 Functions that modify errno cannot be pure. What needs to be done is go
 through the definition of each. For example, strlen, strcmp, etc. can be
 marked as pure. printf and strtok cannot.
 The math functions can be a problem since earlier incarnations often
 messed with global state, but with the advent of thread safe C runtime
 libraries, this shouldn't be an issue anymore.


Isn't errno defined in some implementations to be thread-local?  If so, I guess
we
still have a problem.  Then again, in the long run it probably makes sense to
reimplement a lot of the math stuff that still uses the C std lib in pure D
anyhow
so that things like CTFE work on it, but in the sort run I'm sure that's not
anyone's top priority.
Jan 25 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
dsimcha wrote:

 I assume, when referring to the ones that do throw, you mean functions written
in
 C++ or D, but declared w/ C linkage.  If so, you could make this a per-module
 setting that defaults to not assuming nothrow.  For example, let's say you made
 this pragma(Linkage, nothrow).  Then, if this statement is seen at the top of a
 module, everything declared with extern(Linkage) in that module is assumed to
be
 nothrow.  For standard C, Windows and POSIX API functions and for any library
 written in pure C, I believe (correct me if I'm wrong) this would be a safe
 assumption.  At any rate, it would make nothrow a heck of a lot more usable.


This will do it:

nothrow:
... rest of module ...



 Isn't errno defined in some implementations to be thread-local?


Yes, but thread local doesn't mean pure.


 If so, I guess we
 still have a problem.  Then again, in the long run it probably makes sense to
 reimplement a lot of the math stuff that still uses the C std lib in pure D
anyhow
 so that things like CTFE work on it, but in the sort run I'm sure that's not
 anyone's top priority.


Don has already reimplemented most of them in D, this was done to:

1. ensure a minimum level of performance and accuracy; some C ones are 
crappily done

2. properly support all the D floating point types and overloading rules

3. support NAN and INFINITY correctly
Jan 25 2009
parent reply Don <nospam nospam.com> writes:
Walter Bright wrote:

 dsimcha wrote:

 I assume, when referring to the ones that do throw, you mean functions 
 written in
 C++ or D, but declared w/ C linkage.  If so, you could make this a 
 per-module
 setting that defaults to not assuming nothrow.  For example, let's say 
 you made
 this pragma(Linkage, nothrow).  Then, if this statement is seen at the 
 top of a
 module, everything declared with extern(Linkage) in that module is 
 assumed to be
 nothrow.  For standard C, Windows and POSIX API functions and for any 
 library
 written in pure C, I believe (correct me if I'm wrong) this would be a 
 safe
 assumption.  At any rate, it would make nothrow a heck of a lot more 
 usable.


 
 This will do it:
 
 nothrow:
 ... rest of module ...
 
 

 Isn't errno defined in some implementations to be thread-local?


 
 Yes, but thread local doesn't mean pure.
 

 If so, I guess we
 still have a problem.  Then again, in the long run it probably makes 
 sense to
 reimplement a lot of the math stuff that still uses the C std lib in 
 pure D anyhow
 so that things like CTFE work on it, but in the sort run I'm sure 
 that's not
 anyone's top priority.


 
 Don has already reimplemented most of them in D, this was done to:
 
 1. ensure a minimum level of performance and accuracy; some C ones are 
 crappily done
 
 2. properly support all the D floating point types and overloading rules
 
 3. support NAN and INFINITY correctly


tango.math doesn't use the C library at all, except when inline asm is 
unavailable. Of they differ from the C functions, in that none of them 
set errno!
One really annoying issue still remains, though -- the floating point 
flags in the CPU. They are entirely deterministic, but are they 
considered to be part of the return value of the function? Or would we 
allow them to be ignored?
A compiler could check the exception flags before allowing memoisation. 
But one could also do the same thing for 'errno'.

Likewise, floating point rounding modes. Essentially, the floating point 

to during every floating point operation.


could deal with it by regarding that as a whole-program setting. But 
(depending on the CPU), the old exception flags generally get ORed with 
the new exception flags.

Also, you can set the flags to allow any floating point function to 
throw a hardware exception. It's difficult for any function using 
floating point to claim to be nothrow under ANY circumstances; but 
that's a horrible limitation.
Jan 26 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Don wrote:

 tango.math doesn't use the C library at all, except when inline asm is 
 unavailable. Of they differ from the C functions, in that none of them 
 set errno!
 One really annoying issue still remains, though -- the floating point 
 flags in the CPU. They are entirely deterministic, but are they 
 considered to be part of the return value of the function? Or would we 
 allow them to be ignored?
 A compiler could check the exception flags before allowing memoisation. 
 But one could also do the same thing for 'errno'.
 
 Likewise, floating point rounding modes. Essentially, the floating point 

 during every floating point operation.
 

 could deal with it by regarding that as a whole-program setting. But 
 (depending on the CPU), the old exception flags generally get ORed with 
 the new exception flags.


Those are good points. I don't know what the answer is. My inclination 
is to say if your program relies on changing the rounding mode or 
fiddles with the exception flags, it's undefined behavior.


 Also, you can set the flags to allow any floating point function to 
 throw a hardware exception. It's difficult for any function using 
 floating point to claim to be nothrow under ANY circumstances; but 
 that's a horrible limitation.


I would say that is not supported by D. I've never heard of a use for them.
Jan 26 2009
parent reply Don <nospam nospam.com> writes:
Walter Bright wrote:

 Don wrote:

 tango.math doesn't use the C library at all, except when inline asm is 
 unavailable. Of they differ from the C functions, in that none of them 
 set errno!
 One really annoying issue still remains, though -- the floating point 
 flags in the CPU. They are entirely deterministic, but are they 
 considered to be part of the return value of the function? Or would we 
 allow them to be ignored?
 A compiler could check the exception flags before allowing 
 memoisation. But one could also do the same thing for 'errno'.

 Likewise, floating point rounding modes. Essentially, the floating 

 written to during every floating point operation.


 could deal with it by regarding that as a whole-program setting. But 
 (depending on the CPU), the old exception flags generally get ORed 
 with the new exception flags.


 
 Those are good points. I don't know what the answer is. My inclination 
 is to say if your program relies on changing the rounding mode or 
 fiddles with the exception flags, it's undefined behavior.


One form of error analysis is to run the program with different rounding 
modes, and compare the results.

You can also use rounding modes to implement interval arithmetic, but 
this would normally be restricted to low-level functions. The rounding 
mode would not escape from those functions.

I normally use the exception flags for debugging.


 Also, you can set the flags to allow any floating point function to 
 throw a hardware exception. It's difficult for any function using 
 floating point to claim to be nothrow under ANY circumstances; but 
 that's a horrible limitation.


 
 I would say that is not supported by D. I've never heard of a use for them.


What happens if a nothrow function throws an exception? IMHO a 
satisfactory response would be to abort the program with an error 
message/ drop you into a debugger -- anyway, that's the only thing I use 
when running with FP exceptions enabled.

I guess it's reasonable to argue that using the floating-point flags is 
sufficiently hard-core that pure and nothrow should pretend that they 
don't exist.

Still, some functions (especially correctly-rounded floating-point i/o) 
go to a lot of trouble to support them.  I have a suspicion that it's 
not worth the effort.
Jan 26 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Don wrote:

 Walter Bright wrote:

 Don wrote:

 tango.math doesn't use the C library at all, except when inline asm 
 is unavailable. Of they differ from the C functions, in that none of 
 them set errno!
 One really annoying issue still remains, though -- the floating point 
 flags in the CPU. They are entirely deterministic, but are they 
 considered to be part of the return value of the function? Or would 
 we allow them to be ignored?
 A compiler could check the exception flags before allowing 
 memoisation. But one could also do the same thing for 'errno'.

 Likewise, floating point rounding modes. Essentially, the floating 

 written to during every floating point operation.


 could deal with it by regarding that as a whole-program setting. But 
 (depending on the CPU), the old exception flags generally get ORed 
 with the new exception flags.


 Those are good points. I don't know what the answer is. My inclination 
 is to say if your program relies on changing the rounding mode or 
 fiddles with the exception flags, it's undefined behavior.


 
 One form of error analysis is to run the program with different rounding 
 modes, and compare the results.
 
 You can also use rounding modes to implement interval arithmetic, but 
 this would normally be restricted to low-level functions. The rounding 
 mode would not escape from those functions.
 
 I normally use the exception flags for debugging.
 

 Also, you can set the flags to allow any floating point function to 
 throw a hardware exception. It's difficult for any function using 
 floating point to claim to be nothrow under ANY circumstances; but 
 that's a horrible limitation.


 I would say that is not supported by D. I've never heard of a use for 
 them.


 
 What happens if a nothrow function throws an exception? IMHO a 
 satisfactory response would be to abort the program with an error 
 message/ drop you into a debugger -- anyway, that's the only thing I use 
 when running with FP exceptions enabled.


I'd be ok with saying throwing fp exceptions is a non-recoverable error, 
like a seg fault or stack overflow, and is acceptable in a nothrow function.



 I guess it's reasonable to argue that using the floating-point flags is 
 sufficiently hard-core that pure and nothrow should pretend that they 
 don't exist.
 
 Still, some functions (especially correctly-rounded floating-point i/o) 
 go to a lot of trouble to support them.  I have a suspicion that it's 
 not worth the effort.


So we have two options. One is to say that floating point arithmetic 
cannot be made pure. The other is to ignore the problem (saying it's 
undefined behavior).
Jan 26 2009
next sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2009-01-26 14:21:18 -0500, Walter Bright <newshound1 digitalmars.com> said:


 I guess it's reasonable to argue that using the floating-point flags is 
 sufficiently hard-core that pure and nothrow should pretend that they 
 don't exist.
 
 Still, some functions (especially correctly-rounded floating-point i/o) 
 go to a lot of trouble to support them.  I have a suspicion that it's 
 not worth the effort.


 
 So we have two options. One is to say that floating point arithmetic 
 cannot be made pure. The other is to ignore the problem (saying it's 
 undefined behavior).


I see another: the compiler keeps track of the state of floating point 
flags inside a function, and prevent pure optimisations across those 
boundaries. Of course, the language would have to expose a mean to 
change these flags in order for it to work... and prevent any such 
change from leaking after the lifetime of a function.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Jan 26 2009
prev sibling parent Don <nospam nospam.com> writes:
Walter Bright wrote:

 Don wrote:

 Walter Bright wrote:

 Don wrote:

 tango.math doesn't use the C library at all, except when inline asm 
 is unavailable. Of they differ from the C functions, in that none of 
 them set errno!
 One really annoying issue still remains, though -- the floating 
 point flags in the CPU. They are entirely deterministic, but are 
 they considered to be part of the return value of the function? Or 
 would we allow them to be ignored?
 A compiler could check the exception flags before allowing 
 memoisation. But one could also do the same thing for 'errno'.

 Likewise, floating point rounding modes. Essentially, the floating 

 written to during every floating point operation.


 We could deal with it by regarding that as a whole-program setting. 
 But (depending on the CPU), the old exception flags generally get 
 ORed with the new exception flags.


 Those are good points. I don't know what the answer is. My 
 inclination is to say if your program relies on changing the rounding 
 mode or fiddles with the exception flags, it's undefined behavior.


 One form of error analysis is to run the program with different 
 rounding modes, and compare the results.

 You can also use rounding modes to implement interval arithmetic, but 
 this would normally be restricted to low-level functions. The rounding 
 mode would not escape from those functions.

 I normally use the exception flags for debugging.


 Also, you can set the flags to allow any floating point function to 
 throw a hardware exception. It's difficult for any function using 
 floating point to claim to be nothrow under ANY circumstances; but 
 that's a horrible limitation.


 I would say that is not supported by D. I've never heard of a use for 
 them.


 What happens if a nothrow function throws an exception? IMHO a 
 satisfactory response would be to abort the program with an error 
 message/ drop you into a debugger -- anyway, that's the only thing I 
 use when running with FP exceptions enabled.


 
 I'd be ok with saying throwing fp exceptions is a non-recoverable error, 
 like a seg fault or stack overflow, and is acceptable in a nothrow 
 function.


There are some important uses for catching fp exceptions, including one 
which in the revised IEEE standard: the product of an array of doubles.
eg real.max * real.max * real.min * real.min == 1.0, but you get 
overflows during the calculation.

What you do is enable floating point overflow and underflow as an 
exception, and set up an exception handler inside that function; the 
handler is essentially an inner function. Then you have a seperate 
counter for how many times it has overflowed.
The thing to note about this is that although FP exceptions are used, 
they never leave the function which created them, so there's no stack 
unwinding, no destructors are called, etc.

All functionality would be preserved if it is a non-recoverable error to 
transmit an fp exception across a function boundary (but possible to 
catch the fp exception in the function which generated it).

But possibly such situations are so small in number that they can all be 
put in the standard library.


 I guess it's reasonable to argue that using the floating-point flags 
 is sufficiently hard-core that pure and nothrow should pretend that 
 they don't exist.

 Still, some functions (especially correctly-rounded floating-point 
 i/o) go to a lot of trouble to support them.  I have a suspicion that 
 it's not worth the effort.


 
 So we have two options. One is to say that floating point arithmetic 
 cannot be made pure. The other is to ignore the problem (saying it's 
 undefined behavior).


It might not be a difficulty to ignore the problem. The flags are 
getting very hard to use these days, since the x87 flags are different 
from the SSE flags.

I would recommend stating that when calling a pure function, the state 
of the FPU flags is implementation-dependent. Even better would be if we 
can state that the built-in math functions will respect the flags, 
despite being pure. They are very unlikely to be memoized since they are 
so simple.
Jan 27 2009
prev sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2009-01-25 14:39:52 -0500, Walter Bright <newshound1 digitalmars.com> said:


 I did think of making all functions that are extern(C) automatically 
 nothrow, but was concerned that it would result in a lot of bugs and 
 broken code from ones that did throw.


It's better that way, because it allows you to write proper wrappers 
for C++ functions that may throw exceptions. Like this:

	extern "C"
	int _doSomething_wrapper(int value)
	{
		try
		{
			return doSomething(value);
		}
		catch (const std::exception & e)
		{
			_d_throw_exception(e.what());
		}
	}

with _d_throw_exception written in D and throwing an exception.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Jan 26 2009