www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Static ctors and thin ice

reply Kris <Kris_member pathlink.com> writes:
This is disheartening;

Sooner or later, most D programmers will find the need for a static ctor. In the
Mango libraries, they are used to initialize all manner of singletons; e.g.
Stdout & Stdin, various lookup tables (Uri, ServletContext, etc), even instances
of exceptions. This is done so the rest of the code can make certain valid
assumptions about the environment, which leads to a faster, more manageable, and
more maintainable code-base overall.

Unfortunately, I had assumed the D compiler had a solid foundation for such
things ~ instead, this foundation is pretty much non-existent. I'll tell you
why:

1) Static ctors will often form a dependency graph. Therefore, the order in
which static ctors are invoked must respect any and all interdependencies.
Without this behavior, the utility of static ctors is seriously crippled.
Looking at the compiler source-code, there is precious little evidence that it
does anything particularly useful with regard to this ordering. Hence, whilst
the runtime code (moduleinit.d) tries valiantly to visit the constructors in a
valid order, the data-structures it traverses are laid out in a fashion dictated
by the order of compilation. This is farcical. As it turns out, Walter has
recently added a delicate little footnote to the relevant documentation (“the
order in which they are called is not defined”).

2) There’s also a little secret that you may not know about ~ in order to know
anything about static ctors, the compiler must see them within the modules it
compiles. To wit; the list of said ctors is managed by the compiler only, and
not by the linker. Suppose you are a non-open-source corporate entity, and you
wish to hide the implementation of your algorithmic crown jewels … you must
still expose the source-code for all static constructors, which means all that
precious data which you load up into runtime-containers must be exposed for all
to see. 

There’s lot’s of things to like about D, and there are some things that are
pretty ugly. What bother me most is that working with D often feels like playing
a game of Poker. IMO this is an unreasonable state of affairs, and here’s one
developer who’s pretty fed up with it.

I make a humble request that (1) be resolved quickly; it can be done by building
a simple dependency graph of the imports at compile-time (perhaps in
Module::genmoduleinfo; all the relevant information is ready & accessible
there), and emitting the module-structures in the resultant order. This would
both resolve the issue, and greatly simplify the valiant, yet thoroughly borked,
traversal performed at startup time. It would also permit the ModuleInfo to be
placed in the CONST segment instead. 

How about it, Walter?

As for (2), I don’t see a resolution without the linker being involved.
Feb 13 2005
next sibling parent reply Derek Parnell <derek psych.ward> writes:
On Sun, 13 Feb 2005 23:43:01 +0000 (UTC), Kris wrote:

[snip]
 

 As for (2), I don’t see a resolution without the linker being involved.


Just clarifying this for myself, but are you saying that if the static ctor
is linked in (and not compiled) that the ctor is not run?

-- 
Derek
Melbourne, Australia
14/02/2005 11:00:30 AM
Feb 13 2005
parent Kris <Kris_member pathlink.com> writes:
I may have made a rather glaring booboo regarding that point, Derek. Looking at
the compiler source-code again, I realized it was doing things rather
differently that I had originally thought. So the answer is, I don't know.
Although I thought I did :-(

Felt like I was on thin-ice already with this issue; I think one leg just went
through

- Kris


In article <yelthh1kf26l$.i5r2b5jnerb3.dlg 40tude.net>, Derek Parnell says...

On Sun, 13 Feb 2005 23:43:01 +0000 (UTC), Kris wrote:

[snip]
 

 As for (2), I don’t see a resolution without the linker being involved.


Just clarifying this for myself, but are you saying that if the static ctor
is linked in (and not compiled) that the ctor is not run?

-- 
Derek
Melbourne, Australia
14/02/2005 11:00:30 AM
Feb 13 2005
prev sibling parent reply Kris <Kris_member pathlink.com> writes:
I made a tragic mistake in assessing the method in which ModuleInfo is
constructed, so the original suggestion was nonsense. Armed with a bit more
knowledge though, I'll have another go:

Each object file appears to contain a list of ModuleInfo descriptors,
representing those imports which have static ctors ~ where the list order
follows that of the module source-code. By maintaining such a list within each
module object-file, the design apparently expects to resolve static-ctor
dependencies at startup time by recursively traversing the ModuleInfo structures
across all files linked. I'm guessing there's some voodoo that happens to link
the lists together.

The problem occurs when one module acts as a 'proxy' for another in this regard.
Specifically:

module C has a static ctor
module B imports, and references C, but has no static ctor of its own
module A imports B, and has a static ctor *indirectly* dependent upon C

In this case, A does not need to import C (doesn't even know it exists), though
it must import B. However, the compiler mistakenly drops C from the list
belonging to A because the import of B does not expose any static-ctors therin.
C is orphaned via the lack of a static-ctor in B.

Thus, the chain of dependencies is broken. I checked this by adding a dummy
static ctor (module-level) to B and, sure enough, the graph was restored.

This whole thing might be resolved by ensuring the compiler does not drop C,
just because B does not have a static-ctor. The blunt way to do this would be to
implicitly add a static-ctor to those modules which don't have one themselves,
but which import modules that do (which is already done for static-class-ctors).

I may be wrong in this assessment also, but feel the ice is thicker where I'm
standing now

- Kris


In article <cuoom4$2614$1 digitaldaemon.com>, Kris says...

a simple dependency graph of the imports at compile-time (perhaps in
Module::genmoduleinfo; all the relevant information is ready & accessible
there), and emitting the module-structures in the resultant order. This would
both resolve the issue, and greatly simplify the valiant, yet thoroughly
borked, traversal performed at startup time.
Feb 13 2005
next sibling parent "Regan Heath" <regan netwin.co.nz> writes:
On Mon, 14 Feb 2005 01:36:09 +0000 (UTC), Kris <Kris_member pathlink.com>  
wrote:

 I made a tragic mistake in assessing the method in which ModuleInfo is
 constructed, so the original suggestion was nonsense. Armed with a bit  
 more
 knowledge though, I'll have another go:

 Each object file appears to contain a list of ModuleInfo descriptors,
 representing those imports which have static ctors ~ where the list order
 follows that of the module source-code. By maintaining such a list  
 within each
 module object-file, the design apparently expects to resolve static-ctor
 dependencies at startup time by recursively traversing the ModuleInfo  
 structures
 across all files linked. I'm guessing there's some voodoo that happens  
 to link
 the lists together.

 The problem occurs when one module acts as a 'proxy' for another in this  
 regard.
 Specifically:

 module C has a static ctor
 module B imports, and references C, but has no static ctor of its own
 module A imports B, and has a static ctor *indirectly* dependent upon C

 In this case, A does not need to import C (doesn't even know it exists),  
 though
 it must import B. However, the compiler mistakenly drops C from the list
 belonging to A because the import of B does not expose any static-ctors  
 therin.
 C is orphaned via the lack of a static-ctor in B.

 Thus, the chain of dependencies is broken. I checked this by adding a  
 dummy
 static ctor (module-level) to B and, sure enough, the graph was restored.

 This whole thing might be resolved by ensuring the compiler does not  
 drop C,
 just because B does not have a static-ctor. The blunt way to do this  
 would be to
 implicitly add a static-ctor to those modules which don't have one  
 themselves,
 but which import modules that do (which is already done for  
 static-class-ctors).


It seems to me that:
   "A imports B" && "B imports C" == "A imports C"

regardless of whether any module has a static ctor.

Can the compiler simply give each module a number which refers to the  
deepest level in the dependancy tree it occurs at, then process that list  
 from the biggest entry first?

You can drop from the list an entry that does not have a static ctor  
itself, as long as you still count it as increasing the depth.

I'm not sure whether circular imports will cause trouble?

i.e.

A imports B imports C

A = 0
B = 1
C = 2

process C, process B, process A.


 I may be wrong in this assessment also, but feel the ice is thicker  
 where I'm
 standing now


You're doing more than I am... I'm sitting inside where it's warm drinking  
hot cocoa dreaming about going out on the ice.

Regan
Feb 13 2005
prev sibling next sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
 module C has a static ctor
 module B imports, and references C, but has no static ctor of its own
 module A imports B, and has a static ctor *indirectly* dependent upon C



This thread might be related: 
http://www.digitalmars.com/d/archives/digitalmars/D/bugs/1773.html

In particular check out
http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D.bugs/1780

I thought Walter had fixed all those problems but I guess not.
-Ben
Feb 13 2005
parent pragma <pragma_member pathlink.com> writes:
In article <cup7gp$2ibs$1 digitaldaemon.com>, Ben Hinkle says...

 module C has a static ctor
 module B imports, and references C, but has no static ctor of its own
 module A imports B, and has a static ctor *indirectly* dependent upon C



This thread might be related: 
http://www.digitalmars.com/d/archives/digitalmars/D/bugs/1773.html

In particular check out
http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D.bugs/1780

I thought Walter had fixed all those problems but I guess not.
-Ben


I thought this problem looked familiar.  Looks like it was only pushed /closer/
to completion than before, Walter *did* change the compiler a little bit.  From
what I've gathered from here and on dsource, the problem is still much the same.

Here's a thought: Is it possible that we're seeing a regression instead of a new
bug?

- EricAnderton at yahoo
Feb 14 2005
prev sibling next sibling parent Daan Oosterveld <daan.oosterveld home.nl> writes:
What about using the same way ctors are resolved by classes. This by 
adding sort of inheritance/dependancy to the module statement.

module std.c.stdio : std.stdio;

Which ensures that std.stdio ctors are called first. This way the 
initialisation is mandatory. If you have to do ctors in a specific order 
than the programmer becomes responsable and not a random algorithm 
dictated by the compiler. Or funky link options by reordering the linkage.

Daan

Kris schreef:

 I made a tragic mistake in assessing the method in which ModuleInfo is
 constructed, so the original suggestion was nonsense. Armed with a bit more
 knowledge though, I'll have another go:
 
 Each object file appears to contain a list of ModuleInfo descriptors,
 representing those imports which have static ctors ~ where the list order
 follows that of the module source-code. By maintaining such a list within each
 module object-file, the design apparently expects to resolve static-ctor
 dependencies at startup time by recursively traversing the ModuleInfo
structures
 across all files linked. I'm guessing there's some voodoo that happens to link
 the lists together.
 
 The problem occurs when one module acts as a 'proxy' for another in this
regard.
 Specifically:
 
 module C has a static ctor
 module B imports, and references C, but has no static ctor of its own
 module A imports B, and has a static ctor *indirectly* dependent upon C
 
 In this case, A does not need to import C (doesn't even know it exists), though
 it must import B. However, the compiler mistakenly drops C from the list
 belonging to A because the import of B does not expose any static-ctors therin.
 C is orphaned via the lack of a static-ctor in B.
 
 Thus, the chain of dependencies is broken. I checked this by adding a dummy
 static ctor (module-level) to B and, sure enough, the graph was restored.
 
 This whole thing might be resolved by ensuring the compiler does not drop C,
 just because B does not have a static-ctor. The blunt way to do this would be
to
 implicitly add a static-ctor to those modules which don't have one themselves,
 but which import modules that do (which is already done for
static-class-ctors).
 
 I may be wrong in this assessment also, but feel the ice is thicker where I'm
 standing now
 
 - Kris
 
 
 In article <cuoom4$2614$1 digitaldaemon.com>, Kris says...
 

a simple dependency graph of the imports at compile-time (perhaps in
Module::genmoduleinfo; all the relevant information is ready & accessible
there), and emitting the module-structures in the resultant order. This would
both resolve the issue, and greatly simplify the valiant, yet thoroughly
borked, traversal performed at startup time.
Feb 14 2005
prev sibling parent reply "Walter" <newshound digitalmars.com> writes:
"Kris" <Kris_member pathlink.com> wrote in message
news:cuova9$2biv$1 digitaldaemon.com...

 Each object file appears to contain a list of ModuleInfo descriptors,
 representing those imports which have static ctors ~ where the list order
 follows that of the module source-code. By maintaining such a list within


each

 module object-file, the design apparently expects to resolve static-ctor
 dependencies at startup time by recursively traversing the ModuleInfo


structures

 across all files linked.


That is how it works. The rule used is straightforward: before a module
constructor can be run, all the module constructors in the modules it
imports must have been run.


 I'm guessing there's some voodoo that happens to link
 the lists together.


It's not voodoo, but it is different between windows and linux. It uses the
same technique used by C++ object files to find the static constructors.


 The problem occurs when one module acts as a 'proxy' for another in this


regard.

 Specifically:

 module C has a static ctor
 module B imports, and references C, but has no static ctor of its own
 module A imports B, and has a static ctor *indirectly* dependent upon C

 In this case, A does not need to import C (doesn't even know it exists),


though

 it must import B. However, the compiler mistakenly drops C from the list
 belonging to A because the import of B does not expose any static-ctors


therin.

 C is orphaned via the lack of a static-ctor in B.

 Thus, the chain of dependencies is broken. I checked this by adding a


dummy

 static ctor (module-level) to B and, sure enough, the graph was restored.

 This whole thing might be resolved by ensuring the compiler does not drop


C,

 just because B does not have a static-ctor. The blunt way to do this would


be to

 implicitly add a static-ctor to those modules which don't have one


themselves,

 but which import modules that do (which is already done for


static-class-ctors).

The problem with having the compiler insert implicit static ctors in modules
that don't have them is that it will impose a likely unresolvable order of
construction problem. In other words, it would become impossible to have
circular imports (which is possible now if at least one of the modules in
the circle doesn't have a static ctor).

Putting a static this(){} in B will resolve the problem. It isn't perfect,
but it works. Note that C++ doesn't guarantee any order at all to static
constructors.
Feb 14 2005
next sibling parent reply Kris <Kris_member pathlink.com> writes:
In article <curj1o$2afu$1 digitaldaemon.com>, Walter says...

"Kris" <Kris_member pathlink.com> wrote in message


 module C has a static ctor
 module B imports, and references C, but has no static ctor of its own
 module A imports B, and has a static ctor *indirectly* dependent upon C

 In this case, A does not need to import C (doesn't even know it exists),


though

 it must import B. However, the compiler mistakenly drops C from the list
 belonging to A because the import of B does not expose any static-ctors


therin.

 C is orphaned via the lack of a static-ctor in B.

 Thus, the chain of dependencies is broken. I checked this by adding a


dummy

 static ctor (module-level) to B and, sure enough, the graph was restored.

 This whole thing might be resolved by ensuring the compiler does not drop


C,

 just because B does not have a static-ctor. The blunt way to do this would


be to

 implicitly add a static-ctor to those modules which don't have one


themselves,

 but which import modules that do (which is already done for


static-class-ctors).

The problem with having the compiler insert implicit static ctors in modules
that don't have them is that it will impose a likely unresolvable order of
construction problem. In other words, it would become impossible to have
circular imports (which is possible now if at least one of the modules in
the circle doesn't have a static ctor).


Thanks for the reply;

As I understand it, such a case is detected by the runtime initialization. At
this time, it throws an exception ~ could it be changed to simply terminate the
tail-chasing instead?



Putting a static this(){} in B will resolve the problem. It isn't perfect,
but it works. 


Right. The true problem with this is education ~ the issue is not even
documented. If it were, then you could hardly expect just anyone to say "oh well
.. of course that's the case". It's not a good place for D to be so, better to
find a compiler-based resolution ~ if indeed there is one



 Note that C++ doesn't guarantee any order at all to static
constructors.


Not interested ~ don't mean to be rude, but we're talking about D

- Kris
Feb 14 2005
parent reply "Walter" <newshound digitalmars.com> writes:
"Kris" <Kris_member pathlink.com> wrote in message
news:curk2i$2b91$1 digitaldaemon.com...

 In article <curj1o$2afu$1 digitaldaemon.com>, Walter says...

The problem with having the compiler insert implicit static ctors in




modules

that don't have them is that it will impose a likely unresolvable order




of

construction problem. In other words, it would become impossible to have
circular imports (which is possible now if at least one of the modules in
the circle doesn't have a static ctor).


 Thanks for the reply;

 As I understand it, such a case is detected by the runtime initialization.


At

 this time, it throws an exception ~ could it be changed to simply


terminate the

 tail-chasing instead?


If A and B mutually import each other, that leaves us with the order of A
and B being initialized as unspecified, although they may or may not depend
on each other. It may work on one platform, yet fail on another in a way
that one can't predict. So I think it is better to throw the exception,
forcing the programmer to organize it in such a way that it is not circular.


Putting a static this(){} in B will resolve the problem. It isn't




perfect,

but it works.


 Right. The true problem with this is education ~ the issue is not even
 documented. If it were, then you could hardly expect just anyone to say


"oh well

 .. of course that's the case". It's not a good place for D to be so,


better to

 find a compiler-based resolution ~ if indeed there is one


The documentation can always be improved. To be honest, I hadn't thought of
the proxy situation you're experiencing. I too prefer to solve it within the
language rather than as a documented wart. I'll think about it and see what
I can do.
Feb 14 2005
parent reply Kris <Kris_member pathlink.com> writes:
In article <curr90$2h5h$1 digitaldaemon.com>, Walter says...

"Kris" <Kris_member pathlink.com> wrote in message
news:curk2i$2b91$1 digitaldaemon.com...

 In article <curj1o$2afu$1 digitaldaemon.com>, Walter says...

The problem with having the compiler insert implicit static ctors in




modules

that don't have them is that it will impose a likely unresolvable order




of

construction problem. In other words, it would become impossible to have
circular imports (which is possible now if at least one of the modules in
the circle doesn't have a static ctor).


 Thanks for the reply;

 As I understand it, such a case is detected by the runtime initialization.


At

 this time, it throws an exception ~ could it be changed to simply


terminate the

 tail-chasing instead?


If A and B mutually import each other, that leaves us with the order of A
and B being initialized as unspecified, although they may or may not depend
on each other. It may work on one platform, yet fail on another in a way
that one can't predict. So I think it is better to throw the exception,
forcing the programmer to organize it in such a way that it is not circular.


Sure; that certainly makes sense. To catch that at compile-time would be rather
difficult, since you don't know all the link-units.


Putting a static this(){} in B will resolve the problem. It isn't




perfect,

but it works.


 Right. The true problem with this is education ~ the issue is not even
 documented. If it were, then you could hardly expect just anyone to say


"oh well

 .. of course that's the case". It's not a good place for D to be so,


better to

 find a compiler-based resolution ~ if indeed there is one


The documentation can always be improved. To be honest, I hadn't thought of
the proxy situation you're experiencing. I too prefer to solve it within the
language rather than as a documented wart. I'll think about it and see what
I can do.



Excellent! 

May I suggest something? 

To recap: right now the proxy is excluded by default, which can cause a problem.
Currently, the resolution is to manually add a dummy static ctor to the proxy. 

Assume (for the moment) the proxy were to be included by default. Further
suppose this then causes a circular-reference exception to be thrown. At that
point, you really *do* have a true circular reference (by proxy) which the
programmer should be 'encouraged' to break. I mean, manually adding a ctor in
such a case would throw the exception too ... right? Excluding the proxy (by
default) would actually *mask* that circularity instead.


Alternatively: 

If the concerns were reversed, would the situation be notably more appealing? I
mean, if the proxy were included by default, and there were some means of
excluding it (perhaps via a pragma)? That seems kinda gnarly though ...

Just thoughts;

- Kris
Feb 14 2005
parent reply "Walter" <newshound digitalmars.com> writes:
"Kris" <Kris_member pathlink.com> wrote in message
news:curtjf$2j35$1 digitaldaemon.com...

 Excellent!

 May I suggest something?


Too late! I already figured out a way to fix it. It's 2 lines of code added
to dmd <g>.
Feb 15 2005
next sibling parent "Ben Hinkle" <ben.hinkle gmail.com> writes:
"Walter" <newshound digitalmars.com> wrote in message 
news:cusdno$20i$1 digitaldaemon.com...

 "Kris" <Kris_member pathlink.com> wrote in message
 news:curtjf$2j35$1 digitaldaemon.com...

 Excellent!

 May I suggest something?


 Too late! I already figured out a way to fix it. It's 2 lines of code 
 added
 to dmd <g>.


oh cool! Then forget about my suggestion about adding more info to the 
static ctors. If you fixed the bug without changing how circular 
dependencies work then that is the best solution.
Feb 15 2005
prev sibling parent kris <fu bar.org> writes:
Walter wrote:

 "Kris" <Kris_member pathlink.com> wrote in message
 news:curtjf$2j35$1 digitaldaemon.com...
 

Excellent!

May I suggest something?


 
 
 Too late! I already figured out a way to fix it. It's 2 lines of code added
 to dmd <g>.
 
 


Good; and thanks!
Feb 15 2005
prev sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
 The problem with having the compiler insert implicit static ctors in 
 modules
 that don't have them is that it will impose a likely unresolvable order of
 construction problem. In other words, it would become impossible to have
 circular imports (which is possible now if at least one of the modules in
 the circle doesn't have a static ctor).


Circular imports are only a problem if the static initializers have circular 
dependencies. One way out is to make this more explicit by putting the 
dependencies on the static ctors somehow. In other words instead of saying 
"this module depends on modules foo and bar" have "this static ctor depends 
on modules foo and bar". That could take the pressure off of circular 
modules and put it closer to the real problem of circular static ctors 
(which should be illegal), and that way modules without an explicit static 
ctor can implicitly recurse over their imported modules.
More concretely, line 120 in std.moduleinit.d
  _moduleCtor2(m.importedModules, 0);
should recurse only over those imported modules that are actually needed by 
the static ctor in the current module m.
I don't have a good syntax for expressing the static ctor dependencies, 
though. I guess by default a static ctor would have all of the module 
imports as dependencies.
Feb 14 2005
next sibling parent "Regan Heath" <regan netwin.co.nz> writes:
On Mon, 14 Feb 2005 21:51:45 -0500, Ben Hinkle <ben.hinkle gmail.com>  
wrote:


 The problem with having the compiler insert implicit static ctors in
 modules
 that don't have them is that it will impose a likely unresolvable order  
 of
 construction problem. In other words, it would become impossible to have
 circular imports (which is possible now if at least one of the modules  
 in
 the circle doesn't have a static ctor).


 Circular imports are only a problem if the static initializers have  
 circular
 dependencies. One way out is to make this more explicit by putting the
 dependencies on the static ctors somehow. In other words instead of  
 saying
 "this module depends on modules foo and bar" have "this static ctor  
 depends
 on modules foo and bar". That could take the pressure off of circular
 modules and put it closer to the real problem of circular static ctors
 (which should be illegal), and that way modules without an explicit  
 static
 ctor can implicitly recurse over their imported modules.
 More concretely, line 120 in std.moduleinit.d
   _moduleCtor2(m.importedModules, 0);
 should recurse only over those imported modules that are actually needed  
 by
 the static ctor in the current module m.
 I don't have a good syntax for expressing the static ctor dependencies,
 though. I guess by default a static ctor would have all of the module
 imports as dependencies.


Ideally this would only be required if a circular import was identified.

Identifying circular imports is difficult isn't it? It could be a circle  
of arbitrary length. I guess the compiler can just barf if it gets too  
'deep' and hopefully give the programmer a good indication of where to use  
this syntax to avoid the circle.

Regan
Feb 14 2005
prev sibling parent reply "Walter" <newshound digitalmars.com> writes:
"Ben Hinkle" <ben.hinkle gmail.com> wrote in message
news:curo45$2eea$1 digitaldaemon.com...

 Circular imports are only a problem if the static initializers have


circular

 dependencies. One way out is to make this more explicit by putting the
 dependencies on the static ctors somehow.


While this would work, I think in real life it would be very brittle and
result in many bugs. The trouble is when maintenance programmers change the
code, will they change the dependency list, and if they do, will they get it
right? This is like the syntax in gcc for telling the compiler which
registers are used by the inline assembly, except it's even harder for the
programmer to get right.

Essentially, any language feature where the programmer has to express
something twice is a bug. C header files are one such <g>.
Feb 14 2005
next sibling parent reply John Reimer <brk_6502 yahoo.com> writes:
On Mon, 14 Feb 2005 19:59:06 -0800, Walter wrote:


 
 "Ben Hinkle" <ben.hinkle gmail.com> wrote in message
 news:curo45$2eea$1 digitaldaemon.com...

 Circular imports are only a problem if the static initializers have


 circular

 dependencies. One way out is to make this more explicit by putting the
 dependencies on the static ctors somehow.


 
 While this would work, I think in real life it would be very brittle and
 result in many bugs. The trouble is when maintenance programmers change the
 code, will they change the dependency list, and if they do, will they get it
 right? This is like the syntax in gcc for telling the compiler which
 registers are used by the inline assembly, except it's even harder for the
 programmer to get right.


Interesting that you note that.  I ran into trouble with an inline
assembler conflict in a C library I was compiling for use with D.  The
linux project was originally designed to compile to a static
lib only.  I wanted a shared lib version, so I changed some parameters in
the makefile and compiled away only to find that adding -PIC (a necessary
compile flag for shared library creation) caused the build to fail at an
inline assembler section in one of the source files.  Apparently the
assembler accessed EAX which is important for use when compiling for
position independent code (PIC).  The compiler died with an error
describing the illegal access for this case.  The author obviously had not
considered this possibility. Thankfully, the inline section wasn't a
critical requirement and could be disabled.


 Essentially, any language feature where the programmer has to express
 something twice is a bug. C header files are one such <g>.


Well, I'd say that D import modules amount to the same thing in instances
of closed source libraries that provided stripped modules for public
use. I don't think the world is completely opensource yet. :-)
Feb 14 2005
parent reply "Walter" <newshound digitalmars.com> writes:
"John Reimer" <brk_6502 yahoo.com> wrote in message
news:pan.2005.02.15.04.27.18.367664 yahoo.com...

 Essentially, any language feature where the programmer has to express
 something twice is a bug. C header files are one such <g>.


 Well, I'd say that D import modules amount to the same thing in instances
 of closed source libraries that provided stripped modules for public
 use. I don't think the world is completely opensource yet. :-)


That isn't a problem with the language, it's more a weakness of the current
implementation. There's no reason why, when compiling a module, the compiler
cannot emit a "symbol" file. Then, when another module imports it, the
"symbol" file is read instead. The "symbol" file will only have in it what
is necessary for importation.

That was the original design for DMD, but to get the compiler done and
working I took the expedient route of just reparsing the import source file
instead.
Feb 14 2005
next sibling parent reply kris <fu bar.org> writes:
Walter wrote:

 "John Reimer" <brk_6502 yahoo.com> wrote in message
 news:pan.2005.02.15.04.27.18.367664 yahoo.com...
 

Essentially, any language feature where the programmer has to express
something twice is a bug. C header files are one such <g>.


Well, I'd say that D import modules amount to the same thing in instances
of closed source libraries that provided stripped modules for public
use. I don't think the world is completely opensource yet. :-)


 
 
 That isn't a problem with the language, it's more a weakness of the current
 implementation. There's no reason why, when compiling a module, the compiler
 cannot emit a "symbol" file. Then, when another module imports it, the
 "symbol" file is read instead. The "symbol" file will only have in it what
 is necessary for importation.


Excellent! Pre-compiled "headers" :-)

If the compiler becomes any faster, it might just cause a rupture in the 
fabric of space/time ...
Feb 14 2005
parent "Walter" <newshound digitalmars.com> writes:
"kris" <fu bar.org> wrote in message news:cus9ou$2u39$2 digitaldaemon.com...

 Excellent! Pre-compiled "headers" :-)


Precompiled headers are a kludge to try and get module performance into
C/C++. The C++ "export" is a sorry attempt to run include and module
semantics through a mixmaster.


 If the compiler becomes any faster, it might just cause a rupture in the
 fabric of space/time ...


It being faster would be a nice side effect, but the primary purpose would
be to avoid needing to sometimes create two versions of the module source.
Feb 15 2005
prev sibling next sibling parent reply "Nick Sabalausky" <z a.a> writes:
"Walter" <newshound digitalmars.com> wrote in message 
news:cus7l4$2sa5$1 digitaldaemon.com...

 "John Reimer" <brk_6502 yahoo.com> wrote in message
 news:pan.2005.02.15.04.27.18.367664 yahoo.com...

 Essentially, any language feature where the programmer has to express
 something twice is a bug. C header files are one such <g>.


 Well, I'd say that D import modules amount to the same thing in instances
 of closed source libraries that provided stripped modules for public
 use. I don't think the world is completely opensource yet. :-)


 That isn't a problem with the language, it's more a weakness of the 
 current
 implementation. There's no reason why, when compiling a module, the 
 compiler
 cannot emit a "symbol" file. Then, when another module imports it, the
 "symbol" file is read instead. The "symbol" file will only have in it what
 is necessary for importation.

 That was the original design for DMD, but to get the compiler done and
 working I took the expedient route of just reparsing the import source 
 file
 instead.


Does this mean that symbol files will be added in the future?
Feb 15 2005
parent reply "Walter" <newshound digitalmars.com> writes:
"Nick Sabalausky" <z a.a> wrote in message
news:cusu9i$guj$1 digitaldaemon.com...

 Does this mean that symbol files will be added in the future?


Yes, since it's clear that they add significant value.
Feb 15 2005
parent reply Dave <Dave_member pathlink.com> writes:
In article <cuu836$1tsi$2 digitaldaemon.com>, Walter says...

"Nick Sabalausky" <z a.a> wrote in message
news:cusu9i$guj$1 digitaldaemon.com...

 Does this mean that symbol files will be added in the future?


Yes, since it's clear that they add significant value.


Does what you have planned for the '"symbol" file import' also allow for
function inlining via import (like source file import does now)?

Thanks,

- Dave
Feb 15 2005
parent reply "Walter" <newshound digitalmars.com> writes:
"Dave" <Dave_member pathlink.com> wrote in message
news:cuua18$1vhe$1 digitaldaemon.com...

 In article <cuu836$1tsi$2 digitaldaemon.com>, Walter says...

"Nick Sabalausky" <z a.a> wrote in message
news:cusu9i$guj$1 digitaldaemon.com...

 Does this mean that symbol files will be added in the future?


Yes, since it's clear that they add significant value.


 Does what you have planned for the '"symbol" file import' also allow for
 function inlining via import (like source file import does now)?


Yes.
Feb 16 2005
parent Dave <Dave_member pathlink.com> writes:
In article <cv0d40$13dn$2 digitaldaemon.com>, Walter says...

"Dave" <Dave_member pathlink.com> wrote in message
news:cuua18$1vhe$1 digitaldaemon.com...

 In article <cuu836$1tsi$2 digitaldaemon.com>, Walter says...

"Nick Sabalausky" <z a.a> wrote in message
news:cusu9i$guj$1 digitaldaemon.com...

 Does this mean that symbol files will be added in the future?


Yes, since it's clear that they add significant value.


 Does what you have planned for the '"symbol" file import' also allow for
 function inlining via import (like source file import does now)?


Yes.


Great and Thanks!

- Dave
Feb 17 2005
prev sibling parent John Reimer <brk_6502 yahoo.com> writes:
Walter wrote:

 "John Reimer" <brk_6502 yahoo.com> wrote in message
 news:pan.2005.02.15.04.27.18.367664 yahoo.com...
 

Essentially, any language feature where the programmer has to express
something twice is a bug. C header files are one such <g>.


Well, I'd say that D import modules amount to the same thing in instances
of closed source libraries that provided stripped modules for public
use. I don't think the world is completely opensource yet. :-)


 
 
 That isn't a problem with the language, it's more a weakness of the current
 implementation. There's no reason why, when compiling a module, the compiler
 cannot emit a "symbol" file. Then, when another module imports it, the
 "symbol" file is read instead. The "symbol" file will only have in it what
 is necessary for importation.
 
 That was the original design for DMD, but to get the compiler done and
 working I took the expedient route of just reparsing the import source file
 instead.
 
 


Yes, I see your point.  I very much look forward to a time when that 
"symbol" file is generated by the compiler.
Feb 15 2005
prev sibling next sibling parent "Ben Hinkle" <ben.hinkle gmail.com> writes:
"Walter" <newshound digitalmars.com> wrote in message 
news:cursen$2i8n$1 digitaldaemon.com...

 "Ben Hinkle" <ben.hinkle gmail.com> wrote in message
 news:curo45$2eea$1 digitaldaemon.com...

 Circular imports are only a problem if the static initializers have


 circular

 dependencies. One way out is to make this more explicit by putting the
 dependencies on the static ctors somehow.


 While this would work, I think in real life it would be very brittle and
 result in many bugs. The trouble is when maintenance programmers change 
 the
 code, will they change the dependency list, and if they do, will they get 
 it
 right? This is like the syntax in gcc for telling the compiler which
 registers are used by the inline assembly, except it's even harder for the
 programmer to get right.



And the current system isn't more brittle? There have already been two 
people who tripped up over it. If someone adds static ctors to two modules 
they would have to remember to check if they have to add empty static ctors 
to all the modules "between" them. Remember the static ctor dependencies 
will be optional and should only be needed when the module in question is 
involved in a circular dependency


 Essentially, any language feature where the programmer has to express
 something twice is a bug. C header files are one such <g>.


One wouldn't have to express it twice. Only when more information is needed 
to diagnose the problem. The counter argument to the current design is that 
it forces programmers to add "dead code" to their codebase - if they are 
lucky enough to remember to add that dead code. Wouldn't it be better to 
give more information right at the problem site (the static ctors that have 
a circular dependency) than spread dead-code around the entire codebase?
Feb 15 2005
prev sibling parent Markus Dangl <danglm in.tum.de> writes:
Walter schrieb:

 "Ben Hinkle" <ben.hinkle gmail.com> wrote in message
 news:curo45$2eea$1 digitaldaemon.com...
 

Circular imports are only a problem if the static initializers have


 
 circular
 

dependencies. One way out is to make this more explicit by putting the
dependencies on the static ctors somehow.


 
 
 While this would work, I think in real life it would be very brittle and
 result in many bugs. The trouble is when maintenance programmers change the
 code, will they change the dependency list, and if they do, will they get it
 right? This is like the syntax in gcc for telling the compiler which
 registers are used by the inline assembly, except it's even harder for the
 programmer to get right.
 
 Essentially, any language feature where the programmer has to express
 something twice is a bug. C header files are one such <g>.
 
 


Perhaps you don't have to specify it twice but only if you want to 
resolve a circular import issue. For example:

=====
module a;
import b;

static this()
{
}

module b;
import a;

static this()
// State that a.this() is required
{
}

So if you don't specify it, the order of the execution will be "default".
Feb 15 2005