www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - [announce] D Object Oriented Library - dool

reply Ant <Ant_member pathlink.com> writes:
yet another project, yet another runtime lib.

I started this project to be able to continue development of my
lib DUI and my IDE leds. Since dmd 0.91 I have spend more time
with workarounds then development and I found a new OO lib to
be the long term solution.
dool is to complement phobos where phobos doesn't support he OO paradigm.
read all about it at:

http://sourceforge.net/forum/forum.php?thread_id=1121037&forum_id=394961

contributions to dool all welcome (OO only).

the next versions of DUI and leds will use the new library.
On leds it will be completly tranparent to the user,
on DUI the impact to the developer is minimal.

I also setup the CVS on sourceforge for my 3 active D projects.

There is nothing to see on dool yet, it contains just a few adaptations
from phobos (String, Path, File and not much else) and it was done
in a rush just to be able to compile DUI and leds.
however check the manifest on the sourceforge forum (the link above).

(sourceforge is a bit strange, it reports nothing on CVS but all the projects
are there.)
As I could do nothing more then workarounds, convertion to dool and but fixes
on the past few month the DUI and leds versions
on the CVS are **NOT** better then the current releases.
If you have nothing better to do and are very curiouse about these
projects by all means grab the CVS (but would will be on your own)
you can also browse the CVS from the sourceforge pages
(for instance cvs.sourceforge.net/viewcvs.py/dool/ ).

http://sourceforge.net/projects/dool/ (no home page yet)
http://dui.sourceforge.net/
http://leds.sourceforge.net/projects/

(I hope to submit my projects to dsource soon.)

Ant
Aug 03 2004
next sibling parent Lord Syl <Lord_member pathlink.com> writes:
In article <ceo2od$170c$1 digitaldaemon.com>, Ant says...

yet another project, yet another runtime lib.

I started this project to be able to continue development of my
lib DUI and my IDE leds. Since dmd 0.91 I have spend more time
with workarounds then development and I found a new OO lib to
be the long term solution.
dool is to complement phobos where phobos doesn't support he OO paradigm.
read all about it at:
...


That's good to hear, it's specially adequate for programmers that like Java's
way. :)
Aug 03 2004
prev sibling next sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <ceo2od$170c$1 digitaldaemon.com>, Ant says...


dool is to complement phobos where phobos doesn't support he OO paradigm.
read all about it at:
http://sourceforge.net/forum/forum.php?thread_id=1121037&forum_id=394961


Excellent! You have my wholehearted support.

Just a couple of minor points. (1), I read: "std.conv -> Integer (rename to
Int?)". I realize these are in different namespaces, but there already exists a
class called Int (my unlimited precision integer class). Might I suggest that
the name Integer be retained to avoid conflict?

(2), I read: "std.ctype -> Charater (rename to Char?)". (I assume that was a
typo for "Character"). The problem here is that std.ctype does /not/ deal with
Unicode characters, only with ASCII characters. It is useful for recoding old
legacy applications, but new ones should be encouraged to use Unicode, and,
hence, std.ctype gradually deprecated as replacements become available. Right
now, Hauke's utype module replaces the ctype functions with Unicode functions
having the same name, and, even more comprehensively, there are (non-OO)
functions in etc.unicode which provide vastly more character properties - almost
all of the properties defined by the Unicode Consortium right now, in fact, and
absolutely all of them in the future. A new version of etc.unicode will, I hope,
be available next weekend, complete with linkable lib. I guess what I'm saying
is, I'd rather see an OO wrapper around etc.unicode than around std.ctype. If c
is an instance of a Character/Char, you should be able to do: if
(c.isBidiControl()) {...}, or if (c.isNonCharacterCodePoint()) {...}, or access
any of the other Unicode properties, defined for /every/ Unicode character.

So I would like to suggest that a class which wraps only std.ctype should, in
fact, be called neither Character nor Char. It should be called: ASCIICharacter
or ASCIIChar. I suggest that Character/Char be reserved to wrap the etc.unicode
character functions. As of next weekend, the code which builds etc.unicode will
be uploaded to dsource/Demios - which is open source - so you can import it
directly instead of having to re-implement it, and if you find any bugs you can
get at the source to fix them instead of having to wait for fixed to Phobos.

I think your idea is brilliant. This is just a plea for consistency and
co-operation in naming and forward planning.

Arcane Jill
Aug 03 2004
next sibling parent reply Ant <Ant_member pathlink.com> writes:
In article <ceo686$18ks$1 digitaldaemon.com>, Arcane Jill says...

In article <ceo2od$170c$1 digitaldaemon.com>, Ant says...


dool is to complement phobos where phobos doesn't support he OO paradigm.
read all about it at:
http://sourceforge.net/forum/forum.php?thread_id=1121037&forum_id=394961


Excellent! You have my wholehearted support.

Just a couple of minor points. (1), I read: "std.conv -> Integer (rename to
Int?)". I realize these are in different namespaces, but there already exists a
class called Int (my unlimited precision integer class). Might I suggest that
the name Integer be retained to avoid conflict?


just because Integer and Character sound too javaish, suggestion accepted
(until some thing better comes up).


The problem here is that std.ctype does /not/ deal with
Unicode characters, [...]
 Hauke's utype module replaces the ctype functions with Unicode functions
having the same name, and, even more comprehensively, there are (non-OO)


so, if it's open source, maybe Hauke's utype could be converted to OO
and replace dool's current Character implementation. That's exactly
one of the objectives of dool: easy to change and adopt the best
solution available.


So I would like to suggest that a class which wraps only std.ctype should, in
fact, be called neither Character nor Char. It should be called: ASCIICharacter
or ASCIIChar. I suggest that Character/Char be reserved to wrap the etc.unicode
character functions.


ok, sounds good.


As of next weekend, the code which builds etc.unicode will
be uploaded to dsource/Demios


I'll probably take a look at it.
(be awere that my understanding of Unicode problems is very limited)


I think your idea is brilliant. This is just a plea for consistency and
co-operation in naming and forward planning.


thanks

Ant
Aug 03 2004
parent Arcane Jill <Arcane_member pathlink.com> writes:
In article <ceo7qf$192s$1 digitaldaemon.com>, Ant says...


 Hauke's utype module replaces the ctype functions with Unicode functions
having the same name, and, even more comprehensively, there are (non-OO)


so, if it's open source, maybe Hauke's utype could be converted to OO
and replace dool's current Character implementation. That's exactly
one of the objectives of dool: easy to change and adopt the best
solution available.


Hauke?

I'm not sure I should be speaking on someone else's behalf, but I'm fairly sure
uchar is open source in the sense that the source is public. Hauke posted it on
this forum once. I'm sure he could be persuaded to post it again. I have no idea
what license he uses. I don't know if it's hosted on dsource or anywhere else.

That said, the original (and possibly current) version of uchar worked by
wrapping functions from unichar (another of Hauke's modules), rather than
functions from etc.unicode. Though undeniably excellent, unichar has now been
succeeded by etc.unicode in the sense that the functionality of unichar is a
subset of the functionality of etc.unicode, and the latter library makes the
presumption of aspiring to become a complete implementation of all Unicode
algorithms ... eventually (wheras unichar only handles the most common character
properties). So you have a choice of which implementation to wrap. unichar is
going to smaller; etc.unicode is going to be more comprehensive - take your
pick. Obviously, where they overlap, they should give identical results
(assuming that unichar was built against Unicode 4.0.1).

The future is more interesting. I believe I am correct in saying that Hauke and
I are in agreement that there should be only one Unicode implementation, not two
rivals. I've been out of action for a while (but I'm back now), and once the
etc.unicode codebuilder goes open source, Hauke will have write access to it,
and I know he's pretty good at improving efficiency, as well as understanding
the issues. So, I /think/ that etc.unicode is the way to go, most especially if
uchar could be rewritten to wrap etc.unicode functions. But, like I said, I'm
not sure I should be speaking on someone else's behalf. I'm hoping Hauke will
join in here and speak for himself.




So I would like to suggest that a class which wraps only std.ctype should, in
fact, be called neither Character nor Char. It should be called: ASCIICharacter
or ASCIIChar. I suggest that Character/Char be reserved to wrap the etc.unicode
character functions.


ok, sounds good.


Regardless of the future of Character/Char, the main thing is to name the
std.ctype wrapper ASCIICharacter/ASCIIChar, so as (a) not to mislead people, and
(b) not to tie up the more general name for later.



As of next weekend, the code which builds etc.unicode will
be uploaded to dsource/Demios


I'll probably take a look at it.
(be awere that my understanding of Unicode problems is very limited)


In general, there are a bunch of functions in etc.unicode with names like
getXxxxx(dchar) or isXxxxx(dchar), for instance getHexValue(dchar) or
isLetter(dchar). These functions are pretty easy to understand, and you can use
them just like the old ctype functions. (The functions do have different names,
I'm afraid, but Unicode property names are defined by the Unicode standard, not
by me). The more complex algorithms are yet to come, so actually there's not
much to understand at this point.

Jill
Aug 03 2004
prev sibling parent reply parabolis <parabolis softhome.net> writes:
Arcane Jill wrote:


 (2), I read: "std.ctype -> Charater (rename to Char?)". (I assume that was a
 typo for "Character"). The problem here is that std.ctype does /not/ deal with
 Unicode characters, only with ASCII characters. It is useful for recoding old
 legacy applications, but new ones should be encouraged to use Unicode, and,
 hence, std.ctype gradually deprecated as replacements become available. Right


That statement makes me nervous. I assume however that 
std.c.ctypes will continue to work with legacy apps?


 now, Hauke's utype module replaces the ctype functions with Unicode functions
 having the same name, and, even more comprehensively, there are (non-OO)


What does Unicode need with negative infinity? :)
Aug 03 2004
parent Arcane Jill <Arcane_member pathlink.com> writes:
In article <ceoemp$1bu0$1 digitaldaemon.com>, parabolis says...


 hence, std.ctype gradually deprecated as replacements become available.





That statement makes me nervous. I assume however that 
std.c.ctypes will continue to work with legacy apps?


Relax. I should have chosen my words more carefully. I probably should have said
"hence std.ctype being gradually used less and less frequently, as better
alternatives become available".

In any case, I have no influence over std. I presume Walter will do the sensible
thing and continue to support the ctype functions indefinitely (documenting that
they are defined only for ASCII characters, of course).

Jill
Aug 03 2004
prev sibling next sibling parent reply Sean Kelly <sean f4.ca> writes:
In article <ceo2od$170c$1 digitaldaemon.com>, Ant says...

yet another project, yet another runtime lib.

I started this project to be able to continue development of my
lib DUI and my IDE leds. Since dmd 0.91 I have spend more time
with workarounds then development and I found a new OO lib to
be the long term solution.
dool is to complement phobos where phobos doesn't support he OO paradigm.


Nice.  Now we can say "there is no Phobos only dool!"

(lame ghostbusters reference)


Sean
Aug 03 2004
parent reply Charlie <Charlie_member pathlink.com> writes:
Nice.  Now we can say "there is no Phobos only dool!"
                                   Dana        Zool!
(lame ghostbusters reference)


LOL got it! 

Charlie

In article <ceo9bc$19oo$1 digitaldaemon.com>, Sean Kelly says...

In article <ceo2od$170c$1 digitaldaemon.com>, Ant says...

yet another project, yet another runtime lib.

I started this project to be able to continue development of my
lib DUI and my IDE leds. Since dmd 0.91 I have spend more time
with workarounds then development and I found a new OO lib to
be the long term solution.
dool is to complement phobos where phobos doesn't support he OO paradigm.


Nice.  Now we can say "there is no Phobos only dool!"

(lame ghostbusters reference)


Sean
Aug 03 2004
parent Ant <Ant_member pathlink.com> writes:
In article <ceobf6$1ag6$1 digitaldaemon.com>, Charlie says...

Nice.  Now we can say "there is no Phobos only dool!"
                                   Dana        Zool!
(lame ghostbusters reference)


LOL got it! 


I did consider drool as the name.
for D Runtime OO Lib.

and BTW when I named my DUI I didn't know it means Driving Under the Influense
and I also didn't remember that ant is that insect
(I knew about ant, just did connect.
'Ant' are the first 3 letters of my name.)

(Once a brother of a friend got excelent reviews on his coloring 
skills at kindergarden - later they found out he his colour blind... :)

Ant
Aug 03 2004
prev sibling next sibling parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Just taken a brief look.

"The second problem we find with phobos is that it isn’t a full OO library."

By "a full OO library" do you mean one that uses OO for everything, even 
  if it's overkill?  Leading to such things as

	new DoublePrecisionFloatingPoint(1.5).sin

instead of the more natural notation?

By "problem" do you mean "problem for OO purists"?  As the overview 
says, D isn't meant for purists.

But indeed, some bits of Phobos (like std.date) could do with being OO.

FTM, would structs and unions be allowed as well as classes?  Or do you 
consider those just another "legacy C++ construct"?

"- dool will not contain legacy C++ constructs (pointers are the obvious 
example)"

Do you have an alternative to a pointer into an array that achieves the 
same level of efficiency?

FTM, I'm not sure how you're going to interface many OS APIs without 
using such things at all....

"- new functionalities are always accepted (the exceptions are obvious 
and include malware, legal, and broken submissions)"

Why do you only want illegal stuff?

- std.conv -> Integer (rename to Int?)
- std.ctype -> Charater (rename to Char?)

Depends whether you're going to stick to one naming convention 
throughout all object wrappers for primitive types.

Two possibilities: make them all the same as the same as the D type 
names, or make them all unabbreviated names....

Stewart.

-- 
My e-mail is valid but not my primary mailbox.  Please keep replies on 
the 'group where everyone may benefit.
Aug 03 2004
parent reply Ant <Ant_member pathlink.com> writes:
In article <ceob3h$1acn$1 digitaldaemon.com>, Stewart Gordon says...

Just taken a brief look.

"The second problem we find with phobos is that it isn’t a full OO library."

By "a full OO library" do you mean one that uses OO for everything, even 
  if it's overkill?  Leading to such things as

	new DoublePrecisionFloatingPoint(1.5).sin


no. math is an obvious exception (as long as it doesn't confuses dmd).


By "problem" do you mean "problem for OO purists"?


Obviously.


  As the overview says, D isn't meant for purists.


but dool is.


But indeed, some bits of Phobos (like std.date) could do with being OO.

FTM, would structs and unions be allowed as well as classes?  Or do you 
consider those just another "legacy C++ construct"?


You saying the final classes should be Structs? Why not? probably.
I do use structs. (java does not define OO BTW)
structs can't implement interfaces, that might be the big problem.
(sorry if "legacy C++ construct" doesn't fit well. the first help
dool could have is a revision of the manifest...:)


"- dool will not contain legacy C++ constructs (pointers are the obvious 
example)"

Do you have an alternative to a pointer into an array that achieves the 
same level of efficiency?


no, but do I care? if your goal is speed you might want
to use another lib (complementing dool or not).
String it self might present an effitiency drop over char[].
beside, I might be wrong and adopt different ideas :)
probably the internal implementation should be irrelevant.
(I was wrong once: I thought I had made a mistake but I didn't.
My boss told me that joke :)


FTM, I'm not sure how you're going to interface many OS APIs without 
using such things at all....


I mean the user API.
A programmer has to do what a programmer has to do.
(An old reference for the movie fans)


"- new functionalities are always accepted (the exceptions are obvious 
and include malware, legal, and broken submissions)"

Why do you only want illegal stuff?


Sorry, probably it reads the opposit way I wanted it :{ (?)
(I hope I can change it - I wrote that and revised it on separate days
and still has many, many mistakes... :( )


- std.conv -> Integer (rename to Int?)
- std.ctype -> Charater (rename to Char?)

Depends whether you're going to stick to one naming convention 
throughout all object wrappers for primitive types.

Two possibilities: make them all the same as the same as the D type 
names, or make them all unabbreviated names....


sure, makes sence.

Ant
Aug 03 2004
parent reply Ant <Ant_member pathlink.com> writes:
In article <ceodr8$1bki$1 digitaldaemon.com>, Ant says...

In article <ceob3h$1acn$1 digitaldaemon.com>, Stewart Gordon says...

Just taken a brief look.

"The second problem we find with phobos is that it isn’t a full OO library."

By "a full OO library" do you mean one that uses OO for everything, even 
  if it's overkill?  Leading to such things as

	new DoublePrecisionFloatingPoint(1.5).sin


no. math is an obvious exception (as long as it doesn't confuses dmd).


I should add primitive types also.

Ant
Aug 03 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <ceoet0$1c4q$1 digitaldaemon.com>, Ant says...


no. math is an obvious exception (as long as it doesn't confuses dmd).


I should add primitive types also.


Ah - that makes more sense. In that case, you don't need to wrap char, wchar or
dchar, since these are primitive types.

I was thinking about what I posted earlier and some thoughts occur to me. Now
that I've thought about it, I think it would be a bad idea to put an OO wrapper
around dchar at all. Here's the reason.

Currently, if someone calls isLetter(dchar), and links with etc.unicode, the
linker will add a small number of bytes to their executable - enough for a few
small lookup tables and a function to access them. You're talking a couple of K
at most.

On the other hand, a Char implementation would have to call /every/ Unicode
function, and hence the linker would drag in /every/ function, including the
exotic ones that hardly anyone's ever going to use, and the bloated getName()
funtion, which returns, as an ASCII string, the name of every Unicode character.
If you were to link to /the whole lot/, as a Char wrapper must do, you are going
to pull in the /entire/ of etc.unicode. That's two megabytes. Although that may
be justified in a DLL, no-one is going to want to add that to their executable.

You could make it smaller by choosing which functions to wrap and which to
ignore, or to use unichar (with much the same effect), but, the more I think
about it, the more I don't see the point. dchar is a primitive type. It doesn't
need to be wrapped.

Arcane Jill
Aug 03 2004
next sibling parent Ant <Ant_member pathlink.com> writes:
In article <ceoihh$1drt$1 digitaldaemon.com>, Arcane Jill says...

In article <ceoet0$1c4q$1 digitaldaemon.com>, Ant says...


no. math is an obvious exception (as long as it doesn't confuses dmd).


I should add primitive types also.


Ah - that makes more sense. In that case, you don't need to wrap char, wchar or
dchar, since these are primitive types.



maybe not. It's java influence as most containers in java only support
objects. and also fear of the dreaded "std." conflict on dmd
(that might be converted into the "dool." conflict).

I see the math functions and the primitive types functions 
almost as part of the language, not of the lib (yes, I'm that old),
so it makes sence to me that they be "free functions".



I was thinking about what I posted earlier and some thoughts occur to me. Now
that I've thought about it, I think it would be a bad idea to put an OO wrapper
around dchar at all. Here's the reason.


[...]

valid.

Ant
Aug 03 2004
prev sibling next sibling parent reply parabolis <parabolis softhome.net> writes:
Arcane Jill wrote:


 In article <ceoet0$1c4q$1 digitaldaemon.com>, Ant says...
 
 

no. math is an obvious exception (as long as it doesn't confuses dmd).


I should add primitive types also.


 
 
 Ah - that makes more sense. In that case, you don't need to wrap char, wchar or
 dchar, since these are primitive types.
 
 I was thinking about what I posted earlier and some thoughts occur to me. Now
 that I've thought about it, I think it would be a bad idea to put an OO wrapper
 around dchar at all. Here's the reason.
 
 Currently, if someone calls isLetter(dchar), and links with etc.unicode, the
 linker will add a small number of bytes to their executable - enough for a few
 small lookup tables and a function to access them. You're talking a couple of K
 at most.
 
 On the other hand, a Char implementation would have to call /every/ Unicode
 function, and hence the linker would drag in /every/ function, including the
 exotic ones that hardly anyone's ever going to use, and the bloated getName()
 funtion, which returns, as an ASCII string, the name of every Unicode
character.
 If you were to link to /the whole lot/, as a Char wrapper must do, you are
going
 to pull in the /entire/ of etc.unicode. That's two megabytes. Although that may
 be justified in a DLL, no-one is going to want to add that to their executable.


I have doubts this is true. I have been debating how I feel 
about dool and one thing I tried was checking to see if a static 
function calling a class function would pull the data in. So I 
compiled with -O -inline -release and dumped using obj2asm and 
did not see any unused function names.

If you could give me an example I can try on my own I will be 
easily convinced. However I imagine Ch.isLetter() would probably 
have an almost identical method body as the etc.unicode.isLetter()?
Aug 03 2004
parent Ant <Ant_member pathlink.com> writes:
In article <ceol8g$1f6q$1 digitaldaemon.com>, parabolis says...

Arcane Jill wrote:


 If you were to link to /the whole lot/, as a Char wrapper must do, you are
going
 to pull in the /entire/ of etc.unicode. That's two megabytes. Although that may
 be justified in a DLL, no-one is going to want to add that to their executable.


I have doubts this is true. I have been debating how I feel 
about dool and one thing I tried was checking to see if a static 
function calling a class function would pull the data in. So I 
compiled with -O -inline -release and dumped using obj2asm and 
did not see any unused function names.

If you could give me an example I can try on my own I will be 
easily convinced. However I imagine Ch.isLetter() would probably 
have an almost identical method body as the etc.unicode.isLetter()?


this is great. back to the original idea.

Ant
Aug 03 2004
prev sibling parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Arcane Jill wrote:
<snip>

 On the other hand, a Char implementation would have to call /every/ 
 Unicode function, and hence the linker would drag in /every/ 
 function, including the exotic ones that hardly anyone's ever going 
 to use,


<snip>

 That's two megabytes. Although that may be justified in a DLL, no-one 
 is going to want to add that to their executable.


<snip>

Dead code elimination is one of the most fundamental optimisations ever 
invented.

Stewart.

-- 
My e-mail is valid but not my primary mailbox.  Please keep replies on 
the 'group where everyone may benefit.
Aug 04 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <ceq9g5$25mh$1 digitaldaemon.com>, Stewart Gordon says...

Arcane Jill wrote:
<snip>

 On the other hand, a Char implementation would have to call /every/ 
 Unicode function, and hence the linker would drag in /every/ 
 function, including the exotic ones that hardly anyone's ever going 
 to use,


<snip>

 That's two megabytes. Although that may be justified in a DLL, no-one 
 is going to want to add that to their executable.


<snip>

Dead code elimination is one of the most fundamental optimisations ever 
invented.


From application source code, maybe, but not from a library. When you build a
library which contains an indivisible module which calls hundreds of external
functions, the linker has no idea whether an application which (later) links
against that library is going to call a given function, so it has to call all of
them.

In etc.unicode, the various functions are each in their own module, so if an
application is built directly against etc.unicode then the linker can link in
only those modules which are necessary to complete the link. 

However, suppose that a single, indivisible module within dool.lib were to call
every function in etc.unicode, and suppose that an application linked against
dool.lib and used a Char object. The linker would drag in the dool.lib module
which defines Char. This would create hundreds of unresolved references, which
would in turn cause the whole of etc.unicode to be dragged in to resolve them.
This is because, by the time it gets to the linker stage, it's too late.

I'd be more than happy if I turned out to be wrong on this one, but I don't
think I am. Walter can hopefully confirm or deny this.

Arcane Jill
Aug 04 2004
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Arcane Jill wrote:

<snip>

 From application source code, maybe, but not from a library. When you 
 build a library which contains an indivisible module which calls 
 hundreds of external functions, the linker has no idea whether an 
 application which (later) links against that library is going to call 
 a given function, so it has to call all of them.


<snip>

DCE applies to both compiling and linking.  I don't know if this is 
platform dependent, but AIUI a lib is basically a collection of object 
files.  Each object file has a symbol table that lists the functions it 
defines and the external functions on which it depends.  I'd guess that 
this can be determined for each function too, but I'm not sure.

Linking is then a matter of traversing the dependency graph through the 
object files of both the project itself and the libs it uses.

Stewart.

-- 
My e-mail is valid but not my primary mailbox.  Please keep replies on 
the 'group where everyone may benefit.
Aug 04 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <ceqfqu$284l$1 digitaldaemon.com>, Stewart Gordon says...

Arcane Jill wrote:



DCE [Dead Code Elimination] applies to both compiling and linking.
I don't know if this is 
platform dependent, but AIUI a lib is basically a collection of object 
files.


..with, in the D paradigm, 1 object file = 1 module...


Each object file has a symbol table that lists the functions it 
defines and the external functions on which it depends.


Actually, I think it lists the assembler /symbols/ it defines, and the symbols
upon which it depends. Each symbol is an assembler label or an assembler EQU
directive. Each symbol will correspond to either a function, a global variable,
or a const variable.



I'd guess that 
this can be determined for each function too, but I'm not sure.


Nor I



Linking is then a matter of traversing the dependency graph through the 
object files of both the project itself and the libs it uses.


True, but this process only determines which object files are needed and which
are not. It does not (so far as I know) permit the linker to slice up an object
file into smaller parts and discard some of those fragments. Either the whole
obj file (read D module) is included, or none of it is. There is no in between.

At least, that's what I've always believed. (I wrote my own linker back in the
'80s, but that was for the Commodore Amiga, and I guess linker formats have
changed since then). Anyway, Hauke did the experiment, and concluded (to his
surprise) that D (or its linker) does in fact work the way I described.

If you prove me wrong, I will be very, very happy.

Arcane Jill
Aug 04 2004
next sibling parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Arcane Jill wrote:

<snip>

 Linking is then a matter of traversing the dependency graph through the 
 object files of both the project itself and the libs it uses.


 
 True, but this process only determines which object files are needed and which 
 are not. It does not (so far as I know) permit the linker to slice up an
object 
 file into smaller parts and discard some of those fragments. Either the whole 
 obj file (read D module) is included, or none of it is. There is no in between.


Unless the fragments have their individual symbol tables.  Even if the 
obj file format doesn't support this, then maybe someone could invent a 
new obj file format that does....

Obviously, any lib component that is overly bloated and not going to be 
used by everyone and everything would have a module to itself.  Of 
course, letting modules be as independent of each other as possible does 
help, at least for the time being....


 At least, that's what I've always believed. (I wrote my own linker back in the 
 '80s, but that was for the Commodore Amiga, and I guess linker formats have 
 changed since then). Anyway, Hauke did the experiment, and concluded (to his 
 surprise) that D (or its linker) does in fact work the way I described.


Yes, the DMD linker isn't state of the art just yet.  For example, 
whatever I add to SDWF, it seems that my skeleton program gets bigger. 
When a better optimising linker finally comes along, it'll get smaller 
again.

Stewart.

-- 
My e-mail is valid but not my primary mailbox.  Please keep replies on 
the 'group where everyone may benefit.
Aug 04 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cer8l1$aqq$1 digitaldaemon.com>, Stewart Gordon says...


Obviously, any lib component that is overly bloated and not going to be 
used by everyone and everything would have a module to itself.


Right - which brings me back to my original point, because you see, in D (unlike
C++), every member function of a class /must/ be defined within a single module.


I achieve the one-function-per-module separation in etc.unicode precisely
because the functions were good old fashioned, plain functions (not class member
functions), each taking a dchar as its only parameter. If you were to make them
all member functions of a single class, the resulting (indivisible, so far as we
know) obj file would be two megabytes large, and if an application called a
single function of that class, the whole two megs would get added to the
application. This is not good.

But I don't think that having them as ordinary functions is necessarily a
problem. To me, isLetter(c) is just as readable as c.isLetter(). And, although I
/am/ a great supporter of the OO paradigm in general, I still don't see the need
to wrap the primitive types in structs/classes. If dool lets us write sin(x)
instead of x.sin(), then I don't see why it would have a problem letting us
write isLetter(c) instead of c.isLetter(). There simply is no need to turn
primitive types into objects - and hence, (if you accept this,) no problem to
solve.



Yes, the DMD linker isn't state of the art just yet.  For example, 
whatever I add to SDWF, it seems that my skeleton program gets bigger. 
When a better optimising linker finally comes along, it'll get smaller 
again.


That's acceptable for most projects, but not for a two megabyte library. Telling
people "it will get smaller once someone writes a better linker" is not an
argument I could credibly hold. So it's organized the way it is, to keep
executables as small as possible, with the linker we have right now.

Arcane Jill
Aug 04 2004
parent reply Lars Ivar Igesund <larsivar igesund.net> writes:
It seems that someone believes that the D linker is an immature product. 
The linker used by DMD is not D specific, but is also used by DMC and it 
is called 'optlink' (http://www.digitalmars.com/ctg/optlink.html). It is 
a highly optimized linker (hence the name), at least when it comes to 
execution speed of the linker itself. Sizeoptimization as requested in 
this thread might be done through /EXEPACK, /PACKFUNCTIONS and /WINPACK 
(see http://www.digitalmars.com/ctg/ctgLinkSwitches.html). These 
switches can be passed to the linker through the DMD -L switch. Read 
more on how optlink works at http://www.digitalmars.com/ctg/ctgLinkOps.html

Whether DMD do the COMDAT stuff necessary for 'smart linking' or not, I 
don't know.

Also note that DMD use gcc to link on Linux.

Lars Ivar Igesund
Aug 04 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cerfpb$f73$1 digitaldaemon.com>, Lars Ivar Igesund says...

It seems that someone believes that the D linker is an immature product. 
The linker used by DMD is not D specific, but is also used by DMC and it 
is called 'optlink' (http://www.digitalmars.com/ctg/optlink.html). It is 
a highly optimized linker (hence the name), at least when it comes to 
execution speed of the linker itself. Sizeoptimization as requested in 
this thread might be done through /EXEPACK, /PACKFUNCTIONS and /WINPACK 
(see http://www.digitalmars.com/ctg/ctgLinkSwitches.html). These 
switches can be passed to the linker through the DMD -L switch. Read 
more on how optlink works at http://www.digitalmars.com/ctg/ctgLinkOps.html


Wow! Okay - I'm impressed.

Just as an aside, I would never have discovered those pages by accident, had you
not given us the URLs. It might be useful to add links to linker-related pages
to the Table-of-Contents frame of the D site.




Whether DMD do the COMDAT stuff necessary for 'smart linking' or not, I 
don't know.


I'd certainly be interested to find out.

Thanks for all that brilliant information.
Jill
Aug 04 2004
parent J C Calvarese <jcc7 cox.net> writes:
Arcane Jill wrote:

 In article <cerfpb$f73$1 digitaldaemon.com>, Lars Ivar Igesund says...
 

It seems that someone believes that the D linker is an immature product. 
The linker used by DMD is not D specific, but is also used by DMC and it 
is called 'optlink' (http://www.digitalmars.com/ctg/optlink.html). It is 
a highly optimized linker (hence the name), at least when it comes to 
execution speed of the linker itself. Sizeoptimization as requested in 
this thread might be done through /EXEPACK, /PACKFUNCTIONS and /WINPACK 
(see http://www.digitalmars.com/ctg/ctgLinkSwitches.html). These 
switches can be passed to the linker through the DMD -L switch. Read 
more on how optlink works at http://www.digitalmars.com/ctg/ctgLinkOps.html


 
 
 Wow! Okay - I'm impressed.
 
 Just as an aside, I would never have discovered those pages by accident, had
you
 not given us the URLs. It might be useful to add links to linker-related pages
 to the Table-of-Contents frame of the D site.


That's a good idea. I wish I had known about the linker documentation 
earlier than I did.

On a related note, there's already a wiki page that links to several of 
these tools:

http://www.prowiki.org/wiki4d/wiki.cgi?ReferenceForTools


 
 

Whether DMD do the COMDAT stuff necessary for 'smart linking' or not, I 
don't know.


 
 
 I'd certainly be interested to find out.
 
 Thanks for all that brilliant information.
 Jill


-- 
Justin (a/k/a jcc7)
http://jcc_7.tripod.com/d/
Aug 04 2004
prev sibling next sibling parent reply "Carlos Santander B." <carlos8294 msn.com> writes:
"Arcane Jill" <Arcane_member pathlink.com> escribió en el mensaje
news:cer3bp$6nk$1 digitaldaemon.com
|
| True, but this process only determines which object files are needed and which
| are not. It does not (so far as I know) permit the linker to slice up an
object
| file into smaller parts and discard some of those fragments. Either the whole
| obj file (read D module) is included, or none of it is. There is no in
between.
|

Say I have this:

module a;
void foo() { }

module b;
void bar() { }

module c;
import a, b;

And put the three of them in mylib.lib. Then I do this:

module app;
import c;
void main () { foo(); }

Compile it, and link it with mylib.lib. Will the final exe also have module b?

-----------------------
Carlos Santander Bernal
Aug 04 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <ces64k$rfd$1 digitaldaemon.com>, Carlos Santander B. says...

Say I have this:

module a;
void foo() { }

module b;
void bar() { }

module c;
import a, b;

And put the three of them in mylib.lib. Then I do this:

module app;
import c;
void main () { foo(); }

Compile it, and link it with mylib.lib. Will the final exe also have module b?


No. You'd just get main and a. And that's the priniciple on which I separated my
functions.

Since I posted my post, however, Lars has indicated that the linker is actually
cleverer than I had thought, and might be able to do that sort of thing at the
function level. It doesn't appear to do that automatically, but - if I've
understood the documentation correctly - it can be made to do that with some
command line parameters. More experimentation is needed. When someone gets
function-body-elimination working, I hope they post an example.

Jill
Aug 04 2004
parent reply "Carlos Santander B." <carlos8294 msn.com> writes:
"Arcane Jill" <Arcane_member pathlink.com> escribió en el mensaje
news:ceslnv$164e$1 digitaldaemon.com
| Since I posted my post, however, Lars has indicated that the linker is
actually
| cleverer than I had thought, and might be able to do that sort of thing at the
| function level. It doesn't appear to do that automatically, but - if I've
| understood the documentation correctly - it can be made to do that with some
| command line parameters. More experimentation is needed. When someone gets
| function-body-elimination working, I hope they post an example.
|
| Jill

In a final exe, how can I know if a function body is there or not? If you tell,
I can help with that ;)

-----------------------
Carlos Santander Bernal
Aug 05 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cet74t$1ieu$1 digitaldaemon.com>, Carlos Santander B. says...

In a final exe, how can I know if a function body is there or not? If you tell,
I can help with that ;)


Well, one easy way would be to include the following function - and /don't/ call
it.







If the resulting executable is over a megabyte large then it's a fair bet that
that the unused f and LOOKUP have not been eliminated.

Jill
Aug 05 2004
parent reply "Carlos Santander B." <carlos8294 msn.com> writes:
"Arcane Jill" <Arcane_member pathlink.com> escribió en el mensaje
news:cet8g3$1j5u$1 digitaldaemon.com
| Well, one easy way would be to include the following function - and /don't/
call
| it.
|





|
| If the resulting executable is over a megabyte large then it's a fair bet that
| that the unused f and LOOKUP have not been eliminated.
|
| Jill

(After adding void main () {} )
72200 bytes. Compiler flags don't change anything.

-----------------------
Carlos Santander Bernal
Aug 05 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <ceu358$22e1$1 digitaldaemon.com>, Carlos Santander B. says...

"Arcane Jill" <Arcane_member pathlink.com> escribió en el mensaje
news:cet8g3$1j5u$1 digitaldaemon.com
| Well, one easy way would be to include the following function - and /don't/
call
| it.
|





|
| If the resulting executable is over a megabyte large then it's a fair bet that
| that the unused f and LOOKUP have not been eliminated.
|
| Jill

(After adding void main () {} )
72200 bytes. Compiler flags don't change anything.


Wow. Sounds like you've got it.

You could try changing



to






and if /that/ still works, I'd say you've definitely licked the problem! (This
is almost exactly what my own lookup functions do, you see. If this can be made
to work then we're rolling!

Arcane Jill
Aug 06 2004
parent reply "Carlos Santander B." <carlos8294 msn.com> writes:
"Arcane Jill" <Arcane_member pathlink.com> escribió en el mensaje
news:cevac8$2tsm$1 digitaldaemon.com
| Wow. Sounds like you've got it.
|
| You could try changing
|

|
| to
|




|
| and if /that/ still works, I'd say you've definitely licked the problem! (This
| is almost exactly what my own lookup functions do, you see. If this can be
made
| to work then we're rolling!
|
| Arcane Jill

Sorry, can't do that: DMD ran out of memory on my computer.
Somebody else can try. Compile the file generated by this:

//////////////////////////////
import std.stream;

void main ()
{
    auto MemoryStream ms = new MemoryStream;
    ms.writeLine ( "const byte LOOKUP[1000000] = [" );
    for ( uint i=1; i<50000; ++i )
        ms.writeLine ( "0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9," );
    ms.writeLine ( "0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9];" );
    ms.writeLine ( "byte f(int n) { return LOOKUP[n]; } void main () {}" );
    ms.position = 0;
    auto BufferedFile bf = new BufferedFile ( "test.d", FileMode.Out );
    bf.copyFrom ( ms );
}
//////////////////////////////

The file would be over 2MB.

-----------------------
Carlos Santander Bernal
Aug 06 2004
next sibling parent reply Juanjo =?ISO-8859-15?Q?=C1lvarez?= <juanjuxNO SPAMyahoo.es> writes:
Carlos Santander B. wrote:


 
 The file would be over 2MB.


DMD on Linux, compiled (1 GB RAM) and the size is exactly 2.0Mb.
Aug 06 2004
parent "Carlos Santander B." <carlos8294 msn.com> writes:
"Juanjo Álvarez" <juanjuxNO SPAMyahoo.es> escribió en el mensaje
news:cf1bm6$18t9$1 digitaldaemon.com
| DMD on Linux, compiled (1 GB RAM) and the size is exactly 2.0Mb.

But that'd be for the gcc linker. What about OPTLINK? Can you test it on
Windows?

-----------------------
Carlos Santander Bernal
Aug 06 2004
prev sibling parent Juanjo =?ISO-8859-15?Q?=C1lvarez?= <juanjuxNO SPAMyahoo.es> writes:
Carlos Santander B. wrote:


 "Arcane Jill" <Arcane_member pathlink.com> escribió en el mensaje
 news:cevac8$2tsm$1 digitaldaemon.com
 | Wow. Sounds like you've got it.
 |
 | You could try changing
 |

 |
 | to
 |




 |
 | and if /that/ still works, I'd say you've definitely licked the problem!
 | (This is almost exactly what my own lookup functions do, you see. If
 | this can be
 made
 | to work then we're rolling!
 |
 | Arcane Jill
 
 Sorry, can't do that: DMD ran out of memory on my computer.
 Somebody else can try. Compile the file generated by this:
 
 //////////////////////////////
 import std.stream;
 
 void main ()
 {
     auto MemoryStream ms = new MemoryStream;
     ms.writeLine ( "const byte LOOKUP[1000000] = [" );
     for ( uint i=1; i<50000; ++i )
         ms.writeLine ( "0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9," );
     ms.writeLine ( "0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9];" );
     ms.writeLine ( "byte f(int n) { return LOOKUP[n]; } void main () {}"
     ); ms.position = 0;
     auto BufferedFile bf = new BufferedFile ( "test.d", FileMode.Out );
     bf.copyFrom ( ms );
 }
 //////////////////////////////
 
 The file would be over 2MB.
 


Of course the generated .d file is 2Mb but the executable "test" is 1.1Mb
Aug 06 2004
prev sibling parent Ilya Minkov <minkov cs.tum.edu> writes:
The digitalmars linker understands special directives in modules which 
denote the borders of what needs to be pulled in for each symbol. These 
are generated automatically when compiling into a lib, or have to be 
turned on manually otherwise. Thus it is able to link in separate 
functions and not only whole modules.

Other linkers may or may not have a similar feature, most seem to. The 
GNU ld for Windows apparently doesn't, but ld doesn't have much in 
common between platforms, so elsewhere it probably does.

Where you are right about "indivisibleness" is a class: it pulls in the 
VTable which references all of its functions, which will then have to be 
linked in. However Sather (the language which only supports 
whole-program compilation) also dealed with that, by analyzing what 
methods can potentially be called. It did a grand lot of analysis of 
what can be inlined, eliminated, etc. So i think we should be able to 
escape the bloatware curse this way someday, but we much now make sure 
that the spec doesn't prevent such analysis.

-eye
Aug 15 2004
prev sibling parent reply "Martin M. Pedersen" <martin moeller-pedersen.dk> writes:
"Ant" <Ant_member pathlink.com> skrev i en meddelelse
news:ceo2od$170c$1 digitaldaemon.com...

 dool is to complement phobos where phobos doesn't support he OO paradigm.
 read all about it at:


I'm a bit concerned about the usability of the project - if the manifest is
to be taken literally.

For a library to be really useful, it needs to have stable interfaces, good
documentation, and some formalism to ensure the quality of the
implementation. Consider boost. It has both tests and peer reaview. You
might consider to have two levels of quality: One for libraries that are
somehow formally approved as production quality, and another for
development. It then would make sense to use only production level libraries
except for the parts where one self might participate in the development.

Then consider templates and mixins. Few people wants to use C++ without STL.
Yet, the template mechanism isn't OO, and STL is not OO either. One of the
things that makes is versatile, is that it decouples algorithms from data.
Does "dool" exclude mechanisms such as templates and mixins?

Regards,
Martin M. Pedersen
Aug 03 2004
parent Ant <Ant_member pathlink.com> writes:
In article <ceom81$1fns$1 digitaldaemon.com>, Martin M. Pedersen says...

"Ant" <Ant_member pathlink.com> skrev i en meddelelse
news:ceo2od$170c$1 digitaldaemon.com...

 dool is to complement phobos where phobos doesn't support he OO paradigm.
 read all about it at:


I'm a bit concerned about the usability of the project - if the manifest is
to be taken literally.

For a library to be really useful, it needs to have stable interfaces, good
documentation, and some formalism to ensure the quality of the
implementation. Consider boost. It has both tests and peer reaview. You
might consider to have two levels of quality: One for libraries that are
somehow formally approved as production quality, and another for
development.


that goes without saying (well apparently it doesn't)
I mean dool is just starting, expect changes, I believe it would be
the best way to get it to a rapid start.


Then consider templates and mixins. Few people wants to use C++ without STL.
Yet, the template mechanism isn't OO, and STL is not OO either. One of the
things that makes is versatile, is that it decouples algorithms from data.
Does "dool" exclude mechanisms such as templates and mixins?


I expect the templates to comply to OO and define classes.
(Alghout generic programming is not a new concept for me
I never actually used it before.)
I see mixins (a new concept for me) as complement of the single inheritance
OO paradigm.

Ant
Aug 03 2004