www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Why D is not for me?

reply "Todor Totev" <umbra.tenebris list.ru> writes:
Hello all,
what started as a simple request for help growed up to some kind of essa=
y.
Here it is:

Why D is not for me?
The scene of this little opera:
I'm working for a small firm which products use kernel modules for Windo=
ws.
Part of our tests are built-in in the modules themselves but others
are user-mode programmes. But it is quite common to go one level lower
and use the so called "Native API" to do some tricks.
My boss told me that I need to made a small test programme that exercise=
s
a obscure pattern of behaviour of our product. I thought: this is  =

relatively
a small programme that is not top priority but nevertheless is useful an=
d
with clear goal. A right test for what can D do for me.
And my quest started.

I've read the information on the site, I was eager to start and I  =

downloaded
dmd.zip. I wrote my first programme which uses Native API.
And now is the right time for my first obstacle. As in any
self-respecting RPG quest, it is an easy task:

1. The official distribution of D is the only language distribution seen=
  =

by me
that is not self sufficient.
   1.1. no linker - no linker? How am I supposed to build my programmes?=

   1.2. no libraries - consider the default command line that dmd builds=

        for optlink - kernel32+gdi32.
        Where is kernel32.lib? Where is gdi32.lib?

Please, compare dmd distribution to Python or Ruby ones.
   Ok, I'm not afraid of some digging and I actually read the documentat=
ion,
so I downloaded dmc package and installed it. The miracle happened and n=
ow
I have kernel32.lib. Which leads me to:

2. D is the only language which aims at system programming and in the sa=
me
time cannot inter-operate with the operating system.
   2.1. No easy support for importing Windows types & functions not
        included by default within Phobos.

Want to easily convert a .h file to .d? Yes of course! Not with D.
Because it does not define all basic types so you can go straight to
missing ones. So what, use Core32 from dsource.org! But one must first
find that this project exists! And then, hurray, it conflicts with Phobo=
s!
Good, now I must find out what to comment out. Five minutes later i'm do=
ne
with this task.

   2.2. The libraries included are old. Outdated. Unusable to me. And I =
even
        didn't try to use Transactional NTFS functions from Vista beta  =

WDK/SDK.
        Plain old, boring Windows 2000/XP functions (5 years old, mind t=
his)
        are nowhere to be seen. Functions like FindFirstVolumeMountPoint=

        or GetVolumePathNamesForVolumeName.
   2.3. No out-of-box support for lib files coming with SDK/DDK/IFS/WDK.=

   2.4. No implib functionality so I cannot create up-to date lib files
        from the dll files.

I dig some more and found that there is coff2omf utility, which:
  * produces incompatible lib files, if I want to use the available Borl=
and  =

one
  * is not available free if I want to use the digital mars one
  * actually do not work because it requires special version of
  <b>Microsoft</b> link.exe that cannot be found anywhere.
  So to use Digital Mars product, I must pay to Microsoft and pray
  that I can found their very old linker somewhere in MSDN archives?
I dig some more and found the special version of OpenWatcom wlink exe
that can do the trick. And indeed, it did the trick.
After about 5 hours fight with it, I was able to spell the right  =

incantation
that linked my 20 lines of D code.
That includes special order of library paths so that Watcom's kernel32 i=
s
first and special order of libraries so that snn comes first.
Really, a good job for starter in a completely new language and  =

environment!
D made me feel proud of myself. And I lost 5 hours but now I know why
most people think programming is some kind of wizardry.
If I opted for MS C++ I will be ready in less than a minute.

 =

tests

   And actually that cannot be called "productive". I now must support t=
hree
separate entities: D compiler, DigitalMars C compiler and OpenWatcom  =

linker.
And of course, for every project I start from now, I must build a bat fi=
le
and linker response file tailored for that project. No build.exe for me.=

No simple dmd *.d my.lib my2.lib. Nothing like this.

Ok, my programme is linked and I have an exe file. But as my ex-boss
likes to say: "No-one should commit code in the source control system
unless she had stepped through every one of the new code in the debugger=
."
I know that all the tools of D language & infrastructure are there to
help me spend as little time in the debugger as possible.
Tools like contracts, unit tests, code coverage etc.
But only the debugger can tell you the flow of code.

3. Where is the debugger?
No near me, for sure.
I hear "WinDbg" screamed, am I not?
First, it does not come with D/dmd.
Second, you must find it in the Microsoft site - not an easy task.
Third, half an year earlier when WinDbg out of the blue got
the docking windows and user interface that made it actually useful
some of my colleague go out, bought some beer and celebrated it.
Fourth, that WinDbg which is quite user-unfriendly does not work with D.=

There are anecdotal evidence that version of WinDbg developed circa 1996=

can be used but I'm not sure I want to touch this beast.
   What other languages offer? Lua, Python & Ruby have built-in
debugging facilities. VisualStudio have a debugger which can be extend
to show your data the way you want it. The dotNet SDK comes with a
debugger (and an excellent one). D simply does not have working debugger=
.

To proceed with other problems, I will leave our beautiful scene
and go back in time to February 2005. In that time I had to do another
little project. This time I was choosing between 3 languages:
D, Lua and Io. I chose Lua because:

4. Documentation
Lua comes with the best book for a programming language I read so far
in my life. It is called "Programming in Lua" and is available from
http://www.lua.org/pil/. A book about a language gives a newbie a base
to stand. It gave me confidence. D (and Io) do not have this. Languages =
 =

like
Python and Ruby come with "Programming Python" and "Programming Ruby" bo=
oks
in the zip files you download from their sites. A newbie is lost in the =
 =

scarce
documentation of D. "Programming Python/Ruby/Lua" gives you the boost to=
  =

start.
I cannot stress how important is a good book for the newbie.
Just go to the link I give and read the book. And then compare to D  =

documents.

5. Often changes of the compiler.
This is a controversial issue. But please consider, how many versions of=
  =

dmd
came from February till now? Even the projects in dsource.org cannot cat=
ch
with them. Core32 collides with current phobos. DFL notes that it is tes=
ted
with version 140 and the author cannot help if you use other version.
Lua on the other side stays unchanged from 2001. Most of its bugs are
documented and their solutions are known. Also, the new version that wil=
l  =

be
out this week is known from half an year. Its new features are known. Yo=
u  =

can
prepare for them (I am currently modifying my code to be ready for it an=
d
send patches to authors for bugs I encounter). D just announces new  =

features
and deprecation of old ones and leaves you to adapt.
I can't build my projects on a base that slips and changes every month.
I know Walter thinks of setting version 1.0 but that is in the future
and can't help me now.


Consider the "Major goals of D" and compare to what I said earlier:
  * Reduce software development costs by at least 10% - losing 5 hours j=
ust
  to link a simple 20 lines programme won't reduce the cost of its  =

development.
  The requirements for 3rd party tools (namely Microsoft linker, OpenWat=
com
  suite) and 4th party tools (the patched wlink for DMC) make the situat=
ion
  as bad as possible.
  * Provide low level bare metal access as required - it is not even nea=
r,
  as I really need access to ntdll.dll (the lowest level to bare metal
  in user mode available) and D actively hinders me.
  * Be compatible with the local C application binary interface - Micros=
oft
  tools are de facto standard for Windows platform and D does not talk t=
o  =

them.


To conclude, language D is a daydream coming true to every  =

C/C++ programmer.
But what D as language gives, D as infrastructure takes tenfolds.
I admit all the problems I presented are not unresolvable,
the big problem is that I must resolve or work around them before doing =
any
actual work. Also I know about projects like Poseidon or D Forms Library=

which prove that D is useful. All I state is that D is not useful for me=

in its current form. And I think that the problems I have are applicable=

to other people too. Also it is possible that I mis-read the documentati=
on
and there is an easy way to do what I do and I simply didn't found it.
But I read as much as I can, searched in the newsgroup archives and
I think I made the best possible effort to do the things right.
Last, I know some of the problems are hard to solve.
Microsoft keeps interface of mspdb??.dll files for themselves so noone c=
an
easily produce pdb files in the new format. And it is not redistributabl=
e.
I know that writing a book is a demanding task that often requires more =
 =

than year
to complete. I thought of creating a module that in its contructor  =

dynamically binds
to functions in ntdll & write simple wrappers in D. Et cetera.
I read the newsgroups since version 0.96 or .98 (can't remember exactly)=
.
I'm a competent Windows developer and if Poseidon turns a good IDE I can=

try to write a decent wrapper around dbghelp.dll to help build a proper
debugger. But now I feel that infrastructure of D have a room for  =

improvements.

Regards,
Todor Totev
Dec 27 2005
next sibling parent "Charles" <noone nowhere.com> writes:
All valid concerns , but it sounds like you want to jump into a brand new
language without having to learn or put any effort into it.

D is not an out-of-the-box solution , and as you said isn't even 1.0 yet.

Im afraid you took the long way around trying to get some library working
( you didnt mention which ? ) -- but Ill present a counter example .. have
you ever used dbghelp.dll in an MFC application ?  How many ongodly hoops
must you jump through to get that working ....

I like  how you sepereated out

D the language
D the infranstructure

as your right the two are very seperate , and D the infrastructure does need
alot of work to become a viable commercial solution.

Charlie


"Todor Totev" <umbra.tenebris list.ru> wrote in message
news:op.s2f8okolihwmk4 todor-1-xp.sanbolic.local...


Hello all,
what started as a simple request for help growed up to some kind of essay.
Here it is:

Why D is not for me?
The scene of this little opera:
I'm working for a small firm which products use kernel modules for Windows.
Part of our tests are built-in in the modules themselves but others
are user-mode programmes. But it is quite common to go one level lower
and use the so called "Native API" to do some tricks.
My boss told me that I need to made a small test programme that exercises
a obscure pattern of behaviour of our product. I thought: this is
relatively
a small programme that is not top priority but nevertheless is useful and
with clear goal. A right test for what can D do for me.
And my quest started.

I've read the information on the site, I was eager to start and I
downloaded
dmd.zip. I wrote my first programme which uses Native API.
And now is the right time for my first obstacle. As in any
self-respecting RPG quest, it is an easy task:

1. The official distribution of D is the only language distribution seen
by me
that is not self sufficient.
   1.1. no linker - no linker? How am I supposed to build my programmes?
   1.2. no libraries - consider the default command line that dmd builds
        for optlink - kernel32+gdi32.
        Where is kernel32.lib? Where is gdi32.lib?

Please, compare dmd distribution to Python or Ruby ones.
   Ok, I'm not afraid of some digging and I actually read the documentation,
so I downloaded dmc package and installed it. The miracle happened and now
I have kernel32.lib. Which leads me to:

2. D is the only language which aims at system programming and in the same
time cannot inter-operate with the operating system.
   2.1. No easy support for importing Windows types & functions not
        included by default within Phobos.

Want to easily convert a .h file to .d? Yes of course! Not with D.
Because it does not define all basic types so you can go straight to
missing ones. So what, use Core32 from dsource.org! But one must first
find that this project exists! And then, hurray, it conflicts with Phobos!
Good, now I must find out what to comment out. Five minutes later i'm done
with this task.

   2.2. The libraries included are old. Outdated. Unusable to me. And I even
        didn't try to use Transactional NTFS functions from Vista beta
WDK/SDK.
        Plain old, boring Windows 2000/XP functions (5 years old, mind this)
        are nowhere to be seen. Functions like FindFirstVolumeMountPoint
        or GetVolumePathNamesForVolumeName.
   2.3. No out-of-box support for lib files coming with SDK/DDK/IFS/WDK.
   2.4. No implib functionality so I cannot create up-to date lib files
        from the dll files.

I dig some more and found that there is coff2omf utility, which:
  * produces incompatible lib files, if I want to use the available Borland
one
  * is not available free if I want to use the digital mars one
  * actually do not work because it requires special version of
  <b>Microsoft</b> link.exe that cannot be found anywhere.
  So to use Digital Mars product, I must pay to Microsoft and pray
  that I can found their very old linker somewhere in MSDN archives?
I dig some more and found the special version of OpenWatcom wlink exe
that can do the trick. And indeed, it did the trick.
After about 5 hours fight with it, I was able to spell the right
incantation
that linked my 20 lines of D code.
That includes special order of library paths so that Watcom's kernel32 is
first and special order of libraries so that snn comes first.
Really, a good job for starter in a completely new language and
environment!
D made me feel proud of myself. And I lost 5 hours but now I know why
most people think programming is some kind of wizardry.
If I opted for MS C++ I will be ready in less than a minute.

tests

   And actually that cannot be called "productive". I now must support three
separate entities: D compiler, DigitalMars C compiler and OpenWatcom
linker.
And of course, for every project I start from now, I must build a bat file
and linker response file tailored for that project. No build.exe for me.
No simple dmd *.d my.lib my2.lib. Nothing like this.

Ok, my programme is linked and I have an exe file. But as my ex-boss
likes to say: "No-one should commit code in the source control system
unless she had stepped through every one of the new code in the debugger."
I know that all the tools of D language & infrastructure are there to
help me spend as little time in the debugger as possible.
Tools like contracts, unit tests, code coverage etc.
But only the debugger can tell you the flow of code.

3. Where is the debugger?
No near me, for sure.
I hear "WinDbg" screamed, am I not?
First, it does not come with D/dmd.
Second, you must find it in the Microsoft site - not an easy task.
Third, half an year earlier when WinDbg out of the blue got
the docking windows and user interface that made it actually useful
some of my colleague go out, bought some beer and celebrated it.
Fourth, that WinDbg which is quite user-unfriendly does not work with D.
There are anecdotal evidence that version of WinDbg developed circa 1996
can be used but I'm not sure I want to touch this beast.
   What other languages offer? Lua, Python & Ruby have built-in
debugging facilities. VisualStudio have a debugger which can be extend
to show your data the way you want it. The dotNet SDK comes with a
debugger (and an excellent one). D simply does not have working debugger.

To proceed with other problems, I will leave our beautiful scene
and go back in time to February 2005. In that time I had to do another
little project. This time I was choosing between 3 languages:
D, Lua and Io. I chose Lua because:

4. Documentation
Lua comes with the best book for a programming language I read so far
in my life. It is called "Programming in Lua" and is available from
http://www.lua.org/pil/. A book about a language gives a newbie a base
to stand. It gave me confidence. D (and Io) do not have this. Languages
like
Python and Ruby come with "Programming Python" and "Programming Ruby" books
in the zip files you download from their sites. A newbie is lost in the
scarce
documentation of D. "Programming Python/Ruby/Lua" gives you the boost to
start.
I cannot stress how important is a good book for the newbie.
Just go to the link I give and read the book. And then compare to D
documents.

5. Often changes of the compiler.
This is a controversial issue. But please consider, how many versions of
dmd
came from February till now? Even the projects in dsource.org cannot catch
with them. Core32 collides with current phobos. DFL notes that it is tested
with version 140 and the author cannot help if you use other version.
Lua on the other side stays unchanged from 2001. Most of its bugs are
documented and their solutions are known. Also, the new version that will
be
out this week is known from half an year. Its new features are known. You
can
prepare for them (I am currently modifying my code to be ready for it and
send patches to authors for bugs I encounter). D just announces new
features
and deprecation of old ones and leaves you to adapt.
I can't build my projects on a base that slips and changes every month.
I know Walter thinks of setting version 1.0 but that is in the future
and can't help me now.


Consider the "Major goals of D" and compare to what I said earlier:
  * Reduce software development costs by at least 10% - losing 5 hours just
  to link a simple 20 lines programme won't reduce the cost of its
development.
  The requirements for 3rd party tools (namely Microsoft linker, OpenWatcom
  suite) and 4th party tools (the patched wlink for DMC) make the situation
  as bad as possible.
  * Provide low level bare metal access as required - it is not even near,
  as I really need access to ntdll.dll (the lowest level to bare metal
  in user mode available) and D actively hinders me.
  * Be compatible with the local C application binary interface - Microsoft
  tools are de facto standard for Windows platform and D does not talk to
them.


To conclude, language D is a daydream coming true to every
C/C++ programmer.
But what D as language gives, D as infrastructure takes tenfolds.
I admit all the problems I presented are not unresolvable,
the big problem is that I must resolve or work around them before doing any
actual work. Also I know about projects like Poseidon or D Forms Library
which prove that D is useful. All I state is that D is not useful for me
in its current form. And I think that the problems I have are applicable
to other people too. Also it is possible that I mis-read the documentation
and there is an easy way to do what I do and I simply didn't found it.
But I read as much as I can, searched in the newsgroup archives and
I think I made the best possible effort to do the things right.
Last, I know some of the problems are hard to solve.
Microsoft keeps interface of mspdb??.dll files for themselves so noone can
easily produce pdb files in the new format. And it is not redistributable.
I know that writing a book is a demanding task that often requires more
than year
to complete. I thought of creating a module that in its contructor
dynamically binds
to functions in ntdll & write simple wrappers in D. Et cetera.
I read the newsgroups since version 0.96 or .98 (can't remember exactly).
I'm a competent Windows developer and if Poseidon turns a good IDE I can
try to write a decent wrapper around dbghelp.dll to help build a proper
debugger. But now I feel that infrastructure of D have a room for
improvements.

Regards,
Todor Totev
Dec 27 2005
prev sibling next sibling parent bobef <bobef lessequal.com> writes:
LOL. No offence but this is why products which are driving the 
programming/computers/world down (in my opinion) have such big success. 
People want everything to be a big template. Aranged just for them. 
Without the need to think. Without the need to do anything. Just choose 
from a template. Of course I am looking from a non-comercial 
perpespective. I value D because it is fast, easy, GREAT! Maybe for 
people who want to make money from it these problems matters but still. 

choose-one-from-the-template-wizard crap you can do with D. Sometimes it 
requires more coding and when it doesn't come out of the box with 
library X people don't even think that they can do it on their own. And 
after all if we all code in D  and forget about the slow DND (so called 
programming) crap who will make the people buy faster computers? Because 
you cant make a D program run slow. Unlike the virtual machines where 
you cant make a program run fast.

Todor Totev wrote:

 
 
 Hello all,
 what started as a simple request for help growed up to some kind of essay.
 Here it is:
 
 Why D is not for me?
 The scene of this little opera:
 I'm working for a small firm which products use kernel modules for Windows.
 Part of our tests are built-in in the modules themselves but others
 are user-mode programmes. But it is quite common to go one level lower
 and use the so called "Native API" to do some tricks.
 My boss told me that I need to made a small test programme that exercises
 a obscure pattern of behaviour of our product. I thought: this is  
 relatively
 a small programme that is not top priority but nevertheless is useful and
 with clear goal. A right test for what can D do for me.
 And my quest started.
 
 I've read the information on the site, I was eager to start and I  
 downloaded
 dmd.zip. I wrote my first programme which uses Native API.
 And now is the right time for my first obstacle. As in any
 self-respecting RPG quest, it is an easy task:
 
 1. The official distribution of D is the only language distribution 
 seen  by me
 that is not self sufficient.
   1.1. no linker - no linker? How am I supposed to build my programmes?
   1.2. no libraries - consider the default command line that dmd builds
        for optlink - kernel32+gdi32.
        Where is kernel32.lib? Where is gdi32.lib?
 
 Please, compare dmd distribution to Python or Ruby ones.
   Ok, I'm not afraid of some digging and I actually read the documentation,
 so I downloaded dmc package and installed it. The miracle happened and now
 I have kernel32.lib. Which leads me to:
 
 2. D is the only language which aims at system programming and in the same
 time cannot inter-operate with the operating system.
   2.1. No easy support for importing Windows types & functions not
        included by default within Phobos.
 
 Want to easily convert a .h file to .d? Yes of course! Not with D.
 Because it does not define all basic types so you can go straight to
 missing ones. So what, use Core32 from dsource.org! But one must first
 find that this project exists! And then, hurray, it conflicts with Phobos!
 Good, now I must find out what to comment out. Five minutes later i'm done
 with this task.
 
   2.2. The libraries included are old. Outdated. Unusable to me. And I even
        didn't try to use Transactional NTFS functions from Vista beta  
 WDK/SDK.
        Plain old, boring Windows 2000/XP functions (5 years old, mind this)
        are nowhere to be seen. Functions like FindFirstVolumeMountPoint
        or GetVolumePathNamesForVolumeName.
   2.3. No out-of-box support for lib files coming with SDK/DDK/IFS/WDK.
   2.4. No implib functionality so I cannot create up-to date lib files
        from the dll files.
 
 I dig some more and found that there is coff2omf utility, which:
  * produces incompatible lib files, if I want to use the available 
 Borland  one
  * is not available free if I want to use the digital mars one
  * actually do not work because it requires special version of
  <b>Microsoft</b> link.exe that cannot be found anywhere.
  So to use Digital Mars product, I must pay to Microsoft and pray
  that I can found their very old linker somewhere in MSDN archives?
 I dig some more and found the special version of OpenWatcom wlink exe
 that can do the trick. And indeed, it did the trick.
 After about 5 hours fight with it, I was able to spell the right  
 incantation
 that linked my 20 lines of D code.
 That includes special order of library paths so that Watcom's kernel32 is
 first and special order of libraries so that snn comes first.
 Really, a good job for starter in a completely new language and  
 environment!
 D made me feel proud of myself. And I lost 5 hours but now I know why
 most people think programming is some kind of wizardry.
 If I opted for MS C++ I will be ready in less than a minute.

 tests

   And actually that cannot be called "productive". I now must support three
 separate entities: D compiler, DigitalMars C compiler and OpenWatcom  
 linker.
 And of course, for every project I start from now, I must build a bat file
 and linker response file tailored for that project. No build.exe for me.
 No simple dmd *.d my.lib my2.lib. Nothing like this.
 
 Ok, my programme is linked and I have an exe file. But as my ex-boss
 likes to say: "No-one should commit code in the source control system
 unless she had stepped through every one of the new code in the debugger."
 I know that all the tools of D language & infrastructure are there to
 help me spend as little time in the debugger as possible.
 Tools like contracts, unit tests, code coverage etc.
 But only the debugger can tell you the flow of code.
 
 3. Where is the debugger?
 No near me, for sure.
 I hear "WinDbg" screamed, am I not?
 First, it does not come with D/dmd.
 Second, you must find it in the Microsoft site - not an easy task.
 Third, half an year earlier when WinDbg out of the blue got
 the docking windows and user interface that made it actually useful
 some of my colleague go out, bought some beer and celebrated it.
 Fourth, that WinDbg which is quite user-unfriendly does not work with D.
 There are anecdotal evidence that version of WinDbg developed circa 1996
 can be used but I'm not sure I want to touch this beast.
   What other languages offer? Lua, Python & Ruby have built-in
 debugging facilities. VisualStudio have a debugger which can be extend
 to show your data the way you want it. The dotNet SDK comes with a
 debugger (and an excellent one). D simply does not have working debugger.
 
 To proceed with other problems, I will leave our beautiful scene
 and go back in time to February 2005. In that time I had to do another
 little project. This time I was choosing between 3 languages:
 D, Lua and Io. I chose Lua because:
 
 4. Documentation
 Lua comes with the best book for a programming language I read so far
 in my life. It is called "Programming in Lua" and is available from
 http://www.lua.org/pil/. A book about a language gives a newbie a base
 to stand. It gave me confidence. D (and Io) do not have this. Languages  
 like
 Python and Ruby come with "Programming Python" and "Programming Ruby" books
 in the zip files you download from their sites. A newbie is lost in the  
 scarce
 documentation of D. "Programming Python/Ruby/Lua" gives you the boost 
 to  start.
 I cannot stress how important is a good book for the newbie.
 Just go to the link I give and read the book. And then compare to D  
 documents.
 
 5. Often changes of the compiler.
 This is a controversial issue. But please consider, how many versions 
 of  dmd
 came from February till now? Even the projects in dsource.org cannot catch
 with them. Core32 collides with current phobos. DFL notes that it is tested
 with version 140 and the author cannot help if you use other version.
 Lua on the other side stays unchanged from 2001. Most of its bugs are
 documented and their solutions are known. Also, the new version that 
 will  be
 out this week is known from half an year. Its new features are known. 
 You  can
 prepare for them (I am currently modifying my code to be ready for it and
 send patches to authors for bugs I encounter). D just announces new  
 features
 and deprecation of old ones and leaves you to adapt.
 I can't build my projects on a base that slips and changes every month.
 I know Walter thinks of setting version 1.0 but that is in the future
 and can't help me now.
 
 
 Consider the "Major goals of D" and compare to what I said earlier:
  * Reduce software development costs by at least 10% - losing 5 hours just
  to link a simple 20 lines programme won't reduce the cost of its  
 development.
  The requirements for 3rd party tools (namely Microsoft linker, OpenWatcom
  suite) and 4th party tools (the patched wlink for DMC) make the situation
  as bad as possible.
  * Provide low level bare metal access as required - it is not even near,
  as I really need access to ntdll.dll (the lowest level to bare metal
  in user mode available) and D actively hinders me.
  * Be compatible with the local C application binary interface - Microsoft
  tools are de facto standard for Windows platform and D does not talk 
 to  them.
 
 
 To conclude, language D is a daydream coming true to every  C/C++ 
 programmer.
 But what D as language gives, D as infrastructure takes tenfolds.
 I admit all the problems I presented are not unresolvable,
 the big problem is that I must resolve or work around them before doing any
 actual work. Also I know about projects like Poseidon or D Forms Library
 which prove that D is useful. All I state is that D is not useful for me
 in its current form. And I think that the problems I have are applicable
 to other people too. Also it is possible that I mis-read the documentation
 and there is an easy way to do what I do and I simply didn't found it.
 But I read as much as I can, searched in the newsgroup archives and
 I think I made the best possible effort to do the things right.
 Last, I know some of the problems are hard to solve.
 Microsoft keeps interface of mspdb??.dll files for themselves so noone can
 easily produce pdb files in the new format. And it is not redistributable.
 I know that writing a book is a demanding task that often requires more  
 than year
 to complete. I thought of creating a module that in its contructor  
 dynamically binds
 to functions in ntdll & write simple wrappers in D. Et cetera.
 I read the newsgroups since version 0.96 or .98 (can't remember exactly).
 I'm a competent Windows developer and if Poseidon turns a good IDE I can
 try to write a decent wrapper around dbghelp.dll to help build a proper
 debugger. But now I feel that infrastructure of D have a room for  
 improvements.
 
 Regards,
 Todor Totev
Dec 27 2005
prev sibling next sibling parent "Derek Parnell" <derek psych.ward> writes:
On Wed, 28 Dec 2005 02:44:34 +1100, Todor Totev <umbra.tenebris list.ru>  
wrote:


 Hello all,
 what started as a simple request for help growed up to some kind of  
 essay.
 Here it is:

 Why D is not for me?


A well thought out and restrained essay. I agree with will all you say, to  
varying degrees. I imagine that the primary reason for the omissions you  
detail is the lack of people working on the problems. Walter is the  
bottleneck that all submissions must pass through at the moment. Even  
libraries such as Ares and Mango which go to replace and expand Phobos are  
short on active workers.

Although the language itself, as opposed to tools and implementation, is  
still in flux, it is almost stable.

-- 
Derek Parnell
Melbroune, Australia
Dec 27 2005
prev sibling next sibling parent reply James Dunne <james.jdunne gmail.com> writes:
Todor Totev wrote:

 
 
 Hello all,
 what started as a simple request for help growed up to some kind of essay.


If you're referring to your "Problem with COFF library" post, then I'd 
have to say that

   1) You didn't leave enough time for someone to come and provide a
      clear response to your unclear request for help.
   2) Your request was very far off in a tiny niche of the support corner
      that it makes it very hard to reproduce what it is you're
      discovering (let alone without any code examples given).


 Here it is:
 
 Why D is not for me?
 The scene of this little opera:
 I'm working for a small firm which products use kernel modules for Windows.
 Part of our tests are built-in in the modules themselves but others
 are user-mode programmes. But it is quite common to go one level lower
 and use the so called "Native API" to do some tricks.
 My boss told me that I need to made a small test programme that exercises
 a obscure pattern of behaviour of our product. I thought: this is  
 relatively
 a small programme that is not top priority but nevertheless is useful and
 with clear goal. A right test for what can D do for me.
 And my quest started.
 
 I've read the information on the site, I was eager to start and I  
 downloaded
 dmd.zip. I wrote my first programme which uses Native API.
 And now is the right time for my first obstacle. As in any
 self-respecting RPG quest, it is an easy task:
 
 1. The official distribution of D is the only language distribution 
 seen  by me
 that is not self sufficient.
   1.1. no linker - no linker? How am I supposed to build my programmes?
   1.2. no libraries - consider the default command line that dmd builds
        for optlink - kernel32+gdi32.
        Where is kernel32.lib? Where is gdi32.lib?
 
 Please, compare dmd distribution to Python or Ruby ones.


Ok.  Python and Ruby do not run on the bare metal of the processor. 
They have their own bytecode and interpreter.  Their interpreters for 
the Windows platform were undoubtedly created using Microsoft's C++ 
compiler and hence having full access to the Windows platform and all of 
its libraries.  Writing Microsoft-specific implementations for the 
Python or Ruby standard library functions is now an easy task.  You 
don't need to link your Python or Ruby executable to kernel32.lib or 
gdi32.lib, because the interpreter that runs your Python/Ruby program is 
already linked to those.

This is simply not the case with DMD.  (and you will hear this repeated 
later in this post)


   Ok, I'm not afraid of some digging and I actually read the documentation,
 so I downloaded dmc package and installed it. The miracle happened and now
 I have kernel32.lib. Which leads me to:


DMD is based on the Digital Mars C/C++ toolset as you've discovered.  On 
the download page the two are together and the instructions clearly 
state that 'dmc.zip' is a requirement (from 
http://digitalmars.com/d/dcompiler.html#Win32):

"Requirements

     * 32 bit Windows (Win32) operating system, such as Windows XP
     * dmd.zip (D compiler) for Win32
     * dmc.zip (linker and utilities) for Win32"

Yes, slightly annoying to those of us who'd skip over this and just grab 
what we think we only need... But then again why package the same tools 
in two different archives?  This way you only have to update your dmc 
once (or whenever DMC actually changes which is rare), and just keep 
updating the dmd side of things.  Bandwidth saved.  Time saved (unless 
you didn't read =P).


 2. D is the only language which aims at system programming and in the same
 time cannot inter-operate with the operating system.
   2.1. No easy support for importing Windows types & functions not
        included by default within Phobos.
 


D is not a platform-dependent language.  You are referring to the D 
infrastructure here, specific to the Windows platform.  It is not the 
purpose of Phobos to become a transparent gateway to the raw Windows 
API.  Yes, admittedly, phobos' Windows support is lacking.


 Want to easily convert a .h file to .d? Yes of course! Not with D.


The h2d tool is on dsource.org, but written in C++ requiring the boost 
libraries.


 Because it does not define all basic types so you can go straight to
 missing ones. So what, use Core32 from dsource.org! But one must first
 find that this project exists! And then, hurray, it conflicts with Phobos!
 Good, now I must find out what to comment out. Five minutes later i'm done
 with this task.
 


Well, that wasn't so bad.  I'd contact the author/maintainer of Core32 
and let them know the problem.


   2.2. The libraries included are old. Outdated. Unusable to me. And I even
        didn't try to use Transactional NTFS functions from Vista beta  
 WDK/SDK.
        Plain old, boring Windows 2000/XP functions (5 years old, mind this)
        are nowhere to be seen. Functions like FindFirstVolumeMountPoint
        or GetVolumePathNamesForVolumeName.
   2.3. No out-of-box support for lib files coming with SDK/DDK/IFS/WDK.
   2.4. No implib functionality so I cannot create up-to date lib files
        from the dll files.
 


If you didn't know, the SDK, DDK, IFS, WDK, whatever from Microsoft are 
all proprietary development kits which must be purchased and licensed at 
hefty costs.  I'm no legal expert, but knowing Microsoft there is 
probably a high barrier to entry for compilers wanting to fully support 
the Windows platforms.  I certainly wouldn't expect, let alone demand, 
from Walter that he shell out cash in the neighborhood of 5-6 figures to 
Microsoft to get such support for his compiler.


 I dig some more and found that there is coff2omf utility, which:
  * produces incompatible lib files, if I want to use the available 
 Borland  one


I don't fully understand your note here.  Use what available Borland 
what now?


  * is not available free if I want to use the digital mars one


Digital Mars has to make their income somehow.  The utility CD is $15. 
Sacrifice a night out at the movies (which is most likely to be > $15) 
and grab your copy.  This is incomparable to what Microsoft would charge 
you for their compiler toolset.


  * actually do not work because it requires special version of
  <b>Microsoft</b> link.exe that cannot be found anywhere.
  So to use Digital Mars product, I must pay to Microsoft and pray
  that I can found their very old linker somewhere in MSDN archives?


Again, I'm not sure what you'd need this old LINK utility for?  If you 
use the Digital Mars toolset CD (available for $15 as noted above), then 
you wouldn't have to delve into these murky waters.  I could be wrong, 
since I'm not clear on your intentions here.


 I dig some more and found the special version of OpenWatcom wlink exe
 that can do the trick. And indeed, it did the trick.
 After about 5 hours fight with it, I was able to spell the right  
 incantation
 that linked my 20 lines of D code.
 That includes special order of library paths so that Watcom's kernel32 is
 first and special order of libraries so that snn comes first.


I'm happy to hear you persevered and got your solution, but still I 
believe things might've worked better by just using the readily 
available Digital Mars toolset.

Besides, this COFF and OMF format thing is hardly Digital Mars' fault. 
Microsoft went their own way, invented their own standard, and closed 
the books on it so hardly anyone else could come and share.  This 
doesn't sound like a standard to me.


 Really, a good job for starter in a completely new language and  
 environment!
 D made me feel proud of myself. And I lost 5 hours but now I know why
 most people think programming is some kind of wizardry.


This isn't programming anymore... =P


 If I opted for MS C++ I will be ready in less than a minute.


Well of course, since MS C++ is the same MS that gave you Windows.  It's 
not Digital Mars Windows XP, now is it? =)  Also, you'd be out $750.



 tests






   And actually that cannot be called "productive". I now must support three
 separate entities: D compiler, DigitalMars C compiler and OpenWatcom  
 linker.
 And of course, for every project I start from now, I must build a bat file
 and linker response file tailored for that project. No build.exe for me.
 No simple dmd *.d my.lib my2.lib. Nothing like this.


Derek Parnell's 'build' utility (available also on dsource.org) will get 
you headed in the right direction here.


 Ok, my programme is linked and I have an exe file. But as my ex-boss
 likes to say: "No-one should commit code in the source control system
 unless she had stepped through every one of the new code in the debugger."
 I know that all the tools of D language & infrastructure are there to
 help me spend as little time in the debugger as possible.
 Tools like contracts, unit tests, code coverage etc.
 But only the debugger can tell you the flow of code.


While your ex-boss may be a wise man stating this, it shouldn't be a 
strict requirement that all code must be stepped-through in a debugger 
in order to be bug-free.  True, it may catch many uncaught bugs, but 
this is what (and you realize this) the tools the language provides to 
you should help to eliminate.


 3. Where is the debugger?
 No near me, for sure.
 I hear "WinDbg" screamed, am I not?


First off, again you're assuming Windows platform here.  GDC and GDB are 
perfectly acceptable alternatives.


 First, it does not come with D/dmd.


Nor should it, because it is not a Digital Mars product.


 Second, you must find it in the Microsoft site - not an easy task.


This is Microsoft's fault alone.  Personally, I think it isn't all that 
hard to find.  Your milage may (and did) vary, of course.


 Third, half an year earlier when WinDbg out of the blue got
 the docking windows and user interface that made it actually useful
 some of my colleague go out, bought some beer and celebrated it.


Slapping a GUI onto a tool does not instantly make it "actually useful". 
  It *might* make it more user-friendly and manageable, but the old tool 
was just as useful.


 Fourth, that WinDbg which is quite user-unfriendly does not work with D.


If you mean that it doesn't work with D in that you can't peek at local 
variables' contents, then I should state that this is a problem with 
Microsoft's tools sporting their proprietary PDB file format.


 There are anecdotal evidence that version of WinDbg developed circa 1996
 can be used but I'm not sure I want to touch this beast.
   What other languages offer? Lua, Python & Ruby have built-in
 debugging facilities. VisualStudio have a debugger which can be extend
 to show your data the way you want it. The dotNet SDK comes with a
 debugger (and an excellent one). D simply does not have working debugger.


None of these languages are 'systems programming' languages.  They work 
through bytecode and interpretation (or JIT compilation) which make it 
significantly easier to debug and step-through.  D compiles down to 
native code, which is not so easy to debug.  I believe a D debugger 
project is in the works on dsource.org (dbug?) but I haven't tried it 
myself.


 To proceed with other problems, I will leave our beautiful scene
 and go back in time to February 2005. In that time I had to do another
 little project. This time I was choosing between 3 languages:
 D, Lua and Io. I chose Lua because:
 
 4. Documentation
 Lua comes with the best book for a programming language I read so far
 in my life. It is called "Programming in Lua" and is available from
 http://www.lua.org/pil/. A book about a language gives a newbie a base
 to stand. It gave me confidence. D (and Io) do not have this. Languages  
 like
 Python and Ruby come with "Programming Python" and "Programming Ruby" books
 in the zip files you download from their sites. A newbie is lost in the  
 scarce
 documentation of D. "Programming Python/Ruby/Lua" gives you the boost 
 to  start.
 I cannot stress how important is a good book for the newbie.
 Just go to the link I give and read the book. And then compare to D  
 documents.


D is assuming a solid understanding of C and [C++ or OOP in general] as 
a start for learning D.  While a book would be great for the 
business-oriented newbie who is being paid to be trained in the 
language, I believe there is no substitue for raw experience gained 
while tinkering with the language.  Once you have the basics down, go 
ahead and read the reference books/docs to find out what you may have 
missed in your tinkering sessions.


 5. Often changes of the compiler.
 This is a controversial issue. But please consider, how many versions 
 of  dmd
 came from February till now? Even the projects in dsource.org cannot catch
 with them. Core32 collides with current phobos. DFL notes that it is tested
 with version 140 and the author cannot help if you use other version.
 Lua on the other side stays unchanged from 2001. Most of its bugs are
 documented and their solutions are known. Also, the new version that 
 will  be
 out this week is known from half an year. Its new features are known. 
 You  can
 prepare for them (I am currently modifying my code to be ready for it and
 send patches to authors for bugs I encounter). D just announces new  
 features
 and deprecation of old ones and leaves you to adapt.
 I can't build my projects on a base that slips and changes every month.
 I know Walter thinks of setting version 1.0 but that is in the future
 and can't help me now.
 
 
 Consider the "Major goals of D" and compare to what I said earlier:
  * Reduce software development costs by at least 10% - losing 5 hours just
  to link a simple 20 lines programme won't reduce the cost of its  
 development.


This was your case, and for what it's worth it was probably the 
worst-case scenario.


  The requirements for 3rd party tools (namely Microsoft linker, OpenWatcom
  suite) and 4th party tools (the patched wlink for DMC) make the situation
  as bad as possible.


These are not requirements for regular development.


  * Provide low level bare metal access as required - it is not even near,
  as I really need access to ntdll.dll (the lowest level to bare metal
  in user mode available) and D actively hinders me.


Bare-metal access does not mean access to operating system libraries. 
It means that the code will run on the native processor without 
requiring an interpreter or JIT compiler.


  * Be compatible with the local C application binary interface - Microsoft
  tools are de facto standard for Windows platform and D does not talk 
 to  them.


Microsoft tools were made to force a de facto standard on the Windows 
platform because they use and integrate so well their own proprietary 
technologies to keep others out.

If you had made a case developing for Linux using either the Linux 
version of DMD or the GDC front-end for GCC, I'm sure your experience 
would have been much more enjoyable.


 To conclude, language D is a daydream coming true to every  C/C++ 
 programmer.
 But what D as language gives, D as infrastructure takes tenfolds.


Very good of you to separate the two.  I applaud this clarity of thinking.


 I admit all the problems I presented are not unresolvable,
 the big problem is that I must resolve or work around them before doing any
 actual work. Also I know about projects like Poseidon or D Forms Library
 which prove that D is useful. All I state is that D is not useful for me
 in its current form. And I think that the problems I have are applicable
 to other people too. Also it is possible that I mis-read the documentation
 and there is an easy way to do what I do and I simply didn't found it.
 But I read as much as I can, searched in the newsgroup archives and
 I think I made the best possible effort to do the things right.
 Last, I know some of the problems are hard to solve.



 Microsoft keeps interface of mspdb??.dll files for themselves so noone can
 easily produce pdb files in the new format. And it is not redistributable.
 I know that writing a book is a demanding task that often requires more  
 than year to complete. I thought of creating a module that in its contructor  
 dynamically binds to functions in ntdll & write simple wrappers in D. Et
cetera.


Why not?  Help out a bit.  You've faced these problems yourself, why 
leave the road as bumpy for the next guy?


 I read the newsgroups since version 0.96 or .98 (can't remember exactly).
 I'm a competent Windows developer and if Poseidon turns a good IDE I can
 try to write a decent wrapper around dbghelp.dll to help build a proper
 debugger. But now I feel that infrastructure of D have a room for  
 improvements.
 
 Regards,
 Todor Totev


To be fair, use the tools that are right for the job.  D might not be 
right for your current project, at the current time.  Don't 
overgeneralize to everyone else stating that D is a completely useless 
language in all areas of development.

Regards,
James Dunne
Dec 27 2005
parent reply "Todor Totev" <umbra.tenebris list.ru> writes:
   Hello all,
after I had read your oppinions I decided to make myself as clear as
possible.
First, I posted after I successfully compiled, linked & executed my test=

programme. Second, from the beginning I was trying to prove that D is
excellent as a system programming language. Not application programming
but system programming.

According to Wikipedia (http://en.wikipedia.org/wiki/System_programming)=
:
...the programmer will make assumptions about the hardware and other
properties of the system that the program runs on, and will often exploi=
t
those properties (for example by using an algorithm that is known to be
efficient when used with specific hardware.


 If you're referring to your "Problem with COFF library" post, then I'd=



As I stated, I published my thoughts after I found solution of all  =

engineering
problems I encounter. The solution is within article
http://www.digitalmars.com/d/archives/digitalmars/D/9911.html.
I just didn't realized this at the time I posted that request.



 You don't need to link your Python or Ruby executable to kernel32.lib =


or  =


 gdi32.lib, because the interpreter that runs your Python/Ruby program =


is  =


 already linked to those.


Actually every single process in Windows have access to ntdll.dll
simply because this library is mapped in the process address space even
before the exe file is being read - actually a code inside ntdll is
responsible for starting the process :-) No excuse for D.

See http://www.digitalmars.com/d/overview.html:
"D is a general purpose systems  programming language."
I really think that such a language should give me easy access to the
platform.


 2. D is the only language which aims at system programming and in the=




  =


 same time cannot inter-operate with the operating system.
   2.1. No easy support for importing Windows types & functions not
        included by default within Phobos.


 D is not a platform-dependent language.  You are referring to the D  =



 infrastructure here, specific to the Windows platform.  It is not the =


 =


 purpose of Phobos to become a transparent gateway to the raw Windows  =



 API.  Yes, admittedly, phobos' Windows support is lacking.


Again:
"D is a general purpose systems programming language."
I trust in the intends of language's author and state that it is not
ready for it.


 Want to easily convert a .h file to .d? Yes of course! Not with D.




I really regret for my particular choise of words.
What I really complain is that Phobos only partially defines very basic
Windows types AND in the same time Core32 which is supposed to complemen=
t  =

it
actually clashes with it. So I had to resolve both issues.
Another problem is that I can patch these but a long-term resolution is =
to
fix them inside phobos & core32.
Of course, I'll send what I discovered to Walter and maintainers of core=
32.


        Plain old, boring Windows 2000/XP functions (5 years old, mind=




  =


 this)
        are nowhere to be seen. Functions like FindFirstVolumeMountPoi=




nt

        or GetVolumePathNamesForVolumeName.
   2.3. No out-of-box support for lib files coming with SDK/DDK/IFS/WD=




K.

   2.4. No implib functionality so I cannot create up-to date lib file=




s

        from the dll files.




Again I want to stress that these issues make system programming with D
near to impossible.


 If you didn't know, the SDK, DDK, IFS, WDK, whatever from Microsoft ar=


e  =


 all proprietary development kits which must be purchased and licensed =


at  =


 hefty costs.  I'm no legal expert, but knowing Microsoft there is  =



 probably a high barrier to entry for compilers wanting to fully suppor=


t  =


 the Windows platforms.


I'm the person responsible for MSDN subscription in our firm.
SDK costs 0$ - it is freely available for download.
DDK is free for MSDN subscribers.
IFS costs 900$ but nearly noone needs it.
MSDN subscription does cost some money but I cannot imagine a serious
development for Windows platform without this.


 I certainly wouldn't expect, let alone demand, from Walter that he she=


ll  =


 out cash in the neighborhood of 5-6 figures to Microsoft to get such  =



 support for his compiler.


Neither I. Walter provides compiler & linker. He surelly does not need
DDK, unless he decides to prove that compiler can be written as a device=

driver (humourous note).
It is I who need to bridge D & DDK. And I do have the DDK.
The problem is that I cannot use it with D because optlink does not
understands COFF libraries.
Also, OpenWatcom does understand COFF files in general and MS supplied
ones in particular out of the box. While Optlink can't?


 I dig some more and found that there is coff2omf utility, which:
  * produces incompatible lib files, if I want to use the available  =





 Borland one




Answer:
http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/24468


  * is not available free if I want to use the digital mars one


 Digital Mars has to make their income somehow.  The utility CD is $15.=


  =


 Sacrifice a night out at the movies (which is most likely to be > $15)=


  =


 and grab your copy.  This is incomparable to what Microsoft would char=


ge  =


 you for their compiler toolset.


The reason I don't wnat to purchase this cd is:
* I do not have credit card.
* Things tend to dissapear in Bulgarian customs - i.e. last month
there was a big scandal because two customs officers have stollen the
cell phone of the american ambassador while he was checked with metal  =

detector.
I still wait for my Ubuntu cds. Although a colleague of mine received hi=
s
DVDs so apparently this is related to man's luck.


  * actually do not work because it requires special version of
  <b>Microsoft</b> link.exe that cannot be found anywhere.
  So to use Digital Mars product, I must pay to Microsoft and pray
  that I can found their very old linker somewhere in MSDN archives?


 Again, I'm not sure what you'd need this old LINK utility for?  If you=


  =


 use the Digital Mars toolset CD (available for $15 as noted above), th=


en  =


 you wouldn't have to delve into these murky waters.  I could be wrong,=


  =


 since I'm not clear on your intentions here.


Answer:
http://www.digitalmars.com/ctg/coff2omf.html
clearly states: use microsoft linker.
The Microsoft linker I have access to complains that it does not know th=
e
option /convert so it is useless.


 I dig some more and found the special version of OpenWatcom wlink exe=





 that can do the trick. And indeed, it did the trick.


 I'm happy to hear you persevered and got your solution, but still I  =



 believe things might've worked better by just using the readily  =



 available Digital Mars toolset.


See the reply above.



 Besides, this COFF and OMF format thing is hardly Digital Mars' fault.=


  =


 Microsoft went their own way, invented their own standard, and closed =


 =


 the books on it so hardly anyone else could come and share.  This  =



 doesn't sound like a standard to me.


Answer:
http://www-128.ibm.com/developerworks/power/library/pa-spec12/?ca=3Ddgr-=
lnxw07ELFHero
(look for "Microsoft's PE-COFF was another standard they documented.").
Also, OpenWatcom & Borland support Microsoft COFF files directly.
So I think this indeed is Optlink fault.


 If I opted for MS C++ I will be ready in less than a minute.


 Well of course, since MS C++ is the same MS that gave you Windows.  It=


's  =


 not Digital Mars Windows XP, now is it? =3D)  Also, you'd be out $750.=



 From http://msdn.microsoft.com/vstudio/express/visualc/
you can download a free version of VS2005 that both
works out-of-box with SDk and does have a debugger (a very nice one).
According to Mattew Wilson  =

(http://synesis.com.au/software/stlsoft/compilers.html):
Prior to version 7.1, the Visual C++ compiler had a number of issues wit=
h  =

respect
to language conformance and robustness, but now is one of the leaders in=
  =

compiler
conformance and quality. Their <b>outstanding</b> IDE has very few  =

competitors in
the marketplace.








Because I must translate c .h files to [DllImport] attributes
and make some structs. Basically exactly what I did to make D source.


   And actually that cannot be called "productive". I now must support=




  =


 three
 separate entities: D compiler, DigitalMars C compiler and OpenWatcom =




  =


 linker.
 And of course, for every project I start from now, I must build a bat=




  =


 file
 and linker response file tailored for that project. No build.exe for =




me.

 No simple dmd *.d my.lib my2.lib. Nothing like this.


 Derek Parnell's 'build' utility (available also on dsource.org) will g=


et  =


 you headed in the right direction here.


According to the site, build does not know how to handle wlink.
And my patchy solution relies on OpenWatcom's wlink.
Besides, it is very interesting that on my complain
"No build.exe for me." you responded with "Use build".


 Ok, my programme is linked and I have an exe file. But as my ex-boss
 likes to say: "No-one should commit code in the source control system=





 unless she had stepped through every one of the new code in the  =





 debugger."


 While your ex-boss may be a wise man stating this, it shouldn't be a  =



 strict requirement that all code must be stepped-through in a debugger=


  =


 in order to be bug-free.  True, it may catch many uncaught bugs, but  =



 this is what (and you realize this) the tools the language provides to=


  =


 you should help to eliminate.


Please make a multithreaded windows service that runs under LocalSystem =
 =

account
and synchronizes itself with another survice that runs under user accoun=
t
right from the first time without using the debugger.
After that write a book about how did you achieved this and I will buy i=
t.
I realy do things like thouse to earn my money so I'm touchy on that  =

subject.


 3. Where is the debugger?
 No near me, for sure.
 I hear "WinDbg" screamed, am I not?


 First off, again you're assuming Windows platform here.  GDC and GDB a=


re  =


 perfectly acceptable alternatives.


Sure, but I need Windows 2000. If GDC can handle this, why not?
Remember, I am system programmer doing system programming.


 First, it does not come with D/dmd.


 Nor should it, because it is not a Digital Mars product.


But comes with Utility CD? Pardon me?


 Second, you must find it in the Microsoft site - not an easy task.


 This is Microsoft's fault alone.  Personally, I think it isn't all tha=


t  =


 hard to find.  Your milage may (and did) vary, of course.


 Third, half an year earlier when WinDbg out of the blue got
 the docking windows and user interface that made it actually useful
 some of my colleague go out, bought some beer and celebrated it.


 Slapping a GUI onto a tool does not instantly make it "actually useful=


".  =


   It *might* make it more user-friendly and manageable, but the old to=


ol  =


 was just as useful.


All these points are valid but did not change the thing that debugger
is badly needed by the programmer and D does not have a easy to discover=
  =

one.


 Fourth, that WinDbg which is quite user-unfriendly does not work with=




 D.

 If you mean that it doesn't work with D in that you can't peek at loca=


l  =


 variables' contents, then I should state that this is a problem with  =



 Microsoft's tools sporting their proprietary PDB file format.


In the example I provided in another post 100% of programme's state is  =

inside
local variables. Debugger that hides 100% of program state is not useful=
  =

one.
Of course this is Microsoft problem, as I admitted at end of my post.


   What other languages offer? Lua, Python & Ruby have built-in
 debugging facilities. VisualStudio have a debugger which can be exten=




d

 to show your data the way you want it. The dotNet SDK comes with a
 debugger (and an excellent one). D simply does not have working  =





 debugger.


 None of these languages are 'systems programming' languages.


I use VisualStudio C++ to develop device drivers. Are you sure that
this do not count as system language? Also the user interface of the

the driver's behaviour it is indeed system software.


I wnated to state that every language must have a built-in debugging  =

facilities
to be useful. Actually the selling point of D is its debugging features.=

Also both OpenWatcom and free Borland tools come with debugger.
And they are C++ compilers.


 They work through bytecode and interpretation (or JIT compilation) whi=


ch  =


 make it significantly easier to debug and step-through.


Not for the way I use C++.


 I believe a D debugger project is in the works on dsource.org (dbug?) =


 =


 but I haven't tried it myself.


The current state of dbug is:
http://www.dsource.org/forums/viewtopic.php?t=3D1009
"you can set breakpoints by clicking on the leftmost margin in
the source windows - removing breakpoints isnt finished and its
getting an exception trying to resume from a breakpoint"
Not useful to me.


 D is assuming a solid understanding of C and [C++ or OOP in general] a=


s  =


 a start for learning D.


Completely wrong. D requires from you to learn D and not C/C++.
What need of C you have to learn OOP or template metaprogramming?
Do the messy RTTI of C++ helps you to work with D's one?
Probably the fact that C does not know how large is int really
helps you to learn using the base types of D?
Et cetera, et cetera.


While a book would be great for the

 business-oriented newbie who is being paid to be trained in the  =



 language, I believe there is no substitue for raw experience gained  =



 while tinkering with the language.


Never been in paid to be trained in a language and can't talk for this.
The books help the programmer to get this experience so there is no
substitusion for them. I tried to learn how dotNet framework works
reading MSDN and I understood nothing. I read a book for newbies
and learned how to ask questions and where to look for answers
(I mean that I learned about System.Runtime.InteropServices,
System.Management etc namespaces and was able to figure out in MSDN
what to do to achieve my goals).

Once you have the basics down, go

 ahead and read the reference books/docs to find out what you may have =


 =


 missed in your tinkering sessions.


Completely opposite: Please go to the site of Lua and check the referenc=
e  =

manual
(what D have) and Programming in Lua book. You can decide for yourself w=
hat
you want to use to begin learning a new language - the interpreter, the =
 =

manual
or the book.


  The requirements for 3rd party tools (namely Microsoft linker,  =





 OpenWatcom suite) and 4th party tools (the patched wlink for DMC) mak=




e  =


 the situation as bad as possible.


 These are not requirements for regular development.


My regular development includes inter-operating with the OS at low level=
s.
I cannot write a notepad-type application without some serious help
or a framework but I can write a basic WMI provider for less than a hour=
.
YMMV.


  * Provide low level bare metal access as required - it is not even  =





 near,
  as I really need access to ntdll.dll (the lowest level to bare metal=





  in user mode available) and D actively hinders me.


 Bare-metal access does not mean access to operating system libraries. =


It  =


 means that the code will run on the native processor without requiring=


  =


 an interpreter or JIT compiler.


Obvously you dont know that Win32 API is the ultimate virtual machine.
It is there to shield the programmer from the NT and presents her a
comfortable, compatible with 16 bit Windows, API.
If you escape its confines you can create files with names that
contains embedded zeroes, or that differs only by case of letters, etc.
Most of the functions inside kernel32/advapi32 either do the work
without going to the OS level or simply convert ASCIIZ to counted string=
s
and ANSI to UNICODE and gives them to the real OS to process them.
This looks like a tipical behaviour of a VM to me.


  * Be compatible with the local C application binary interface -  =





 Microsoft tools are de facto standard for Windows platform andD does =




 =


 not talk to them.


 Microsoft tools were made to force a de facto standard on the Windows =


 =


 platform because they use and integrate so well their own proprietary =


 =


 technologies to keep others out.


Dmd is a proprietary technology so you cannot blame MS doing the same  =

thing.

Lets give you a hipotetical example. You develop a text editor that
implements "search in files" functionality.
Look at this code:

import std.recls;
import std.stdio;

void main()
{
   Search s =3D new Search(null, "*",
     RECLS_FLAG.RECLS_F_RECURSIVE|RECLS_FLAG.RECLS_F_DIRECTORIES);
   foreach (Entry e; s)
     writefln(e.GetPath());
}

Do you see the problem?
Alice as an advanced user had made hard links in her file system.
But the problem is that one of the hard link goes UP in the file system
thus if blindly followed forms a never-ending cycle.
One can recreate the Alice setup using a tool from
http://www.sysinternals.com/utilities/junction.html
You can do this yourself and enjoy the show.
After that come back and tell again that you do not need to
inter-operate with your OS.


 If you had made a case developing for Linux using either the Linux  =



 version of DMD or the GDC front-end for GCC, I'm sure your experience =


 =


 would have been much more enjoyable.


Windows 2000. It's what concerns me.


 To be fair, use the tools that are right for the job.  D might not be =


 =


 right for your current project, at the current time.  Don't  =



 overgeneralize to everyone else stating that D is a completely useless=


  =


 language in all areas of development.


Consider the native string type:
struct UNICODE_STRING {
     USHORT Length;
     USHORT MaximumLength;
     PWSTR Buffer;
}
Now look at D dynamic wchar array
struct {
   uint size;
   wchar* ptr;
}

Do you see the resemblance? Both are counted strings and as such
do not need trailing zero. Both are happy with embedded zeroes.
Both use UTF-16. D IS the best choice for the job.
The compiler is flawless! The Optlink is the guardian sitting
between me and my eternal happiness...
And if there is a statement in my post that can be interpreted
as "D is a completely useless language in all areas of development"
then I really regret for the choice of words.
All I state is that Dmd is very hard to be used as system programming
language.

Thanks for your profound replay, I hope I made myself more clear
on what my problems are.

Regards,
Todor
Dec 28 2005
next sibling parent reply Lars Ivar Igesund <larsivar igesund.net> writes:
I'm just curious here: Did you try to dynamically load ntdll.dll using
std.loader? I know this module isn't currently in the documentation (it
hasn't been actively mantained for a while), but it should be compiled into
phobos. At least it is included with the source. It is just a wrapper for
LoadLibrary, but I've used it to load windows functions that is too new for
the definitions and libs provided by Digital Mars. If this works for you, I
know that it isn't your only missing link...

Lars Ivar
Dec 28 2005
parent reply "Todor Totev" <umbra.tenebris list.ru> writes:
 I'm just curious here: Did you try to dynamically load ntdll.dll using=



 std.loader? At least it is included with the source. It is just a  =



 wrapper for
 LoadLibrary, but I've used it to load windows functions that is too ne=


w  =


 for the definitions and libs provided by Digital Mars. If this works f=


or  =


 you, I know that it isn't your only missing link...

 Lars Ivar


No but I know that it will work just fine for my example.
But I think it's a short-term solution - dynamic loading is for function=
s
that present only in some versions of OS or functions you may not always=
  =

need.

Regards,
Todor
Dec 28 2005
parent reply Carlos Santander <csantander619 gmail.com> writes:
Todor Totev escribió:

 I'm just curious here: Did you try to dynamically load ntdll.dll using
 std.loader? At least it is included with the source. It is just a  
 wrapper for
 LoadLibrary, but I've used it to load windows functions that is too 
 new  for the definitions and libs provided by Digital Mars. If this 
 works for  you, I know that it isn't your only missing link...

 Lars Ivar




I tried that before you posted, and it worked as far as I can tell. Ok, it 
doesn't really work but that's because I don't know what NtOpenFile does or how 
it works, etc., so that's where I stopped. It took me less than half an hour. I 
attached a modified public_test.d. You have to compile with win32\winnt.d and 
win32\winbase.d. Also, guess what: nothing more than dmd and (opt)link.


 
 No but I know that it will work just fine for my example.
 But I think it's a short-term solution - dynamic loading is for functions


(I guess Thunderbird didn't download the message correctly)

You must know more than I do about Windows programming, but let me point you at 
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/devnotes/win
rog/ntopenfile.asp, 
which is the MSDN documentation for NtOpenFile. That document links to 
http://msdn.microsoft.com/library/en-us/devnotes/winprog/callin
_internal_apis.asp, 
which says "There is no associated import library, so developers must use 
run-time dynamic linking to call the functions described in this header file", 
and later "you can access them through run-time dynamic linking using 
LoadLibrary and GetProcAddress", which is what Lars suggested and what I did.

--

Don't get me wrong: I understand your problems, but I think 5 hours was a lot 
more than what you really needed. You mentioned problems with the DM libraries 
being old, and you're right. That has been brought up quite some times. But 
there's a practical and useful solution for that. Doing so, also makes your 
linking problems void. And most (if not all) of your problems with Core32 are 
solved by NOT importing std.c.windows.windows. Debugging is a problem too, but 
WinDbg does a fairly good job, and you could've read Wiki4D and find out more 
about debugging D programs.

Anyway, I hope this particular bad experience doesn't turn you away from D.


-- 
Carlos Santander Bernal
Dec 28 2005
next sibling parent Carlos Santander <csantander619 gmail.com> writes:
Carlos Santander escribió:

 
 I tried that before you posted, and it worked as far as I can tell. Ok, 
 it doesn't really work but that's because I don't know what NtOpenFile 
 does or how it works, etc., so that's where I stopped. It took me less 
 than half an hour. I attached a modified public_test.d. You have to 
 compile with win32\winnt.d and win32\winbase.d. Also, guess what: 
 nothing more than dmd and (opt)link.
 


I forgot the attachment. Here it goes now.

-- 
Carlos Santander Bernal
Dec 28 2005
prev sibling parent reply "Todor Totev" <umbra.tenebris list.ru> writes:
On Thu, 29 Dec 2005 02:55:34 +0200, Carlos Santander  =

<csantander619 gmail.com> wrote:

 I tried that before you posted, and it worked as far as I can tell. Ok=


,  =


 it doesn't really work but that's because I don't know what NtOpenFile=


  =


 does or how it works, etc., so that's where I stopped. It took me less=


  =


 than half an hour. I attached a modified public_test.d. You have to  =



 compile with win32\winnt.d and win32\winbase.d. Also, guess what:  =



 nothing more than dmd and (opt)link.


Carlos, I tried to present a problem where I have a real-world problem a=
nd
a valid programme to solve it which is not well handled by the existing
Dmd infrastructure.
To solve the problem I changed the infrastructure to match my programme.=

And it takes bloody ages.
On the other hand, you change the programme to match the infrastructure.=

I admit the way you walk is the easier one.
My personal opinion is that both solutions work and have their drawbacks=
 as
both are not the solutions of the problem but patches to it.

Also the code I provided was intended to prove that debugger is needed.
Before I had posted it I rolled back to a previous vesion that contains
a tiny mistake and mischievously removed the existing writelfn line
that can easily reveal the wrong code.
What you think is NtOpenFile problem is actually a human problem
left to demonstrate that a debugger is needed.


 http://msdn.microsoft.com/library/default.asp?url=3D/library/en-us/dev=


notes/winprog/ntopenfile.asp,  =


 which is the MSDN documentation for NtOpenFile. That document links to=


  =


 http://msdn.microsoft.com/library/en-us/devnotes/winprog/calling_inter=


nal_apis.asp,

Both links are 404 now


 which says "There is no associated import library, so developers must =


 =


 use run-time dynamic linking to call the functions described in this  =



 header file", and later "you can access them through run-time dynamic =


 =


 linking using LoadLibrary and GetProcAddress", which is what Lars  =



 suggested and what I did.


Either Microsoft are so big that MSDN department does not know what OS
department do or MS programmers do not read MSDN.
I have 5 copies of ntdll.lib on my hard (4 from the Micriosoft supplied =
DKK
and one from OpenWatcom compiler). I can only think that MS does not wan=
t
a common application developer to deal with that dll.
Also the Win32 WMI provider implemented in cimwin32.dll, which is a norm=
al
user-mode dll statically links to ntdll.dll so I have an example to foll=
ow.



 ...problems with Core32 are solved by NOT importing  =



 std.c.windows.windows.


Core32 is intended to complement std.windows as I was made to believe.
Actually this is my fault but is counter-intuitive for me.


 Debugging is a problem too, but WinDbg does a fairly good job, and you=


  =


 could've read Wiki4D and find out more about debugging D programs.


The most peculiar thing with Wiki4D is that I use the UltraEdit
syntax highlighter from it. I really can't explain why I missed the  =

debugger
related pages. Thanks for the tip.


 Anyway, I hope this particular bad experience doesn't turn you away fr=


om  =


 D.


You are kidding! My little programme is working, I really like how D sid=
e
of it interacts with Windows Native side of it. I have plans to improve =
it
and I'm currently downloading my copy of VisualStudio Express which I  =

intend
to convert to dedicated D debugger.

I still miss the ability to conviniently use existing lib files but will=

look at build and try to patch it so it starts speaking the language of
Watcom's wlink.

Thanks for the replay, it helped me alot.
Todor
Dec 29 2005
next sibling parent reply "Walter Bright" <newshound digitalmars.com> writes:
"Todor Totev" <umbra.tenebris list.ru> wrote in message 
news:op.s2jo0pu7ihwmk4 todor-1-xp.sanbolic.local...

Carlos, I tried to present a problem where I have a real-world problem and
a valid programme to solve it which is not well handled by the existing
Dmd infrastructure.


I appreciate your long and detailed post illustrating the difficulties 
you're having, and how you got around them. Your concerns are real, and we 
need to address them.


 I really can't explain why I missed the debugger related pages.


The problem with newer Windbg's from Microsoft is that it can no longer read 
Microsoft codeview debug information. The old Windbg does work with codeview 
debug data, it does see local variables in D and D class members. It comes 
on the Digital Mars CD.


I still miss the ability to conviniently use existing lib files but will
look at build and try to patch it so it starts speaking the language of
Watcom's wlink.


Writing a utility to convert Microsoft's dll import libraries to a module 
definition file, which can then be fed through implib to create an OMF 
import library, wouldn't be hard. It just takes someone to sit down and do 
it. Want to take a stab at it?
Dec 29 2005
next sibling parent d d.com writes:
In article <dp1ce3$m64$1 digitaldaemon.com>, Walter Bright says...

The problem with newer Windbg's from Microsoft is that it can no longer read 
Microsoft codeview debug information. The old Windbg does work with codeview 
debug data, it does see local variables in D and D class members. It comes 
on the Digital Mars CD.


Why not put the "old Windbg" on you D web-site for download? -- because of some
license issue?

Or let us know which old version of WinDbg works with D, maybe we can find it
somewhere on the web.
Dec 29 2005
prev sibling parent Kyle Furlong <kylefurlong gmail.com> writes:
Walter Bright wrote:

 "Todor Totev" <umbra.tenebris list.ru> wrote in message 
 news:op.s2jo0pu7ihwmk4 todor-1-xp.sanbolic.local...
 

Carlos, I tried to present a problem where I have a real-world problem and
a valid programme to solve it which is not well handled by the existing
Dmd infrastructure.


 
 
 I appreciate your long and detailed post illustrating the difficulties 
 you're having, and how you got around them. Your concerns are real, and we 
 need to address them.
 
 

I really can't explain why I missed the debugger related pages.


 
 
 The problem with newer Windbg's from Microsoft is that it can no longer read 
 Microsoft codeview debug information. The old Windbg does work with codeview 
 debug data, it does see local variables in D and D class members. It comes 
 on the Digital Mars CD.
 
 

I still miss the ability to conviniently use existing lib files but will
look at build and try to patch it so it starts speaking the language of
Watcom's wlink.


 
 
 Writing a utility to convert Microsoft's dll import libraries to a module 
 definition file, which can then be fed through implib to create an OMF 
 import library, wouldn't be hard. It just takes someone to sit down and do 
 it. Want to take a stab at it?
 
 
 


Eric's DDL is the place to start probably. Check it out at dsource.org
Dec 30 2005
prev sibling parent Carlos Santander <csantander619 gmail.com> writes:
Todor Totev escribió:

 
 Carlos, I tried to present a problem where I have a real-world problem and
 a valid programme to solve it which is not well handled by the existing
 Dmd infrastructure.
 To solve the problem I changed the infrastructure to match my programme.
 And it takes bloody ages.
 On the other hand, you change the programme to match the infrastructure.


I don't understand how's that. Could you elaborate, please?


 I admit the way you walk is the easier one.
 My personal opinion is that both solutions work and have their drawbacks as
 both are not the solutions of the problem but patches to it.
 
 Also the code I provided was intended to prove that debugger is needed.
 Before I had posted it I rolled back to a previous vesion that contains
 a tiny mistake and mischievously removed the existing writelfn line
 that can easily reveal the wrong code.
 What you think is NtOpenFile problem is actually a human problem
 left to demonstrate that a debugger is needed.
 


I said I didn't know what NtOpenFile does, and since you said something about 
modifying boot.ini, when the program failed, I said "great" ;) That's why I 
didn't try harder. Oh, and I was at work.


 http://msdn.microsoft.com/library/default.asp?url=/library/en-us/devnotes/win
rog/ntopenfile.asp,  
 which is the MSDN documentation for NtOpenFile. That document links 
 to  
 http://msdn.microsoft.com/library/en-us/devnotes/winprog/callin
_internal_apis.asp, 


 
 
 Both links are 404 now


Sorry about that. Here's another try: 
http://msdn.microsoft.com/library/en-us/devnotes/winprog/ntopen
ile.asp?frame=true, 
and 
http://msdn.microsoft.com/library/en-us/devnotes/winprog/calling_internal_apis.asp


 

 which says "There is no associated import library, so developers must  
 use run-time dynamic linking to call the functions described in this  
 header file", and later "you can access them through run-time dynamic  
 linking using LoadLibrary and GetProcAddress", which is what Lars  
 suggested and what I did.


 
 
 Either Microsoft are so big that MSDN department does not know what OS
 department do or MS programmers do not read MSDN.
 I have 5 copies of ntdll.lib on my hard (4 from the Micriosoft supplied DKK
 and one from OpenWatcom compiler). I can only think that MS does not want
 a common application developer to deal with that dll.
 Also the Win32 WMI provider implemented in cimwin32.dll, which is a normal
 user-mode dll statically links to ntdll.dll so I have an example to follow.
 
 

 ...problems with Core32 are solved by NOT importing  
 std.c.windows.windows.


 
 
 Core32 is intended to complement std.windows as I was made to believe.
 Actually this is my fault but is counter-intuitive for me.
 


I didn't know that either, I just saw conflicts between std.c.windows.windows 
and Core32, so I tried not importing std.c.windows.windows, and it worked. Then 
other symbols were missing, so I added more Core32 modules.


 Debugging is a problem too, but WinDbg does a fairly good job, and 
 you  could've read Wiki4D and find out more about debugging D programs.


 
 The most peculiar thing with Wiki4D is that I use the UltraEdit
 syntax highlighter from it. I really can't explain why I missed the  
 debugger
 related pages. Thanks for the tip.
 

 Anyway, I hope this particular bad experience doesn't turn you away 
 from  D.


 
 
 You are kidding! My little programme is working, I really like how D side
 of it interacts with Windows Native side of it. I have plans to improve it
 and I'm currently downloading my copy of VisualStudio Express which I  
 intend
 to convert to dedicated D debugger.
 


Good!


 I still miss the ability to conviniently use existing lib files but will
 look at build and try to patch it so it starts speaking the language of
 Watcom's wlink.
 
 Thanks for the replay, it helped me alot.
 Todor


You're welcome.

-- 
Carlos Santander Bernal
Dec 29 2005
prev sibling next sibling parent James Dunne <james.jdunne gmail.com> writes:
Todor Totev wrote:

   Hello all,
 after I had read your oppinions I decided to make myself as clear as
 possible.


Cool.


 First, I posted after I successfully compiled, linked & executed my test
 programme. Second, from the beginning I was trying to prove that D is
 excellent as a system programming language. Not application programming
 but system programming.


I don't think it (proving that D is an excellent system programming 
language) came across this way... Sounded like a long-winded rant to me. 
  *sigh*.


 
 According to Wikipedia (http://en.wikipedia.org/wiki/System_programming):
 ...the programmer will make assumptions about the hardware and other
 properties of the system that the program runs on, and will often exploit
 those properties (for example by using an algorithm that is known to be
 efficient when used with specific hardware.
 

 If you're referring to your "Problem with COFF library" post, then I'd


 
 
 As I stated, I published my thoughts after I found solution of all  
 engineering
 problems I encounter. The solution is within article
 http://www.digitalmars.com/d/archives/digitalmars/D/9911.html.
 I just didn't realized this at the time I posted that request.
 


Don't you wish there was an 'undo post' feature? =P  Well, not really. 
It's good to post your experience on newsgroups that others may learn 
from your mistakes.  It should follow naturally that you should reply to 
your post stating that you solved your problem and how you went about 
it, but it's not required.  I like everything to be out in the open. =)


 

 You don't need to link your Python or Ruby executable to kernel32.lib 
 or  gdi32.lib, because the interpreter that runs your Python/Ruby 
 program is  already linked to those.


 
 Actually every single process in Windows have access to ntdll.dll
 simply because this library is mapped in the process address space even
 before the exe file is being read - actually a code inside ntdll is
 responsible for starting the process :-) No excuse for D.
 


Ah yes, countless debugging sessions have taught me this knowledge as 
well, just plum forgot about it =).  But, for D to have *access* to 
those ntdll.dll functions, it must declare function prototypes for every 
function in the dll and I guess nobody got around to doing that yet (nor 
completing the Windows headers translations).


 Want to easily convert a .h file to .d? Yes of course! Not with D.




 
 
 I really regret for my particular choise of words.
 What I really complain is that Phobos only partially defines very basic
 Windows types AND in the same time Core32 which is supposed to 
 complement  it
 actually clashes with it. So I had to resolve both issues.
 Another problem is that I can patch these but a long-term resolution is to
 fix them inside phobos & core32.
 Of course, I'll send what I discovered to Walter and maintainers of core32.
 


I understand.


        Plain old, boring Windows 2000/XP functions (5 years old, 
 mind  this)
        are nowhere to be seen. Functions like FindFirstVolumeMountPoint
        or GetVolumePathNamesForVolumeName.
   2.3. No out-of-box support for lib files coming with SDK/DDK/IFS/WDK.
   2.4. No implib functionality so I cannot create up-to date lib files
        from the dll files.




 
 
 Again I want to stress that these issues make system programming with D
 near to impossible.
 


It seems to be that Walter is concentrating mostly on the design and 
implementation of his language, rather than the details of platform 
compatibility and standard library development.  Many have complained 
about the latter, but few have stepped up to actually do something about 
it.  I applaud those few (no need to name names).

Now, I think you are an excellent candidate for solving some of these 
problems.  Why not apply your experience working with the Windows APIs 
and convert some headers for us all?  It's a one time task that everyone 
can benefit from.

As for me, I don't like to rely on OS-specific behavior and prefer the 
cross-platform approach to develop my software, which is why I like the 
phobos approach.


 If you didn't know, the SDK, DDK, IFS, WDK, whatever from Microsoft 
 are  all proprietary development kits which must be purchased and 
 licensed at  hefty costs.  I'm no legal expert, but knowing Microsoft 
 there is  probably a high barrier to entry for compilers wanting to 
 fully support  the Windows platforms.


 
 
 I'm the person responsible for MSDN subscription in our firm.
 SDK costs 0$ - it is freely available for download.
 DDK is free for MSDN subscribers.
 IFS costs 900$ but nearly noone needs it.
 MSDN subscription does cost some money but I cannot imagine a serious
 development for Windows platform without this.
 


Sure, for us we can just grab a MSDN subscription, but I was leaning 
more towards the compiler-vendor side of things.  Like I said before, I 
don't know much about this area at all and I'm just assuming things are 
different for this market.


 I certainly wouldn't expect, let alone demand, from Walter that he 
 shell  out cash in the neighborhood of 5-6 figures to Microsoft to get 
 such  support for his compiler.


 
 Neither I. Walter provides compiler & linker. He surelly does not need
 DDK, unless he decides to prove that compiler can be written as a device
 driver (humourous note).


=)

Yes, I know that might've come off wrong, but I wasn't assuming you were 
expecting so either.  I meant the cost in terms of the MSDN subscription 
and/or license-for-redistribution fees.


 It is I who need to bridge D & DDK. And I do have the DDK.
 The problem is that I cannot use it with D because optlink does not
 understands COFF libraries.
 Also, OpenWatcom does understand COFF files in general and MS supplied
 ones in particular out of the box. While Optlink can't?
 


The COFF2OMF really doesn't do the trick here?  I've never used it 
myself, so I'm just asking for future reference.  Argh, I don't like 
when tools do half the job.


  * is not available free if I want to use the digital mars one



 Digital Mars has to make their income somehow.  The utility CD is 
 $15.  Sacrifice a night out at the movies (which is most likely to be 

 $15)  and grab your copy.  This is incomparable to what Microsoft 


 would charge  you for their compiler toolset.


 
 
 The reason I don't wnat to purchase this cd is:
 * I do not have credit card.
 * Things tend to dissapear in Bulgarian customs - i.e. last month
 there was a big scandal because two customs officers have stollen the
 cell phone of the american ambassador while he was checked with metal  
 detector.
 I still wait for my Ubuntu cds. Although a colleague of mine received his
 DVDs so apparently this is related to man's luck.
 


Wow, that sucks.  Anything I can do to help?  I'm in the U.S. so I can 
probably purchase an extra copy for you and send you a digital image via 
secure transmission.








 
 Because I must translate c .h files to [DllImport] attributes
 and make some structs. Basically exactly what I did to make D source.
 


Ah okay.


   And actually that cannot be called "productive". I now must 
 support  three
 separate entities: D compiler, DigitalMars C compiler and 
 OpenWatcom   linker.
 And of course, for every project I start from now, I must build a 
 bat  file
 and linker response file tailored for that project. No build.exe for me.
 No simple dmd *.d my.lib my2.lib. Nothing like this.



 Derek Parnell's 'build' utility (available also on dsource.org) will 
 get  you headed in the right direction here.


 
 
 According to the site, build does not know how to handle wlink.
 And my patchy solution relies on OpenWatcom's wlink.
 Besides, it is very interesting that on my complain
 "No build.exe for me." you responded with "Use build".
 


Hehehe, sorry about that.  I wasn't sure what 'build.exe' you were 
referring to.

I'm sure it wouldn't be too tough to patch the build source to handle 
wlink compatibility.  Perhaps another area for your experience to come 
in to help here?


 Ok, my programme is linked and I have an exe file. But as my ex-boss
 likes to say: "No-one should commit code in the source control system
 unless she had stepped through every one of the new code in the  
 debugger."



 While your ex-boss may be a wise man stating this, it shouldn't be a  
 strict requirement that all code must be stepped-through in a 
 debugger  in order to be bug-free.  True, it may catch many uncaught 
 bugs, but  this is what (and you realize this) the tools the language 
 provides to  you should help to eliminate.


 
 Please make a multithreaded windows service that runs under LocalSystem  
 account
 and synchronizes itself with another survice that runs under user account
 right from the first time without using the debugger.
 After that write a book about how did you achieved this and I will buy it.
 I realy do things like thouse to earn my money so I'm touchy on that  
 subject.


I figured you might come back with a multi-threaded example.  Yes, that 
stuff is tricky.

Regarding the source control commit, IMO it's still a good idea to 
commit current working code just so you have a backup somewhere and can 
easily revert changes.  Of course, the granularity of commits is 
completely up to the developer and you only get out what you put in.


 3. Where is the debugger?
 No near me, for sure.
 I hear "WinDbg" screamed, am I not?



 First off, again you're assuming Windows platform here.  GDC and GDB 
 are  perfectly acceptable alternatives.


 
 
 Sure, but I need Windows 2000. If GDC can handle this, why not?
 Remember, I am system programmer doing system programming.
 

 First, it does not come with D/dmd.



 Nor should it, because it is not a Digital Mars product.


 
 
 But comes with Utility CD? Pardon me?
 


I believe that version contained on the utility CD is an old unsupported 
version for which nobody else distributes anymore.  Again, I could be wrong.


 Second, you must find it in the Microsoft site - not an easy task.



 This is Microsoft's fault alone.  Personally, I think it isn't all 
 that  hard to find.  Your milage may (and did) vary, of course.


 Third, half an year earlier when WinDbg out of the blue got
 the docking windows and user interface that made it actually useful
 some of my colleague go out, bought some beer and celebrated it.



 Slapping a GUI onto a tool does not instantly make it "actually 
 useful".    It *might* make it more user-friendly and manageable, but 
 the old tool  was just as useful.


 
 All these points are valid but did not change the thing that debugger
 is badly needed by the programmer and D does not have a easy to 
 discover  one.
 


A valid point, but I wasn't commenting on that fact. =)


 Fourth, that WinDbg which is quite user-unfriendly does not work with D.


 If you mean that it doesn't work with D in that you can't peek at 
 local  variables' contents, then I should state that this is a problem 
 with  Microsoft's tools sporting their proprietary PDB file format.


 
 In the example I provided in another post 100% of programme's state is  
 inside
 local variables. Debugger that hides 100% of program state is not 
 useful  one.
 Of course this is Microsoft problem, as I admitted at end of my post.
 


Yes.  It's a shame.


   What other languages offer? Lua, Python & Ruby have built-in
 debugging facilities. VisualStudio have a debugger which can be extend
 to show your data the way you want it. The dotNet SDK comes with a
 debugger (and an excellent one). D simply does not have working  
 debugger.


 None of these languages are 'systems programming' languages.


 
 I use VisualStudio C++ to develop device drivers. Are you sure that
 this do not count as system language?


I didn't notice the VisualStudio in there when I replied earlier.  I was 
mainly replying about Lua, Python & Ruby here.  Yes, of course C/C++ is 
a systems programming language. =)



 As its sole purpose is to tweak the driver's behaviour it is indeed



I don't agree here.  My school of thought is that if the language alone 
can be used to develop an operating system with then I consider it a 
systems programming language.  By your inductive logic here, one could 
technically consider any language which gives you interop abilities to 
the OS a system programming language.

IMO, D is a systems programming language because 1) it compiles to 
native code, 2) you can write an operating system with it, and 3) you 
can interface to other operating systems with it without special interop 
abilities needed.


 
 I wnated to state that every language must have a built-in debugging  
 facilities
 to be useful. Actually the selling point of D is its debugging features.
 Also both OpenWatcom and free Borland tools come with debugger.
 And they are C++ compilers.
 

 They work through bytecode and interpretation (or JIT compilation) 
 which  make it significantly easier to debug and step-through.


 
 
 Not for the way I use C++.
 


Again, I was referring to Lua, Python & Ruby here.  Sorry for the C++ 
blanket cover.


 I believe a D debugger project is in the works on dsource.org (dbug?)  
 but I haven't tried it myself.


 
 The current state of dbug is:
 http://www.dsource.org/forums/viewtopic.php?t=1009
 "you can set breakpoints by clicking on the leftmost margin in
 the source windows - removing breakpoints isnt finished and its
 getting an exception trying to resume from a breakpoint"
 Not useful to me.
 


Apparently not; good of you to check though.


 D is assuming a solid understanding of C and [C++ or OOP in general] 
 as  a start for learning D.


 
 
 Completely wrong. D requires from you to learn D and not C/C++.
 What need of C you have to learn OOP or template metaprogramming?
 Do the messy RTTI of C++ helps you to work with D's one?
 Probably the fact that C does not know how large is int really
 helps you to learn using the base types of D?
 Et cetera, et cetera.
 


What I meant by a 'solid understanding' of C and [C++ or OOP in general] 
was the flow and structure of C programs (expressions, statements, etc.) 
combined with the OOP paradigm.  Of course one does not have to learn 
RTTI of C++ in order to learn D.


 
 While a book would be great for the
 

 business-oriented newbie who is being paid to be trained in the  
 language, I believe there is no substitue for raw experience gained  
 while tinkering with the language.


 
 Never been in paid to be trained in a language and can't talk for this.
 The books help the programmer to get this experience so there is no
 substitusion for them. I tried to learn how dotNet framework works
 reading MSDN and I understood nothing. I read a book for newbies
 and learned how to ask questions and where to look for answers
 (I mean that I learned about System.Runtime.InteropServices,
 System.Management etc namespaces and was able to figure out in MSDN
 what to do to achieve my goals).

 Once you have the basics down, go ahead and read the reference books/docs
 to find out what you may have missed in your tinkering sessions.


 
 Completely opposite: Please go to the site of Lua and check the 
 reference  manual
 (what D have) and Programming in Lua book. You can decide for yourself what
 you want to use to begin learning a new language - the interpreter, the  
 manual
 or the book.
 


Everyone learns differently; I was just stating how I learn for a 
counter-example of why a D book is needed.  I should've pointed that out 
in my statement.  Self-taught is my way to go. =)


  The requirements for 3rd party tools (namely Microsoft linker,  
 OpenWatcom suite) and 4th party tools (the patched wlink for DMC) 
 make  the situation as bad as possible.


 These are not requirements for regular development.


 
 My regular development includes inter-operating with the OS at low levels.
 I cannot write a notepad-type application without some serious help
 or a framework but I can write a basic WMI provider for less than a hour.
 YMMV.
 


I meant "regular development" in regular user-mode software that isn't 
strictly dependent on OS APIs or external linkers to get the job done. =)


  * Provide low level bare metal access as required - it is not even  
 near,
  as I really need access to ntdll.dll (the lowest level to bare metal
  in user mode available) and D actively hinders me.


 Bare-metal access does not mean access to operating system libraries. 
 It  means that the code will run on the native processor without 
 requiring  an interpreter or JIT compiler.


 
 Obvously you dont know that Win32 API is the ultimate virtual machine.
 It is there to shield the programmer from the NT and presents her a
 comfortable, compatible with 16 bit Windows, API.
 If you escape its confines you can create files with names that
 contains embedded zeroes, or that differs only by case of letters, etc.
 Most of the functions inside kernel32/advapi32 either do the work
 without going to the OS level or simply convert ASCIIZ to counted strings
 and ANSI to UNICODE and gives them to the real OS to process them.
 This looks like a tipical behaviour of a VM to me.
 


Now I understand what you mean.  Okay, you're speaking of the separation 
between the "high-level" Win32 API and the "low-level" functions present 
in NTDLL.DLL.  Still, I wouldn't consider this "bare-metal" access.  I 
would consider this "low-level OS access", i.e. as close to kernel mode 
programming as you can get in user mode.


  * Be compatible with the local C application binary interface -  
 Microsoft tools are de facto standard for Windows platform andD does  
 not talk to them.



 Microsoft tools were made to force a de facto standard on the Windows  
 platform because they use and integrate so well their own proprietary  
 technologies to keep others out.


 
 
 Dmd is a proprietary technology so you cannot blame MS doing the same  
 thing.
 


??  DMD uses open standards.  I was referring to the 'de facto standard' 
of their own PE-COFF and PDB formats that their tools use.  Sorry for 
the confusion.


 Lets give you a hipotetical example. You develop a text editor that
 implements "search in files" functionality.
 Look at this code:
 
 import std.recls;
 import std.stdio;
 
 void main()
 {
   Search s = new Search(null, "*",
     RECLS_FLAG.RECLS_F_RECURSIVE|RECLS_FLAG.RECLS_F_DIRECTORIES);
   foreach (Entry e; s)
     writefln(e.GetPath());
 }
 
 Do you see the problem?
 Alice as an advanced user had made hard links in her file system.
 But the problem is that one of the hard link goes UP in the file system
 thus if blindly followed forms a never-ending cycle.
 One can recreate the Alice setup using a tool from
 http://www.sysinternals.com/utilities/junction.html
 You can do this yourself and enjoy the show.
 After that come back and tell again that you do not need to
 inter-operate with your OS.
 


Not sure where this came from, but yes I do see the problem. =)


 If you had made a case developing for Linux using either the Linux  
 version of DMD or the GDC front-end for GCC, I'm sure your experience  
 would have been much more enjoyable.


 
 Windows 2000. It's what concerns me.
 


Have you tried GDC on cygwin yet?  I'd be interested to see how the 
cross-compilation tools that GCC provides work with Windows at the low 
levels you're working at.


 To be fair, use the tools that are right for the job.  D might not be  
 right for your current project, at the current time.  Don't  
 overgeneralize to everyone else stating that D is a completely 
 useless  language in all areas of development.


 Consider the native string type:
 struct UNICODE_STRING {
     USHORT Length;
     USHORT MaximumLength;
     PWSTR Buffer;
 }
 Now look at D dynamic wchar array
 struct {
   uint size;
   wchar* ptr;
 }
 
 Do you see the resemblance? Both are counted strings and as such
 do not need trailing zero. Both are happy with embedded zeroes.
 Both use UTF-16. D IS the best choice for the job.
 The compiler is flawless! The Optlink is the guardian sitting
 between me and my eternal happiness...


How does a D to C translator sound to you?


 And if there is a statement in my post that can be interpreted
 as "D is a completely useless language in all areas of development"
 then I really regret for the choice of words.
 All I state is that Dmd is very hard to be used as system programming
 language.


Sorry for the assumption, but it sounded like you were coming down on D 
in a long-winded rant.  I would say the definition of "systems 
programming language" means different things to different people.


 
 Thanks for your profound replay, I hope I made myself more clear
 on what my problems are.
 
 Regards,
 Todor
Dec 28 2005
prev sibling parent J C Calvarese <technocrat7 gmail.com> writes:
In article <op.s2h92jx8ihwmk4 todor-1-xp.sanbolic.local>, Todor Totev says...

I really regret for my particular choise of words.
What I really complain is that Phobos only partially defines very basic
Windows types AND in the same time Core32 which is supposed to complement

it actually clashes with it. So I had to resolve both issues.
Another problem is that I can patch these but a long-term resolution is
to fix them inside phobos & core32.

Of course, I'll send what I discovered to Walter and maintainers of core=
32.


You're right: Core32 is meant to complement Phobos. We've tried to add
"version(STANDALONE)" to all of the statements that are also in Phobos. DMD
doesn't complain about conflicts until they are actually mentioned in code that
uses both. Some of the conflicts were found by tediously comparing std.c.windows
to the Core32 files, and other conflicts were found by "blah conflicts with
blah" error statements from DMD. I guess if I were smart I'd come up with some
sort of automated tool to find all of the conflicts once and for all, but for
now I just version out the conflicts when I become aware of them.

As the guy who maintains Core32, I'd appreciate it if you could help me fix any
of the conflicts. Either a new topic in the forum
(http://www.dsource.org/forums/viewforum.php?f=16) with the changes or an e-mail
with the changed file should do the trick. 

I'll also add any definitions that are missing if it's spelled out for me. I
know Core32 is out-of-date and incomplete. Mike Wynn did the original
conversions from Microsoft's headers, and I don't have the time or talent to
convert a bunch of new headers myself. But I'll accept contributions.

jcc7
Dec 28 2005
prev sibling next sibling parent Hasan Aljudy <hasan.aljudy gmail.com> writes:
Excellent / well though-out post. I think your concerns are very legitimate.

I have absolutely no experience in your situation, but I encountered the 
same kind of problems when I tried to use Linux as a desktop 
environment. (i.e. endless compatibility issues.)
Linux seems to have developed a culture which does not care about these 
details and expects the users to waste their time and solve them (see 
some of the replies to your post).

One thing though, the problem of not finding a linker can be solved by 
downloading the dmc.zip file from the same page where you found the 
compiler.




Todor Totev wrote:

 
 
 Hello all,
 what started as a simple request for help growed up to some kind of essay.
 Here it is:
 
 Why D is not for me?
 The scene of this little opera:
 I'm working for a small firm which products use kernel modules for Windows.
 Part of our tests are built-in in the modules themselves but others
 are user-mode programmes. But it is quite common to go one level lower
 and use the so called "Native API" to do some tricks.
 My boss told me that I need to made a small test programme that exercises
 a obscure pattern of behaviour of our product. I thought: this is  
 relatively
 a small programme that is not top priority but nevertheless is useful and
 with clear goal. A right test for what can D do for me.
 And my quest started.
 
 I've read the information on the site, I was eager to start and I  
 downloaded
 dmd.zip. I wrote my first programme which uses Native API.
 And now is the right time for my first obstacle. As in any
 self-respecting RPG quest, it is an easy task:
 
 1. The official distribution of D is the only language distribution 
 seen  by me
 that is not self sufficient.
   1.1. no linker - no linker? How am I supposed to build my programmes?
   1.2. no libraries - consider the default command line that dmd builds
        for optlink - kernel32+gdi32.
        Where is kernel32.lib? Where is gdi32.lib?
 
 Please, compare dmd distribution to Python or Ruby ones.
   Ok, I'm not afraid of some digging and I actually read the documentation,
 so I downloaded dmc package and installed it. The miracle happened and now
 I have kernel32.lib. Which leads me to:
 
 2. D is the only language which aims at system programming and in the same
 time cannot inter-operate with the operating system.
   2.1. No easy support for importing Windows types & functions not
        included by default within Phobos.
 
 Want to easily convert a .h file to .d? Yes of course! Not with D.
 Because it does not define all basic types so you can go straight to
 missing ones. So what, use Core32 from dsource.org! But one must first
 find that this project exists! And then, hurray, it conflicts with Phobos!
 Good, now I must find out what to comment out. Five minutes later i'm done
 with this task.
 
   2.2. The libraries included are old. Outdated. Unusable to me. And I even
        didn't try to use Transactional NTFS functions from Vista beta  
 WDK/SDK.
        Plain old, boring Windows 2000/XP functions (5 years old, mind this)
        are nowhere to be seen. Functions like FindFirstVolumeMountPoint
        or GetVolumePathNamesForVolumeName.
   2.3. No out-of-box support for lib files coming with SDK/DDK/IFS/WDK.
   2.4. No implib functionality so I cannot create up-to date lib files
        from the dll files.
 
 I dig some more and found that there is coff2omf utility, which:
  * produces incompatible lib files, if I want to use the available 
 Borland  one
  * is not available free if I want to use the digital mars one
  * actually do not work because it requires special version of
  <b>Microsoft</b> link.exe that cannot be found anywhere.
  So to use Digital Mars product, I must pay to Microsoft and pray
  that I can found their very old linker somewhere in MSDN archives?
 I dig some more and found the special version of OpenWatcom wlink exe
 that can do the trick. And indeed, it did the trick.
 After about 5 hours fight with it, I was able to spell the right  
 incantation
 that linked my 20 lines of D code.
 That includes special order of library paths so that Watcom's kernel32 is
 first and special order of libraries so that snn comes first.
 Really, a good job for starter in a completely new language and  
 environment!
 D made me feel proud of myself. And I lost 5 hours but now I know why
 most people think programming is some kind of wizardry.
 If I opted for MS C++ I will be ready in less than a minute.

 tests

   And actually that cannot be called "productive". I now must support three
 separate entities: D compiler, DigitalMars C compiler and OpenWatcom  
 linker.
 And of course, for every project I start from now, I must build a bat file
 and linker response file tailored for that project. No build.exe for me.
 No simple dmd *.d my.lib my2.lib. Nothing like this.
 
 Ok, my programme is linked and I have an exe file. But as my ex-boss
 likes to say: "No-one should commit code in the source control system
 unless she had stepped through every one of the new code in the debugger."
 I know that all the tools of D language & infrastructure are there to
 help me spend as little time in the debugger as possible.
 Tools like contracts, unit tests, code coverage etc.
 But only the debugger can tell you the flow of code.
 
 3. Where is the debugger?
 No near me, for sure.
 I hear "WinDbg" screamed, am I not?
 First, it does not come with D/dmd.
 Second, you must find it in the Microsoft site - not an easy task.
 Third, half an year earlier when WinDbg out of the blue got
 the docking windows and user interface that made it actually useful
 some of my colleague go out, bought some beer and celebrated it.
 Fourth, that WinDbg which is quite user-unfriendly does not work with D.
 There are anecdotal evidence that version of WinDbg developed circa 1996
 can be used but I'm not sure I want to touch this beast.
   What other languages offer? Lua, Python & Ruby have built-in
 debugging facilities. VisualStudio have a debugger which can be extend
 to show your data the way you want it. The dotNet SDK comes with a
 debugger (and an excellent one). D simply does not have working debugger.
 
 To proceed with other problems, I will leave our beautiful scene
 and go back in time to February 2005. In that time I had to do another
 little project. This time I was choosing between 3 languages:
 D, Lua and Io. I chose Lua because:
 
 4. Documentation
 Lua comes with the best book for a programming language I read so far
 in my life. It is called "Programming in Lua" and is available from
 http://www.lua.org/pil/. A book about a language gives a newbie a base
 to stand. It gave me confidence. D (and Io) do not have this. Languages  
 like
 Python and Ruby come with "Programming Python" and "Programming Ruby" books
 in the zip files you download from their sites. A newbie is lost in the  
 scarce
 documentation of D. "Programming Python/Ruby/Lua" gives you the boost 
 to  start.
 I cannot stress how important is a good book for the newbie.
 Just go to the link I give and read the book. And then compare to D  
 documents.
 
 5. Often changes of the compiler.
 This is a controversial issue. But please consider, how many versions 
 of  dmd
 came from February till now? Even the projects in dsource.org cannot catch
 with them. Core32 collides with current phobos. DFL notes that it is tested
 with version 140 and the author cannot help if you use other version.
 Lua on the other side stays unchanged from 2001. Most of its bugs are
 documented and their solutions are known. Also, the new version that 
 will  be
 out this week is known from half an year. Its new features are known. 
 You  can
 prepare for them (I am currently modifying my code to be ready for it and
 send patches to authors for bugs I encounter). D just announces new  
 features
 and deprecation of old ones and leaves you to adapt.
 I can't build my projects on a base that slips and changes every month.
 I know Walter thinks of setting version 1.0 but that is in the future
 and can't help me now.
 
 
 Consider the "Major goals of D" and compare to what I said earlier:
  * Reduce software development costs by at least 10% - losing 5 hours just
  to link a simple 20 lines programme won't reduce the cost of its  
 development.
  The requirements for 3rd party tools (namely Microsoft linker, OpenWatcom
  suite) and 4th party tools (the patched wlink for DMC) make the situation
  as bad as possible.
  * Provide low level bare metal access as required - it is not even near,
  as I really need access to ntdll.dll (the lowest level to bare metal
  in user mode available) and D actively hinders me.
  * Be compatible with the local C application binary interface - Microsoft
  tools are de facto standard for Windows platform and D does not talk 
 to  them.
 
 
 To conclude, language D is a daydream coming true to every  C/C++ 
 programmer.
 But what D as language gives, D as infrastructure takes tenfolds.
 I admit all the problems I presented are not unresolvable,
 the big problem is that I must resolve or work around them before doing any
 actual work. Also I know about projects like Poseidon or D Forms Library
 which prove that D is useful. All I state is that D is not useful for me
 in its current form. And I think that the problems I have are applicable
 to other people too. Also it is possible that I mis-read the documentation
 and there is an easy way to do what I do and I simply didn't found it.
 But I read as much as I can, searched in the newsgroup archives and
 I think I made the best possible effort to do the things right.
 Last, I know some of the problems are hard to solve.
 Microsoft keeps interface of mspdb??.dll files for themselves so noone can
 easily produce pdb files in the new format. And it is not redistributable.
 I know that writing a book is a demanding task that often requires more  
 than year
 to complete. I thought of creating a module that in its contructor  
 dynamically binds
 to functions in ntdll & write simple wrappers in D. Et cetera.
 I read the newsgroups since version 0.96 or .98 (can't remember exactly).
 I'm a competent Windows developer and if Poseidon turns a good IDE I can
 try to write a decent wrapper around dbghelp.dll to help build a proper
 debugger. But now I feel that infrastructure of D have a room for  
 improvements.
 
 Regards,
 Todor Totev
Dec 27 2005
prev sibling next sibling parent Tom <Tom_member pathlink.com> writes:
Two thoughts of my own experience for you:

- D infrastructure is this way because it's pre-1.0 version (the language and
the compiler are at kind of BETA stage). Not ready for serious project
development (so here a lot of your complaints gets demolished).

- If you would just downloaded DMC linker (and the other tools coming with it)
*as instructions states clearly*, all that 5 hours of wasted hard work would had
been spended in a more useful way. I had no problems the first time i wanted to
try D. It was about minutes before getting simple demos compiling/linking and
running.

Though I would agree with people saying D is taking too long to get out of its
current BETA stage. Walter should open the project or dedicate more of his time
in it (i don't know about the disadvantages of doing the first).

Tom;

In article <op.s2f8okolihwmk4 todor-1-xp.sanbolic.local>, Todor Totev says...

Hello all,
what started as a simple request for help growed up to some kind of essa=
y.
Here it is:

Why D is not for me?
The scene of this little opera:
I'm working for a small firm which products use kernel modules for Windo=
ws.
Part of our tests are built-in in the modules themselves but others
are user-mode programmes. But it is quite common to go one level lower
and use the so called "Native API" to do some tricks.
My boss told me that I need to made a small test programme that exercise=
s
a obscure pattern of behaviour of our product. I thought: this is  =

relatively
a small programme that is not top priority but nevertheless is useful an=
d
with clear goal. A right test for what can D do for me.
And my quest started.

I've read the information on the site, I was eager to start and I  =

downloaded
dmd.zip. I wrote my first programme which uses Native API.
And now is the right time for my first obstacle. As in any
self-respecting RPG quest, it is an easy task:

1. The official distribution of D is the only language distribution seen=
  =

by me
that is not self sufficient.
   1.1. no linker - no linker? How am I supposed to build my programmes?=

   1.2. no libraries - consider the default command line that dmd builds=

        for optlink - kernel32+gdi32.
        Where is kernel32.lib? Where is gdi32.lib?

Please, compare dmd distribution to Python or Ruby ones.
   Ok, I'm not afraid of some digging and I actually read the documentat=
ion,
so I downloaded dmc package and installed it. The miracle happened and n=
ow
I have kernel32.lib. Which leads me to:

2. D is the only language which aims at system programming and in the sa=
me
time cannot inter-operate with the operating system.
   2.1. No easy support for importing Windows types & functions not
        included by default within Phobos.

Want to easily convert a .h file to .d? Yes of course! Not with D.
Because it does not define all basic types so you can go straight to
missing ones. So what, use Core32 from dsource.org! But one must first
find that this project exists! And then, hurray, it conflicts with Phobo=
s!
Good, now I must find out what to comment out. Five minutes later i'm do=
ne
with this task.

   2.2. The libraries included are old. Outdated. Unusable to me. And I =
even
        didn't try to use Transactional NTFS functions from Vista beta  =

WDK/SDK.
        Plain old, boring Windows 2000/XP functions (5 years old, mind t=
his)
        are nowhere to be seen. Functions like FindFirstVolumeMountPoint=

        or GetVolumePathNamesForVolumeName.
   2.3. No out-of-box support for lib files coming with SDK/DDK/IFS/WDK.=

   2.4. No implib functionality so I cannot create up-to date lib files
        from the dll files.

I dig some more and found that there is coff2omf utility, which:
  * produces incompatible lib files, if I want to use the available Borl=
and  =

one
  * is not available free if I want to use the digital mars one
  * actually do not work because it requires special version of
  <b>Microsoft</b> link.exe that cannot be found anywhere.
  So to use Digital Mars product, I must pay to Microsoft and pray
  that I can found their very old linker somewhere in MSDN archives?
I dig some more and found the special version of OpenWatcom wlink exe
that can do the trick. And indeed, it did the trick.
After about 5 hours fight with it, I was able to spell the right  =

incantation
that linked my 20 lines of D code.
That includes special order of library paths so that Watcom's kernel32 i=
s
first and special order of libraries so that snn comes first.
Really, a good job for starter in a completely new language and  =

environment!
D made me feel proud of myself. And I lost 5 hours but now I know why
most people think programming is some kind of wizardry.
If I opted for MS C++ I will be ready in less than a minute.

 =

tests

   And actually that cannot be called "productive". I now must support t=
hree
separate entities: D compiler, DigitalMars C compiler and OpenWatcom  =

linker.
And of course, for every project I start from now, I must build a bat fi=
le
and linker response file tailored for that project. No build.exe for me.=

No simple dmd *.d my.lib my2.lib. Nothing like this.

Ok, my programme is linked and I have an exe file. But as my ex-boss
likes to say: "No-one should commit code in the source control system
unless she had stepped through every one of the new code in the debugger=
."
I know that all the tools of D language & infrastructure are there to
help me spend as little time in the debugger as possible.
Tools like contracts, unit tests, code coverage etc.
But only the debugger can tell you the flow of code.

3. Where is the debugger?
No near me, for sure.
I hear "WinDbg" screamed, am I not?
First, it does not come with D/dmd.
Second, you must find it in the Microsoft site - not an easy task.
Third, half an year earlier when WinDbg out of the blue got
the docking windows and user interface that made it actually useful
some of my colleague go out, bought some beer and celebrated it.
Fourth, that WinDbg which is quite user-unfriendly does not work with D.=

There are anecdotal evidence that version of WinDbg developed circa 1996=

can be used but I'm not sure I want to touch this beast.
   What other languages offer? Lua, Python & Ruby have built-in
debugging facilities. VisualStudio have a debugger which can be extend
to show your data the way you want it. The dotNet SDK comes with a
debugger (and an excellent one). D simply does not have working debugger=
.

To proceed with other problems, I will leave our beautiful scene
and go back in time to February 2005. In that time I had to do another
little project. This time I was choosing between 3 languages:
D, Lua and Io. I chose Lua because:

4. Documentation
Lua comes with the best book for a programming language I read so far
in my life. It is called "Programming in Lua" and is available from
http://www.lua.org/pil/. A book about a language gives a newbie a base
to stand. It gave me confidence. D (and Io) do not have this. Languages =
 =

like
Python and Ruby come with "Programming Python" and "Programming Ruby" bo=
oks
in the zip files you download from their sites. A newbie is lost in the =
 =

scarce
documentation of D. "Programming Python/Ruby/Lua" gives you the boost to=
  =

start.
I cannot stress how important is a good book for the newbie.
Just go to the link I give and read the book. And then compare to D  =

documents.

5. Often changes of the compiler.
This is a controversial issue. But please consider, how many versions of=
  =

dmd
came from February till now? Even the projects in dsource.org cannot cat=
ch
with them. Core32 collides with current phobos. DFL notes that it is tes=
ted
with version 140 and the author cannot help if you use other version.
Lua on the other side stays unchanged from 2001. Most of its bugs are
documented and their solutions are known. Also, the new version that wil=
l  =

be
out this week is known from half an year. Its new features are known. Yo=
u  =

can
prepare for them (I am currently modifying my code to be ready for it an=
d
send patches to authors for bugs I encounter). D just announces new  =

features
and deprecation of old ones and leaves you to adapt.
I can't build my projects on a base that slips and changes every month.
I know Walter thinks of setting version 1.0 but that is in the future
and can't help me now.


Consider the "Major goals of D" and compare to what I said earlier:
  * Reduce software development costs by at least 10% - losing 5 hours j=
ust
  to link a simple 20 lines programme won't reduce the cost of its  =

development.
  The requirements for 3rd party tools (namely Microsoft linker, OpenWat=
com
  suite) and 4th party tools (the patched wlink for DMC) make the situat=
ion
  as bad as possible.
  * Provide low level bare metal access as required - it is not even nea=
r,
  as I really need access to ntdll.dll (the lowest level to bare metal
  in user mode available) and D actively hinders me.
  * Be compatible with the local C application binary interface - Micros=
oft
  tools are de facto standard for Windows platform and D does not talk t=
o  =

them.


To conclude, language D is a daydream coming true to every  =

C/C++ programmer.
But what D as language gives, D as infrastructure takes tenfolds.
I admit all the problems I presented are not unresolvable,
the big problem is that I must resolve or work around them before doing =
any
actual work. Also I know about projects like Poseidon or D Forms Library=

which prove that D is useful. All I state is that D is not useful for me=

in its current form. And I think that the problems I have are applicable=

to other people too. Also it is possible that I mis-read the documentati=
on
and there is an easy way to do what I do and I simply didn't found it.
But I read as much as I can, searched in the newsgroup archives and
I think I made the best possible effort to do the things right.
Last, I know some of the problems are hard to solve.
Microsoft keeps interface of mspdb??.dll files for themselves so noone c=
an
easily produce pdb files in the new format. And it is not redistributabl=
e.
I know that writing a book is a demanding task that often requires more =
 =

than year
to complete. I thought of creating a module that in its contructor  =

dynamically binds
to functions in ntdll & write simple wrappers in D. Et cetera.
I read the newsgroups since version 0.96 or .98 (can't remember exactly)=
.
I'm a competent Windows developer and if Poseidon turns a good IDE I can=

try to write a decent wrapper around dbghelp.dll to help build a proper
debugger. But now I feel that infrastructure of D have a room for  =

improvements.

Regards,
Todor Totev
Dec 27 2005
prev sibling next sibling parent reply "Todor Totev" <umbra.tenebris list.ru> writes:
   Hello all,
to make my point clear, I present you a simple task:

Build a programme that discovers where boot.ini file resides and prints =
 =

its contents on the screen.

For all that don't know, boot.ini file resides on the system partition o=
f  =

Windows NT & up
and tells the system loader from where it can boot the OS.

For all that think that this is not a "typical" task, I will answer that=
  =

this summer I had not only to
do this but also to MODIFY boot.ini. And I have limited time to do this.=

Yes, I'm a system programmer - for my living I write drivers,
WMI providers, and Windows services all of which runs on Windows  =

2000/XP/2003.

I attach the solution I forged in D. Please, disregard the quality of th=
e  =

D portion of code -
this is my first programme in D so I'm sure it can be improved.

You can compile, link, run the program & enjoy the results.
I will post as replays to this message some hints of what must be done.
If you want to check how much time you need to run this programme, don't=
  =

read them.

You must know that you need kernel32.lib advapi32.lib & ntdll.lib which =
 =

corresponds to
Windows 2000 system dlls to successfully build the code.

Enjoy,
Todor
Dec 28 2005
parent "Todor Totev" <umbra.tenebris list.ru> writes:
Problem 1: where is the damned ntdll.lib?

Solutions:

1. Digital mars one - use implib on ntdll.dll in your system folder and
make sure the resulted lib will work with windows 2000,XP,2003.

2. Official one - get Microsoft DDK and use the provided official import=
  =

library ntdll.lib -
this is what I tried to do from the beginning but will cost you some $ i=
f  =

you do not have already one

3. The poor bastard that just whants to play with D one -
download Open Watcom form www.openwatcom.org and use the ntdll.lib from
\watcom\lib386\nt

4. The hacker one - modify the source until it compiles & links

Please tell me which route you take and how much time you need to finish=
  =

it.

Enjoy the show,
Todor
Dec 28 2005
prev sibling next sibling parent reply Ilya Minkov <minkov cs.tum.edu> writes:
It all revolves about resources available. Go start a company, hire 200 
employees and soon (in a year or 2) you will have a perfect working 
environment with IDE, debugger etc. Or maybe not.

And then we shall see who will pay for it. Did you pay for your MSVC? I 
haven't - i have mine from the MS university support programme. I 
wouldn't be able to afford it myself. And don't let fool yourself my MS 
giving from time to time limited working versions of their software: in 
the long run, they have to pay their deveopment costs. The situation is 
not much different in the world of free software: it is being developed 
by companies with large commercial interest - sometimes it just makes 
more sense to improve a free product for everyone than to license a 
commercial one. If not for Apple, IBM and many other giants, there would 
be no GCC, no Gnome, no KDE, no high-performance free unix clones. Free 
software would not have evolved to be an alternative to commercial 
software in many areas it has become.

So saying "MSVC is so much better" is cynical at best. Also, consider: 
did you have your first MSVC programme running within minutes? Did you 
find the documentation, the header files at once? Did you know what 
every element of an IDE does by just looking at it for the first time?

-eye
Jan 01 2006
parent reply Lars Ivar Igesund <larsivar igesund.net> writes:
Ilya Minkov wrote:


 And then we shall see who will pay for it. Did you pay for your MSVC? I
 haven't - i have mine from the MS university support programme. I
 wouldn't be able to afford it myself. And don't let fool yourself my MS
 giving from time to time limited working versions of their software: in
 the long run, they have to pay their deveopment costs. 



 -eye


MS don't do this to earn money on the development software itself, it is all
a part of the strategy to tie the developers into creating software for
Windows. The more exclusive Windows software, the more money to MS. It is
their absurdly huge income from Windows, Office and different server
products that fund the development of hugely productive tools. Now, I think
I recently heard that such type of tools (highly productive 4th generation
stuff) makes the developer less aware of issues related to code quality,
and even reuse, that these tools in theory should support. I certainly know
that these tools hide quite a lot of the details that interest _me_, and
unless you are a wizz at the tool itself, it becomes quite troublesome to
debug problems in the toolchain.

And I suppose this is offtopic...

Lars Ivar Igesund
Jan 01 2006
parent reply "Matthew" <matthew hat.stlsoft.dot.org> writes:
"Lars Ivar Igesund" <larsivar igesund.net> wrote in message
news:dp8qkr$3tt$1 digitaldaemon.com...

 Ilya Minkov wrote:


 And then we shall see who will pay for it. Did you pay for your MSVC? I
 haven't - i have mine from the MS university support programme. I
 wouldn't be able to afford it myself. And don't let fool yourself my MS
 giving from time to time limited working versions of their software: in
 the long run, they have to pay their deveopment costs.



 -eye


 MS don't do this to earn money on the development software itself, it is


all

 a part of the strategy to tie the developers into creating software for
 Windows. The more exclusive Windows software, the more money to MS. It is
 their absurdly huge income from Windows, Office and different server
 products that fund the development of hugely productive tools. Now, I


think

 I recently heard that such type of tools (highly productive 4th generation
 stuff) makes the developer less aware of issues related to code quality,
 and even reuse, that these tools in theory should support. I certainly


know

 that these tools hide quite a lot of the details that interest _me_, and
 unless you are a wizz at the tool itself, it becomes quite troublesome to
 debug problems in the toolchain.

 And I suppose this is offtopic...


Not at all. Interesting, apposite, and, to the best of my understanding,
correct. :-)
Jan 01 2006
parent Ilya Minkov <minkov cs.tum.edu> writes:
Matthew wrote:

 "Lars Ivar Igesund" <larsivar igesund.net> wrote in message


[...]


I recently heard that such type of tools (highly productive 4th generation
stuff) makes the developer less aware of issues related to code quality,
and even reuse, that these tools in theory should support. I certainly know
that these tools hide quite a lot of the details that interest _me_, and
unless you are a wizz at the tool itself, it becomes quite troublesome to
debug problems in the toolchain.

And I suppose this is offtopic...


 
 Not at all. Interesting, apposite, and, to the best of my understanding,
 correct. :-)


While drifting offtopic, i recalled a recent essay by Charles Petzold, 
who has been writing popular books on Windows programming back when i 
was a kid poking around on a ZX-Spectrum clone.

http://charlespetzold.com/etc/DoesVisualStudioRotTheMind.html

-i.
Jan 02 2006
prev sibling parent reply "Walter Bright" <newshound digitalmars.com> writes:
"Todor Totev" <umbra.tenebris list.ru> wrote in message 
news:op.s2f8okolihwmk4 todor-1-xp.sanbolic.local...

   2.2. The libraries included are old. Outdated. Unusable to me. And I 
 even
        didn't try to use Transactional NTFS functions from Vista beta 
 WDK/SDK.
        Plain old, boring Windows 2000/XP functions (5 years old, mind 
 this)
        are nowhere to be seen. Functions like FindFirstVolumeMountPoint
        or GetVolumePathNamesForVolumeName.
   2.3. No out-of-box support for lib files coming with SDK/DDK/IFS/WDK.
   2.4. No implib functionality so I cannot create up-to date lib files
        from the dll files.


This problem is now resolved with the coffimplib utility.
Jan 02 2006
parent reply F <F_member pathlink.com> writes:
Maybe I am wrong, but I understand that:
-implib/coffimplib are required to generate a .lib file from a .dll to be linked
with dmd generated code.
I wanted to ask if would be possible to directly integrate this into the linker,
so one could directly pass the .dll in command line. Maybe something like this:
http://www.jorgon.freeserve.co.uk/#linker
So, if .lib can be generated from .dll to be linked with, why use .lib at all?

F



In article <dpc8b3$29li$1 digitaldaemon.com>, Walter Bright says...

"Todor Totev" <umbra.tenebris list.ru> wrote in message 
news:op.s2f8okolihwmk4 todor-1-xp.sanbolic.local...

   2.2. The libraries included are old. Outdated. Unusable to me. And I 
 even
        didn't try to use Transactional NTFS functions from Vista beta 
 WDK/SDK.
        Plain old, boring Windows 2000/XP functions (5 years old, mind 
 this)
        are nowhere to be seen. Functions like FindFirstVolumeMountPoint
        or GetVolumePathNamesForVolumeName.
   2.3. No out-of-box support for lib files coming with SDK/DDK/IFS/WDK.
   2.4. No implib functionality so I cannot create up-to date lib files
        from the dll files.


This problem is now resolved with the coffimplib utility.
Jan 03 2006
parent reply "Walter Bright" <newshound digitalmars.com> writes:
"F" <F_member pathlink.com> wrote in message 
news:dpeeth$mvs$1 digitaldaemon.com...

 Maybe I am wrong, but I understand that:
 -implib/coffimplib are required to generate a .lib file from a .dll to be 
 linked
 with dmd generated code.
 I wanted to ask if would be possible to directly integrate this into the 
 linker,
 so one could directly pass the .dll in command line. Maybe something like 
 this:
 http://www.jorgon.freeserve.co.uk/#linker
 So, if .lib can be generated from .dll to be linked with, why use .lib at 
 all?


Good question. The answer is that Microsoft set up the system dlls with 
names that *DO NOT MATCH* the names generated by the C compiler. Yes, you 
heard that right! So, simply scanning the system dlls does not give enough 
information.

The import library provides a translation between the 'internal name' and 
the 'external name'. The former is missing from the dll's.
Jan 03 2006
next sibling parent James Dunne <james.jdunne gmail.com> writes:
Walter Bright wrote:

 "F" <F_member pathlink.com> wrote in message 
 news:dpeeth$mvs$1 digitaldaemon.com...
 

Maybe I am wrong, but I understand that:
-implib/coffimplib are required to generate a .lib file from a .dll to be 
linked
with dmd generated code.
I wanted to ask if would be possible to directly integrate this into the 
linker,
so one could directly pass the .dll in command line. Maybe something like 
this:
http://www.jorgon.freeserve.co.uk/#linker
So, if .lib can be generated from .dll to be linked with, why use .lib at 
all?


 
 
 Good question. The answer is that Microsoft set up the system dlls with 
 names that *DO NOT MATCH* the names generated by the C compiler. Yes, you 
 heard that right! 


Heard what now? =P


 So, simply scanning the system dlls does not give enough 
 information.
 
 The import library provides a translation between the 'internal name' and 
 the 'external name'. The former is missing from the dll's. 
 


That is seriously messed up.
Jan 03 2006
prev sibling parent Don Clugston <dac nospam.com.au> writes:
Walter Bright wrote:

So, if .lib can be generated from .dll to be linked with, why use .lib at 
all?


 
 Good question. The answer is that Microsoft set up the system dlls with 
 names that *DO NOT MATCH* the names generated by the C compiler. Yes, you 
 heard that right! So, simply scanning the system dlls does not give enough 
 information.
 
 The import library provides a translation between the 'internal name' and 
 the 'external name'. The former is missing from the dll's. 


This is one of the (rare) cases where I prefer the original VB solution: 
specify the external name as a alias when defining the function. So that 
the names are in a human-readable header file, instead of yet another 
complex binary file format.

The thing I really don't understand is why Microsoft changed the 
definition of __declspec(export) with extern "C" in VC++ so that it 
didn't disable name mangling. When they first introduced 
__declspec(export), you could create DLLs that didn't need DEF files or 
implibs. It was fabulous. With the next compiler release, it was back to 
the Bad Old Days. I think that was the release where they removed 80-bit 
reals, too.

They've made some really strange decisions over the years. I wonder if 
internal politics are involved.
Jan 04 2006