www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Win32 headers: Problem with COM interface indirection levels

reply Michael Butscher <mbutscher gmx.de> writes:
Hi,

there seems to be a rather fundamental error in the translation of COM 
interfaces by the Win32 bindings project:

The C++ headers defines an interface as a class (more precisely a 
struct) with some virtual methods. This means, an instance of the 
interface is the actual object data, mainly containing a pointer to the 
vtable.

In D, an interface is a pointer to the object data which contains in 
turn the pointer to the vtable.

Because of the different level of indirection e.g. the IShellFolder 
interface in D is in fact equivalent to the LPSHELLFOLDER type in C++.


I think there are two possible solutions:

1. Modify parameter types of all functions using the interface, e.g. 
translate C++ definition:

  HRESULT SHGetDesktopFolder(LPSHELLFOLDER*);


to:
  
  HRESULT SHGetDesktopFolder(LPSHELLFOLDER);

or:

  HRESULT SHGetDesktopFolder(IShellFolder*);




2. Rewrite the definitions of the LP... types as aliases of the 
interfaces (I would prefer that), e.g.

C++:

  typedef IShellFolder *LPSHELLFOLDER;


to D:

  alias IShellFolder LPSHELLFOLDER;




I think this is important so the decision should be documented in the 
Translation instructions.



Michael
Aug 08 2007
next sibling parent reply Carlos Santander <csantander619 gmail.com> writes:
Michael Butscher escribió:

 Hi,
 
 there seems to be a rather fundamental error in the translation of COM 
 interfaces by the Win32 bindings project:
 
 The C++ headers defines an interface as a class (more precisely a 
 struct) with some virtual methods. This means, an instance of the 
 interface is the actual object data, mainly containing a pointer to the 
 vtable.
 
 In D, an interface is a pointer to the object data which contains in 
 turn the pointer to the vtable.
 
 Because of the different level of indirection e.g. the IShellFolder 
 interface in D is in fact equivalent to the LPSHELLFOLDER type in C++.
 
 
 I think there are two possible solutions:
 
 1. Modify parameter types of all functions using the interface, e.g. 
 translate C++ definition:
 
   HRESULT SHGetDesktopFolder(LPSHELLFOLDER*);
 
 
 to:
   
   HRESULT SHGetDesktopFolder(LPSHELLFOLDER);
 
 or:
 
   HRESULT SHGetDesktopFolder(IShellFolder*);
 
 
 
 
 2. Rewrite the definitions of the LP... types as aliases of the 
 interfaces (I would prefer that), e.g.
 
 C++:
 
   typedef IShellFolder *LPSHELLFOLDER;
 
 
 to D:
 
   alias IShellFolder LPSHELLFOLDER;
 
 
 
 
 I think this is important so the decision should be documented in the 
 Translation instructions.
 
 
 
 Michael


I don't use Windows, but I haven't heard of anyone having this problem before. 
Can you test that with a simpler case? Remember that there's a ComInterface (or 
something like that) in std.c.windows.com (I think) which gets special
treatment 
by the compiler. Maybe the compiler performs some magic there in order to get 
everything working fine.

Also, maybe this question is better suited in the Bindings project Dsource
forum?

-- 
Carlos Santander Bernal
Aug 08 2007
parent reply Michael Butscher <mbutscher gmx.de> writes:
Carlos Santander wrote:

 Michael Butscher escribió:


[...]

 Because of the different level of indirection e.g. the IShellFolder 
 interface in D is in fact equivalent to the LPSHELLFOLDER type in C++.




[...]


 I don't use Windows, but I haven't heard of anyone having this problem before.


This might be an indication that this part of the headers is currently 
not used widely (maybe by some people changing their copy of the 
headers privately). But the published version definitely can't work.



 Can you test that with a simpler case?


I have made a little test already. The provided headers lead to a 
access violation. Modifying it according to my request make them work.

I found out in the meantime that some headers (namely "d3d9.d") were 
modified already in a similar way, but it was not documented in the 
translation instructions.



 Remember that there's a ComInterface (or 
 something like that) in std.c.windows.com (I think) which gets special
treatment 
 by the compiler.


It seems that every interface named "IUnknown" (and derived interfaces) 
is handled specially, regardless of the containing module. This is not 
the problem.



 Maybe the compiler performs some magic there in order to get 
 everything working fine.
 
 Also, maybe this question is better suited in the Bindings project Dsource
forum?


You are right. I only found forums a bit impractical, but that's not a 
good argument.



Michael
Aug 09 2007
next sibling parent reply jcc7 <technocrat7 gmail.com> writes:
== Quote from Michael Butscher (mbutscher gmx.de)'s article

 Carlos Santander wrote:

 Michael Butscher escribió:


 [...]


...

 I found out in the meantime that some headers (namely
 "d3d9.d") were modified already in a similar way, but it was not
 documented in the translation instructions.


It's been mentioned on the wiki page at
http://www.prowiki.org/wiki4d/wiki.cgi?WindowsAPI, but I think you're right that
it's not in the "Translation instructions" section (and it probably should be
there).

I get the feeling that the WindowsAPI page at Wiki4D has gotten way too big to
be
efficient. I think that it should probably be split into subpages, but I would
be
afraid of stepping on the toes of the WindowsAPI team by doing it myself.

(Also, I wonder if it would work better to store the content that's current at
Wiki4D in the bindings wiki instead [http://www.dsource.org/projects/bindings],
since it is part of the bindings project and dsource wikis have a cool way of
formatting D source code.)



 Remember that there's a ComInterface (or
 something like that) in std.c.windows.com (I think) which gets
 special treatment by the compiler.


 It seems that every interface named "IUnknown" (and derived
 interfaces) is handled specially, regardless of the containing
 module. This is not the problem.

 Maybe the compiler performs some magic there in order to get
 everything working fine.

 Also, maybe this question is better suited in the Bindings project
 Dsource forum?


 You are right. I only found forums a bit impractical, but that's not
 a good argument.


Actually by my count, it's already been mentioned twice by other people in the
Bindings forum.

http://www.dsource.org/forums/viewtopic.php?t=2458


(The bindings project doesn't really have formal leadership, though, so I'm not
sure who's reading that forum anyway.)
Aug 09 2007
next sibling parent reply Michael Butscher <mbutscher gmx.de> writes:
jcc7 wrote:

 == Quote from Michael Butscher (mbutscher gmx.de)'s article

 Carlos Santander wrote:

 Michael Butscher escribió:


 [...]


 ...

 I found out in the meantime that some headers (namely
 "d3d9.d") were modified already in a similar way, but it was not
 documented in the translation instructions.


 
 It's been mentioned on the wiki page at
 http://www.prowiki.org/wiki4d/wiki.cgi?WindowsAPI, but I think you're right
that
 it's not in the "Translation instructions" section (and it probably should be
there).
 
 I get the feeling that the WindowsAPI page at Wiki4D has gotten way too big to
be
 efficient. I think that it should probably be split into subpages, but I would
be
 afraid of stepping on the toes of the WindowsAPI team by doing it myself.


Personally I prefer one large page as I'm using a dialup connection, so 
I have to load only one page and can read it offline easily.



 (Also, I wonder if it would work better to store the content that's current at
 Wiki4D in the bindings wiki instead [http://www.dsource.org/projects/bindings],
 since it is part of the bindings project and dsource wikis have a cool way of
 formatting D source code.)


At least there should be only one place for the instructions. Currently 
there exists also a readme file in the SVN-repository with similar 
instructions.

Either the file should point to the wiki page(s) or vice versa.


[...]

 Also, maybe this question is better suited in the Bindings project
 Dsource forum?


 You are right. I only found forums a bit impractical, but that's not
 a good argument.


 
 Actually by my count, it's already been mentioned twice by other people in the
 Bindings forum.
 
 http://www.dsource.org/forums/viewtopic.php?t=2458



I thought that somebody would have written it in the instructions if 
the problem was realized already therefore I didn't search for it in 
the forum.




Michael
Aug 10 2007
next sibling parent reply jcc7 <technocrat7 gmail.com> writes:
== Quote from Michael Butscher (mbutscher gmx.de)'s article

 jcc7 wrote:

 == Quote from Michael Butscher (mbutscher gmx.de)'s article

 Carlos Santander wrote:

 Michael Butscher escribió:


 [...]


 ...

 I found out in the meantime that some headers (namely
 "d3d9.d") were modified already in a similar way, but it was not
 documented in the translation instructions.


 It's been mentioned on the wiki page at
 http://www.prowiki.org/wiki4d/wiki.cgi?WindowsAPI, but I think you're right
that
 it's not in the "Translation instructions" section (and it probably should be




there).

 I get the feeling that the WindowsAPI page at Wiki4D has gotten way too big to
be
 efficient. I think that it should probably be split into subpages, but I would
be
 afraid of stepping on the toes of the WindowsAPI team by doing it myself.


 Personally I prefer one large page as I'm using a dialup connection, so
 I have to load only one page and can read it offline easily.


I understand the reason for your preference, but I still think the page is way
too
long. Perhaps the instructions could stay on the main page, but the list of
contributor and the discussions could be kept on separate pages. Anyway, I don't
want to interfere with the project, so I'll let someone actually involved in the
project make that change if they want to take my (unsolicited) advice.


 (Also, I wonder if it would work better to store the content that's current at
 Wiki4D in the bindings wiki instead [http://www.dsource.org/projects/bindings],
 since it is part of the bindings project and dsource wikis have a cool way of
 formatting D source code.)


 At least there should be only one place for the instructions. Currently
 there exists also a readme file in the SVN-repository with similar
 instructions.


That does sound like a problem.



 Either the file should point to the wiki page(s) or vice versa.


I agree.

Personally, I would prefer to have a wiki page as the sole place due to the
greater options for formatting.


 [...]

 Also, maybe this question is better suited in the Bindings project
 Dsource forum?


 You are right. I only found forums a bit impractical, but that's not
 a good argument.


 Actually by my count, it's already been mentioned twice by other people in the
 Bindings forum.

 http://www.dsource.org/forums/viewtopic.php?t=2458



 I thought that somebody would have written it in the instructions if
 the problem was realized already therefore I didn't search for it in
 the forum.
 Michael


I don't know how the people API project are talking to each other. I assumed it
was via the wiki page, but for all I know they could be using IRC, AIM, or an
e-mail mailing list. I was just trying to make the point that apparently it
doesn't matter if anyone posts in the bindings forum because the people who
might
want to know that information don't seem to even be reading that forum.

I'm not a leader in the API project (or even a member). I expect that the
leadership probably doesn't want just anyone editing their instructions
section. I
suspect that why the people that posted on forum didn't edit the instructions
themselves. Or maybe they weren't aware of the Wiki4D page? (I'm not sure how
the
API project is organized.)

I did at least make a comment on that page, but it's become such an unwieldy
page
that I don't think anyone has read it from beginning to end recently.
Aug 10 2007
next sibling parent reply Michael Butscher <mbutscher gmx.de> writes:
jcc7 wrote:

 == Quote from Michael Butscher (mbutscher gmx.de)'s article

 jcc7 wrote:




[...]

 I get the feeling that the WindowsAPI page at Wiki4D has gotten way too big to
be
 efficient. I think that it should probably be split into subpages, but I would
be
 afraid of stepping on the toes of the WindowsAPI team by doing it myself.


 Personally I prefer one large page as I'm using a dialup connection, so
 I have to load only one page and can read it offline easily.


 
 I understand the reason for your preference, but I still think the page is way
too
 long. Perhaps the instructions could stay on the main page, but the list of
 contributor and the discussions could be kept on separate pages. Anyway, I
don't
 want to interfere with the project, so I'll let someone actually involved in
the
 project make that change if they want to take my (unsolicited) advice.


Now I have posted a message to the Bindings forum about that. Let's see 
what happens.


[...]

 I thought that somebody would have written it in the instructions if
 the problem was realized already therefore I didn't search for it in
 the forum.
 Michael


 
 I don't know how the people API project are talking to each other. I assumed it
 was via the wiki page, but for all I know they could be using IRC, AIM, or an
 e-mail mailing list. I was just trying to make the point that apparently it
 doesn't matter if anyone posts in the bindings forum because the people who
might
 want to know that information don't seem to even be reading that forum.
 
 I'm not a leader in the API project (or even a member). I expect that the
 leadership probably doesn't want just anyone editing their instructions
section. I
 suspect that why the people that posted on forum didn't edit the instructions
 themselves. Or maybe they weren't aware of the Wiki4D page? (I'm not sure how
the
 API project is organized.)


The start page of the bindings project wiki has a link to the Wiki4D 
page, so the "WindowsAPI" page seems to be "official".



Michael
Aug 11 2007
parent reply Justin C Calvarese <technocrat7 gmail.com> writes:
Michael Butscher wrote:

 jcc7 wrote:

 == Quote from Michael Butscher (mbutscher gmx.de)'s article

 jcc7 wrote:




 [...]

 I get the feeling that the WindowsAPI page at Wiki4D has gotten way too big to
be
 efficient. I think that it should probably be split into subpages, but I would
be
 afraid of stepping on the toes of the WindowsAPI team by doing it myself.


 Personally I prefer one large page as I'm using a dialup connection, so
 I have to load only one page and can read it offline easily.


 I understand the reason for your preference, but I still think the page is way
too
 long. Perhaps the instructions could stay on the main page, but the list of
 contributor and the discussions could be kept on separate pages. Anyway, I
don't
 want to interfere with the project, so I'll let someone actually involved in
the
 project make that change if they want to take my (unsolicited) advice.


 
 Now I have posted a message to the Bindings forum about that. Let's see 
 what happens.


Fair enough.



 [...]

 I thought that somebody would have written it in the instructions if
 the problem was realized already therefore I didn't search for it in
 the forum.
 Michael


 I don't know how the people API project are talking to each other. I assumed it
 was via the wiki page, but for all I know they could be using IRC, AIM, or an
 e-mail mailing list. I was just trying to make the point that apparently it
 doesn't matter if anyone posts in the bindings forum because the people who
might
 want to know that information don't seem to even be reading that forum.

 I'm not a leader in the API project (or even a member). I expect that the
 leadership probably doesn't want just anyone editing their instructions
section. I
 suspect that why the people that posted on forum didn't edit the instructions
 themselves. Or maybe they weren't aware of the Wiki4D page? (I'm not sure how
the
 API project is organized.)


 
 The start page of the bindings project wiki has a link to the Wiki4D 
 page, so the "WindowsAPI" page seems to be "official".


Actually, I was the one who added that link, but it's been over a year, 
so if anyone objected, I hope they would've corrected it by now. ;)
http://www.dsource.org/projects/bindings/wiki/WikiStart?action=diff&version=6&old_version=5

-- 
jcc7
Aug 11 2007
parent "Stewart Gordon" <smjg_1998 yahoo.com> writes:
"Justin C Calvarese" <technocrat7 gmail.com> wrote in message 
news:f9lsdf$m5b$1 digitalmars.com...

 Michael Butscher wrote:


<snip>

 The start page of the bindings project wiki has a link to the Wiki4D 
 page, so the "WindowsAPI" page seems to be "official".


 Actually, I was the one who added that link, but it's been over a year, so 
 if anyone objected, I hope they would've corrected it by now. ;)
 http://www.dsource.org/projects/bindings/wiki/WikiStart?action=diff&version=6&old_version=5


The page on Wiki4D is indeed the official one at the moment.

Stewart.
Aug 27 2007
prev sibling parent "Stewart Gordon" <smjg_1998 yahoo.com> writes:
"jcc7" <technocrat7 gmail.com> wrote in message 
news:f9hu7a$2j79$1 digitalmars.com...
<snip>

 I don't know how the people API project are talking to each other. I 
 assumed it
 was via the wiki page, but for all I know they could be using IRC, AIM, or 
 an
 e-mail mailing list. I was just trying to make the point that apparently 
 it
 doesn't matter if anyone posts in the bindings forum because the people 
 who might
 want to know that information don't seem to even be reading that forum.


<snip>

It is mostly via the wiki page and these newsgroups.  The problem is that 
not enough people are working on it, and those that do tend to disappear 
after a while.

Stewart.
Aug 27 2007
prev sibling parent "Stewart Gordon" <smjg_1998 yahoo.com> writes:
"Michael Butscher" <mbutscher gmx.de> wrote in message 
news:MPG.21268ee9722f11e99896a9 news.digitalmars.com...
<snip>

 At least there should be only one place for the instructions. Currently
 there exists also a readme file in the SVN-repository with similar
 instructions.


They're meant to be the same, but probably have got a bit out of sync.


 Either the file should point to the wiki page(s) or vice versa.


<snip>

Having it in the SVN at least means that we have access to a revision 
history of the instructions.  But migrating it onto the Dsource wiki would 
be one way to avoid losing this.  Good thinking, whosever idea this was!

Stewart.
Aug 27 2007
prev sibling parent reply "Stewart Gordon" <smjg_1998 yahoo.com> writes:
"jcc7" <technocrat7 gmail.com> wrote in message 
news:f9f7oc$2ovj$1 digitalmars.com...
<snip>

 It's been mentioned on the wiki page at
 http://www.prowiki.org/wiki4d/wiki.cgi?WindowsAPI, but I think you're 
 right that
 it's not in the "Translation instructions" section (and it probably should 
 be there).

 I get the feeling that the WindowsAPI page at Wiki4D has gotten way too 
 big to be
 efficient. I think that it should probably be split into subpages, but I 
 would be
 afraid of stepping on the toes of the WindowsAPI team by doing it myself.


It probably could be split, though I'm not sure what would be the best 
approach.


 (Also, I wonder if it would work better to store the content that's 
 current at
 Wiki4D in the bindings wiki instead 
 [http://www.dsource.org/projects/bindings],
 since it is part of the bindings project and dsource wikis have a cool way 
 of
 formatting D source code.)


<snip>

Maybe that would be a good idea.  I haven't used Dsource's wiki engine 
much - how does it compare to prowiki?  At least it seems able to show 
arbitrary versions of  a page like MediaWiki, rather than just the current 
version and the most recent edit.

Stewart.
Aug 27 2007
parent jcc7 <technocrat7 gmail.com> writes:
== Quote from Stewart Gordon (smjg_1998 yahoo.com)'s article

 "jcc7" <technocrat7 gmail.com> wrote in message
 news:f9f7oc$2ovj$1 digitalmars.com...
 <snip>

 It's been mentioned on the wiki page at
 http://www.prowiki.org/wiki4d/wiki.cgi?WindowsAPI, but I think you're
 right that
 it's not in the "Translation instructions" section (and it probably should
 be there).

 I get the feeling that the WindowsAPI page at Wiki4D has gotten way too
 big to be
 efficient. I think that it should probably be split into subpages, but I
 would be
 afraid of stepping on the toes of the WindowsAPI team by doing it myself.


 It probably could be split, though I'm not sure what would be the best
 approach.

 (Also, I wonder if it would work better to store the content that's
 current at
 Wiki4D in the bindings wiki instead
 [http://www.dsource.org/projects/bindings],
 since it is part of the bindings project and dsource wikis have a cool way
 of
 formatting D source code.)


 <snip>
 Maybe that would be a good idea.  I haven't used Dsource's wiki engine
 much - how does it compare to prowiki?  At least it seems able to show
 arbitrary versions of  a page like MediaWiki, rather than just the current
 version and the most recent edit.


If you click on the "Archive" link, it will show you more versions, but you're
right that Trac does keep track of edits more thoroughly.

As you noticed, Trac keeps all of the version of a file as it's changed (and it
always keeps track of who changed it, too). On the other hand, I haven't quite
been able to figure out how often ProWiki saves a version in the archive or even
if they're saved permanently. And with ProWiki, it can be difficult to determine
who changed what.

Trac is really good at formatting D code automatically (for an example, see:
http://www.dsource.org/projects/tutorials/wiki/PrintfExample). By the way, the
old
formatter that I had worked on has been replaced with a much cooler formatter
that
is Pygments-based.

Trac allows you to get an RSS feed of project events (such when people change
wiki
pages and when SVN changes are committed). ProWiki will also give an RSS feed
(which I think will even let you restrict your news to a particular page), but I
think it's helpful to have the SVN changes and the wiki changes in the same
feed.
(I've been using RSS to watch that no one is being mischievous in the Tutorials
project for a while.)

Also, every wiki engine seems to have it's own flavor of wiki syntax, of course,
but you probably already realized that.

I think that using the Trac wiki pages would make more sense that using the
ProWiki page. It probably would take a little time to change the pages from
ProWiki syntax to   Trac wiki syntax, but I don't think it'd be that bad. And
you
wouldn't have to change all of the content at once. You could move one section
of
the page, then delete it from ProWiki, and so on. I'd even help you if you want.
Aug 28 2007
prev sibling parent reply "Stewart Gordon" <smjg_1998 yahoo.com> writes:
"Michael Butscher" <mbutscher gmx.de> wrote in message 
news:MPG.212522dc134a28d69896a8 news.digitalmars.com...
<snip>

 This might be an indication that this part of the headers is currently
 not used widely (maybe by some people changing their copy of the
 headers privately). But the published version definitely can't work.


Which headers - std.c.windows or my project?

And if people are creating private forks of my project, why?  True, it's PD, 
so who's to stop them, but what's to gain by doing this instead of 
contributing to the project?

<snip>

 I found out in the meantime that some headers (namely "d3d9.d") were
 modified already in a similar way, but it was not documented in the
 translation instructions.


The D3D9 headers have been completely retranslated from M$'s own headers by 
somebody who missed the point.  Is MinGW 3.9 as up to date as this?  I've 
just deassigned these modules - would somebody like to redo them from MinGW 
3.9?

<snip>

 Also, maybe this question is better suited in the Bindings project 
 Dsource forum?


 You are right. I only found forums a bit impractical, but that's not a
 good argument.


And I don't tend to read the Dsource forums....

Stewart.
Aug 27 2007
parent Michael Butscher <mbutscher gmx.de> writes:
Stewart Gordon wrote:

 "Michael Butscher" <mbutscher gmx.de> wrote in message 
 news:MPG.212522dc134a28d69896a8 news.digitalmars.com...
 <snip>

 This might be an indication that this part of the headers is currently
 not used widely (maybe by some people changing their copy of the
 headers privately). But the published version definitely can't work.


 
 Which headers - std.c.windows or my project?


I meant your headers. The std.c.windows headers work.



 And if people are creating private forks of my project, why?  True, it's PD, 
 so who's to stop them, but what's to gain by doing this instead of 
 contributing to the project?


Well, laziness is part of human nature.

First one should ask if the changes made privately were done in the 
right way to not produce chaos in the repository.

One has to find the subversion URL (I really had some problems with 
this step), check out the files, change it, check if the testcompile 
script works (which it didn't in the past due to, AFAIR d3d9.d) and 
finally commit the changes.


[...]

 Also, maybe this question is better suited in the Bindings project 
 Dsource forum?


 You are right. I only found forums a bit impractical, but that's not a
 good argument.


 
 And I don't tend to read the Dsource forums....


With the bindings forum there is also the additional problem that it is 
used by multiple subprojects and one has to guess by subject line if 
the topic belongs to Windows bindings or something else.



Michael
Sep 03 2007
prev sibling parent Don Clugston <dac nospam.com.au> writes:
Michael Butscher wrote:

 Hi,
 
 there seems to be a rather fundamental error in the translation of COM 
 interfaces by the Win32 bindings project:
 
 The C++ headers defines an interface as a class (more precisely a 
 struct) with some virtual methods. This means, an instance of the 
 interface is the actual object data, mainly containing a pointer to the 
 vtable.
 
 In D, an interface is a pointer to the object data which contains in 
 turn the pointer to the vtable.
 
 Because of the different level of indirection e.g. the IShellFolder 
 interface in D is in fact equivalent to the LPSHELLFOLDER type in C++.
 
 
 I think there are two possible solutions:
 
 1. Modify parameter types of all functions using the interface, e.g. 
 translate C++ definition:
 
   HRESULT SHGetDesktopFolder(LPSHELLFOLDER*);
 
 
 to:
   
   HRESULT SHGetDesktopFolder(LPSHELLFOLDER);
 
 or:
 
   HRESULT SHGetDesktopFolder(IShellFolder*);
 
 
 
 
 2. Rewrite the definitions of the LP... types as aliases of the 
 interfaces (I would prefer that), e.g.
 
 C++:
 
   typedef IShellFolder *LPSHELLFOLDER;
 
 
 to D:
 
   alias IShellFolder LPSHELLFOLDER;


I think you're right (and I'm probably responsible for most of it). It's a 
straightforward but fundamental error. Essentially, there's already a * in the 
definition of a D class. The second way is how it should be.
Should be straightforward to fix with a simple bit of grepping.
Aug 09 2007