www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Header Files

reply bitwise <bitwise.pvt gmail.com> writes:
./main.d
./pack/foo.d
./pack/sub/bar.d

dmd main.d pack/foo.d pack/sub/bar.d -ofTest -H

This dumps all the *.di files into the output directory ignoring the  
directory structure.

Is there some rational for it being this way?
Wouldn't it be much more useful if directories were preserved?

   Bit
May 09 2015
next sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Sat, 09 May 2015 10:01:25 -0400, bitwise <bitwise.pvt gmail.com> wrote:


 ./main.d
 ./pack/foo.d
 ./pack/sub/bar.d

 dmd main.d pack/foo.d pack/sub/bar.d -ofTest -H

 This dumps all the *.di files into the output directory ignoring the  
 directory structure.

 Is there some rational for it being this way?
 Wouldn't it be much more useful if directories were preserved?

    Bit



At this point, I'm thinking I should file a bug report for this.

You can't build a codebase with multiple packages because package.d or any  
other files that share the same filename will be overwritten.

Is there really no way to preserve the directory structure when creating  
.di files?

   Bit
May 09 2015
parent reply "bitwise" <bitwise.pvt gmail.com> writes:
On Sunday, 10 May 2015 at 01:00:31 UTC, bitwise wrote:

 Is there really no way to preserve the directory structure when 
 creating .di files?
   Bit


Why hello, Bitwise! I believe the '-op' flag is what you're 
looking for!

   Bitwiser
May 10 2015
parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 05/10/2015 09:11 AM, bitwise wrote:


 On Sunday, 10 May 2015 at 01:00:31 UTC, bitwise wrote:

 Is there really no way to preserve the directory structure when
 creating .di files?
   Bit


 Why hello, Bitwise! I believe the '-op' flag is what you're looking for!

    Bitwiser


Wow! Walter has already implemented it by going back in time. :)

Ali
May 10 2015
prev sibling next sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 05/09/2015 07:01 AM, bitwise wrote:

 ./main.d
 ./pack/foo.d
 ./pack/sub/bar.d

 dmd main.d pack/foo.d pack/sub/bar.d -ofTest -H

 This dumps all the *.di files into the output directory ignoring the
 directory structure.

 Is there some rational for it being this way?
 Wouldn't it be much more useful if directories were preserved?

    Bit


As far as I know, every other similar tool works the same way:

   $ gcc -c foo/foo.c

foo.o is outputted to the current directory.

The common solution that I know is to use makefiles to manage file 
dependencies. For example, the rule for .di specifies the file that it 
depends on (the .d corresponding to that .di) and 'make' handles it 
automatically.

Ali
May 09 2015
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Sat, 09 May 2015 21:09:33 -0400, Ali =C7ehreli <acehreli yahoo.com> w=
rote:


 On 05/09/2015 07:01 AM, bitwise wrote:

 ./main.d
 ./pack/foo.d
 ./pack/sub/bar.d

 dmd main.d pack/foo.d pack/sub/bar.d -ofTest -H

 This dumps all the *.di files into the output directory ignoring the
 directory structure.

 Is there some rational for it being this way?
 Wouldn't it be much more useful if directories were preserved?

    Bit


 As far as I know, every other similar tool works the same way:

    $ gcc -c foo/foo.c

 foo.o is outputted to the current directory.

 The common solution that I know is to use makefiles to manage file  =



 dependencies. For example, the rule for .di specifies the file that it=


  =


 depends on (the .d corresponding to that .di) and 'make' handles it  =



 automatically.

 Ali


I'm not sure I understand what you mean, but my point is, that as is, th=
is  =

feature of DMD is broken.

If people are successfully using dmd -H right now, they must not be usin=
g  =

packages, or their 'package.di' files would be overwritten. So I'm  =

wondering if I should do a PR to fix this.

   Bit
May 09 2015
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 05/09/2015 06:18 PM, bitwise wrote:


 I'm not sure I understand what you mean, but my point is, that as is,
 this feature of DMD is broken.


Arguably broken but compared to other tools it behaves in the same way.


 If people are successfully using dmd -H right now, they must not be
 using packages, or their 'package.di' files would be overwritten. So I'm
 wondering if I should do a PR to fix this.


The same can be said for gcc and other tools as well and it is not 
limited to package.d: any two files with the same name would have the 
same problem, meaning that the particular build system that puts those 
unrelated files in the same directory is in error, not the compiler.

The way I see it, the right solution is to provide correct paths to dmd 
for each file.

Ali
May 09 2015
parent bitwise <bitwise.pvt gmail.com> writes:
On Sat, 09 May 2015 21:31:20 -0400, Ali =C7ehreli <acehreli yahoo.com> w=
rote:


 On 05/09/2015 06:18 PM, bitwise wrote:

  > I'm not sure I understand what you mean, but my point is, that as i=


s,

  > this feature of DMD is broken.

 Arguably broken but compared to other tools it behaves in the same way=


.

dmd is the reference compiler, so the other compilers _should_ behave th=
e  =

same way. I suppose this could be considered arguable, because there may=
  =

exist a build system(I don't know of any) that will plan for this defect=
  =

and generate the headers one by one while building out the directories o=
n  =

it's own. So my options are to be burdened by some build tool I don't ne=
ed  =

just for this one feature, or waste my time writing custom build scripts=
  =

when this is something that can trivially be implemented in the compiler=
.

In the interest of not breaking existing build systems, an additional fl=
ag  =

could be added to dmd. Something like -Hp for "preserve paths when  =

outputting headers".


 The way I see it, the right solution is to provide correct paths to dm=


d  =


 for each file.


Given a build command like the one below, I don't understand how you wou=
ld  =

achieve this without building the files one by one.


 ./main.d
 ./pack/foo.d
 ./pack/sub/bar.d
  dmd main.d pack/foo.d pack/sub/bar.d -ofTest -H


The output is:

./main.di
./foo.di
./bar.di

And it should be(or could be, if -Hp was passed)

./main.di
./pack/foo.di
./pack/sub/bar.di

   Bit
May 09 2015
prev sibling parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Sat, 09 May 2015 10:01:25 -0400
schrieb bitwise <bitwise.pvt gmail.com>:


 ./main.d
 ./pack/foo.d
 ./pack/sub/bar.d
 
 dmd main.d pack/foo.d pack/sub/bar.d -ofTest -H
 
 This dumps all the *.di files into the output directory ignoring the  
 directory structure.
 
 Is there some rational for it being this way?
 Wouldn't it be much more useful if directories were preserved?
 
    Bit


I agree, D modules are hierarchical and sometimes share the
same base name. Header files are very important to short
circuit the transitive import chain. We've already seen it in
Phobos. Many modules ends up pulling in most of Phobos, and
those are library modules.

Now imagine there was an operating system like Linux/GNU with
all libraries written in D and no .di files: The program entry
point in a project the size of OpenOffice would pull in so many
imports that no super computer in the world could compile it
given CTFE memory use: all of the project files, font library,
math library, image library, entire GUI toolkit, Java binding,
accessibility library, RPC library, configuration library,
OpenGL, audio, GhostScript and PDF, Python, XML, MS Office
file type readers, ICU, SSL, Phobos, X11 or similar, color
management, printing, zip, databases, ...

We often shun .di files, thinking of them as C++ relics, but
the big difference is that we don't write them. The compiler
generates them and they are still a key to a successful
eco-system with huge projects, just the way they are in C/C++.

One thing I'm missing is a way to tell the compiler or a build
tool which symbols or modules are part of the library API.
There is no sense in generating an "internal.di" and similar
files that are never supposed to be used by the outside.
(They'd also be listed unnecessarily in import
auto-completions of Mono-D and similar IDEs.)

-- 
Marco
May 10 2015
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Sun, 10 May 2015 04:20:45 -0400, Marco Leise <Marco.Leise gmx.de> wrote:

 I agree, D modules are hierarchical and sometimes share the
 same base name. Header files are very important to short
 circuit the transitive import chain. We've already seen it in
 Phobos. Many modules ends up pulling in most of Phobos, and
 those are library modules.

 Now imagine there was an operating system like Linux/GNU with
 all libraries written in D and no .di files: The program entry
 point in a project the size of OpenOffice would pull in so many
 imports that no super computer in the world could compile it
 given CTFE memory use: all of the project files, font library,
 math library, image library, entire GUI toolkit, Java binding,
 accessibility library, RPC library, configuration library,
 OpenGL, audio, GhostScript and PDF, Python, XML, MS Office
 file type readers, ICU, SSL, Phobos, X11 or similar, color
 management, printing, zip, databases, ...

 We often shun .di files, thinking of them as C++ relics, but
 the big difference is that we don't write them. The compiler
 generates them and they are still a key to a successful
 eco-system with huge projects, just the way they are in C/C++.

 One thing I'm missing is a way to tell the compiler or a build
 tool which symbols or modules are part of the library API.
 There is no sense in generating an "internal.di" and similar
 files that are never supposed to be used by the outside.
 (They'd also be listed unnecessarily in import
 auto-completions of Mono-D and similar IDEs.)


I'm not sure I understand what you're trying to say.

   Bit
May 10 2015
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Sun, 10 May 2015 10:37:13 -0400
schrieb bitwise <bitwise.pvt gmail.com>:


 I'm not sure I understand what you're trying to say.
 
    Bit


I'm trying to hijack your thread and communicate that .di
files are important for D since they allow the compiler to
stop recursively importing modules at some point rather than
eagerly importing until "the world" is in compiler memory.

-- 
Marco
May 11 2015
parent reply "weaselcat" <weaselcat gmail.com> writes:
On Monday, 11 May 2015 at 12:46:59 UTC, Marco Leise wrote:

 Am Sun, 10 May 2015 10:37:13 -0400
 schrieb bitwise <bitwise.pvt gmail.com>:


 I'm not sure I understand what you're trying to say.
 
    Bit


 I'm trying to hijack your thread and communicate that .di
 files are important for D since they allow the compiler to
 stop recursively importing modules at some point rather than
 eagerly importing until "the world" is in compiler memory.


Is this something that already works?
May 11 2015
parent Marco Leise <Marco.Leise gmx.de> writes:
Am Mon, 11 May 2015 12:50:48 +0000
schrieb "weaselcat" <weaselcat gmail.com>:


 On Monday, 11 May 2015 at 12:46:59 UTC, Marco Leise wrote:

 Am Sun, 10 May 2015 10:37:13 -0400
 schrieb bitwise <bitwise.pvt gmail.com>:


 I'm not sure I understand what you're trying to say.
 
    Bit


 I'm trying to hijack your thread and communicate that .di
 files are important for D since they allow the compiler to
 stop recursively importing modules at some point rather than
 eagerly importing until "the world" is in compiler memory.


 
 Is this something that already works?


The idea is very simple. You have some .d file like this:

import a;
import internal;
import std.algorithm;

SomeStruct foo()
{
	return SomeStruct(min(1, 2));
}

The compiler loads and analyzes 'a', 'internal',
'std.algorithm' and publically imported modules. If there are
multiple declarations of SomeStruct it errors out. If
SomeStruct is only defined in 'a', a .di file would reduce to
this:

import a;

SomeStruct foo();

So we remove the dependency on that Phobos module
'std.algorithm' for 'min' and module 'internal'.

Ideally only the public API of a module remains in a .di file,
but we need to know more, like what size it is and if it has a
dtor. There's also compile-time reflection to iterate all the
(private and public) fields in a struct or class making it
harder to completely remove internal modules from the public
API:

import internal_types;

struct IUseInternalStructures
{
private:
	SomeInternalType data;
	uint handle;
        float area;
}

Although for most purposes it is enough to know the .init of
this struct and that it has no dtor (not even one from
SomeInternalData), we cannot really remove the import here in
the .di file.

-- 
Marco
May 11 2015