www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - CMake with D support early snapshot

reply "Trent Forkert" <trentforkert gmail.com> writes:
Hello all,

Given the recent chatter and movement on CMake D support, I've 
decided to go public with a project of mine earlier than I had 
intended.

Before I go further, a request: do *not* post this to 
HN/reddit/etc just yet. It is still in early stages and an 
upstream CMake 3.0 release (without D support) is incoming. 
Announcing this far and wide will only yield confusion at this 
time.

So, what is this?

It's CMake, with various modifications to work toward making D a 
first-class citizen of the CMake world.

While other projects exist that attempt to add D support, they 
all do so without touching CMake's C++ sources. This means that 
they will inevitably fall short of the mark.

Additionally, when I first started toying with this several 
months ago, there were a lot of implementation/design issues in 
the existing projects, that went against the way CMake's 
internals expect things to be done. I'm not sure how the current 
scene is in that regard.

Status

* GDC is fully supported, as is DMD master
* LDC and older DMD's will work for simpler projects, but won't 
handle linking external libraries at the moment. I had a hack 
workaround for this before, but recently removed it when 
restructuring things a bit.
* 32-bit DMD on Windows can't really be used for a C/C++/D mixed 
project right now because of problems I'm having with Optlink
* VisualD generation works (tested on VS 2010 and VS 2012)
* Makefile generation (and similar generators) work
* Works on Windows and Linux. OS X ought to work, but is 
untested, as I don't have my OS X dev environment set up at the 
moment
* D is listed on the CMake Qt GUI, which is nice

Github: https://github.com/trentforkert/cmake
Wiki: https://github.com/trentforkert/cmake/wiki
Binaries: 
https://drive.google.com/folderview?id=0B5vzzNch4TtET09HM0NLWURKV1U&usp=drive_web#list


As is tradition here, destroy!

  - Trent
Mar 25 2014
next sibling parent Ben Boeckel <mathstuf gmail.com> writes:
On Tue, Mar 25, 2014 at 17:50:30 +0000, Trent Forkert wrote:

 It's CMake, with various modifications to work toward making D a
 first-class citizen of the CMake world.
 
 While other projects exist that attempt to add D support, they all do
 so without touching CMake's C++ sources. This means that they will
 inevitably fall short of the mark.


For full-featured support, I agree.


 Additionally, when I first started toying with this several months
 ago, there were a lot of implementation/design issues in the existing
 projects, that went against the way CMake's internals expect things
 to be done. I'm not sure how the current scene is in that regard.


The compiler detection and such hasn't changed in CMake much (AFAICT),
so if they weren't working as expected before, nothing has changed.


 * GDC is fully supported, as is DMD master
 * LDC and older DMD's will work for simpler projects, but won't
 handle linking external libraries at the moment. I had a hack
 workaround for this before, but recently removed it when
 restructuring things a bit.
 * 32-bit DMD on Windows can't really be used for a C/C++/D mixed
 project right now because of problems I'm having with Optlink
 * VisualD generation works (tested on VS 2010 and VS 2012)
 * Makefile generation (and similar generators) work
 * Works on Windows and Linux. OS X ought to work, but is untested, as
 I don't have my OS X dev environment set up at the moment
 * D is listed on the CMake Qt GUI, which is nice


Much more comprehensive than my attempts :) . The depfile support would
be nice to add in so that dependencies are calculated properly. Make
will be left out with DMD/LDC, but GDC will work with Ninja and Make
already (LDC/DMD+Ninja has a pull request with Ninja).

I'll try it out when I get a chance.

--Ben
Mar 25 2014
prev sibling next sibling parent reply Ben Boeckel <mathstuf gmail.com> writes:
On Tue, Mar 25, 2014 at 14:12:45 -0400, Ben Boeckel wrote:

 I'll try it out when I get a chance.


I've pushed some fixes to my CMake fork to work with GDC on Linux. I've
also fixed up some things I saw and opened a bug on your fork (which
isn't linked with Kitware/CMake, so PRs are unavailable…an unfortunate
GitHub limitation; I've contacted support about this). I now have it
working for a "dummy" project.

I've also commented on the branch as a whole on that issue as well.

--Ben
Mar 25 2014
parent reply "Trent Forkert" <trentforkert gmail.com> writes:
On Wednesday, 26 March 2014 at 06:17:57 UTC, Ben Boeckel wrote:

 On Tue, Mar 25, 2014 at 14:12:45 -0400, Ben Boeckel wrote:

 I'll try it out when I get a chance.


 I've pushed some fixes to my CMake fork to work with GDC on 
 Linux. I've
 also fixed up some things I saw and opened a bug on your fork 
 (which
 isn't linked with Kitware/CMake, so PRs are unavailable…an 
 unfortunate
 GitHub limitation; I've contacted support about this). I now 
 have it
 working for a "dummy" project.

 I've also commented on the branch as a whole on that issue as 
 well.

 --Ben


As I told Ben on Github, I didn't realize CMake had a github 
mirror. So, I waved my magic wand and made trentforkert/cmake a 
github fork of Kitware/CMake so that PRs should work now.

The original (non-github-fork) repo is now preserved at 
trentforkert/cmake_old, just in case we need it for something.

  - Trent
Mar 26 2014
parent reply "Trent Forkert" <trentforkert gmail.com> writes:
I have a quandary you might be able to help with, Ben.

I've been working on moving dependency resolution things to 
cmDependsD.cxx, and I've got it mostly working.

However, if we have gdc produce make-style dependencies (which 
will still require processing to get CMake to be happy), that 
requires me to put a special case in the C++ that I'd rather not 
have.

The way I see it, there are three possibilities here:
1. Keep gdc producing make-style deps, and deal with that as 
needed.
2. Have gdc produce dmd-style deps, and let the patched Ninja you 
mentioned use that.
3. Have a separate flag registered for the two different styles 
of dependencies.

I'm quite partial to number 3, as I could then use that to check 
which flags are defined for which style is supported, and act 
accordingly. I prefer that to hardcoding "gdc does this. Anything 
else does this other thing."

It should be noted that (AFAICT), the Ninja generator is the only 
thing that even considers ${CMAKE_DEPFILE_FLAGS_<lang>}. All the 
Makefile generators (basically everything but Ninja, VS, and 
possibly XCode) use the cmDepends system.

Thoughts?

  - Trent
Mar 26 2014
next sibling parent reply Ben Boeckel <mathstuf gmail.com> writes:
On Wed, Mar 26, 2014 at 22:52:43 +0000, Trent Forkert wrote:

 However, if we have gdc produce make-style dependencies (which will
 still require processing to get CMake to be happy), that requires me
 to put a special case in the C++ that I'd rather not have.


I'm fine with either, but make-style is preferred since it can exclude
system files (which I've asked LDC about supporting as well; should ask
dmd too).


 The way I see it, there are three possibilities here:
 1. Keep gdc producing make-style deps, and deal with that as needed.
 2. Have gdc produce dmd-style deps, and let the patched Ninja you
 mentioned use that.
 3. Have a separate flag registered for the two different styles of
 dependencies.
 
 I'm quite partial to number 3, as I could then use that to check
 which flags are defined for which style is supported, and act
 accordingly. I prefer that to hardcoding "gdc does this. Anything
 else does this other thing."


What about:

  4. Add depfile support to Makefile generators.

It seems that it shouldn't be *too* hard[1] to do. Obviously, dmd-style
deps will still have problems with make, but we could try and ask the
dmd and ldc upstream to at least support Make-style dependency files
(this would mean old dmd/ldc + make isn't supported, but at least Ninja
would be an option there). If that fails, some CMake code to generate
Make-style .d files from dmd-style shouldn't be too bad (some regex
matching and character escaping should do the trick).


 It should be noted that (AFAICT), the Ninja generator is the only
 thing that even considers ${CMAKE_DEPFILE_FLAGS_<lang>}. All the
 Makefile generators (basically everything but Ninja, VS, and possibly
 XCode) use the cmDepends system.


Yeah, seems so. If we could get rid of the cmDepends* stuff, that'd be
nice. I only see it included from Source/cmLocalUnixMakefileGenerator3.*
which means if the Makefile learns about depfiles, we can start removing
cmDepends* (yay!). I guess the IDE generators rely on IDE magic to get
dependencies correct?

So...it looks like this is what we're aiming for:

  - DMD/LDC
    - Support make-style depfiles (optional; unlikely)
    - Support filtering out excess dependencies (private import, system
      files) (preferred)
  - Ninja
    - dmd depfile support (preferred; no comment from martine yet)
  - CMake
    - Add depfile support to Make (preferred)
    - Add dmd -> make depfile translator  (likely necessary[2])

--Ben

[1]http://stackoverflow.com/a/16969086
[2]The dmd-depfile format probably won't fly with make upstream since
make-depfiles aren't actually a thing with make, but gcc.
Mar 26 2014
parent reply "Trent Forkert" <trentforkert gmail.com> writes:
On Wednesday, 26 March 2014 at 23:17:31 UTC, Ben Boeckel wrote:

 What about:

   4. Add depfile support to Makefile generators.


That's basically what I'm doing, though only in the context of D.
cmDependsD::WriteDependencies() gets called for every D object,
and has the D compiler produce a dmd-style depfile of the 
relevant sources.
It then scans each of the

     foo (.../foo.d) : public : object (.../object.di)

lines, pulling the filepaths from the () to construct the file
depends.make in the correct CMakeFiles/<target>.dir, which 
contains a list
of lines like:

     .../foo.d.o: .../foo.d
     .../foo.d.o: .../object.di


 Yeah, seems so. If we could get rid of the cmDepends* stuff, 
 that'd be
 nice. I only see it included from 
 Source/cmLocalUnixMakefileGenerator3.*
 which means if the Makefile learns about depfiles, we can start 
 removing
 cmDepends* (yay!). I guess the IDE generators rely on IDE magic 
 to get
 dependencies correct?


Y'know, I didn't really know. So, I spent longer than I'd care to 
admit in a find|grep loop examining the source. Raw notes as I 
progressed are appended to the bottom of this post, but the 
takeaway is that there are only really four kinds of generators:

* Makefiles
* Ninja (Some depfile solution)
* Visual Studio (maybe VisualD will help handle things for us)
* XCode (absolutley no idea)

Everything else is implemented on top of those. All the non-VS 
non-XC generators ultimately go through cmLocalUnixMakefile3 and 
cmDepends, or they go through Ninja.

So, we'll still need to get Ninja, VS, and XCode dependency 
separately, but cmDepends handles the all other generator's 
dependency resolution.



 So...it looks like this is what we're aiming for:

   - DMD/LDC
     - Support make-style depfiles (optional; unlikely)
     - Support filtering out excess dependencies (private 
 import, system
       files) (preferred)


Right now I'm doing direct dependencies only, so if module foo 
imports std.stdio, it adds a dependency on stdio.di, but nothing 
else. I think we might actually want full (recursive) dependency 
listings though, because of how templates work. Alternatively, I 
can just check the module names and filter out std.* and similar.


   - Ninja
     - dmd depfile support (preferred; no comment from martine 
 yet)
   - CMake
     - Add depfile support to Make (preferred)
     - Add dmd -> make depfile translator  (likely necessary[2])


Like I said above, that's what I'm doing in cmDependsD at the 
moment. Since Ninja, VS and XCode all do their own thing, I think 
its fine to leave that translator inside cmDependsD.


 --Ben

 [1]http://stackoverflow.com/a/16969086
 [2]The dmd-depfile format probably won't fly with make upstream 
 since
 make-depfiles aren't actually a thing with make, but gcc.


Whatever CMake does with their depends.make (I didn't go that 
deep), it works on all of the different Makefile generators CMake 
supports.

  - Trent

What follows is my notes as I sorted things out:

The method 
cmLocalUnixMakefileGenerator3::WriteDependLanguageInfo() calls 
into the cmDepends* system. This method is called by 
cmMakefileTargetGenerator::WriteTargetDependRules(). This in turn 
is called by WriteRuleFiles in 
cmMakefileExecutableTargetGenerator and 
cmMakefileLibraryTargetGenerator, which is called by 
cmLocalUnixMakefileGenerator3::Generate().

The cmLUMG3 will be set as the local generator for every 
cmGlobal*Generator except for Kdevelop, Ninja, VisualStudio and 
XCode.

Upon inspection:

* Kdevelop uses the Unix Makefile generator behind the scenes
* Ninja does its own thing
* VisualStudio does its own thing
* XCode appears to do its own thing

There are also cmExtra*Generators, for completeness, I'll inspect 
them too.

* CodeBlocks: uses another Makefile generator
* CodeLight: uses a Makefile generator or Ninja
* Eclipse: uses a Makefile generator or Ninja
* Kate: Makefile or Ninja
* Sublime: Makefile or Ninja
Mar 26 2014
parent reply Ben Boeckel <mathstuf gmail.com> writes:
On Thu, Mar 27, 2014 at 00:38:05 +0000, Trent Forkert wrote:

 On Wednesday, 26 March 2014 at 23:17:31 UTC, Ben Boeckel wrote:

  4. Add depfile support to Makefile generators.


 
 That's basically what I'm doing, though only in the context of D.


No, I meant, using the DEPFLAGS during the make build (like ninja does)
rather than doing it at configure time.


 Y'know, I didn't really know. So, I spent longer than I'd care to
 admit in a find|grep loop examining the source.


'git grep' is your friend :) . ('git grep -l cmDepends')


 So, we'll still need to get Ninja, VS, and XCode dependency
 separately, but cmDepends handles the all other generator's
 dependency resolution.


Ninja is fine with the 'deps = dmd' solution. VS will need VisualD I
imagine (probably fine to require), and XCode will need someone who
cares enough to look into it (not me...) ;) . A cursory search
finds a[1] few[2] solutions[3], but that's about the limit of my
attention span for XCode+D right now.


 Right now I'm doing direct dependencies only, so if module foo
 imports std.stdio, it adds a dependency on stdio.di, but nothing
 else. I think we might actually want full (recursive) dependency
 listings though, because of how templates work. Alternatively, I can
 just check the module names and filter out std.* and similar.


Recursive public (non-static?) imports is the proper, minimal way to do
it; you may be getting an import implicitly via another forwarding
module.


 Like I said above, that's what I'm doing in cmDependsD at the moment.
 Since Ninja, VS and XCode all do their own thing, I think its fine to
 leave that translator inside cmDependsD.


I think I was unclear: I'd like to see the dependency resolution done at
build time, not configure time. That's why there'd be something like:

    %.o: %.d
    %.d: %.dmd.d
        -cmake -DINPUT="$<" -DOUTPUT="$>" -P cmake_root/CMakeDMDToMakeDeps.cmake
    -include $(wildcard *.d)

or something in the Makefile if depfile support is detected (that
snippet is a circular dependency, but maybe the CMakeDMDToMakeDeps.cmake
could be placed into the %.o recipe after the compile step and make
knows nothing of .dmd.d files. I'll have to ask Brad why Make was never
converted off of cmDependsC when DEPFILE support was added.


 The method cmLocalUnixMakefileGenerator3::WriteDependLanguageInfo()
 calls into the cmDepends* system. This method is called by
 cmMakefileTargetGenerator::WriteTargetDependRules(). This in turn is
 called by WriteRuleFiles in cmMakefileExecutableTargetGenerator and
 cmMakefileLibraryTargetGenerator, which is called by
 cmLocalUnixMakefileGenerator3::Generate().


My suggestion is to remove the cmDepends calls with rules to chain up a
.d depends rule supplement when available.


 There are also cmExtra*Generators, for completeness, I'll inspect
 them too.
 
 * CodeBlocks: uses another Makefile generator
 * CodeLight: uses a Makefile generator or Ninja
 * Eclipse: uses a Makefile generator or Ninja
 * Kate: Makefile or Ninja
 * Sublime: Makefile or Ninja


My understanding (without looking): the extra generators basically do
some scaffolding to write IDE files which just tell the IDE how to run
the internal generator, possibly with a list of targets, sources, and
whatnot for those which can't manually inspect the build files
themselves.

--Ben

[1]http://dblog.aldacron.net/2010/03/22/d-for-xcode-now-with-dmd-support/
[2]http://michelf.ca/projects/d-for-xcode/
[3]http://prowiki.org/wiki4d/wiki.cgi?EditorSupport/AppleXcode
Mar 26 2014
parent reply "Trent Forkert" <trentforkert gmail.com> writes:
On Thursday, 27 March 2014 at 01:16:57 UTC, Ben Boeckel wrote:

 On Thu, Mar 27, 2014 at 00:38:05 +0000, Trent Forkert wrote:

 On Wednesday, 26 March 2014 at 23:17:31 UTC, Ben Boeckel wrote:

  4. Add depfile support to Makefile generators.


 
 That's basically what I'm doing, though only in the context of 
 D.


 No, I meant, using the DEPFLAGS during the make build (like 
 ninja does)
 rather than doing it at configure time.


I've tested this by creating a wrapper script around dmd to log 
calls to it in a file. Using that, I can confirm that cmDependsD 
does nothing at configure time. Granted it didn't refresh the 
deps file when I updated my D code, but...

I've also tested with a simple C project (to confirm I no 
D-related bugs get in the way) that depend.make is generated and 
updated at build time. After building, I went and updated my C 
file to point to a new header it hadn't touched before, re-ran 
make (not cmake), and checked that depend.make listed now the 
header as a dependency (it did). To be extra sure, I reverted the 
change to the C code, and ran make again. depend.make was updated 
to remove the header dependency.

The reason cmDependsD didn't update deps at build-time when I 
tried appears to be a matter of implementing another method I 
haven't got to yet.

You're right that examine_d_source only worked at configure time, 
but cmDepends works at build time.


 Ninja is fine with the 'deps = dmd' solution.


I will go with that then.


 VS will need VisualD I
 imagine (probably fine to require)


It absolutely does.


 and XCode will need someone who
 cares enough to look into it (not me...) ;) . A cursory search
 finds a[1] few[2] solutions[3], but that's about the limit of my
 attention span for XCode+D right now.


Yeah, I'm set up to work on Windows and Linux at the moment. 
Despite using a Macbook, I almost never touch OS X ^_^.


 Recursive public (non-static?) imports is the proper, minimal 
 way to do
 it; you may be getting an import implicitly via another 
 forwarding
 module.


 Like I said above, that's what I'm doing in cmDependsD at the 
 moment.
 Since Ninja, VS and XCode all do their own thing, I think its 
 fine to
 leave that translator inside cmDependsD.


 I think I was unclear: I'd like to see the dependency 
 resolution done at
 build time, not configure time. That's why there'd be something 
 like:

     %.o: %.d
     %.d: %.dmd.d
         -cmake -DINPUT="$<" -DOUTPUT="$>" -P 
 cmake_root/CMakeDMDToMakeDeps.cmake
     -include $(wildcard *.d)


I can do something like that if its needed, I think. 
cmDependsFortran appears to generate dependency rules that call 
CMake, and 'cmake -E cmake_depends ...' is the command that is 
actually used to generate a depend.make. But, as I said above, 
cmDepends is a build-time thing. It's more obtuse about it than 
the Ninja generator is, but it is still build-time.


 My understanding (without looking): the extra generators 
 basically do
 some scaffolding to write IDE files which just tell the IDE how 
 to run
 the internal generator, possibly with a list of targets, 
 sources, and
 whatnot for those which can't manually inspect the build files
 themselves.


Correct.

  - Trent
Mar 26 2014
parent Ben Boeckel <mathstuf gmail.com> writes:
On Thu, Mar 27, 2014 at 01:52:00 +0000, Trent Forkert wrote:

 I've tested this by creating a wrapper script around dmd to log calls
 to it in a file. Using that, I can confirm that cmDependsD does
 nothing at configure time. Granted it didn't refresh the deps file
 when I updated my D code, but...
 
 I've also tested with a simple C project (to confirm I no D-related
 bugs get in the way) that depend.make is generated and updated at
 build time. After building, I went and updated my C file to point to
 a new header it hadn't touched before, re-ran make (not cmake), and
 checked that depend.make listed now the header as a dependency (it
 did). To be extra sure, I reverted the change to the C code, and ran
 make again. depend.make was updated to remove the header dependency.
 
 The reason cmDependsD didn't update deps at build-time when I tried
 appears to be a matter of implementing another method I haven't got
 to yet.


Well, that shows my ignorance of now cmDepends works. I now see that it
is hooked through 'cmake -E cmake_depends' and not run during configure
(as it living within a generator led me to believe). So the goal of
using the compiler for this rather than C++ code would be to not have
the C++ code in the first place. I'd guess it'd also have make build a
lot faster (CMake performance being another recent and ongoing focus of
mine)...


 Yeah, I'm set up to work on Windows and Linux at the moment. Despite
 using a Macbook, I almost never touch OS X ^_^.


XCode updates are a cause for dread from CMake developers. Lots of
things always break :( . I don't know if I'll be able to find any
willing victims (currently, Ninja bails ASAP on Fortran; making XCode do
so for D is probably workable until someone cares enough to write
patches).


 I can do something like that if its needed, I think. cmDependsFortran
 appears to generate dependency rules that call CMake, and 'cmake -E
 cmake_depends ...' is the command that is actually used to generate a
 depend.make. But, as I said above, cmDepends is a build-time thing.
 It's more obtuse about it than the Ninja generator is, but it is
 still build-time.


I'll have to ask Brad what he perfers for cmDepends* going forward. I
don't know the full ramifications of removing it.

--Ben
Mar 26 2014
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On 26 March 2014 23:17, Ben Boeckel <mathstuf gmail.com> wrote:

 On Wed, Mar 26, 2014 at 22:52:43 +0000, Trent Forkert wrote:

 However, if we have gdc produce make-style dependencies (which will
 still require processing to get CMake to be happy), that requires me
 to put a special case in the C++ that I'd rather not have.


 I'm fine with either, but make-style is preferred since it can exclude
 system files (which I've asked LDC about supporting as well; should ask
 dmd too).


 The way I see it, there are three possibilities here:
 1. Keep gdc producing make-style deps, and deal with that as needed.
 2. Have gdc produce dmd-style deps, and let the patched Ninja you
 mentioned use that.
 3. Have a separate flag registered for the two different styles of
 dependencies.

 I'm quite partial to number 3, as I could then use that to check
 which flags are defined for which style is supported, and act
 accordingly. I prefer that to hardcoding "gdc does this. Anything
 else does this other thing."


 What about:

   4. Add depfile support to Makefile generators.

 It seems that it shouldn't be *too* hard[1] to do. Obviously, dmd-style
 deps will still have problems with make, but we could try and ask the
 dmd and ldc upstream to at least support Make-style dependency files
 (this would mean old dmd/ldc + make isn't supported, but at least Ninja
 would be an option there). If that fails, some CMake code to generate
 Make-style .d files from dmd-style shouldn't be too bad (some regex
 matching and character escaping should do the trick).


Ain't the dmd-style deps legacy from a failed D build system?

It's just kept around cause more recent/current build systems /may/
find it's output useful.
Mar 27 2014
prev sibling parent reply Ben Boeckel <mathstuf gmail.com> writes:
On Thu, Mar 27, 2014 at 17:21:45 +0000, Iain Buclaw wrote:

 Ain't the dmd-style deps legacy from a failed D build system?
 
 It's just kept around cause more recent/current build systems /may/
 find it's output useful.


Well, it's what we have currently. We could add support for
gcc-compatible depfiles, but then you're stuck with only new versions of
DMD or LDC if you use CMake. Adding support to CMake and Ninja is much
easier than upgrading the compiler since they're much smaller and more
easily upgradable (fewer moving parts, lower chance of getting subtle
new behavior[1], more reasonable to expect or ask users to have an
up-to-date version, etc.).

Getting the -M flags recognized and implemented in DMD and LDC are good
goals to shoot for, but we're constrained by the tools we have
available…

--Ben

[1]CMake is (ideally) loud when this occurs through its policy system.m
Mar 27 2014
parent "Trent Forkert" <trentforkert gmail.com> writes:
FWIW, I just pushed my implementation of cmDependsD[1], and found 
dmd-style deps much nicer to work with. I used the >=2.064 
ability to do

     dmd -c -o- -deps foo.d

since that also contains file imports, which are technically 
dependencies. I missed when/if this was discussed on the list, 
but it seems that form of dependency output at least is intended 
to stay, even if it forces us to drop support for older compiler 
versions.

  - Trent

[1]https://github.com/trentforkert/cmake/commit/bbaa6b1f82d6c55154a95d1995007e408c31103e
Mar 27 2014
prev sibling next sibling parent reply "w0rp" <devw0rp gmail.com> writes:
This is all very interesting. I have two questions.

1. Do you plan to get this merged back into CMake proper?
2. Could I use it to handle my weird build process?

At the moment I am working on something which requires
a strange build process, and I have been struggling to
implement it with existing build tools. First it needs to
compile and run a D program, which generates .d and .cpp files. 
Then,
it needs to take all of those D and C++ sources and build one 
library
out of them. This all works, I just need to get a build tool that
does it all in one 'make' command or similar.
Apr 01 2014
parent reply "Trent Forkert" <trentforkert gmail.com> writes:
On Tuesday, 1 April 2014 at 12:46:54 UTC, w0rp wrote:

 This is all very interesting. I have two questions.

 1. Do you plan to get this merged back into CMake proper?


Ideally, yes. But things are still in early stages, and that is 
up to people who aren't me. Perhaps Ben can shed light on if 
Kitware wants something like this upstream. However, I 
effectively need CMake for a personal project, so I'll continue 
maintaining this even if Kitware refuses an upstream merge. Once 
things stabilize a bit more, I'll keep up-to-date binaries 
published, too.


 2. Could I use it to handle my weird build process?

 At the moment I am working on something which requires
 a strange build process, and I have been struggling to
 implement it with existing build tools. First it needs to
 compile and run a D program, which generates .d and .cpp files. 
 Then,
 it needs to take all of those D and C++ sources and build one 
 library
 out of them. This all works, I just need to get a build tool 
 that
 does it all in one 'make' command or similar.


Yep, that sort of thing works:

CMakeLists.txt:

     cmake_minimum_required(VERSION 3.0)
     project(example D)

     add_executable(codegen codegen.d)
     add_custom_command(
         OUTPUT generated.d
         COMMAND codegen
     )
     add_library(generated generated.d)


codegen.d:
     import std.file;

     void main()
     {
         "generated.d".write(q{
             int myfunc()
             {
                 return 42;
             }
         });
     }

Then, to use CMake to generate Makefiles (or various other build 
systems) and build:

     mkdir ../path/to/builddir
     cd ../path/to/builddir
     cmake ../path/to/sourcedir
     make -j4

Luckily for you, an old pet project of mine does something 
similar to what you describe. I intend on reviving it in the near 
future, and moving from a hacky shell script build system to a 
CMake one. So, if something breaks, I'll be sure to fix it.

Be warned, however, that this project is still subject to 
significant changes. For instance, I'm in the process of 
replacing include_text_directories with 
include_directories(TEXT), and thus improving the way -J flags 
are managed. I'm sure other things will come up and need changed 
as we encounter bugs.

  - Trent
Apr 01 2014
parent Ben Boeckel <mathstuf gmail.com> writes:
On Tue, Apr 01, 2014 at 14:47:51 +0000, Trent Forkert wrote:

 On Tuesday, 1 April 2014 at 12:46:54 UTC, w0rp wrote:

1. Do you plan to get this merged back into CMake proper?


 
 Ideally, yes. But things are still in early stages, and that is up to
 people who aren't me. Perhaps Ben can shed light on if Kitware wants
 something like this upstream. However, I effectively need CMake for a
 personal project, so I'll continue maintaining this even if Kitware
 refuses an upstream merge. Once things stabilize a bit more, I'll
 keep up-to-date binaries published, too.


I believe it'd only be accepted if you (or someone else from the D
community) stick around and help with support. I'm the only one here who
seems to have an interest in D (and, most likely, mine is really only
here for as long as I'm working with the abagames ports I'm doing in my
free time). I doubt it'd be accepted if it is just a code dump onto
Kitware since CMake has some strict backwards compatibility requirements
and there are no D experts readily available.

--Ben
Apr 01 2014
prev sibling parent reply "Meta" <jared771 gmail.com> writes:
On Tuesday, 25 March 2014 at 17:50:32 UTC, Trent Forkert wrote:

 Hello all,

 Given the recent chatter and movement on CMake D support, I've 
 decided to go public with a project of mine earlier than I had 
 intended.

 Before I go further, a request: do *not* post this to 
 HN/reddit/etc just yet. It is still in early stages and an 
 upstream CMake 3.0 release (without D support) is incoming. 
 Announcing this far and wide will only yield confusion at this 
 time.

 So, what is this?

 It's CMake, with various modifications to work toward making D 
 a first-class citizen of the CMake world.

 While other projects exist that attempt to add D support, they 
 all do so without touching CMake's C++ sources. This means that 
 they will inevitably fall short of the mark.

 Additionally, when I first started toying with this several 
 months ago, there were a lot of implementation/design issues in 
 the existing projects, that went against the way CMake's 
 internals expect things to be done. I'm not sure how the 
 current scene is in that regard.

 Status

 * GDC is fully supported, as is DMD master
 * LDC and older DMD's will work for simpler projects, but won't 
 handle linking external libraries at the moment. I had a hack 
 workaround for this before, but recently removed it when 
 restructuring things a bit.
 * 32-bit DMD on Windows can't really be used for a C/C++/D 
 mixed project right now because of problems I'm having with 
 Optlink
 * VisualD generation works (tested on VS 2010 and VS 2012)
 * Makefile generation (and similar generators) work
 * Works on Windows and Linux. OS X ought to work, but is 
 untested, as I don't have my OS X dev environment set up at the 
 moment
 * D is listed on the CMake Qt GUI, which is nice

 Github: https://github.com/trentforkert/cmake
 Wiki: https://github.com/trentforkert/cmake/wiki
 Binaries: 
 https://drive.google.com/folderview?id=0B5vzzNch4TtET09HM0NLWURKV1U&usp=drive_web#list


 As is tradition here, destroy!

  - Trent


Is this ready to be posted to HN/Reddit, or still no-go?
Apr 01 2014
next sibling parent reply Ben Boeckel <mathstuf gmail.com> writes:
On Tue, Apr 01, 2014 at 20:11:02 +0000, Meta wrote:

 Is this ready to be posted to HN/Reddit, or still no-go?


No idea, it's Trent's work for the most part, so that's his call.
However, I would like to get this to the CMake list sometime soon
(within a month?) for feedback before too many start using it externally
(especially since getting it on the radar for 3.1 would be nice).

For example, it seems that there are new signatures that need vetting
(include_directories(TEXT)), some new INTERFACE properties which need
plumbed (text include directories at least, possibly DDoc stuff?), and
other minor things (cmDependsD not working for older compilers last I
looked, something needs to be done about the Ninja patch I made, removal
of unnecessary code in the Platform/ and Compiler/ files, probably
more).

--Ben
Apr 01 2014
parent reply "Trent Forkert" <trentforkert gmail.com> writes:
On Wednesday, 2 April 2014 at 03:34:14 UTC, Ben Boeckel wrote:

 However, I would like to get this to the CMake list sometime 
 soon
 (within a month?) for feedback before too many start using it 
 externally
 (especially since getting it on the radar for 3.1 would be 
 nice).


I can probably manage that. I'd also rather it go through CMake 
vetting before HN/reddit vetting, as I feel like the former would 
be a lot more productive than the latter.


 For example, it seems that there are new signatures that need 
 vetting
 (include_directories(TEXT)), some new INTERFACE properties 
 which need
 plumbed (text include directories at least, possibly DDoc 
 stuff?),


My understanding of INTERFACE properties is that they are used 
when dealing with importing/exporting targets. If this is the 
case, I'm not sure of a case when text includes would need to be 
provided through this. Text imports work at compile time, 
literally putting a string of the contents of the specified file 
where the import() was.


 and
 other minor things (cmDependsD not working for older compilers 
 last I
 looked,


Yeah, I'm not sure there's much I can do about that. I have to 
choose between scanning textually imported files as dependencies 
(which they are) or supporting older compilers. Granted, the 
Ninja dependency resolution doesn't register text imports, since 
I have to specify the filename in the arguments. "-deps > 
<DEPFILE>" could work, but we would have to ensure that it comes 
last on the command line. "-deps=<DEPFILE>" doesn't output all 
the same information as "-deps > <DEPFILE>". Same holds for GDC 
and LDC equivalents.

  - Trent
Apr 01 2014
parent Ben Boeckel <mathstuf gmail.com> writes:
On Wed, Apr 02, 2014 at 03:59:47 +0000, Trent Forkert wrote:

 My understanding of INTERFACE properties is that they are used when
 dealing with importing/exporting targets. If this is the case, I'm
 not sure of a case when text includes would need to be provided
 through this. Text imports work at compile time, literally putting a
 string of the contents of the specified file where the import() was.


They're also used within a project. This would be required if, e.g., a
project installs modules intended to be text-imported and installs them
to somewhere like /usr/include/d/_text/path/to/module.d. Granted, I have
no idea if this even makes sense, but that would be one example :) .
There also may be things like DDoc flags if cross-referencing is
supported (such as where to look for any equivalent of Doxygen's .tag
files).


 Yeah, I'm not sure there's much I can do about that. I have to choose
 between scanning textually imported files as dependencies (which they
 are) or supporting older compilers. Granted, the Ninja dependency
 resolution doesn't register text imports, since I have to specify the
 filename in the arguments. "-deps > <DEPFILE>" could work, but we
 would have to ensure that it comes last on the command line.


So I talked to Brad about the rationale behind the cmDepends files:

  - Fortran requires it because Fortran is Fortran[1];
  - Java is lacking compiler support; and
  - C is supported because it is considerably faster than the compiler
    (it looks at a target-at-a-time rather than per-TU so headers
    included multiple times are parsed once). One downside is that #if
    is completely ignored and if CMake can't find a file it assumes the
    build is sane and skips ignores the #include directive[2].

One way Brad suggested is to do this is to overload the (internal) -E
cmake_depends command to do execute the D compiler internally, read the
depfiles over a pipe and write a (Make-format) depfile itself. This
would absolve Ninja from needing to read the DMD depfile format (at the
expense of any non-CMake build system not also doing the work), but also
mean that make VERBOSE=1 doesn't have the actual compilation command
verbatim for easy inspection. It also means we're not executing the
compiler then executing CMake to execute the compiler again with
slightly different flags.

Honestly, I think supporting older, existing compilers is more important
because they are usually harder to upgrade and the build tool should not
dictate such things (to a point; complaining that GCC 1.0 is unsupported
is just being a troll at this point :D ).

Also, if you can support a single-shot per-target dependency scanning
like C, you'd likely beat the compiler anyways since you're only reading
duplicate files once.


 "-deps=<DEPFILE>" doesn't output all the same information as "-deps >
 <DEPFILE>". Same holds for GDC and LDC equivalents.


That's…unfortunate :( . Though knowing where the -deps= writer lives in
DMD (the implementation of the import keyword rather than the lexer),
this probably doesn't surprise me.

--Ben

[1]At compile time, you may discover that you depend on a .mod file
generated by another source file's compilation, and not just the .f
file[3]. This means that A.f requiring B.mod from B.f can't be decided
until B is compiled before A. You can make it work with depfiles and
"compile iteratively until it discovers all the links", but that's
asinine to request of users. AFAIK, D does not do such silliness.
[2]Also unsupported is things like #include <MY_HEADER> under
-DMY_HEADER=stdio.h silliness. Yes, this works (look at boost headers
and search for BOOST_PP_FILENAME).
[3]http://compgroups.net/comp.lang.fortran/gfortran-and-m/2696739
Apr 01 2014
prev sibling parent "Trent Forkert" <trentforkert gmail.com> writes:
On Tuesday, 1 April 2014 at 20:11:03 UTC, Meta wrote:

 Is this ready to be posted to HN/Reddit, or still no-go?


I'm going to have to say no. A couple of major problems off the 
top of my head:

1. I've recently had to shrink the fully supported compilers list 
to DMD master and GDC 2.064 or greater, owing to a need for the 
compiler to handle shared libs passed directly to it and the 
-deps flag, respectively. LDC will be back on the list pending it 
gaining support for those things.
2. The major use case of mixing C/C++/D code is still 
fundamentally broken with Win32 DMD because of Optlink problems 
my poor Linux brain has yet to sort out. This is even more 
problematic, since the needed GDC version doesn't work on (or at 
least doesn't have a build for) Windows.
3. I'm still worried about creating confusion around CMake's 
pending 3.0 release.

I want it to be as friction-less as possible for C/C++ & CMake 
users to swap out their CMake install, add a D compiler, and 
start using D alongside C/C++ in their projects. Right now, such 
an effort would most likely be met with frustration pretty 
quickly. That frustration will end up aimed at D, even if it 
should be aimed at my work on this project. That won't benefit 
anybody.

I've got a list of things in my head I want to be fixed/supported 
before publishing this too far and wide. I'll work on moving 
those from my head to the bug tracker tonight, so I don't forget 
about them.

  - Trent
Apr 01 2014