www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - [GSoC] Header Generation for C/C++

reply Eduard Staniloiu <edi33416 gmail.com> writes:
Hi everyone,

At the end of May I've started working on my GSoC project, Header 
Generation for C/C++

Introduction
------------

In recent years, the D programming language has gained more and 
more
attention and existing C and C++ codebases are starting to 
incrementally integrate D
components.

In order to be able to use D components, a C or C++ interface to 
them must be
provided; in C and C++, this is done through header files. 
Currently, this process is entirely
manual, with the responsibility of writing a header file falling 
on shoulders of the
programmer. The larger the D portion of a codebase is, the more 
tedious the task
becomes: the best example being the DMD frontend which amounts to 
roughly ~310000
lines of code for which the C++ header files that are used by 
other backend
implementations (gdc, ldc) are manually managed. This is a 
repetitive, time consuming,
and rather boring task: this is the perfect job for a machine.

Project goal
------------

The deliverable of the project is a tool that automatically 
generates C and C++
header files from D module files. This can be achieved either by 
a library solution using
DMD as a Library, or by adding this feature in the DMD frontend 
through a compiler
switch.

The advantage of using DMD as a Library is that this wouldn’t 
increase the
complexity of compiler frontend codebase. The disadvantage will 
be that the user will be
required to install a third-party tool. Contrasting to this, the 
addition of the feature to the
frontend would result in a smoother integration with all the 
backends that use the DMD
frontend.

We have decided to go with the compiler switch approach.

One major milestone (and success marker) for the project is to 
automatically generate the
DMD frontend headers required by GDC/LDC.

Implementation strategy
-----------------------

The feature will require the implementation of a `Visitor` class 
that will traverse
the `AST` resulted after the parsing phase of the D code. For 
each top-level `Dsymbol`
(variable, function, struct, class etc.) the associated C++ 
correspondent will be written in
the header file.

The visitor will override the visiting methods of two types of 
nodes:
* Traversal nodes - these nodes simply implement the `AST` 
traversal logic:
`ModuleDeclaration`, `ScopeDeclaration`, etc.
* Output nodes - these nodes will implement the actual header 
generation logic:
`FuncDeclaration`, `StructDeclaration`, `VarDeclaration`, etc.

The header file will consist of declarations from `public extern 
(C++)` and `public extern (C)`
declarations/definitions from D modules.

Project status
--------------

I've started work [0] with the revival of DMD's PR 8591 [1], 
rebasing it and converting it into
a compiler switch.

The next step was to add a bunch of tests for the existing code, 
which revealed the following issues
* StructDeclaration:
   - align different than 1 does nothing; we should support 
align(n), where `n` in [1, 2, 4, 8, 16]
   - align(n): inside struct definition doesn’t add alignment, but 
breaks generation of default ctors
   - default ctors should be generated only if struct has no ctors
   - if a struct has ctors defined, only default ctor (S() { … }) 
should be generated to init members to default values, and the 
defined ctors must be declared
   - if a struct has a void initializer (`member = void`), the 
code segfaults
   - a struct should only define ctors if it’s `extern (C++)`

   As you can see, a bunch of the issues above are related to 
auto-generated ctor definitions.
   You might wonder "But why are there any definitions?"; the 
default ctors are there because D initializes
   member fields with a default value, while C and C++ do not, and 
this might break existing GDC/LDC behaviour.
   Ideally, we wouldn't generate any definitions, and if we can 
confirm the ctor definitions aren't needed, we'll remove them.

* ClassDeclaration:
   - align(n) does nothing. You can use align on classes in C++, 
though It is generally regarded as bad practice and should be 
avoided

* FuncDeclaration:
   - default arguments can be any valid D code, including a lambda 
function or a complex expression; we don't want to go down the 
path of generating C or C++ code, so for now default arguments 
get ignored.

* TemplateDeclaration:
   - templates imply code generation, so for now we don't support 
them

After writing the tests and understanding what are the issues, I 
got more comfortable with the codebase and I got on to the next 
(current) step: generating the DMD frontend header files from 
DMD's `*.d` frontend modules.

This took quite some time and sweat to get going: the major pain 
point here is given by templates.
There is `dmd/root/array.d` which has a templated `Array(T)` that 
is used throughout the codebase.
Since we don't support templates, we decided to keep the manual 
management of the `dmd/root/*.h` headers, but things aren't that 
simple.

The issue: while we don't explicitly pass in any of the 
`dmd/root/*.d` modules, some of them are processed during the 
semantic analysis phase, which will generate the definition of 
some `struct`s and `enum`s from `dmd/root/*.d` into the generated 
frontend header. When the generated header is used in conjunction 
with the manually managed header files from `dmd/root/*.h` a 
`struct`/`enum` re-definition error will be thrown by the 
compiler.

I kept scratching my head at how to avoid this, and in the end I 
went with explicitly ignoring anything that comes from a 
`dmd/root/*.d` module. Ideally, this special casing shouldn't be 
needed, and it should go away if we can add support for some 
simple D -> C++ templates.

So now, the current state of affairs is that the code in the PR 
[0] can link with and pass the `cxx-unittests`.

How to use it
-------------

The current PR [0] code is generating a `C++` header file out of 
a list of `.d` modules passed at compile time.

The simplest form of the CLI switch is `dmd -HC a.d b.d`

This will visit the ASTs of modules `a` and `b` and output a 
single header file at `stdout`.

By using the `-HCf=<file-name>` switch, the above result will be 
written in specified file name. Using `-HCd=<path>` will write 
the `file-name` in the specified `path`.

So, by running,
`dmd -HCf=ab.h -HCd=mypath/ a.d b.d` will write the generated 
header in `mypath/ab.h`, relative to the current directory.

If you have some spare time and curiosity I would appreciate your 
`test drive` and bug reports :)

This month
----------

I'll be working on generating the frontend headers, cleaning up 
the code and fixing issues and addressing PR comments.

Closing note
------------

I deeply apologize for this long overdue post.

Looking forward to your replies,
Edi

[0] - https://github.com/dlang/dmd/pull/9971
[1] - https://github.com/dlang/dmd/pull/8591
Jul 16 2019
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
So currently there is no way to restrict it to just extern(C)?
I ask this because -HC makes me think C not C++ headers.
Jul 16 2019
parent reply Eduard Staniloiu <edi33416 gmail.com> writes:
On Tuesday, 16 July 2019 at 13:46:56 UTC, rikki cattermole wrote:

 So currently there is no way to restrict it to just extern(C)?
 I ask this because -HC makes me think C not C++ headers.


Currently the outputted headers are C++ headers, but we were 
thinking of wrapping `extern (C++)` definitions inside an `#ifdef 
__cplusplus` block, and prefixing any `extern (C)` definitions 
with the following `EXTERNC` macro

```
#ifdef __cplusplus
#define EXTERNC extern(C)
#else
#define EXTERNC
#endif
```

This way, the generated header could be used in both C and C++.

What do you think?
Jul 17 2019
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 17 July 2019 at 11:05:21 UTC, Eduard Staniloiu 
wrote:

 On Tuesday, 16 July 2019 at 13:46:56 UTC, rikki cattermole 
 wrote:

 So currently there is no way to restrict it to just extern(C)?
 I ask this because -HC makes me think C not C++ headers.


 Currently the outputted headers are C++ headers, but we were 
 thinking of wrapping `extern (C++)` definitions inside an 
 `#ifdef __cplusplus` block, and prefixing any `extern (C)` 
 definitions with the following `EXTERNC` macro

 ```
 #ifdef __cplusplus
 #define EXTERNC extern(C)
 #else
 #define EXTERNC
 #endif
 ```

 This way, the generated header could be used in both C and C++.

 What do you think?


It should be pretty trivial to just check the linkage of the 
symbols and only output extern(C) symbols.

Also you've clearly been doing too much D programming! (which is 
probably a good thing) That would be:
#define EXTERNC extern "C" {

and you need to macro the closing brace as well.
Jul 17 2019
parent reply Eduard Staniloiu <edi33416 gmail.com> writes:
On Wednesday, 17 July 2019 at 11:52:17 UTC, Nicholas Wilson wrote:

 On Wednesday, 17 July 2019 at 11:05:21 UTC, Eduard Staniloiu 
 wrote:

 On Tuesday, 16 July 2019 at 13:46:56 UTC, rikki cattermole 
 wrote:

 So currently there is no way to restrict it to just extern(C)?
 I ask this because -HC makes me think C not C++ headers.


 Currently the outputted headers are C++ headers, but we were 
 thinking of wrapping `extern (C++)` definitions inside an 
 `#ifdef __cplusplus` block, and prefixing any `extern (C)` 
 definitions with the following `EXTERNC` macro

 ```
 #ifdef __cplusplus
 #define EXTERNC extern(C)
 #else
 #define EXTERNC
 #endif
 ```

 This way, the generated header could be used in both C and C++.

 What do you think?


 It should be pretty trivial to just check the linkage of the 
 symbols and only output extern(C) symbols.


This would require two compiler switches, one for C (-HC ?) and 
one for C++ (-HCPP ?).
Was trying to keep it as one compiler switch.


 Also you've clearly been doing too much D programming! (which 
 is probably a good thing) That would be:
 #define EXTERNC extern "C" {

 and you need to macro the closing brace as well.


Good catch! Thank you
Jul 17 2019
parent Manu <turkeyman gmail.com> writes:
On Wed, Jul 17, 2019 at 5:55 AM Eduard Staniloiu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Wednesday, 17 July 2019 at 11:52:17 UTC, Nicholas Wilson wrote:

 On Wednesday, 17 July 2019 at 11:05:21 UTC, Eduard Staniloiu
 wrote:

 On Tuesday, 16 July 2019 at 13:46:56 UTC, rikki cattermole
 wrote:

 So currently there is no way to restrict it to just extern(C)?
 I ask this because -HC makes me think C not C++ headers.


 Currently the outputted headers are C++ headers, but we were
 thinking of wrapping `extern (C++)` definitions inside an
 `#ifdef __cplusplus` block, and prefixing any `extern (C)`
 definitions with the following `EXTERNC` macro

 ```
 #ifdef __cplusplus
 #define EXTERNC extern(C)
 #else
 #define EXTERNC
 #endif
 ```

 This way, the generated header could be used in both C and C++.

 What do you think?


 It should be pretty trivial to just check the linkage of the
 symbols and only output extern(C) symbols.


 This would require two compiler switches, one for C (-HC ?) and
 one for C++ (-HCPP ?).
 Was trying to keep it as one compiler switch.


No, just emit one header with all the normal guards for C compilers.

#ifdef __cplusplus

#else

#endif

// C declarations
EXTERN_C void cfunc();
EXTERN_C void cfunc2();
...

// C++ declarations:
#ifdef __cplusplus
void cppFunc();
...
#endif


If you detect that there are no C++ symbols at all, then you could simplify to:

#ifdef __cplusplus
extern "C" {
#endif

// C declarations
void cFunc();
...

#ifdef __cplusplus
}
#endif
Jul 17 2019
prev sibling next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
I considered suggesting something like this.
But I'm not sure either way, it really depends on how portable it would be.

Also there are COM classes to think about too.
Jul 17 2019
prev sibling parent Manu <turkeyman gmail.com> writes:
On Wed, Jul 17, 2019 at 4:10 AM Eduard Staniloiu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Tuesday, 16 July 2019 at 13:46:56 UTC, rikki cattermole wrote:

 So currently there is no way to restrict it to just extern(C)?
 I ask this because -HC makes me think C not C++ headers.


 Currently the outputted headers are C++ headers, but we were
 thinking of wrapping `extern (C++)` definitions inside an `#ifdef
 __cplusplus` block, and prefixing any `extern (C)` definitions
 with the following `EXTERNC` macro

 ```
 #ifdef __cplusplus
 #define EXTERNC extern(C)
 #else
 #define EXTERNC
 #endif
 ```

 This way, the generated header could be used in both C and C++.

 What do you think?


Sure, that sort of thing is normal and sensible.
Jul 17 2019
prev sibling next sibling parent reply bachmeier <no spam.net> writes:
On Tuesday, 16 July 2019 at 13:16:50 UTC, Eduard Staniloiu wrote:


 So, by running,
 `dmd -HCf=ab.h -HCd=mypath/ a.d b.d` will write the generated 
 header in `mypath/ab.h`, relative to the current directory.


Will it be possible to extend this for other languages? That 
would be a killer application. For instance

dmd -HRuby=ab.rb -HCd=mypath/ a.d b.d

goes a step further, converting what would have been the 
generated C header to the file ab.rb containing (taken from 
https://github.com/ffi/ffi/wiki/Examples)

module ab
   extend FFI::Library
   ffi_lib "path/to/ab.so"
   attach_function :calculate_something, [:int, :float], :double

functions taking zero arguments
   attach_function :create_object, [:string], :pointer
   attach_function :calculate_something_else, [:double, :pointer], 
:double
   attach_function :free_object, [:pointer], :void
end

Then the Ruby script would call ab.rb as

require 'ffi'
require 'ab'


just fine
if ( (errcode = ab.error_code()) != 0)

   exit 1
end


literals as well
d = ab.calculate_something_else(c, objptr)
ab.free_object(objptr)
Jul 16 2019
parent Eduard Staniloiu <edi33416 gmail.com> writes:
On Tuesday, 16 July 2019 at 19:10:41 UTC, bachmeier wrote:

 On Tuesday, 16 July 2019 at 13:16:50 UTC, Eduard Staniloiu 
 wrote:


 So, by running,
 `dmd -HCf=ab.h -HCd=mypath/ a.d b.d` will write the generated 
 header in `mypath/ab.h`, relative to the current directory.


 Will it be possible to extend this for other languages? That 
 would be a killer application. For instance

 dmd -HRuby=ab.rb -HCd=mypath/ a.d b.d

 goes a step further, converting what would have been the 
 generated C header to the file ab.rb containing (taken from 
 https://github.com/ffi/ffi/wiki/Examples)

 [ ... ]


This would best be done as a separate RubyVisitor that visits the 
AST nodes and
writes the expected FFI interface. If I didn't misunderstand the 
FFI readme, it only works with C interfaces, so this should 
probably simplify a great deal from the complexity.

 From the example it looks like FFI is meant to work only with 
opaque pointers, which means that you would only be interested in 
declaring `struct`s, defining `enum`s and function declarations.

The memory management bindings might be trickier, or they could 
be easy as I don't know either Ruby or FFI.

This being said, I believe that it should be done as a separate 
visitor so it wouldn't add more complexity to the C/C++ one.

This would be an interesting project :)
Jul 17 2019
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On Tue, Jul 16, 2019 at 6:20 AM Eduard Staniloiu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 [...]


I'm super-excited to give this a spin. Thanks for having a go at it!!
One hopefully trivial request, I wouldn't want the compiler to emit a
single header; I would want it to emit one .h per .d file into the
output directory.
Jul 16 2019
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 17 July 2019 at 04:05:57 UTC, Manu wrote:

 One hopefully trivial request, I wouldn't want the compiler to 
 emit a single header; I would want it to emit one .h per .d 
 file into the output directory.


Forward declarations make that a PITA.
Jul 16 2019
parent reply Manu <turkeyman gmail.com> writes:
On Tue, Jul 16, 2019 at 9:40 PM Nicholas Wilson via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Wednesday, 17 July 2019 at 04:05:57 UTC, Manu wrote:

 One hopefully trivial request, I wouldn't want the compiler to
 emit a single header; I would want it to emit one .h per .d
 file into the output directory.


 Forward declarations make that a PITA.


Hmmm...
Jul 16 2019
next sibling parent reply Eduard Staniloiu <edi33416 gmail.com> writes:
On Wednesday, 17 July 2019 at 04:43:52 UTC, Manu wrote:

 On Tue, Jul 16, 2019 at 9:40 PM Nicholas Wilson via 
 Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On Wednesday, 17 July 2019 at 04:05:57 UTC, Manu wrote:

 One hopefully trivial request, I wouldn't want the compiler 
 to emit a single header; I would want it to emit one .h per 
 .d file into the output directory.


 Forward declarations make that a PITA.


 Hmmm...


I'm glad to hear that you are excited and eager to give it a spin 
:D

As Nicholas pointed out, forward declarations make it a pain.
Say you have the following example modules, `a.d` and `b.d`
```
// a.d
module a;

import b;

extern (C++) struct A
{
   TestEnum e;
}

// b.d
module b;

enum TestEnum
{
   aa,
   bb
}
```

For `a.d` you'll get
```
// a.h
#pragma once

enum TestEnum
{
     TESTENUMaa = 0,
     TESTENUMbb = 1
};

struct A
{
     TestEnum e;
};
```

For `b.d` you'll get
```
// b.h
#pragma once

enum TestEnum
{
     TESTENUMaa = 0,
     TESTENUMbb = 1
};
```

When you'll bring the two headers together, you'll get a 
re-definition error for the enum.

One way to go with this would be to have wrap the definitions 
inside an `#ifndef \ #define` block.

Another one would be to check from which module does the forward 
declaration come from and replace that with an `#include "b.h"`, 
but I don't know how complicated this would be, as not all 
forward declarations come from a different module.
Jul 17 2019
parent reply Manu <turkeyman gmail.com> writes:
On Wed, Jul 17, 2019 at 4:40 AM Eduard Staniloiu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 enum TestEnum
 {
      TESTENUMaa = 0,
      TESTENUMbb = 1
 };


What in earth is that?
Jul 17 2019
parent reply Eduard Staniloiu <edi33416 gmail.com> writes:
On Wednesday, 17 July 2019 at 17:41:23 UTC, Manu wrote:

 On Wed, Jul 17, 2019 at 4:40 AM Eduard Staniloiu via 
 Digitalmars-d <digitalmars-d puremagic.com> wrote:

 enum TestEnum
 {
      TESTENUMaa = 0,
      TESTENUMbb = 1
 };


 What in earth is that?


This is the convention that is used in the manually written 
headers in order to avoid name clashing between fields of 
different enums in the C headers
Jul 17 2019
next sibling parent reply Manu <turkeyman gmail.com> writes:
On Wed, Jul 17, 2019 at 11:40 AM Eduard Staniloiu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Wednesday, 17 July 2019 at 17:41:23 UTC, Manu wrote:

 On Wed, Jul 17, 2019 at 4:40 AM Eduard Staniloiu via
 Digitalmars-d <digitalmars-d puremagic.com> wrote:

 enum TestEnum
 {
      TESTENUMaa = 0,
      TESTENUMbb = 1
 };


 What in earth is that?


 This is the convention that is used in the manually written
 headers in order to avoid name clashing between fields of
 different enums in the C headers


Okay... but it's also the worst thing ever, and no real user would
want this code to be emit from the compiler ;)
Safe to say this is a strictly DMD-specific naming pattern, and I
don't think that should be our benchmark for a public-facing feature.

A better non-C++11 name might be `TestEnum_aa`?
Jul 17 2019
parent reply Jacob Carlborg <doob me.com> writes:
On 2019-07-17 22:59, Manu wrote:


 Okay... but it's also the worst thing ever, and no real user would
 want this code to be emit from the compiler ;)
 Safe to say this is a strictly DMD-specific naming pattern, and I
 don't think that should be our benchmark for a public-facing feature.
 
 A better non-C++11 name might be `TestEnum_aa`?


In Apple's frameworks, in particular the Objective-C frameworks, they 
define enums like this:

typedef NS_ENUM(int, MKAnnotationViewDragState)
{
     MKAnnotationViewDragStateNone = 0,
     MKAnnotationViewDragStateStarting,
     MKAnnotationViewDragStateDragging,
     MKAnnotationViewDragStateCanceling,
     MKAnnotationViewDragStateEnding
}

Where NS_ENUM expands, in the above case, to:

typedef int MKAnnotationViewDragState; enum
{
     MKAnnotationViewDragStateNone = 0,
     MKAnnotationViewDragStateStarting,
     MKAnnotationViewDragStateDragging,
     MKAnnotationViewDragStateCanceling,
     MKAnnotationViewDragStateEnding
}

Probably makes it easier to present a nice interface in Swift.

-- 
/Jacob Carlborg
Jul 22 2019
parent reply Manu <turkeyman gmail.com> writes:
On Mon, Jul 22, 2019 at 2:51 AM Jacob Carlborg via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 2019-07-17 22:59, Manu wrote:


 Okay... but it's also the worst thing ever, and no real user would
 want this code to be emit from the compiler ;)
 Safe to say this is a strictly DMD-specific naming pattern, and I
 don't think that should be our benchmark for a public-facing feature.

 A better non-C++11 name might be `TestEnum_aa`?


 In Apple's frameworks, in particular the Objective-C frameworks, they
 define enums like this:

 typedef NS_ENUM(int, MKAnnotationViewDragState)
 {
      MKAnnotationViewDragStateNone = 0,
      MKAnnotationViewDragStateStarting,
      MKAnnotationViewDragStateDragging,
      MKAnnotationViewDragStateCanceling,
      MKAnnotationViewDragStateEnding
 }

 Where NS_ENUM expands, in the above case, to:

 typedef int MKAnnotationViewDragState; enum
 {
      MKAnnotationViewDragStateNone = 0,
      MKAnnotationViewDragStateStarting,
      MKAnnotationViewDragStateDragging,
      MKAnnotationViewDragStateCanceling,
      MKAnnotationViewDragStateEnding
 }

 Probably makes it easier to present a nice interface in Swift.

 --
 /Jacob Carlborg


Great idea!
We can make the generated header emit this:

#ifndef D_ENUM

#endif
#ifndef D_ENUM_KEY

#endif
#ifndef D_ENUM_KEY_VAL

#endif

D_ENUM(DEnum, int)
{
  D_ENUM_KEY(A, DEnum),
  D_ENUM_KEY_VAL(B, 10, DEnum),
};



And then someone can override those with:

#define D_ENUM(name, type) enum class name : type
#define D_ENUM_KEY(name, enumType) name
#define D_ENUM_KEY_VAL(name, value, enumType) name = value

#include "my_d_header.h"



And for over-powered win, define the macros like this:




And then the header emits:
D_ENUM(DEnum, denum, DENUM, int)
{
  D_ENUM_KEY(MyKey, mykey, MYKEY, DEnum, denum, DENUM),
}

This allows maximum flexibility! It can support the D compilers weird
naming strategy without adding additional options to the compiler.
Jul 22 2019
parent reply Gregor =?UTF-8?B?TcO8Y2ts?= <gregormueckl gmx.de> writes:
On Monday, 22 July 2019 at 18:12:01 UTC, Manu wrote:

 Great idea!
 We can make the generated header emit this:

 #ifndef D_ENUM

 #endif
 #ifndef D_ENUM_KEY

 #endif
 #ifndef D_ENUM_KEY_VAL

 = value
 #endif

 D_ENUM(DEnum, int)
 {
   D_ENUM_KEY(A, DEnum),
   D_ENUM_KEY_VAL(B, 10, DEnum),
 };



 And then someone can override those with:

 #define D_ENUM(name, type) enum class name : type
 #define D_ENUM_KEY(name, enumType) name
 #define D_ENUM_KEY_VAL(name, value, enumType) name = value

 #include "my_d_header.h"



 And for over-powered win, define the macros like this:



 typeUpper) ...

 And then the header emits:
 D_ENUM(DEnum, denum, DENUM, int)
 {
   D_ENUM_KEY(MyKey, mykey, MYKEY, DEnum, denum, DENUM),
 }

 This allows maximum flexibility! It can support the D compilers 
 weird naming strategy without adding additional options to the 
 compiler.


That's a pretty neat hack!

I'm just worried about two things:
- it makes the generated code harder to read (not that big of a 
deal)
- if you happen to include headers with conflicting D_ENUM* 
defines (e.g. a potential 3rd party D wrapper), you might get 
some nasty surprises :/
Jul 28 2019
parent reply Manu <turkeyman gmail.com> writes:
On Sun, Jul 28, 2019 at 5:20 PM Gregor Mückl via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Monday, 22 July 2019 at 18:12:01 UTC, Manu wrote:

 Great idea!
 We can make the generated header emit this:

 #ifndef D_ENUM

 #endif
 #ifndef D_ENUM_KEY

 #endif
 #ifndef D_ENUM_KEY_VAL

 = value
 #endif

 D_ENUM(DEnum, int)
 {
   D_ENUM_KEY(A, DEnum),
   D_ENUM_KEY_VAL(B, 10, DEnum),
 };



 And then someone can override those with:

 #define D_ENUM(name, type) enum class name : type
 #define D_ENUM_KEY(name, enumType) name
 #define D_ENUM_KEY_VAL(name, value, enumType) name = value

 #include "my_d_header.h"



 And for over-powered win, define the macros like this:



 typeUpper) ...

 And then the header emits:
 D_ENUM(DEnum, denum, DENUM, int)
 {
   D_ENUM_KEY(MyKey, mykey, MYKEY, DEnum, denum, DENUM),
 }

 This allows maximum flexibility! It can support the D compilers
 weird naming strategy without adding additional options to the
 compiler.


 That's a pretty neat hack!

 I'm just worried about two things:
 - it makes the generated code harder to read (not that big of a
 deal)


Barely... and a generated header is not really for human consumption either way.


 - if you happen to include headers with conflicting D_ENUM*
 defines (e.g. a potential 3rd party D wrapper), you might get
 some nasty surprises :/


C/C++ programmers now how to do macros and what to expect.
Jul 28 2019
parent reply Eduard Staniloiu <edi33416 gmail.com> writes:
On Monday, 29 July 2019 at 01:19:36 UTC, Manu wrote:

 On Sun, Jul 28, 2019 at 5:20 PM Gregor Mückl via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Monday, 22 July 2019 at 18:12:01 UTC, Manu wrote:

 [...]


 That's a pretty neat hack!

 I'm just worried about two things:
 - it makes the generated code harder to read (not that big of a
 deal)


 Barely... and a generated header is not really for human 
 consumption either way.


 - if you happen to include headers with conflicting D_ENUM* 
 defines (e.g. a potential 3rd party D wrapper), you might get 
 some nasty surprises :/


 C/C++ programmers now how to do macros and what to expect.


So should we go ahead with generating code based on the 
`-extern-std` value or by using defines?

After sleeping a bit on this, I prefer `-extern-std`.
Jul 30 2019
next sibling parent reply Eduard Staniloiu <edi33416 gmail.com> writes:
On Tuesday, 30 July 2019 at 14:43:01 UTC, Eduard Staniloiu wrote:

 On Monday, 29 July 2019 at 01:19:36 UTC, Manu wrote:

 On Sun, Jul 28, 2019 at 5:20 PM Gregor Mückl via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Monday, 22 July 2019 at 18:12:01 UTC, Manu wrote:

 [...]


 That's a pretty neat hack!

 I'm just worried about two things:
 - it makes the generated code harder to read (not that big of 
 a
 deal)


 Barely... and a generated header is not really for human 
 consumption either way.


 - if you happen to include headers with conflicting D_ENUM* 
 defines (e.g. a potential 3rd party D wrapper), you might get 
 some nasty surprises :/


 C/C++ programmers now how to do macros and what to expect.


 So should we go ahead with generating code based on the 
 `-extern-std` value or by using defines?

 After sleeping a bit on this, I prefer `-extern-std`.


BTW, I went with `-extern-std` in the current implementation (on 
GH).

Another question is how should `align(n)` be generated. See the 
following two:

1) __attribute__((packed, aligned(n)))

or

2) __attribute__((aligned(n)))

Note the missing `packed` between 1) and 2)

Currently (on GH) the entire generated output looks like this

struct
#if defined(__GNUC__) || defined(__clang__)
     __attribute__((packed, aligned(8)))
#elif defined(_MSC_VER)
     __declspec(align(8))
#elif defined(__DMC__)
     #pragma pack(push, 8)
#endif
MyStruct
{
   /* fields */
}
#if defined(__DMC__)
     #pragma pack(pop)
#endif
Jul 30 2019
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.com> writes:
On 7/30/19 10:50 AM, Eduard Staniloiu wrote:

 On Tuesday, 30 July 2019 at 14:43:01 UTC, Eduard Staniloiu wrote:

 On Monday, 29 July 2019 at 01:19:36 UTC, Manu wrote:

 On Sun, Jul 28, 2019 at 5:20 PM Gregor Mückl via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Monday, 22 July 2019 at 18:12:01 UTC, Manu wrote:

 [...]


 That's a pretty neat hack!

 I'm just worried about two things:
 - it makes the generated code harder to read (not that big of a
 deal)


 Barely... and a generated header is not really for human consumption 
 either way.


 - if you happen to include headers with conflicting D_ENUM* defines 
 (e.g. a potential 3rd party D wrapper), you might get some nasty 
 surprises :/


 C/C++ programmers now how to do macros and what to expect.


 So should we go ahead with generating code based on the `-extern-std` 
 value or by using defines?

 After sleeping a bit on this, I prefer `-extern-std`.


 
 BTW, I went with `-extern-std` in the current implementation (on GH).
 
 Another question is how should `align(n)` be generated. See the 
 following two:
 
 1) __attribute__((packed, aligned(n)))
 
 or
 
 2) __attribute__((aligned(n)))
 
 Note the missing `packed` between 1) and 2)
 
 Currently (on GH) the entire generated output looks like this
 
 struct
 #if defined(__GNUC__) || defined(__clang__)
      __attribute__((packed, aligned(8)))
 #elif defined(_MSC_VER)
      __declspec(align(8))
 #elif defined(__DMC__)
      #pragma pack(push, 8)
 #endif
 MyStruct
 {
    /* fields */
 }
 #if defined(__DMC__)
      #pragma pack(pop)
 #endif
 


Better yet:

#if defined(__GNUC__) || defined(__clang__)
     #define BEGIN_PACK(n) __attribute__((packed, aligned(n)))
#elif defined(_MSC_VER)
     #define BEGIN_PACK(n) __declspec(align(n))
#elif defined(__DMC__)
     #define BEGIN_PACK(n) #pragma pack(push, n)
#endif

#if defined(__DMC__)
     #define END_PACK() #pragma pack(pop)
#else
     #define END_PACK()
#endif
Jul 30 2019
parent reply Manu <turkeyman gmail.com> writes:
On Tue, Jul 30, 2019 at 8:10 AM Andrei Alexandrescu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 7/30/19 10:50 AM, Eduard Staniloiu wrote:

 On Tuesday, 30 July 2019 at 14:43:01 UTC, Eduard Staniloiu wrote:

 On Monday, 29 July 2019 at 01:19:36 UTC, Manu wrote:

 On Sun, Jul 28, 2019 at 5:20 PM Gregor Mückl via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 22 July 2019 at 18:12:01 UTC, Manu wrote:

 [...]


 That's a pretty neat hack!

 I'm just worried about two things:
 - it makes the generated code harder to read (not that big of a
 deal)


 Barely... and a generated header is not really for human consumption
 either way.


 - if you happen to include headers with conflicting D_ENUM* defines
 (e.g. a potential 3rd party D wrapper), you might get some nasty
 surprises :/


 C/C++ programmers now how to do macros and what to expect.


 So should we go ahead with generating code based on the `-extern-std`
 value or by using defines?

 After sleeping a bit on this, I prefer `-extern-std`.


 BTW, I went with `-extern-std` in the current implementation (on GH).

 Another question is how should `align(n)` be generated. See the
 following two:

 1) __attribute__((packed, aligned(n)))

 or

 2) __attribute__((aligned(n)))

 Note the missing `packed` between 1) and 2)

 Currently (on GH) the entire generated output looks like this

 struct
 #if defined(__GNUC__) || defined(__clang__)
      __attribute__((packed, aligned(8)))
 #elif defined(_MSC_VER)
      __declspec(align(8))
 #elif defined(__DMC__)
      #pragma pack(push, 8)
 #endif
 MyStruct
 {
    /* fields */
 }
 #if defined(__DMC__)
      #pragma pack(pop)
 #endif


 Better yet:

 #if defined(__GNUC__) || defined(__clang__)
      #define BEGIN_PACK(n) __attribute__((packed, aligned(n)))
 #elif defined(_MSC_VER)
      #define BEGIN_PACK(n) __declspec(align(n))
 #elif defined(__DMC__)
      #define BEGIN_PACK(n) #pragma pack(push, n)
 #endif

 #if defined(__DMC__)
      #define END_PACK() #pragma pack(pop)
 #else
      #define END_PACK()
 #endif


This is precisely what I was just about to propose ;)
Jul 30 2019
parent reply Eduard Staniloiu <edi33416 gmail.com> writes:
On Tuesday, 30 July 2019 at 18:00:44 UTC, Manu wrote:

 On Tue, Jul 30, 2019 at 8:10 AM Andrei Alexandrescu via 
 Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On 7/30/19 10:50 AM, Eduard Staniloiu wrote:

[...]


 Better yet:

 #if defined(__GNUC__) || defined(__clang__)
      #define BEGIN_PACK(n) __attribute__((packed, aligned(n)))
 #elif defined(_MSC_VER)
      #define BEGIN_PACK(n) __declspec(align(n))
 #elif defined(__DMC__)
      #define BEGIN_PACK(n) #pragma pack(push, n)
 #endif

 #if defined(__DMC__)
      #define END_PACK() #pragma pack(pop)
 #else
      #define END_PACK()
 #endif


 This is precisely what I was just about to propose ;)


Will use this, but my question still stands:
__attribute__ packed or not?

Given the following struct
```
struct S
{
     int x;
     char y;
}
```

With `struct __attribute__((packed, aligned(8))) S` -> sizeof(S) 
is 5

With `struct __attribute__((aligned(8))) S` -> sizeof(S) is 8
Jul 30 2019
parent kinke <noone nowhere.com> writes:
On Tuesday, 30 July 2019 at 18:40:04 UTC, Eduard Staniloiu wrote:

 On Tuesday, 30 July 2019 at 18:00:44 UTC, Manu wrote:

 On Tue, Jul 30, 2019 at 8:10 AM Andrei Alexandrescu via 
 Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On 7/30/19 10:50 AM, Eduard Staniloiu wrote:

[...]


 Better yet:

 #if defined(__GNUC__) || defined(__clang__)
      #define BEGIN_PACK(n) __attribute__((packed, aligned(n)))
 #elif defined(_MSC_VER)
      #define BEGIN_PACK(n) __declspec(align(n))
 #elif defined(__DMC__)
      #define BEGIN_PACK(n) #pragma pack(push, n)
 #endif

 #if defined(__DMC__)
      #define END_PACK() #pragma pack(pop)
 #else
      #define END_PACK()
 #endif


 This is precisely what I was just about to propose ;)


 Will use this, but my question still stands:
 __attribute__ packed or not?

 Given the following struct
 ```
 struct S
 {
     int x;
     char y;
 }
 ```

 With `struct __attribute__((packed, aligned(8))) S` -> 
 sizeof(S) is 5

 With `struct __attribute__((aligned(8))) S` -> sizeof(S) is 8


I've only found 2 explicit alignments in the *frontend* files 
(dmd/{libomf,scanmscoff}.d aren't part of the frontend):

1) dmd/root/longdouble.d:
-----
struct longdouble_soft
{
     // DMD's x87 `real` on Windows is packed (alignof = 2 -> 
sizeof = 10).
     align(2) ulong mantissa;
     ushort exp_sign;
}
-----

2) dmd/expression.d:
-----
align(8) union __AnonStruct__u { ... }
-----

These 2 instances can be altered not to require any explicit 
alignment. Then there wouldn't be any current need for align() 
support.

Instance 1, `longdouble_soft`, is only relevant for MSVC. It 
could consist of data `ushort[4] mantissa_; ushort exp_sign` 
(natural alignment of 2 and size 10) and feature a `ref ulong 
mantissa()` property performing a reinterpret-cast.

Instance 2 only needs an additional `long __for_alignment_only` 
union member (union alignment is already handled like this in 
another union I don't remember OTOH).
Jul 30 2019
prev sibling parent reply Gregor =?UTF-8?B?TcO8Y2ts?= <gregormueckl gmx.de> writes:
On Tuesday, 30 July 2019 at 14:50:15 UTC, Eduard Staniloiu wrote:

 Another question is how should `align(n)` be generated. See the 
 following two:

 1) __attribute__((packed, aligned(n)))

 or

 2) __attribute__((aligned(n)))

 Note the missing `packed` between 1) and 2)

 Currently (on GH) the entire generated output looks like this

 struct
 #if defined(__GNUC__) || defined(__clang__)
     __attribute__((packed, aligned(8)))
 #elif defined(_MSC_VER)
     __declspec(align(8))
 #elif defined(__DMC__)
     #pragma pack(push, 8)
 #endif
 MyStruct
 {
   /* fields */
 }
 #if defined(__DMC__)
     #pragma pack(pop)
 #endif


This only covers structure packing rules, right? So alignment of 
the aggregate as a whole (equivalent to C++ alignas(n) statement) 
is out of scope?

If so,

#pragma pack(push,n)

and

#pragma pack(pop,n)

work on MSVC, GCC and clang in my experience. DMC is documented 
to have the same behavior. So you should be able to dispense with 
the #ifdef's completely.
Aug 02 2019
parent reply Eduard Staniloiu <edi33416 gmail.com> writes:
On Friday, 2 August 2019 at 13:17:46 UTC, Gregor Mückl wrote:

 On Tuesday, 30 July 2019 at 14:50:15 UTC, Eduard Staniloiu 
 wrote:

 Another question is how should `align(n)` be generated. See 
 the following two:

 1) __attribute__((packed, aligned(n)))

 or

 2) __attribute__((aligned(n)))

 Note the missing `packed` between 1) and 2)

 Currently (on GH) the entire generated output looks like this

 struct
 #if defined(__GNUC__) || defined(__clang__)
     __attribute__((packed, aligned(8)))
 #elif defined(_MSC_VER)
     __declspec(align(8))
 #elif defined(__DMC__)
     #pragma pack(push, 8)
 #endif
 MyStruct
 {
   /* fields */
 }
 #if defined(__DMC__)
     #pragma pack(pop)
 #endif


 This only covers structure packing rules, right? So alignment 
 of the aggregate as a whole (equivalent to C++ alignas(n) 
 statement) is out of scope?

 If so,

 #pragma pack(push,n)

 and

 #pragma pack(pop,n)

 work on MSVC, GCC and clang in my experience. DMC is documented 
 to have the same behavior. So you should be able to dispense 
 with the #ifdef's completely.


I missed your reply. Sorry about that.

`#pragma pack(push/pop, n)` was initially used, but when testing 
it didn't have the same behaviour as `__attribute__((packed, 
aligned(n)))` so I ended up with the entire `#ifdef` tree above.

I could generate `alignas(n)` when the user compiles with 
`-extern-std=c++11` and above, but I'm currently working on 
correctly generating the ctors and dtors declarations.
Sep 05 2019
next sibling parent reply Eduard Staniloiu <edi33416 gmail.com> writes:
 On Thursday, 5 September 2019 at 11:25:47 UTC, Eduard Staniloiu 
 wrote:
 [...]
 I'm currently working on correctly generating the ctors and 
 dtors declarations.


More details about the above.

When I began working on the project, the code was generating two 
ctors for each `struct`
   1. a default ctor that was initializing all the struct fields 
with their default value
   2. a ctor that took all the struct field types as arguments and 
assigned the supplied arguments to each struct field (Ex. 
`this->x = x`).

The above two replicate (almost) the D behavior of constructing a 
struct that has no ctor defined. I say almost because you can 
either use the default ctor, or you **must** pass all the field 
arguments to the ctor (in D you can pass as many as you want, not 
all).

When I first saw this and ran it with Razvan and Andrei and we 
said "Thats odd. The tool should only generate declarations, not 
definitions". I asked Iain about it and he said he couldn't 
remember why it was there, but probably he needed it at some 
point. So, in the light of this, we took the decision of dropping 
the ctor definition generation until further notice.

Well, this week I had an epiphany: for any struct, we have to 
generate the default ctor definition because on the D side, 
fields are default initialized to their `.init` value. If one 
would construct a struct (Ex. on the stack) on the C++ side, then 
the code could break.

Let's see the following simple struct example
```
// D module

extern (C++) struct S
{
   int* ptr;
   void apiFun() {
     if (ptr) {
       /* do stuff */
     }
   }
}

void bar()
{
   S s;
   s.apiFun();
}

// C++ code

// Generated struct header
struct S
{
   int* ptr;
   void apiFun();
};

void gun()
{
   S s;
   s.apiFun();
}
```

The `bar` function, being on the D side, is fine.
The `gun` function on the other hand, will be UB.

This is bad.

Starting from this, I came to the following conclusions:
   - For any `extern (C++)` struct, a default ctor must be 
generated
   - If a struct defines a dtor, a declaration must be generated: 
`virtual ~A();`
   - For any defined ctors, a declaration must be generated

I think that if a struct defines a copy ctor, an opAssign, then 
an equivalent C++ declaration must be generated.

What do you think, folks?

Cheers,
Edi
Sep 05 2019
parent Manu <turkeyman gmail.com> writes:
On Thu, Sep 5, 2019 at 5:01 AM Eduard Staniloiu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Thursday, 5 September 2019 at 11:25:47 UTC, Eduard Staniloiu
 wrote:
 [...]
 I'm currently working on correctly generating the ctors and
 dtors declarations.


 More details about the above.

 When I began working on the project, the code was generating two
 ctors for each `struct`
    1. a default ctor that was initializing all the struct fields
 with their default value
    2. a ctor that took all the struct field types as arguments and
 assigned the supplied arguments to each struct field (Ex.
 `this->x = x`).

 The above two replicate (almost) the D behavior of constructing a
 struct that has no ctor defined. I say almost because you can
 either use the default ctor, or you **must** pass all the field
 arguments to the ctor (in D you can pass as many as you want, not
 all).

 When I first saw this and ran it with Razvan and Andrei and we
 said "Thats odd. The tool should only generate declarations, not
 definitions". I asked Iain about it and he said he couldn't
 remember why it was there, but probably he needed it at some
 point. So, in the light of this, we took the decision of dropping
 the ctor definition generation until further notice.

 Well, this week I had an epiphany: for any struct, we have to
 generate the default ctor definition because on the D side,
 fields are default initialized to their `.init` value. If one
 would construct a struct (Ex. on the stack) on the C++ side, then
 the code could break.

 Let's see the following simple struct example
 ```
 // D module

 extern (C++) struct S
 {
    int* ptr;
    void apiFun() {
      if (ptr) {
        /* do stuff */
      }
    }
 }

 void bar()
 {
    S s;
    s.apiFun();
 }

 // C++ code

 // Generated struct header
 struct S
 {
    int* ptr;
    void apiFun();
 };

 void gun()
 {
    S s;
    s.apiFun();
 }
 ```

 The `bar` function, being on the D side, is fine.
 The `gun` function on the other hand, will be UB.

 This is bad.

 Starting from this, I came to the following conclusions:
    - For any `extern (C++)` struct, a default ctor must be
 generated


We have discussed responding to the C++ language version before, and
for >= C++11 it would be nice to use member initialisers rather than a
default ctor.


    - If a struct defines a dtor, a declaration must be generated:
 `virtual ~A();`


Not virtual though right?


    - For any defined ctors, a declaration must be generated


A constructor is no different than any other method. I expect you're
emiting all the methods right?


 I think that if a struct defines a copy ctor, an opAssign, then
 an equivalent C++ declaration must be generated.


These are just normal methods too... if a method has a C++
counterpart, then emit a declaration for it. This includes all
constructors (except postblit, that guy's a problem).
You could translate operators (perhaps as a stretch goal), ie;
opAssign -> operator=


 What do you think, folks?


I think declarations for all methods that have a valid C++ counterpart
should be emit... no?


On a tangent, can you confirm where you landed with respect to the
enum discussion? I want to be sure we landed in a satisfying place.
Did you go with the fully-loaded macro solution, so the client can
define the macro's to taste?
Sep 05 2019
prev sibling parent Gregor =?UTF-8?B?TcO8Y2ts?= <gregormueckl gmx.de> writes:
On Thursday, 5 September 2019 at 11:25:47 UTC, Eduard Staniloiu 
wrote:

 `#pragma pack(push/pop, n)` was initially used, but when 
 testing it didn't have the same behaviour as 
 `__attribute__((packed, aligned(n)))` so I ended up with the 
 entire `#ifdef` tree above.


Can you give a quick summary of the differences that you 
encountered? I believe you, but I'm curious because this goes 
completely against my expectations.
Sep 06 2019
prev sibling next sibling parent reply Seb <seb wilzba.ch> writes:
On Tuesday, 30 July 2019 at 14:43:01 UTC, Eduard Staniloiu wrote:

 On Monday, 29 July 2019 at 01:19:36 UTC, Manu wrote:

 On Sun, Jul 28, 2019 at 5:20 PM Gregor Mückl via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Monday, 22 July 2019 at 18:12:01 UTC, Manu wrote:

 [...]


 That's a pretty neat hack!

 I'm just worried about two things:
 - it makes the generated code harder to read (not that big of 
 a
 deal)


 Barely... and a generated header is not really for human 
 consumption either way.


 - if you happen to include headers with conflicting D_ENUM* 
 defines (e.g. a potential 3rd party D wrapper), you might get 
 some nasty surprises :/


 C/C++ programmers now how to do macros and what to expect.


 So should we go ahead with generating code based on the 
 `-extern-std` value or by using defines?

 After sleeping a bit on this, I prefer `-extern-std`.


Advantage of using defines is that you can use one header to rule 
them all.
I.e. you generate it once and then make it available to your 
clients/users.
=> it's immediately ready to be distributed

With -extern-std, you have to generate n-interfaces and 
distribute them to your clients + probably add a few defines 
yourself.

A comprise could be that by default you emit all combinations and 
only if -extern-std is defined you emit the headers for that 
standard.
Jul 30 2019
parent Manu <turkeyman gmail.com> writes:
On Tue, Jul 30, 2019 at 9:20 AM Seb via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Tuesday, 30 July 2019 at 14:43:01 UTC, Eduard Staniloiu wrote:

 On Monday, 29 July 2019 at 01:19:36 UTC, Manu wrote:

 On Sun, Jul 28, 2019 at 5:20 PM Gregor Mückl via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 22 July 2019 at 18:12:01 UTC, Manu wrote:

 [...]


 That's a pretty neat hack!

 I'm just worried about two things:
 - it makes the generated code harder to read (not that big of
 a
 deal)


 Barely... and a generated header is not really for human
 consumption either way.


 - if you happen to include headers with conflicting D_ENUM*
 defines (e.g. a potential 3rd party D wrapper), you might get
 some nasty surprises :/


 C/C++ programmers now how to do macros and what to expect.


 So should we go ahead with generating code based on the
 `-extern-std` value or by using defines?

 After sleeping a bit on this, I prefer `-extern-std`.


 Advantage of using defines is that you can use one header to rule
 them all.
 I.e. you generate it once and then make it available to your
 clients/users.
 => it's immediately ready to be distributed

 With -extern-std, you have to generate n-interfaces and
 distribute them to your clients + probably add a few defines
 yourself.

 A comprise could be that by default you emit all combinations and
 only if -extern-std is defined you emit the headers for that
 standard.


Nar.. just emit the defines, and let the `-extern-std` drive the most
likely desired default for when the macros are not defined by the
user.
Jul 30 2019
prev sibling parent Manu <turkeyman gmail.com> writes:
On Tue, Jul 30, 2019 at 7:45 AM Eduard Staniloiu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Monday, 29 July 2019 at 01:19:36 UTC, Manu wrote:

 On Sun, Jul 28, 2019 at 5:20 PM Gregor Mückl via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 22 July 2019 at 18:12:01 UTC, Manu wrote:

 [...]


 That's a pretty neat hack!

 I'm just worried about two things:
 - it makes the generated code harder to read (not that big of a
 deal)


 Barely... and a generated header is not really for human
 consumption either way.


 - if you happen to include headers with conflicting D_ENUM*
 defines (e.g. a potential 3rd party D wrapper), you might get
 some nasty surprises :/


 C/C++ programmers now how to do macros and what to expect.


 So should we go ahead with generating code based on the
 `-extern-std` value or by using defines?

 After sleeping a bit on this, I prefer `-extern-std`.


It's possible that the correct answer is both.
The define technique I described empowers DMD to use its bizarre
naming strategy without special hacks in the header gen code.
I think perhaps the defines should be used, but the default code (when
the macros aren't specified by the user) could be driven by
-extern-std as you have it now.
Jul 30 2019
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On Wed, 17 Jul 2019 at 22:59, Manu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Wed, Jul 17, 2019 at 11:40 AM Eduard Staniloiu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Wednesday, 17 July 2019 at 17:41:23 UTC, Manu wrote:

 On Wed, Jul 17, 2019 at 4:40 AM Eduard Staniloiu via
 Digitalmars-d <digitalmars-d puremagic.com> wrote:

 enum TestEnum
 {
      TESTENUMaa = 0,
      TESTENUMbb = 1
 };


 What in earth is that?


 This is the convention that is used in the manually written
 headers in order to avoid name clashing between fields of
 different enums in the C headers


 Okay... but it's also the worst thing ever, and no real user would
 want this code to be emit from the compiler ;)
 Safe to say this is a strictly DMD-specific naming pattern, and I
 don't think that should be our benchmark for a public-facing feature.

 A better non-C++11 name might be `TestEnum_aa`?



I guess the most typical C/C++98 style would be:

enum xml_status {
  XML_STATUS_ERROR = 0,
  XML_STATUS_OK = 1,
  XML_STATUS_SUSPENDED = 2
};

Some may want the enum to be called xml_status_flags, or some may want
a different prefix for the enum members.

-- 
Iain
Jul 17 2019
prev sibling parent Manu <turkeyman gmail.com> writes:
On Wed, Jul 17, 2019 at 9:57 PM Iain Buclaw via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Wed, 17 Jul 2019 at 22:59, Manu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Wed, Jul 17, 2019 at 11:40 AM Eduard Staniloiu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Wednesday, 17 July 2019 at 17:41:23 UTC, Manu wrote:

 On Wed, Jul 17, 2019 at 4:40 AM Eduard Staniloiu via
 Digitalmars-d <digitalmars-d puremagic.com> wrote:

 enum TestEnum
 {
      TESTENUMaa = 0,
      TESTENUMbb = 1
 };


 What in earth is that?


 This is the convention that is used in the manually written
 headers in order to avoid name clashing between fields of
 different enums in the C headers


 Okay... but it's also the worst thing ever, and no real user would
 want this code to be emit from the compiler ;)
 Safe to say this is a strictly DMD-specific naming pattern, and I
 don't think that should be our benchmark for a public-facing feature.

 A better non-C++11 name might be `TestEnum_aa`?



 I guess the most typical C/C++98 style would be:

 enum xml_status {
   XML_STATUS_ERROR = 0,
   XML_STATUS_OK = 1,
   XML_STATUS_SUSPENDED = 2
 };

 Some may want the enum to be called xml_status_flags, or some may want
 a different prefix for the enum members.


Surely the keys should be created from string material that actually
exists? Applying random formatting to the strings feels pointless to
me. Just prefix the key with the enum name, and `_` is the
overwhelmingly established C identifier separator.
If you want your codebase to feel natural to C in the way you describe
above, then you can write:

enum XML_STATUS
{
  ERROR = 0,
  OK = 1,
  SUSPENDED = 2
}

D has a lot of meta that generates identifiers from things; if there's
arbitrary re-formatting rules in this tool, then they would end out
being mirrored around the place.
Jul 18 2019
prev sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 17 July 2019 at 04:43:52 UTC, Manu wrote:

 On Tue, Jul 16, 2019 at 9:40 PM Nicholas Wilson via 
 Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On Wednesday, 17 July 2019 at 04:05:57 UTC, Manu wrote:

 One hopefully trivial request, I wouldn't want the compiler 
 to emit a single header; I would want it to emit one .h per 
 .d file into the output directory.


 Forward declarations make that a PITA.


 Hmmm...


You could possibly achieve something sensible with package at a 
time compilation though.
Jul 17 2019
prev sibling parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Tue, 16 Jul 2019 at 15:20, Eduard Staniloiu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 Project goal
 ------------

 The deliverable of the project is a tool that automatically
 generates C and C++
 header files from D module files. This can be achieved either by
 a library solution using
 DMD as a Library, or by adding this feature in the DMD frontend
 through a compiler
 switch.

 The advantage of using DMD as a Library is that this wouldn’t
 increase the
 complexity of compiler frontend codebase. The disadvantage will
 be that the user will be
 required to install a third-party tool. Contrasting to this, the
 addition of the feature to the
 frontend would result in a smoother integration with all the
 backends that use the DMD
 frontend.

 We have decided to go with the compiler switch approach.

 One major milestone (and success marker) for the project is to
 automatically generate the
 DMD frontend headers required by GDC/LDC.


As there will be for a long time the need to bootstrap, I think the
desired end result would be to have the generated header checked in to
dmd, and for this header to be rebuilt during the 'make' build
process.

The CI infrastructure we have can then assert that this header doesn't
change during the build, so PRs should never forget to check-in the
new header.


 So now, the current state of affairs is that the code in the PR
 [0] can link with and pass the `cxx-unittests`.


So it's probably worth giving the generated header a test against gdc then.

https://github.com/gcc-mirror/gcc/tree/ibuclaw/gdc



 How to use it
 -------------

 The current PR [0] code is generating a `C++` header file out of
 a list of `.d` modules passed at compile time.

 The simplest form of the CLI switch is `dmd -HC a.d b.d`

 This will visit the ASTs of modules `a` and `b` and output a
 single header file at `stdout`.

 By using the `-HCf=<file-name>` switch, the above result will be
 written in specified file name. Using `-HCd=<path>` will write
 the `file-name` in the specified `path`.

 So, by running,
 `dmd -HCf=ab.h -HCd=mypath/ a.d b.d` will write the generated
 header in `mypath/ab.h`, relative to the current directory.

 If you have some spare time and curiosity I would appreciate your
 `test drive` and bug reports :)


So this would be part of the compiler rather than a standalone
application?  What's the rationale there?

-- 
Iain
Jul 17 2019
parent reply Eduard Staniloiu <edi33416 gmail.com> writes:
On Wednesday, 17 July 2019 at 12:40:38 UTC, Iain Buclaw wrote:

 On Tue, 16 Jul 2019 at 15:20, Eduard Staniloiu via 
 Digitalmars-d <digitalmars-d puremagic.com> wrote:

 [...]


 So it's probably worth giving the generated header a test 
 against gdc then.

 https://github.com/gcc-mirror/gcc/tree/ibuclaw/gdc


I'll give this a shot. Could you please give me a helping hand on 
how to get gdc to compile with the generated `frontend.h`?


 [...]


 So this would be part of the compiler rather than a standalone 
 application?  What's the rationale there?


Keeping it as part of the compiler doesn't require to redo all 
steps that `mars.d` does and it is also consistent with the 
current `.di` and `.json` generators.
Jul 17 2019
next sibling parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Wed, 17 Jul 2019 at 15:25, Eduard Staniloiu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Wednesday, 17 July 2019 at 12:40:38 UTC, Iain Buclaw wrote:

 On Tue, 16 Jul 2019 at 15:20, Eduard Staniloiu via
 Digitalmars-d <digitalmars-d puremagic.com> wrote:

 [...]


 So it's probably worth giving the generated header a test
 against gdc then.

 https://github.com/gcc-mirror/gcc/tree/ibuclaw/gdc


 I'll give this a shot. Could you please give me a helping hand on
 how to get gdc to compile with the generated `frontend.h`?


Should be straightforward, all front-end includes have the explicit path, e.g:

#include "dmd/aggregate.h"

Just replace them with:

#include "dmd/frontend.h"

And see if it fails to compile or bootstrap. :-)


 [...]


 So this would be part of the compiler rather than a standalone
 application?  What's the rationale there?


 Keeping it as part of the compiler doesn't require to redo all
 steps that `mars.d` does and it is also consistent with the
 current `.di` and `.json` generators.


But apart from the construction (and possible destruction) of all
frontend types, there shouldn't be anything duplicated.

Did you consider cross compiler generation?

-- 
Iain
Jul 17 2019
parent reply Seb <seb wilzba.ch> writes:
On Wednesday, 17 July 2019 at 13:35:35 UTC, Iain Buclaw wrote:

 But apart from the construction (and possible destruction) of 
 all frontend types, there shouldn't be anything duplicated.

 Did you consider cross compiler generation?


Motivations for doing it inside DMD instead of a separate library:

- DMD frontend isn't stable (AST nodes are often changed)
- DMD frontend isn't SemVer-tagged (so you can only rely on 
~master)
- people expressed a lot of interest in this being integrated 
directly
- easier bootstrapping (though not really a concern if we require 
the generated headers to be checked into the dmd repository)
Jul 17 2019
parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Wed, 17 Jul 2019 at 15:50, Seb via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Wednesday, 17 July 2019 at 13:35:35 UTC, Iain Buclaw wrote:

 But apart from the construction (and possible destruction) of
 all frontend types, there shouldn't be anything duplicated.

 Did you consider cross compiler generation?


 Motivations for doing it inside DMD instead of a separate library:

 - DMD frontend isn't stable (AST nodes are often changed)
 - DMD frontend isn't SemVer-tagged (so you can only rely on
 ~master)
 - people expressed a lot of interest in this being integrated
 directly
 - easier bootstrapping (though not really a concern if we require
 the generated headers to be checked into the dmd repository)


It doesn't make bootstrapping easier, as you will be starting from a
base where either:
1. There is no D compiler
2. The D compiler does not understand -HC

Having a standalone application means you only need to worry about (1)
in order to be able to generate headers to then build the D compiler.

-- 
Iain
Jul 17 2019
parent Eduard Staniloiu <edi33416 gmail.com> writes:
On Wednesday, 17 July 2019 at 14:26:11 UTC, Iain Buclaw wrote:

 On Wed, 17 Jul 2019 at 15:50, Seb via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Wednesday, 17 July 2019 at 13:35:35 UTC, Iain Buclaw wrote:

 But apart from the construction (and possible destruction) 
 of all frontend types, there shouldn't be anything 
 duplicated.

 Did you consider cross compiler generation?


 Motivations for doing it inside DMD instead of a separate 
 library:

 - DMD frontend isn't stable (AST nodes are often changed)
 - DMD frontend isn't SemVer-tagged (so you can only rely on
 ~master)
 - people expressed a lot of interest in this being integrated
 directly
 - easier bootstrapping (though not really a concern if we 
 require
 the generated headers to be checked into the dmd repository)


 It doesn't make bootstrapping easier, as you will be starting 
 from a
 base where either:
 1. There is no D compiler
 2. The D compiler does not understand -HC

 Having a standalone application means you only need to worry 
 about (1) in order to be able to generate headers to then build 
 the D compiler.


Maybe I'm missing something, but wouldn't the bootstrapping stay 
the same?
You start from (1) - no D compiler, you bootstrap the D compiler, 
and then you get `-HC`, right?

With the tool approach, you're still at (1), you bootstrap the D 
compiler and then build the tool with the bootstrapped D compiler.

Am I missing something?
Jul 17 2019
prev sibling parent reply Eduard Staniloiu <edi33416 gmail.com> writes:
I want to give a heads-up to everyone for the following issue 
(this had me scratching my head for a couple of days unit I 
caught it):

The current code doesn't take into account `enum BaseType`s, ex:
```
enum TOK : ubyte { /* ... */ }

class Expression
{
   TOK op;
   /* ... */
}
```

The `enum TOK` above gets generated as
```
enum TOK { /* ... */ }
```

According to the C standard, the compilers are free to pick the 
type that can fit the enum and most of them will pick `int` as a 
base type; thus `sizeof(TOK) -> 4UL`.
As you can see, this is a problem as the D object files will 
consider `TOK` to be one byte and the C object files will 
consider it to be four bytes.

The manual header implementation did this clever trick to solve 
the problem
```
#typedef unsigned char TOK;
enum
{
   TOKmem1,
   /* ... */
};

class Expression
{
   TOK op;
   /* ... */
}
```

The above takes advantage of the fact that enum member fields are 
in the global namespace, so the values will exist, and since they 
can fit in an unsigned char, the code will work through the 
typedef.

All of the above is required because C++98/03 doesn't have 
support for enum base types.

I have the following question: Can we use C++11 or would that 
break GDC / LDC? With C++11 we could use `enum class` which would 
solve this nicely. The issue with `enum class` is that it will 
break code since not the fields need to be prefixed with the enum 
name.

But still, what do you think? What are the pros and cons of 
supporting/using C++11?
Jul 17 2019
next sibling parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Wed, 17 Jul 2019 at 15:40, Eduard Staniloiu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 I have the following question: Can we use C++11 or would that
 break GDC / LDC? With C++11 we could use `enum class` which would
 solve this nicely. The issue with `enum class` is that it will
 break code since not the fields need to be prefixed with the enum
 name.

 But still, what do you think? What are the pros and cons of
 supporting/using C++11?


No you cannot use C++11, yes it will break GDC bootstrap.

-- 
Iain
Jul 17 2019
parent reply Eduard Staniloiu <edi33416 gmail.com> writes:
On Wednesday, 17 July 2019 at 14:28:34 UTC, Iain Buclaw wrote:

 On Wed, 17 Jul 2019 at 15:40, Eduard Staniloiu via 
 Digitalmars-d <digitalmars-d puremagic.com> wrote:

 I have the following question: Can we use C++11 or would that 
 break GDC / LDC? With C++11 we could use `enum class` which 
 would solve this nicely. The issue with `enum class` is that 
 it will break code since not the fields need to be prefixed 
 with the enum name.

 But still, what do you think? What are the pros and cons of 
 supporting/using C++11?


 No you cannot use C++11, yes it will break GDC bootstrap.


Can you elaborate a bit on this, please? I'm curious what are the 
reasons why we cannot use it.

Not really pushing for C++11, but I'd like to understand what is/ 
which are the blocker/s.

Thank you
Jul 17 2019
parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On Wed, 17 Jul 2019 at 18:05, Eduard Staniloiu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Wednesday, 17 July 2019 at 14:28:34 UTC, Iain Buclaw wrote:

 On Wed, 17 Jul 2019 at 15:40, Eduard Staniloiu via
 Digitalmars-d <digitalmars-d puremagic.com> wrote:

 I have the following question: Can we use C++11 or would that
 break GDC / LDC? With C++11 we could use `enum class` which
 would solve this nicely. The issue with `enum class` is that
 it will break code since not the fields need to be prefixed
 with the enum name.

 But still, what do you think? What are the pros and cons of
 supporting/using C++11?


 No you cannot use C++11, yes it will break GDC bootstrap.


 Can you elaborate a bit on this, please? I'm curious what are the
 reasons why we cannot use it.

 Not really pushing for C++11, but I'd like to understand what is/
 which are the blocker/s.


The stage1 gcc bootstrap is built with -std=gnu++98, assumedly to be
compatible with the widest range of platforms possible that gcc
supports.  This is something controlled by the toplevel, and not in
the per-language build config.

-- 
Iain
Jul 17 2019
prev sibling next sibling parent Eduard Staniloiu <edi33416 gmail.com> writes:
On Wednesday, 17 July 2019 at 13:38:44 UTC, Eduard Staniloiu 
wrote:

 I want to give a heads-up to everyone for the following issue 
 (this had me scratching my head for a couple of days unit I 
 caught it):

 The current code doesn't take into account `enum BaseType`s, ex:


Update: now the code takes into account enum base types. The 
generated code uses the `typedef` trick to have things work.
Jul 17 2019
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On Wed, Jul 17, 2019 at 6:40 AM Eduard Staniloiu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 I want to give a heads-up to everyone for the following issue
 (this had me scratching my head for a couple of days unit I
 caught it):

 The current code doesn't take into account `enum BaseType`s, ex:
 ```
 enum TOK : ubyte { /* ... */ }

 class Expression
 {
    TOK op;
    /* ... */
 }
 ```

 The `enum TOK` above gets generated as
 ```
 enum TOK { /* ... */ }
 ```


So... use:

enum class TOK : unsigned char { /* */ };

??
Jul 17 2019
parent reply Eduard Staniloiu <edi33416 gmail.com> writes:
On Wednesday, 17 July 2019 at 17:43:21 UTC, Manu wrote:

 On Wed, Jul 17, 2019 at 6:40 AM Eduard Staniloiu via 
 Digitalmars-d <digitalmars-d puremagic.com> wrote:

 I want to give a heads-up to everyone for the following issue 
 (this had me scratching my head for a couple of days unit I 
 caught it):

 The current code doesn't take into account `enum BaseType`s, 
 ex:
 ```
 enum TOK : ubyte { /* ... */ }

 class Expression
 {
    TOK op;
    /* ... */
 }
 ```

 The `enum TOK` above gets generated as
 ```
 enum TOK { /* ... */ }
 ```


 So... use:

 enum class TOK : unsigned char { /* */ };

 ??


That’s what I would like to use, but `enum class`es are starting 
with C++11

This will also lead to breaking code as the fields must now be 
accessed as `TOK::field`.

That being said, I think C++11 would be nice
Jul 17 2019
next sibling parent Manu <turkeyman gmail.com> writes:
On Wed, Jul 17, 2019 at 11:45 AM Eduard Staniloiu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Wednesday, 17 July 2019 at 17:43:21 UTC, Manu wrote:

 On Wed, Jul 17, 2019 at 6:40 AM Eduard Staniloiu via
 Digitalmars-d <digitalmars-d puremagic.com> wrote:

 I want to give a heads-up to everyone for the following issue
 (this had me scratching my head for a couple of days unit I
 caught it):

 The current code doesn't take into account `enum BaseType`s,
 ex:
 ```
 enum TOK : ubyte { /* ... */ }

 class Expression
 {
    TOK op;
    /* ... */
 }
 ```

 The `enum TOK` above gets generated as
 ```
 enum TOK { /* ... */ }
 ```


 So... use:

 enum class TOK : unsigned char { /* */ };

 ??


 That’s what I would like to use, but `enum class`es are starting
 with C++11

 This will also lead to breaking code as the fields must now be
 accessed as `TOK::field`.

 That being said, I think C++11 would be nice


Breaking what? This feature doesn't exist yet! There's nobody with
code written against these generated headers...
Jul 17 2019
prev sibling parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Wed, 17 Jul 2019 at 23:00, Manu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Wed, Jul 17, 2019 at 11:45 AM Eduard Staniloiu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Wednesday, 17 July 2019 at 17:43:21 UTC, Manu wrote:

 On Wed, Jul 17, 2019 at 6:40 AM Eduard Staniloiu via
 Digitalmars-d <digitalmars-d puremagic.com> wrote:

 I want to give a heads-up to everyone for the following issue
 (this had me scratching my head for a couple of days unit I
 caught it):

 The current code doesn't take into account `enum BaseType`s,
 ex:
 ```
 enum TOK : ubyte { /* ... */ }

 class Expression
 {
    TOK op;
    /* ... */
 }
 ```

 The `enum TOK` above gets generated as
 ```
 enum TOK { /* ... */ }
 ```


 So... use:

 enum class TOK : unsigned char { /* */ };

 ??


 That’s what I would like to use, but `enum class`es are starting
 with C++11

 This will also lead to breaking code as the fields must now be
 accessed as `TOK::field`.

 That being said, I think C++11 would be nice


 Breaking what? This feature doesn't exist yet! There's nobody with
 code written against these generated headers...


LDC and GDC builds will break as a result of switching to
auto-generated headers, but that's expected when things switch to
mechanical naming anyway...  we'll need to update our side of the
compiler regardless.

-- 
Iain
Jul 17 2019
parent Eduard Staniloiu <edi33416 gmail.com> writes:
On Thursday, 18 July 2019 at 05:02:03 UTC, Iain Buclaw wrote:

 On Wed, 17 Jul 2019 at 23:00, Manu via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Wed, Jul 17, 2019 at 11:45 AM Eduard Staniloiu via 
 Digitalmars-d <digitalmars-d puremagic.com> wrote:

 [...]


 Breaking what? This feature doesn't exist yet! There's nobody 
 with code written against these generated headers...


 LDC and GDC builds will break as a result of switching to 
 auto-generated headers, but that's expected when things switch 
 to mechanical naming anyway...  we'll need to update our side 
 of the compiler regardless.


Yep, that's the breaking I was referring to
Jul 18 2019
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On Wed, Jul 17, 2019 at 7:29 AM Iain Buclaw via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Wed, 17 Jul 2019 at 15:40, Eduard Staniloiu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 I have the following question: Can we use C++11 or would that
 break GDC / LDC? With C++11 we could use `enum class` which would
 solve this nicely. The issue with `enum class` is that it will
 break code since not the fields need to be prefixed with the enum
 name.

 But still, what do you think? What are the pros and cons of
 supporting/using C++11?


 No you cannot use C++11, yes it will break GDC bootstrap.


This needs to be optional. I can't NOT use C++11 for instance...
We could read the --extern-std flag to drive the output perhaps?
Jul 17 2019
parent reply Eduard Staniloiu <edi33416 gmail.com> writes:
On Wednesday, 17 July 2019 at 18:00:38 UTC, Manu wrote:

 On Wed, Jul 17, 2019 at 7:29 AM Iain Buclaw via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Wed, 17 Jul 2019 at 15:40, Eduard Staniloiu via 
 Digitalmars-d <digitalmars-d puremagic.com> wrote:

 I have the following question: Can we use C++11 or would 
 that break GDC / LDC? With C++11 we could use `enum class` 
 which would solve this nicely. The issue with `enum class` 
 is that it will break code since not the fields need to be 
 prefixed with the enum name.

 But still, what do you think? What are the pros and cons of 
 supporting/using C++11?


 No you cannot use C++11, yes it will break GDC bootstrap.


 This needs to be optional. I can't NOT use C++11 for 
 instance... We could read the --extern-std flag to drive the 
 output perhaps?


We could use `#if __cplusplus < 201103L` to emit for C++98 and 
C++11, but I was hoping we could avoid that.

I guess it will come to inserting a bunch of `#if`s
Jul 17 2019
parent Manu <turkeyman gmail.com> writes:
On Wed, Jul 17, 2019 at 11:55 AM Eduard Staniloiu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Wednesday, 17 July 2019 at 18:00:38 UTC, Manu wrote:

 On Wed, Jul 17, 2019 at 7:29 AM Iain Buclaw via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Wed, 17 Jul 2019 at 15:40, Eduard Staniloiu via
 Digitalmars-d <digitalmars-d puremagic.com> wrote:

 I have the following question: Can we use C++11 or would
 that break GDC / LDC? With C++11 we could use `enum class`
 which would solve this nicely. The issue with `enum class`
 is that it will break code since not the fields need to be
 prefixed with the enum name.

 But still, what do you think? What are the pros and cons of
 supporting/using C++11?


 No you cannot use C++11, yes it will break GDC bootstrap.


 This needs to be optional. I can't NOT use C++11 for
 instance... We could read the --extern-std flag to drive the
 output perhaps?


 We could use `#if __cplusplus < 201103L` to emit for C++98 and
 C++11, but I was hoping we could avoid that.

 I guess it will come to inserting a bunch of `#if`s


No, that doesn't work, because then code written against the header
needs to be adapted according to the -std used to compile.
I think it would be fine to use the -extern-std supplied to DMD to
drive the output that way.
Jul 17 2019
prev sibling next sibling parent David Nadlinger <code klickverbot.at> writes:
On Wednesday, 17 July 2019 at 13:38:44 UTC, Eduard Staniloiu 
wrote:

 I have the following question: Can we use C++11 or would that 
 break GDC / LDC?


LDC has been requiring C+11 to build for a long time now. —David
Jul 17 2019
prev sibling next sibling parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Wednesday, 17 July 2019 at 13:38:44 UTC, Eduard Staniloiu 
wrote:

 I want to give a heads-up to everyone for the following issue 
 (this had me scratching my head for a couple of days unit I 
 caught it):

 [...]


it would be needed for pure C also. So would not work.

As for smaller sized enum part of a struct I sometime use

struct{
    enum TOK op:CHAR_BIT;
}

works in most cases except when you need to take its address or 
its offset.
Jul 17 2019
prev sibling next sibling parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Wed, 17 Jul 2019 at 20:01, Manu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Wed, Jul 17, 2019 at 7:29 AM Iain Buclaw via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Wed, 17 Jul 2019 at 15:40, Eduard Staniloiu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 I have the following question: Can we use C++11 or would that
 break GDC / LDC? With C++11 we could use `enum class` which would
 solve this nicely. The issue with `enum class` is that it will
 break code since not the fields need to be prefixed with the enum
 name.

 But still, what do you think? What are the pros and cons of
 supporting/using C++11?


 No you cannot use C++11, yes it will break GDC bootstrap.


 This needs to be optional. I can't NOT use C++11 for instance...
 We could read the --extern-std flag to drive the output perhaps?


I assumed the question was only about dmd headers, and not the tool itself.

Naturally, there should be a way to configure the style or language
standards you want the code to be outputted in (and this is another
reason why I think a standalone app is better).

-- 
Iain
Jul 17 2019
parent reply Johannes Pfau <nospam example.com> writes:
Am Thu, 18 Jul 2019 06:41:16 +0200 schrieb Iain Buclaw:


 
 Naturally, there should be a way to configure the style or language
 standards you want the code to be outputted in (and this is another
 reason why I think a standalone app is better).


Has extending the DMD json output, then writing a tool to process that 
json into c++ .h files been considered?

This way you don't have to worry about compiler API stability and you 
could make the c++ generation as flexible as you want (and you can target 
other, non-C++ languages easily).

-- 
Johannes
Jul 19 2019
parent Gregor =?UTF-8?B?TcO8Y2ts?= <gregormueckl gmx.de> writes:
On Friday, 19 July 2019 at 21:08:12 UTC, Johannes Pfau wrote:

 Am Thu, 18 Jul 2019 06:41:16 +0200 schrieb Iain Buclaw:


 
 Naturally, there should be a way to configure the style or 
 language standards you want the code to be outputted in (and 
 this is another reason why I think a standalone app is better).


 Has extending the DMD json output, then writing a tool to 
 process that json into c++ .h files been considered?

 This way you don't have to worry about compiler API stability 
 and you could make the c++ generation as flexible as you want 
 (and you can target other, non-C++ languages easily).


Any such external tool would have to replicate a pretty decent 
chunk of the D language AST to cover all the relevant nuances of 
definitions in D. To me, that sounds like writing a lot of code 
that duplicates stuff that's already in the compiler.
Jul 28 2019
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On Wed, Jul 17, 2019 at 9:41 PM Iain Buclaw via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Wed, 17 Jul 2019 at 20:01, Manu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Wed, Jul 17, 2019 at 7:29 AM Iain Buclaw via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Wed, 17 Jul 2019 at 15:40, Eduard Staniloiu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 I have the following question: Can we use C++11 or would that
 break GDC / LDC? With C++11 we could use `enum class` which would
 solve this nicely. The issue with `enum class` is that it will
 break code since not the fields need to be prefixed with the enum
 name.

 But still, what do you think? What are the pros and cons of
 supporting/using C++11?


 No you cannot use C++11, yes it will break GDC bootstrap.


 This needs to be optional. I can't NOT use C++11 for instance...
 We could read the --extern-std flag to drive the output perhaps?


 I assumed the question was only about dmd headers, and not the tool itself.

 Naturally, there should be a way to configure the style or language
 standards you want the code to be outputted in (and this is another
 reason why I think a standalone app is better).


A stand-alone app would be unbelievably disappointing. I think I would
not use this if it were a stand-alone app; it would have failed at its
entire value proposition for us. We would stick with writing C++ code
and D as an afterthought because there would be no friction advantage
this way around.
The point is to simplify the build environment, not make it a whole lot worse :/
Jul 18 2019
parent Gregor =?UTF-8?B?TcO8Y2ts?= <gregormueckl gmx.de> writes:
On Thursday, 18 July 2019 at 18:56:28 UTC, Manu wrote:

 A stand-alone app would be unbelievably disappointing. I think 
 I would
 not use this if it were a stand-alone app; it would have failed 
 at its
 entire value proposition for us. We would stick with writing 
 C++ code
 and D as an afterthought because there would be no friction 
 advantage
 this way around.
 The point is to simplify the build environment, not make it a 
 whole lot worse :/


I second this. I've tried to create this C++ header feature a few 
months ago, but had to put it aside for lack of time. But the 
compiler frontend provides a lot of infrastructure that would 
need to be duplicated in any potential external tool that it's 
just going to be easier to have this as part of the compiler 
itself.

Besides, getting updated C++ headers for virtually free on each D 
compiler run can eliminate a whole host of potential bugs 
stemming from mismatched definitions because of incorrectly 
scripted build tooling. Multi-language/multi-compiler builds are 
unpleasant enough to automate as it is.
Jul 28 2019