www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Now that the dmd backend is in D...?

reply Jonathan Marler <johnnymarler gmail.com> writes:
Now that the dmd backend is in D, are there plans to remove the 
old C++ organization?

Today, the backend is split into backend headers and backend 
sources, much like a C/C++ project.  The backend sources are 
compiled separately from DMD into a library, and the backend 
headers are compiled with DMD in order to link to the precompiled 
backend library.

You can see which files belong to which by looking at the 
posix.mak file:

BACK_HDRS=$C/cc.d $C/cdef.d $C/cgcv.d $C/code.d $C/cv4.d $C/dt.d 
$C/el.d $C/global.d \
	$C/obj.d $C/oper.d $C/outbuf.d $C/rtlsym.d $C/code_x86.d 
$C/iasm.d $C/codebuilder.d \
	$C/ty.d $C/type.d $C/exh.d $C/mach.d $C/mscoff.d $C/dwarf.d 
$C/dwarf2.d $C/xmm.d \
	$C/dlist.d $C/melf.d $C/varstats.di $C/dt.d

BACK_DOBJS = bcomplex.o evalu8.o divcoeff.o dvec.o go.o gsroa.o 
glocal.o gdag.o gother.o gflow.o \
	out.o \
	gloop.o compress.o cgelem.o cgcs.o ee.o cod4.o cod5.o nteh.o 
blockopt.o mem.o cg.o cgreg.o \
	dtype.o debugprint.o fp.o symbol.o elem.o dcode.o cgsched.o 
cg87.o cgxmm.o cgcod.o cod1.o cod2.o \
	cod3.o cv8.o dcgcv.o pdata.o util2.o var.o md5.o backconfig.o 
ph2.o drtlsym.o dwarfeh.o ptrntab.o \
	aarray.o dvarstats.o dwarfdbginf.o elfobj.o cgen.o os.o goh.o 
barray.o cgcse.o elpicpie.o


However, now that it's all in D, is there a plan to remove the 
backend headers?  Then once that is done, would we remove all the 
extern (C++) modifiers?  I realize that the extern (C++) 
modifiers cannot be removed until the backend headers are removed 
because making everything extern (D) would cause the module name 
to be used in the mangled symbol so the headers and sources would 
no longer match.

So is there any reason to keep the separate headers/sources in 
the DMD backend?
Jul 31 2019
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/31/2019 8:03 PM, Jonathan Marler wrote:

 Now that the dmd backend is in D, are there plans to remove the old C++ 
 organization?


No. It's still used to compile the DMC++ compiler. Trying to reshuffle all that 
around will cause me major problems.
Aug 01 2019
next sibling parent Jonathan Marler <johnnymarler gmail.com> writes:
On Thursday, 1 August 2019 at 08:59:28 UTC, Walter Bright wrote:

 On 7/31/2019 8:03 PM, Jonathan Marler wrote:

 Now that the dmd backend is in D, are there plans to remove 
 the old C++ organization?


 No. It's still used to compile the DMC++ compiler. Trying to 
 reshuffle all that around will cause me major problems.


I see...that would make it pretty hard to link it with the DMC++ 
compiler :)
Aug 01 2019
prev sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Thursday, 1 August 2019 at 08:59:28 UTC, Walter Bright wrote:

 On 7/31/2019 8:03 PM, Jonathan Marler wrote:

 Now that the dmd backend is in D, are there plans to remove 
 the old C++ organization?


 No. It's still used to compile the DMC++ compiler. Trying to 
 reshuffle all that around will cause me major problems.


Obviously we need a way to ensure that all changes to the backend 
don't cause any problems for DMC++.

Would you be interested in the community setting up a repo with 
both dmd and dmc, so each change to the backend (whether it is a 
bug fix, enhancement or refactoring) is tested against both dmd 
and dmc - they both need to compile and have their respective 
test suites pass.

If you're interested, we can brainstorm different possibilities:

* Move dmc repo to dmd/src/dmc, move dmc's tests to dmd/tests/dmc
* Include dmd as a git submodule in the dmc repo - a PR that 
affects the backend will need to be first merged in dmc and then 
dmd
* etc...
Aug 01 2019
next sibling parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Thursday, 1 August 2019 at 10:20:33 UTC, Petar Kirov 
[ZombineDev] wrote:

 On Thursday, 1 August 2019 at 08:59:28 UTC, Walter Bright wrote:

 [...]


 Obviously we need a way to ensure that all changes to the 
 backend don't cause any problems for DMC++.

 Would you be interested in the community setting up a repo with 
 both dmd and dmc, so each change to the backend (whether it is 
 a bug fix, enhancement or refactoring) is tested against both 
 dmd and dmc - they both need to compile and have their 
 respective test suites pass.

 If you're interested, we can brainstorm different possibilities:

 * Move dmc repo to dmd/src/dmc, move dmc's tests to 
 dmd/tests/dmc
 * Include dmd as a git submodule in the dmc repo - a PR that 
 affects the backend will need to be first merged in dmc and 
 then dmd
 * etc...


I think DMC++ is closed source.
Aug 01 2019
parent Dennis <dkorpel gmail.com> writes:
On Thursday, 1 August 2019 at 10:31:23 UTC, Jonathan Marler wrote:

 I think DMC++ is closed source.


It used to be.
https://github.com/DigitalMars/Compiler
Aug 01 2019
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/1/2019 3:20 AM, Petar Kirov [ZombineDev] wrote:

 On Thursday, 1 August 2019 at 08:59:28 UTC, Walter Bright wrote:

 On 7/31/2019 8:03 PM, Jonathan Marler wrote:

 Now that the dmd backend is in D, are there plans to remove the old C++ 
 organization?


 No. It's still used to compile the DMC++ compiler. Trying to reshuffle all 
 that around will cause me major problems.


 
 Obviously we need a way to ensure that all changes to the backend don't cause 
 any problems for DMC++.
 
 Would you be interested in the community setting up a repo with both dmd and 
 dmc, so each change to the backend (whether it is a bug fix, enhancement or 
 refactoring) is tested against both dmd and dmc - they both need to compile
and 
 have their respective test suites pass.
 
 If you're interested, we can brainstorm different possibilities:
 
 * Move dmc repo to dmd/src/dmc, move dmc's tests to dmd/tests/dmc
 * Include dmd as a git submodule in the dmc repo - a PR that affects the
backend 
 will need to be first merged in dmc and then dmd
 * etc...


That is possible now that DMC++ is Boost licensed. It's currently at:

https://github.com/DigitalMars/Compiler

and is reasonably in sync with DMD. However, having endured multiple grand 
reshufflings of the DMD source code, and having recently headed off yet another 
one, I'm not much in favor of it.

If you look at the history of the back end, you'll see I've been incrementally 
updating it to be more modern and more tractable. I expect to continue with
that.

What I'd like to see, however, is to get the backend covered by the code 
coverage thing in the D test suite. Currently, only the front end is.
Aug 01 2019
next sibling parent reply Seb <seb wilzba.ch> writes:
On Thursday, 1 August 2019 at 20:24:12 UTC, Walter Bright wrote:

 What I'd like to see, however, is to get the backend covered by 
 the code coverage thing in the D test suite. Currently, only 
 the front end is.


This wouldn't be too hard: https://github.com/dlang/dmd/pull/10259
Aug 01 2019
parent reply Seb <seb wilzba.ch> writes:
On Thursday, 1 August 2019 at 21:27:49 UTC, Seb wrote:

 On Thursday, 1 August 2019 at 20:24:12 UTC, Walter Bright wrote:

 What I'd like to see, however, is to get the backend covered 
 by the code coverage thing in the D test suite. Currently, 
 only the front end is.


 This wouldn't be too hard: 
 https://github.com/dlang/dmd/pull/10259


Now available on CodeCov: 
https://codecov.io/gh/dlang/dmd/tree/master/src/dmd/backend
Aug 01 2019
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/1/2019 3:49 PM, Seb wrote:

 On Thursday, 1 August 2019 at 21:27:49 UTC, Seb wrote:

 On Thursday, 1 August 2019 at 20:24:12 UTC, Walter Bright wrote:

 What I'd like to see, however, is to get the backend covered by the code 
 coverage thing in the D test suite. Currently, only the front end is.


 This wouldn't be too hard: https://github.com/dlang/dmd/pull/10259


 
 Now available on CodeCov: 
 https://codecov.io/gh/dlang/dmd/tree/master/src/dmd/backend


Thanks much for the quick action!
Aug 01 2019
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/1/2019 3:49 PM, Seb wrote:

 Now available on CodeCov: 
 https://codecov.io/gh/dlang/dmd/tree/master/src/dmd/backend


This is the first time for a coverage report for the back end! It's better than 
I expected, but still not good.
Aug 01 2019
prev sibling parent reply Exil <Exil gmall.com> writes:
On Thursday, 1 August 2019 at 20:24:12 UTC, Walter Bright wrote:

 and is reasonably in sync with DMD. However, having endured 
 multiple grand reshufflings of the DMD source code, and having 
 recently headed off yet another one, I'm not much in favor of 
 it.


So D suffers and can't make any significant changes to the 
backend in order to accommodate an old dead project that isn't 
being actively developed?
Aug 01 2019
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/1/2019 4:05 PM, Exil wrote:

 On Thursday, 1 August 2019 at 20:24:12 UTC, Walter Bright wrote:

 and is reasonably in sync with DMD. However, having endured multiple grand 
 reshufflings of the DMD source code, and having recently headed off yet 
 another one, I'm not much in favor of it.


 
 So D suffers and can't make any significant changes to the backend in order to 
 accommodate an old dead project that isn't being actively developed?


In my not-so-humble experience, shuffling code about is too easy, usually an 
illusion of progress (look ma! I changed lots of lines of code!).

Refactoring to improve the design and flow of the code is much harder, usually 
requiring a thorough understanding of the code. Switching the backend to using 
CodeBuilder is an example of such.
Aug 01 2019
parent Exil <Exil gmall.com> writes:
On Thursday, 1 August 2019 at 23:54:28 UTC, Walter Bright wrote:

 On 8/1/2019 4:05 PM, Exil wrote:

 On Thursday, 1 August 2019 at 20:24:12 UTC, Walter Bright 
 wrote:

 and is reasonably in sync with DMD. However, having endured 
 multiple grand reshufflings of the DMD source code, and 
 having recently headed off yet another one, I'm not much in 
 favor of it.


 
 So D suffers and can't make any significant changes to the 
 backend in order to accommodate an old dead project that isn't 
 being actively developed?


 In my not-so-humble experience, shuffling code about is too 
 easy, usually an illusion of progress (look ma! I changed lots 
 of lines of code!).


Who said it was for "progress"? Don't paint it as something it 
isn't. It's usually about maintainability and readability. 
Something the backend lacks. Lack of comments, horrible naming 
convention, etc... With documentation like this:

     /*****************
      * Free code
      */
     void code_free(code *cstart)


 Refactoring to improve the design and flow of the code is much 
 harder, usually requiring a thorough understanding of the code. 
 Switching the backend to using CodeBuilder is an example of 
 such.


There's more to it than that. I once had to refactor a monolithic 
class. It had dozens of functions that it really shouldn't have. 
I didn't write the code, someone else did and other developers 
over the years kept adding onto it until it got to the state it 
was in. It was first easier to -reshuffle- refactor the code into 
separate files to better understand how it works and to decouple 
everything. It made the code more readable and easier to 
understand, and from there to easily be redesigned into 
appropriate.

Kind of like how when you go from C -> D, rather than rewriting 
it in idiomatic D straight away, it is easier and reduces bugs to 
do a direct conversion from C to D. I think this is something you 
should be familiar with.
Aug 01 2019