www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Moving druntime into the DMD repository

reply Seb <seb wilzba.ch> writes:
This a thread to explore whether it would be feasible to do so.

Motivation
----------

DRuntime and DMD heavily depend on each other.

It happens very often that a PR needs to touch both and then a 
complicated three-step (or sometimes four-step PR series) needs 
to be done to keep the CIs happy.
Moreover, as the whole testsuite is at dmd, it happens more often 
that an error is caught in the testsuite, but would require a 
druntime PR or the opposite you make a PR to druntime and want 
its accompanying test to be checked by the CI.

How would this happen?
----------------------

A potential migration could look like this:

- lock druntime (e.g. only allowing admins to merge)
- move source code to dmd (of course with the git history, see 
e.g. [1])
- update the druntime/posix.mak build script to forward to the 
new source location
- update other dlang Makefiles
- remove everything else from druntime (optional check for the 
Makefile update step)
- update CIs and build scripts (more on this below)

[1] https://stackoverflow.com/a/10548919/3026245

What would need to be updated?
-----------------------------

- Makefiles for DMD, Druntime, Phobos, Tools, Dlang.org (mostly 
path updates)
- LDC/GDC (I think LDC already includes druntime as git submodule)

If we update the druntime/posix.mak file to call the 
druntime/posix.mak at its new location, then there would be zero 
breakage (druntime already requires dmd to cloned) and over time 
these can be updated:

- CI scripts (all of them explicitly clone druntime, but removing 
that line should be all)
- Release scripts at dlang/installer
- Packager build scripts

(most distros just ship the the libphobos2.{so,a} libraries, but 
if druntime is shipped too, it will require a path update for the 
release after such a potential migration.)

What do you think?
------------------

- Has the dmd/druntime split being annoying you too?
- Do you have a better suggestion?
- Would this break your workflow in a drastic way?
Jul 27 2018
next sibling parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Friday, 27 July 2018 at 11:03:50 UTC, Seb wrote:


 - Do you have a better suggestion?


Do we have a rich enough CI API to recognize dependencies between 
repositories?  For example, if I create a local branch in my dmd 
repository named `fix_bug` and a local branch in my druntime 
repository also named `fix_bug` and submit both PRs to their 
corresponding repositories, can the CI scripts recognize that 
they have the same branch name and pull them both for testing?

Mike
Jul 27 2018
parent Seb <seb wilzba.ch> writes:
On Friday, 27 July 2018 at 11:25:37 UTC, Mike Franklin wrote:

 On Friday, 27 July 2018 at 11:03:50 UTC, Seb wrote:


 - Do you have a better suggestion?


 Do we have a rich enough CI API to recognize dependencies 
 between repositories?  For example, if I create a local branch 
 in my dmd repository named `fix_bug` and a local branch in my 
 druntime repository also named `fix_bug` and submit both PRs to 
 their corresponding repositories, can the CI scripts recognize 
 that they have the same branch name and pull them both for 
 testing?

 Mike


It's already implemented for branches under the respective dlang 
repository.
This roughly happens on every CI:

```sh
for proj in druntime phobos; do
     if [ $BRANCH != master ] && [ $BRANCH != stable ] &&
            ! git ls-remote --exit-code --heads 
https://github.com/dlang/$proj.git $BRANCH > /dev/null; then

branches
         clone https://github.com/dlang/$proj.git ../$proj master
     else
         clone https://github.com/dlang/$proj.git ../$proj $BRANCH
     fi
done
```

However, this only applies if the PR is targeting the respective 
branch, which means the workflow is a bit more annoying

* push dmd branch to dlang/druntime-dmd
* open PR at dmd targeting master (from druntime-dmd)
* create druntime-dmd branch at dlang/druntime
* push changes to private branch
* open PR at druntime targetting druntime-dmd
* merge druntime-dmd back to master once the druntime PR has been 
approved and merged

It's a bit of an overhead for small changes though :/

Checking out the branch from your repository is problematic, 
because it's not exposed as environment variable and we would 
need to query the GitHub API to find this. Now the GitHub API 
gets rate-limited pretty quick, we would have to use our own 
GitHub API cache layer and ensure the requests coming from the 
CIs are really ours.
Jul 29 2018
prev sibling next sibling parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Friday, 27 July 2018 at 11:03:50 UTC, Seb wrote:


 What do you think?
 ------------------


Also, is there any other compelling reasons besides the test 
dependency problem to motivate merging the two repositories?  
What other benefits would it provide, if any?

Mike
Jul 27 2018
parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On 27 July 2018 at 13:32, Mike Franklin via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, 27 July 2018 at 11:03:50 UTC, Seb wrote:


 What do you think?
 ------------------



 Also, is there any other compelling reasons besides the test dependency
 problem to motivate merging the two repositories?  What other benefits would
 it provide, if any?

 Mike


I can't think of any, but such ABI breakages should be avoided.  The
preferred route for any such changes that require synchronisation
between dmd and druntime should be in order of:

1. Commit new functions/feature to druntime as dead/unused code.
2. Commit changes to dmd which activates the feature you added to druntime.
3. Add/turn on tests in both dmd and druntime.

Iain
Jul 27 2018
prev sibling next sibling parent reply Russel Winder <russel winder.org.uk> writes:
On Fri, 2018-07-27 at 11:03 +0000, Seb via Digitalmars-d wrote:

 This a thread to explore whether it would be feasible to do so.
=20
 Motivation
 ----------
=20
 DRuntime and DMD heavily depend on each other.
=20


But DMD is only one of the compilers, there is LDC and GDC. Is Druntime
unique to DMD or do the other compilers also use it?

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk
Jul 27 2018
parent Seb <seb wilzba.ch> writes:
On Friday, 27 July 2018 at 12:02:50 UTC, Russel Winder wrote:

 On Fri, 2018-07-27 at 11:03 +0000, Seb via Digitalmars-d wrote:

 This a thread to explore whether it would be feasible to do so.
 
 Motivation
 ----------
 
 DRuntime and DMD heavily depend on each other.
 


 But DMD is only one of the compilers, there is LDC and GDC. Is 
 Druntime unique to DMD or do the other compilers also use it?


It's unique to the DMD Frontend which is used by LDC/GDC ;-)
They use it with small modifications, e.g. for the LLVM 
intrinsincs and a few LDC specific features:

https://github.com/ldc-developers/druntime
https://github.com/dlang/druntime/compare/master...ldc-developers:ldc
Jul 27 2018
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, July 27, 2018 5:03:50 AM MDT Seb via Digitalmars-d wrote:

 What do you think?
 ------------------

 - Has the dmd/druntime split being annoying you too?
 - Do you have a better suggestion?
 - Would this break your workflow in a drastic way?


It would break all existing tools and scripts that are used to build the
existing repos - many of which are not official tools. So, yes, it would be
very annoying. That's not necessarily a good enough reason not to do it, but
IMHO, it really needs to provide solid benefits to rearrange things like
that for it to be worth breaking all of the existing tools that anyone uses
for building dmd, druntime, and Phobos.

It also arguably makes no sense in that a lot of what's in druntime has
nothing to do with dmd - e.g. all of the OS C bindings are in there. There
are also several modules that need to be in druntime, because druntime uses
them but don't really have much to do with the compiler (e.g. core.time and
core.thread). And not only from a code organizational standpoint does the
current separation make a lot of sense for a lot of what's in druntime, but
it also matters from the standpoint of who has permissions to do what. Right
now, it's reasonable to give privileges to folks to merge PRs for druntime
who have no business merging PRs for dmd. If the repos are merged, then
we're forced to either give some of those folks dmd merge rights or make the
smaller pool of folks who have merge rights for dmd deal with those PRs.

Also, given that druntime gets linked into Phobos, having that repo be part
of the compiler is that much weirder. In that sense, it belongs more with
Phobos than dmd.

The dependencies between the three repos do occasionally cause problems, but
overall, I think that the separation makes a lot of sense.

- Jonathan M Davis
Jul 27 2018
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
+1 leave it
Jul 27 2018
prev sibling next sibling parent reply Seb <seb wilzba.ch> writes:
On Friday, 27 July 2018 at 12:04:18 UTC, Jonathan M Davis wrote:

 On Friday, July 27, 2018 5:03:50 AM MDT Seb via Digitalmars-d 
 wrote:

 What do you think?
 ------------------

 - Has the dmd/druntime split being annoying you too?
 - Do you have a better suggestion?
 - Would this break your workflow in a drastic way?


 It would break all existing tools and scripts that are used to 
 build the existing repos - many of which are not official 
 tools. So, yes, it would be very annoying.


FWIW I don't think this would be the case. All build scripts I 
have seen call the Makefile (except for LDC/GDC, of course). The 
Makefile build would continue to work at any stage.


 That's not necessarily a good enough reason not to do it, but 
 IMHO, it really needs to provide solid benefits to rearrange 
 things like that for it to be worth breaking all of the 
 existing tools that anyone uses for building dmd, druntime, and 
 Phobos.


As mentioned, I don't know of any tool that builds druntime 
itself. Everyone calls the Makefiles.
If there are tools that build druntime on their own, it would be 
a very good point yes.


 It also arguably makes no sense in that a lot of what's in 
 druntime has nothing to do with dmd - e.g. all of the OS C 
 bindings are in there. There are also several modules that need 
 to be in druntime, because druntime uses them but don't really 
 have much to do with the compiler (e.g. core.time and 
 core.thread). And not only from a code organizational 
 standpoint does the current separation make a lot of sense for 
 a lot of what's in druntime, but it also matters from the 
 standpoint of who has permissions to do what. Right now, it's 
 reasonable to give privileges to folks to merge PRs for 
 druntime who have no business merging PRs for dmd.



 If the repos are merged, then we're forced to either give some 
 of those folks dmd merge rights or make the smaller pool of 
 folks who have merge rights for dmd deal with those PRs.


This shouldn't be a problem either as GitHub's CODEOWNERS files, 
are made for this use case and
all we need would be sth. this:

*  team-dmd
src/druntime  team-druntime

https://help.github.com/articles/enabling-required-reviews-for-pull-requests
(Step 8)


 Also, given that druntime gets linked into Phobos, having that 
 repo be part of the compiler is that much weirder. In that 
 sense, it belongs more with Phobos than dmd.

 The dependencies between the three repos do occasionally cause 
 problems, but overall, I think that the separation makes a lot 
 of sense.


It causes A LOT of problems in terms of maintaining the CI and 
build scripts,
moving things between DMD and adding test for druntime features 
to the dmd testsuite.
Jul 27 2018
parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, July 27, 2018 6:28:07 AM MDT Seb via Digitalmars-d wrote:

 On Friday, 27 July 2018 at 12:04:18 UTC, Jonathan M Davis wrote:

 On Friday, July 27, 2018 5:03:50 AM MDT Seb via Digitalmars-d

 wrote:

 What do you think?
 ------------------

 - Has the dmd/druntime split being annoying you too?
 - Do you have a better suggestion?
 - Would this break your workflow in a drastic way?


 It would break all existing tools and scripts that are used to
 build the existing repos - many of which are not official
 tools. So, yes, it would be very annoying.


 FWIW I don't think this would be the case. All build scripts I
 have seen call the Makefile (except for LDC/GDC, of course). The
 Makefile build would continue to work at any stage.


I'd be _very_ surprised if it doesn't break the tools that I use to build
the repos and keep them in sync. The biggest ones to look at though would
probably be DVM and digger.

I'd probably be okay with the build process being changed if we were
switching to something that was more maintainable than the makefiles, but to
rearrange repos? Not so much. In general, the build process is one thing
that I'd very much not like to see touched without something that's going to
significantly improve it, because pretty every time someone mucks with it, I
have to waste time fixing stuff so that it will build for me again, and
usually, the changes don't seem like they really improved things, just moved
them around.


 If the repos are merged, then we're forced to either give some
 of those folks dmd merge rights or make the smaller pool of
 folks who have merge rights for dmd deal with those PRs.


 This shouldn't be a problem either as GitHub's CODEOWNERS files,
 are made for this use case and
 all we need would be sth. this:

 *  team-dmd
 src/druntime  team-druntime


Well, that would certainly help, but it still seems like things that should
be separate are being shoved together. dmd and druntime are separate beasts
and IMHO should be kept as such. Also, even if the codeowners thing deals
with the permissions, it still makes it harder for folks to look through the
PR list for stuff that's relevant to what they might be merging, since the
dmd and druntime PR queues would presumably be the same one at that point.
Anyone who works on druntime but not dmd would have to deal with a ton of
dmd PRs, whereas currently, they don't.

_Maybe_ it would make sense if we were talking about putting all three repos
in one with clear subdirectories for the three projects, since then the
repos could be completely in sync at all times, and it would be easier for
folks to figure out how to build everything, whereas right now, it's a bit
of a pain to deal with it (hence when a number of us have written scripts or
tools to deal with the process), but I don't know if that really makes sense
when everything is considered - especially with how the PRs would all be in
one bucket. We have enough problems with PRs being properly reviewed in a
timely manner as it is. Either way, just putting two of them in a single
repo seems like it's causing all of the problems that merging them all
together would cause without really getting any of the benefits.


 The dependencies between the three repos do occasionally cause
 problems, but overall, I think that the separation makes a lot
 of sense.


 It causes A LOT of problems in terms of maintaining the CI and
 build scripts,
 moving things between DMD and adding test for druntime features
 to the dmd testsuite.


Does it really cause that many problems? If so, I'd honestly rather that we
look into adjusting how the CI works than merging repos. And if the problem
si that we have a lot of PRs where we need a dmd and druntime PR at the same
time, then I think that that's something to be concerned about. I'd be
worried if the compiler API needs to be changed that frequently.

And from a code cleanliness and maintenance standpoint, I really don't think
that it makes sense to merge dmd and druntime. Some pieces are
interconnected, yes, but most of druntime is a very separate thing, and for
the most part, it should be treated as separate.

- Jonathan M Davis
Jul 27 2018
prev sibling parent John Colvin <john.loughran.colvin gmail.com> writes:
On Friday, 27 July 2018 at 12:04:18 UTC, Jonathan M Davis wrote:

 On Friday, July 27, 2018 5:03:50 AM MDT Seb via Digitalmars-d 
 wrote:

 What do you think?
 ------------------

 - Has the dmd/druntime split being annoying you too?
 - Do you have a better suggestion?
 - Would this break your workflow in a drastic way?


 It would break all existing tools and scripts that are used to 
 build the existing repos - many of which are not official 
 tools. So, yes, it would be very annoying. That's not 
 necessarily a good enough reason not to do it, but IMHO, it 
 really needs to provide solid benefits to rearrange things like 
 that for it to be worth breaking all of the existing tools that 
 anyone uses for building dmd, druntime, and Phobos.


Not necessarily. The druntime repo could just contain makefiles 
that forward to the dmd ones.
Jul 27 2018
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 27 July 2018 at 11:03:50 UTC, Seb wrote:

 This a thread to explore whether it would be feasible to do so.

 What do you think?
 ------------------


One is a compiler the other is a (optional!) library.
merging the two means forcing druntime onto people who want to 
use dmd.

the seperation of the repositories is a good thing, since it 
prompt people to think before breaking compatibilty with the 
runtime.

Which _should_ not happen often anyhow!

druntime and dmd are seperate things.


 - Has the dmd/druntime split being annoying you too?


The split has never anonyned me. I find it to be helpful in fact.


 - Do you have a better suggestion?


Yes, try to implement features in non-breaking, non 
runtime-dependent way.


 - Would this break your workflow in a drastic way?


I'd have to see the effects, to be definite about this, but 
likely the answer is yes.
Jul 27 2018
prev sibling next sibling parent Joakim <dlang joakim.fea.st> writes:
On Friday, 27 July 2018 at 11:03:50 UTC, Seb wrote:

 This a thread to explore whether it would be feasible to do so.

 Motivation
 ----------

 [...]


Not much.


 - Do you have a better suggestion?


No.


 - Would this break your workflow in a drastic way?


No, don't really use the official repos anymore, been using the 
LDC forks instead, so doesn't matter to me how the official git 
repos for dmd are organized.

LDC splits off the DMD testsuite as its own git repo too, so any 
merged directory can always be split off to another git repo if 
wanted:

https://github.com/ldc-developers/dmd-testsuite
Jul 27 2018
prev sibling next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/27/18 7:03 AM, Seb wrote:

 This a thread to explore whether it would be feasible to do so.
 
 Motivation
 ----------
 
 DRuntime and DMD heavily depend on each other.
 
 It happens very often that a PR needs to touch both and then a 
 complicated three-step (or sometimes four-step PR series) needs to be 
 done to keep the CIs happy.
 Moreover, as the whole testsuite is at dmd, it happens more often that 
 an error is caught in the testsuite, but would require a druntime PR or 
 the opposite you make a PR to druntime and want its accompanying test to 
 be checked by the CI.


The auto tester is home-grown. We can adjust it if needed to test PRs in 
tandem.

In fact, I believe we used to use Daniel Murphy's branch to test mutual 
PRs this way (kind of ad-hoc, but it worked, and was not frequently needed).

I'd MUCH rather not merge these two repos. Note that this sometimes even 
happens with Phobos too!

If needed, I'd rather there be an interface on the auto-tester to tag 
which mutual PR is needed from any of the other repos.

-Steve
Jul 27 2018
prev sibling next sibling parent Anton Fediushin <fediushin.anton yandex.com> writes:
On Friday, 27 July 2018 at 11:03:50 UTC, Seb wrote:

 This a thread to explore whether it would be feasible to do so.


[...]

 What do you think?
 ------------------

 - Has the dmd/druntime split being annoying you too?


I'm not familiar with contributing to the dmd/druntime, but 
having druntime in a separate repository never made sense to me. 
If it makes it easier for people to work on dmd/druntime, I'm all 
up for that.


 - Do you have a better suggestion?


Not at all.


 - Would this break your workflow in a drastic way?


Nope, in my workflow I either use dmd packages from pacman/apt or 
install.sh script. If these get updated on time I won't even 
notice that change.
Jul 29 2018
prev sibling next sibling parent RazvanN <razvan.nitu1305 gmail.com> writes:
On Friday, 27 July 2018 at 11:03:50 UTC, Seb wrote:

 This a thread to explore whether it would be feasible to do so.


 From my experience, whenever a change in dmd required a change in 
druntime the protocol was pretty straightforward and not at all 
annoying:

-> you make the dmd PR and it causes druntime breakage
-> you make the druntime PR to fix the problem

Problem solved. The only overhead here is :

git checkout -b druntime_fix
git push -f origin druntime_fix

Having druntime and dmd as separate repos makes sense because the 
two are logically separated. If druntime and dmd are merged 
together based on your motivation, there's only a matter of time 
until someone, following the same logic as you did, will propose 
to also merge phobos with dmd + druntime since modifying the 
latter often requires changing the former.

The only situation where your proposal has any benefit is when a 
circular dependency arises: you first need the druntime patch to 
be able to merge the dmd one and at the same time you first need 
the dmd patch to merge the druntime one. I don't even know if 
such a situation is possible, but even so, in this case we can 
just bite the bullet and merge them both at the (pseudo)same time.

RazvanN
Jul 30 2018
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2018-07-27 13:03, Seb wrote:

 This a thread to explore whether it would be feasible to do so.

 Motivation
 ----------

 DRuntime and DMD heavily depend on each other.

 It happens very often that a PR needs to touch both and then a
 complicated three-step (or sometimes four-step PR series) needs to be
 done to keep the CIs happy.
 Moreover, as the whole testsuite is at dmd, it happens more often that
 an error is caught in the testsuite, but would require a druntime PR or
 the opposite you make a PR to druntime and want its accompanying test to
 be checked by the CI.


Another approach could be to separate the compiler dependent API, i.e. 
"rt" and the user facing API, i.e. "core". The "rt" API could be moved 
to the compiler repository while the user facing API could stay.

-- 
/Jacob Carlborg
Jul 31 2018
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/27/2018 4:03 AM, Seb wrote:

 This a thread to explore whether it would be feasible to do so.


Since DMD and Druntime require each other, it is the right thing to do.
Jul 31 2018
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/31/18 2:24 PM, Walter Bright wrote:

 On 7/27/2018 4:03 AM, Seb wrote:

 This a thread to explore whether it would be feasible to do so.


 
 Since DMD and Druntime require each other, it is the right thing to do.


I think this is an incorrect relationship.

DMD does NOT require Druntime, *testing* DMD requires Druntime.

In fact, I thought the whole point of the -betterC feature was to 
eliminate any dependency on druntime.

As we move to more and more library-provided hooks and away from 
"compiler magic", this coupling will become less and less prevalent.

In fact, most of the changes that require both projects to be 
simultaneously to be updated are these magic-removing ones.

If anything makes sense, it would be to remove the compiler-dependencies 
out of druntime into a smaller runtime library that is included in the 
dmd project. Most of druntime is dependencies for the platform, not the 
compiler.

-Steve
Jul 31 2018
next sibling parent reply Seb <seb wilzba.ch> writes:
On Tuesday, 31 July 2018 at 19:54:20 UTC, Steven Schveighoffer 
wrote:

 On 7/31/18 2:24 PM, Walter Bright wrote:

 Since DMD and Druntime require each other, it is the right 
 thing to do.


 I think this is an incorrect relationship.

 DMD does NOT require Druntime, *testing* DMD requires Druntime.


Well DMD ships just happens to ship with Druntime ;-)


 In fact, I thought the whole point of the -betterC feature was 
 to eliminate any dependency on druntime.


That's only partially true. Even with -betterC, `object.d` will 
still be imported by default and actually many language features 
that work in -betterC will be lowered to druntime (even in 
betterC):

One simple example: https://run.dlang.io/is/41vvoO


 As we move to more and more library-provided hooks and away 
 from "compiler magic", this coupling will become less and less 
 prevalent.


On the contrary, DMD will depend more and more on druntime as the 
magic is now in druntime and not in the dmd source code.


 In fact, most of the changes that require both projects to be 
 simultaneously to be updated are these magic-removing ones.


Yes and no. Whenever you want to update a library hooks (which 
are implementation-defined and not part of the specification), 
it's rather complicated to do with keeping all CIs happy.


 If anything makes sense, it would be to remove the 
 compiler-dependencies out of druntime into a smaller runtime 
 library that is included in the dmd project. Most of druntime 
 is dependencies for the platform, not the compiler.


Good idea!
Jul 31 2018
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/31/18 4:48 PM, Seb wrote:

 On Tuesday, 31 July 2018 at 19:54:20 UTC, Steven Schveighoffer wrote:

 On 7/31/18 2:24 PM, Walter Bright wrote:

 Since DMD and Druntime require each other, it is the right thing to do.


 I think this is an incorrect relationship.

 DMD does NOT require Druntime, *testing* DMD requires Druntime.


 
 Well DMD ships just happens to ship with Druntime ;-)


What I mean is that one can build and use DMD without druntime (see Mike 
Franklin's project from 2013). The test suite requires it because we are 
testing the interactions between the DMD binary and the runtime that it 
uses. This whole thread is really about testing.


 In fact, I thought the whole point of the -betterC feature was to 
 eliminate any dependency on druntime.


 
 That's only partially true. Even with -betterC, `object.d` will still be 
 imported by default and actually many language features that work in 
 -betterC will be lowered to druntime (even in betterC):
 
 One simple example: https://run.dlang.io/is/41vvoO


True, you can probably say that most of object.d belongs to the 
compiler. I wouldn't have any problem moving that.


 As we move to more and more library-provided hooks and away from 
 "compiler magic", this coupling will become less and less prevalent.


 
 On the contrary, DMD will depend more and more on druntime as the magic 
 is now in druntime and not in the dmd source code.


Not really. There is a confusion here about what is a dependency, and 
what is the interface. The agreed-upon interface between compiler and 
runtime is really the only thing that requires careful updates and 
breaks the CI. As long as the rest is independent, it can be done on one 
side or the other.

What I meant by the statement above is, the runtime is going to be 
deciding more of what actually happens instead of the compiler. When 
these decisions are moved out of the compiler and into the runtime (in 
the form of templates), the compiler becomes more streamlined and less 
crowded with "code generating code". The hooks become simpler and 
naturally more stable, because the place where you "play with" the 
functionality is in the library. This means you need to change those 
hooks less often, and breaking projects becomes less common.

A glaring example is the AA library. If the compiler simply lowered any 
Value[Key] to AssociativeArray!(Key, Value), and let the runtime define 
*everything*, when would you have to change the compiler again after 
that? But of course, until then, the compiler and AA runtime have to be 
altered in lock-step.


 In fact, most of the changes that require both projects to be 
 simultaneously to be updated are these magic-removing ones.


 
 Yes and no. Whenever you want to update a library hooks (which are 
 implementation-defined and not part of the specification), it's rather 
 complicated to do with keeping all CIs happy.


Yes, exactly my point! The removal of magic means removal of the 
complicated hooks.


 

 If anything makes sense, it would be to remove the 
 compiler-dependencies out of druntime into a smaller runtime library 
 that is included in the dmd project. Most of druntime is dependencies 
 for the platform, not the compiler.


 
 Good idea!


I think object.d can probably go to the compiler, but some stuff in 
there should probably be moved into a druntime-specific module. I'd have 
to look more deeply at the rt package to see what makes sense, but 
everything else should stay in druntime (core, gc).

-Steve
Jul 31 2018
prev sibling next sibling parent reply Seb <seb wilzba.ch> writes:
On Tuesday, 31 July 2018 at 19:54:20 UTC, Steven Schveighoffer 
wrote:

 If anything makes sense, it would be to remove the 
 compiler-dependencies out of druntime into a smaller runtime 
 library that is included in the dmd project. Most of druntime 
 is dependencies for the platform, not the compiler.


Moving object.d to the dmd repository would be rather easy:

https://github.com/dlang/dmd/pull/8529
https://github.com/dlang/druntime/pull/2262

and already help a lot in decoupling both. Maybe that's a 
compromise that can be made?
Jul 31 2018
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 01/08/2018 12:07 PM, Seb wrote:

 On Tuesday, 31 July 2018 at 19:54:20 UTC, Steven Schveighoffer wrote:

 If anything makes sense, it would be to remove the 
 compiler-dependencies out of druntime into a smaller runtime library 
 that is included in the dmd project. Most of druntime is dependencies 
 for the platform, not the compiler.


 
 Moving object.d to the dmd repository would be rather easy:
 
 https://github.com/dlang/dmd/pull/8529
 https://github.com/dlang/druntime/pull/2262
 
 and already help a lot in decoupling both. Maybe that's a compromise 
 that can be made?


I think we're on to a winner now.

Just a matter now of figuring out if we should have an object.d module 
at all, and if we can split stuff out from it e.g. TypeInfo. There is of 
course other things like rt.* that could go with it.
Jul 31 2018
prev sibling parent Mike Franklin <slavo5150 yahoo.com> writes:
On Tuesday, 31 July 2018 at 19:54:20 UTC, Steven Schveighoffer 
wrote:


 If anything makes sense, it would be to remove the 
 compiler-dependencies out of druntime into a smaller runtime 
 library that is included in the dmd project. Most of druntime 
 is dependencies for the platform, not the compiler.


I think this is right-on.  I don't see the solution yet, but I 
don't think it's just a matter of moving object.d.

Mike
Jul 31 2018
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/31/2018 11:24 AM, Walter Bright wrote:

 On 7/27/2018 4:03 AM, Seb wrote:

 This a thread to explore whether it would be feasible to do so.


 
 Since DMD and Druntime require each other, it is the right thing to do.


Perhaps my judgement is premature. There are a lot of good points raised in
this 
thread.
Jul 31 2018
prev sibling parent Radu <rad.racariu gmail.com> writes:
On Friday, 27 July 2018 at 11:03:50 UTC, Seb wrote:

 This a thread to explore whether it would be feasible to do so.

 Motivation
 ----------

 [...]


Druntime is not exclusively DMD. For example when porting to 
other platforms or cruntimes is already hard because of the sync 
and merge you need to do when you move your changes upstream. 
Most of the time those changes have nothing to do with dmd.

Having dmd source internwined with druntime will make things even 
harder.

Better find a solution on the CI server side rather then merge 
dmd with druntime.
Jul 31 2018