www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - {dmd, druntime} -> {dmd+druntime} i.e. Monorepo?

reply max haughton <maxhaton gmail.com> writes:
Thesis: Develop dmd and druntime together in a monorepo, since 
they are developed and shipped together. I think this could 
potentially save a lot of hassle for both developers and 
consumers, especially the former: No more waiting for druntime 
PRs to go through, and hopefully less CI-senbugs.

Antithesis?
Aug 22 2021
next sibling parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Monday, 23 August 2021 at 01:01:34 UTC, max haughton wrote:

 Thesis: Develop dmd and druntime together in a monorepo, since 
 they are developed and shipped together. I think this could 
 potentially save a lot of hassle for both developers and 
 consumers, especially the former: No more waiting for druntime 
 PRs to go through, and hopefully less CI-senbugs.

 Antithesis?


Are there any drawbacks of doing it?
Aug 23 2021
prev sibling next sibling parent reply IGotD- <nise nise.com> writes:
On Monday, 23 August 2021 at 01:01:34 UTC, max haughton wrote:

 Thesis: Develop dmd and druntime together in a monorepo, since 
 they are developed and shipped together. I think this could 
 potentially save a lot of hassle for both developers and 
 consumers, especially the former: No more waiting for druntime 
 PRs to go through, and hopefully less CI-senbugs.

 Antithesis?


Yes, if you for example want to develop a special druntime, then 
it is better to have dmd and druntime separated as you don't need 
the extra dmd baggage.

A similar argument can be done to merge phobos and druntime 
because they are somewhat connected as well. I'm not suggesting 
this however.

If you look at LDC, they have solved it by adding druntime and 
phobos as git submodules. This is a good idea as you get the 
exact snapshots of each version and don't have to manually set 
the version of each separate git. Why haven't the dmd repos done 
the same?
Aug 23 2021
next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Monday, 23 August 2021 at 07:51:52 UTC, IGotD- wrote:

 Yes, if you for example want to develop a special druntime, 
 then it is better to have dmd and druntime separated as you 
 don't need the extra dmd baggage.

 A similar argument can be done to merge phobos and druntime 
 because they are somewhat connected as well. I'm not suggesting 
 this however.


These two are totally dissimilar cases.

Everything in `core` is part of the language. Any D compiler is 
allowed to specially recognize functions in the `core` API. And 
vice-versa, any D compiler is allowed to translate language 
constructs to DRuntime calls. The latter is done in DMD for 
`assert` failures and `new`, for example. It follows that what 
DRuntime is fundamentally dependent on the compiler 
implementation, and there is a good case for having a same 
repository for both of them.

Conversely the `std` is just a library that obeys normal language 
rules. It's implementation should try to avoid compiler 
implementation dependency where possible, and no D compiler 
should handle anything in `std` namespace as special. Ideally, 
you could compile multiple different versions of Phobos with any 
D compiler. I don't think Phobos is nearly that portable in 
practice but that's the ideal. It follows that it should always 
be a separate repository from the compiler and/or DRuntime.
Aug 23 2021
parent Denis Feklushkin <denis.feklushin gmail.com> writes:
On Monday, 23 August 2021 at 13:04:37 UTC, Dukc wrote:


 Conversely the `std` is just a library that obeys normal 
 language rules. It's implementation should try to avoid 
 compiler implementation dependency where possible, and no D 
 compiler should handle anything in `std` namespace as special. 
 Ideally, you could compile multiple different versions of 
 Phobos with any D compiler. I don't think Phobos is nearly that 
 portable in practice but that's the ideal. It follows that it 
 should always be a separate repository from the compiler and/or 
 DRuntime.


In fact, Phobos full of platform-dependent code same as Druntime.
Rather, the topic should have been called "(druntime + phobos) 
monorepo"
Aug 24 2021
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Monday, 23 August 2021 at 07:51:52 UTC, IGotD- wrote:

 On Monday, 23 August 2021 at 01:01:34 UTC, max haughton wrote:

 Thesis: Develop dmd and druntime together in a monorepo, since 
 they are developed and shipped together. I think this could 
 potentially save a lot of hassle for both developers and 
 consumers, especially the former: No more waiting for druntime 
 PRs to go through, and hopefully less CI-senbugs.

 Antithesis?


 Yes, if you for example want to develop a special druntime, 
 then it is better to have dmd and druntime separated as you 
 don't need the extra dmd baggage.

 A similar argument can be done to merge phobos and druntime 
 because they are somewhat connected as well. I'm not suggesting 
 this however.

 If you look at LDC, they have solved it by adding druntime and 
 phobos as git submodules. This is a good idea as you get the 
 exact snapshots of each version and don't have to manually set 
 the version of each separate git. Why haven't the dmd repos 
 done the same?


git filter-branch --prune-empty --subdirectory-filter "druntime/"

And voila. the result could even be pushed int he existing 
druntime repo on a regular basis by a cronjob.

This is a non problem.
Aug 24 2021
prev sibling next sibling parent deadalnix <deadalnix gmail.com> writes:
On Monday, 23 August 2021 at 01:01:34 UTC, max haughton wrote:

 Thesis: Develop dmd and druntime together in a monorepo, since 
 they are developed and shipped together. I think this could 
 potentially save a lot of hassle for both developers and 
 consumers, especially the former: No more waiting for druntime 
 PRs to go through, and hopefully less CI-senbugs.

 Antithesis?


Yes please.
Aug 24 2021
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 8/22/21 9:01 PM, max haughton wrote:

 Thesis: Develop dmd and druntime together in a monorepo, since they are 
 developed and shipped together. I think this could potentially save a 
 lot of hassle for both developers and consumers, especially the former: 
 No more waiting for druntime PRs to go through, and hopefully less 
 CI-senbugs.
 
 Antithesis?


druntime is comprised of several pieces:

1. Compiler support (GC, [associative] array runtime, TypeInfo, etc.)
2. Runtime startup (running the module constructors/destructors, 
handling exceptions not caught by `main`, etc.)
3. Library code that is needed for items 1 and 2. For example, core.time 
isn't really "runtime", but is needed by the runtime for many parts.

It's hard to split any of these out. I don't know that core.time should 
be part of the compiler. Or core.thread. Really, only the 
compiler-linked parts really belong in the compiler repository. For 
instance the functions that implement string switching. I listed the GC 
in that section because the compiler has hooks into the GC, but I don't 
know if I'd count the GC as part of the compiler.

More stuff gets shoved into druntime because we don't want to implement 
it twice (once in druntime, once in phobos), and as the features of the 
compiler become more implementable in the library, we can move things 
there to simplify and streamline the compiler.

I think a great project would be to do an analysis of all the 
dependencies between the compiler and the runtime (and phobos!), and 
then discuss how we would design it today if we had the chance, knowing 
what we know now.

As far as merging the two repositories, it's more of the same shoving of 
stuff where it doesn't belong because something is difficult (i.e. 
co-dependent PRs) or inconvenient. There's alternatives as well, we 
could use some sort of tagging to identify co-dependent PRs to make sure 
they are tested together.

-Steve
Aug 24 2021
parent max haughton <maxhaton gmail.com> writes:
On Tuesday, 24 August 2021 at 13:18:11 UTC, Steven Schveighoffer 
wrote:

 On 8/22/21 9:01 PM, max haughton wrote:

 [...]


 druntime is comprised of several pieces:

 1. Compiler support (GC, [associative] array runtime, TypeInfo, 
 etc.)
 2. Runtime startup (running the module 
 constructors/destructors, handling exceptions not caught by 
 `main`, etc.)
 3. Library code that is needed for items 1 and 2. For example, 
 core.time isn't really "runtime", but is needed by the runtime 
 for many parts.

 It's hard to split any of these out. I don't know that 
 core.time should be part of the compiler. Or core.thread. 
 Really, only the compiler-linked parts really belong in the 
 compiler repository. For instance the functions that implement 
 string switching. I listed the GC in that section because the 
 compiler has hooks into the GC, but I don't know if I'd count 
 the GC as part of the compiler.

 More stuff gets shoved into druntime because we don't want to 
 implement it twice (once in druntime, once in phobos), and as 
 the features of the compiler become more implementable in the 
 library, we can move things there to simplify and streamline 
 the compiler.

 I think a great project would be to do an analysis of all the 
 dependencies between the compiler and the runtime (and 
 phobos!), and then discuss how we would design it today if we 
 had the chance, knowing what we know now.

 As far as merging the two repositories, it's more of the same 
 shoving of stuff where it doesn't belong because something is 
 difficult (i.e. co-dependent PRs) or inconvenient. There's 
 alternatives as well, we could use some sort of tagging to 
 identify co-dependent PRs to make sure they are tested together.

 -Steve


The actual topology of the codebases would stay the same (or at 
least changes to it don't matter for the purposes of what I am 
suggesting), it's just the simplest way I can think of to 
streamline our current process. Semantically they are tied 
together very strongly and the same is true of our development 
process.

I also don't think our CI is particularly reliable and I think 
this would help.
Aug 24 2021