www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Monorepo?

reply Atila Neves <atila.neves gmail.com> writes:
Now that dmd/druntime is in one repo, is there a good reason to 
not include Phobos?
Feb 06 2023
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Monday, 6 February 2023 at 10:22:32 UTC, Atila Neves wrote:
 Now that dmd/druntime is in one repo, is there a good reason to 
 not include Phobos?
No. Do it.
Feb 06 2023
prev sibling next sibling parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Monday, 6 February 2023 at 10:22:32 UTC, Atila Neves wrote:
 Now that dmd/druntime is in one repo, is there a good reason to 
 not include Phobos?
Better to have Phobos as an external source library though compiled by dub/reggae/rdmd/whatever. https://code.dlang.org/packages/phobos
Feb 06 2023
parent reply max haughton <maxhaton gmail.com> writes:
On Monday, 6 February 2023 at 14:26:45 UTC, Iain Buclaw wrote:
 On Monday, 6 February 2023 at 10:22:32 UTC, Atila Neves wrote:
 Now that dmd/druntime is in one repo, is there a good reason 
 to not include Phobos?
Better to have Phobos as an external source library though compiled by dub/reggae/rdmd/whatever. https://code.dlang.org/packages/phobos
Things choosing and thus eventually disagreeing on which Phobos version they want to use is potentially not so great.
Feb 06 2023
parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Monday, 6 February 2023 at 14:28:45 UTC, max haughton wrote:
 Things choosing and thus eventually disagreeing on which Phobos 
 version they want to use is potentially not so great.
Phobos being pre-compiled with an unknown set of flags, and that mismatching every other project reliant on Phobos is not great either.
Feb 06 2023
parent reply deadalnix <deadalnix gmail.com> writes:
On Monday, 6 February 2023 at 15:04:24 UTC, Iain Buclaw wrote:
 On Monday, 6 February 2023 at 14:28:45 UTC, max haughton wrote:
 Things choosing and thus eventually disagreeing on which 
 Phobos version they want to use is potentially not so great.
Phobos being pre-compiled with an unknown set of flags, and that mismatching every other project reliant on Phobos is not great either.
This is generally not a problem with C++, why should we expect this to be one for D? If we introduced flags that bifurcate the language, then it seems to me that these flags should be removed.
Feb 06 2023
next sibling parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Monday, 6 February 2023 at 22:58:02 UTC, deadalnix wrote:
 On Monday, 6 February 2023 at 15:04:24 UTC, Iain Buclaw wrote:
 On Monday, 6 February 2023 at 14:28:45 UTC, max haughton wrote:
 Things choosing and thus eventually disagreeing on which 
 Phobos version they want to use is potentially not so great.
Phobos being pre-compiled with an unknown set of flags, and that mismatching every other project reliant on Phobos is not great either.
This is generally not a problem with C++, why should we expect this to be one for D?
Every kind of code generation flag can and will affect the signature of a function, whether a template gets codegen.
 If we introduced flags that bifurcate the language, then it 
 seems to me that these flags should be removed.
Goodbye -release, goodbye -dip1000, goodbye -checkaction=, goodbye -preview=, goodbye -debug=, goodbye -unittest, goodbye -version=, goodbye -allinst...
Feb 06 2023
next sibling parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Monday, 6 February 2023 at 23:42:01 UTC, Iain Buclaw wrote:
 Goodbye -release, goodbye -dip1000, goodbye -checkaction=, 
 goodbye -preview=, goodbye -debug=, goodbye -unittest, goodbye 
 -version=, goodbye -allinst...
i have a potential solution http://dpldocs.info/this-week-in-d/Blog.Posted_2022_11_14.html#redesign-for-template-emission-woes
Feb 06 2023
parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On Monday, 6 February 2023 at 23:49:54 UTC, Adam D Ruppe wrote:
 On Monday, 6 February 2023 at 23:42:01 UTC, Iain Buclaw wrote:
 Goodbye -release, goodbye -dip1000, goodbye -checkaction=, 
 goodbye -preview=, goodbye -debug=, goodbye -unittest, goodbye 
 -version=, goodbye -allinst...
i have a potential solution http://dpldocs.info/this-week-in-d/Blog.Posted_2022_11_14.html#redesign-for-template-emission-woes
Agreed that none of them are strictly bugs once you explain the rationale. There are historically two ways to go about it. 1. Reference a template repository during building of each module, package, library, project, ...; whichever way your build is broken down into. Add templates to the repo as you progress through building your project. 2. Go gung-ho with emissions, relying on "used" flags to determine whether any calls/address references to the symbol actually remain in the graph (after inlining), and prune it all away before writing to assembly/object file.
Feb 06 2023
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On Monday, 6 February 2023 at 23:42:01 UTC, Iain Buclaw wrote:
 On Monday, 6 February 2023 at 22:58:02 UTC, deadalnix wrote:
 If we introduced flags that bifurcate the language, then it 
 seems to me that these flags should be removed.
Goodbye -release, goodbye -dip1000, goodbye -checkaction=, goodbye -preview=, goodbye -debug=, goodbye -unittest, goodbye -version=, goodbye -allinst...
Yes, there are bug reports covering pretty much all of these, for example. https://issues.dlang.org/show_bug.cgi?id=20802 https://issues.dlang.org/show_bug.cgi?id=22700 https://issues.dlang.org/show_bug.cgi?id=22902 https://issues.dlang.org/show_bug.cgi?id=23255
Feb 06 2023
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Monday, 6 February 2023 at 23:42:01 UTC, Iain Buclaw wrote:
 If we introduced flags that bifurcate the language, then it 
 seems to me that these flags should be removed.
Goodbye -release, goodbye -dip1000, goodbye -checkaction=, goodbye -preview=, goodbye -debug=, goodbye -unittest, goodbye -version=, goodbye -allinst...
Where do I sign?
Feb 06 2023
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 6 February 2023 at 22:58:02 UTC, deadalnix wrote:
 On Monday, 6 February 2023 at 15:04:24 UTC, Iain Buclaw wrote:
 On Monday, 6 February 2023 at 14:28:45 UTC, max haughton wrote:
 Things choosing and thus eventually disagreeing on which 
 Phobos version they want to use is potentially not so great.
Phobos being pre-compiled with an unknown set of flags, and that mismatching every other project reliant on Phobos is not great either.
This is generally not a problem with C++, why should we expect this to be one for D? If we introduced flags that bifurcate the language, then it seems to me that these flags should be removed.
It's curious now that you mention it that C++ doesn't have this problem, seeing as how they also have a lot of compiler flags. Off the top of my head I can't figure out why.
Feb 07 2023
parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 7 February 2023 at 10:33:58 UTC, Atila Neves wrote:
 If we introduced flags that bifurcate the language, then it 
 seems to me that these flags should be removed.
It's curious now that you mention it that C++ doesn't have this problem, seeing as how they also have a lot of compiler flags. Off the top of my head I can't figure out why.
The flags don't tend to create a new, incompatible, language. What is affected by the flag of the consumer is provided as templates and/or macros.
Feb 07 2023
parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On Tuesday, 7 February 2023 at 14:37:35 UTC, deadalnix wrote:
 On Tuesday, 7 February 2023 at 10:33:58 UTC, Atila Neves wrote:
 If we introduced flags that bifurcate the language, then it 
 seems to me that these flags should be removed.
It's curious now that you mention it that C++ doesn't have this problem, seeing as how they also have a lot of compiler flags. Off the top of my head I can't figure out why.
The flags don't tend to create a new, incompatible, language.
Erm... yes they do (C/C++ flags).
Feb 07 2023
prev sibling next sibling parent reply Mathias LANG <geod24 gmail.com> writes:
On Monday, 6 February 2023 at 10:22:32 UTC, Atila Neves wrote:
 Now that dmd/druntime is in one repo, is there a good reason to 
 not include Phobos?
Because it will inevitably lead to circular dependencies, something Walter is already complaining about. There was an advantage in merging DMD and druntime: They are tightly coupled, and we often enough situations where you had to merge a PR in one repo to see if things were working in the other. We don't have that requirement in Phobos.
Feb 06 2023
next sibling parent reply "H. S. Teoh" <hsteoh qfbox.info> writes:
On Mon, Feb 06, 2023 at 04:48:51PM +0000, Mathias LANG via Digitalmars-d wrote:
 On Monday, 6 February 2023 at 10:22:32 UTC, Atila Neves wrote:
 Now that dmd/druntime is in one repo, is there a good reason to not
 include Phobos?
Because it will inevitably lead to circular dependencies, something Walter is already complaining about. There was an advantage in merging DMD and druntime: They are tightly coupled, and we often enough situations where you had to merge a PR in one repo to see if things were working in the other. We don't have that requirement in Phobos.
OTOH there have been a good number of occasions in the past where changes in DMD/druntime had to be coordinated with Phobos. Even today, I'm almost certain that you can only use a specific version of Phobos with a specific version of DMD; it's likely to fail compilation or behave incorrectly otherwise. T -- The fact that anyone still uses AOL shows that even the presence of options doesn't stop some people from picking the pessimal one. - Mike Ellis
Feb 06 2023
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/6/23 11:57 AM, H. S. Teoh wrote:

 OTOH there have been a good number of occasions in the past where
 changes in DMD/druntime had to be coordinated with Phobos. Even today,
 I'm almost certain that you can only use a specific version of Phobos
 with a specific version of DMD; it's likely to fail compilation or
 behave incorrectly otherwise.
That's a one-way dependency, and normal. DMD/druntime do not depend on phobos (any more, I believe the only issues were with intrinsics, which now are all in druntime) FWIW, I would vote for a phobos dependency in code.dlang.org, but it needs to be versioned properly (not easy). I doubt we have the manpower to make sure that is the case. -Steve
Feb 06 2023
prev sibling parent Atila Neves <atila.neves gmail.com> writes:
On Monday, 6 February 2023 at 16:57:13 UTC, H. S. Teoh wrote:
 On Mon, Feb 06, 2023 at 04:48:51PM +0000, Mathias LANG via 
 Digitalmars-d wrote:
 [...]
OTOH there have been a good number of occasions in the past where changes in DMD/druntime had to be coordinated with Phobos. Even today, I'm almost certain that you can only use a specific version of Phobos with a specific version of DMD; it's likely to fail compilation or behave incorrectly otherwise. T
That's happened more times than I can count to me, so I'm surprised at the comment that we don't have that requirement for Phobos.
Feb 06 2023
prev sibling next sibling parent Adam D Ruppe <destructionator gmail.com> writes:
On Monday, 6 February 2023 at 16:48:51 UTC, Mathias LANG wrote:
 There was an advantage in merging DMD and druntime: They are 
 tightly coupled, and we often enough situations where you had 
 to merge a PR in one repo to see if things were working in the 
 other. We don't have that requirement in Phobos.
In theory, Phobos is supposed to be decoupled, but in practice, it isn't. That said, I'd rather fix the practice than the theory here. It *should* work better than it does.
Feb 06 2023
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/6/2023 8:48 AM, Mathias LANG wrote:
 On Monday, 6 February 2023 at 10:22:32 UTC, Atila Neves wrote:
 Now that dmd/druntime is in one repo, is there a good reason to not include 
 Phobos?
Because it will inevitably lead to circular dependencies, something Walter is already complaining about.
More specifically, I complained about the order in which things were built and tested. There is a test suite designed to test the compiler. There are many hundreds of these tests. Each is standalone, only a few lines long. I.e. they are already reduced and isolated. (The source of most of these were bugs that were fixed.) When something in that test suite fails, one can usually go directly to the problem. But what happens with the way the test suite is run is: 1. the compiler gets built 2. then the compiler is used to compile druntime. Boom, the compiler crashes. Now you've got hours of work to reduce the problem. 3. or the compiler is used to build checkwhitespace. Boom, the compiler crashes, or checkwhitespace crashes. Hours of work ahead. 4. or the compiler is used to bootstrap itself. Boom, it creates a compiler that crashes, or the created compiler compiles code badly. Hours of work ahead. 5. or the compiler is used to build build.d. build itself then crashes. Hours of work ahead. 6. or the compiler is used to build Phobos. It crashes building Phobos, or the Phobos unittests fail. Hours of work ahead. --- OR --- the freshly built compiler is used to run the compiler test suite. test1234.d fails. A few minutes of work identifying what went wrong, because you've only got 6 lines of code to look at rather than 100,000 lines. 2,3,4,5,6 have all routinely happened. At least (mostly) the compiler test suite does not depend on a functioning Phobos. Myself (and others) have removed nearly all of those dependencies. Another advantage of running the compiler test suite first is it is relatively small and runs fast. Hence, you get quick turnaround.
Feb 06 2023
next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 6 February 2023 at 18:36:56 UTC, Walter Bright wrote:
 There is a test suite designed to test the compiler. There are 
 many hundreds of these tests. Each is standalone, only a few 
 lines long. I.e. they are already reduced and isolated. (The 
 source of most of these were bugs that were fixed.) When 
 something in that test suite fails, one can usually go directly 
 to the problem.
I wish this were true, but actually there are a bunch of tests in the compiler's test suite that rely on Phobos. I know this because every now and then I will attempt to run the test suite without compiling Phobos first, and it will flood my terminal with error messages. Perhaps some housekeeping is in order.
Feb 06 2023
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/6/2023 11:36 AM, Paul Backus wrote:
 I wish this were true, but actually there are a bunch of tests in the
compiler's 
 test suite that rely on Phobos. I know this because every now and then I will 
 attempt to run the test suite without compiling Phobos first, and it will
flood 
 my terminal with error messages.
I removed some. If there are any left, name and shame!
Feb 06 2023
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/6/23 1:36 PM, Walter Bright wrote:

 More specifically, I complained about the order in which things were 
 built and tested.
 
 There is a test suite designed to test the compiler. There are many 
 hundreds of these tests. Each is standalone, only a few lines long. I.e. 
 they are already reduced and isolated. (The source of most of these were 
 bugs that were fixed.) When something in that test suite fails, one can 
 usually go directly to the problem.
 
 But what happens with the way the test suite is run is:
 
 1. the compiler gets built
 
 2. then the compiler is used to compile druntime. Boom, the compiler 
 crashes. Now you've got hours of work to reduce the problem.
This is probably unavoidable. druntime and the compiler are tightly coupled. Potentially, you could run some tests before this? Off the top of my head, you could run fail-compilation tests before a valid druntime is available?
 3. or the compiler is used to build checkwhitespace. Boom, the compiler 
 crashes, or checkwhitespace crashes. Hours of work ahead.
We should build checkwhitespace with the bootstrap compiler, not with the built compiler. You can rebuild checkwhitespace at the end as a further test. In any case, changing the compiler shouldn't break this part of the step.
 4. or the compiler is used to bootstrap itself. Boom, it creates a 
 compiler that crashes, or the created compiler compiles code badly. 
 Hours of work ahead.
This should be done after the test suite, right?
 5. or the compiler is used to build build.d. build itself then crashes. 
 Hours of work ahead.
Same here.
 6. or the compiler is used to build Phobos. It crashes building Phobos, 
 or the Phobos unittests fail. Hours of work ahead.
Yeah, we need to make sure phobos isn't needed to run the full test suite. possibly we could delay phobos-dependent tests until after the standalone tests finish. Sometimes compiler bugs are only tickled with insanely convoluted conditions, which might include specific parts of phobos. Reducing them is ideal, but that takes hours too.
 the freshly built compiler is used to run the compiler test suite. 
 test1234.d fails. A few minutes of work identifying what went wrong, 
 because you've only got 6 lines of code to look at rather than 100,000 
 lines.
 
 2,3,4,5,6 have all routinely happened.
 
 At least (mostly) the compiler test suite does not depend on a 
 functioning Phobos. Myself (and others) have removed nearly all of those 
 dependencies.
 
 Another advantage of running the compiler test suite first is it is 
 relatively small and runs fast. Hence, you get quick turnaround.
 
All agreed. We should have a standing rule that any tests that are run based on code external to dmd/druntime need to happen *after* all the dmd/druntime tests are done. And tools should be built with a known good compiler. The "checkwhitespace" problem seems an incredibly foolish requirement to shove in front of making sure you have a sane compiler. -Steve
Feb 06 2023
parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Monday, 6 February 2023 at 20:21:35 UTC, Steven Schveighoffer 
wrote:
 All agreed. We should have a standing rule that any tests that 
 are run based on code external to dmd/druntime need to happen 
 *after* all the dmd/druntime tests are done. And tools should 
 be built with a known good compiler. The "checkwhitespace" 
 problem seems an incredibly foolish requirement to shove in 
 front of making sure you have a sane compiler.
You're not _really_ going to believe that checkwhitespace is built with the compiler under test, are you? ;-)
Feb 06 2023
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/6/23 6:35 PM, Iain Buclaw wrote:
 On Monday, 6 February 2023 at 20:21:35 UTC, Steven Schveighoffer wrote:
 All agreed. We should have a standing rule that any tests that are run 
 based on code external to dmd/druntime need to happen *after* all the 
 dmd/druntime tests are done. And tools should be built with a known 
 good compiler. The "checkwhitespace" problem seems an incredibly 
 foolish requirement to shove in front of making sure you have a sane 
 compiler.
You're not _really_ going to believe that checkwhitespace is built with the compiler under test, are you? ;-)
I haven't looked. Just going by what Walter said. If it is, it shouldn't be. -Steve
Feb 06 2023
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Monday, 6 February 2023 at 16:48:51 UTC, Mathias LANG wrote:
 On Monday, 6 February 2023 at 10:22:32 UTC, Atila Neves wrote:
 Now that dmd/druntime is in one repo, is there a good reason 
 to not include Phobos?
Because it will inevitably lead to circular dependencies, something Walter is already complaining about. There was an advantage in merging DMD and druntime: They are tightly coupled, and we often enough situations where you had to merge a PR in one repo to see if things were working in the other. We don't have that requirement in Phobos.
Just don't add things in the include path, and no dependencies will be created. If the build system doesn't allow for this, then change the build system.
Feb 06 2023
prev sibling next sibling parent reply user1234 <user1234 12.de> writes:
On Monday, 6 February 2023 at 10:22:32 UTC, Atila Neves wrote:
 Now that dmd/druntime is in one repo, is there a good reason to 
 not include Phobos?
This could lead to more breaking changes. Monorepo means that only BuildKite will be able to detect them, **unless** it is specified that it's not allowed to commit changes in phobos just because required by a change in dmd/druntime. The need for this rule says it all.
Feb 06 2023
parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 6 February 2023 at 17:09:36 UTC, user1234 wrote:
 On Monday, 6 February 2023 at 10:22:32 UTC, Atila Neves wrote:
 Now that dmd/druntime is in one repo, is there a good reason 
 to not include Phobos?
This could lead to more breaking changes. Monorepo means that only BuildKite will be able to detect them, **unless** it is specified that it's not allowed to commit changes in phobos just because required by a change in dmd/druntime. The need for this rule says it all.
I don't understand, could you please explain how there'd be more breaking changes?
Feb 06 2023
parent reply user1234 <user1234 12.de> writes:
On Monday, 6 February 2023 at 17:56:33 UTC, Atila Neves wrote:
 On Monday, 6 February 2023 at 17:09:36 UTC, user1234 wrote:
 On Monday, 6 February 2023 at 10:22:32 UTC, Atila Neves wrote:
 Now that dmd/druntime is in one repo, is there a good reason 
 to not include Phobos?
This could lead to more breaking changes. Monorepo means that only BuildKite will be able to detect them, **unless** it is specified that it's not allowed to commit changes in phobos just because required by a change in dmd/druntime. The need for this rule says it all.
I don't understand, could you please explain how there'd be more breaking changes?
I'll try to reword a bit: In case of an unexpected breaking change that would manifest itself only in the Phobos test suite and not in the BuildKite CI, and without "the rule", authors will be tempted to fix the problem in phobos in the same PR as the one initially only supposed to change DMD. As special rule is required, then why put phobos in the main D repo ? The fact that a special rule is required is a sign that a monorepo is a bad idea.
Feb 06 2023
next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 6 February 2023 at 18:15:00 UTC, user1234 wrote:
 On Monday, 6 February 2023 at 17:56:33 UTC, Atila Neves wrote:
 On Monday, 6 February 2023 at 17:09:36 UTC, user1234 wrote:
 On Monday, 6 February 2023 at 10:22:32 UTC, Atila Neves wrote:
 [...]
This could lead to more breaking changes. Monorepo means that only BuildKite will be able to detect them, **unless** it is specified that it's not allowed to commit changes in phobos just because required by a change in dmd/druntime. The need for this rule says it all.
I don't understand, could you please explain how there'd be more breaking changes?
I'll try to reword a bit: In case of an unexpected breaking change that would manifest itself only in the Phobos test suite and not in the BuildKite CI, and without "the rule", authors will be tempted to fix the problem in phobos in the same PR as the one initially only supposed to change DMD. As special rule is required, then why put phobos in the main D repo ? The fact that a special rule is required is a sign that a monorepo is a bad idea.
What special rule? If a dmd change breaks Phobos, doesn't it make more sense to fix Phobos in the same PR than submitting a separate PR to the Phobos repo (and having to explain that it's
Feb 07 2023
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 07/02/2023 11:15 PM, Atila Neves wrote:
 What special rule? If a dmd change breaks Phobos, doesn't it make more 
 sense to fix Phobos in the same PR than submitting a separate PR to the 

If a Phobos change breaks MIR, doesn't it make more sense to fix MIR in the same PR than submitting a separate PR to the MIR repo (and having to We can apply this same logic to literally any code base that is in the auto tester. The dmd repository is going to get quite big when we keep including what should be isolated code bases administratively... Personally I'm not sure that the zlib folks are going to want to move development into the dmd repository. But hey, you guys have pull so maybe you'll archive it!
Feb 07 2023
next sibling parent John Colvin <john.loughran.colvin gmail.com> writes:
On Wednesday, 8 February 2023 at 00:40:44 UTC, Richard (Rikki) 
Andrew Cattermole wrote:
 On 07/02/2023 11:15 PM, Atila Neves wrote:
 What special rule? If a dmd change breaks Phobos, doesn't it 
 make more sense to fix Phobos in the same PR than submitting a 
 separate PR to the Phobos repo (and having to explain that 

If a Phobos change breaks MIR, doesn't it make more sense to fix MIR in the same PR than submitting a separate PR to the MIR repo (and having to explain that its because of a Phobos PR We can apply this same logic to literally any code base that is in the auto tester. The dmd repository is going to get quite big when we keep including what should be isolated code bases administratively... Personally I'm not sure that the zlib folks are going to want to move development into the dmd repository. But hey, you guys have pull so maybe you'll archive it!
There are competing effects at play here. While of course merging everything in to dmd would maximise one thing at the expense of others, that doesn’t have much bearing on whether merging any individual thing is worth doing. There is no slippery slope to fall down, just the usual balancing of factors based on insight from experience and theory.
Feb 08 2023
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 8 February 2023 at 00:40:44 UTC, Richard (Rikki) 
Andrew Cattermole wrote:
 On 07/02/2023 11:15 PM, Atila Neves wrote:
 What special rule? If a dmd change breaks Phobos, doesn't it 
 make more sense to fix Phobos in the same PR than submitting a 
 separate PR to the Phobos repo (and having to explain that 

If a Phobos change breaks MIR, doesn't it make more sense to fix MIR in the same PR than submitting a separate PR to the MIR repo (and having to explain that its because of a Phobos PR We can apply this same logic to literally any code base that is in the auto tester. The dmd repository is going to get quite big when we keep including what should be isolated code bases administratively... Personally I'm not sure that the zlib folks are going to want to move development into the dmd repository. But hey, you guys have pull so maybe you'll archive it!
It's interesting that you have all the elements to answer this dilemma yet do not see it. The proliferation of repositories solve one problem, and one problem only: allow disparate organizations to collaborate without each other's processes, timeline, and whatnot disturbing the other too much. It creates an abstraction layer, which, while having a cost, remain beneficial because you'd have cross org friction instead, and it's much more costly. When the different part fall under the same organization, you start paying that abstraction cost for no reason at all. Case in point: dustmite exists. While I certainly command Vladimir for his work,
Feb 11 2023
parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 12 February 2023 at 00:15:51 UTC, deadalnix wrote:
 Case in point: dustmite exists. While I certainly command 
 Vladimir for his work,
I would rather he has spend his time and energy on something that provide value to the world rather than fixing limitation in the current toolchain.
Feb 11 2023
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 12 February 2023 at 00:18:35 UTC, deadalnix wrote:
 On Sunday, 12 February 2023 at 00:15:51 UTC, deadalnix wrote:
 Case in point: dustmite exists. While I certainly command 
 Vladimir for his work,
I would rather he has spend his time and energy on something that provide value to the world rather than fixing limitation in the current toolchain.
I think you mean 'Digger'. [https://github.com/CyberShadow/Digger]
Feb 11 2023
parent deadalnix <deadalnix gmail.com> writes:
On Sunday, 12 February 2023 at 00:49:04 UTC, Stefan Koch wrote:
 On Sunday, 12 February 2023 at 00:18:35 UTC, deadalnix wrote:
 On Sunday, 12 February 2023 at 00:15:51 UTC, deadalnix wrote:
 Case in point: dustmite exists. While I certainly command 
 Vladimir for his work,
I would rather he has spend his time and energy on something that provide value to the world rather than fixing limitation in the current toolchain.
I think you mean 'Digger'. [https://github.com/CyberShadow/Digger]
Yes, wow. This message got stuck due to a server outage, and I recovered it half written, and didn't do a great job at fixing it XD
Feb 12 2023
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 8 February 2023 at 00:40:44 UTC, Richard (Rikki) 
Andrew Cattermole wrote:
 On 07/02/2023 11:15 PM, Atila Neves wrote:
 What special rule? If a dmd change breaks Phobos, doesn't it 
 make more sense to fix Phobos in the same PR than submitting a 
 separate PR to the Phobos repo (and having to explain that 

If a Phobos change breaks MIR, doesn't it make more sense to fix MIR in the same PR than submitting a separate PR to the MIR repo (and having to explain that its because of a Phobos PR
I don't understand how this analogy applies unless you're advocating for a monorepo with all dub packages, which I doubt is the case.
 We can apply this same logic to literally any code base that is 
 in the auto tester. The dmd repository is going to get quite 
 big when we keep including what should be isolated code bases 
 administratively...
I don't think we can, because I'm asking about putting code that's under control by the same group of people in the same repository and I don't see how that applies to private individual projects.
Feb 13 2023
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 14/02/2023 12:58 AM, Atila Neves wrote:
 On Wednesday, 8 February 2023 at 00:40:44 UTC, Richard (Rikki) Andrew 
 Cattermole wrote:
 On 07/02/2023 11:15 PM, Atila Neves wrote:
 What special rule? If a dmd change breaks Phobos, doesn't it make 
 more sense to fix Phobos in the same PR than submitting a separate PR 
 to the Phobos repo (and having to explain that it's because of dmd PR 

If a Phobos change breaks MIR, doesn't it make more sense to fix MIR in the same PR than submitting a separate PR to the MIR repo (and
I don't understand how this analogy applies unless you're advocating for a monorepo with all dub packages, which I doubt is the case.
My point is that the arguments being made for it, are easily applied to other projects than just Phobos.
 We can apply this same logic to literally any code base that is in the 
 auto tester. The dmd repository is going to get quite big when we keep 
 including what should be isolated code bases administratively...
I don't think we can, because I'm asking about putting code that's under control by the same group of people in the same repository and I don't see how that applies to private individual projects.
It shouldn't be the same group! Its a vastly different skill set to develop a compiler than it is to develop a standard library. Same goes for things like GC's, it is its own specialty. We should be moving things out of dmd's repository, not moving them in!
Feb 13 2023
parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 14 February 2023 at 01:57:43 UTC, Richard (Rikki) 
Andrew Cattermole wrote:
 Same goes for things like GC's, it is its own specialty. We 
 should be moving things out of dmd's repository, not moving 
 them in!
What practical problem would moving the GC out of the DMD repo solve?
Feb 13 2023
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 14/02/2023 3:21 PM, deadalnix wrote:
 On Tuesday, 14 February 2023 at 01:57:43 UTC, Richard (Rikki) Andrew 
 Cattermole wrote:
 Same goes for things like GC's, it is its own specialty. We should be 
 moving things out of dmd's repository, not moving them in!
What practical problem would moving the GC out of the DMD repo solve?
Not just the GC, almost all of the druntime shouldn't be in there. There are three things directly tied to the compiler that do need to be in there: 1. Attributes 2. Intrinsics (*sigh* atomics not picking that battle with dmd right now) 3. Hooks, must be templated (has its own benefits in simplifying the compiler quite significantly) Everything else is library code like bindings which super doesn't need a compiler developer to look at them. And that is the point, its different groups that should be maintaining it, merging it will not encourage it, it will only mean people step on each others toes more.
Feb 13 2023
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Tuesday, 14 February 2023 at 02:27:42 UTC, Richard (Rikki) 
Andrew Cattermole wrote:
 On 14/02/2023 3:21 PM, deadalnix wrote:
 On Tuesday, 14 February 2023 at 01:57:43 UTC, Richard (Rikki) 
 Andrew Cattermole wrote:
 Same goes for things like GC's, it is its own specialty. We 
 should be moving things out of dmd's repository, not moving 
 them in!
What practical problem would moving the GC out of the DMD repo solve?
Not just the GC, almost all of the druntime shouldn't be in there. There are three things directly tied to the compiler that do need to be in there: 1. Attributes 2. Intrinsics (*sigh* atomics not picking that battle with dmd right now) 3. Hooks, must be templated (has its own benefits in simplifying the compiler quite significantly) Everything else is library code like bindings which super doesn't need a compiler developer to look at them. And that is the point, its different groups that should be maintaining it, merging it will not encourage it, it will only mean people step on each others toes more.
Ok, so I gather that no practical problem is being solved by said maneuver.
Feb 13 2023
prev sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Tuesday, 14 February 2023 at 02:27:42 UTC, Richard (Rikki) 
Andrew Cattermole wrote:
 On 14/02/2023 3:21 PM, deadalnix wrote:
 On Tuesday, 14 February 2023 at 01:57:43 UTC, Richard (Rikki) 
 Andrew Cattermole wrote:
 Same goes for things like GC's, it is its own specialty. We 
 should be moving things out of dmd's repository, not moving 
 them in!
What practical problem would moving the GC out of the DMD repo solve?
Not just the GC, almost all of the druntime shouldn't be in there. There are three things directly tied to the compiler that do need to be in there: 1. Attributes 2. Intrinsics (*sigh* atomics not picking that battle with dmd right now) 3. Hooks, must be templated (has its own benefits in simplifying the compiler quite significantly) Everything else is library code like bindings which super doesn't need a compiler developer to look at them. And that is the point, its different groups that should be maintaining it, merging it will not encourage it, it will only mean people step on each others toes more.
In what way would they step on each others toes (more than they do separately)? A single repo solves a real problem, which is all the PITA caused when doing cross-cutting changes and an overly complex developer experience full of silly pitfalls and careful juggling (oh, right, I need to git pull on the other repo as well, maybe? Which dmd is this building with again? Oh damn I forgot to switch branches in 3 places at once). Putting aside status quo bias, any separation should also solve something comparable.
Feb 13 2023
parent reply Mathias LANG <geod24 gmail.com> writes:
On Tuesday, 14 February 2023 at 06:55:50 UTC, John Colvin wrote:
 In what way would they step on each others toes (more than they 
 do separately)? A single repo solves a real problem, which is 
 all the PITA caused when doing cross-cutting changes and an 
 overly complex developer experience full of silly pitfalls and 
 careful juggling (oh, right, I need to git pull on the other 
 repo as well, maybe? Which dmd is this building with again? Oh 
 damn I forgot to switch branches in 3 places at once). Putting 
 aside status quo bias, any separation should also solve 
 something comparable.
Agreed that the developer experience is currently sub-par. The way I see the monorepo reasoning, it is attempting to solve systemic problems that surface when a few dimensions (at least projects & members, perhaps others) reach a very large scale. When you have more than 10k developers, employing someone to save them 10 minutes each per day is actually a 10x improvement. What we will mostly loose by having a monorepo is a visible interface between core language and standard library. The interface will be less apparent, and easier to break. We have very concrete evidence that this is something that leads to problems (Tango/Phobos), and the separation exists today because of this. In recent meetings, Walter complained that the DMD test suite depends on Phobos, and he's right. Fixing this problem would alleviate some of the issues you're describing it. LDC fixed that issue, and I think GDC did too, but DMD didn't yet.
Feb 14 2023
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 14 February 2023 at 10:24:36 UTC, Mathias LANG wrote:
 What we will mostly loose by having a monorepo is a visible 
 interface between core language and standard library. The 
 interface will be less apparent, and easier to break. We have 
 very concrete evidence that this is something that leads to 
 problems (Tango/Phobos), and the separation exists today 
 because of this.
[1]
 In recent meetings, Walter complained that the DMD test suite 
 depends on Phobos, and he's right. Fixing this problem would 
 alleviate some of the issues you're describing it. LDC fixed 
 that issue, and I think GDC did too, but DMD didn't yet.
[2] I would like to point our that [2] is concrete evidence that [1] clearly is not solved by the use of separate repos.
Feb 14 2023
parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Tuesday, 14 February 2023 at 12:46:33 UTC, deadalnix wrote:
 On Tuesday, 14 February 2023 at 10:24:36 UTC, Mathias LANG 
 wrote:
 What we will mostly loose by having a monorepo is a visible 
 interface between core language and standard library. The 
 interface will be less apparent, and easier to break. We have 
 very concrete evidence that this is something that leads to 
 problems (Tango/Phobos), and the separation exists today 
 because of this.
[1]
 In recent meetings, Walter complained that the DMD test suite 
 depends on Phobos, and he's right. Fixing this problem would 
 alleviate some of the issues you're describing it. LDC fixed 
 that issue, and I think GDC did too, but DMD didn't yet.
[2] I would like to point our that [2] is concrete evidence that [1] clearly is not solved by the use of separate repos.
They are not in any way related. Having phobos in the same repo as dmd does not make [2] go away. The lament being made is that Phobos is being built before running the testsuite, not that Phobos is in a separate repo. Regarding [1], I know the reference, however it's not clearly spelled out. D used to be maintained in one repository, it nearly killed the language. In the most pessimistic outlook, the split took the wind out of D's sails, and left the language permanently on the horizon. The end result of the fallout being druntime got removed from phobos, severing any ties between core and standard runtime libraries so that the likes of Tango or Ocean could co-exist with D compiler/core.
Feb 20 2023
parent deadalnix <deadalnix gmail.com> writes:
On Monday, 20 February 2023 at 23:58:44 UTC, Iain Buclaw wrote:
 They are not in any way related.  Having phobos in the same 
 repo as dmd does not make [2] go away. The lament being made is 
 that Phobos is being built before running the testsuite, not 
 that Phobos is in a separate repo.
The claim never was that having phobos in the same repo makes the problem go away. In fact there is no such claim. However, it was claimed that using separate repos prevent this types of issues, which is a self detonating claim.
 Regarding [1], I know the reference, however it's not clearly 
 spelled out.  D used to be maintained in one repository, it 
 nearly killed the language. In the most pessimistic outlook, 
 the split took the wind out of D's sails, and left the language 
 permanently on the horizon. The end result of the fallout being 
 druntime got removed from phobos, severing any ties between 
 core and standard runtime libraries so that the likes of Tango 
 or Ocean could co-exist with D compiler/core.
While I was already a D user back then, I wasn't involved in the way I am now, so I do not have this much background. However, it is relatively obvious that a language ending up with 2 standard library is the byproduct of some severe problem and I would be very surprised if the explaination was as simple as a repository organisation question. In fact, if I had to bet, I would bet against this hypothesis even without the full knwoledge of the background.
Feb 20 2023
prev sibling parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Tuesday, 14 February 2023 at 10:24:36 UTC, Mathias LANG wrote:
 What we will mostly loose by having a monorepo is a visible 
 interface between core language and standard library. The 
 interface will be less apparent, and easier to break.
Although the original question was about Phobos, the scope doesn't stop there. Don't forget there's also dlang/tools, dlang/installer, dlang/dlang.org, dlang/dub, and dlang/downloads.dlang.org. There would be absolutely zero point or gain in merging dlang/phobos into dlang/dmd unless everything else goes in too.
Feb 20 2023
parent max haughton <maxhaton gmail.com> writes:
On Tuesday, 21 February 2023 at 00:18:51 UTC, Iain Buclaw wrote:
 On Tuesday, 14 February 2023 at 10:24:36 UTC, Mathias LANG 
 wrote:
 What we will mostly loose by having a monorepo is a visible 
 interface between core language and standard library. The 
 interface will be less apparent, and easier to break.
Although the original question was about Phobos, the scope doesn't stop there. Don't forget there's also dlang/tools, dlang/installer, dlang/dlang.org, dlang/dub, and dlang/downloads.dlang.org. There would be absolutely zero point or gain in merging dlang/phobos into dlang/dmd unless everything else goes in too.
The spec probably could be in the with the implementation. They're modified in lockstep only with the barrier of an additional PR being required for the spec. I also think it's hyperbolic to claim there is "zero" utility to it regardless of how you are calculating it.
Feb 20 2023
prev sibling parent Atila Neves <atila.neves gmail.com> writes:
On Monday, 6 February 2023 at 18:15:00 UTC, user1234 wrote:
 On Monday, 6 February 2023 at 17:56:33 UTC, Atila Neves wrote:
 On Monday, 6 February 2023 at 17:09:36 UTC, user1234 wrote:
 [...]
I don't understand, could you please explain how there'd be more breaking changes?
I'll try to reword a bit: In case of an unexpected breaking change that would manifest itself only in the Phobos test suite and not in the BuildKite CI, and without "the rule", authors will be tempted to fix the problem in phobos in the same PR as the one initially only supposed to change DMD.
And this would be a bad thing because...?
Feb 13 2023
prev sibling next sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
I am very much not a fan of horror movies. Let alone at 8am.

Let's not explore such horror topics such as mixing two sets of teams 
code bases which absolutely needs to be completely isolated from one 
another in such early hours.

But seriously, WHY ARE THERE COMPILER HOOKS IN PHOBOS (std.math)!
Feb 06 2023
next sibling parent user1234 <user1234 12.de> writes:
On Monday, 6 February 2023 at 19:13:35 UTC, Richard (Rikki) 
Andrew Cattermole wrote:
 I am very much not a fan of horror movies. Let alone at 8am.

 Let's not explore such horror topics such as mixing two sets of 
 teams code bases which absolutely needs to be completely 
 isolated from one another in such early hours.

 But seriously, WHY ARE THERE COMPILER HOOKS IN PHOBOS 
 (std.math)!
I'd say that they should be moved in druntime.
Feb 06 2023
prev sibling next sibling parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Monday, 6 February 2023 at 19:13:35 UTC, Richard (Rikki) 
Andrew Cattermole wrote:
 I am very much not a fan of horror movies. Let alone at 8am.

 Let's not explore such horror topics such as mixing two sets of 
 teams code bases which absolutely needs to be completely 
 isolated from one another in such early hours.

 But seriously, WHY ARE THERE COMPILER HOOKS IN PHOBOS 
 (std.math)!
Because somebody wanted a power operator. Also, WIP: https://github.com/dlang/dmd/pull/14297
Feb 06 2023
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/6/2023 11:38 AM, Iain Buclaw wrote:
 Because somebody wanted a power operator.
Maybe that should be revisited (i.e. use a function rather than an operator).
Feb 06 2023
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 07/02/2023 9:05 AM, Walter Bright wrote:
 On 2/6/2023 11:38 AM, Iain Buclaw wrote:
 Because somebody wanted a power operator.
Maybe that should be revisited (i.e. use a function rather than an operator).
Nah, its just the implementation is in the wrong place and its implementation needs revisiting. https://issues.dlang.org/show_bug.cgi?id=23538
Feb 06 2023
parent reply "H. S. Teoh" <hsteoh qfbox.info> writes:
On Tue, Feb 07, 2023 at 09:34:08AM +1300, Richard (Rikki) Andrew Cattermole via
Digitalmars-d wrote:
 On 07/02/2023 9:05 AM, Walter Bright wrote:
 On 2/6/2023 11:38 AM, Iain Buclaw wrote:
 Because somebody wanted a power operator.
Maybe that should be revisited (i.e. use a function rather than an operator).
Nah, its just the implementation is in the wrong place and its implementation needs revisiting. https://issues.dlang.org/show_bug.cgi?id=23538
Since ^^ is a built-in operator in the language, the implementation (and its dependencies) really should be in druntime. Any dependent functions like exp() that's currently in std.math should be moved to druntime, and std.math should just alias the current symbol to the druntime implementation. T -- The irony is that Bill Gates claims to be making a stable operating system and Linus Torvalds claims to be trying to take over the world. -- Anonymous
Feb 06 2023
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/6/23 3:46 PM, H. S. Teoh wrote:
 On Tue, Feb 07, 2023 at 09:34:08AM +1300, Richard (Rikki) Andrew Cattermole
via Digitalmars-d wrote:
 On 07/02/2023 9:05 AM, Walter Bright wrote:
 On 2/6/2023 11:38 AM, Iain Buclaw wrote:
 Because somebody wanted a power operator.
Maybe that should be revisited (i.e. use a function rather than an operator).
Nah, its just the implementation is in the wrong place and its implementation needs revisiting. https://issues.dlang.org/show_bug.cgi?id=23538
Since ^^ is a built-in operator in the language, the implementation (and its dependencies) really should be in druntime. Any dependent functions like exp() that's currently in std.math should be moved to druntime, and std.math should just alias the current symbol to the druntime implementation.
Yes. That's what has happened with all the other intrinsics, though this sounds less like an intrinsic, and more like a magic symbol. Or, alternatively. Create a small Phobos with just the pieces needed to make the test suite run, and don't ever change it. -Steve
Feb 06 2023
prev sibling parent ryuukk_ <ryuukk.dev gmail.com> writes:
On Monday, 6 February 2023 at 19:13:35 UTC, Richard (Rikki) 
Andrew Cattermole wrote:
 I am very much not a fan of horror movies. Let alone at 8am.

 Let's not explore such horror topics such as mixing two sets of 
 teams code bases which absolutely needs to be completely 
 isolated from one another in such early hours.

 But seriously, WHY ARE THERE COMPILER HOOKS IN PHOBOS 
 (std.math)!
I agree with that, the standard library is a library, it should be a separate thing The runtime lives with the compiler, making it easier to port the whole thing to new targets instead of having to do a whole mess all at once
Feb 07 2023
prev sibling next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Monday, 6 February 2023 at 10:22:32 UTC, Atila Neves wrote:
 Now that dmd/druntime is in one repo, is there a good reason to 
 not include Phobos?
I would recommend a discussion and vote (among normal participants) at one of the upcoming Dlang foundation monthly meetings. Ask two people to kick off the discussion with each one taking a different side (invite someone who doesn't typically attend those meetings if needed). One thing that might be useful to this discussion is some kind of criterion that participants can to judge the decision. Any choice will have pros and cons, but how do you judge what is most important? Are you trying to make the build process as smooth as possible? Or maybe to optimize dmd/druntime developers' time? Or maybe to minimize breakage in phobos from dmd/druntime changes? Or maybe to minimize burdens on new contributors? Or maybe some balance of these potentially conflicting objectives.
Feb 07 2023
prev sibling parent Johan <j j.nl> writes:
On Monday, 6 February 2023 at 10:22:32 UTC, Atila Neves wrote:
 Now that dmd/druntime is in one repo, is there a good reason to 
 not include Phobos?
I'm in favor of this. In this thread I have seen no arguments against it that are relevant, because they do not relate to having things in one or more repositories. The order in which things are built, or mutual independence, etc, those are not related to having things in one repository. It is related to the CI and build system in place. You will need to have checked out all repositories to complete CI testing; can't ship the compiler without Phobos included. FYI, LDC has effectively had things in one repository (using submodule) for a very long time, definitely since I joined the project many years ago. LLVM now also has _Everything_ in one repository (runtime, libcxx, clang, flang (another language!), lld, ...). It's convenient. For DMD development, having things in one repo will make it much easier to deal with breaking changes (and will also make git --bisect easier). Just mentally imagine what currently needs to be done when a DMD change breaks Phobos compilation (the coordination of PR merging and keeping track of which DMD commithash requires which Phobos commithash); and then imagine what that would look like in a monorepo. -Johan
Feb 08 2023