www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - D Language Foundation Quarterly Meeting, October 2021

reply Mike Parker <aldacron gmail.com> writes:
This meeting took place on October 22nd at 13:00 UTC. Apologies 
for the delayed summary.

This was a quarterly meeting, meaning there were two parts. The 
first was focused on industry reps and contributors using D in 
production; the second was the monthly D Language Foundation 
meeting. If you are using D in production and would like to 
participate in our quarterly meetings, please let me know.

The meeting lasted three hours. The following people attended:

**Industry Reps**
Iain Buclaw for GDC
Martin Kinkelin for LDC
Mario Kröplin for Funkwerk
Mathias Lang for BPF Korea
Robert Schadek for Symmetry Investments
Joseph Wakeling for Frequenz

**Contributors**
Petar Kirov
Vladimir Panteleev
Steven Schveighoffer

**D Language Foundation**
Andrei Alexandrescu
Walter Bright
Ali Çehreli
Max Haughton
Átila Neves
Razvan Nitu


Joseph had no issues to report, but he brought up his concerns 
about the state of D in relation to Rust. This led to a 
discussion that touched on the excitement of things happening in 
the Rust space (new projects, particularly in the field of Data 
Science), the availability of libraries in the Rust ecosystem (in 
many cases, it's easier to get a new project going in Rust than 
in D), and marketing (he knows programmers who are highly 
enthused by Rust's offerings and unenthusiastic about the 
potential of D by comparison).

This led to a recurring discussion throughout the meeting about 
marketing, branding, and the availability of libraries. Some 
salient points were raised throughout:

* Word of mouth is a powerful form of marketing, and languages 
with solid tooling support (IDEs that almost write the code for 
you) and a large library ecosystem (like Go's "lego style" 
programming, where much of what you need for modern programming 
has already been written) have a lot more to talk about. We have 
a great language to talk about, but we're missing the other stuff.
* We may be appealing to the kind of programmer who is happy with 
Emacs and has no problem writing the support libraries they need, 
but that sort of programmer is not the future of programming.
* We don't need to be a better C, because no one wants to write 
new code in C. We need to just be *better*.
* HN/Reddit/Etc have a constant stream of "I wrote X in Rust" 
articles. We need more people who are using D to write about/talk 
about what they are doing so that others can see it.
* The biggest languages have a story. (Java: write once run 
anywhere, the language of the internet; Rust: memory safety 
without a GC; Go: goroutines). What is D's story? It's not one 
big thing, but a lot of little things, and that's not a strong 
story.
* Branding. We need to determine what we are as a brand and 
should consider hiring a branding expert.


Mario said Funkwerk wants to create containers that can accept 
arbitrary types, and that includes accepting `immutable` and 
`const` types without the containers, in turn, becoming 
`immutable`/`const`. Unfortunately, they could not use the Phobos 
`Rebindable` for this, as it doesn't wrap structs. Mathis Beer 
worked on a solution, and Mario asked him to [submit it to the 
forums for 
feeback](https://forum.dlang.org/post/qnzxruolyeozohflretb forum.dlang.org). It
received very little. And they aren't happy with it as it's a "very tricky,
very ugly" solution that relies on compiler behavior. Should they continue to
use it and risk it breaking if the compiler is changed? Átila said he would
look at their code.

A second issue was regarding a Funkwerk PR to dfmt that adds a 
`single_indent` option to dfmt for continuation lines since dfmt 
only supported double indent continuations, something that was 
rejected for the D Style Guide. The PR got held up waiting for 
feedback from Brian Schott, and a later request to merge it 
anyway received no response. Most of the people present at the 
meeting have merge rights to dfmt and agreed it should be merged, 
[and Razvan did 
so](https://github.com/dlang-community/dfmt/pull/535).


Robert said he has no major issues with his current project. He 
did circle back to the libraries/marketing discussion, and I 
summarized his remarks above. Other than that, he is happy using 
D every day.

He also said he has [continued working on 
"dud"](https://forum.dlang.org/post/cnkbatgkqtdmbldlnewl forum.dlang.org) as
his Fridays project at Symmetry. He is currently working on dependency
resolution, based on the work done in dub. He intends to do a talk about dud
and its dependency resolution at the next real-world DConf. He mentioned that
in adding parsing support for JSON/SDLang dub recipes, he found several SDLang
recipes that have unexpected syntax. Finally, though he expects to keep a
legacy dub-like system, where the user types `dud` and the project builds and
runs, he would like to connect to Átila's reggae for anything more complex.



three months ago, in code written by someone experienced with D. 
They would love a solution for it.

A few months ago, they were also hit by a bug in 
`-preview=dtorfields` that [was triggered in 
vibe.d](https://github.com/vibe-d/vibe-core/issues/283) and 
[reported in Issue 
https://issues.dlang.org/show_bug.cgi?id=21989) after 
Sönke Ludwig tracked it down. A PR for this issue by 
MoonlightSentinel is currently stalled, but the preview switch 
[was enabled by default in 
2.098.0](https://dlang.org/changelog/2.098.0.html#dtorfileds). 
BPFKorea was forced to disable the feature with 
`-revert=dtorfields`.

They also have some exception-related bugs on Alpine Linux, but 
the stack traces on Alpine don't work yet. He has done some work 
on getting them working but has not had time to finish it. He is 
hoping to fix that when he is able.


Martin reported that LDC's DLL support is working well and has 
been used successfully at Symmetry.


Over the last quarter, Iain has submitted patches to GCC for 
bootstrapping the D compiler and is waiting on approval. Once he 
has that, he'll be able to delete the current C++ front end and 
libraries and add in the master versions of dmd (the d front 
end), druntime, and phobos. With that, GDC will be at the latest 
version of D.

Iain has been working with Walter on Import C. A recent PR from 
Walter adding support for `va_start` and `va_end` now enables 
them to cleanly compile a preprocessed zlib to object code. Iain 
will soon be testing the output against zlib's test suite to see 
if there are any hidden run-time bugs in ImportC. They've 
selected zlib as a test target because it is compiled in Phobos 
with DMC, so it would be great to get it compiling with DMD 
instead.

He also brought up our aging server infrastructure, including 
digitalmars.com, dlang.org, the auto tester, and more, which has 
been a topic of discussion in past meetings. This led to a 
discussion about the auto tester (e.g., what is it covering that 
our other CI solutions are not, how switching to a monorepo would 
benefit the auto tester, how can we avoid unnecessary builds, 
etc.), our CI services, and the state of the current auto tester 
server stack.

(A couple of related notes.

I participated in a non-foundation meeting yesterday with 
Vladimir, Petar, Mathias, Max, and Razvan as a preliminary 
discussion of what it would take to merge some of the core 
repositories into a single repo. This resulted in a plan to put 
together a proposal and bring it to a monthly Foundation meeting, 
tentatively targeting our November meeting.

I am attending a foundation meeting later today with someone who 
has volunteered to advise us on bringing our server 
infrastructure under Foundation management and multiple admins. 
This meeting is intended to provide our adviser with a clear 
picture of the existing servers and our goals so that he can 
formulate some suggestions and bring them to one of our future 
monthly meetings.)


[Vladimir's open-source btdu 
project](https://github.com/CyberShadow/btdu) has been picking up 
recently among users of [the btrfs file 
system](https://btrfs.wiki.kernel.org/index.php/Main_Page). A 
problem he's encountered is one of packaging and distribution. 
For example, on Debian: DMD isn't packaged in the official 
distro, GDC has an old front end, and the LDC packages are 
outdated. This has led to a poor experience for new users of his 
software, who often decide to use something else. This led to a 
discussion about how to get D compiler packages into more 
distros. Most distros do not allow people to just start 
submitting packages without first establishing trust in some 
form. The best approach will likely be to sponsor someone who 
already has access.

He also discussed some plans he has for some changes to DustMite 
that he hopes will introduce D to more people. I will say no more 
about that for now and leave it to Vladimir to make the 
announcement when he is ready.


Petar is the CTO of a small startup using D in a limited 
capacity. He brought up two minor issues they have encountered. 
The first: `-chackaction=context` causes linker errors in release 
builds. For now, they are fine using the switch only in unit test 
builds. The second: they could not find an easy way to customize 
the visualization of `DateTime` in vibe.d, but they were able to 
work around it. He feels that though the project is small it was 
a success story for its use case.

Petar is the only active maintainer of [DLang 
Tour](https://tour.dlang.org/) but is not happy with its current 
state. He has ideas on how to improve it but has not been able to 
make the time for it. He would be happy to guide anyone who would 
like to volunteer to do some work on it and will resume work on 
it himself when he has more time. If anyone is interested please 
let us know in this thread. I will make a more visible effort to 
recruit someone later. One of the things he would like to do is 
use DLang Tour to make all of the examples in Ali's book runnable.

Finally, Petar believes the tools that Hackerpilot (Brian Schott) 
developed (dcd, dfmt, dsymbol, etc.) are extremely useful, but it 
can be annoying to contribute to them while they are in separate 
repositories. Sometimes, I change in one requires a change in 
another. He would like to see them all merged into a single 
repository.


Steve is teaching programming to some kids. He was originally 
using JavaScript, then Lua and Roblox, and now D and raylib-d (of 
which he has since become the maintainer). This has shown him 
that most of our existing D resources are targeted at people who 
already know how to program. We have very few resources geared 
toward teaching D to people who do not know how to program. 
Related to that, it's difficult to teach the kids how to 
interpret error messages. This would be much easier if compiler 
error messages and Phobos/DRuntime error messages were improved. 
For example, a `RangeError` on an invalid array index should 
report what the number was that caused the problem; failed 
asserts should report the offending expression; etc. This would 
be useful to everyone, not just beginners.


`-preview=in` came up briefly at a previous meeting, in the 
context of a larger discussion, as a mistake that should be 
killed. Given that it was implemented by Mathias Lang, we decided 
to bring him into a future meeting to present his case for 
keeping the feature around and ultimately moving it out of 
preview.

Mathias provided a bit of background on the feature. `in` used to 
mean `scope const`, but then DIP 1000 came along, and `in` was 
changed to just `const`. Then later, a preview switch was 
introduced to make `in` mean `scope const` again. Mathias felt 
that `in` by itself was not very useful, so he saw an opportunity 
to improve it based on a use case in BPFKorea's codebase. So he 
changed its behavior, behind `-preview=in`, such that `in` means 
`const scope ref`, with the addition that the compiler is free to 
optimize to pass by value when it can (e.g., small POD structs) 
and it accepts rvalue references. The big objection raised in 
past discussions was aliasing. Mathias feels that is a more 
general problem, is a rare problem, and ` live` is supposed to 
help with that. BPFKorea has been using the feature in production 
for over a year, and in that time it has only caused a single 
bug, which came down to the fact that they weren't using 
`preview=dip1000`, so `scope` wasn't being enforced. Their code 
is sensitive about how things get copied around, and they have 
had memory corruption in the past. `-preview=in` solved that 
problem. He then mentioned some ideas he had to make it more 
useful, which led to some discussion that isn't relevant to this 
summary.

This was a long discussion that took up a full hour.

Martin, Petar, and Ali voiced strong support for the feature. 
Some of their arguments in favor:

* it avoids the situation in templated code with an `in ref` 
parameter where you have, e.g., an `int` being passed by 
reference; the compiler can optimize this to pass the `int` by 
value.
* you don't have to concern yourself with deciding whether to 
pass a `const` parameter by reference or by value; the compiler 
will do for you the best thing on the current platform.
* this eliminates the need for multiple function overloads and/or 
`static if` blocks to do manually what this does for you, which 
simplifies the language (and is something easier to understand).
* it eliminates the need for `-preview=rvaluerefparams` (and [the 
DIP on which that feature is 
based](https://github.com/dlang/DIPs/pull/218)) and does not 
allow rvalues to be mutated.


Átila and Walter had strong reservations. Their two major 
objections:

* (Átila) if `-preview=rvaluerefparams` is broken, or the DIP is 
wrong, we should fix it.
* (Walter) the issue with `-preview=in` and aliasing isn't that 
it can happen, it's that it can sometimes happen and sometimes 
not depending on if the compiler optimizes to pass by value, and 
that can vary from platform to platform; this is a major 
inconsistency in that the behavior is unpredictable; that 
aliasing is rare makes this a more pernicious problem that can 
silently introduce memory corruption.

When reading the above, you will probably think of things to say 
for or against specific arguments. I'm willing to bet that it was 
already said in this meeting. They drilled down deep into the 
weeds and raised several points and counterpoints related to the 
above arguments and other, more minor arguments. In the end, they 
found their way to an agreement on how to proceed.

`-preview=in` will not be killed. It needs to be changed such 
that:

* `in` always means `const scope ref`; the compiler will not 
attempt to pass by value based on platform-specific heuristics.
* `in` will accept rvalue references.
* the compiler can in the future optimize to pass by value to 
`in` with more appropriate heuristics (e.g., it can prove there 
is no aliasing).


Phew! That was a longer summary than I expected to write. Our 
next meeting is a normal monthly meeting (no industry reps) and 
will take place on Friday, November 26th, at 14:00 UTC (the 
American clocks will be moved back an hour before then, so we 
have to move from 13:00 to 14:00 UTC so that we can keep it at 
06:00 PST for Walter and Ali).

It looks like the primary agenda item for the next meeting will 
be a discussion on merging the druntime repository into the dmd 
repository. If you have anything you'd like us to discuss, or 
would like to discuss with us, please let me know and I'll tell 
you if I can squeeze you in.
Nov 05 2021
next sibling parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Friday, 5 November 2021 at 11:57:40 UTC, Mike Parker wrote:

 This meeting took place on October 22nd at 13:00 UTC. Apologies 
 for the delayed summary.

 [...]


Nice summary! I got more confidence in the future of D after 
reading this.
Nov 05 2021
prev sibling next sibling parent Guillaume Piolat <first.last gmail.com> writes:
On Friday, 5 November 2021 at 11:57:40 UTC, Mike Parker wrote:

 Phew! That was a longer summary than I expected to write.


Whoa! Thank you Michael. That and the recent Youtube videos are 
both excellent!
Nov 05 2021
prev sibling next sibling parent Dukc <ajieskola gmail.com> writes:
On Friday, 5 November 2021 at 11:57:40 UTC, Mike Parker wrote:

 * The biggest languages have a story. (Java: write once run 
 anywhere, the language of the internet; Rust: memory safety 
 without a GC; Go: goroutines). What is D's story? It's not one 
 big thing, but a lot of little things, and that's not a strong 
 story.


I wonder if the underlying theme, that separates us from the 
rest, is "low or high-level, just as home"? At least when it 
comes to the stance on GC, that seems to be the hot potato on HN, 
that applies. Rust advertises working well without the GC. Go 
advertises working well with the GC. We lobby for working with or 
without based on situation.
Nov 05 2021
prev sibling next sibling parent reply zjh <fqbqrr 163.com> writes:
On Friday, 5 November 2021 at 11:57:40 UTC, Mike Parker wrote:

 This meeting took place on October 22nd at 13:00 UTC.


D can aim at `experts`, especially `meta programming users`.
On this point,`rust` can't compete.
`Silky general meta programming`.
Use my `strengths` to attack theirs weaknesses.

It is more important to take time to do a good job of the product 
and fix various bugs than to vigorously publicize it now.
If the user `comes`, and the experience is bad, which is a bad 
thing.
We shouldn't be in a hurry. We should take our time.
Need to work on tools, especially `ide` or `vim` plug-ins.
Many people do not insist on using D. because the tools are not 
good enough.
Nov 05 2021
parent reply JN <666total wp.pl> writes:
On Friday, 5 November 2021 at 13:19:24 UTC, zjh wrote:

 D can aim at `experts`, especially `meta programming users`.
 On this point,`rust` can't compete.
 `Silky general meta programming`.
 Use my `strengths` to attack theirs weaknesses.


This is much less of a strength than you think. For 90% of cases, 
lack of metaprogramming is resolved by putting a Python script in 
build step that autogenerates the necessary code.
Nov 06 2021
next sibling parent zjh <fqbqrr 163.com> writes:
On Saturday, 6 November 2021 at 15:46:57 UTC, JN wrote:

 On Friday, 5 November 2021 at 13:19:24 UTC, zjh wrote:

 D can aim at `experts`, especially `meta programming users`.


Python script.


Although practical, it's ugly. It's no different from C++ macros.
Nov 06 2021
prev sibling next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Saturday, 6 November 2021 at 15:46:57 UTC, JN wrote:

 This is much less of a strength than you think. For 90% of 
 cases, lack of metaprogramming is resolved by putting a Python 
 script in build step that autogenerates the necessary code.


Yes, I agree. For a single project metaprogramming has little 
impact.

But such scripts are rarely reused between projects. Where 
metaprogramming has high potential is in creating more adaptive 
frameworks that can reused in many projects.

It does requires high level of sophistication and insight (and 
experimentation) to build such frameworks though.
Nov 07 2021
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Saturday, 6 November 2021 at 15:46:57 UTC, JN wrote:

 On Friday, 5 November 2021 at 13:19:24 UTC, zjh wrote:

 D can aim at `experts`, especially `meta programming users`.
 On this point,`rust` can't compete.
 `Silky general meta programming`.
 Use my `strengths` to attack theirs weaknesses.


 This is much less of a strength than you think. For 90% of 
 cases, lack of metaprogramming is resolved by putting a Python 
 script in build step that autogenerates the necessary code.


Python has no idea about D syntax or semantics (or any other 
language not called Python), and therefore can't even do 
something as simple as "what are all the D structs in module x". 
You'd have to invent a templating language on top of the code 
you're already writing, then write some Python code to parse and 
generate on top of *that*.

To me, that's like saying that C/Fortran aren't that big of a 
deal compared to assembly. Who needs for loops, amirite?
Nov 08 2021
parent Tobias Pankrath <tobias pankrath.net> writes:
On Monday, 8 November 2021 at 12:02:43 UTC, Atila Neves wrote:

 On Saturday, 6 November 2021 at 15:46:57 UTC, JN wrote:

 On Friday, 5 November 2021 at 13:19:24 UTC, zjh wrote:

 D can aim at `experts`, especially `meta programming users`.
 On this point,`rust` can't compete.
 `Silky general meta programming`.
 Use my `strengths` to attack theirs weaknesses.


 This is much less of a strength than you think. For 90% of 
 cases, lack of metaprogramming is resolved by putting a Python 
 script in build step that autogenerates the necessary code.


 Python has no idea about D syntax or semantics (or any other 
 language not called Python), and therefore can't even do 
 something as simple as "what are all the D structs in module 
 x". You'd have to invent a templating language on top of the 
 code you're already writing, then write some Python code to 
 parse and generate on top of *that*.

 To me, that's like saying that C/Fortran aren't that big of a 
 deal compared to assembly. Who needs for loops, amirite?



to let users hook into the compilation step and generate code on 
the fly given full compiler knowledge to the already existing 
code prior to generation.
Nov 08 2021
prev sibling next sibling parent reply Matheus <matheus gmail.com> writes:
On Friday, 5 November 2021 at 11:57:40 UTC, Mike Parker wrote:

 ...


Nice summary, but why these meetings are no recorded or streamed?

Matheus.
Nov 05 2021
parent Mike Parker <aldacron gmail.com> writes:
On Friday, 5 November 2021 at 13:19:30 UTC, Matheus wrote:

 On Friday, 5 November 2021 at 11:57:40 UTC, Mike Parker wrote:

 ...


 Nice summary, but why these meetings are no recorded or 
 streamed?

 Matheus.


I do record the meetings to help me with the summaries. In this 
case, I was unable to attend, so the recording was my primary 
source. I don't think that would serve any purpose to publish it 
or to stream the meetings beyond what the summaries already 
provide.

The big issue for me: when people have the public eye on them, it 
will inevitably impact what they say in the meeting. One 
particular opinion or remark can become the news rather than the 
topic about which it was made. We want everyone participating to 
be able to speak freely without self-censoring, or worrying about 
what viewers would think of any specific thing they say. This is 
especially true now that we are inviting contributors to 
participate, as some of them may be more reluctant to do so if 
their comments are public.

There are also times when we make tentative plans for something 
and we don't want to make the details public until we are more 
certain that it is going to happen. I see no benefit in exposing 
the details of these conversations.

And then there are times when we discuss topics related to one of 
the D shops that they don't want to be made public knowledge. 
This doesn't just happen in the quarterly meetings when the 
industry reps are present, but also sometimes in the smaller 
monthly meetings.

And even if we did stream them or publish a recording, I would 
still need to write a summary for the people who don't want to 
sit through a 2+ hour stream or skim through a recording looking 
for the highlights. Consider the summary of the 
marketing/ecosystem discussion. That discussion was scattered 
throughout the first two hours of the meeting as people kept 
coming back to it. Who has the time to skip back and forth 
through the video to piece it all together? So I'll have to edit 
out any sensitive topics that we discussed *and* write the 
summary. No thanks.
Nov 05 2021
prev sibling next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
Thank you, Mike. I love these digests.

On 11/5/21 4:57 AM, Mike Parker wrote:


 One of the things
 [Petar Kirov] would like to do is use DLang Tour to make all of the 


examples in

 Ali's book runnable.


(That discussion must have happened when I had to leave to for another 
meeting.)

Interesting... Perhaps I can link from the chapters like "run this code 
on DLang Tour."


 For example, a `RangeError` on an invalid array index should
 report what the number was that caused the problem; failed asserts
 should report the offending expression; etc.


There is great progress there already:

auto foo(int[] arr) {
   return arr[42];
}

void main() {
   foo([1, 2]);
}

core.exception.ArrayIndexError deneme.d(1755): index [42] exceeds array 
of length 2

Awesome! :)






 Martin, Petar, and Ali voiced strong support for the feature. Some of
 their arguments in favor:


Another argument was "intent". As in, the programmer means "this is an 
input to this function" and the compiler does the rest. Today, both of 
the following are input parameters:

void foo(ref const(Foo) a) {}
void foo(Foo a) {}

But wait... the latter may not be an input because it may be a struct 
that has indirections that I may modify through?

How about the following "input" parameter:

void foo(const(Foo) a) {}

That 'const' has no meaning and no place there if Foo is purely a value 
type (e.g. int). Why should the knowledge of the implementation's not 
modifying that copy leak out to the interface?

I think all of that can be fixed with in(tent). ;)

void foo(in Foo a) {}

Done. In other words, dear compiler, you deal with it! :)


 `-preview=in` will not be killed. It needs to be changed such that:


Yay!

Ali
Nov 05 2021
prev sibling parent reply Kagamin <spam here.lot> writes:
On Friday, 5 November 2021 at 11:57:40 UTC, Mike Parker wrote:

 `-preview=in` will not be killed. It needs to be changed such 
 that:

 * `in` always means `const scope ref`; the compiler will not 
 attempt to pass by value based on platform-specific heuristics.


What about C functions? The ABI changes.
Nov 08 2021
parent Mathias LANG <geod24 gmail.com> writes:
On Monday, 8 November 2021 at 21:48:03 UTC, Kagamin wrote:

 On Friday, 5 November 2021 at 11:57:40 UTC, Mike Parker wrote:

 `-preview=in` will not be killed. It needs to be changed such 
 that:

 * `in` always means `const scope ref`; the compiler will not 
 attempt to pass by value based on platform-specific heuristics.


 What about C functions? The ABI changes.


`in` doesn't exist in C. All DRuntime bindings have been switched 
not to use `in` AFAIK (I'll do another pass on druntime / Phobos 
when my other changes are ready).
There's a PR to disable it for non D/C++: 
https://github.com/dlang/dmd/pull/12242 but that was before this 
meeting.
Nov 08 2021