www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - named arguments, string interpolation, please stop.

reply deadalnix <deadalnix gmail.com> writes:
I have been in the D community for a very long time. I have seen 
D successfully deployed in companies, and the pain points 
associated with it. I have seen D fails to catch on in companies 
and why that is has well.

Let me tell you, none of this has anything to do with feature D 
has or does not have. At large, D has more features than most 
languages.

D chasing the next feature like a crack addict chase his next 
dose. With the same level of success.

The main problem people face with D in the real world are almost 
exclusively of the implementation kind. The list is endless (and 
yes, there are many bugs reports about these things). I recently 
made a post about how the OOP implementation is extremely sub-par 
vs what people in OOP languages would expect. no change of the 
language required to fix. See here: 
https://forum.dlang.org/post/hteuczyclxajakrisxjd forum.dlang.org

But if you are not convinced, here are a few more example of 
thing being implemented wrong or existing feature not working 
right:
  - D runtime is unable to see thread started manually (for 
instance with pthread-create) leading to all kind of bizarre 
behavior.
  - Template symbols are generated as weak, which prevents 
inlining (!).
  - Pretty much no cross module inlining, making helper function 
absurdly costly.
  - scope(success) generates exception handling code.
  - D goes virtual by default for class methods, but LTO is unable 
to finalize (contrary to every other languages going virtual by 
default).
  - The GC implementation is nowhere close to where it needs to be.
  - in contracts are dynamically bound (and in the callee) instead 
of statically bounds and in the caller.

These are just simple thing that I have on top of my mind, but 
there are a ton more. I have seen some of the above cause 
projects to fail. None of them require any significant language 
change.

There is nothing features like string interpolations or named 
argument can bring to the table that could pay for the 
implementations problem of existing feature. The cost benefit 
analysis is just a big L for D: the fail to address the main pain 
points, while causing massive breakage in the tooling ecosystem 
(syntax highlighting support in 3rd party IDE, code formatter, 
etc...), and it cost real time and resource to upgrade these, or 
come at the cost of other quality of life stuff nullifying their 
benefit (for instance, the quality of syntax highlighting for D 
has degraded significantly in vim and sublime text over the past 
few years).

In addition, some recent D features, such as  nogc, has been a 
productivity disaster int he wild. While the impact might not be 
felt on smaller codebases, the infectious nature of the feature 
makes large codebase significantly harder the refactor than they 
used to be.

Each time we take steps in that direction, D becomes a harder 
sell.
Jan 11
next sibling parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Thursday, 11 January 2024 at 13:07:52 UTC, deadalnix wrote:

 There is nothing features like string interpolations or named 
 argument can bring to the table that could pay for the 
 implementations problem of existing feature.


FWIW: I anticipate named arguments significantly improving our 
code. Large parts of boilerplate's generated builders exist only 
due to D's lack of native named arguments.
Jan 11
next sibling parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Thursday, 11 January 2024 at 13:09:56 UTC, FeepingCreature 
wrote:

 On Thursday, 11 January 2024 at 13:07:52 UTC, deadalnix wrote:

 There is nothing features like string interpolations or named 
 argument can bring to the table that could pay for the 
 implementations problem of existing feature.


 FWIW: I anticipate named arguments significantly improving our 
 code. Large parts of boilerplate's generated builders exist 
 only due to D's lack of native named arguments.


+1, same here for named arguments.
I will leave to Theo the the reply about syntax highlighting 
[Joking :-)]

I sympathise with your  opinion, but it seems to me that one 
thing does not exclude polishing the harsh points of the language 
that you listed. BUT, polishing hard point involve  long term 
contributors, and we are going back to the "mother of all the 
problems".

I would add also that I've the impression that the IDE tooling 
quality improved al lot since years ago, and it's gaining more 
attention lately, web-freak is doing a great job, and Prajwal is 
working on dfmt.

/P
Jan 11
parent "H. S. Teoh" <hsteoh qfbox.info> writes:
On Thu, Jan 11, 2024 at 01:25:28PM +0000, Paolo Invernizzi via Digitalmars-d
wrote:

 On Thursday, 11 January 2024 at 13:09:56 UTC, FeepingCreature wrote:

 On Thursday, 11 January 2024 at 13:07:52 UTC, deadalnix wrote:

 There is nothing features like string interpolations or named
 argument can bring to the table that could pay for the
 implementations problem of existing feature.


 
 FWIW: I anticipate named arguments significantly improving our code.
 Large parts of boilerplate's generated builders exist only due to
 D's lack of native named arguments.


 
 +1, same here for named arguments.
 I will leave to Theo the the reply about syntax highlighting [Joking
 :-)]


lol, my reputation precedes me, I see. :-P

But I will say, just because *I* don't care for syntax highlighting,
doesn't mean everyone else doesn't. If that's what will draw more users,
then more power to them. As long as I have the option of turning it off.
;-)



 I sympathise with your  opinion, but it seems to me that one thing
 does not exclude polishing the harsh points of the language that you
 listed. BUT, polishing hard point involve  long term contributors, and
 we are going back to the "mother of all the problems".


Yes, the social problem. D has overcome seemingly impossible technical
odds, but the social problem remains. And from all appearances, it is
unlikely ever to change.  So this is never going to be solved.



 I would add also that I've the impression that the IDE tooling quality
 improved al lot since years ago, and it's gaining more attention
 lately, web-freak is doing a great job, and Prajwal is working on
 dfmt.


[...]

+1, credit where credit is due.


T

-- 
Sometimes the best solution to morale problems is just to fire all of the
unhappy people. -- despair.com
Jan 11
prev sibling next sibling parent Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Thursday, 11 January 2024 at 13:09:56 UTC, FeepingCreature 
wrote:

 On Thursday, 11 January 2024 at 13:07:52 UTC, deadalnix wrote:

 There is nothing features like string interpolations or named 
 argument can bring to the table that could pay for the 
 implementations problem of existing feature.


 FWIW: I anticipate named arguments significantly improving our 
 code. Large parts of boilerplate's generated builders exist 
 only due to D's lack of native named arguments.


I think it's the other way around

Also, we already have such arguments: structs :-)
Jan 11
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 11 January 2024 at 13:09:56 UTC, FeepingCreature 
wrote:

 On Thursday, 11 January 2024 at 13:07:52 UTC, deadalnix wrote:

 There is nothing features like string interpolations or named 
 argument can bring to the table that could pay for the 
 implementations problem of existing feature.


 FWIW: I anticipate named arguments significantly improving our 
 code. Large parts of boilerplate's generated builders exist 
 only due to D's lack of native named arguments.


Sure on the other hand, of the top level languages, very few do 
have named arguments.

named argument. Sure, some of the usual suspect do, such as 
python, but they are a minority, and the conclusion remains: 
named arguments may be nice to have, but aren't what's preventing 
anyone from using D.

There is a design mistake I see people making again and again, 
and it goes way beyond software: they design for what their 
current most satisfied users want. But that's really dumb, these 
people are already happy with the product! Anyone in this forum 
fall into that bucket, by the way. You got to observe how D is 
used int he wild by people not frequenting this space. What 
problem do they encounter? What frustrates them? What make the 
stop using it? These are the people you want to design for if you 
want to actually improve the product.
Jan 11
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 1/11/24 19:02, deadalnix wrote:

 ...
 There is a design mistake I see people making again and again, and it 
 goes way beyond software: they design for what their current most 
 satisfied users want.


This is perhaps a mistake if your primary goal is to increase your user 
base, but on the other hand popularity is mostly a function of marketing 
and inertia, technical merit is of lesser importance. Keeping your 
current users happy is therefore also important.


 But that's really dumb, these people are already happy with the product!


I think it's a bit more complicated than that in the context of an open 
source project. Those people can afford to use a project whose potential 
has not been fully realized and they can actively participate in shaping 
the product, which is part of the value proposition. You are not going 
to be able to keep people engaged enough to fix long-standing tricky 
issues that do not impact them a lot if they cannot even make an impact 
on something they care about that would be simple to implement.


 Anyone in this forum fall into that bucket, by 
 the way. You got to observe how D is used int he wild by people not 
 frequenting this space.


Well, that is pretty hard to do. I don't know most of those people, so 
this is not something that is actionable to me. The subset that I do 
know prioritize similar things to be fixed or improved that I do.


 What problem do they encounter? What frustrates 
 them? What make the stop using it? These are the people you want to 
 design for if you want to actually improve the product.


In any case, I think everything on your list (and probably anything 
further that will come to mind) also improves the experience for 
existing users. They are just maybe not the blockers they had been in 
your experience for everyone and/or they are hard to fix due to 
technical debt.
Jan 11
prev sibling next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Thursday, 11 January 2024 at 18:02:46 UTC, deadalnix wrote:

 [snip]
 There is a design mistake I see people making again and again, 
 and it goes way beyond software: they design for what their 
 current most satisfied users want. But that's really dumb, 
 these people are already happy with the product! Anyone in this 
 forum fall into that bucket, by the way. You got to observe how 
 D is used int he wild by people not frequenting this space. 
 What problem do they encounter? What frustrates them? What make 
 the stop using it? These are the people you want to design for 
 if you want to actually improve the product.


Like Abraham Wald and fighter planes in WW2.

https://en.wikipedia.org/wiki/Survivorship_bias#Military
Jan 11
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 1/11/24 19:02, deadalnix wrote:

 
 Sure on the other hand, of the top level languages, very few do have 
 named arguments.

 argument.


https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/named-and-optional-arguments

https://stitcher.io/blog/php-8-named-arguments

https://www.jetbrains.com/help/go/inlay-hints.html

Ok, I am cheating with the last one.
Jan 11
prev sibling next sibling parent Hipreme <msnmancini hotmail.com> writes:
On Thursday, 11 January 2024 at 13:07:52 UTC, deadalnix wrote:

 I have been in the D community for a very long time. I have 
 seen D successfully deployed in companies, and the pain points 
 associated with it. I have seen D fails to catch on in 
 companies and why that is has well.

 Let me tell you, none of this has anything to do with feature D 
 has or does not have. At large, D has more features than most 
 languages.

 D chasing the next feature like a crack addict chase his next 
 dose. With the same level of success.

 The main problem people face with D in the real world are 
 almost exclusively of the implementation kind. The list is 
 endless (and yes, there are many bugs reports about these 
 things). I recently made a post about how the OOP 
 implementation is extremely sub-par vs what people in OOP 
 languages would expect. no change of the language required to 
 fix. See here: 
 https://forum.dlang.org/post/hteuczyclxajakrisxjd forum.dlang.org

 But if you are not convinced, here are a few more example of 
 thing being implemented wrong or existing feature not working 
 right:
  - D runtime is unable to see thread started manually (for 
 instance with pthread-create) leading to all kind of bizarre 
 behavior.
  - Template symbols are generated as weak, which prevents 
 inlining (!).
  - Pretty much no cross module inlining, making helper function 
 absurdly costly.
  - scope(success) generates exception handling code.
  - D goes virtual by default for class methods, but LTO is 
 unable to finalize (contrary to every other languages going 
 virtual by default).
  - The GC implementation is nowhere close to where it needs to 
 be.
  - in contracts are dynamically bound (and in the callee) 
 instead of statically bounds and in the caller.

 These are just simple thing that I have on top of my mind, but 
 there are a ton more. I have seen some of the above cause 
 projects to fail. None of them require any significant language 
 change.

 There is nothing features like string interpolations or named 
 argument can bring to the table that could pay for the 
 implementations problem of existing feature. The cost benefit 
 analysis is just a big L for D: the fail to address the main 
 pain points, while causing massive breakage in the tooling 
 ecosystem (syntax highlighting support in 3rd party IDE, code 
 formatter, etc...), and it cost real time and resource to 
 upgrade these, or come at the cost of other quality of life 
 stuff nullifying their benefit (for instance, the quality of 
 syntax highlighting for D has degraded significantly in vim and 
 sublime text over the past few years).

 In addition, some recent D features, such as  nogc, has been a 
 productivity disaster int he wild. While the impact might not 
 be felt on smaller codebases, the infectious nature of the 
 feature makes large codebase significantly harder the refactor 
 than they used to be.

 Each time we take steps in that direction, D becomes a harder 
 sell.


Named arguments surely was a pretty useful feature, not for any 
function, but for one which D was missing that even C had:
```c
struct A a;
a = (struct A){a: 50, b:100};
```

Before named arguments, you could only use that syntax if you 
were using in the same line of declaration:
```d
///compiles
A a = {
    a: 50, b: 100
};

///Does not
A a;
a = {
   a: 50, b: 100
}

```

Right now, it is possible to achieve that in any place:
```d
A a;
a = A(a: 50, b: 100);
```
To me, this was a big win already for making cleaner syntax code, 
specially inside configuration parts.

But I feel like obliged to completely agree with your position 
and also, am waiting for all the improvements you last commented 
on classes which looked like a big win with no code breakage.
Template inlining is fairly essential to D, specially because of 
the Range interface nature.
Jan 11
prev sibling next sibling parent reply zjh <fqbqrr 163.com> writes:
On Thursday, 11 January 2024 at 13:07:52 UTC, deadalnix wrote:

 I have been in the D community for a very long time.



 ...



D needs a `global task list` and arranges for people to execute 
it.
What D needs is `organization`, and it is a `strong organization`!
Jan 11
next sibling parent reply zjh <fqbqrr 163.com> writes:
On Thursday, 11 January 2024 at 13:29:30 UTC, zjh wrote:

 D needs a `global task list` and arranges for people to execute 
 it.
 What D needs is `organization`, and it is a `strong 
 organization`!



Just like a `task scheduler`, various tasks need to be arranged 
`reasonably`.
Jan 11
parent reply zjh <fqbqrr 163.com> writes:
On Thursday, 11 January 2024 at 13:34:24 UTC, zjh wrote:

 Just like a `task scheduler`, various tasks need to be arranged 
 `reasonably`.


```d
///compiles
A a = {
    a: 50, b: 100
};
```

Why can't we construct it directly like C++?

```d
//C++ :
A a = {
    50,100
};
```
Jan 11
parent reply Hipreme <msnmancini hotmail.com> writes:
On Thursday, 11 January 2024 at 13:42:20 UTC, zjh wrote:


 Why can't we construct it directly like C++?

 ```d
 //C++ :
 A a = {
    50,100
 };
 ```



We already have the function call syntax instead:
```d
A a = A(50, 100); //Does the same as C++ {} syntax
```
But this syntax is ugly and bad when you have a struct with 10+ 
members. This is where named arguments done a change for me.

Also, as Paolo said, one thing does not exclude the other by the 
way.
Jan 11
parent zjh <fqbqrr 163.com> writes:
On Thursday, 11 January 2024 at 13:46:26 UTC, Hipreme wrote:


 ```d
 A a = A(50, 100); //Does the same as C++ {} syntax
 ```



VS:

```d
//C++:
A a{50, 100};
```


`a struct with 10+ members` is not a good `design`!
Jan 11
prev sibling parent reply Bradley Chatha <sealabjaster gmail.com> writes:
On Thursday, 11 January 2024 at 13:29:30 UTC, zjh wrote:

 D needs a `global task list` and arranges for people to execute 
 it.
 What D needs is `organization`, and it is a `strong 
 organization`!


We used to have something similar (but not exactly the same) as 
this via the leadership's vision documents: 
https://wiki.dlang.org/Vision_statements

It'd contain things like a wishlist of features, "champions 
needed" tasks, etc.

My favourite is the 2016H2 document, because of the Phobos 
wishlist:

```
     Eliminate Phobos dependency on GC (in other words, make GC 
opt out a viable and simple option).
     Review all Phobos modules for compatibility with ranges - 
std.zip, for example, was done before ranges and does not work 
with them
     Replace modules that are lacking in quality: json, xml
     Strengthen definition of ranges, possibly improve API
     New modules such as: containers, SI units, rational numbers, 
fixed-point numbers, big decimals, unit testing framework, 
benchmarking, linear algebra.
     Use -cov to improve code coverage of Phobos modules
     Make sure every function in Phobos has an example
     Make sure every function in Phobos has Params: and Returns: 
sections 
http://www.digitalmars.com/d/archives/digitalmars/D/Phobos_Documentation_-_call_to_action_258777.html
     Take inspiration from popular modules in other languages for 
adding modules to Phobos
     Improve green threads/fiber support
     Create a module that enables code to be run on GPUs
     Create the interface code to the C++ STL
     Review all of Phobos for  safe compatibility
     Remove dependency on autodecode from Phobos
```

(I wish I had a better skillset back then to try to help out :( )

Sadly these wishlists never really lead to much progress from 
what I remember, so they got semi-silently dropped.
Jan 11
parent reply zjh <fqbqrr 163.com> writes:
On Thursday, 11 January 2024 at 14:51:21 UTC, Bradley Chatha 
wrote:


 My favourite is the 2016H2 document, because of the Phobos 
 wishlist:

 ```
     Eliminate Phobos dependency on GC (in other words, make GC 
 opt out a viable and simple option).
     Review all Phobos modules for compatibility with ranges - 
 std.zip, for example, was done before ranges and does not work 
 with them
    ...



The D leadership should `refine` these visions instead of 
focusing on `specific details`!

`Specific details` can be discussed separately,

But shouldn't the `D leadership` be more important in 
`distributing tasks` and `refining the vision`?

D leadership, although can focus on specific technical aspects, 
what is more important is to `refine the tasks, refine the 
vision, and then delegate the tasks to the corresponding 
personnel`!

Instead of focusing too much on technology

The D leadership should focus on `explaining the structure of the 
'DMD' code, making it easier for others to modify`, distributing 
tasks, refining tasks, and `supervising tasks` while `addressing 
technical difficulties`.
Jan 11
parent zjh <fqbqrr 163.com> writes:
On Thursday, 11 January 2024 at 15:45:55 UTC, zjh wrote:


 The D leadership should focus on `explaining the structure of 
 the 'DMD' code, making it easier for others to modify`, 
 distributing tasks, refining tasks, and `supervising tasks` 
 while `addressing technical difficulties`.



`Task List+Implementation+Schedule`!
Just like `engineering management`, all `D` need is a `task 
schedule`.
Jan 11
prev sibling next sibling parent Guillaume Piolat <first.name gmail.com> writes:
Disagree on that one:

On Thursday, 11 January 2024 at 13:07:52 UTC, deadalnix wrote:

  - D runtime is unable to see thread started manually (for 
 instance with pthread-create) leading to all kind of bizarre 
 behavior.


Well this is working as intended and needed here in shared libs. 
Registering and deregistering threads endlessly in shared 
libraries cost time. druntime cannot be used in full everywhere 
so at least provide escape hatch. Either use Thread or register 
your thread to the druntime. The common misconception is that you 
could just register threads once in a shared library but in 
reality when the threads dies elsewhere the GC will scan memory 
that doesn't exist anymore.
Jan 11
prev sibling next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Thursday, 11 January 2024 at 13:07:52 UTC, deadalnix wrote:

 [snip]
 The main problem people face with D in the real world are 
 almost exclusively of the implementation kind. The list is 
 endless (and yes, there are many bugs reports about these 
 things). I recently made a post about how the OOP 
 implementation is extremely sub-par vs what people in OOP 
 languages would expect. no change of the language required to 
 fix. See here: 
 https://forum.dlang.org/post/hteuczyclxajakrisxjd forum.dlang.org

 But if you are not convinced, here are a few more example of 
 thing being implemented wrong or existing feature not working 
 right:
  - D runtime is unable to see thread started manually (for 
 instance with pthread-create) leading to all kind of bizarre 
 behavior.
  - Template symbols are generated as weak, which prevents 
 inlining (!).
  - Pretty much no cross module inlining, making helper function 
 absurdly costly.
  - scope(success) generates exception handling code.
  - D goes virtual by default for class methods, but LTO is 
 unable to finalize (contrary to every other languages going 
 virtual by default).
  - The GC implementation is nowhere close to where it needs to 
 be.
  - in contracts are dynamically bound (and in the callee) 
 instead of statically bounds and in the caller.

 


Discussing new features is kind of the language equivalent of 
bikeshedding. I don't have anywhere near the skills needed to fix 
any of those issues, but I could vaguely form an opinion on 
whether I like new feature X.

I think everyone has their own explanation for why D isn't 
catching on as much as people would hope. These aren't my 
reasons, but I wouldn't be surprised if there are some people who 
are turned away by these issues.

But your list is very much in line with one of Walter's recent 
posts on the other thread where he mentions how much time he 
spends working on things that no one else is working on. Some of 
these issues are undoubtedly hard to fix and not many have the 
expertise to fix them. The response from those sympathetic to the 
fork might be something along the lines that the burden often 
falls on Walter because some contributors leave. Its a tricky 
organizational problem. It's hard to have high quality/standards 
with a lot of contributors.
Jan 11
next sibling parent reply zjh <fqbqrr 163.com> writes:
On Thursday, 11 January 2024 at 14:34:57 UTC, jmh530 wrote:
Its a

 tricky organizational problem. It's hard to have high 
 quality/standards with a lot of contributors.


This is an organizational issue, how to arrange `manpower, time, 
and tasks`?
Jan 11
next sibling parent reply bachmeier <no spam.net> writes:
On Thursday, 11 January 2024 at 14:47:16 UTC, zjh wrote:

 On Thursday, 11 January 2024 at 14:34:57 UTC, jmh530 wrote:
 Its a

 tricky organizational problem. It's hard to have high 
 quality/standards with a lot of contributors.


 This is an organizational issue, how to arrange `manpower, 
 time, and tasks`?


Not so much. It's an ownership issue. You have to tell someone 
that they get to choose how to do the work. If it meets certain 
criteria, such as passing a set of tests specified in advance, it 
gets merged.

Very few people are willing to spend hours of their free time 
doing work, only to have it rejected because it's not the way the 
person in charge would have done it. I sure wouldn't. Especially 
considering that the criteria for rejection are arbitrary and 
revealed only after the fact.

If the folks at the very top of the organization want to maintain 
ownership of everything, the only work that gets done is the work 
they do.
Jan 11
parent "H. S. Teoh" <hsteoh qfbox.info> writes:
On Thu, Jan 11, 2024 at 04:13:07PM +0000, bachmeier via Digitalmars-d wrote:

 On Thursday, 11 January 2024 at 14:47:16 UTC, zjh wrote:

 On Thursday, 11 January 2024 at 14:34:57 UTC, jmh530 wrote:

 Its a tricky organizational problem. It's hard to have high
 quality/standards with a lot of contributors.


 
 This is an organizational issue, how to arrange `manpower, time, and
 tasks`?


 
 Not so much. It's an ownership issue. You have to tell someone that
 they get to choose how to do the work. If it meets certain criteria,
 such as passing a set of tests specified in advance, it gets merged.


Exactly. The core problem is not technical, it is social.  We have no
problem with technical issues -- Walter is an expert at that, no doubt
about it, there's no problem on that front.  The social side has been a
problem since day 1, and it's clear by now that this is not going to
change.  So this problem will persist.

This project is open source in the sense of the license for the code,
but in terms of development and management it is absolutely run like a
closed-source, proprietary project.  I don't say whether this is good or
bad -- arguments can be made for both -- but as far as community
participation and retaining contributors is concerned, we're dreaming if
we think it will somehow magically work out.



 Very few people are willing to spend hours of their free time doing
 work, only to have it rejected because it's not the way the person in
 charge would have done it. I sure wouldn't. Especially considering
 that the criteria for rejection are arbitrary and revealed only after
 the fact.

 If the folks at the very top of the organization want to maintain
 ownership of everything, the only work that gets done is the work they
 do.


Yep.  D's history proves this beyond any doubt.


T

-- 
There's light at the end of the tunnel. It's the oncoming train.
Jan 11
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2024 6:47 AM, zjh wrote:

 On Thursday, 11 January 2024 at 14:34:57 UTC, jmh530 wrote:
 Its a

 tricky organizational problem. It's hard to have high quality/standards with a 
 lot of contributors.


 
 This is an organizational issue, how to arrange `manpower, time, and tasks`?



It's a repeating occurrence that people often ask me what they can work on. I 
provide a list of A,B,C,D that would be of great benefit. They do Q, because Q 
interests them more.

In order to assign tasks, one needs to actually employ people, as I cannot 
assign anything to volunteers.

(Some of our volunteers do do things that aren't much fun, but need to be
worked 
on. Those are our most valued contributors.)
Jan 11
next sibling parent reply zjh <fqbqrr 163.com> writes:
On Thursday, 11 January 2024 at 19:53:10 UTC, Walter Bright wrote:


 In order to assign tasks, one needs to actually employ people, 
 as I cannot assign anything to volunteers.



You don't need to `hire` anyone, you just need to carefully 
explain the `structure` of the `'dmd'` code, and then list 
`countless tasks`. Similarly, `break down` the big tasks into 
`small ones`. Create a list that `every user` can see, and then 
create an `incentive mechanism`!

`Break down` big tasks into small ones, and then let users solve 
them `one by one`. The difficulties are handed over to 
`management`, and the details are handed over to `ordinary users`.

`Encourage` users to solve practical problems related to 
themselves and tell them `where` to solve them. Users are like 
`writing plugins`.

What the user needs is, where can I solve my `small problem`? If 
possible, the user can solve it themselves! It is very important 
for users to `understand the compiler`!
Jan 11
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2024 4:57 PM, zjh wrote:

 You don't need to `hire` anyone, you just need to carefully explain the 
 `structure` of the `'dmd'` code, and then list `countless tasks`. Similarly, 
 `break down` the big tasks into `small ones`. Create a list that `every user` 
 can see, and then create an `incentive mechanism`!



We do have a task list:

https://issues.dlang.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bugidtype=include&list_id=246714&order=Bug%20Number&query_format=advanced
Jan 11
parent reply Monkyyy <crazymonkyyy gmail.com> writes:
On Friday, 12 January 2024 at 05:22:16 UTC, Walter Bright wrote:

 On 1/11/2024 4:57 PM, zjh wrote:

 You don't need to `hire` anyone, you just need to carefully 
 explain the `structure` of the `'dmd'` code, and then list 
 `countless tasks`. Similarly, `break down` the big tasks into 
 `small ones`. Create a list that `every user` can see, and 
 then create an `incentive mechanism`!



 We do have a task list:

 https://issues.dlang.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bugidtype=include&list_id=246714&order=Bug%20Number&query_format=advanced


Ive seen poeple say a todo list with 10 items lack priorities, 
the bug list is very very very much longer.
Jan 12
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/12/2024 3:02 AM, Monkyyy wrote:

 Ive seen poeple say a todo list with 10 items lack priorities, the bug list is 
 very very very much longer.


The buglist is indeed very long. But it can be sliced and diced into subsets in 
many ways, such as by content, by severity, by keyword, etc.

It's not necessary to read it all, just read till you find something that you 
want to fix.
Jan 12
parent monkyyy <crazymonkyyy gmail.com> writes:
On Friday, 12 January 2024 at 20:49:36 UTC, Walter Bright wrote:

 On 1/12/2024 3:02 AM, Monkyyy wrote:

 Ive seen poeple say a todo list with 10 items lack priorities, 
 the bug list is very very very much longer.


 The buglist is indeed very long. But it can be sliced and diced 
 into subsets in many ways, such as by content, by severity, by 
 keyword, etc.

 It's not necessary to read it all, just read till you find 
 something that you want to fix.


that will obviously lead to the wack-a-mole people complain about 
and will make matching your bug report to whatever search query 
you use be this silly game
Jan 12
prev sibling parent reply zjh <fqbqrr 163.com> writes:
On Thursday, 11 January 2024 at 19:53:10 UTC, Walter Bright wrote:


 (Some of our volunteers do do things that aren't much fun, but 
 need to be worked on. Those are our most valued contributors.)



Now there is a `discussion group` on GitHub, which can be turned 
into a `task list`. Each discussion is a `task`.

What users need is a `'Q'`, so you can encourage them to 
establish a `'Q'` discussion, and then `interested users` can 
organize their own discussions!

And, we should encourage `discussion` and merge `features`!

A very simple example is a user, like me, who really wants a `C++ 
class level private`. This' private 'can be achieved by adding a 
`keyword`, you only need to add one keyword, and then implement 
the corresponding function. Then, you get a bunch of `potential 
C++ users`. But `D` team refuse this `pr` , the user leaves. 
Originally, it could bring a bunch of its friends to enter D. In 
the end, D loses all and has nothing. There is `no function`, 
There are `no D users`.
Jan 11
next sibling parent zjh <fqbqrr 163.com> writes:
On Friday, 12 January 2024 at 01:11:44 UTC, zjh wrote:


 A very simple example is a user, like me, who really wants a 
 `C++ class level private`.


Don't nitpick about `user needs` anymore, users are the `most 
important` part of the D ecosystem. Without users, D is useless.
Jan 11
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2024 5:11 PM, zjh wrote:

 A very simple example is a user, like me, who really wants a `C++ class level 
 private`. This' private 'can be achieved by adding a `keyword`, you only need
to 
 add one keyword, and then implement the corresponding function. Then, you get
a 
 bunch of `potential C++ users`. But `D` team refuse this `pr` , the user
leaves. 
 Originally, it could bring a bunch of its friends to enter D. In the end, D 
 loses all and has nothing. There is `no function`, There are `no D users`.


What is the criteria for if a feature should be merged or not? I cannot think
of 
one where somebody isn't going to be unhappy about the result.

For example, there are D users who want the GC removed from the language, and 
another group that want D focused on the GC. We have attempted to embrace both 
sets of users with BetterC, but there are people still unhappy with that.

It reminds me of a story my dad told me. He talked to the mayor of the small 
town he lived in, and asked the mayor "what's your biggest problem in managing 
the town?" The mayor replied "there are two groups of people - those who like 
dogs, and those who do not. They are each roughly half of the population. There 
is no reconciliation between those two groups. It's constant friction."
Jan 14
parent reply zjh <fqbqrr 163.com> writes:
On Sunday, 14 January 2024 at 21:39:33 UTC, Walter Bright wrote:


 What is the criteria for if a feature should be merged or not? 
 I cannot think of one where somebody isn't going to be unhappy 
 about the result.




The standard is very simple: whether to `increase users `and 
whether to `make users happy`! Adding a simple keyword like this 
can attract `a large number of C++users`, which is `net profit`!

On the contrary, you did not add this feature, you lost all! `The 
users` has left, and D has not `improved` either!

Perhaps it's not important to you, but for those who are 
accustomed to C++, this is a `very important thing`. At the class 
level, you don't want other builds in the same module to access 
it, which is a `basic requirement`.

Perhaps you are not used to it, but users have already become 
accustomed to it
This is `basic class level encapsulation`!
For `C++users`, it's like `eating and drinking water`!


Now `'openD'` has taken away pure `'GC'` users, which is a good 
thing. We can fully focus on serving and competing with 
`'C++/rust'`!

You can definitely locate the target of `'D'` more accurately! 
Competing with `'rust'` and facilitating the writing of 
`'rust/C++'` wrappers should all be the goals of `'D'`!

The previous article about `'interfacing rust/C++'` was very 
good! In this way, although I don't use `'rust/C++'`, I can still 
make use of the ecosystem of `'rust/C++'`!

It's also good for users, don't have to `endure` the `ugly 
syntax` of `'rust'`! But you can enjoy the features of the 'rust' 
community! `Rust's is D's`!
Jan 14
next sibling parent reply Lance Bachmeier <no spam.net> writes:
On Monday, 15 January 2024 at 01:21:33 UTC, zjh wrote:

 On Sunday, 14 January 2024 at 21:39:33 UTC, Walter Bright wrote:


 What is the criteria for if a feature should be merged or not? 
 I cannot think of one where somebody isn't going to be unhappy 
 about the result.




 The standard is very simple: whether to `increase users `and 
 whether to `make users happy`! Adding a simple keyword like 
 this can attract `a large number of C++users`, which is `net 
 profit`!

 On the contrary, you did not add this feature, you lost all! 
 `The users` has left, and D has not `improved` either!


The fastest way to make users leave/avoid any language is to make 
it as complicated as C++. Not a single programmer will move to D 
because of this. They'll just say they already have that in C++.
Jan 14
parent reply zjh <fqbqrr 163.com> writes:
On Monday, 15 January 2024 at 01:32:42 UTC, Lance Bachmeier wrote:

 The fastest way to make users leave/avoid any language is to 
 make it as complicated as C++. Not a single programmer will 
 move to D because of this. They'll just say they already have 
 that in C++.




You're wrong, things themselves are `complex`. The `complexity` 
of `C++` is used to solve `specific problems`. Using complexity 
to solve complex problems, makes it easier for `users`!

You think it's `complicated`, that's because things `themselves` 
are already complex!

The problem with `C++` is that `the response` is too slow! Many 
`features` are already available in other languages, but the 
problem with `'D'` is its `inaccurate positioning`. If you need 
`'GC'` now, I recommend them to go to `'openD'`.

And D focuses on the competition of `C++/rust`!
This way, the `target` is `smaller` and the `positioning` is more 
`accurate`!
Jan 14
parent reply Sergey <kornburn yandex.ru> writes:
On Monday, 15 January 2024 at 01:44:36 UTC, zjh wrote:

 And D focuses on the competition of `C++/rust`!
 This way, the `target` is `smaller` and the `positioning` is 
 more `accurate`!


For that you have a betterC.
And what do you mean by competing C++/rust?
Jan 14
parent zjh <fqbqrr 163.com> writes:
On Monday, 15 January 2024 at 01:50:55 UTC, Sergey wrote:


 For that you have a betterC.
 And what do you mean by competing C++/rust?



`Being` a `truely system language`!

In this way, you can even use `'opend'` in the upper `application 
layer` and `'d'` in the lower `system layer`, and `'d'` can also 
wrapps `'C++/rust/C/... '`,thus make use of their ecology.
Isn't this positioning `fragrant`?
Jan 14
prev sibling parent zjh <fqbqrr 163.com> writes:
On Monday, 15 January 2024 at 01:21:33 UTC, zjh wrote:


 The previous article about `'interfacing rust/C++'` was very 
 good!



[Here](https://forum.dlang.org/post/guencibxmfoaosurgjpg forum.dlang.org), such
man should be introduced into the `D core team`!
To encourage others to work hard to do a good job in the 
integration of `C++/Rust`!
The `current language competition` is not only about `language 
competition`, but also includes `ecology`, such `wrappers of 
C++/Rust`, which are very important for D!
Similarly, it indicates that any `features`  or `ecology` 
beneficial to D, will provide you with `good treatment` and 
increase `your voice` in the D community!
Jan 14
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 11 January 2024 at 14:34:57 UTC, jmh530 wrote:

 I think everyone has their own explanation for why D isn't 
 catching on as much as people would hope. These aren't my 
 reasons, but I wouldn't be surprised if there are some people 
 who are turned away by these issues.


Don't focus on the exact list of issues. It's indeed going to 
differs, but there is a pattern, and what matters is the pattern. 
These issue are all implementation problem with current features 
that we have now today. I could have added 10x more by spending a 
little more time. For instance, demangling support for various 
debuggers would be tremendously valuable.

Large D project suffers much more from poor or problematic 
implementation of current feature  than they do from the lack of 
this or that language gizmo.

In addition, improving these things is direct value added to 
users at little no no cost to the them. They get the same things 
as before, it just works better.

On the other hand, the cost of new feature - even desirable ones, 
is greatly underestimated by what I can only assume is a self 
selected crowd of people who use little tools when coding.
Jan 11
next sibling parent reply "H. S. Teoh" <hsteoh qfbox.info> writes:
On Thu, Jan 11, 2024 at 05:52:20PM +0000, deadalnix via Digitalmars-d wrote:

 On Thursday, 11 January 2024 at 14:34:57 UTC, jmh530 wrote:

 I think everyone has their own explanation for why D isn't catching
 on as much as people would hope. These aren't my reasons, but I
 wouldn't be surprised if there are some people who are turned away
 by these issues.
 


 
 Don't focus on the exact list of issues. It's indeed going to differs,
 but there is a pattern, and what matters is the pattern. These issue
 are all implementation problem with current features that we have now
 today.


[...]

I heard this story once, about a company that wanted to succeed and be
the best of the best.  So they hired only the best people -- you had to
be either the best in your area, or you're not hired.  With every
employee an expert, the company should succeed, right?

Unfortunately, what actually happened is that projects started to fail
and people starting quitting.  Why?  Because nobody was interested to
take care of the menial, but necessary tasks.  The experts found it too
boring to work on maintenance when they could be inventing new features.
So nobody took care of the basic stuff and projects were failing.  When
management realized this, they started assigning people to work on it.
That's when people started quitting -- the experts were hired to do the
interesting work, they did not want to deal with the boring stuff.
Eventually the company went the way of the dodo.

//

It's always more exciting to work on new features, to invent the next
thing that will revolutionize D.  Improving the quality of existing
features?  Too boring, too tedious, and totally unrewarding.  Guess what
gets done, and what doesn't.


T

-- 
The two rules of success: 1. Don't tell everything you know. -- YHL
Jan 11
parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 11 January 2024 at 18:20:29 UTC, H. S. Teoh wrote:

 It's always more exciting to work on new features, to invent 
 the next thing that will revolutionize D.  Improving the 
 quality of existing features?  Too boring, too tedious, and 
 totally unrewarding.  Guess what gets done, and what doesn't.


It looks like management need to start saying no. And do bad if 
some people quit. The alternative is the whole thing stagnate of 
sink.
Jan 11
parent reply "H. S. Teoh" <hsteoh qfbox.info> writes:
On Thu, Jan 11, 2024 at 06:24:46PM +0000, deadalnix via Digitalmars-d wrote:

 On Thursday, 11 January 2024 at 18:20:29 UTC, H. S. Teoh wrote:

 It's always more exciting to work on new features, to invent the
 next thing that will revolutionize D.  Improving the quality of
 existing features?  Too boring, too tedious, and totally
 unrewarding.  Guess what gets done, and what doesn't.
 


 
 It looks like management need to start saying no. And do bad if some
 people quit. The alternative is the whole thing stagnate of sink.


That's what the aforementioned company did, and they went under.

The root of the problem lies in their policy of hiring only the best
people, people who are not interested in doing the menial, but necessary
work.


T

-- 
In order to understand recursion you must first understand recursion.
Jan 11
parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 11 January 2024 at 18:39:29 UTC, H. S. Teoh wrote:

 On Thu, Jan 11, 2024 at 06:24:46PM +0000, deadalnix via 
 Digitalmars-d wrote:

 On Thursday, 11 January 2024 at 18:20:29 UTC, H. S. Teoh wrote:

 It's always more exciting to work on new features, to invent 
 the next thing that will revolutionize D.  Improving the 
 quality of existing features?  Too boring, too tedious, and 
 totally unrewarding.  Guess what gets done, and what doesn't.
 


 
 It looks like management need to start saying no. And do bad 
 if some people quit. The alternative is the whole thing 
 stagnate of sink.


 That's what the aforementioned company did, and they went under.

 The root of the problem lies in their policy of hiring only the 
 best people, people who are not interested in doing the menial, 
 but necessary work.


 T


These are clearly not the best people.
Jan 11
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2024 9:52 AM, deadalnix wrote:

 I could 
 have added 10x more by spending a little more time. For instance, demangling 
 support for various debuggers would be tremendously valuable.


I would appreciate you spending the time on that list. Eliminating all those 
little rocks in one's shoe is very worth while.

I can't do anything about them if I don't know what they are.

There is probably some low hanging fruit that we can easily take care of. We 
just have to know what they are!
Jan 11
prev sibling next sibling parent reply IGotD- <nise nise.com> writes:
On Thursday, 11 January 2024 at 13:07:52 UTC, deadalnix wrote:

  - D runtime is unable to see thread started manually (for 
 instance with pthread-create) leading to all kind of bizarre 
 behavior.


Threads that are manually created can be attached by using

ThreadT thread_attachThis_tpl(ThreadT)();
Jan 11
parent reply IGotD- <nise nise.com> writes:
On Thursday, 11 January 2024 at 14:54:47 UTC, IGotD- wrote:

 On Thursday, 11 January 2024 at 13:07:52 UTC, deadalnix wrote:

  - D runtime is unable to see thread started manually (for 
 instance with pthread-create) leading to all kind of bizarre 
 behavior.


 Threads that are manually created can be attached by using

 ThreadT thread_attachThis_tpl(ThreadT)();


Should be:

Thread thread_attachThis();
Jan 11
parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 11 January 2024 at 14:55:46 UTC, IGotD- wrote:

 On Thursday, 11 January 2024 at 14:54:47 UTC, IGotD- wrote:

 On Thursday, 11 January 2024 at 13:07:52 UTC, deadalnix wrote:

  - D runtime is unable to see thread started manually (for 
 instance with pthread-create) leading to all kind of bizarre 
 behavior.


 Threads that are manually created can be attached by using

 ThreadT thread_attachThis_tpl(ThreadT)();


 Should be:

 Thread thread_attachThis();


But that doesn't work. If a GC cycle kicks in in between the 
thread starting and you having the chance to attach it, you may 
end up collecting live objects. This is what killed the project, 
BTW.

Having thread that are not managed by the GC is a fine thing to 
want, but you got to do it the other way around by detaching the 
thread from the GC.
Jan 11
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2024 9:56 AM, deadalnix wrote:

 But that doesn't work. If a GC cycle kicks in in between the thread starting
and 
 you having the chance to attach it, you may end up collecting live objects.
This 
 is what killed the project, BTW.
 
 Having thread that are not managed by the GC is a fine thing to want, but you 
 got to do it the other way around by detaching the thread from the GC.


If you could pseudocode a solution, we can add it to the bugzilla issue. 
Threading issues are not something I have much expertise in.
Jan 11
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Thursday, 11 January 2024 at 19:58:43 UTC, Walter Bright wrote:

 On 1/11/2024 9:56 AM, deadalnix wrote:

 But that doesn't work. If a GC cycle kicks in in between the 
 thread starting and you having the chance to attach it, you 
 may end up collecting live objects. This is what killed the 
 project, BTW.
 
 Having thread that are not managed by the GC is a fine thing 
 to want, but you got to do it the other way around by 
 detaching the thread from the GC.


 If you could pseudocode a solution, we can add it to the 
 bugzilla issue. Threading issues are not something I have much 
 expertise in.


I've no ideas how you can have a list or external created 
pthreads without horrible hack like hooking pthread_create.

/P
Jan 11
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 11 January 2024 at 20:03:36 UTC, Paolo Invernizzi 
wrote:

 I've no ideas how you can have a list or external created 
 pthreads without horrible hack like hooking pthread_create.

 /P


Yes. This is a fairly standard stunt. boehm does it for instance. 
tsan as well.
Jan 11
prev sibling parent reply Guillaume Piolat <first.name gmail.com> writes:
On Thursday, 11 January 2024 at 20:03:36 UTC, Paolo Invernizzi 
wrote:

 I've no ideas how you can have a list or external created 
 pthreads without horrible hack like hooking pthread_create.



Ocaml is exactly like D with that: 
https://v2.ocaml.org/manual/intfc.html#ss:c-thread-register
Jan 11
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Friday, 12 January 2024 at 02:21:10 UTC, Guillaume Piolat 
wrote:

 On Thursday, 11 January 2024 at 20:03:36 UTC, Paolo Invernizzi 
 wrote:

 I've no ideas how you can have a list or external created 
 pthreads without horrible hack like hooking pthread_create.



 Ocaml is exactly like D with that: 
 https://v2.ocaml.org/manual/intfc.html#ss:c-thread-register


Yeah, I think every language is in the same D boat, if it doesn't 
hook pthread_create.
Jan 12
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 12/01/2024 10:11 PM, Paolo Invernizzi wrote:

 On Friday, 12 January 2024 at 02:21:10 UTC, Guillaume Piolat wrote:

 On Thursday, 11 January 2024 at 20:03:36 UTC, Paolo Invernizzi wrote:

 I've no ideas how you can have a list or external created pthreads 
 without horrible hack like hooking pthread_create.



 Ocaml is exactly like D with that: 
 https://v2.ocaml.org/manual/intfc.html#ss:c-thread-register


 
 Yeah, I think every language is in the same D boat, if it doesn't hook 
 pthread_create.


Yes.

I couldn't find a way to do it when I was building my thread abstraction 
for linux.

Real shame, Windows solution with the 4 functions of DllMain in PE-COFF 
is briliant.
Jan 12
prev sibling parent Guillaume Piolat <first.name gmail.com> writes:
On Thursday, 11 January 2024 at 17:56:51 UTC, deadalnix wrote:

 Having thread that are not managed by the GC is a fine thing to 
 want, but you got to do it the other way around by detaching 
 the thread from the GC.


No I need to do it the current way, by using system API to create 
threads like with any other language runtime. Also how would you 
create a thread in betterC else?

The rule for unattached threads differently is that their stack 
is not traced and unrealiable to stay alive. So either they just 
borrow GC memory while it's pointer elsewhere, either they don't 
use GC memory.
The issues arise when you let a unattached thread be the sole 
owner of GC memory created by a legitimate registered thread, and 
yes it's very awkward and unfamiliar but well if you were using 
Java you wouldn't expect pthread_create to register to the Java 
runtime.
Jan 11
prev sibling next sibling parent reply Dejan Lekic <dejan.lekic gmail.com> writes:
On Thursday, 11 January 2024 at 13:07:52 UTC, deadalnix wrote:

 Each time we take steps in that direction, D becomes a harder 
 sell.


Now watch this thread grow into oblivion with more than 100 
replies... 20+ years of reading D newsgroups/forum taught me that 
serious stuff should not be discussed here.
Jan 11
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 11 January 2024 at 17:08:50 UTC, Dejan Lekic wrote:

 On Thursday, 11 January 2024 at 13:07:52 UTC, deadalnix wrote:

 Each time we take steps in that direction, D becomes a harder 
 sell.


 Now watch this thread grow into oblivion with more than 100 
 replies... 20+ years of reading D newsgroups/forum taught me 
 that serious stuff should not be discussed here.


Unfortunately, you are correct.
Jan 11
prev sibling parent Elias (0xEAB) <desisma heidel.beer> writes:
On Thursday, 11 January 2024 at 17:08:50 UTC, Dejan Lekic wrote:

 Now watch this thread grow into oblivion with more than 100 
 replies...


I’m impressed. /s
Not even two weeks into 2024 and we’ve already got two threads on 
the newsgroup that could qualify as yet another “crybaby thread”.
Jan 12
prev sibling next sibling parent reply max haughton <maxhaton gmail.com> writes:
On Thursday, 11 January 2024 at 13:07:52 UTC, deadalnix wrote:

 I have been in the D community for a very long time. I have 
 seen D successfully deployed in companies, and the pain points 
 associated with it. I have seen D fails to catch on in 
 companies and why that is has well.

 Let me tell you, none of this has anything to do with feature D 
 has or does not have. At large, D has more features than most 
 languages


I don't think D is that big by modern standards (anymore at 
least), but anyway I agree. String interpolation however is one 
which I think passes the test; I argue this point below.

Also mentioned briefly is a note that the system (the way D as a 
piece of software is laid out, people can be surprisingly 
irrelevant in this sense) is biased towards shit outcomes, the 
convexity is not in the right direction.


  - Template symbols are generated as weak, which prevents 
 inlining (!).


Practically this just gets ignored with the exception of gdc 
assuming Iain kept that change, which can also be told to ignore 
it.


  - Pretty much no cross module inlining, making helper function 
 absurdly costly.


LDC can do that, no? Either with LTO or the 
-enable-cross-module-inlining flag.


  - The GC implementation is nowhere close to where it needs to 
 be.


This is indeed important, I agree. I am working on making the 
druntime test suite less brittle in part with the aim of being 
able to trust the "swap in your own GC" stuff more (a secondary 
interest but it is in my mind).


 There is nothing features like string interpolations or named 
 argument can bring to the table that could pay for the 
 implementations problem of existing feature. The cost benefit 
 analysis is just a big L for D: the fail to address the main 
 pain points, while causing massive breakage in the tooling 
 ecosystem (syntax highlighting support in 3rd party IDE, code 
 formatter, etc...), and it cost real time and resource to 
 upgrade these, or come at the cost of other quality of life 
 stuff nullifying their benefit (for instance, the quality of 
 syntax highlighting for D has degraded significantly in vim and 
 sublime text over the past few years).


Isn't this cost benefit like assuming a car company can't make 
the brakes better while also making the seats more comfortable?

A good GC might be the best part of half a million dollars worth 
of programmer-budget whereas (say) string interpolation is 
basically free i.e. its a tiny lowering right at the beginning of 
semantic analysis, not like adding dependent types - the 
complexity is not a fractal (whereas some changes mostly 
definitely are)

And on top of that provides real value - the patterns Adam's 
proposal enables are a genuine win all kinds off stuff beyond 
just format!"" style string interpolation i.e. the applications 
when talking to databases and so on are obvious but there is 
massive transformational potential all over the shop.

Javascript goes about it with a more abstract, but less conducive 
to introspection, but similar idea, even in a incohesive language 
with no type system it has great power i.e. was the difference 
between writing a simple function versus either writing or being 
a compiler for a recent project I did.

In the abstract sense I've never really cared that much about 
named arguments (i.e. rarely use them in languages that have it), 
but as a concrete example about tooling - the PR to enable named 
arguments supports in all the "official" tools was done literally 
years before the upstream work in the compiler to actually do the 
semantic analysis on them.



Most of the issues with tooling in this sense when things change 
is purely just brought about because things are too separated. 
Along those lines I am planning to try to merge in all the tools 
as they use the dmd frontend more and more. The current layout of 
dfmt et al, and the compiler/s themselves is fundamentally biased 
towards making this kind of change difficult, fix that and there 
would be a lot less to complain about.

Wrt to Vim and so on I've very rarely been using the syntax 
highlighting that comes with an editor so no idea on that front.

Is highlighting in sublime text worth halting any change? I 
assume its done lexically so if a token looks a different colour 
does it matter?

There is an LSP easily within reach of any sufficiently modern or 
sufficiently configurable editor, these are extremely popular and 
standardized now.
Jan 11
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 11 January 2024 at 17:45:09 UTC, max haughton wrote:

 [...]

 Isn't this cost benefit like assuming a car company can't make 
 the brakes better while also making the seats more comfortable?


Car companies pretty much never change the UI, even accross 
brand. Imagine a car manufacturer working and making acceleration 
happen by pressing a button on the steering wheel instead of 
using a pedal rather than work on more confy seats or better 
brake and you have a good analogy of what we are doing.

I'm sure there are a lot of reasons for making such a change, for 
instead people with no feet can no accelerate! Nevertheless, it's 
unlikely to go well for that company.


 A good GC might be the best part of half a million dollars 
 worth of programmer-budget whereas (say) string interpolation 
 is basically free i.e. its a tiny lowering right at the 
 beginning of semantic analysis, not like adding dependent types 
 - the complexity is not a fractal (whereas some changes mostly 
 definitely are)


I don't think so. Here is the problem we have, we don't read 
either the research papers on the topic or the implementation 
other have done. Building something new from scratch? Sure, you 
are right. Reading boehm, the dotnet runtime, tcmalloc and 
jemalloc, and reusing the techniques in there, not where close to 
that cost.

We don't have to reinvent the wheel.


 Most of the issues with tooling in this sense when things 
 change is purely just brought about because things are too 
 separated. Along those lines I am planning to try to merge in 
 all the tools as they use the dmd frontend more and more. The 
 current layout of dfmt et al, and the compiler/s themselves is 
 fundamentally biased towards making this kind of change 
 difficult, fix that and there would be a lot less to complain 
 about.


It will always be. All kind of people work on all kind of tools 
and most are willing to do a best effort to support D. However, 
that support is dwindling, when it should be growing. Time 
passing is playing in our favor there, we could just be cruising 
and accumulate support.

Most of these tools will not use the D frontend or whatever else 
to do that, for the simple reason that most of these tools will 
have their own idiosyncratic way of doing it. People are always 
going to come up with new idea about what tools can do, and we 
can benefit from that.

In fact, the gradual loss of support is a good indicator that the 
features don't pay for themselves. if they did, people would be 
upgrading. There are ac ouple of lever one can use to solve this:
1/ Reduce the cost of upgrade, for instance by delivering the new 
goodies in batch infrequently.
2/ Work on feature that deliver more value, which probably 
involve high quality implementation of said features, and on that 
front I bring you attention to the original post.


 Wrt to Vim and so on I've very rarely been using the syntax 
 highlighting that comes with an editor so no idea on that front.

 Is highlighting in sublime text worth halting any change? I 
 assume its done lexically so if a token looks a different 
 colour does it matter?


Vim and sublime are widely used text editors. Both probably have 
more users than D by at least an order of magnitude.


 There is an LSP easily within reach of any sufficiently modern 
 or sufficiently configurable editor, these are extremely 
 popular and standardized now.


Yes. Who maintain these for each editor? Nobody because cost > 
benefits. And for as long as this is the case, the ecosystem will 
shrink.
Jan 11
parent reply max haughton <maxhaton gmail.com> writes:
On Thursday, 11 January 2024 at 18:23:00 UTC, deadalnix wrote:


 Yes. Who maintain these for each editor? Nobody because cost > 
 benefits. And for as long as this is the case, the ecosystem 
 will shrink.


Its a protocol. You only need one; there is one 
https://github.com/Pure-D/serve-d
Jan 11
parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 11 January 2024 at 18:38:37 UTC, max haughton wrote:

 On Thursday, 11 January 2024 at 18:23:00 UTC, deadalnix wrote:


 Yes. Who maintain these for each editor? Nobody because cost > 
 benefits. And for as long as this is the case, the ecosystem 
 will shrink.


 Its a protocol. You only need one; there is one 
 https://github.com/Pure-D/serve-d


https://xkcd.com/927/
Jan 11
parent max haughton <maxhaton gmail.com> writes:
On Thursday, 11 January 2024 at 23:41:56 UTC, deadalnix wrote:

 On Thursday, 11 January 2024 at 18:38:37 UTC, max haughton 
 wrote:

 On Thursday, 11 January 2024 at 18:23:00 UTC, deadalnix wrote:


 Yes. Who maintain these for each editor? Nobody because cost

 benefits. And for as long as this is the case, the


 ecosystem will shrink.


 Its a protocol. You only need one; there is one 
 https://github.com/Pure-D/serve-d


 https://xkcd.com/927/


Which floating point standard does your computer use?

LSP is extremely widely implemented and supported by almost 
everything under the sun 
https://microsoft.github.io/language-server-protocol/implementors/servers/ -
other than nano and notepad every text editor in use by more than 2 people that
I can think of has it (which includes sublime)
Jan 11
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2024 9:45 AM, max haughton wrote:

 Most of the issues with tooling in this sense when things change is purely
just 
 brought about because things are too separated. Along those lines I am
planning 
 to try to merge in all the tools as they use the dmd frontend more and more.
The 
 current layout of dfmt et al, and the compiler/s themselves is fundamentally 
 biased towards making this kind of change difficult, fix that and there would
be 
 a lot less to complain about.


On my end, I have fixed the lexer/parser so they are independent of the rest of 
the compiler.

This is targeted towards enabling other tools to deal with D code.
Jan 11
prev sibling next sibling parent monkyyy <crazymonkyyy gmail.com> writes:
On Thursday, 11 January 2024 at 13:07:52 UTC, deadalnix wrote:

 There is nothing features like string interpolations or named 
 argument can bring to the table

 Each time we take steps in that direction, D becomes a harder 
 sell.


I would only want the simplest string interpolation but I imagine 
we are long past that point i"foo:$foo ($__LINE__)" if your are 
constructing strings your probably doing something slow(if not 
use binary data), debug related, or compatibility with some 
outside nonsense like making json or bash commands, where you 
guess and check. Like I wouldnt have let it be a debate for 5 
years about speed, template expansion, or adding math to it, etc.

but I believe named arguments + templates will make for very very 
nice api's no one else can manage. I think named arguments are 
the strongest evidence that d is moving forward at all(note: 
relative).
Jan 11
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2024 5:07 AM, deadalnix wrote:

 These are just simple thing that I have on top of my mind, but there are a ton 
 more.


Thanks for taking the time to write this list. I would like to see all of your 
issues, though! Let's make a list and then have a bugzilla entry for each of 
them, so we can start picking them off.


 such as  nogc, has been a productivity disaster


I don't really understand this. Just don't use  nogc?
Jan 11
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 1/11/24 20:33, Walter Bright wrote:

 On 1/11/2024 5:07 AM, deadalnix wrote:

 These are just simple thing that I have on top of my mind, but there 
 are a ton more.


 Thanks for taking the time to write this list. I would like to see all 
 of your issues, though! Let's make a list and then have a bugzilla entry 
 for each of them, so we can start picking them off.
 
  > such as  nogc, has been a productivity disaster
 
 I don't really understand this. Just don't use  nogc?


Perhaps a library you want to use is ` nogc` and you have to provide a 
callback, override a class method, or fork it to add a new feature you need.

Maybe you want to avoid implicit GC allocations in some part of the 
code, but now you have to mark the entire function ` nogc` to get the 
checking, and then you can no longer use exceptions in that function.

etc.
Jan 11
parent reply Dennis <dkorpel gmail.com> writes:
On Thursday, 11 January 2024 at 20:55:52 UTC, Timon Gehr wrote:

 On 1/11/24 20:33, Walter Bright wrote:
 Maybe you want to avoid implicit GC allocations in some part of 
 the code, but now you have to mark the entire function ` nogc` 
 to get the checking, and then you can no longer use exceptions 
 in that function.

 etc.


Considering the compiler doesn't use  nogc for anything that 
matters (unlike nothrow, which changes code generation), I wonder 
if we could degrade  nogc from a type attribute to a linting 
tool. It would still point out accidental array literals or 
closures, but not complain when calling a function that hasn't 
been proven to be  nogc.

It can still complain when calling a function that has been 
inferred  gc, and if you want to be GC free 100% certain, don't 
link it and the linker will complain about missing symbols.
Jan 11
next sibling parent reply DrDread <no no.no> writes:
On Thursday, 11 January 2024 at 21:06:08 UTC, Dennis wrote:

 On Thursday, 11 January 2024 at 20:55:52 UTC, Timon Gehr wrote:

 On 1/11/24 20:33, Walter Bright wrote:
 Maybe you want to avoid implicit GC allocations in some part 
 of the code, but now you have to mark the entire function 
 ` nogc` to get the checking, and then you can no longer use 
 exceptions in that function.

 etc.


 Considering the compiler doesn't use  nogc for anything that 
 matters (unlike nothrow, which changes code generation), I 
 wonder if we could degrade  nogc from a type attribute to a 
 linting tool. It would still point out accidental array 
 literals or closures, but not complain when calling a function 
 that hasn't been proven to be  nogc.

 It can still complain when calling a function that has been 
 inferred  gc, and if you want to be GC free 100% certain, don't 
 link it and the linker will complain about missing symbols.


this is discarding the use case that you want to mark only parts 
of your codebase nogc, but enforce it'S really nogc. we do use 
some realtime threads, but the rest off the app is  gc.
Jan 11
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2024 1:18 PM, DrDread wrote:

 this is discarding the use case that you want to mark only parts of your 
 codebase nogc, but enforce it'S really nogc. we do use some realtime threads, 
 but the rest off the app is  gc.


This subthread suggests to me that the problem with  nogc is it works.

It reminds me of a similar debate when we added `const` to the type system. 
Unlike C++, const in D is ruthlessly enforced. People wanted something called 
"logical const" where some construct pretends to be const when it was actually 
being mutated.

Logical const code cannot take advantage of it being const because it isn't 
constant.

I don't see much utility in a "logical  nogc" that allows using the gc anyway.

If you want to use the gc, don't use  nogc.
Jan 11
next sibling parent reply Adam Wilson <flyboynw gmail.com> writes:
On Thursday, 11 January 2024 at 21:37:01 UTC, Walter Bright wrote:

 On 1/11/2024 1:18 PM, DrDread wrote:

 this is discarding the use case that you want to mark only 
 parts of your codebase nogc, but enforce it'S really nogc. we 
 do use some realtime threads, but the rest off the app is  gc.


 This subthread suggests to me that the problem with  nogc is it 
 works.


 nogc works just fine. We recently spent a good chunk of time in 
Discord educating a newbie on what it *actually* does.

What  nogc is specified to do: Prevent GC allocations from 
occurring. Fantastic.
What people actually do with  nogc: Use it to selectively disable 
the GC without using GC.disable().

The reason for this stems from a side-effect of how the *current* 
GC operates. Because allocations are the trigger for collections, 
by preventing allocations, collections are also prevented. And 
what people really want to do is disable collections because they 
don't like the collection pauses. They don't *actually* care 
about the allocations per se because that is generally as fast as 
a malloc and they are going to have to allocate at some point 
anyways.

So  nogc works exactly as specified, but because of an 
unspecified implementation side-effect, that is not guaranteed to 
hold true in the future, the  nogc crowd writes their code as if 
 nogc does something else entirely.  And we end up here in this 
thread.
Jan 11
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2024 2:02 PM, Adam Wilson wrote:

  nogc works just fine. We recently spent a good chunk of time in Discord 
 educating a newbie on what it *actually* does.
 
 What  nogc is specified to do: Prevent GC allocations from occurring.
Fantastic.
 What people actually do with  nogc: Use it to selectively disable the GC
without 
 using GC.disable().
 
 The reason for this stems from a side-effect of how the *current* GC operates. 
 Because allocations are the trigger for collections, by preventing
allocations, 
 collections are also prevented. And what people really want to do is disable 
 collections because they don't like the collection pauses. They don't
*actually* 
 care about the allocations per se because that is generally as fast as a
malloc 
 and they are going to have to allocate at some point anyways.
 
 So  nogc works exactly as specified, but because of an unspecified 
 implementation side-effect, that is not guaranteed to hold true in the future, 
 the  nogc crowd writes their code as if  nogc does something else entirely. 
And 
 we end up here in this thread.


Sounds like better documentation is needed for both  nogc and GC.disable().

https://issues.dlang.org/show_bug.cgi?id=24331
Jan 11
prev sibling next sibling parent Siarhei Siamashka <siarhei.siamashka gmail.com> writes:
On Thursday, 11 January 2024 at 22:02:16 UTC, Adam Wilson wrote:

 [...]
 And what people really want to do is disable collections 
 because they don't like the collection pauses. They don't 
 *actually* care about the allocations per se because that is 
 generally as fast as a malloc


It is generally as *slow* as malloc.


 and they are going to have to allocate at some point anyways.


In many cases it's possible to write performance critical inner 
loops without any heap allocations at all. The ` nogc` attribute 
could work as a safeguard to ensure that *unnecessary* 
allocations are not happening if we don't expect them there. The 
compiler could be helpful and yet it isn't. That's your loss. And 
that's the reason why you have to deal with unhappy users.


 So  nogc works exactly as specified, but because of an 
 unspecified implementation side-effect, that is not guaranteed 
 to hold true in the future, the  nogc crowd writes their code 
 as if  nogc does something else entirely.  And we end up here 
 in this thread.


Please don't make any weird assumptions. I'm not telling you any 
new information and you are supposed to already know this since a 
very long time ago.
Jan 11
prev sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Thursday, January 11, 2024 3:02:16 PM MST Adam Wilson via Digitalmars-d 
wrote:

 On Thursday, 11 January 2024 at 21:37:01 UTC, Walter Bright wrote:

 On 1/11/2024 1:18 PM, DrDread wrote:

 this is discarding the use case that you want to mark only
 parts of your codebase nogc, but enforce it'S really nogc. we
 do use some realtime threads, but the rest off the app is  gc.


 This subthread suggests to me that the problem with  nogc is it
 works.


  nogc works just fine. We recently spent a good chunk of time in
 Discord educating a newbie on what it *actually* does.

 What  nogc is specified to do: Prevent GC allocations from
 occurring. Fantastic.
 What people actually do with  nogc: Use it to selectively disable
 the GC without using GC.disable().

 The reason for this stems from a side-effect of how the *current*
 GC operates. Because allocations are the trigger for collections,
 by preventing allocations, collections are also prevented. And
 what people really want to do is disable collections because they
 don't like the collection pauses. They don't *actually* care
 about the allocations per se because that is generally as fast as
 a malloc and they are going to have to allocate at some point
 anyways.

 So  nogc works exactly as specified, but because of an
 unspecified implementation side-effect, that is not guaranteed to
 hold true in the future, the  nogc crowd writes their code as if
  nogc does something else entirely.  And we end up here in this
 thread.


 nogc doesn't even prevent collections running while a function is called.
It just prevents that specific function from triggering the collection
(thanks to no allocations triggering it). Another thread could trigger a
collection while that function is running. So, yes, in a single-threaded
program, it's equivalent to calling GC.disable, but in the general case, it
isn't.

So, anyone actually relying on  nogc preventing collections is arguably
asking for trouble even with the current GC - though it seems like the kind
of folks who like to use  nogc are often the kind of folks who are likely to
avoid the GC entirely, so if  nogc is used everywhere in their code, then
they won't end up with either GC allocations or collections. However, anyone
using it selectively really can't rely on it to avoid collections unless
their program is single-threaded and will always remain so.

- Jonathan M Davis
Jan 11
next sibling parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 12/01/2024 12:09 PM, Jonathan M Davis wrote:

  nogc doesn't even prevent collections running while a function is 
 called. It just prevents that specific function from triggering the 
 collection (thanks to no allocations triggering it). Another thread 
 could trigger a collection while that function is running. So, yes, in a 
 single-threaded program, it's equivalent to calling GC.disable, but in 
 the general case, it isn't.
 
 
 So, anyone actually relying on  nogc preventing collections is arguably 
 asking for trouble even with the current GC - though it seems like the 
 kind of folks who like to use  nogc are often the kind of folks who are 
 likely to avoid the GC entirely, so if  nogc is used everywhere in their 
 code, then they won't end up with either GC allocations or collections. 
 However, anyone using it selectively really can't rely on it to avoid 
 collections unless their program is single-threaded and will always 
 remain so.


Yes.

It is fundamentally an attempt at informed consent at using the GC in 
selected code.

Which it does a terrible job at.
Jan 11
prev sibling next sibling parent reply Adam Wilson <flyboynw gmail.com> writes:
On Thursday, 11 January 2024 at 23:09:46 UTC, Jonathan M Davis 
wrote:

  nogc doesn't even prevent collections running while a function 
 is called. It just prevents that specific function from 
 triggering the collection (thanks to no allocations triggering 
 it). Another thread could trigger a collection while that 
 function is running. So, yes, in a single-threaded program, 
 it's equivalent to calling GC.disable, but in the general case, 
 it isn't.

 So, anyone actually relying on  nogc preventing collections is 
 arguably asking for trouble even with the current GC - though 
 it seems like the kind of folks who like to use  nogc are often 
 the kind of folks who are likely to avoid the GC entirely, so 
 if  nogc is used everywhere in their code, then they won't end 
 up with either GC allocations or collections. However, anyone 
 using it selectively really can't rely on it to avoid 
 collections unless their program is single-threaded and will 
 always remain so.

 - Jonathan M Davis


Indeed. I think that highly experienced D devs understand these 
subtle distinctions. But to the average dev it looks like an easy 
out. And IMO, this is the source of much of the angst over  nogc. 
The name implies that the GC is disabled during execution, and in 
single threaded programs it effectively is, but step outside of 
that box and the GC pause magically reappears despite  nogc being 
applied. So it *appears* to the average person that the language 
isn't doing as instructed. What people expect  nogc to do versus 
what it's specified to do are different things. But because 
people typically presume that their assumptions are correct they 
launch rants on the NGs before they bother to read the spec.

To some extent Rikki is right that  nogc might be better as a 
linter, because what it actually does is more in line with what a 
linter is expected to do.  nogc is a hygiene feature that can 
easily be misunderstood as doing something more than that.
Jan 11
parent reply claptrap <clap trap.com> writes:
On Friday, 12 January 2024 at 00:24:52 UTC, Adam Wilson wrote:

 On Thursday, 11 January 2024 at 23:09:46 UTC, Jonathan M Davis


 To some extent Rikki is right that  nogc might be better as a 
 linter, because what it actually does is more in line with what 
 a linter is expected to do.  nogc is a hygiene feature that can 
 easily be misunderstood as doing something more than that.


That might make some sense if all you're doing is trying to 
minimize GC usage for performance reasons. But if you are writing 
code that has real time requirements you absolutely need to be 
100% sure the GC is not called into. And if you are doing real 
time code, you'll know enough to make sure the GC doesnt even 
know about the real time thread, so other threads using GC wont 
matter. It probably wont anyway as it'll likely be an external 
callback.

So im not sure if it matters if average joe has a 
misunderstanding about what nogc does (im not sure they actually 
do either tbh), but the people who actually really need it have 
to understand it and a lot more on top.

So "average joe doesnt understand nogc therefor its useless" is a 
strawman IMO.



 Jan 12



 next sibling parent  Paolo Invernizzi <paolo.invernizzi gmail.com>  writes:


On Friday, 12 January 2024 at 11:04:09 UTC, claptrap wrote:

 On Friday, 12 January 2024 at 00:24:52 UTC, Adam Wilson wrote:

 On Thursday, 11 January 2024 at 23:09:46 UTC, Jonathan M Davis


 To some extent Rikki is right that  nogc might be better as a 
 linter, because what it actually does is more in line with 
 what a linter is expected to do.  nogc is a hygiene feature 
 that can easily be misunderstood as doing something more than 
 that.





 So "average joe doesnt understand nogc therefor its useless" is 
 a strawman IMO.


+1 !!!



 Jan 12



prev sibling  parent reply Adam Wilson <flyboynw gmail.com>  writes:


On Friday, 12 January 2024 at 11:04:09 UTC, claptrap wrote:

 So im not sure if it matters if average joe has a 
 misunderstanding about what nogc does (im not sure they 
 actually do either tbh), but the people who actually really 
 need it have to understand it and a lot more on top.

 So "average joe doesnt understand nogc therefor its useless" is 
 a strawman IMO.


If it doesn't matter, or they really do know, then please present 
an alternative theory as to why we keep having the same 
misunderstandings over  nogc.

Small problem, I didn't raise a strawman, you did. I never said 
it was useless. You decided that was what I meant to say and 
attacked that instead. I said that it's a hygiene feature that is 
frequently misunderstood and that it *could* be done as a linting 
item.



 Jan 12



  parent reply claptrap <clap trap.com>  writes:


On Friday, 12 January 2024 at 23:03:20 UTC, Adam Wilson wrote:

 On Friday, 12 January 2024 at 11:04:09 UTC, claptrap wrote:

 So im not sure if it matters if average joe has a 
 misunderstanding about what nogc does (im not sure they 
 actually do either tbh), but the people who actually really 
 need it have to understand it and a lot more on top.

 So "average joe doesnt understand nogc therefor its useless" 
 is a strawman IMO.


 If it doesn't matter, or they really do know, then please 
 present an alternative theory as to why we keep having the same 
 misunderstandings over  nogc.


I just went through 9 months of the learn group, i didn't find 
any posts regarding nogc being confusing, just few post on why 
certain functions can't be called with nogc.

I'm saying I dont see lots of people who are confused about nogc.



 Small problem, I didn't raise a strawman, you did. I never said 
 it was useless.


Yeah sorry, you're right.



 Jan 12



  parent reply Elias (0xEAB) <desisma heidel.beer>  writes:


On Saturday, 13 January 2024 at 00:22:25 UTC, claptrap wrote:

 I'm saying I dont see lots of people who are confused about 
 nogc.


I see people who confuse it with an imaginary ` noAlloc` on a 
regular basis.



 Jan 12



  parent reply claptrap <clap trap.com>  writes:


On Saturday, 13 January 2024 at 01:25:51 UTC, Elias (0xEAB) wrote:

 On Saturday, 13 January 2024 at 00:22:25 UTC, claptrap wrote:

 I'm saying I dont see lots of people who are confused about 
 nogc.


 I see people who confuse it with an imaginary ` noAlloc` on a 
 regular basis.


What is it that they expect it to do?



 Jan 13



  parent reply Elias (0xEAB) <desisma heidel.beer>  writes:


On Saturday, 13 January 2024 at 19:22:53 UTC, claptrap wrote:

 I'm saying I dont see lots of people who are confused about 
 nogc.


 I see people who confuse it with an imaginary ` noAlloc` on a 
 regular basis.


 What is it that they expect it to do?


Prevent allocations (e.g. in the hot paths of their code).



 Jan 13



 next sibling parent  Elias (0xEAB) <desisma heidel.beer>  writes:


On Sunday, 14 January 2024 at 05:08:07 UTC, Elias (0xEAB) wrote:

 On Saturday, 13 January 2024 at 19:22:53 UTC, claptrap wrote:

 What is it that they expect it to do?


 Prevent allocations (e.g. in the hot paths of their code).


Or prevent collections (which would be GC.disable(), in fact).



 Jan 13



prev sibling  parent  claptrap <clap trap.com>  writes:


On Sunday, 14 January 2024 at 05:08:07 UTC, Elias (0xEAB) wrote:

 On Saturday, 13 January 2024 at 19:22:53 UTC, claptrap wrote:

 I'm saying I dont see lots of people who are confused about 
 nogc.


 I see people who confuse it with an imaginary ` noAlloc` on a 
 regular basis.


 What is it that they expect it to do?


 Prevent allocations (e.g. in the hot paths of their code).

 or prevent collections (which would be GC.disable(), in fact).


So it's not that  nogc is hard to understand. It's that some 
newbies dont understand how GC works.

Maybe we should go through D.learn and find the top 3 features 
that confuse newbies and kill them. I wonder what they'd be?



 Jan 14



prev sibling  parent reply Siarhei Siamashka <siarhei.siamashka gmail.com>  writes:


On Thursday, 11 January 2024 at 23:09:46 UTC, Jonathan M Davis 
wrote:

  nogc doesn't even prevent collections running while a function 
 is called. It just prevents that specific function from 
 triggering the collection (thanks to no allocations triggering 
 it). Another thread could trigger a collection while that 
 function is running. So, yes, in a single-threaded program, 
 it's equivalent to calling GC.disable, but in the general case, 
 it isn't.


Or another thread could just call GC.enable/GC.collect and we are 
in a trouble either way, no matter what we do. Wouldn't it make 
sense to implement this other thread in a way that it properly 
cooperates with the main thread?


 So, anyone actually relying on  nogc preventing collections is 
 arguably asking for trouble even with the current GC - though 
 it seems like the kind of folks who like to use  nogc are often 
 the kind of folks who are likely to avoid the GC entirely, so 
 if  nogc is used everywhere in their code, then they won't end 
 up with either GC allocations or collections.


You are probably talking about the OS kernel developers. 
Something like PowerNex, d-virtio and maybe a few other seemingly 
inactive/dead D projects.

But a common scenario for games is to load resources taking 
advantage of the GC. And then try to avoid GC allocations in the 
main loop for performance reasons. Do D language maintainers 
refuse to acknowledge the existence of game developers? I think 
that there are a few of them in this forum.


 However, anyone using it selectively really can't rely on it to 
 avoid collections unless their program is single-threaded and 
 will always remain so.


What if the other threads are also ` nogc`?



 Jan 11



  parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz>  writes:


On 12/01/2024 2:36 PM, Siarhei Siamashka wrote:

     So, anyone actually relying on  nogc preventing collections is
     arguably asking for trouble even with the current GC - though it
     seems like the kind of folks who like to use  nogc are often the
     kind of folks who are likely to avoid the GC entirely, so if  nogc
     is used everywhere in their code, then they won't end up with either
     GC allocations or collections.
 
 You are probably talking about the OS kernel developers. Something like 
 PowerNex, d-virtio and maybe a few other seemingly inactive/dead D projects.
 
 But a common scenario for games is to load resources taking advantage of 
 the GC. And then try to avoid GC allocations in the main loop for 
 performance reasons. Do D language maintainers refuse to acknowledge the 
 existence of game developers? I think that there are a few of them in 
 this forum.


No, nobody is refusing such a clear use case. Especially when a few have 
been very vocal over the years and have had language features 
implemented for them.

What you are suggesting is that `` nogc`` should for each function, call 
into the GC to disable/enable it automatically.

This is slow. You need to be responsible for doing this.
It should not be automated.

We can only guarantee in the compiler that this set of branches in the 
call stack will not trigger the GC. It is not possible to do other 
(potentially unknown) branches across all threads.


     However, anyone using it selectively really can't rely on it to
     avoid collections unless their program is single-threaded and will
     always remain so.
 
 What if the other threads are also | nogc|?


"then they won't end up with either GC allocations or collections."



 Jan 11



  parent reply Siarhei Siamashka <siarhei.siamashka gmail.com>  writes:


On Friday, 12 January 2024 at 01:50:25 UTC, Richard (Rikki) 
Andrew Cattermole wrote:

 What you are suggesting is that `` nogc`` should for each 
 function, call into the GC to disable/enable it automatically.


I haven't suggested anything like this.

I'm in favor of something simple like a ` localnogc` attribute, 
though the details still need to be clarified to see how it 
applies to the actual ` nogc`-compatible libraries and benefits 
them. But Walter opposes this idea in principle: 
https://forum.dlang.org/post/unpn1j$1617$1 digitalmars.com ("I 
don't see much utility in a "logical  nogc" that allows using the 
gc anyway"). So no progress can be made without a fork. But the 
OpenD fork doesn't care about the ` nogc` use case.



 Jan 11



  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 1/12/24 03:27, Siarhei Siamashka wrote:

 But the OpenD fork doesn't care about the ` nogc` use case.


I don't think it's exactly like that, they mostly just dislike the 
pattern of rejecting new features for introducing implicit GC 
allocations. Anyway, personally I think something like  localnogc can be 
part of embracing the GC.



 Jan 11



prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 1/11/24 22:37, Walter Bright wrote:

 On 1/11/2024 1:18 PM, DrDread wrote:

 this is discarding the use case that you want to mark only parts of 
 your codebase nogc, but enforce it'S really nogc. we do use some 
 realtime threads, but the rest off the app is  gc.


 
 This subthread suggests to me that the problem with  nogc is it works.
 ...


It does not work. There are two options:

- Mark the function and callback ` nogc`, then GC users cannot use your 
library.
- Mark the function and callback not ` nogc`, then ` nogc` users cannot 
use your library.

There are two options and both are wrong. There is no right answer 
except to accept that ` nogc` actually does not work for library 
writers, and neither do the other attributes.


 It reminds me of a similar debate when we added `const` to the type 
 system. Unlike C++, const in D is ruthlessly enforced. People wanted 
 something called "logical const" where some construct pretends to be 
 const when it was actually being mutated.
 
 Logical const code cannot take advantage of it being const because it 
 isn't constant.
 
 I don't see much utility in a "logical  nogc" that allows using the gc 
 anyway.
 
 If you want to use the gc, don't use  nogc.


I do not use ` nogc`. As a result some of my code is liable to memory 
leaks due to bad escape analysis and I am having a productivity headache 
when I have to interface with ` nogc` library code. So not using ` nogc` 
is also bad. I use `const` only for POD data.



 Jan 11



 next sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz>  writes:


On 12/01/2024 11:17 AM, Timon Gehr wrote:

 On 1/11/24 22:37, Walter Bright wrote:
 
     On 1/11/2024 1:18 PM, DrDread wrote:
 
         this is discarding the use case that you want to mark only parts
         of your codebase nogc, but enforce it'S really nogc. we do use
         some realtime threads, but the rest off the app is  gc.
 
     This subthread suggests to me that the problem with  nogc is it
     works. ...
 
 It does not work. There are two options:
 
   * Mark the function and callback | nogc|, then GC users cannot use
     your library.
   * Mark the function and callback not | nogc|, then | nogc| users
     cannot use your library.


Indeed, what I want is contract invalidation.

```d
void func(void delegate()  safe  nogc del)  safe  nogc {
	del();
}
```

If we pass in `` nogc`` delegate, it works as is.

If we don't the caller sees:

```d
void func(void delegate()  system del)  system {
	del();
}
```

It's `` system`` simply because the delegate is not ``scope`` and 
therefore can escape, otherwise with DIP1000 and ``scope`` would still 
be `` safe``.

This would not replace attribute inference. But instead help interfacing 
code that needs those guarantees versus those that don't.



 Jan 11



  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 1/11/24 23:24, Richard (Rikki) Andrew Cattermole wrote:

 
 On 12/01/2024 11:17 AM, Timon Gehr wrote:

 On 1/11/24 22:37, Walter Bright wrote:

     On 1/11/2024 1:18 PM, DrDread wrote:

         this is discarding the use case that you want to mark only parts
         of your codebase nogc, but enforce it'S really nogc. we do use
         some realtime threads, but the rest off the app is  gc.

     This subthread suggests to me that the problem with  nogc is it
     works. ...

 It does not work. There are two options:

   * Mark the function and callback | nogc|, then GC users cannot use
     your library.
   * Mark the function and callback not | nogc|, then | nogc| users
     cannot use your library.


 
 Indeed, what I want is contract invalidation.
 
 ```d
 void func(void delegate()  safe  nogc del)  safe  nogc {
      del();
 }
 ```
 
 If we pass in `` nogc`` delegate, it works as is.
 
 If we don't the caller sees:
 
 ```d
 void func(void delegate()  system del)  system {
      del();
 }
 ```
 
 It's `` system`` simply because the delegate is not ``scope`` and 
 therefore can escape, otherwise with DIP1000 and ``scope`` would still 
 be `` safe``.
 
 This would not replace attribute inference. But instead help interfacing 
 code that needs those guarantees versus those that don't.


If you do that D can no longer correctly implement a higher-order 
function that forms the composed function from its two arguments. It 
looks good on paper for a few minutes, but this is not a good solution 
to apply by default.



 Jan 11



  parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz>  writes:


On 12/01/2024 11:33 AM, Timon Gehr wrote:

 If you do that D can no longer correctly implement a higher-order 
 function that forms the composed function from its two arguments. It 
 looks good on paper for a few minutes, but this is not a good solution 
 to apply by default.


Unless we get something different that solves the same premise, opt-in 
is fine.

I'm thinking explicitly for things like ``opApply``. Having giant mixin 
templates that handle all combinations is far worse.



 Jan 11



  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 1/11/24 23:37, Richard (Rikki) Andrew Cattermole wrote:

 On 12/01/2024 11:33 AM, Timon Gehr wrote:

 If you do that D can no longer correctly implement a higher-order 
 function that forms the composed function from its two arguments. It 
 looks good on paper for a few minutes, but this is not a good solution 
 to apply by default.


 
 Unless we get something different that solves the same premise, opt-in 
 is fine.
 
 I'm thinking explicitly for things like ``opApply``. Having giant mixin 
 templates that handle all combinations is far worse.


Yes, attribute polymorphism is needed for that case, but currently there 
is no way to abstract over attributes and this is invariably the point 
where language features become a bit too esoteric for people (both 
language designers and users) who are not type system experts and then 
ad-hoc hacks proliferate.



 Jan 11



  parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz>  writes:


On 12/01/2024 11:56 AM, Timon Gehr wrote:

 On 1/11/24 23:37, Richard (Rikki) Andrew Cattermole wrote:

 On 12/01/2024 11:33 AM, Timon Gehr wrote:

 If you do that D can no longer correctly implement a higher-order 
 function that forms the composed function from its two arguments. It 
 looks good on paper for a few minutes, but this is not a good 
 solution to apply by default.


 Unless we get something different that solves the same premise, opt-in 
 is fine.

 I'm thinking explicitly for things like ``opApply``. Having giant 
 mixin templates that handle all combinations is far worse.


 
 Yes, attribute polymorphism is needed for that case, but currently there 
 is no way to abstract over attributes and this is invariably the point 
 where language features become a bit too esoteric for people (both 
 language designers and users) who are not type system experts and then 
 ad-hoc hacks proliferate.


I am very open to a counter proposal that covers the points you have 
outlined.

Right now, contract invalidation + finely grained template solution for 
when you need more control is the bar to beat, at least for me.

Either way, right now this problem is dividing the community and causing 
significant issues with attributes in practice. It needs solving. It 
would be a major pressure release for this subject for many people.



 Jan 11



  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 1/12/24 00:04, Richard (Rikki) Andrew Cattermole wrote:

 On 12/01/2024 11:56 AM, Timon Gehr wrote:

 On 1/11/24 23:37, Richard (Rikki) Andrew Cattermole wrote:

 On 12/01/2024 11:33 AM, Timon Gehr wrote:

 If you do that D can no longer correctly implement a higher-order 
 function that forms the composed function from its two arguments. It 
 looks good on paper for a few minutes, but this is not a good 
 solution to apply by default.


 Unless we get something different that solves the same premise, 
 opt-in is fine.

 I'm thinking explicitly for things like ``opApply``. Having giant 
 mixin templates that handle all combinations is far worse.


 Yes, attribute polymorphism is needed for that case, but currently 
 there is no way to abstract over attributes and this is invariably the 
 point where language features become a bit too esoteric for people 
 (both language designers and users) who are not type system experts 
 and then ad-hoc hacks proliferate.


 
 I am very open to a counter proposal that covers the points you have 
 outlined.
 ...


The way this is usually handled is via generic parameters.

First, make attributes first class, so stuff like this just works:

alias nice=pure  nogc  safe nothrow;

void foo() nice{}

Then, you need generic parameters, I'll indicate them with `[]`:

```d
void opApply[functionAttribute a](scope int delegate()a dg)a{
     ....
}
```

Done. Here, everything in `[]` exists only at compile time (think of it 
as ghost variables), and there is only one `opApply` function at runtime.


 Right now, contract invalidation + finely grained template solution for 
 when you need more control is the bar to beat, at least for me.
 
 Either way, right now this problem is dividing the community and causing 
 significant issues with attributes in practice. It needs solving. It 
 would be a major pressure release for this subject for many people.
 


I am not saying don't do the ad-hoc hacks, but understand that this is 
what you are doing. `inout` was a decent hack, until you need to do 
something a bit more involved, e.g. `opApply`. Opt-in invalidation is a 
decent option, until you run into the same issue again at a slightly 
higher level of abstraction, necessitating the next hack. The mess that 
will be left behind will be more complicated than the proper solution 
would have been, even though now it may appears that what I am 
suggesting is more complicated.



 Jan 11



 next sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz>  writes:


On 12/01/2024 12:12 PM, Timon Gehr wrote:

 On 1/12/24 00:04, Richard (Rikki) Andrew Cattermole wrote:

 On 12/01/2024 11:56 AM, Timon Gehr wrote:

 On 1/11/24 23:37, Richard (Rikki) Andrew Cattermole wrote:

 On 12/01/2024 11:33 AM, Timon Gehr wrote:

 If you do that D can no longer correctly implement a higher-order 
 function that forms the composed function from its two arguments. 
 It looks good on paper for a few minutes, but this is not a good 
 solution to apply by default.


 Unless we get something different that solves the same premise, 
 opt-in is fine.

 I'm thinking explicitly for things like ``opApply``. Having giant 
 mixin templates that handle all combinations is far worse.


 Yes, attribute polymorphism is needed for that case, but currently 
 there is no way to abstract over attributes and this is invariably 
 the point where language features become a bit too esoteric for 
 people (both language designers and users) who are not type system 
 experts and then ad-hoc hacks proliferate.


 I am very open to a counter proposal that covers the points you have 
 outlined.
 ...


 
 The way this is usually handled is via generic parameters.
 
 First, make attributes first class, so stuff like this just works:
 
 alias nice=pure  nogc  safe nothrow;
 
 void foo() nice{}
 
 Then, you need generic parameters, I'll indicate them with `[]`:
 
 ```d
 void opApply[functionAttribute a](scope int delegate()a dg)a{
      ....
 }
 ```
 
 Done. Here, everything in `[]` exists only at compile time (think of it 
 as ghost variables), and there is only one `opApply` function at runtime.


What you have presented here is what I mean by template solution (minus 
the template).


 Right now, contract invalidation + finely grained template solution 
 for when you need more control is the bar to beat, at least for me.

 Either way, right now this problem is dividing the community and 
 causing significant issues with attributes in practice. It needs 
 solving. It would be a major pressure release for this subject for 
 many people.


 
 I am not saying don't do the ad-hoc hacks, but understand that this is 
 what you are doing. `inout` was a decent hack, until you need to do 
 something a bit more involved, e.g. `opApply`. Opt-in invalidation is a 
 decent option, until you run into the same issue again at a slightly 
 higher level of abstraction, necessitating the next hack. The mess that 
 will be left behind will be more complicated than the proper solution 
 would have been, even though now it may appears that what I am 
 suggesting is more complicated.


What I'm wondering now is if we can reframe the problem, so that 
contract invalidation is an application of the template solution, so 
that it can scale without further hacks.



 Jan 11



  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 1/12/24 00:18, Richard (Rikki) Andrew Cattermole wrote:

 
 On 12/01/2024 12:12 PM, Timon Gehr wrote:

 On 1/12/24 00:04, Richard (Rikki) Andrew Cattermole wrote:

 On 12/01/2024 11:56 AM, Timon Gehr wrote:

 On 1/11/24 23:37, Richard (Rikki) Andrew Cattermole wrote:

 On 12/01/2024 11:33 AM, Timon Gehr wrote:

 If you do that D can no longer correctly implement a higher-order 
 function that forms the composed function from its two arguments. 
 It looks good on paper for a few minutes, but this is not a good 
 solution to apply by default.


 Unless we get something different that solves the same premise, 
 opt-in is fine.

 I'm thinking explicitly for things like ``opApply``. Having giant 
 mixin templates that handle all combinations is far worse.


 Yes, attribute polymorphism is needed for that case, but currently 
 there is no way to abstract over attributes and this is invariably 
 the point where language features become a bit too esoteric for 
 people (both language designers and users) who are not type system 
 experts and then ad-hoc hacks proliferate.


 I am very open to a counter proposal that covers the points you have 
 outlined.
 ...


 The way this is usually handled is via generic parameters.

 First, make attributes first class, so stuff like this just works:

 alias nice=pure  nogc  safe nothrow;

 void foo() nice{}

 Then, you need generic parameters, I'll indicate them with `[]`:

 ```d
 int opApply[functionAttribute a](scope int delegate()a dg)a{
      ....
 }
 ```

 Done. Here, everything in `[]` exists only at compile time (think of 
 it as ghost variables), and there is only one `opApply` function at 
 runtime.


 
 What you have presented here is what I mean by template solution (minus 
 the template).
 

 Right now, contract invalidation + finely grained template solution 
 for when you need more control is the bar to beat, at least for me.

 Either way, right now this problem is dividing the community and 
 causing significant issues with attributes in practice. It needs 
 solving. It would be a major pressure release for this subject for 
 many people.


 I am not saying don't do the ad-hoc hacks, but understand that this is 
 what you are doing. `inout` was a decent hack, until you need to do 
 something a bit more involved, e.g. `opApply`. Opt-in invalidation is 
 a decent option, until you run into the same issue again at a slightly 
 higher level of abstraction, necessitating the next hack. The mess 
 that will be left behind will be more complicated than the proper 
 solution would have been, even though now it may appears that what I 
 am suggesting is more complicated.


 
 What I'm wondering now is if we can reframe the problem, so that 
 contract invalidation is an application of the template solution, so 
 that it can scale without further hacks.


Well, you can view

```d
int opApply( called scope int delegate() dg)b{ ... }
```

As syntactic sugar for:

```d
int opApply[functionAttribute __a37 :> b](scope int delegate()__a37 
dg)__a37{ ... }
```

Where `a :> b` means you can call a `b` function from an `a` context, 
i.e., `__a37` may weaken the guarantees given by `b` and allow function 
types that can type more values into the function.



 Jan 11



  parent  "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz>  writes:


On 12/01/2024 12:29 PM, Timon Gehr wrote:

     What I'm wondering now is if we can reframe the problem, so that
     contract invalidation is an application of the template solution, so
     that it can scale without further hacks.
 
 Well, you can view
 
 |int opApply( called scope int delegate() dg)b{ ... } |
 
 As syntactic sugar for:
 
 |int opApply[functionAttribute __a37 :> b](scope int delegate()__a37 
 dg)__a37{ ... } |
 
 Where |a :> b| means you can call a |b| function from an |a| context, 
 i.e., |__a37| may weaken the guarantees given by |b| and allow function 
 types that can type more values into the function.


Yes.

We'd still need to keep the ``scope`` stuff on the delegate wrt. 
`` safe`` as part of the weakening logic.

However I think the main thing is that the function body gets typed 
checked with the assumption that the most restrictive set holds. And 
only the caller sees that the contract has been invalidated in the error 
case.

Now we really only need to come up with a story that can be easily 
described to the user in both simple and complex cases, and describe it 
to the compiler also.



 Jan 11



prev sibling  parent reply zjh <fqbqrr 163.com>  writes:


On Thursday, 11 January 2024 at 23:12:36 UTC, Timon Gehr wrote:


 alias nice=pure  nogc  safe nothrow;

 void foo() nice{}

 Then, you need generic parameters, I'll indicate them with `[]`:

 ```d
 void opApply[functionAttribute a](scope int delegate()a dg)a{
     ....
 }
 ```



The attribute should be a `set`(making it easier to determine if 
it exists), and every common building block (function, 
enumeration, variable,etc.) can have one, and even all `building 
blocks` can have the same `set name`!

This way, all `desired constructs` can be extracted at compile 
time!



 Jan 11



  parent reply zjh <fqbqrr 163.com>  writes:


On Friday, 12 January 2024 at 01:42:42 UTC, zjh wrote:


```d
void opApply[functionAttribute a](scope int delegate()a dg)a{
     ....
}
//=>
void opApply[aset b](scope int delegate()a dg)a{
     ....
}
```

Here, `'b'` represents the set of attributes, and compare with 
the set of `a`'s attributes to indicate whether or `a` meet `b`.



 Jan 11



  parent  zjh <fqbqrr 163.com>  writes:


On Friday, 12 January 2024 at 01:49:34 UTC, zjh wrote:

 On Friday, 12 January 2024 at 01:42:42 UTC, zjh wrote:


 ```d



 void opApply[aset b](scope int delegate()a dg)a{
     ....
 }
 ```



Here, `'aset'` should be a '`predict`' and there should be a 
`'attribute'` function.



 Jan 11



prev sibling  parent  Jonathan M Davis <newsgroup.d jmdavisprog.com>  writes:


On Thursday, January 11, 2024 3:17:05 PM MST Timon Gehr via Digitalmars-d 
wrote:

 On 1/11/24 22:37, Walter Bright wrote:

 On 1/11/2024 1:18 PM, DrDread wrote:

 this is discarding the use case that you want to mark only parts of
 your codebase nogc, but enforce it'S really nogc. we do use some
 realtime threads, but the rest off the app is  gc.


 This subthread suggests to me that the problem with  nogc is it works.
 ...


 It does not work. There are two options:

 - Mark the function and callback ` nogc`, then GC users cannot use your
 library.
 - Mark the function and callback not ` nogc`, then ` nogc` users cannot
 use your library.

 There are two options and both are wrong. There is no right answer
 except to accept that ` nogc` actually does not work for library
 writers, and neither do the other attributes.


Yeah, to an extent, the problem can be solved with templates, but that only
works if the attributes of the library code depend entirely on the template
arguments, and it becomes _very_ easy to break other people's code when
changing the implementation by accidentally changing which attributes are
inferred.

Once the code isn't templated, you're either restricting what the library
can do with its implementation by using attributes on it which then prevent
you from being able to refactor the code in a way that doesn't work with
those attributes - or you're restricting what the user code can do with your
library, because they don't support one or more attributes that they
require. Either way, the existance of those attributes restricts what can be
done with your library regardless of which set of attributes you end up
using.

Another way to look at the problem is that many of these attributes are
effectively leaking the implementation details of your library. If you can
mark something as const or pure or  nogc, etc. then that says something
about what your library is currently doing. And because the code using your
library then potentially relies on those attributes being there, you then
can't change your implementation, because those details leaked through the
API.

- Jonathan M Davis



 Jan 11



prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 1/11/24 22:37, Walter Bright wrote:

 
 Logical const code cannot take advantage of it being const because it 
 isn't constant.


Well, people can and do do it.

https://issues.dlang.org/show_bug.cgi?id=9149

I have had to debate people on whether or not this is even a bug because 
they had been relying on the behavior in order to implement logical `const`.



 Jan 11



 next sibling parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 1/11/2024 2:32 PM, Timon Gehr wrote:

 On 1/11/24 22:37, Walter Bright wrote:

 Logical const code cannot take advantage of it being const because it isn't 
 constant.


 
 Well, people can and do do it.
 
 https://issues.dlang.org/show_bug.cgi?id=9149
 
 I have had to debate people on whether or not this is even a bug because they 
 had been relying on the behavior in order to implement logical `const`.


It is a bug, and as the issue shows, there is a solution.



 Jan 13



prev sibling  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 1/11/2024 2:32 PM, Timon Gehr wrote:

 https://issues.dlang.org/show_bug.cgi?id=9149



I added the `safe` keyword so anyone looking for safety issues can find it.



 Jan 13



prev sibling  parent reply Guillaume Piolat <first.name gmail.com>  writes:


On Thursday, 11 January 2024 at 21:37:01 UTC, Walter Bright wrote:

 This subthread suggests to me that the problem with  nogc is it 
 works.



 nogc could get an escape hatch maybe?
pure is especially annoying because no escape hatch.

Perhaps we can live with their UB in a per-attribute basis.
UB of  nogc is allocating  with GC, it only annoys in cases 
without GC, else well, type system was broken.

It's pretty nice to enforce statically no accidental allocation 
in a time critical callback, in short the reasons that made  nogc 
invented still apply.



 Jan 11



 next sibling parent  Paolo Invernizzi <paolo.invernizzi gmail.com>  writes:


On Thursday, 11 January 2024 at 22:49:51 UTC, Guillaume Piolat 
wrote:

 On Thursday, 11 January 2024 at 21:37:01 UTC, Walter Bright 
 wrote:

 This subthread suggests to me that the problem with  nogc is 
 it works.



  nogc could get an escape hatch maybe?
 pure is especially annoying because no escape hatch.

 Perhaps we can live with their UB in a per-attribute basis.
 UB of  nogc is allocating  with GC, it only annoys in cases 
 without GC, else well, type system was broken.

 It's pretty nice to enforce statically no accidental allocation 
 in a time critical callback, in short the reasons that made 
  nogc invented still apply.


+1, I was posting the same. If  nogc code _could_ call _explicit_ 
` gc` functions that could be a solution.



 Jan 11



prev sibling next sibling parent  Guillaume Piolat <first.name gmail.com>  writes:


On Thursday, 11 January 2024 at 22:49:51 UTC, Guillaume Piolat 
wrote:

  nogc could get an escape hatch maybe?
 pure is especially annoying because no escape hatch.

 Perhaps we can live with their UB in a per-attribute basis.
 UB of  nogc is allocating  with GC, it only annoys in cases 
 without GC, else well, type system was broken.

 It's pretty nice to enforce statically no accidental allocation 
 in a time critical callback, in short the reasons that made 
  nogc invented still apply.


I honestly would rather have nothrow and pure go than  nogc, 
which is going to be used on no-runtime, minimal runtimes (such 
as those that go in limited environments), runtimes with a GC 
that.

It's not avoiding the new that is difficult, it's the array 
literals, the accidental closures, and whatever also allocates 
quietly.



 Jan 11



prev sibling next sibling parent reply Nickolay Bukreyev <buknik95 ya.ru>  writes:


On Thursday, 11 January 2024 at 22:49:51 UTC, Guillaume Piolat 
wrote:

  nogc could get an escape hatch maybe?
 pure is especially annoying because no escape hatch.

 Perhaps we can live with their UB in a per-attribute basis.
 UB of  nogc is allocating  with GC, it only annoys in cases 
 without GC, else well, type system was broken.


There is an escape hatch for them already.

```d
void fakeNoGc(ref Appender!string)  nogc;

pragma(mangle, fakeNoGc.mangleof)
void fakeNoGcImpl(ref Appender!string app) {
     app ~= "text";
}

void test() {
     auto app = appender!string();
     app.reserve(8);
     ()  nogc {
         fakeNoGc(app); // If it actually reallocates, it's UB.
     }();
}
```

The compiler blindly trusts us that `fakeNoGc` won’t touch the 
GC. It does not check anything.

It’s a very dangerous pattern. Are you certain it needs a more 
convenient syntax?



 Jan 11



  parent  Guillaume Piolat <first.name gmail.com>  writes:


On Friday, 12 January 2024 at 03:38:07 UTC, Nickolay Bukreyev 
wrote:

 It’s a very dangerous pattern. Are you certain it needs a more 
 convenient syntax?


Yes we use that hack as a basis of our work? so that you can call 
class destructors (they may throw and use GC!).

In the same way you need  trusted to use  safe and  system in a 
nice and complete way, it's the same with  nogc and pure.



 Jan 12



prev sibling  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 1/11/2024 2:49 PM, Guillaume Piolat wrote:

 pure is especially annoying because no escape hatch.


There is. You take a pointer to the pure code, can forcibly cast it to an
impure 
pointer. However, that only works in  system code. There's a `pureMalloc` 
somewhere in the library that does this.

The same technique can be used to remove  nogc.

But when one subverts the type system like that, one must own the consequences.

BTW, `pure` is not enforced within `debug` blocks. This is handy so they can be 
instrumented with things like `printf`.



 Jan 13



prev sibling next sibling parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 1/11/24 22:06, Dennis wrote:

 On Thursday, 11 January 2024 at 20:55:52 UTC, Timon Gehr wrote:

 On 1/11/24 20:33, Walter Bright wrote:
 Maybe you want to avoid implicit GC allocations in some part of the 
 code, but now you have to mark the entire function ` nogc` to get the 
 checking, and then you can no longer use exceptions in that function.

 etc.


 
 Considering the compiler doesn't use  nogc for anything that matters 
 (unlike nothrow, which changes code generation), I wonder if we could 
 degrade  nogc from a type attribute to a linting tool. It would still 
 point out accidental array literals or closures, but not complain when 
 calling a function that hasn't been proven to be  nogc.
 ...


Maybe, but sometimes you really don't want any allocations, as they 
would lead to a memory leak. The exception case in this example is 
special exactly because of its implications on control flow.


 It can still complain when calling a function that has been inferred 
  gc,


Well, but then I can't allocate an exception again.


 and if you want to be GC free 100% certain, don't link it and the 
 linker will complain about missing symbols.


Personally, I want to use GC, perhaps just not in the innermost loop 
performing latency-critical operations behind a `GC.disable()` and not 
for the sole reason that the compiler's escape analysis was overly 
conservative.



 Jan 11



prev sibling  parent  "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz>  writes:


On 12/01/2024 10:06 AM, Dennis wrote:

 On Thursday, 11 January 2024 at 20:55:52 UTC, Timon Gehr wrote:

 On 1/11/24 20:33, Walter Bright wrote:
 Maybe you want to avoid implicit GC allocations in some part of the 
 code, but now you have to mark the entire function ` nogc` to get the 
 checking, and then you can no longer use exceptions in that function.

 etc.


 
 Considering the compiler doesn't use  nogc for anything that matters 
 (unlike nothrow, which changes code generation), I wonder if we could 
 degrade  nogc from a type attribute to a linting tool. It would still 
 point out accidental array literals or closures, but not complain when 
 calling a function that hasn't been proven to be  nogc.
 
 It can still complain when calling a function that has been inferred 
  gc, and if you want to be GC free 100% certain, don't link it and the 
 linker will complain about missing symbols.


 localnogc is what I want.

Check everything but function calls.

Easy to implement, and helps make sure if you care that a closure is 
stack allocated, it won't heap allocate.



 Jan 11



prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com>  writes:


On Thursday, January 11, 2024 12:33:48 PM MST Walter Bright via Digitalmars-d 
wrote:

 On 1/11/2024 5:07 AM, deadalnix wrote:

 These are just simple thing that I have on top of my mind, but there are a
 ton more.


 Thanks for taking the time to write this list. I would like to see all of
 your issues, though! Let's make a list and then have a bugzilla entry for
 each of them, so we can start picking them off.

  > such as  nogc, has been a productivity disaster

 I don't really understand this. Just don't use  nogc?


If he's talking about what I've discussed with him previously, the issue
really relates to attributes in general rather than  nogc specifically. The
big problem comes with refactoring code in a large codebase. The analogy
that John Colvin came up with for the issue is that putting attributes on
your codebase ends up being like putting concrete on it, making it
incredibly difficult to refactor anything.

For instance, because function doX requires some set of attributes,
everything it calls even indirectly then requires a compatible set of
attributes. So, when you go to make a change to anything that doX ends up
calling (even indirectly), you're then restricted by the (often arbitrary)
attribute requirements that doX has. And of course, since you're dealing
with a large codebase, it's not just one function that's the issue. It's a
whole bunch of functions, many of which call each other. So, making changes
that seem like they should be simple can then become incredibly difficult
because of all of the compilation errors you get related to attributes -
most of which you really don't care about in practice, but you need to make
the compiler happy.

Attributes in general can be useful in a fairly restricted set of code, but
once a lot of code is involved, they can become quite problematic - to the
point that we're trying to rip out many of the attributes on the codebase I
work on at Symmetry. And in some cases, the fact that some third party
library developer decided that a function should require a specific set of
attributes then becomes problematic when making code changes. Third party
code that we could use just fine before suddenly becomes problematic,
because we need to make a change to a type which in most languages would
have been localized and encapsulated, but attributes cause the changes to
cascade out all over the place and potentially make it so that we can't use
third party code that worked just fine before. And even if the problems are
in our own code, it becomes a time sink to figure out how to rework it all
so that it works with the changed attributes.

So, for instance, a piece of code could require that the types and functions
used with it be pure. The developer decided at the time that that's what
they thought would make sense with what they were trying to do, and the code
that was written at the time worked that way. But some time later, a change
needs to happen in some other part of the codebase, and something that was
pure before can no longer be pure. Suddenly, that part of the code that
required pure doesn't compile anymore, and the change could be in a
completely different part of the codebase to a type that just so happens to
end up being used (maybe even indirectly) by the code requiring pure. So,
suddenly, a bunch of code has to be refactored just because of an attribute
that really doesn't do much in practice. Realistically, no optimizations
were being done based on pure, and at best, they told the developers that
nothing like global variables were being used (which realistically weren't
being used anyway, because almost no one ever does that, because it's not
maintainable). But because some piece of code was doing something simple
that didn't work wtih pure but which would have been a perfectly fine and
small change in C++, a bunch of code has to be changed. So, the attribute
was likely adding zero value, and it just cost a ton of time to make changes
because of it.

If  nogc had been used, the situation would have largely been the same. Some
other attributes (like  safe) would also cause issues, but would be easier
to solve, because there are backdoors. But too often, refactoring the code
is then going to end up with stuff like  trusted being slapped on code just
to shut the compiler up. And for the ones without easy backdoors, casts will
sometimes end up being used to shut the compiler up - maybe with the idea
that it's a temporary solution, but obviously, that kind of thing can end up
sticking around long term. Depending on the situation and the attribute, it
could then be a time bomb in the making, or it could actually be fine. It's
certainly not great practice, but in practice, it's the sort of thing that
the language's design encourages with larger codebases, because it sometimes
becomes by far the simplest way to deal with a change in attributes.

Attributes in general simply don't seem to pay for themselves and come at
too high a cost once the codebase gets large enough. They often place what
are essentially arbitrary requirements on code which may have seemed
reasonable to the developer working on that code at the time but which can
become incredibly difficult to change later when you need to. And the
benefits that they provide are often minimal in practice, making the whole
thing that much worse.

I'm increasingly inclined to think that most attributes are primarily of
theoretical benefit rather than of actual, practical benefit, and even if
they are of practical benefit, if the codebase gets large enough, the way
that they make refactoring difficult tends to make their cost too high to be
worth it - and unfortunately, by the point that that becomes clear, you're
already using them all over the place and wasting a ton of time because of
decisions made months or years ago.

Often, attributes are used because they make sense for how a piece of code
is currently written, and they work with that version of the code. But as
the needs of the code change, the situation changes, and those attributes
might not work anymore, which can ultimately cost a _lot_ of time or even
make certain refactorings impossible, particularly as the amount of code
involved grows. Really, that's the biggest problem here. We can discuss
individual attributes and why they do or don't make sense (and  nogc is
particularly bad), but to an extent, _all_ attributes are a problem here.

If we were to be designing D from scratch, I would be arguing very strongly
that we should be much, _much_ pickier about the attributes that we have in
the language (e.g. I would argue strongly against both pure and  nogc), but
as it is, it's going to tend to mean that the advice for anyone working on a
codebase of any significant size is going to be to minimize the list of
attributes they use to those where there's going to be clear and obvious
benefit and which will be much less likely to cause issues with refactoring
later. Trying to use as many attributes as possibly (which many D developers
think is best practice) really does seem to be like you're putting concrete
on your code, making refactoring far, far harder than it would be in most
languages - or if you'd just minimize the attributes that you're using.

- Jonathan M Davis



 Jan 11



 next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


This is a reasonable argument. I've often suggested that people just write code 
and don't use attributes. The code will still work just fine.

Where attributes make sense is when you want a rigorous check on something. For 
example, to share a data structure among threads, making it `immutable` will 
ensure that no synchronization is necessary.

The beauty of pure functions is you can reason about them, knowing that there
is 
no "side loading" hanky panky going on. It can be verified by the compiler.
This 
is highly useful for doing parallel execution of things.

`scope` is handy for verifing the lifetime of a pointer, which is essential if 
you need to pinch off all memory leaks.

Another way to deal with attributes is to use templates, as the compiler will 
infer attributes for template functions.

Any piece of code that accepts lambdas, callbacks, etc., should be very clear 
about attribute use and how that will apply to the lambdas, etc.



 Jan 11



  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 1/11/24 23:41, Walter Bright wrote:

 
 Any piece of code that accepts lambdas, callbacks, etc., should be very 
 clear about attribute use and how that will apply to the lambdas, etc.


The issue is, sometimes the piece of code that accepts lambdas is 
entirely unopinionated about what the right attributes are, but the 
callback and the calling code have a very strong opinion. There is no 
way to annotate this now that makes sense, you have to duplicate the 
code with different attribute annotations. I am not sure exactly how 
many overloads of `opApply` you need to cater to all use cases, but I 
think it is more than ten.

This is a fundamental issue with D's type system, and it is a common 
issue programming languages run into. Type theorists solve it with 
attribute polymorphism. `inout` was an attempt at that, albeit not an 
extremely successful one.



 Jan 11



 next sibling parent  "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz>  writes:


On 12/01/2024 12:05 PM, Timon Gehr wrote:

 On 1/11/24 23:41, Walter Bright wrote:
 
     Any piece of code that accepts lambdas, callbacks, etc., should be
     very clear about attribute use and how that will apply to the
     lambdas, etc.
 
 The issue is, sometimes the piece of code that accepts lambdas is 
 entirely unopinionated about what the right attributes are, but the 
 callback and the calling code have a very strong opinion. There is no 
 way to annotate this now that makes sense, you have to duplicate the 
 code with different attribute annotations. I am not sure exactly how 
 many overloads of |opApply| you need to cater to all use cases, but I 
 think it is more than ten.


I have it implemented as a permutation...

This is awful for exports. It should be one, not 24 on top of a template 
opApply that accepts anything internally.

https://github.com/Project-Sidero/basic_memory/blob/main/source/sidero/base/internal/meta.d#L4



 Jan 11



prev sibling  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 1/11/2024 3:05 PM, Timon Gehr wrote:

 The issue is, sometimes the piece of code that accepts lambdas is entirely 
 unopinionated about what the right attributes are, but the callback and the 
 calling code have a very strong opinion. There is no way to annotate this now 
 that makes sense, you have to duplicate the code with different attribute 
 annotations. I am not sure exactly how many overloads of `opApply` you need to 
 cater to all use cases, but I think it is more than ten.



opApply was itself a mistake, I'd prefer it to be removed, but alas. The better 
method is to use a lambda as a template alias parameter, and let the compiler 
infer the right attributes for it.



 Jan 11



  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 1/12/24 08:10, Walter Bright wrote:

 On 1/11/2024 3:05 PM, Timon Gehr wrote:

 The issue is, sometimes the piece of code that accepts lambdas is 
 entirely unopinionated about what the right attributes are, but the 
 callback and the calling code have a very strong opinion. There is no 
 way to annotate this now that makes sense, you have to duplicate the 
 code with different attribute annotations. I am not sure exactly how 
 many overloads of `opApply` you need to cater to all use cases, but I 
 think it is more than ten.


 
 
 opApply was itself a mistake, I'd prefer it to be removed, but alas. The 
 better method is to use a lambda as a template alias parameter, and let 
 the compiler infer the right attributes for it.


Well, I have wished for an `opApply` with template alias sometimes, but 
this does not always work. E.g., `opApply` may be a virtual function 
(which comprises quite a lot of my own `opApply` use cases). Anyway, 
`opApply` is just one example that happens to be particularly familiar 
to many D programmers. Do you think virtual methods and runtime 
delegates were a mistake in general?



 Jan 12



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 1/12/2024 6:54 AM, Timon Gehr wrote:

 Do you 
 think virtual methods and runtime delegates were a mistake in general?


I don't like opApply() because it does a hidden rewrite of the statements in
the 
function that makes the semantic analysis fragile, ugly and clumsy.

I'm a bit amazed it works at all :-/

Virtual methods and runtime delegates don't have this problem.



 Jan 12



  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 1/12/24 21:52, Walter Bright wrote:

 On 1/12/2024 6:54 AM, Timon Gehr wrote:

 Do you think virtual methods and runtime delegates were a mistake in 
 general?


 
 I don't like opApply() because it does a hidden rewrite of the 
 statements in the function that makes the semantic analysis fragile, 
 ugly and clumsy.
 
 I'm a bit amazed it works at all :-/
 
 Virtual methods and runtime delegates don't have this problem.


My own `opApply` implementation is relatively straightforward:

https://github.com/tgehr/d-compiler/blob/master/semantic.d#L8868-L8915
https://github.com/tgehr/d-compiler/blob/master/semantic.d#L9174-L9227
https://github.com/tgehr/d-compiler/blob/master/semantic.d#L9882-L9889
https://github.com/tgehr/d-compiler/blob/master/semantic.d#L9972-L9985
https://github.com/tgehr/d-compiler/blob/master/semantic.d#L10037-L10047
https://github.com/tgehr/d-compiler/blob/master/semantic.d#L10059-L10064



 Jan 12



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 1/12/2024 3:13 PM, Timon Gehr wrote:

 On 1/12/24 21:52, Walter Bright wrote:

 On 1/12/2024 6:54 AM, Timon Gehr wrote:

 Do you think virtual methods and runtime delegates were a mistake in general?


 I don't like opApply() because it does a hidden rewrite of the statements in 
 the function that makes the semantic analysis fragile, ugly and clumsy.

 I'm a bit amazed it works at all :-/

 Virtual methods and runtime delegates don't have this problem.


 
 My own `opApply` implementation is relatively straightforward:
 
 https://github.com/tgehr/d-compiler/blob/master/semantic.d#L8868-L8915
 https://github.com/tgehr/d-compiler/blob/master/semantic.d#L9174-L9227
 https://github.com/tgehr/d-compiler/blob/master/semantic.d#L9882-L9889
 https://github.com/tgehr/d-compiler/blob/master/semantic.d#L9972-L9985
 https://github.com/tgehr/d-compiler/blob/master/semantic.d#L10037-L10047
 https://github.com/tgehr/d-compiler/blob/master/semantic.d#L10059-L10064


I'm not sure if it is straightforward or not! Consider that the original 
implementation was implemented in C-with-Classes, not D templates!



 Jan 12



prev sibling  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 1/11/24 22:45, Jonathan M Davis wrote:

 On Thursday, January 11, 2024 12:33:48 PM MST Walter Bright via Digitalmars-d
 wrote:

 On 1/11/2024 5:07 AM, deadalnix wrote:

 These are just simple thing that I have on top of my mind, but there are a
 ton more.


 Thanks for taking the time to write this list. I would like to see all of
 your issues, though! Let's make a list and then have a bugzilla entry for
 each of them, so we can start picking them off.

   > such as  nogc, has been a productivity disaster

 I don't really understand this. Just don't use  nogc?


 
 ...
 I'm increasingly inclined to think that most attributes are primarily of
 theoretical benefit


No, theory predicts many of those issues with the way attributes are 
designed in D currently. x)


 ... Trying to use as many attributes as possibly (which many D developers
 think is best practice) really does seem to be like you're putting concrete
 on your code, making refactoring far, far harder than it would be in most
 languages - or if you'd just minimize the attributes that you're using.
 ...


Yes. Unfortunately it seems to be very hard to make new D developers 
understand this point until they run into the problem themself.

You put an attribute if the code is wrong if it does not conform to the 
attribute, otherwise you should not put it. But in D, often both choices 
are incorrect if you want to interact with code that does have 
attributes applied to it.



 Jan 11



prev sibling  parent reply deadalnix <deadalnix gmail.com>  writes:


On Thursday, 11 January 2024 at 19:33:48 UTC, Walter Bright wrote:

 On 1/11/2024 5:07 AM, deadalnix wrote:

 such as  nogc, has been a productivity disaster


 I don't really understand this. Just don't use  nogc?


And don't use any library. Got it.



 Jan 11



  parent reply max haughton <maxhaton gmail.com>  writes:


On Thursday, 11 January 2024 at 23:44:02 UTC, deadalnix wrote:

 On Thursday, 11 January 2024 at 19:33:48 UTC, Walter Bright 
 wrote:

 On 1/11/2024 5:07 AM, deadalnix wrote:

 such as  nogc, has been a productivity disaster


 I don't really understand this. Just don't use  nogc?


 And don't use any library. Got it.


The thing with  nogc and ESPECIALLY betterC in a library context 
is that it adds more and more decisions everywhere - O(2^N)? And 
that's before all the attribute spam.



 Jan 11



  parent reply "H. S. Teoh" <hsteoh qfbox.info>  writes:


On Fri, Jan 12, 2024 at 12:42:44AM +0000, max haughton via Digitalmars-d wrote:

 On Thursday, 11 January 2024 at 23:44:02 UTC, deadalnix wrote:

 On Thursday, 11 January 2024 at 19:33:48 UTC, Walter Bright wrote:

 On 1/11/2024 5:07 AM, deadalnix wrote:

 such as  nogc, has been a productivity disaster


 
 I don't really understand this. Just don't use  nogc?


 
 And don't use any library. Got it.


 
 The thing with  nogc and ESPECIALLY betterC in a library context is
 that it adds more and more decisions everywhere - O(2^N)? And that's
 before all the attribute spam.


Exactly, it's causing D's already-small ecosystem to be fragmented even
more.

Now instead of one library that works for all D code, you have libraries
that use the GC so they're incompatible with betterC and  nogc projects,
libraries that don't use GC but still use some D feature not in betterC,
so the betterC people are left out, then you have  nogc libraries that
the GC crowd wouldn't use because it requires you to pass allocators and
all the rest of the spam associated with manual memory management.

The only way a library author can deal with this mess is (1) write in
the most restricted language subset, i.e., betterC +  nogc, so that it's
usable by everybody. But the GC crowd will be unlikely to use it,
because it will lack the conveniences they're accustomed to, the API
will have manual memory management paraphrenalia that doesn't fit well
with the rest of user code. Plus this is also a much higher bar for the
library author, so you're further limiting the already small number of
people who are writing D libraries.  Or (2) be opinionated and write a
GC-using library, in which case the betterC /  nogc people won't use it.
Either alternative leads to ecosystem fragmentation. Or write a  nogc
library but the betterC people won't use it.  Etc..

//

Now, this is only at the library level.  When you come down to the
function level it gets worse. Take for example a library that has a
function to register a callback to be triggered when some event occurs:

	void registerCallback(void delegate() cb);

Problem:  safe code can't call this.  nogc code can't call this. BetterC
code can't call it either. What to do? Add the most restrictive
attributes so that everyone can call it:

	extern(C) void registerCallback(void delegate() cb)  safe  nogc;

Oops, BetterC does not support delegates. Guess we have to lose that
subset of D users:

	void registerCallback(void delegate() cb)  safe  nogc;

Still no good. The function will happily take a delegate in the `cb`
parameter, but it can't do anything with it, it cannot be called from
anywhere, the delegate is  system and may allocate.  Next stab:

	void registerCallback(void delegate()  safe  nogc cb)  safe  nogc;

OK, now the function body can actually call `cb`. But wait, now the
function is no longer usable with  system callbacks. Nor with allocating
callbacks.  Solution?  Templatize it and let the compiler figure out the
attribute soup:

	void registerCallback()(void delegate() cb);

Finally, a solution?  Nope, there can only be one instantiation of this
template, and the inferred attributes will be the most restrictive that
still allows the function body to compile.  No matter what attributes
the compiler infers for it, it will exclude *some* use cases. For
example if  safe was inferred, it makes the function unusable with
 system callbacks.  If  nogc was inferred, it cannot be used with
allocating callbacks.  If it was inferred pure, it cannot be used with
an impure callback.  So the only code that can actually call this
function is pure  safe  nogc nothrow.  Meaning that it's useless for any
code that's impure,  system, allocating, or uses Exceptions.

The only way around this is to create 2^N versions of this function, one
with each combination of attributes, so that it's fully generic for all
of its intended audience.

	void registerCallbackSystem(void delegate()  system cb)  system;
	void registerCallbackSafe(void delegate()  safe cb)  safe;
	void registerCallbackNoGc(void delegate()  nogc cb) nogc;
	void registerCallbackSafeNoGc(void delegate()  safe  nogc cb)  safe  nogc;

Most library authors aren't going to bother doing this; they will just
pick whatever attribute set suits them, and not bother about the rest.
Net result: ecosystem fragmentation.


T

-- 
In theory, there is no difference between theory and practice.



 Jan 11



 next sibling parent  "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz>  writes:


On 12/01/2024 2:21 PM, H. S. Teoh wrote:

 Oops, BetterC does not support delegates. Guess we have to lose that 
 subset of D users:
 
 
 |void registerCallback(void delegate() cb)  safe  nogc;|


What? -betterC supports delegates.

It does not support closure creation with GC however.

But you can create a delegate yourself just fine.

In fact if you make that scope, it'll create the closure just fine with 
-betterC.



 Jan 11



prev sibling next sibling parent reply zjh <fqbqrr 163.com>  writes:


On Friday, 12 January 2024 at 01:21:32 UTC, H. S. Teoh wrote:

 On Fri, Jan 12, 2024 at



```d
void registerCallbackSystem(void delegate()  system cb)  system;
void registerCallbackSafe(void delegate()  safe cb)  safe;
...
```
I think `'abstract attributes'` is also a good idea. Can we turn 
the above function into:
```d
bool  attribute f(set b){
     return b.hasone( system);
}

void registerCallback(void delegate() cb)  safe{
     static if(f(attrs(cb))){
         ...
     }
     static if(attrs(cb)== nogc){
         ...
     }
     ...
}
```
Combining `attribute functions` to avoid `function splitting`.



 Jan 11



  parent reply zjh <fqbqrr 163.com>  writes:


On Friday, 12 January 2024 at 02:31:21 UTC, zjh wrote:


 Combining `attribute functions` to avoid `function splitting`.


It's best to be able to `deduct` function etc. Then, you can use 
the information deducted from the deduction for metaprogramming.

`Deduct`the common attribute, Is it ` nogc`? Is it 
` pure`?Or` system`,` safety`?.



 Jan 11



  parent  zjh <fqbqrr 163.com>  writes:


On Friday, 12 January 2024 at 02:43:56 UTC, zjh wrote:


 `Deduct`the common attribute, Is it ` nogc`? Is it 
 ` pure`?Or` system`,` safety`?.



A `function` (`variable`, construction block) has an '`attribute 
dictionary`'. This attribute dict can contain four common 
attributes: `' nogc,  pure, system, safe', and user attributes`. 
Each '`attribute dict`' can be calculated

This kind of `attribute dictionary` should be very useful!



 Jan 11



prev sibling next sibling parent reply Guillaume Piolat <first.name gmail.com>  writes:


On Friday, 12 January 2024 at 01:21:32 UTC, H. S. Teoh wrote:

 The only way a library author can deal with this mess is (1) 
 write in the most restricted language subset, i.e., betterC + 
  nogc, so that it's usable by everybody. But the GC crowd will 
 be unlikely to use it, because it will lack the conveniences 
 they're accustomed to, the API will have manual memory 
 management paraphrenalia that doesn't fit well with the rest of 
 user code. Plus this is also a much higher bar for the library 
 author, so you're further limiting the already small number of 
 people who are writing D libraries.  Or (2) be opinionated and 
 write a GC-using library, in which case the betterC /  nogc 
 people won't use it. Either alternative leads to ecosystem 
 fragmentation. Or write a  nogc library but the betterC people 
 won't use it.  Etc..


Interesting post thanks. Describes the situation very well.
I think more escape hatches ( localnogc?) may help with the 
delegate situation you related.



 Jan 11



  parent  "H. S. Teoh" <hsteoh qfbox.info>  writes:


On Fri, Jan 12, 2024 at 03:13:58AM +0000, Guillaume Piolat via Digitalmars-d
wrote:

 On Friday, 12 January 2024 at 01:21:32 UTC, H. S. Teoh wrote:

 
 The only way a library author can deal with this mess is (1) write
 in the most restricted language subset, i.e., betterC +  nogc, so
 that it's usable by everybody. But the GC crowd will be unlikely to
 use it, because it will lack the conveniences they're accustomed to,
 the API will have manual memory management paraphrenalia that
 doesn't fit well with the rest of user code. Plus this is also a
 much higher bar for the library author, so you're further limiting
 the already small number of people who are writing D libraries.  Or
 (2) be opinionated and write a GC-using library, in which case the
 betterC /  nogc people won't use it.  Either alternative leads to
 ecosystem fragmentation. Or write a  nogc library but the betterC
 people won't use it.  Etc..
 


 
 Interesting post thanks. Describes the situation very well.
 I think more escape hatches ( localnogc?) may help with the delegate
 situation you related.


Escape hatches is not the answer.  What we need is (1) a consistent way
of enabling/disabling an attribute, and (2) a way of treating them as
first-class citizens, i.e., be able to express "this delegate's
attributes is equal to the attributes of the parent function".  Timon
already described this, I won't repeat the details.


T

-- 
Doubtless it is a good thing to have an open mind, but a truly open mind should
be open at both ends, like the food-pipe, with the capacity for excretion as
well as absorption. -- Northrop Frye



 Jan 11



prev sibling next sibling parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 1/11/2024 5:21 PM, H. S. Teoh wrote:

 	void registerCallback(void delegate() cb);
 
 Problem:  safe code can't call this.  nogc code can't call this. BetterC
 code can't call it either.


```d
extern(C) void registerCallback(void delegate() cb)
{
     cb();
}

void boo()
{
     extern (C) void delegate() cb;
     registerCallback(cb);
}
```
compiles with -betterC



 Jan 11



prev sibling next sibling parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 1/11/2024 5:21 PM, H. S. Teoh wrote:

 	void registerCallbackSystem(void delegate()  system cb)  system;
 	void registerCallbackSafe(void delegate()  safe cb)  safe;
 	void registerCallbackNoGc(void delegate()  nogc cb) nogc;
 	void registerCallbackSafeNoGc(void delegate()  safe  nogc cb)  safe  nogc;


To make it work with both  safe and  system code, use  trusted.

To make code usable with both -betterC and regular D, just don't use the gc in 
it. No need to put  nogc in it.



 Jan 11



prev sibling next sibling parent reply Paolo Invernizzi <paolo.invernizzi gmail.com>  writes:


On Friday, 12 January 2024 at 01:21:32 UTC, H. S. Teoh wrote:

 On Fri, Jan 12, 2024 at 12:42:44AM +0000, max haughton via 
 Digitalmars-d wrote:

 [...]


 Exactly, it's causing D's already-small ecosystem to be 
 fragmented even more.

 [...]


I don't understand why a library author needs to satisfy everyone.

If part of an IO library API is  safe  nogc, for example, I'm 
expecting the author cared about avoiding allocation in that part 
and cared about being safe _following a specific design target_, 
and aiming to praise application _following a similar design 
target for IO_.

That doesn't align with other design? Well, it's simply 
impossible to reach all goals at the same time, it's better to 
have different solutions (aka libraries) for different 
philosophy, and it's easier also.

BTW, that's also because I'm skeptical about the success of 
editions, you can't have _everything_ in the compiler, and at the 
same time expect to have it simple and manageable. I'm expecting 
a big grow in complexity.

/P



 Jan 12



 next sibling parent reply "H. S. Teoh" <hsteoh qfbox.info>  writes:


On Fri, Jan 12, 2024 at 09:02:24AM +0000, Paolo Invernizzi via Digitalmars-d
wrote:

 On Friday, 12 January 2024 at 01:21:32 UTC, H. S. Teoh wrote:

 On Fri, Jan 12, 2024 at 12:42:44AM +0000, max haughton via Digitalmars-d
 wrote:

 [...]


 
 Exactly, it's causing D's already-small ecosystem to be fragmented
 even more.
 
 [...]


 
 I don't understand why a library author needs to satisfy everyone.


[...]

They don't.  They cater to their own use case, leaving everybody else
out.  That's why D's library ecosystem is fragmenting.  That was the
point: things like  nogc, betterC, etc., are fragmenting the ecosystem.


T

-- 
Gone Chopin. Bach in a minuet.



 Jan 12



 next sibling parent  Guillaume Piolat <first.name gmail.com>  writes:


On Friday, 12 January 2024 at 15:18:48 UTC, H. S. Teoh wrote:

 They don't.  They cater to their own use case, leaving 
 everybody else out.  That's why D's library ecosystem is 
 fragmenting.  That was the point: things like  nogc, betterC, 
 etc., are fragmenting the ecosystem.


It does, but it's not like the cases where people want to use 
betterC or  nogc aren't valid, like claptrap said. A static 
guarantee is really helpful here all the time.

The most restricted sort of D can be used almost anywhere, such 
as the (pretty much un-D in spirit) nothrow  nogc no-exceptions 
betterC-compatible struct-only subset.
So in large parts it is possible to not fragment some of the 
low-level unopinionated libraries.

When this is done, the biggest issue left is plain old manual 
handling, as bad as ever.

The "true fix" for that is a druntime that can go anywhere in any 
situation (statically linked, with GC hooks) and this is it seems 
the call of "OpenD" afaik, but that's not what we have now.

Even if possible there are cases where you may just want a C 
runtime, such as was highlighted by the Mir project. Or Inochi2D 
planning to go  nogc for portability reasons. What's really new 
here is Hipreme being able to run a sizeable D subset with a 
custom runtime, that only requires no-exception AFAIK.



 Jan 12



prev sibling  parent  deadalnix <deadalnix gmail.com>  writes:


On Friday, 12 January 2024 at 15:18:48 UTC, H. S. Teoh wrote:

 On Fri, Jan 12, 2024 at 09:02:24AM +0000, Paolo Invernizzi via 
 Digitalmars-d wrote:

 I don't understand why a library author needs to satisfy 
 everyone.


 [...]

 They don't.  They cater to their own use case, leaving 
 everybody else out.  That's why D's library ecosystem is 
 fragmenting.  That was the point: things like  nogc, betterC, 
 etc., are fragmenting the ecosystem.


This. Indeed, you can't expect people to bend over backward to 
accommodate everybody's use case. But, to grow an ecosystem, you 
want that one person catering to their own use case to, by 
default, cater to as many people's use case as possible.

Feature that work against this create a maintenance nightmare.



 Jan 13



prev sibling  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 1/12/2024 1:02 AM, Paolo Invernizzi wrote:

 BTW, that's also because I'm skeptical about the success of editions, you
can't 
 have _everything_ in the compiler, and at the same time expect to have it
simple 
 and manageable. I'm expecting a big grow in complexity.


That is indeed a risk. We're going to have to be careful with that.



 Jan 14



prev sibling  parent  JN <666total wp.pl>  writes:


On Friday, 12 January 2024 at 01:21:32 UTC, H. S. Teoh wrote:

 Now instead of one library that works for all D code, you have 
 libraries that use the GC so they're incompatible with betterC 
 and  nogc projects, libraries that don't use GC but still use 
 some D feature not in betterC, so the betterC people are left 
 out, then you have  nogc libraries that the GC crowd wouldn't 
 use because it requires you to pass allocators and all the rest 
 of the spam associated with manual memory management.


I wonder how many people actually willingly use betterC. I feel 
like WASM is the main usecase for betterC, and if WASM worked in 
"normal D mode" betterC would mostly fade into obscurity for 
special cases when you need to interop with existing C code.



 Jan 14



prev sibling next sibling parent  Don Allen <donaldcallen gmail.com>  writes:


On Thursday, 11 January 2024 at 13:07:52 UTC, deadalnix wrote:

 I have been in the D community for a very long time. I have 
 seen D successfully deployed in companies, and the pain points 
 associated with it. I have seen D fails to catch on in 
 companies and why that is has well.

 Let me tell you, none of this has anything to do with feature D 
 has or does not have. At large, D has more features than most 
 languages.

 D chasing the next feature like a crack addict chase his next 
 dose. With the same level of success.

 The main problem people face with D in the real world are 
 almost exclusively of the implementation kind. The list is 
 endless (and yes, there are many bugs reports about these 
 things). I recently made a post about how the OOP 
 implementation is extremely sub-par vs what people in OOP 
 languages would expect. no change of the language required to 
 fix. See here: 
 https://forum.dlang.org/post/hteuczyclxajakrisxjd forum.dlang.org

 But if you are not convinced, here are a few more example of 
 thing being implemented wrong or existing feature not working 
 right:
  - D runtime is unable to see thread started manually (for 
 instance with pthread-create) leading to all kind of bizarre 
 behavior.
  - Template symbols are generated as weak, which prevents 
 inlining (!).
  - Pretty much no cross module inlining, making helper function 
 absurdly costly.
  - scope(success) generates exception handling code.
  - D goes virtual by default for class methods, but LTO is 
 unable to finalize (contrary to every other languages going 
 virtual by default).
  - The GC implementation is nowhere close to where it needs to 
 be.
  - in contracts are dynamically bound (and in the callee) 
 instead of statically bounds and in the caller.

 These are just simple thing that I have on top of my mind, but 
 there are a ton more. I have seen some of the above cause 
 projects to fail. None of them require any significant language 
 change.

 There is nothing features like string interpolations or named 
 argument can bring to the table that could pay for the 
 implementations problem of existing feature. The cost benefit 
 analysis is just a big L for D: the fail to address the main 
 pain points, while causing massive breakage in the tooling 
 ecosystem (syntax highlighting support in 3rd party IDE, code 
 formatter, etc...), and it cost real time and resource to 
 upgrade these, or come at the cost of other quality of life 
 stuff nullifying their benefit (for instance, the quality of 
 syntax highlighting for D has degraded significantly in vim and 
 sublime text over the past few years).

 In addition, some recent D features, such as  nogc, has been a 
 productivity disaster int he wild. While the impact might not 
 be felt on smaller codebases, the infectious nature of the 
 feature makes large codebase significantly harder the refactor 
 than they used to be.

 Each time we take steps in that direction, D becomes a harder 
 sell.


While I don't have a long history with D or deep knowledge of the 
implementation details, the thrust of this message sounds right 
to me.

The calls I've seen recently for more language features strike me 
as misguided. As the above says, D is already a big language, 
lotsa features. It's also well-documented, compared with what 
I've seen with comparable languages, even Rust, whose 
"affectionately named" (seriously?) Book is simply awful, in my 
opinion.

I think Walter and Co. need to create a plan to address D's real 
weaknesses, e.g., tooling, issues with Phobos, and problems with 
the implementations of existing features. Lack of features is not 
among them, in my opinion.

This should be discussed with this community and we should, in 
turn,  avoid the temptation to derail the Big Picture discussion 
by descending into talk of technical details prematurely. Make 
your best plan first. Then you can talk about how to implement 
it. This may cause an iteration back to the plan, adjusting with 
what you learned by considering implementation problems.

I'm, in effect, supporting Walter's preference for 
specifications, but at an even higher level than a particular 
DIP. I also generally support Walter's tendency to say "no" 
(perhaps improving communication of his reasoning), because it's 
consistent with what I'm advocating here. Make sure what you have 
is at a high professional level before moving on to the next 
great thing.

I don't think D's future lies in trying to be more dazzling than 
the competition. I think being better engineered (Walter's 
strength) and better crafted are the dimensions that suit this 
project. Think about products like Toyotas or Seiko watches. 
While both companies have produced some highly innovative 
products, we mostly know and admire them because their 
main-stream stuff Just Works. I think of D in that category. I 
believe that if the project focuses on this and people start to 
discover that in addition to writing programs in D being a huge 
improvement over wrestling with C or C++, all the tooling is 
there and of good quality (I'm really looking forward to the new 
dfmt), so that the D software-development experience, end-to-end, 
is as problem-free as possible. People don't want to fight with 
their tools; they want to use them to accomplish their goal. 
Serve that and you've got a winner.



 Jan 11



prev sibling  parent reply Elias (0xEAB) <desisma heidel.beer>  writes:


On Thursday, 11 January 2024 at 13:07:52 UTC, deadalnix wrote:

 the quality of syntax highlighting for D has degraded 
 significantly


For code-d it improved in my experience. Although, the switch to 
a different grammar around 2017ish brought a bunch of quirks, the 
overall experience got better. And some of said quirkiness has 
been vetted.

Syntax highlighting for D has been pretty disappointing in 
general (across all implementations) ever since in the first 
place.
In my perception, it is as it has always been: Better than no 
highlighting.
…I think this summarizes it quite well.



 Jan 12



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 1/12/2024 5:35 PM, Elias (0xEAB) wrote:

 Syntax highlighting for D has been pretty disappointing in general (across all 
 implementations) ever since in the first place.
 In my perception, it is as it has always been: Better than no highlighting.
 …I think this summarizes it quite well.


My editor (microEmacs) pretends to do syntax highlighting by simply showing 
keywords, comments, text, and string literals in different colors. I had to 
switch it to muted colors because the durn thing presented like a Christmas
tree.

I kinda enjoy the look, but am unsure of its value.



 Jan 12