www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - The point of const, scope, and other attributes

reply Walter Bright <newshound2 digitalmars.com> writes:
The original D did not have them.

Many people who have written D code without them, and then added them. Then
they 
noticed that although those attributes are meant to prevent bugs, adding them
to 
existing code did not find any bugs. So what is the point?

A small anecdote. Before the backend was recently converted to D, I was invited 
to try out one of the good C++ bug checkers. I thought great, this will find 
bugs in the backend.

While it generated about a thousand false positives, it didn't find a single
bug.

But I realized that I was running the checker over code that had already been 
thoroughly debugged through a lot of usage. Of course it didn't find anything. 
All the memory corruption bugs, bad casts, etc., had already been squeezed out.

The point of the checker was to find bugs in fresh code, and find them before 
even trying to run the code. The checker saves you the trouble of finding those 
bugs yourself. It saves you the trouble caused by shipping those bugs.

---

Another story. Back when I was programming in C, and was inexperienced, my code 
had a lot of bugs in it. I made every mistake, over and over. Having a memory 
corruption bug was a depressingly routine problem, and they were always hard to 
debug. But, over time, and endless experience, I gradually learned to avoid 
writing those bugs. I can't even recall the last time I wrote a memory 
corruption bug. (Of course, saying this, tomorrow for sure I'll make that
mistake!)

But this is really doing things the hard way. I'm lazy, I like the easy way.

---

There are other reasons for the attributes, such as making code more 
understandable. A `const` parameter tells you the function doesn't modify that 
argument, or what it points to. A `scope` parameter tells you the function 
doesn't save a copy of it, so the caller can confidently delete the data. And
so on.
May 10 2022
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
The C++ checker did gripe about some of the debug printf format strings not 
quite matching the arguments, such as printing a pointer with %x. The errors 
were benign, though. printf format mismatch was another situation where I wrote 
lots of bugs.

But that experience helped push me towards getting D to do printf format 
checking. The checking doesn't find bugs anymore in existing code, because
those 
errors never make it through a compilation. It also trained me to not make
those 
mistakes anymore.

So is that useful? Hell yes. I don't even have to bother checking the formats 
anymore when reviewing code. Neither need anyone else. Ain't that the shiznit? 
Why didn't we do that eons ago? Sigh.
May 10 2022
parent Dennis <dkorpel gmail.com> writes:
On Wednesday, 11 May 2022 at 05:15:00 UTC, Walter Bright wrote:

 The checking doesn't find bugs anymore in existing code, 
 because those errors never make it through a compilation.


Never say never ;)

https://github.com/dlang/dmd/pull/13987
https://issues.dlang.org/show_bug.cgi?id=23020
May 11 2022
prev sibling next sibling parent Salih Dincer <salihdb hotmail.com> writes:
On Wednesday, 11 May 2022 at 05:02:31 UTC, Walter Bright wrote:

 There are other reasons for the attributes, such as making code 
 more understandable. A `const` parameter tells you the function 
 doesn't modify that argument, or what it points to. A `scope` 
 parameter tells you the function doesn't save a copy of it, so 
 the caller can confidently delete the data. And so on.


Very satisfying information that I have encountered for the first 
time on subjects that I have never used, thank you. Maybe I'll 
use it more often now :)

SDB 79
May 10 2022
prev sibling next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Wednesday, 11 May 2022 at 05:02:31 UTC, Walter Bright wrote:

 [snip]


I am increasingly of the opinion that DIP1000 is too complicated 
for the average programmer to understand well. I find myself 
often stratching my head with it, it would be overwhelming for a 
beginner.

Still, I do not advocate going back to pre-DIP1000 semantics 
because those are simply unsafe when it comes to slicing static 
arrays. I think we should start teaching two rules:

1: Never, ever brute force your way through DIP1000 errors with 
` trusted` because you do not understand `return`, `scope` and 
`ref`.
2: If you're having trouble, GC-allocate the variables you're 
trying to use.

The nice thing about DIP1000 is that one can't accidently 
introduce bugs as long as s/he follows rule 1.

Regarding other attributes, I think it's good that our 
const/immutable/shared are transitive. Otherwise the attribute 
soup problem we have would be even worse.
May 11 2022
next sibling parent reply IGotD- <nise nise.com> writes:
On Wednesday, 11 May 2022 at 07:21:02 UTC, Dukc wrote:

 I am increasingly of the opinion that DIP1000 is too 
 complicated for the average programmer to understand well. I 
 find myself often stratching my head with it, it would be 
 overwhelming for a beginner.



it doesn't make much sense because there is a lot of runtime 

better because it is simpler for the programmer.

Another problem is that all this badging tend to get out of hand 
and it breeds more badges until it doesn't mean anything. For 
example what will happen if D adds a mutable attribute?
May 11 2022
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/11/2022 3:48 AM, IGotD- wrote:


make 
 much sense because there is a lot of runtime violations of const underneath. 



It's not simpler for someone trying to read the code. Function documentation 
almost never says "by the way, this `getValue` function also tweaks the
database."



 Another problem is that all this badging tend to get out of hand and it breeds 
 more badges until it doesn't mean anything. For example what will happen if D 
 adds a mutable attribute?


It already has a mutable attribute - nothing!
May 11 2022
parent reply Fry <fry131313 gmail.com> writes:
On Wednesday, 11 May 2022 at 18:18:18 UTC, Walter Bright wrote:

 On 5/11/2022 3:48 AM, IGotD- wrote:


 because it doesn't make much sense because there is a lot of 

 aproach better because it is simpler for the programmer.


 It's not simpler for someone trying to read the code. Function 
 documentation almost never says "by the way, this `getValue` 
 function also tweaks the database."


Good documentation will. Really const doesn't help with making 
code more readable either. I avoided using const in D cause it 
just got in the way more than it was helping. The most useful 
feature for const to exist in C++ was that it allows you to do 
`const T&` which will allow you to pass anything to that 
function, rvalue or lvalue without making a copy. D doesn't do 

would have just added complexity without much benefit.


 Another problem is that all this badging tend to get out of 
 hand and it breeds more badges until it doesn't mean anything. 
 For example what will happen if D adds a mutable attribute?


 It already has a mutable attribute - nothing!


I think he means a mutable attribute that allows you to modify a 
value even if the object is const. Like C++'s mutable.
May 13 2022
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, May 13, 2022 at 06:25:22PM +0000, Fry via Digitalmars-d wrote:

 On Wednesday, 11 May 2022 at 18:18:18 UTC, Walter Bright wrote:

 On 5/11/2022 3:48 AM, IGotD- wrote:








[...]

 It's not simpler for someone trying to read the code. Function
 documentation almost never says "by the way, this `getValue`
 function also tweaks the database."


 
 Good documentation will. Really const doesn't help with making code
 more readable either. I avoided using const in D cause it just got in
 the way more than it was helping.


Yeah, more than one long-time D user (including myself) has come to this
conclusion. In my own experience, const is mainly useful at the lowest
levels of code (strings, leaf-node modules that operate on some
self-contained data structure that doesn't depend on anything else).
Once you get to a high enough level of abstraction, it starts getting in
your way and becomes far too much more work than the benefits it offers;
it's just not worth the trouble.  So these days I don't really bother
with const, except in small pockets of leaf-node code.



 The most useful feature for const to exist in C++ was that it allows
 you to do `const T&` which will allow you to pass anything to that
 function, rvalue or lvalue without making a copy. D doesn't do this
 though.


[...]

I thought `auto ref` was supposed to do this?  But I recall people
hating `auto ref` for various reasons...


T

-- 
Gone Chopin. Bach in a minuet.
May 13 2022
parent reply Tejas <notrealemail gmail.com> writes:
On Friday, 13 May 2022 at 18:43:11 UTC, H. S. Teoh wrote:

 On Fri, May 13, 2022 at 06:25:22PM +0000, Fry via Digitalmars-d 
 wrote:

 [...]






 [...]

 [...]


 Yeah, more than one long-time D user (including myself) has 
 come to this conclusion. In my own experience, const is mainly 
 useful at the lowest levels of code (strings, leaf-node modules 
 that operate on some self-contained data structure that doesn't 
 depend on anything else). Once you get to a high enough level 
 of abstraction, it starts getting in your way and becomes far 
 too much more work than the benefits it offers; it's just not 
 worth the trouble.  So these days I don't really bother with 
 const, except in small pockets of leaf-node code.



 [...]


 [...]

 I thought `auto ref` was supposed to do this?  But I recall 
 people hating `auto ref` for various reasons...


 T


Both `auto ref` and `in` make a copy if it's an rvalue.

Idk why people hate `auto ref` though... most likely the fact 
that one might have to do `__traits(isRef, symbol)` anyways for 
some edge cases
May 13 2022
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 5/13/22 17:04, Tejas wrote:

 On Friday, 13 May 2022 at 18:43:11 UTC, H. S. Teoh wrote:

 On Fri, May 13, 2022 at 06:25:22PM +0000, Fry via Digitalmars-d wrote:

 [...]






 [...]

 [...]


 Yeah, more than one long-time D user (including myself) has come to
 this conclusion. In my own experience, const is mainly useful at the
 lowest levels of code (strings, leaf-node modules that operate on some
 self-contained data structure that doesn't depend on anything else).
 Once you get to a high enough level of abstraction, it starts getting
 in your way and becomes far too much more work than the benefits it
 offers; it's just not worth the trouble.  So these days I don't really
 bother with const, except in small pockets of leaf-node code.



 [...]


 [...]

 I thought `auto ref` was supposed to do this?  But I recall people
 hating `auto ref` for various reasons...


 T


 Both `auto ref` and `in` make a copy if it's an rvalue.


'in' may not copy rvalues when compiled with -preview=in:

import std.stdio;

struct S(size_t N) {
   int[N] i; // Can be veeery big

   this (int i) {
     writeln("When constructing: ", &this.i);
   }
}

void foo(T)(in T s) {
   writeln("Inside foo       : ", &s.i);
}

void main() {
   foo(S!1(42));
   foo(S!1000(43));
}

foo() receives two rvalue objects, second of which is passed by 
reference just because the compiler decided to do so:

When constructing: 7FFD44B79EA0
Inside foo       : 7FFD44B79E78  // <-- Copied for N==1; who cares. :)
When constructing: 7FFD44B79EB0
Inside foo       : 7FFD44B79EB0  // <-- rvalue passed by reference.

That prooves D does indeed support rvalue references. :D


 Idk why people hate `auto ref` though... most likely the fact that one
 might have to do `__traits(isRef, symbol)` anyways for some edge cases


My discomfort with 'auto ref' is, it comes with a guideline: Treat the 
parameter 'const'. If not, an rvalue object would be mutated but the 
outside world would not know about it (mostly). Why would it be 
different for lvalues? Because they are passed by reference, the caller 
has an interest in the mutation, right?

So, I can't imagine a use case where lvalue mutation is important but 
rvalue mutation is not.

Ali
May 13 2022
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Saturday, 14 May 2022 at 01:11:03 UTC, Ali Çehreli wrote:

 So, I can't imagine a use case where lvalue mutation is 
 important but rvalue mutation is not.


Maybe for a function that wants to define optional out parameters 
that the caller can care about or not?  It's a trivial example, 
but something like:

```D
import std.stdio: writefln;

void foo () (int input,
              auto ref int optional_output = 0 /* dummy rvalue 
default */)
{
     writefln!"Input received: %s"(input);
     optional_output = input * 2;
}

void main()
{
     foo(7);  // `optional_out` parameter is effectively ignored
     writefln("Output parameter was not used");
     int output;
     foo(19, output);  // `optional_out` parameter writes to lvalue
     writefln!"Output received: %s"(output);
     assert(output == 38);
}
```

A similar pattern might also be useful for passing optional 
context object to which a function might write, say, information 
on its progress.

Not sure that either of these would be wise, but at least they 
are conceivable options.
May 14 2022
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sat, May 14, 2022 at 09:34:51AM +0000, Joseph Rushton Wakeling via
Digitalmars-d wrote:

 On Saturday, 14 May 2022 at 01:11:03 UTC, Ali Çehreli wrote:

 So, I can't imagine a use case where lvalue mutation is important
 but rvalue mutation is not.


 
 Maybe for a function that wants to define optional out parameters that
 the caller can care about or not?  It's a trivial example, but
 something like:
 
 ```D
 import std.stdio: writefln;
 
 void foo () (int input,
              auto ref int optional_output = 0 /* dummy rvalue default */)
 {


[...]

 }
 ```
 
 A similar pattern might also be useful for passing optional context
 object to which a function might write, say, information on its
 progress.


TBH, for cases like these, I'd just use a pointer and check for null:

	void foo()(int input, int* optional_output = null) {
		...
		if (optional_output !is null)
			*optional_output = ...;
		...
	}

No need to fear pointers, they're in the language for a reason.


T

-- 
Build a man a fire, and he is warm for a night. Set a man on fire, and he is
warm for the rest of his life.
May 14 2022
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Saturday, 14 May 2022 at 09:45:33 UTC, H. S. Teoh wrote:

 TBH, for cases like these, I'd just use a pointer and check for 
 null

 [...]

 No need to fear pointers, they're in the language for a reason.


Quite fair, of course.  One doesn't _need_ mutable auto ref to 
achieve the use cases above ... but one could conceivably use it 
if one was so inclined.
May 14 2022
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/11/2022 12:21 AM, Dukc wrote:

 Regarding other attributes, I think it's good that our const/immutable/shared 
 are transitive. Otherwise the attribute soup problem we have would be even
worse.


My experience with non-transitive const in C and C++ is it is more 
"documentation" than anything reliable. For example, if you're handing a 
reference to a non-trivial data structure to a function, there's no way to say 
"just read the data structure, don't change it". People schlepp a `const` on it 
anyway, but it doesn't work.
May 11 2022
prev sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 11 May 2022 at 05:02:31 UTC, Walter Bright wrote:

 The original D did not have them.

 [snip]


I'm generally pro-attribute, but I can't help but find myself 
scratching my head a bit with when to use return ref scope. And 
then for whatever reason there was a change so that now scope 
return has to be return scope.

So I think it might help to lay out some principles vis a vis 
attributes.

The first principle I would emphasize is "time to first plot". I 
learned about this from the Julia language, where they are 
talking about actual time it takes for Julia to boot up and be 
able to plot something. However, I think you can think about it a 
bit more metaphorically as "time for a new user to do something." 
This principle implies that D's opt-in approach to attributes is 
good since a new user can start doing work without needing to 
worry about them at first. On the basis of this principle, moving 
toward  safe by default or const/immutable by default would be 
bad.

The second principle is harder to describe, but it is related to 
the relationship between complexity, levels of abstraction, and 
mental burden. In the back of my head, I am recalling a chart 
that compared C++ and D with D listed as less complex than C++ (I 
don't recall what the other axis was, maybe something like 
expressive power). Anyway, if a programming language is 
considering adding a feature and there is a less complex version 
that abstracts away from some details versus a more complex 
version that operates on a higher level of abstraction, then it 
can be the case that the more complex version operating on a 
higher level of abstraction might actually have a lower mental 
burden. Consider the treatment of lifetimes in DIP 1000 vs 
lifetimes in Rust. DIP 1000's use of return probably covers most 
peoples' use cases, but might also require a higher mental burden 
to figure out exactly what it means since it is less abstract. By 
contrast, Rust lifetimes are implemented in a more complex way, 
but also operating at a higher level of abstraction. The higher 
level of abstraction might also result in a lower mental burden. 
Long story short, sometimes adding a more complex feature can be 
a good thing if it comes associated with a higher level of 
abstraction.

Of course, the trade off between the complexity and the ability 
to generate improved performance/safety is another important 
principle.
May 11 2022
next sibling parent reply Dennis <dkorpel gmail.com> writes:
On Wednesday, 11 May 2022 at 12:54:31 UTC, jmh530 wrote:

 By contrast, Rust lifetimes are implemented in a more complex 
 way, but also operating at a higher level of abstraction. The 
 higher level of abstraction might also result in a lower mental 
 burden.


I don't know what you mean with 'higher level of abstraction' 
here, can you give an example?
May 11 2022
parent jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 11 May 2022 at 13:56:32 UTC, Dennis wrote:

 On Wednesday, 11 May 2022 at 12:54:31 UTC, jmh530 wrote:

 By contrast, Rust lifetimes are implemented in a more complex 
 way, but also operating at a higher level of abstraction. The 
 higher level of abstraction might also result in a lower 
 mental burden.


 I don't know what you mean with 'higher level of abstraction' 
 here, can you give an example?


I don't know if I'm using it in the computer science way...

I mean it like how if you look at the rules of arithmetic 
(distributive, associative, etc.) these apply to what we normally 
think of as numbers (integers, reals, etc.) but then they apply 
different to other things (like matrices). You can abstract from 
these ideas to group theory and rings and make statements about 
all algebras with particular rules.

So with respect to lifetimes, in D we can apply the return 
attribute to a function parameter. From Rust's perspective, this 
is equivalent to an explicit lifetime for a function parameter 
that lasts as long as the return. I was phrasing it as the Rust 
approach is a higher level of of abstraction, but you could also 
think of it as the D approach is a subset of the Rust approach. I 
think this was done on purpose, acknowledging that the return 
annotation is the most often way Rust lifetimes are used. My 
response would be that is all well and fine, but I still find 
this confusing:
https://dlang.org/spec/function.html#ref-return-scope-parameters
May 11 2022
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/11/2022 5:54 AM, jmh530 wrote:

 And then for whatever reason there was 
 a change so that now scope return has to be return scope.


The reason was does "ref scope return" mean return-ref scope, or does it mean 
ref return-scope?
May 11 2022
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 11 May 2022 at 18:21:29 UTC, Walter Bright wrote:

 On 5/11/2022 5:54 AM, jmh530 wrote:

 And then for whatever reason there was a change so that now 
 scope return has to be return scope.


 The reason was does "ref scope return" mean return-ref scope, 
 or does it mean ref return-scope?


I get that it was done for a reason...it's just a confusing 
reason...
May 11 2022
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/11/2022 4:07 PM, jmh530 wrote:

 I get that it was done for a reason...it's just a confusing reason...


The idea was to reduce the confusion!
May 11 2022