www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Head Const

reply Walter Bright <newshound2 digitalmars.com> writes:
rears its head again :-)

Head Const is what C++ has for const, i.e. it is not transitive, applies to one 
level only. D has transitive const.

What head const will do for us:

1. make it easy to interface to C++ code that uses const, as currently it is
not 
very practical to do so, you have to resort to pragma(mangle)

2. supports single assignment style of programming, even if the data is 
otherwise mutable

The downside is, of course, language complexity.
Feb 15 2016
next sibling parent reply ZombineDev <valid_email he.re> writes:
On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote:

 rears its head again :-)

 Head Const is what C++ has for const, i.e. it is not 
 transitive, applies to one level only. D has transitive const.

 What head const will do for us:

 1. make it easy to interface to C++ code that uses const, as 
 currently it is not very practical to do so, you have to resort 
 to pragma(mangle)

 2. supports single assignment style of programming, even if the 
 data is otherwise mutable

 The downside is, of course, language complexity.


I'll leave to others to discuss whether a language solution is 
worth it, but I just wanted to point out there are two library 
solutions in this area:
HeadConst [1] - PR pending, and
Rebindable (tail const) [2] - already part of Phobos.

Maybe if the language could express head const, it also makes 
sense to have tail const.

[1]: https://github.com/D-Programming-Language/phobos/pull/3862
[2]:
Feb 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/15/2016 3:11 PM, ZombineDev wrote:

 I'll leave to others to discuss whether a language solution is worth it, but I
 just wanted to point out there are two library solutions in this area:
 HeadConst [1] - PR pending, and
 Rebindable (tail const) [2] - already part of Phobos.

 Maybe if the language could express head const, it also makes sense to have
tail
 const.

 [1]: https://github.com/D-Programming-Language/phobos/pull/3862



Transitive const is tail const.
Feb 15 2016
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 15 February 2016 at 23:28:28 UTC, Walter Bright wrote:

 Transitive const is tail const.


const(char)[] foo is tail const

const(Object) ref is the analog the language is missing so phobos 
tries to catch up
Feb 15 2016
prev sibling parent reply ZombineDev <valid_email he.re> writes:
On Monday, 15 February 2016 at 23:28:28 UTC, Walter Bright wrote:

 On 2/15/2016 3:11 PM, ZombineDev wrote:

 I'll leave to others to discuss whether a language solution is 
 worth it, but I
 just wanted to point out there are two library solutions in 
 this area:
 HeadConst [1] - PR pending, and
 Rebindable (tail const) [2] - already part of Phobos.

 Maybe if the language could express head const, it also makes 
 sense to have tail
 const.

 [1]: https://github.com/D-Programming-Language/phobos/pull/3862
 [2]: 



 Transitive const is tail const.


I used the term "tail const" as a mutable pointer/reference to 
const/immutable object, where transitivity starts from the object 
(not from the pointer, which is what currently happens if you 
type `const Object o`).
Feb 15 2016
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/15/2016 3:39 PM, ZombineDev wrote:

 I used the term "tail const" as a mutable pointer/reference to const/immutable
 object, where transitivity starts from the object (not from the pointer, which
 is what currently happens if you type `const Object o`).


Makes sense.
Feb 15 2016
prev sibling parent reply ZombineDev <valid_email he.re> writes:
On Monday, 15 February 2016 at 23:39:56 UTC, ZombineDev wrote:

 On Monday, 15 February 2016 at 23:28:28 UTC, Walter Bright 
 wrote:

 On 2/15/2016 3:11 PM, ZombineDev wrote:

 I'll leave to others to discuss whether a language solution 
 is worth it, but I
 just wanted to point out there are two library solutions in 
 this area:
 HeadConst [1] - PR pending, and
 Rebindable (tail const) [2] - already part of Phobos.

 Maybe if the language could express head const, it also makes 
 sense to have tail
 const.

 [1]: 
 https://github.com/D-Programming-Language/phobos/pull/3862
 [2]: 



 Transitive const is tail const.


 I used the term "tail const" as a mutable pointer/reference to 
 const/immutable object, where transitivity starts from the 
 object (not from the pointer, which is what currently happens 
 if you type `const Object o`).


Just to clarify:
struct S;
const(S)* mutablePointerToS;
const(S)[] mutableArrayOfConstObjects;
Are examles of tail const. The only missing piece is the ability 
to have a tail-const references to class objects.
Feb 15 2016
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Mon, Feb 15, 2016 at 11:45:26PM +0000, ZombineDev via Digitalmars-d wrote:
[...]

I used the term "tail const" as a mutable pointer/reference to
const/immutable object, where transitivity starts from the object
(not from the pointer, which is what currently happens if you type
`const Object o`).


 
 Just to clarify:
 struct S;
 const(S)* mutablePointerToS;
 const(S)[] mutableArrayOfConstObjects;

 Are examles of tail const. The only missing piece is the ability to
 have a tail-const references to class objects.


Just out of curiosity, since we already have Rebindable in Phobos and
HeadConst is being proposed, what are the disadvantages / shortcomings
of a library solution that would justify adding yet another feature to
the language?


T

-- 
Today's society is one of specialization: as you grow, you learn more
and more about less and less. Eventually, you know everything about
nothing.
Feb 15 2016
next sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Mon, 15 Feb 2016 16:14:23 -0800, H. S. Teoh via Digitalmars-d wrote:

 Just out of curiosity, since we already have Rebindable in Phobos and
 HeadConst is being proposed, what are the disadvantages / shortcomings
 of a library solution that would justify adding yet another feature to
 the language?


C++ interfacing requires either a well-known type that the compiler 
special-cases or a builtin attribute. D could ignore this for all 
purposes besides name mangling, though, and that would be an okay 
compromise.
Feb 15 2016
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Feb 16, 2016 at 12:21:38AM +0000, Chris Wright via Digitalmars-d wrote:

 On Mon, 15 Feb 2016 16:14:23 -0800, H. S. Teoh via Digitalmars-d wrote:

 Just out of curiosity, since we already have Rebindable in Phobos
 and HeadConst is being proposed, what are the disadvantages /
 shortcomings of a library solution that would justify adding yet
 another feature to the language?


 
 C++ interfacing requires either a well-known type that the compiler
 special-cases or a builtin attribute. D could ignore this for all
 purposes besides name mangling, though, and that would be an okay
 compromise.


What about besides C++ integration?  'cos I remember some people were
complaining that a library solution is bad, but I've forgotten what the
reasons were.

My thinking is that if the current library solution is bad, or messy, or
whatever, and other such library solutions exhibit similar problems,
then perhaps what we should be looking at is how to make the language
more conducive to implementing nice, usable library solutions to this
and a host of other problems, rather than throwing everything and the
kitchen sink into the language and end up with an unmaintainable mess.


T

-- 
This sentence is false.
Feb 15 2016
parent Chris Wright <dhasenan gmail.com> writes:
On Mon, 15 Feb 2016 16:57:44 -0800, H. S. Teoh via Digitalmars-d wrote:


 What about besides C++ integration?  'cos I remember some people were
 complaining that a library solution is bad, but I've forgotten what the
 reasons were.


The first problem mentioned was C++ integration, and the minimal thing 
required to make that work in a nice way is to supply a  cppconst 
attribute with no semantics that is used only to alter name mangling. 
It's the least complex solution available.

I did get tripped up not too long ago on a lack of a `final` storage 
class in the context of members. It is sometimes useful.

There's also a question of whether we want the same thing for C++ name 
mangling as for preventing reassignment. It might be clearer to have both 
final and  cppconst.
Feb 15 2016
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 16.02.2016 01:14, H. S. Teoh via Digitalmars-d wrote:

 Just out of curiosity, since we already have Rebindable in Phobos and
 HeadConst is being proposed, what are the disadvantages / shortcomings
 of a library solution that would justify adding yet another feature to
 the language?


struct S{
     int[] a;
     void foo()headconst{
         a[0]=1; // ok
         // arr.length=2 // error
     }
}

void main(){
     headconst(S) s([0,2,3]);
     s.foo();
     assert(s.a==[1,2,3]);
}

How to do this in the library?
Feb 16 2016
parent Nick Treleaven <ntrel-pub mybtinternet.com> writes:
On Tuesday, 16 February 2016 at 10:32:45 UTC, Timon Gehr wrote:

 struct S{
     int[] a;
     void foo()headconst{
         a[0]=1; // ok
         // arr.length=2 // error
     }
 }

 void main(){
     headconst(S) s([0,2,3]);
     s.foo();
     assert(s.a==[1,2,3]);
 }

 How to do this in the library?


Maybe make foo a free function:

struct S{
     int[] a;
}

void foo(HConst!S self){
     self.a[0]=1; // ok
     self.arr.length=2 // no effect
}

Haven't tried, but maybe HConst can allow access to fields of S 
by value, not ref. That could prevent mutation like C++ const, 
but it wouldn't disallow mutation of the temporary copy. Not sure 
if that's good enough. I suppose the compiler could warn about 
unused temporary mutation.
Feb 16 2016
prev sibling next sibling parent Laeeth Isharc <laeethnospam nospam.laeeth.com> writes:
On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote:

 rears its head again :-)

 Head Const is what C++ has for const, i.e. it is not 
 transitive, applies to one level only. D has transitive const.

 What head const will do for us:

 1. make it easy to interface to C++ code that uses const, as 
 currently it is not very practical to do so, you have to resort 
 to pragma(mangle)

 2. supports single assignment style of programming, even if the 
 data is otherwise mutable

 The downside is, of course, language complexity.


so keep const as it is and add headconst?
Feb 15 2016
prev sibling next sibling parent reply Daniel Murphy <yebbliesnospam gmail.com> writes:
On 16/02/2016 9:48 AM, Walter Bright wrote:

 rears its head again :-)

 Head Const is what C++ has for const, i.e. it is not transitive, applies
 to one level only. D has transitive const.

 What head const will do for us:

 1. make it easy to interface to C++ code that uses const, as currently
 it is not very practical to do so, you have to resort to pragma(mangle)


I'd much rather improve pragma(mangle) than add more C++ features to D.


 2. supports single assignment style of programming, even if the data is
 otherwise mutable


Like 'final'?  We did get rid of that...
Feb 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/15/2016 4:15 PM, Daniel Murphy wrote:

 1. make it easy to interface to C++ code that uses const, as currently
 it is not very practical to do so, you have to resort to pragma(mangle)


 I'd much rather improve pragma(mangle) than add more C++ features to D.


It's currently difficult to interface with C++, and always will be, but 
smoothing out what we can can be a big opportunity for D.



 2. supports single assignment style of programming, even if the data is
 otherwise mutable


 Like 'final'?  We did get rid of that...


Maybe we should resurrect it.
Feb 15 2016
next sibling parent rsw0x <anonymous anonymous.com> writes:
On Tuesday, 16 February 2016 at 01:04:44 UTC, Walter Bright wrote:

 On 2/15/2016 4:15 PM, Daniel Murphy wrote:

 1. make it easy to interface to C++ code that uses const, as 
 currently
 it is not very practical to do so, you have to resort to 
 pragma(mangle)


 I'd much rather improve pragma(mangle) than add more C++ 
 features to D.


 It's currently difficult to interface with C++, and always will 
 be, but smoothing out what we can can be a big opportunity for 
 D.


Hmm, I wonder if Stroustrup said something similar.
Feb 15 2016
prev sibling next sibling parent reply Daniel Murphy <yebbliesnospam gmail.com> writes:
On 16/02/2016 12:04 PM, Walter Bright wrote:

 It's currently difficult to interface with C++, and always will be, but
 smoothing out what we can can be a big opportunity for D.


I'm starting to think we should give up on implementing C++ support in 
the language and move it to the library.

eg

mixin(cppFunctionBinding("unsigned long long NameSpace::myFunc(char * 
const && x, long double y)");

expands to some combination of pragma(mangle) and extern(C++)

With the limitation that only declarations can be parsed, it's not that 
bad to implement a ctfe C++ parser, and we can stop the creep of C++ 
features and hack into D.

As a bonus, it could generate wrappers when we really can't match the 
semantics well enough.
Feb 16 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 16 February 2016 at 09:04:48 UTC, Daniel Murphy wrote:

 I'm starting to think we should give up on implementing C++ 
 support in the language and move it to the library.

 eg

 mixin(cppFunctionBinding("unsigned long long 
 NameSpace::myFunc(char * const && x, long double y)");

 expands to some combination of pragma(mangle) and extern(C++)

 With the limitation that only declarations can be parsed, it's 
 not that bad to implement a ctfe C++ parser, and we can stop 
 the creep of C++ features and hack into D.

 As a bonus, it could generate wrappers when we really can't 
 match the semantics well enough.


I agree with the principle, but not as a library function, 
because:

1. you want virtual functions to work out ok

2. making D more reliant on macroish string processing is not good


You would need something along the lines of:

1. «extern "C++"» the essence of the class definition in plain 
C++ syntax

2. add to this syntax a translation for each parameter what it 
means in D.


E.g.

extern "C++" {

class X {
   mutable int rc;
   virtual func1(const A* ptr);  reinterpret(ptr, head_const_ptr!A)
   virtual func2(const A* ptr);  reinterpret(ptr, const A*)
   virtual func3(A* ptr);
   virtual func4(const A* ptr);  reinterpret(ptr, const_rc!A*)
};

}
Feb 16 2016
parent reply Daniel Murphy <yebbliesnospam gmail.com> writes:
On 16/02/2016 8:29 PM, Ola Fosheim Grøstad wrote:

 I agree with the principle, but not as a library function, because:

 1. you want virtual functions to work out ok


virtual functions don't even need mangling.  But even if they did it 
would work just fine anyway.


 2. making D more reliant on macroish string processing is not good


It's not macroish string processing, it's embedding a subset of C++ 
declarations like a DSL.  The difference is that the C++ can be fully 
type-checked and semantically analysed, errors will not leak into the 
generated source.


 You would need something along the lines of:

 1. «extern "C++"» the essence of the class definition in plain C++ syntax

 2. add to this syntax a translation for each parameter what it means in D.


 E.g.

 extern "C++" {

 class X {
    mutable int rc;
    virtual func1(const A* ptr);  reinterpret(ptr, head_const_ptr!A)
    virtual func2(const A* ptr);  reinterpret(ptr, const A*)
    virtual func3(A* ptr);
    virtual func4(const A* ptr);  reinterpret(ptr, const_rc!A*)
 };

 }


We don't 'need' compiler support beyond what we have, for any of this.
Feb 17 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Thursday, 18 February 2016 at 06:49:34 UTC, Daniel Murphy 
wrote:

 It's not macroish string processing, it's embedding a subset of 
 C++ declarations like a DSL.  The difference is that the C++ 
 can be fully type-checked and semantically analysed, errors 
 will not leak into the generated source.


Ok, if you can resolve everything this way.

I agree that you end up spending way too much effort on bringing 
C++ into D, for it to be meaningful. My headerfiles used to be 
like 10-20% of my C++ source. Now it is at 40-50% of my C++ 
source... My C++ library code is at 90% header file only. Things 
like concepts will probably just ensure that this trend persists.

So I am not sure if interfacing with compiled C++ only is 
particularly useful. I think you need support for generating C++ 
source code stubs for interfacing so that you can use inlined 
non-instantiated C++ code (or just stick to C).

LLVM whole program optimization (linking LLVM IR) would "melt" it 
all together reasonably efficiently.
Feb 18 2016
prev sibling parent reply Guillaume Piolat <contact gam3sfrommars.fr> writes:
On Tuesday, 16 February 2016 at 01:04:44 UTC, Walter Bright wrote:

 2. supports single assignment style of programming, even if 
 the data is
 otherwise mutable


 Like 'final'?  We did get rid of that...


 Maybe we should resurrect it.


I'm trying to say it politely.
D2 const story is more complicated than its competitors.

Both D1 "final" and C++ const always felt more useful and 
practical to me that the whole D2 immutable/const/inout thing. 
The current scheme seems to have marginal value in practice, lots 
of complexity, and is harder to use well (Unqual, inout) etc. 
Constructors can break it. I don't know why we should be that 
happy about our constness, maybe someone can explain.
Feb 16 2016
next sibling parent Shachar Shemesh <shachar weka.io> writes:
On 16/02/16 12:31, Guillaume Piolat wrote:

 I'm trying to say it politely.
 D2 const story is more complicated than its competitors.

 Both D1 "final" and C++ const always felt more useful and practical to
 me that the whole D2 immutable/const/inout thing. The current scheme
 seems to have marginal value in practice, lots of complexity, and is
 harder to use well (Unqual, inout) etc. Constructors can break it. I
 don't know why we should be that happy about our constness, maybe
 someone can explain.


Seconded. It is one of the main tools I was using in C++ to avoid 
introducing bugs that I find completely unusable in D.

I've tried to introduce it into code I'm writing. I'm used to such 
things being a problem to add later for code that was not written this 
way in advance. That's the case in C++ too. Here, however, there are 
many cases where I simply couldn't get it to the finish line. Between 
not being able to have head const, needing to mutate vars in a const 
object (a la refcount) and the compiler deciding functions are pure, I 
often end up facing a problem where I say "okay, forget it".

And that's a real shame. We have at least one case where we're 
introducing runtime checks to make sure no one is accidentally changing 
a global var put there for reference, where declaring it const would 
have provided compile time assurance that we're okay.

Shachar
Feb 17 2016
prev sibling parent reply rsw0x <anonymous anonymous.com> writes:
On Tuesday, 16 February 2016 at 10:31:05 UTC, Guillaume Piolat 
wrote:

 On Tuesday, 16 February 2016 at 01:04:44 UTC, Walter Bright 
 wrote:

 2. supports single assignment style of programming, even if 
 the data is
 otherwise mutable


 Like 'final'?  We did get rid of that...


 Maybe we should resurrect it.


 I'm trying to say it politely.
 D2 const story is more complicated than its competitors.

 Both D1 "final" and C++ const always felt more useful and 
 practical to me that the whole D2 immutable/const/inout thing. 
 The current scheme seems to have marginal value in practice, 
 lots of complexity, and is harder to use well (Unqual, inout) 
 etc. Constructors can break it. I don't know why we should be 
 that happy about our constness, maybe someone can explain.


+1

It's weird because usually D prefers the practical solution over 
the ivory tower one. Nearly every time I end up using immutable 
or const for anything beyond say, a trivial function parameter, I 
always end up removing it.
My C++ code is often far, far more 'const correct' than my D code.
Feb 17 2016
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, 17 February 2016 at 17:25:40 UTC, rsw0x wrote:

 It's weird because usually D prefers the practical solution 
 over the ivory tower one. Nearly every time I end up using 
 immutable or const for anything beyond say, a trivial function 
 parameter, I always end up removing it.
 My C++ code is often far, far more 'const correct' than my D 
 code.


Well, Walter's take on it is that because C++'s const doesn't 
actually guarantee anything, it really isn't much different from 
simply documenting your code, and it's essentially useless. And 
to a point, he's right. All C++'s const does is prevent 
accidental mutation and serve as documentation that the code 
isn't supposed to mutate the logical state of the object. It's 
trivially circumvented to the point that it doesn't actually 
guarantee anything and can't actually be relied on for anything.

So, essentially the argument is that C++'s const isn't actually 
practical. It's just fooling you into thinking that you're 
getting guarantees that you're not actually getting.

But in practice, it does prevent accidental mutation, and as long 
as the programmer is behaving themselves reasonably well, it 
functions as documentation. And IMHO that definitely is worth 
something, much as Walter thinks that it isn't. So, while C++'s 
const doesn't really guarantee much, I'd much rather have it than 
nothing.

A lot of what it comes down to though from a purely practical 
standpoint is that because we have immutable, const naturally 
becomes more restrictive, and while immutable obviously can't be 
used in many cases, it _does_ allow for cleaner, more efficient 
code where it can be used. So, as soon as you have immutable, 
you're almost forced to go to where D is with const - at least as 
long as const can refer to something that's immutable.

Unfortunately, the net result is that while const is still very 
useful, there are a lot of cases in D where you can't use it, and 
you have to be a lot more careful about how and when you use it 
such that it gets used far less than in C++, and while where it 
_is_ used does provide much better guarantees, you lose out on a 
lot of protection against accidental mutation that your typical 
C++ code gets.

Personally, I'm quite divided on the matter. The extra guarantees 
of D's const are very nice to have, but since it doesn't have any 
backdoors, there are a number of idioms that simply can't be done 
in D as long as const is involved, which definitely sucks. But 
interestingly, the more I've used D, the less happy I've been 
with C++'s const. It just has too many holes. In particular, the 
fact that it's not transitive is incredibly annoying, making it 
trivial to have cases where you give mutable access to something 
where you intended for it to be const, but it was only partially 
const. Even if we were to do something like add  mutable to D, I 
never want to see non-transitive const in D, and I wish that 
C++'s const were transitive, even if all of the rest of it was 
the same.

- Jonathan M Davis
Feb 17 2016
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 17 February 2016 at 17:47:02 UTC, Jonathan M Davis 
wrote:

 definitely sucks. But interestingly, the more I've used D, the 
 less happy I've been with C++'s const. It just has too many 
 holes. In particular, the fact that it's not transitive is


What you could consider in C++ is to fully encapsulate your field 
values and use member functions to access them. Which is 
considered good modelling anyway.

So if you access a member as "get_child() const", you return a 
const reference, which makes it transitive.

What is annoying in C++ is that there is no getter/setter 
syntax...
Feb 17 2016
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, 17 February 2016 at 18:16:54 UTC, Ola Fosheim 
Grøstad wrote:

 On Wednesday, 17 February 2016 at 17:47:02 UTC, Jonathan M 
 Davis wrote:

 definitely sucks. But interestingly, the more I've used D, the 
 less happy I've been with C++'s const. It just has too many 
 holes. In particular, the fact that it's not transitive is


 What you could consider in C++ is to fully encapsulate your 
 field values and use member functions to access them. Which is 
 considered good modelling anyway.

 So if you access a member as "get_child() const", you return a 
 const reference, which makes it transitive.


As soon as you start using pointers or containers or pointers or 
anything like that, pretty quickly, you can get situations like 
where the container is const, but its elements aren't. So, you 
may not be able to mutate the stuff that's directly a member of 
the class, you can still mutate stuff that it refers to or even 
owns. D doesn't allow that, which I think is a definite 
improvement. If you return a reference or pointer to a member 
variable from a const member function in D, it's const through 
and through with no holes, and I think that that is unequivocally 
a good thing.

Where things get annoying is the type of situation where you'd 
mark something mutable in C++, since those just don't work at all 
in D. You're forced to not use const or redesign what you're 
doing so that it doesn't need mutable (which usually means not 
using const).

- Jonathan M Davis
Feb 17 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 17 February 2016 at 19:03:25 UTC, Jonathan M Davis 
wrote:

 As soon as you start using pointers or containers or pointers 
 or anything like that, pretty quickly, you can get situations 
 like where the container is const, but its elements aren't.


Yes, generic containers is an issue in C++. You can't just recast 
the template parameter as const, as the internals could depend on 
it not being const when the container was instantiated (if-test 
on const in the constructor). This is a problem we get when 
meta-programming becomes powerful enough.

The easy way out is to use a wrapper and implement the constness 
in the wrapper (a class with just a pointer and methods). 
Basically implement a reference type for container access.
Feb 17 2016
prev sibling parent Laeeth Isharc <laeethnospam laeethnospam.com> writes:
On Wednesday, 17 February 2016 at 17:47:02 UTC, Jonathan M Davis 
wrote:

 On Wednesday, 17 February 2016 at 17 you're not actually 
 getting.

 But in practice, it does prevent accidental mutation, and as 
 long as the programmer is behaving themselves reasonably well,



 Unfortunately, the net result is that while const is still very 
 useful, there are a lot of cases in D where you can't use it, 
 and you have to be a lot more careful about how and when you 
 use it such that it gets used far less than in C++, and while 
 where it _is_ used does provide much better guarantees, you 
 lose out on a lot of protection against accidental mutation 
 that your typical C++ code gets.


Things would be a bit friendlier for newcomers if the 
documentation were a bit clearer on the 'having to be a lot more 
careful on how and when you use it' so one doesnt end up finding 
this out the hard way.
Feb 17 2016
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote:

 rears its head again :-)

 Head Const is what C++ has for const, i.e. it is not 
 transitive, applies to one level only. D has transitive const.

 What head const will do for us:

 1. make it easy to interface to C++ code that uses const, as 
 currently it is not very practical to do so, you have to resort 
 to pragma(mangle)

 2. supports single assignment style of programming, even if the 
 data is otherwise mutable

 The downside is, of course, language complexity.


Straight up head-const is utterly useless IMHO. That's what Java 
has. C++ has something far more complicated where any individual 
piece (or pieces) of a pointer type can be declared const without 
necessarily making the whole thing const. So, _that_ is what we'd 
need to interact with C++, and it's a downright ugly feature 
IMHO. Tail-const has some value, because it allows you to refer 
to a const object with a mutable pointer or reference, whereas 
head-const just makes things ugly. So, if it weren't for 
interacting with C++, I would give a resounding "no" to this. 
It's not even vaguely worth the complexity.

As for interoperability with C++, I don't know. Previously, you 
stated that we weren't going to do things like support 
interacting with C++ templates (aside from specific 
instantiations which weren't typed as templates), because it 
would mean putting a C++ compiler into D, which you didn't want 
to do. But increasingly, it seems like you're heading in the 
direction of doing that in an attempt to be able to have 
fantastic C++ interoperability. On the one on hand, that seems 
great, since being able to have your C++ code work with your D 
code is great, but on the other, it seems like it's going to make 
it so that D is contaminated by a lot of extra C++ muck just to 
be able to interoperate. At some point, we either need to decide 
that we're just not going to interoperate with C++ in some manner 
and lose out on some capability, or we're going to need to fully 
interoperate with C++ and pretty much put a C++ compiler in the D 
compiler, and I'd prefer that we didn't go that far.

In this particular case, doesn't this really just come down to 
mangling? It's undefined behavior in D to mutate a const object 
(even if it was constructed as mutable), so we can't just slap D 
const on C++ types and have that work, since the C++ code could 
legally mutate the object. But since C++ considers it defined 
behavior to mutate a const object by casting away const (or at 
least it does in all but some very specific cases), can we just 
get away with the D code treating D const as mutable? If so, then 
it's purely a matter of mangling. And for that, we could do 
something like add  cppconst or cppconst that the compiler 
recognizes and which is only valid in extern(C++) declarations. 
It would unfortunately have to be more than a simple attribute, 
because it would have to apply to parts of a parameter's type 
like C++'s const does (instead of the whole type at once), but if 
it were implemented such that it was just something that went on 
the extern(C++) function parameter types (or return types) for 
mangling purposes, and the D code just treated it as mutable, 
then it would at least restrict the muck to extern(C++).

Now, that does have the downside that the D code has no 
protection against mutating a C++ object that the C++ code marks 
as const (whereas while the C++ code can mutate, you at least 
have to cast away const or use the mutable keyword first), but 
that seems a whale of a lot better to me than trying to add 
non-transitive const to D.

- Jonathan M Davis
Feb 15 2016
next sibling parent reply w0rp <devw0rp gmail.com> writes:
I think the point about name mangling is very true. That's the 
most important thing, being able to call all of the C++ functions.

I personally love that const and immutable are transitive in D. I 
get annoyed in other languages when I have const objects 
containing mutable objects, and no real protection from the 
compiler from stopping the objects in the inside from being 
mutated. The way I see it, C++ const is a weaker form of const 
which doesn't complain about mutations enough.

I think if we could avoid putting head const in the language, 
we'd be better off. If you ever really, really need to mutate 
something inside a const type, you can cast away const. It will 
rightly look ugly, and the compiler can complain that it's not 
 safe.
Feb 15 2016
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, 16 February 2016 at 07:46:11 UTC, w0rp wrote:

 I think if we could avoid putting head const in the language, 
 we'd be better off. If you ever really, really need to mutate 
 something inside a const type, you can cast away const. It will 
 rightly look ugly, and the compiler can complain that it's not 
  safe.


Actually, in D, it's undefined behavior to cast away const and 
mutate. D's const is supposed to guarantee that the data cannot 
be mutated via a const reference to the data. Given that very few 
optimizations can be made based solely on const (and I'm not sure 
that the compiler currently does any of them), it will almost 
certainly work as long as the underlying data is mutable rather 
than immutable, but it _is_ undefined behavior, and you're 
violating the guarantees that the type system provides when you 
go around it like that.

- Jonathan M Davis
Feb 16 2016
parent reply ZombineDev <valid_email he.re> writes:
On Tuesday, 16 February 2016 at 08:03:31 UTC, Jonathan M Davis 
wrote:

 ...
 Given that very few optimizations can be made based solely on 
 const (and I'm not sure that the compiler currently does any of 
 them) ...


I think that the fact that we have shared can help a lot in the 
area of optimization.
Taking a non-shared const parameter should be as good as taking 
an immutable one (if the function pure-like - only does 
computation based on the parameter and doesn't escape it).
Or am I missing something?
Feb 16 2016
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, 16 February 2016 at 09:45:34 UTC, ZombineDev wrote:

 On Tuesday, 16 February 2016 at 08:03:31 UTC, Jonathan M Davis 
 wrote:

 ...
 Given that very few optimizations can be made based solely on 
 const (and I'm not sure that the compiler currently does any 
 of them) ...


 I think that the fact that we have shared can help a lot in the 
 area of optimization.
 Taking a non-shared const parameter should be as good as taking 
 an immutable one (if the function pure-like - only does 
 computation based on the parameter and doesn't escape it).
 Or am I missing something?


The fact that variables are thread-local by default in D 
definitely helps, but it's still far from sufficient. If you have

     auto result = func(foo);

where foo is const, for the compiler to know that foo was not 
mutated by the call to func, it has to know that there's no way 
that func could have accessed a mutable reference to the object 
that foo refers to. The fact that we have shared means that it 
doesn't have to worry about other, arbitrary code mutating foo 
while func is called, but it still has to be able to determine 
that neither func no anything that it accesses is able to mutate 
foo.

If func is pure, that gets us a good chunk of the way, because 
then the only way that func can access a mutable reference to foo 
is via its arguments. If it's just foo, then - unless I'm missing 
something - that should be enough, since foo would not be able to 
access a mutable reference to itself except via a non-pure 
function, which wouldn't be callable from func if it's pure. But 
if func is a member function, or if it took addition, mutable 
arguments, e.g.

     auto result = func(foo, bar);

then the compiler has to know that func can't access a mutable 
foo via the other arguments - and if they're not const, that's 
hard to do.

In general, I think that the compiler pretty much has to be able 
to see where foo was created and that no mutable references _can_ 
exist that func would have access to. e.g.

     const foo = new Foo(42);
     auto result = func(foo, bar);

or

     auto mutableFoo = new Foo(42);
     const foo = mutableFoo;
     auto result = func(foo, bar);

would be enough. But without a lot of code flow analysis, it 
doesn't take much for it to be too much for the compiler to 
figure out, and unless it has access to the bodies of the 
functions that are being called and does deep inspection of what 
they do and call, simply calling a function (especially a 
non-pure one) very quickly makes it so that the compiler doesn't 
know enough to guarantee that func can't possibly access foo via 
a mutable reference to the same data. e.g. in

     auto mutableFoo = new Foo(42);
     auto result1 = otherFunc(mutableFoo, bar);
     const foo = mutableFoo;
     auto result2 = func(foo, bar);

otherFunc might have given bar access to mutableFoo which it then 
accessed inside of func.

So, yes, the combination of non-shared, pure, and const does 
allow for some optimizations, but it really doesn't take much to 
make it so that the compiler can't guarantee that const object 
hasn't been changed by a function call without it doing a lot of 
deep analysis that compilers just don't normally do.

- Jonathan M Davis
Feb 16 2016
parent ZombineDev <valid_email he.re> writes:
On Tuesday, 16 February 2016 at 10:09:19 UTC, Jonathan M Davis 
wrote:

 On Tuesday, 16 February 2016 at 09:45:34 UTC, ZombineDev wrote:

 On Tuesday, 16 February 2016 at 08:03:31 UTC, Jonathan M Davis 
 wrote:

 ...
 Given that very few optimizations can be made based solely on 
 const (and I'm not sure that the compiler currently does any 
 of them) ...


 I think that the fact that we have shared can help a lot in 
 the area of optimization.
 Taking a non-shared const parameter should be as good as 
 taking an immutable one (if the function pure-like - only does 
 computation based on the parameter and doesn't escape it).
 Or am I missing something?


 The fact that variables are thread-local by default in D 
 definitely helps, but it's still far from sufficient. If you 
 have

     auto result = func(foo);

 where foo is const, for the compiler to know that foo was not 
 mutated by the call to func, it has to know that there's no way 
 that func could have accessed a mutable reference to the object 
 that foo refers to. The fact that we have shared means that it 
 doesn't have to worry about other, arbitrary code mutating foo 
 while func is called, but it still has to be able to determine 
 that neither func no anything that it accesses is able to 
 mutate foo.

 If func is pure, that gets us a good chunk of the way, because 
 then the only way that func can access a mutable reference to 
 foo is via its arguments. If it's just foo, then - unless I'm 
 missing something - that should be enough, since foo would not 
 be able to access a mutable reference to itself except via a 
 non-pure function, which wouldn't be callable from func if it's 
 pure. But if func is a member function, or if it took addition, 
 mutable arguments, e.g.

     auto result = func(foo, bar);

 then the compiler has to know that func can't access a mutable 
 foo via the other arguments - and if they're not const, that's 
 hard to do.

 In general, I think that the compiler pretty much has to be 
 able to see where foo was created and that no mutable 
 references _can_ exist that func would have access to. e.g.

     const foo = new Foo(42);
     auto result = func(foo, bar);

 or

     auto mutableFoo = new Foo(42);
     const foo = mutableFoo;
     auto result = func(foo, bar);

 would be enough. But without a lot of code flow analysis, it 
 doesn't take much for it to be too much for the compiler to 
 figure out, and unless it has access to the bodies of the 
 functions that are being called and does deep inspection of 
 what they do and call, simply calling a function (especially a 
 non-pure one) very quickly makes it so that the compiler 
 doesn't know enough to guarantee that func can't possibly 
 access foo via a mutable reference to the same data. e.g. in

     auto mutableFoo = new Foo(42);
     auto result1 = otherFunc(mutableFoo, bar);
     const foo = mutableFoo;
     auto result2 = func(foo, bar);

 otherFunc might have given bar access to mutableFoo which it 
 then accessed inside of func.

 So, yes, the combination of non-shared, pure, and const does 
 allow for some optimizations, but it really doesn't take much 
 to make it so that the compiler can't guarantee that const 
 object hasn't been changed by a function call without it doing 
 a lot of deep analysis that compilers just don't normally do.

 - Jonathan M Davis


I agree with everything you said. What I meant was that **in 
theory** const + thread-local can give similar guarantees as 
immutable (if the compiler can perform the necessary analysis).
That means it's more of an implementation problem than a langauge 
problem.
Given that separate compilation and .di header-only libraries are 
far more rare in D that in C++ (D has faster compilation and 
templates with attribute inference provide even more incentive), 
whole program optimization seems more easily doable in D.
After all the Rust compiler has proven that immutable borrowing 
(of a mutable object) is not hard to enforce, so I don't see a 
hard obstacle for this to be implemented in D. At least not an 
obstacle comming from the language design.
Feb 16 2016
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/15/2016 10:04 PM, Jonathan M Davis wrote:

 Straight up head-const is utterly useless IMHO. That's what Java has. C++ has
 something far more complicated where any individual piece (or pieces) of a
 pointer type can be declared const without necessarily making the whole thing
 const. So, _that_ is what we'd need to interact with C++, and it's a downright
 ugly feature IMHO.


That's right. 'headconst' cannot be just a storage class, it has to be a type 
constructor.

You're right that mangling is the big issue with C++ interoperability in
regards 
to const.
Feb 16 2016
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, 16 February 2016 at 06:04:42 UTC, Jonathan M Davis 
wrote:

 On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright 
 wrote:

 rears its head again :-)

 Head Const is what C++ has for const, i.e. it is not 
 transitive, applies to one level only. D has transitive const.


 In this particular case, doesn't this really just come down to 
 mangling? It's undefined behavior in D to mutate a const object 
 (even if it was constructed as mutable), so we can't just slap 
 D const on C++ types and have that work, since the C++ code 
 could legally mutate the object. But since C++ considers it 
 defined behavior to mutate a const object by casting away const 
 (or at least it does in all but some very specific cases), can 
 we just get away with the D code treating D const as mutable? 
 If so, then it's purely a matter of mangling. And for that, we 
 could do something like add  cppconst or cppconst that the 
 compiler recognizes and which is only valid in extern(C++) 
 declarations. It would unfortunately have to be more than a 
 simple attribute, because it would have to apply to parts of a 
 parameter's type like C++'s const does (instead of the whole 
 type at once), but if it were implemented such that it was just 
 something that went on the extern(C++) function parameter types 
 (or return types) for mangling purposes, and the D code just 
 treated it as mutable, then it would at least restrict the muck 
 to extern(C++).


Actually, one could argue that the fact that we support const at 
all with other languages - even with C - is broken in that C/C++ 
code can cast away our const and mutate the variable we passed 
in, breaking our type system. We rely on the programmer to know 
that the C/C++ code isn't going to do that, and most of the time, 
you really have no clue. You just assume that it won't - and it 
probably won't, but you don't know for sure.

So, most of the time, it will work just fine, but const with 
extern(C) and extern(C++) really does not provide the guarantees 
that const is supposed to provide.

Of course, there's also the issue that with extern(C) and 
extern(C++), everything is really __gshared, and yet we typically 
treat  it like it's thread-local just like the D code is. 
Arguably, all parameters and return types with extern(C) and 
extern(C++) should be automatically treated as shared, and not 
doing so is risking a world of pain if the C/C++ code misbehaves, 
but having them be treated as shared would be really annoying to 
deal with, and most C/C++ code that's being called probably isn't 
doing anything with threads that could violate D's guarantees 
about thread-local variables (just like most C/C++ code 
presumably isn't casting away const and mutating when we pass it 
something from D) - but there's no guarantee that it isn't.

So, as far as compiler guarantees and the type system go, 
extern(C) and extern(C++) are a bit of a disaster, much as they 
generally work in practice. But I don't know what we can really 
do about that other than trying to educate people about the risks 
and possibly have the compiler avoid certain types of 
optimizations around extern(C) and extern(C++) code (which it may 
already do).

- Jonathan M Davis
Feb 16 2016
prev sibling parent reply karabuta <karabutaworld gmail.com> writes:
On Tuesday, 16 February 2016 at 06:04:42 UTC, Jonathan M Davis 
wrote:

 On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright 
 wrote:

 rears its head again :-)

 Head Const is what C++ has for const, i.e. it is not 
 transitive, applies to one level only. D has transitive const.

 What head const will do for us:

 1. make it easy to interface to C++ code that uses const, as 
 currently it is not very practical to do so, you have to 
 resort to pragma(mangle)

 2. supports single assignment style of programming, even if 
 the data is otherwise mutable

 The downside is, of course, language complexity.





 Previously, you stated that we weren't going to do things like 
 support interacting with C++ templates (aside from specific 
 instantiations which weren't typed as templates), because it 
 would mean putting a C++ compiler into D, which you didn't want 
 to do. But increasingly, it seems like you're heading in the 
 direction of doing that in an attempt to be able to have 
 fantastic C++ interoperability. On the one on hand, that seems 
 great, since being able to have your C++ code work with your D 
 code is great, but on the other, it seems like it's going to 
 make it so that D is contaminated by a lot of extra C++ muck 
 just to be able to interoperate. At some point, we either need 
 to decide that we're just not going to interoperate with C++ in 
 some manner and lose out on some capability, or we're going to 
 need to fully interoperate with C++ and pretty much put a C++ 
 compiler in the D compiler, and I'd prefer that we didn't go 
 that far.

 - Jonathan M Davis


Hahaha. Well, I think it is already happening. Like the 
reincarnation of C to C++ story.
Feb 16 2016
parent rsw0x <anonymous anonymous.com> writes:
On Tuesday, 16 February 2016 at 18:06:05 UTC, karabuta wrote:

 Hahaha. Well, I think it is already happening. Like the 
 reincarnation of C to C++ story.


The focus on interfacing D with C++ lately has been very 
disconcerting, especially considering features from TDPL are 
still unfinished nearly 6 years later.
Feb 16 2016
prev sibling next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote:

 rears its head again :-)

 Head Const is what C++ has for const, i.e. it is not 
 transitive, applies to one level only. D has transitive const.

 What head const will do for us:

 1. make it easy to interface to C++ code that uses const, as 
 currently it is not very practical to do so, you have to resort 
 to pragma(mangle)

 2. supports single assignment style of programming, even if the 
 data is otherwise mutable

 The downside is, of course, language complexity.


Maybe you can get away with adding "mutable", which is needed for 
intrusive ref counting as well:

C++:
Type * const ptr;

struct ConstType {
mutable int rc;
}


D:
const mutable(Type)* ptr;

struct ConstType {
mutable(int) rc;
}
Feb 15 2016
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, 16 February 2016 at 07:20:00 UTC, Ola Fosheim Grøstad 
wrote:

 On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright 
 wrote:

 rears its head again :-)

 Head Const is what C++ has for const, i.e. it is not 
 transitive, applies to one level only. D has transitive const.

 What head const will do for us:

 1. make it easy to interface to C++ code that uses const, as 
 currently it is not very practical to do so, you have to 
 resort to pragma(mangle)

 2. supports single assignment style of programming, even if 
 the data is otherwise mutable

 The downside is, of course, language complexity.


 Maybe you can get away with adding "mutable", which is needed 
 for intrusive ref counting as well:

 C++:
 Type * const ptr;

 struct ConstType {
 mutable int rc;
 }


 D:
 const mutable(Type)* ptr;

 struct ConstType {
 mutable(int) rc;
 }


For that to work in D, we'd lose out on the guarantee that a 
const object can't be mutated via a const reference to that data 
- though with the current state of things, const does become 
limiting enough that you can't use it under a number of 
circumstances that you might like to (e.g. ref-counting), and 
some folks mistakenly think that it's defined behavior to cast 
away const and mutate and do it in their code. So, such a change 
might be worth it, but it's not something that should be 
considered lightly. If we did go that route though, having 
mutable or  mutable would allow for it to be much more controlled 
than simply casting away const and mutating (e.g. it could 
guarantee that no immutable objects were involved, since an 
object with an  mutable member couldn't be immutable, since that 
would violate immutable).

But even if we went that route, that's still a whole other 
ballgame from non-transitive const. There are some major 
implications for const if we making it legal to cast it away and 
mutate as long as the underlying data is mutable rather than 
immutable, but from a language complexity standpoint, it's not 
particularly problematic, whereas non-transitive const is.

- Jonathan M Davis
Feb 16 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 16 February 2016 at 08:04:32 UTC, Jonathan M Davis 
wrote:

 For that to work in D, we'd lose out on the guarantee that a 
 const object can't be mutated via a const reference to that 
 data - though with the current state of things, const does 
 become limiting enough that you can't use it under a number of 
 circumstances that you might like to (e.g. ref-counting), and 
 some folks mistakenly think that it's defined behavior to cast 
 away const and mutate and do it in their code.


Yes,  casting away transitive const should be impossible.

The real issue is how to deal with calling overloaded C++ 
functions. Like, some C++ functions implements transitive const, 
others do head const combined with mutable. So it is insufficient 
to look at the C++ sigature... Which means D ought to have some 
stronger way to specifing constness for C++ signatures than C++ 
provides.


 But even if we went that route, that's still a whole other 
 ballgame from non-transitive const. There are some major 
 implications for const if we making it legal to cast it away 
 and mutate as long as the underlying data is mutable rather 
 than immutable, but from a language complexity standpoint, it's 
 not particularly problematic, whereas non-transitive const is.


Yes, please don't allow casting from const to mutable, D should 
aim for stronger typing and better optimization.
Feb 16 2016
prev sibling parent reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig outerproduct.org> writes:
Am 16.02.2016 um 08:20 schrieb Ola Fosheim Grøstad:

 On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote:

 rears its head again :-)

 Head Const is what C++ has for const, i.e. it is not transitive,
 applies to one level only. D has transitive const.

 What head const will do for us:

 1. make it easy to interface to C++ code that uses const, as currently
 it is not very practical to do so, you have to resort to pragma(mangle)

 2. supports single assignment style of programming, even if the data
 is otherwise mutable

 The downside is, of course, language complexity.


 Maybe you can get away with adding "mutable", which is needed for
 intrusive ref counting as well:

 C++:
 Type * const ptr;

 struct ConstType {
 mutable int rc;
 }


 D:
 const mutable(Type)* ptr;

 struct ConstType {
 mutable(int) rc;
 }


As a bonus, this would also provide a natural syntax to define 
tail-const class references:

mutable const(C) something;
Feb 16 2016
parent reply ZombineDev <valid_email he.re> writes:
On Tuesday, 16 February 2016 at 09:39:31 UTC, Sönke Ludwig wrote:

 Am 16.02.2016 um 08:20 schrieb Ola Fosheim Grøstad:

 On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright 
 wrote:

 rears its head again :-)

 Head Const is what C++ has for const, i.e. it is not 
 transitive,
 applies to one level only. D has transitive const.

 What head const will do for us:

 1. make it easy to interface to C++ code that uses const, as 
 currently
 it is not very practical to do so, you have to resort to 
 pragma(mangle)

 2. supports single assignment style of programming, even if 
 the data
 is otherwise mutable

 The downside is, of course, language complexity.


 Maybe you can get away with adding "mutable", which is needed 
 for
 intrusive ref counting as well:

 C++:
 Type * const ptr;

 struct ConstType {
 mutable int rc;
 }


 D:
 const mutable(Type)* ptr;

 struct ConstType {
 mutable(int) rc;
 }


 As a bonus, this would also provide a natural syntax to define 
 tail-const class references:

 mutable const(C) something;


Another bonus to introducing the mutable keyword is the option to 
make everything immutable by default (in a future version of D) 
and allow the users to have mutable objects only if they use the 
mutable keyword.
Feb 16 2016
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, 16 February 2016 at 10:06:12 UTC, ZombineDev wrote:

 Another bonus to introducing the mutable keyword is the option 
 to make everything immutable by default (in a future version of 
 D) and allow the users to have mutable objects only if they use 
 the mutable keyword.


While some folks do bring that up from time to time, I think that 
it's pretty clear that that would be so restrictive that it would 
risk killing D. As it is, many programmers avoid const 
altogether, because it's too restrictive. Heck, ranges are 
designed in such a way that they require mutation to work, and 
they're everywhere.

immutable has its uses to be sure, but I don't see how it's 
anything but a pipe dream to expect any version of D to be 
immutable by default. For most programmers, it would be way too 
annoying and way too verbose, because they'd be forced to slap 
mutable on most everything.

Regardless, there isn't much point in planning for a future 
version of D. We don't know what we're going to want to do at 
that point, and if we're actually willing to break backwards 
compatibility in a serious way, what D2 looks like doesn't really 
matter much for D3. And we don't even know whether there will 
ever be a D3. What matters to us now is what we do with D2 for 
making it a good language now and not what we may or may not do 
with a future version of the language. Planning for D3 now would 
be like planning for D when working on finishing up C++98.

- Jonathan M Davis
Feb 16 2016
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 16 February 2016 at 10:17:05 UTC, Jonathan M Davis 
wrote:

 version of D. We don't know what we're going to want to do at 
 that point, and if we're actually willing to break backwards 
 compatibility in a serious way, what D2 looks like doesn't 
 really matter much for D3. And we don't even know whether there 
 will ever be a D3. What matters to us now is what we do with D2 
 for making it a good language now and not what we may or may 
 not do with a future version of the language. Planning for D3 
 now would be like planning for D when working on finishing up 
 C++98.


This is true, in a way, but also a bit too pessimistic.

Here is a possible strategy:

1. add semantics that are desirable to D2
2. express both D2 and D3 in a single intermediate representation
3. work on a new D3 syntax that does not support deprecated 
features
4. have two parsers, one for D2 and one for D3

Let compiler vendors decide whether they want to support D2 or 
not.
Feb 16 2016
parent reply ZombineDev <valid_email he.re> writes:
On Tuesday, 16 February 2016 at 11:02:38 UTC, Ola Fosheim Grøstad 
wrote:

 On Tuesday, 16 February 2016 at 10:17:05 UTC, Jonathan M Davis 
 wrote:

 version of D. We don't know what we're going to want to do at 
 that point, and if we're actually willing to break backwards 
 compatibility in a serious way, what D2 looks like doesn't 
 really matter much for D3. And we don't even know whether 
 there will ever be a D3. What matters to us now is what we do 
 with D2 for making it a good language now and not what we may 
 or may not do with a future version of the language. Planning 
 for D3 now would be like planning for D when working on 
 finishing up C++98.


 This is true, in a way, but also a bit too pessimistic.

 Here is a possible strategy:

 1. add semantics that are desirable to D2
 2. express both D2 and D3 in a single intermediate 
 representation
 3. work on a new D3 syntax that does not support deprecated 
 features
 4. have two parsers, one for D2 and one for D3

 Let compiler vendors decide whether they want to support D2 or 
 not.


A small comment on the way you express most of your opinions:
You have many ideas on how and why D should change, however all 
of your proposals are so vague that no one can figure out any 
concrete action items in them. Furthermore you don't seem to be 
familiar with implementation details and even don't seem to have 
any experience in writting large programs in D, because you never 
talk about concrete problems and never contribute DIPs or PRs 
(which other people serious about using D do). It seems that 
you're not interested in using D for anything at all, which is 
very strange considering how much time you spend on the forums.
Instead of trying critisize other people's concrete plans/ideas 
by saying that instead they should be focusing on something very 
abstract that only you understand, I think it would be a lot more 
beneficial for the D community and your agenda, if you turn your 
words into actions and show **by example** that your 
design/implementation has **objective, measurable** benefits. 
Otherwise no one will find practical real-world benefits from 
your theoretical ideas and we would just waste time arguing with 
each other, while something practical could be accomplished in 
the same time.
Be the change you want to see in the world.

Back to your concrete comment:

It sounds nice in theory, however there are too many refactorings 
in the AST and semantic analysis done for each release, which 
prevents creating a stable intermediate representation.
The only stable IRs are the DMD backend IR, the GCC IR and the 
LLVM IR.
By the way those sementics (maybe) are already representable in 
the current AST abstractions in the frontend. What benefits would 
a dedicated frontend IR bring? Hint: I have already watched the 
LLVM talk about the Swift IR and I'm familiar with the reasons 
for Rust introducing MIR. I just don't see the value such a large 
initiative would bring to D.
The best way to defend your point would be look at the DMD source 
code and propose specific API changes/additions which would bring 
objective benefits. Otherwise you sound like someone telling a 
car manufacturer that they should be focusing on flying cars or 
teleportation, because clearly that's the future :D
Feb 16 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 16 February 2016 at 12:03:48 UTC, ZombineDev wrote:

 familiar with implementation details and even don't seem to 
 have any experience in writting large programs in D, because


Of course I don't. My goal for D is that it becomes a solid base 
language which it makes sense to build upon. My D programming is 
currently only language related experiments on a rather low level 
(syntax and basic functionality), I have used D for other things 
in the past, but currently it does not stack up against Python or 
C++ for production in my case.

It would be nice if it did, of course.



 Otherwise no one will find practical real-world benefits from 
 your theoretical ideas and we would just waste time arguing 
 with each other, while something practical could be 
 accomplished in the same time.


The foundation has to be solid. Clang went against common wisdom 
when compiling directly to a low level IR without. Common wisdom 
is to compile to a high level IR first. In the case of C++ it has 
worked out ok for clang. That does not mean that there are no 
benefits to using a high level IR. GCC has a higher level IR at 
the top level, and I suspect they have benefitted from that on 
some optimizations.


 would bring objective benefits. Otherwise you sound like 
 someone telling a car manufacturer that they should be focusing 
 on flying cars or teleportation, because clearly that's the 
 future :D


No, it is more like discussing the chassis rather than the 
polish. Car manufacturers can design many models over the same 
chassis, if they put extra effort into it.
Feb 16 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 16 February 2016 at 12:15:03 UTC, Ola Fosheim Grøstad 
wrote:

 Common wisdom is to compile to a high level IR first. In the 
 case of C++ it has worked out ok for clang. That does not mean 
 that there are no benefits to using a high level IR.


Let me expand on this:

1. it makes the compiler easier to maintain and debug as the 
compiler becomes more modular with clear separation

2. it makes it possible to verify  safe (hopefully)

3. it makes it easier to use multiple backends

4. it can make it easier to exploit multithreading

5. you can have separate compilation that retains more type 
information for whole program linkage or optimization

6. It makes it much easier to get people to work on the compiler 
as you only need to understand the IR when implementing features, 
i.e. static analysis.
Feb 16 2016
prev sibling next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 16.02.2016 11:17, Jonathan M Davis wrote:

 On Tuesday, 16 February 2016 at 10:06:12 UTC, ZombineDev wrote:

 Another bonus to introducing the mutable keyword is the option to make
 everything immutable by default (in a future version of D) and allow
 the users to have mutable objects only if they use the mutable keyword.


 While some folks do bring that up from time to time, I think that it's
 pretty clear that that would be so restrictive that it would risk
 killing D. As it is, many programmers avoid const altogether, because
 it's too restrictive.


Immutable by default would not be what we have now (transitive immutable).
Feb 16 2016
prev sibling parent ZombineDev <valid_email he.re> writes:
On Tuesday, 16 February 2016 at 10:17:05 UTC, Jonathan M Davis 
wrote:

 On Tuesday, 16 February 2016 at 10:06:12 UTC, ZombineDev wrote:

 Another bonus to introducing the mutable keyword is the option 
 to make everything immutable by default (in a future version 
 of D) and allow the users to have mutable objects only if they 
 use the mutable keyword.


 While some folks do bring that up from time to time, I think 
 that it's pretty clear that that would be so restrictive that 
 it would risk killing D. As it is, many programmers avoid const 
 altogether, because it's too restrictive.


Not if immutable by default is opt-in - read below (my second 
paragraph).


 Heck, ranges are designed in such a way that they require 
 mutation to work, and they're everywhere.


Ranges are not a hard problem, they're more of a too much good 
code written with them to be worth the breakage.
Since ranges are mostly meant to be cheap value types, we can 
just make popFront return a new range, just like slicing an array 
slice returns a new slice. However for this to work we would need 
to rewrite a ton of std.range, std.algorithm, etc. so it probably 
won't provide a good enough cost/benefit ratio.
I'm working on a different proposal to generalize ranges to 
support push data flow model, in addition to the current 
pull-only, which would allow them to be used for processing async 
data (e.g. comming from I/O or another thread performing 
expensive computation for each element) in a **non-blocking** 
way. Processing async data with the current range primitives 
implies polling/blocking on popFront or front, which is a 
performance killer and diminishes the value of composition.
I hope my proposal would provide enough value to be included in 
phobos as an alternative to the current InputRange primitives. If 
it proves to be good enough, maybe it would be worth adding new 
versions of most of the current range algorithms/transformations 
that leverage the new prmitives.
I haven't yet figured out the best way to bring these API ideas 
from Clojure and C++ to D (see 
http://forum.dlang.org/thread/xfjoktyjtoojhszrlugi forum.dlang.org for more
info on me exploring one possibility of using higher-order functions), because
there are plenty of ways to do this which I want to explore, but I hope to have
something concrete in a couple of months.



 immutable has its uses to be sure, but I don't see how it's 
 anything but a pipe dream to expect any version of D to be 
 immutable by default. For most programmers, it would be way too 
 annoying and way too verbose, because they'd be forced to slap 
 mutable on most everything.


Immutable by default has two sides:
1) The programmer writing the code
2) The user of the code

The change to immutable by default should only affect 1). 2) 
should be the same as if currently you make something immutable. 
This means **no code breakage**.
A good way forward, IMO, is to be able to indicate at the 
beginning of the module your general preferences e.g. immutable, 
 nogc, pure, non-virtual by default. This should affect **only** 
the current declaration scope, just as if you have put `nothrow:` 
at the beginning. The only thing that's needed is a way to revert 
this. The proposed solution is `pure(false)`, `final(false)`, 
etc., which Andrei already approved.



 Regardless, there isn't much point in planning for a future 
 version of D. We don't know what we're going to want to do at 
 that point, and if we're actually willing to break backwards 
 compatibility in a serious way, what D2 looks like doesn't 
 really matter much for D3. And we don't even know whether there 
 will ever be a D3. What matters to us now is what we do with D2 
 for making it a good language now and not what we may or may 
 not do with a future version of the language. Planning for D3 
 now would be like planning for D when working on finishing up 
 C++98.

 - Jonathan M Davis


I agree. I think we should focus on features that have strong 
benefits and don't break backwards compatibility. There are 
plenty of stuff to explore in this area. From what I've read, the 
D1/D2 and Phobos/Tango devisions was hurtful for the community. 
We should instead focus on unitying and improving what we already 
have in order to bring real benefits to our community.
Feb 16 2016
prev sibling next sibling parent reply Dicebot <public dicebot.lv> writes:
On 02/16/2016 12:48 AM, Walter Bright wrote:

 rears its head again :-)
 
 Head Const is what C++ has for const, i.e. it is not transitive, applies
 to one level only. D has transitive const.
 
 What head const will do for us:
 
 1. make it easy to interface to C++ code that uses const, as currently
 it is not very practical to do so, you have to resort to pragma(mangle)
 
 2. supports single assignment style of programming, even if the data is
 otherwise mutable
 
 The downside is, of course, language complexity.


Moderately skeptical here. Adding features to the language for the sake
of C++ compatibility is good way to ensure C++ compatibility will be its
only domain of triumph. I'd expect more justification for users who
don't care about C++ but want language which doesn't collapse under own
weight.
Feb 16 2016
parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Tuesday, 16 February 2016 at 11:27:56 UTC, Dicebot wrote:

 On 02/16/2016 12:48 AM, Walter Bright wrote:

 rears its head again :-)
 
 Head Const is what C++ has for const, i.e. it is not 
 transitive, applies to one level only. D has transitive const.
 
 What head const will do for us:
 
 1. make it easy to interface to C++ code that uses const, as 
 currently it is not very practical to do so, you have to 
 resort to pragma(mangle)
 
 2. supports single assignment style of programming, even if 
 the data is otherwise mutable
 
 The downside is, of course, language complexity.


 Moderately skeptical here. Adding features to the language for 
 the sake of C++ compatibility is good way to ensure C++ 
 compatibility will be its only domain of triumph. I'd expect 
 more justification for users who don't care about C++ but want 
 language which doesn't collapse under own weight.


` mutable` OTOH would be a useful for both C++, reference 
counting, caching, lazy initialization... But we need to find a 
way to keep most of the existing guarantees, especially 
concerning shareability.
Feb 16 2016
parent reply Dicebot <public dicebot.lv> writes:
On 02/16/2016 02:49 PM, Marc Schütz wrote:

 ` mutable` OTOH would be a useful for both C++, reference counting,
 caching, lazy initialization... But we need to find a way to keep most
 of the existing guarantees, especially concerning shareability.


In my opinion  mutable would be a disaster of much higher destructive
impact than head const. I am very opposed to it no matter how it is
designed. Once you start considering it, you are better at simply
throwing away existing const system and starting it all from scratch
with D3. Logical const is harmful as it doesn't give and serious
guarantees but gives developer a false sense of confidence.
Feb 16 2016
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 16 February 2016 at 13:35:56 UTC, Dicebot wrote:

 matter how it is designed. Once you start considering it, you 
 are better at simply throwing away existing const system and 
 starting it all from scratch with D3. Logical const is harmful 
 as it doesn't give and serious guarantees but gives developer a 
 false sense of confidence.


Why is that? Doesn't non-shared const promise that the protected 
values aren't modified, until the const reference is no longer 
alive?

Meaning: does the language allow non-const aliasing with const in 
function calls?

   // prototype f(const A *, A*);
   f(ptr,ptr);

Is this disallowed? Or is it legal D? (not talking about the 
compiler, but the language spec)
Feb 16 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 16.02.2016 14:41, Ola Fosheim Grøstad wrote:

 On Tuesday, 16 February 2016 at 13:35:56 UTC, Dicebot wrote:

 matter how it is designed. Once you start considering it, you are
 better at simply throwing away existing const system and starting it
 all from scratch with D3. Logical const is harmful as it doesn't give
 and serious guarantees but gives developer a false sense of confidence.


 Why is that? Doesn't non-shared const promise that the protected values
 aren't modified, until the const reference is no longer alive?

 Meaning: does the language allow non-const aliasing with const in
 function calls?

    // prototype f(const A *, A*);
    f(ptr,ptr);

 Is this disallowed? Or is it legal D? (not talking about the compiler,
 but the language spec)


Legal.
Feb 16 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 16 February 2016 at 15:10:40 UTC, Timon Gehr wrote:

 Legal.


Ouch... :-/
Feb 16 2016
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, 16 February 2016 at 15:12:00 UTC, Ola Fosheim Grøstad 
wrote:

 On Tuesday, 16 February 2016 at 15:10:40 UTC, Timon Gehr wrote:

 Legal.


 Ouch... :-/


const guarantees that the data will not be mutated via that 
reference to the data. It guarantees nothing about other 
references to that data. For that, you need immutable. But as 
long as const guarantees that the data can't be mutated via that 
reference, then there's plenty of code where the programmer can 
look at it and know for sure that that data isn't being mutated, 
because they know that the code in question can't have access to 
the same data via a mutable reference (the compiler can see that 
too, but in a much more limited fashion than a programmer is 
going to be able to know).

If casting away const and mutating is defined behavior, then you 
lose that guarantee, and instead of knowing that some code 
doesn't have access to a mutable reference to that data being 
enough, you have to actually know what the code does to know that 
it's not casting away const and mutating.

Having  mutable would be better than allowing casting away const 
and mutating to be defined behavior (at least when the underlying 
data is mutable rather than immutable), because it would at least 
restrict the data that can be mutated, and it would guarantee 
that you don't accidentally mutate an immutable variable (since 
it would make no sense to have a type with an  mutable member be 
immutable, whereas if you're casting away const, getting that 
right would be completely up to the programmer). But having 
 mutable would  still mean that you can't rely on a const 
variable not being mutated when you pass it to a function that 
you know doesn't have access to a mutable reference to the same 
data. If you knew that no  mutable was involved, it would be the 
same as now, but you'd have to know for sure that  mutable was 
involved, and even if it isn't now, it could be later after 
refactoring.

So, having  mutable would be far better than having it be defined 
behavior to cast away const and mutate (assuming that the 
underlying data is mutable rather than immutable), but you're 
still losing out on guarantees that we have now. After all, most 
code isn't going to do the equivalent of

     // prototype f(const A *, A*);
     f(ptr,ptr);

Ultimately, I think that the question is whether the guarantees 
that we currently get with const are worth more or whether the 
idioms that don't work with const right now but would work with 
 mutable are worth more. But regardless, the guarantees provided 
by const obviously pale in comparison to the ones that immutable 
provides.

- Jonathan M Davis
Feb 16 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 16 February 2016 at 15:33:03 UTC, Jonathan M Davis 
wrote:

 const guarantees that the data will not be mutated via that 
 reference to the data. It guarantees nothing about other 
 references to that data. For that, you need immutable.


Yes, it's as weak as C++ const in most cases from an optimization 
POV.

In C (and some C++ compilers) you can say that there is no 
aliasing by using "restrict", so I guess what I want is something 
like "const restrict".

https://en.wikipedia.org/wiki/Restrict


 But as long as const guarantees that the data can't be mutated 
 via that reference, then there's plenty of code where the 
 programmer can look at it and know for sure that that data 
 isn't being mutated, because they know that the code in


When you are calling a function that is no-global-access "pure", 
yes.


 If casting away const and mutating is defined behavior, then 
 you lose that guarantee, and instead of knowing that some code 
 doesn't have access to a mutable reference to that data being 
 enough, you have to actually know what the code does to know 
 that it's not casting away const and mutating.


Yes, I agree that casting away const as is possible in C++ is 
bad. The only reasonable setting for doing so is when you call 
legacy functions that are not defined to be const, but 
implemented as such.


 Having  mutable would be better than allowing casting away 
 const and mutating to be defined behavior (at least when the 
 underlying data is mutable rather than immutable), because it 
 would at least restrict the data that can be mutated, and it 
 would guarantee that you don't accidentally mutate an immutable 
 variable (since it would make no sense to have a type with an 
  mutable member be immutable, whereas if you're casting away 
 const, getting that right would be completely up to the 
 programmer).


Maybe I misunderstand, but I am likely to want an immutable 
object with a mutable field, e.g. ref-counted caching. You use a 
CAS instruction on the mutable ref-counter and share the data. 
Then you can flush the cache and release all objects that have a 
refcount of only 1 (the cache index pointer).


 But having  mutable would  still mean that you can't rely on a 
 const variable not being mutated when you pass it to a function 
 that you know doesn't have access to a mutable reference to the 
 same data. If you knew that no  mutable was involved, it would 
 be the same as now, but you'd have to know for sure that 
  mutable was involved, and even if it isn't now, it could be 
 later after refactoring.


Well, yes, this is where you have to be very clear about what 
kind of semantics a struct is meant to have. I don't feel this is 
very clear in the current language.

To me the value of immutable is that fields that are immutable 
will remain immutable forever. Like for a key that is inserted 
into a hash table. You know that it cannot change so you never 
need to recompute the hash value.

Then I'd like to see a "limited time immutable", like the 
"restrict const" suggested above.

But apparently there are use-cases where you'd want to say "whole 
object is immutable" and other use-cases where you want to say 
"all non-forced-mutable fields are immutable".


 So, having  mutable would be far better than having it be 
 defined behavior to cast away const and mutate (assuming that 
 the underlying data is mutable rather than immutable), but 
 you're still losing out on guarantees that we have now.


Yes. But I think you could distinguish between what you specify 
and what you require. So you could specify "I only want wholly 
immutable things" in a function signature, and specify "this 
object is mostly immutable except for these mutable exceptions" 
when defining it.

So I think you can have both?
Feb 16 2016
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, 16 February 2016 at 16:01:26 UTC, Ola Fosheim Grøstad 
wrote:

 On Tuesday, 16 February 2016 at 15:33:03 UTC, Jonathan M Davis 
 wrote:

 So, having  mutable would be far better than having it be 
 defined behavior to cast away const and mutate (assuming that 
 the underlying data is mutable rather than immutable), but 
 you're still losing out on guarantees that we have now.


 Yes. But I think you could distinguish between what you specify 
 and what you require. So you could specify "I only want wholly 
 immutable things" in a function signature, and specify "this 
 object is mostly immutable except for these mutable exceptions" 
 when defining it.

 So I think you can have both?


const and immutable are transitive. This is particularly critical 
with immutable, which is implicitly shared. Having portions of an 
immutable object be mutable wouldn't play nicely at all with how 
immutable works and would complicate things considerably. Andrei 
recently posted about possibly having a reference count be in the 
memory next to an immutable object but not having it in the 
object itself, and that might work:

http://forum.dlang.org/post/n9larg$23c9$1 digitalmars.com

But I really don't think that getting rid of transivity is going 
to fly for immutable (e.g. it's not like you can share part of 
object across threads and have part of it be thread-local), and 
I'm opposed to introducing non-transitive const like Walter 
brought up with this thread. It's just not worth the complexity.

- Jonathan M Davis
Feb 16 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 16 February 2016 at 16:27:54 UTC, Jonathan M Davis 
wrote:

 considerably. Andrei recently posted about possibly having a 
 reference count be in the memory next to an immutable object 
 but not having it in the object itself, and that might work:


I don't think I understand the semantic difference.

Transitive immutability is mostly useless. You usually want at 
least to be able to have a back pointer to a mutable index so 
that you can move the object to another index. This is also not 
something you can prevent. I can simply have a global set of 
indices and refer to that global set through an index ID in the 
immutable object. That is equivalent to breaking immutable 
transitivity.

So... you cannot have transitive immutability if you allow global 
data structures. Then you would have to require "pure" functions 
everywhere.

I also don't see how a reinterpret cast to a negative offset can 
be a good solution. If you need that, then you have a strong sign 
that something important is missing in the type
system.  Where would this be allowed? Not in "pure" functions, 
for sure?



 But I really don't think that getting rid of transivity is 
 going to fly for immutable (e.g. it's not like you can share 
 part of object across threads and have part of it be 
 thread-local)


It shouldn't pose a problem for a system level language to hand 
memory access through a thread local pointer to another thread if 
need be, but not sure if I follow the argument.



, and I'm opposed to introducing non-transitive
 const like Walter brought up with this thread. It's just not 
 worth the complexity.


But you do at the very least need a mechanism for triggering 
overloads on "const A&", "A&&" etc. C++ libraries tend to be 
heavily overloaded...
Feb 16 2016
prev sibling next sibling parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Tuesday, 16 February 2016 at 13:35:56 UTC, Dicebot wrote:

 On 02/16/2016 02:49 PM, Marc Schütz wrote:

 ` mutable` OTOH would be a useful for both C++, reference 
 counting, caching, lazy initialization... But we need to find 
 a way to keep most of the existing guarantees, especially 
 concerning shareability.


 In my opinion  mutable would be a disaster of much higher 
 destructive impact than head const. I am very opposed to it no 
 matter how it is designed. Once you start considering it, you 
 are better at simply throwing away existing const system and 
 starting it all from scratch with D3. Logical const is harmful 
 as it doesn't give and serious guarantees but gives developer a 
 false sense of confidence.


The last sentence in my post is crucial: "keep most of the 
existing guarantees". If we can ensure that access to  mutable 
data is strongly restricted and properly encapsulated, we don't 
lose anything. The type system would stay as it is, there would 
just be a way for it to be broken locally without provoking 
undefined behaviour.

For example, it's always possible to use a global mutable 
associative array to store additional data connected with an 
immutable or const object (ignoring purity issues for the 
moment). That's safe because from the outside, there's no 
observable change to the state of the object itself, and the 
global AA's type (shared/thread-local) prevents race conditions.

There's no reason why we can't have the same guarantees for 
embedded members, without resorting to an external AA. Have a 
look at my DIP [1] for lazy initialization of const members, 
which I now realize can be adapted to this use case. Basically, 
just replace `lazy` by ` mutable`, and most of the DIP is still 
valid. I'll try updating it when I find the time.

[1] http://wiki.dlang.org/DIP85
Feb 16 2016
next sibling parent Danni Coy via Digitalmars-d <digitalmars-d puremagic.com> writes:
speaking as a lay user - if this is going to be done maybe it should
something like
__gconst?
similiar to __gshared which is ugly enough to make you concider not
using it, but it's there if you really need it.

On Wed, Feb 17, 2016 at 5:29 AM, Marc Schütz via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Tuesday, 16 February 2016 at 13:35:56 UTC, Dicebot wrote:

 On 02/16/2016 02:49 PM, Marc Schütz wrote:

 ` mutable` OTOH would be a useful for both C++, reference counting,
 caching, lazy initialization... But we need to find a way to keep most of
 the existing guarantees, especially concerning shareability.



 In my opinion  mutable would be a disaster of much higher destructive
 impact than head const. I am very opposed to it no matter how it is
 designed. Once you start considering it, you are better at simply throwing
 away existing const system and starting it all from scratch with D3. Logical
 const is harmful as it doesn't give and serious guarantees but gives
 developer a false sense of confidence.



 The last sentence in my post is crucial: "keep most of the existing
 guarantees". If we can ensure that access to  mutable data is strongly
 restricted and properly encapsulated, we don't lose anything. The type
 system would stay as it is, there would just be a way for it to be broken
 locally without provoking undefined behaviour.

 For example, it's always possible to use a global mutable associative array
 to store additional data connected with an immutable or const object
 (ignoring purity issues for the moment). That's safe because from the
 outside, there's no observable change to the state of the object itself, and
 the global AA's type (shared/thread-local) prevents race conditions.

 There's no reason why we can't have the same guarantees for embedded
 members, without resorting to an external AA. Have a look at my DIP [1] for
 lazy initialization of const members, which I now realize can be adapted to
 this use case. Basically, just replace `lazy` by ` mutable`, and most of the
 DIP is still valid. I'll try updating it when I find the time.

 [1] http://wiki.dlang.org/DIP85
Feb 16 2016
prev sibling next sibling parent reply Dicebot <public dicebot.lv> writes:
On 02/16/2016 09:29 PM, Marc Schütz wrote:

 The last sentence in my post is crucial: "keep most of the existing
 guarantees". If we can ensure that access to  mutable data is strongly
 restricted and properly encapsulated, we don't lose anything. The type
 system would stay as it is, there would just be a way for it to be
 broken locally without provoking undefined behaviour.


One example of a existing guarantee you won't be able to keep, for
example, is that any immutable allocation can be completely put into
separate read-only memory. A very important property for building
optimized allocators if you keep in mind sharing goals of immutability.


 For example, it's always possible to use a global mutable associative
 array to store additional data connected with an immutable or const
 object (ignoring purity issues for the moment). That's safe because from
 the outside, there's no observable change to the state of the object
 itself, and the global AA's type (shared/thread-local) prevents race
 conditions.


Yes and this is how I believe it must be done.


 There's no reason why we can't have the same guarantees for embedded
 members, without resorting to an external AA. Have a look at my DIP [1]
 for lazy initialization of const members, which I now realize can be
 adapted to this use case. Basically, just replace `lazy` by ` mutable`,
 and most of the DIP is still valid. I'll try updating it when I find the
 time.
 
 [1] http://wiki.dlang.org/DIP85


The problem with this DIP is that it speaks about type system semantics
and what matters first is memory model (which is currently very
under-defined as soon as you step from a "the GC" world).

Physical immutability is demanding but it also has great value in its
simplicity and being hard to fool. Any language change that is going to
reject this notion has to be really strongly justified in terms of what
is gained and what is lost and not come simply because expressing such
semantics is possible.
Feb 16 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/16/2016 6:20 PM, Dicebot wrote:

 The problem with this DIP is that it speaks about type system semantics
 and what matters first is memory model (which is currently very
 under-defined as soon as you step from a "the GC" world).

 Physical immutability is demanding but it also has great value in its
 simplicity and being hard to fool. Any language change that is going to
 reject this notion has to be really strongly justified in terms of what
 is gained and what is lost and not come simply because expressing such
 semantics is possible.


Yeah, I think we're on the same page with this.

Most of what people find frustrating about D's transitive const/immutable is
its 
very nature of not being sloppy and full of holes.
Feb 16 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 02/16/2016 09:53 PM, Walter Bright wrote:

 Most of what people find frustrating about D's transitive
 const/immutable is its very nature of not being sloppy and full of holes.


I thought it was the constructors and postblit that are the matter. (It 
may seem I'm on a snarky streak, but I do think that's the remaining 
issue with immutable). -- Andrei
Feb 17 2016
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, 17 February 2016 at 11:56:29 UTC, Andrei 
Alexandrescu wrote:

 On 02/16/2016 09:53 PM, Walter Bright wrote:

 Most of what people find frustrating about D's transitive
 const/immutable is its very nature of not being sloppy and 
 full of holes.


 I thought it was the constructors and postblit that are the 
 matter. (It may seem I'm on a snarky streak, but I do think 
 that's the remaining issue with immutable). -- Andrei


Yes, those are design problems with D's const that we really need 
to figure out how to fix, but they're not necessarily what people 
complain about. Many folks want to be able to do things like lazy 
initialization or caching or have a mutable reference count in a 
const/immutable object. The problems that usually come up 
basically boil down to folks wanting logical const rather than 
the compiler actually guaranteeing that data cannot be mutated 
via a const reference to that data without violating the type 
system. Lots of people basically want holes in const, because 
that's what C++ has, and not having holes in const prevents a 
number of potentially useful idioms (heck, you were trying to get 
around it with a container that you posted in the main newsgroup 
semi-recently and in RCString when you posted about it in 
dlang-study the other day).

We need to solve the constructor/postblit issues in order for D's 
const to really work like it's supposed to, but even if that 
problem were solved, the basic tenet of D's design for const - 
that it has no backdoors - is what really makes it so that many 
folks avoid const completely, whereas others use it but violate 
the type system by casting away const and mutating, not realizing 
that unlike C++, it's undefined behavior in D.

Regardless of whether Walter's (and thus D's) stance on const is 
the right one, it clearly doesn't jive with what a lot of folks 
(particularly those from C++) are looking for or expecting.

- Jonathan M Davis
Feb 17 2016
parent sclytrack <sclytrack fake.com> writes:
On Wednesday, 17 February 2016 at 12:46:08 UTC, Jonathan M Davis 
wrote:

 but violate the type system by casting away const and mutating, 
 not realizing that unlike C++, it's undefined behavior in D.

 Regardless of whether Walter's (and thus D's) stance on const 
 is the right one, it clearly doesn't jive with what a lot of 
 folks (particularly those from C++) are looking for or 
 expecting.

 - Jonathan M Davis


	struct (bla1, bla2) Test
	{
		int rc;
		bla1 int number;
		(bla2) Payload payload;

		void doSomething() (shared, immutable)
		{
		}
	}
	
	(shared, immutable) Test obj;

	bla1 becomes shared and bla2 becomes immutable, but
	the payload might not be immutable because it is
	(bla2) instead of just bla2.
	---
	Can it be made in a way that the blablas do not
	participate in the static if?
	----
	Could "shared", "immutable" be separated from
	the type int, float?
	
	(shared, immutable) Test!(int, float) obj;
	---
	I did some "const(immutable) int" and the the
	compiler says basic type expected.

	What is the basic type from "const(shared, immutable)",
	would that be "const(shared const, const)"?

	And then the basic type from "const(const, immutable)"
	would be "const".
	---
	Also would it be possible to determine the implicit
	conversion rules based solely on these qualifiers and
	not on the content of the object?
	
	(const, const) Test t = new (const, immutable) Test();
	const Test t = new (const,const) Test();
	---
Feb 17 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2016 3:56 AM, Andrei Alexandrescu wrote:

 On 02/16/2016 09:53 PM, Walter Bright wrote:

 Most of what people find frustrating about D's transitive
 const/immutable is its very nature of not being sloppy and full of holes.


 I thought it was the constructors and postblit that are the matter. (It may
seem
 I'm on a snarky streak, but I do think that's the remaining issue with
 immutable). -- Andrei


Logical const!
Feb 17 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 02/17/2016 04:07 PM, Walter Bright wrote:

 On 2/17/2016 3:56 AM, Andrei Alexandrescu wrote:

 On 02/16/2016 09:53 PM, Walter Bright wrote:

 Most of what people find frustrating about D's transitive
 const/immutable is its very nature of not being sloppy and full of
 holes.


 I thought it was the constructors and postblit that are the matter.
 (It may seem
 I'm on a snarky streak, but I do think that's the remaining issue with
 immutable). -- Andrei


 Logical const!


How do you mean that? I see no connection. -- Andrei
Feb 17 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2016 2:38 PM, Andrei Alexandrescu wrote:

 On 02/17/2016 04:07 PM, Walter Bright wrote:

 On 2/17/2016 3:56 AM, Andrei Alexandrescu wrote:

 On 02/16/2016 09:53 PM, Walter Bright wrote:

 Most of what people find frustrating about D's transitive
 const/immutable is its very nature of not being sloppy and full of
 holes.


 I thought it was the constructors and postblit that are the matter.
 (It may seem
 I'm on a snarky streak, but I do think that's the remaining issue with
 immutable). -- Andrei


 Logical const!


 How do you mean that? I see no connection. -- Andrei


People want to relax transitive immutability so they can implement logical
const.
Feb 17 2016
prev sibling parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Wednesday, 17 February 2016 at 02:20:15 UTC, Dicebot wrote:

 One example of a existing guarantee you won't be able to keep, 
 for example, is that any immutable allocation can be completely 
 put into separate read-only memory.


Yes, and it would be rejected statically (rule 2). I therefor 
don't consider this a problem.


 A very important property for building optimized allocators if 
 you keep in mind sharing goals of immutability.


This is considered too (rule 3). An object can only be immutable 
if all its embedded  mutable members are marked as shared.


 For example, it's always possible to use a global mutable 
 associative array to store additional data connected with an 
 immutable or const object (ignoring purity issues for the 
 moment). That's safe because from the outside, there's no 
 observable change to the state of the object itself, and the 
 global AA's type (shared/thread-local) prevents race 
 conditions.


 Yes and this is how I believe it must be done.


 There's no reason why we can't have the same guarantees for 
 embedded members, without resorting to an external AA. Have a 
 look at my DIP [1] for lazy initialization of const members, 
 which I now realize can be adapted to this use case. 
 Basically, just replace `lazy` by ` mutable`, and most of the 
 DIP is still valid. I'll try updating it when I find the time.
 
 [1] http://wiki.dlang.org/DIP85


 The problem with this DIP is that it speaks about type system 
 semantics and what matters first is memory model (which is 
 currently very under-defined as soon as you step from a "the 
 GC" world).


I'd say most of the memory model issues are the same as those 
with shared, which is just as undefined currently. That leaves 
possible reordering of accesses to immutable data. I don't have a 
complete answer to that at the moment, but most of it is likely 
also covered by the fact that the  mutable members must be shared 
in this case.


 Physical immutability is demanding but it also has great value 
 in its simplicity and being hard to fool. Any language change 
 that is going to reject this notion has to be really strongly 
 justified in terms of what is gained and what is lost and not 
 come simply because expressing such semantics is possible.
Feb 17 2016
parent reply Dicebot <public dicebot.lv> writes:
On 02/17/2016 08:48 PM, Marc Schütz wrote:

 On Wednesday, 17 February 2016 at 02:20:15 UTC, Dicebot wrote:

 One example of a existing guarantee you won't be able to keep, for
 example, is that any immutable allocation can be completely put into
 separate read-only memory.


 
 Yes, and it would be rejected statically (rule 2). I therefor don't
 consider this a problem.


You pretty much prohibit any but typed allocators this way. I do see it
as a problem.


 A very important property for building optimized allocators if you
 keep in mind sharing goals of immutability.


 
 This is considered too (rule 3). An object can only be immutable if all
 its embedded  mutable members are marked as shared.


That sounds rather weird considering immutable data neither needs nor
synchronization nor CPU cache reloading and  mutable shared one needs both.
Feb 18 2016
parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Thursday, 18 February 2016 at 11:12:40 UTC, Dicebot wrote:

 On 02/17/2016 08:48 PM, Marc Schütz wrote:

 On Wednesday, 17 February 2016 at 02:20:15 UTC, Dicebot wrote:

 One example of a existing guarantee you won't be able to 
 keep, for example, is that any immutable allocation can be 
 completely put into separate read-only memory.


 
 Yes, and it would be rejected statically (rule 2). I therefor 
 don't
 consider this a problem.


 You pretty much prohibit any but typed allocators this way. I 
 do see it as a problem.


Rule 2 only forbids _static immutables_ with  mutable members, 
precisely because those could end up in read-only memory. I don't 
see how allocators are affected at all. An allocator would be 
mutable, or at least manage mutable memory.


 A very important property for building optimized allocators 
 if you keep in mind sharing goals of immutability.


 
 This is considered too (rule 3). An object can only be 
 immutable if all its embedded  mutable members are marked as 
 shared.


 That sounds rather weird considering immutable data neither 
 needs nor synchronization nor CPU cache reloading and  mutable 
 shared one needs both.


It's a necessary consequence of what you just said. Consider 
reference counting: normal const RC objects are thread-local, and 
you can use simple increments/decrements to manipulate the 
refcounts. Because immutable ones are implicitly sharable, you 
need to use atomic operations. Again, this is well encapsulated: 
the end-user doesn't get access to the  mutable members.

By the way, the AA approach is troublesome with implicit sharing: 
if you declare an immutable RC object that uses a thread-local AA 
to store its counters, and it gets sent to another thread, that 
one would suddenly find the reference count to be 0 (non-existing 
in its own thread-local AA). And I don't see how you could 
prevent someone from accidentally doing this with the current 
language.
Feb 18 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/18/2016 6:04 AM, Marc Schütz wrote:

 Rule 2 only forbids _static immutables_ with  mutable members, precisely
because
 those could end up in read-only memory. I don't see how allocators are affected
 at all. An allocator would be mutable, or at least manage mutable memory.


You could create a bunch of immutable data structures, then make the page they 
are in read-only.
Feb 18 2016
parent Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Friday, 19 February 2016 at 00:45:00 UTC, Walter Bright wrote:

 On 2/18/2016 6:04 AM, Marc Schütz wrote:

 Rule 2 only forbids _static immutables_ with  mutable members, 
 precisely because
 those could end up in read-only memory. I don't see how 
 allocators are affected
 at all. An allocator would be mutable, or at least manage 
 mutable memory.


 You could create a bunch of immutable data structures, then 
 make the page they are in read-only.


You can do this with normal mutable data structures as well, 
which already shows that it's a problem on a different level. 
(And `mmap()` is  system.)
Feb 21 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/16/2016 11:29 AM, Marc Schütz wrote:

 For example, it's always possible to use a global mutable associative array to
 store additional data connected with an immutable or const object (ignoring
 purity issues for the moment). That's safe because from the outside, there's no
 observable change to the state of the object itself, and the global AA's type
 (shared/thread-local) prevents race conditions.


The trouble with that is you're relying on the programmer to ensure
correctness. 
It'll revert D to C++ "trust the programmer" semantics.

Furthermore, with the current mechanical guarantee of immutability, it opens up 
the possibility of relying on such beyond merely observable behavior - such as 
immutable data being placed in ROM, or special GC semantics.
Feb 16 2016
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 17 February 2016 at 02:51:06 UTC, Walter Bright 
wrote:

 The trouble with that is you're relying on the programmer to 
 ensure correctness. It'll revert D to C++ "trust the 
 programmer" semantics.


You might end up in a worse position than C++, as people will 
cast away constness / immutability in third party libraries to 
get around the limitations.

C++ added "mutable" keyword for a reason... with head const which 
is much less limiting to begin with...



 Furthermore, with the current mechanical guarantee of 
 immutability, it opens up the possibility of relying on such 
 beyond merely observable behavior - such as immutable data 
 being placed in ROM, or special GC semantics.


A nonissue.

1. The compiler has access to all reachable types. It can easily 
determine if there are reachable types with  mutable fields.

2. This is not a good argument for transitivity of immutability 
as head-immutability is sufficient.
Feb 16 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/16/2016 9:56 PM, Ola Fosheim Grøstad wrote:

 1. The compiler has access to all reachable types.


Nope. Opaque types are supported.
Feb 16 2016
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 02/16/2016 09:51 PM, Walter Bright wrote:

 On 2/16/2016 11:29 AM, Marc Schütz wrote:

 For example, it's always possible to use a global mutable associative
 array to
 store additional data connected with an immutable or const object
 (ignoring
 purity issues for the moment). That's safe because from the outside,
 there's no
 observable change to the state of the object itself, and the global
 AA's type
 (shared/thread-local) prevents race conditions.


 The trouble with that is you're relying on the programmer to ensure
 correctness. It'll revert D to C++ "trust the programmer" semantics.


That doesn't seem the case to me. -- Andrei
Feb 17 2016
prev sibling parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Wednesday, 17 February 2016 at 02:51:06 UTC, Walter Bright 
wrote:

 On 2/16/2016 11:29 AM, Marc Schütz wrote:

 For example, it's always possible to use a global mutable 
 associative array to
 store additional data connected with an immutable or const 
 object (ignoring
 purity issues for the moment). That's safe because from the 
 outside, there's no
 observable change to the state of the object itself, and the 
 global AA's type
 (shared/thread-local) prevents race conditions.


 The trouble with that is you're relying on the programmer to 
 ensure correctness. It'll revert D to C++ "trust the 
 programmer" semantics.


Are you referring to the quoted passage, or my DIP? If the 
latter, please note that it was initially written for a different 
purpose. Operations with  mutables will be  system, which is our 
usual way to deal with situations in which the programmer has to 
ensure correctness. The unsafe ( trusted) part is very small and 
well encapsulated, the unsafety doesn't leak.


 Furthermore, with the current mechanical guarantee of 
 immutability, it opens up the possibility of relying on such 
 beyond merely observable behavior - such as immutable data 
 being placed in ROM, or special GC semantics.


That's already covered in the DIP, see my reply to Dicebot.
Feb 17 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2016 10:54 AM, Marc Schütz wrote:

 That's already covered in the DIP, see my reply to Dicebot.


Note that D supports opaque types, meaning it is not always possible to 
transitively follow all types to see if they have mutable members.
Feb 17 2016
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, 17 February 2016 at 22:06:20 UTC, Walter Bright 
wrote:

 On 2/17/2016 10:54 AM, Marc Schütz wrote:

 That's already covered in the DIP, see my reply to Dicebot.


 Note that D supports opaque types, meaning it is not always 
 possible to transitively follow all types to see if they have 
 mutable members.


We _could_ have something like  mutable on member variables which 
made them mutable in spite of being in a const object and which 
made it illegal for that type to ever be immutable, but we'd be 
forced to have an additional attribute on the type itself (e.g. 
 contains_mutable) because of the opaque type issue, and at that 
point, the compiler could just look at the attribute on the type 
to know that it couldn't be immutable, and similar to abstract, 
it could then require that any type that it's a member of t hen 
be marked with that attribute. So, it's uglier than simply 
marking a member variable with  mutable, but it's certainly 
feasible.

The question really is whether we want such a backdoor in const 
and whether it's worth the extra complication. It _would_ be 
explicitly marked, making it fairly easy to determine when a 
const type might actually mutate some of its state (though 
presumably, if it's used responsibly, it would only mutate state 
that was really part of the logical state of the object - e.g. a 
mutex or a reference count). So, while const would be weakened on 
some level, it wouldn't be completely invisible, and you couldn't 
accidentally have a member variable with an  mutable member, 
since you'd be forced to mark the type containing it with 
 contains_mutable. And it _would_ allow a number of idioms that 
many folks keep wanting and complaining about without actually 
making it defined behavior to cast away const and mutate (even 
Andrei has been running into issues lately with containers and 
RCString where he's needed  mutable and then done stuff like cast 
away const and mutating, thinking that that was okay as long as 
the underlying object wasn't immutable). It would be a backdoor, 
but it would be a well-defined one rather than allowing a 
free-for-all.

I honestly don't know whether adding something like  mutable is a 
good idea or not, but it is very clear that there are a number of 
idioms that are impossible in D when const is involved that lead 
a lot of programmers to either not use const or to cast it away 
and mutate, thinking that that's okay, because it's okay in C++, 
and it does tend to work as long as immutable isn't involved, 
much as it's undefined behavior. Having  mutable would make const 
usable for a lot of programs that otherwise have to mostly 
abandon it, but it would add some extra complication to the 
language, and it would mean that there's a backdoor in const, 
even if it's one that's actually safe to use (unlike casting away 
const and mutating).

But what _is_ clear to me is that transitive const has been a big 
win, and while I can definitely live with C++'s version of const 
if I have to, it drives me nuts now that it's not transitive. 
Without transitivity, it's too easy to end up with mutable 
references to stuff from const functions, just because they're 
not directly part of the object. Transitivity fixes all that.

- Jonathan M Davis
Feb 17 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2016 2:31 PM, Jonathan M Davis wrote:

 We _could_ have something like  mutable on member variables which made them
 mutable in spite of being in a const object and which made it illegal for that
 type to ever be immutable, but we'd be forced to have an additional attribute
on
 the type itself (e.g.  contains_mutable) because of the opaque type issue, and
 at that point, the compiler could just look at the attribute on the type to
know
 that it couldn't be immutable, and similar to abstract, it could then require
 that any type that it's a member of t hen be marked with that attribute. So,
 it's uglier than simply marking a member variable with  mutable, but it's
 certainly feasible.


It would seem that implementing headconst as a type constructor would let
people 
who wanted mutable members have their way, without introducing backdoors in
const.
Feb 17 2016
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, 17 February 2016 at 22:44:27 UTC, Walter Bright 
wrote:


 It would seem that implementing headconst as a type constructor 
 would let people who wanted mutable members have their way, 
 without introducing backdoors in const.


I really don't see how that's the same thing at all. The main 
problems with const pop up when what folks need is something like 
C++'s mutable, because they need to be able to do something like 
a reference count or a mutex which is not really part of the 
logical state of the object but is still part of the object. 
Having some form of non-transitive const allows for things like a 
const pointer to non-const data, but it doesn't help at all when 
what you need to do is treat most of the object as const while 
treating a small portion of it as mutable. Sure, that's not 
completely transitive, because parts of the object are mutable, 
but it's specific parts of the object, not which part of a 
pointer declaration is const and which isn't. Maybe I'm missing 
something, but I don't see how adding non-transitive const to D 
in order to solve the C++ declaration problem would help at all 
with the complaints that folks have with D's const. The 
complaints with D's const are almost entirely about the lack of 
backdoors (be they well-defined like a mutable attribute or a 
free-for-all like casting away const and mutating).

- Jonathan M Davis
Feb 17 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2016 3:12 PM, Jonathan M Davis wrote:

 On Wednesday, 17 February 2016 at 22:44:27 UTC, Walter Bright wrote:


 It would seem that implementing headconst as a type constructor would let
 people who wanted mutable members have their way, without introducing
 backdoors in const.


 I really don't see how that's the same thing at all. The main problems with
 const pop up when what folks need is something like C++'s mutable, because they
 need to be able to do something like a reference count or a mutex which is not
 really part of the logical state of the object but is still part of the object.
 Having some form of non-transitive const allows for things like a const pointer
 to non-const data, but it doesn't help at all when what you need to do is treat
 most of the object as const while treating a small portion of it as mutable.
 Sure, that's not completely transitive, because parts of the object are
mutable,
 but it's specific parts of the object, not which part of a pointer declaration
 is const and which isn't. Maybe I'm missing something, but I don't see how
 adding non-transitive const to D in order to solve the C++ declaration problem
 would help at all with the complaints that folks have with D's const. The
 complaints with D's const are almost entirely about the lack of backdoors (be
 they well-defined like a mutable attribute or a free-for-all like casting away
 const and mutating).


If the headconst was one level up, the struct can have mutating members.
Feb 17 2016
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, 17 February 2016 at 23:47:54 UTC, Walter Bright 
wrote:

 On 2/17/2016 3:12 PM, Jonathan M Davis wrote:

 On Wednesday, 17 February 2016 at 22:44:27 UTC, Walter Bright 
 wrote:


 It would seem that implementing headconst as a type 
 constructor would let
 people who wanted mutable members have their way, without 
 introducing
 backdoors in const.


 I really don't see how that's the same thing at all. The main 
 problems with
 const pop up when what folks need is something like C++'s 
 mutable, because they
 need to be able to do something like a reference count or a 
 mutex which is not
 really part of the logical state of the object but is still 
 part of the object.
 Having some form of non-transitive const allows for things 
 like a const pointer
 to non-const data, but it doesn't help at all when what you 
 need to do is treat
 most of the object as const while treating a small portion of 
 it as mutable.
 Sure, that's not completely transitive, because parts of the 
 object are mutable,
 but it's specific parts of the object, not which part of a 
 pointer declaration
 is const and which isn't. Maybe I'm missing something, but I 
 don't see how
 adding non-transitive const to D in order to solve the C++ 
 declaration problem
 would help at all with the complaints that folks have with D's 
 const. The
 complaints with D's const are almost entirely about the lack 
 of backdoors (be
 they well-defined like a mutable attribute or a free-for-all 
 like casting away
 const and mutating).


 If the headconst was one level up, the struct can have mutating 
 members.


Yes, but that really isn't going to help much code. It would be 
useless for ref-counting const objects, it wouldn't allow you to 
put a mutex in a const object, and you couldn't do anything with 
caching calculated properties in a const object. All it means is 
that you can do something like have a const pointer to a mutable 
object, which in my experience is usually useless. There have 
been plenty of complaints about having to use Rebindable instead 
of having tail-const for classes, but I've rarely seen anyone 
complain about the lack of head-const. And what's usually the 
case is that folks want logical const, and I don't see how that's 
possible without either having something similar to C++'s mutable 
or make it well-defined to cast away const and mutate.

Having something similar to C++'s mutable would at least solve 
most of the problem while making it explicit and safe, whereas 
allowing casting away const and mutating would just throw all of 
the guarantees of const out the window and risk serious problems 
with immutable, making it so that all const really did was 
prevent accidental mutation and serve as documentation of intent. 
Something like  mutable would at least retain the guarantees that 
we currently have when  mutable isn't used and restrict the 
effects of removing const to very well defined areas such that it 
wouldn't run afoul of immutable.

So, if we want to actually solve the problems that folks 
typically complain about with const, I think that something like 
 mutable is the way to go. Now, we can certainly decide that 
that's not worth it and that those idioms simply won't work in D 
as long as const is used, but I really don't think that 
head-const is going to solve the same problem at all or really 
make much of anyone much happier aside from how it helps with C++ 
interoperability. Adding tail-const for classes to the language 
would make a whale of a lot more people happy than head-const 
would even though Rebindable ostensibly solves that problem 
already.

- Jonathan M Davis
Feb 17 2016
next sibling parent reply Bottled Gin <Gin Bottled.com> writes:
Greetings

Having coded a multithreaded library in D, I sorely miss headcost 
like specifier.

The library code dealt with lots of composite class object 
hierarchy where many element objects of a parent are *effectively 
immutable*. You build an effectively immutable object in the 
constructor of the parent class and then you are done since you 
do not assign to these object handles again. As a result there is 
no need to lock the parent object while accessing an *effectively 
immutable* object. The parent as well as child objects are 
mutable otherwise.

Dlang's immutable and const do not provide such a contract.

So +1 for headconst if this specifier can be used with class 
elements as well and not just in the function parameter list.

- Puneet
Feb 17 2016
parent Bottled Gin <Gin Bottled.com> writes:
Too illustrate how headconst can help with multithreading (shared 
memory) code, I have created this small snippet. In my 
experience, there are lots of *effectively immutable* objects 
(like Foo.bar in the code snippet). If these variables can be 
declared headconst, a lot of hierarchical synchronization locking 
(and therefor deadlock situations) can be saved. In the present 
Dlang semantics we do not have any specifier to make *effective 
mutability* effective. So presently I have to create a naming and 
coding discipline when declaring and using effectively mutable 
hierarchical objects.


class Bar {
   void funcBar() { }
}

class Frop {
   void funcFrop() { }
}

class Foo {
   this() {
     synchronized(this) {
       bar = new Bar();
     }
   }

   void setFrop() {
     frop = new Frop();
   }

   headconst Bar bar;		// effectively immutable
   Frop frop;			// mutable

   void funcBar() {		// no locking required
     bar.funcBar();		// since bar is effectively immutable
   }

   void funcFrop() {
     synchronized(this) {	// lock necessary since frop
       frop.funcFrop();		// is mutable
     }
   }
}

void main() {
   Foo foo = new Foo();
   // yield for other threads
   // foo.setFrop();    // in some other thread
   foo.funcBar();
   foo.funcFrop();
}
Feb 17 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2016 4:41 PM, Jonathan M Davis wrote:

 Yes, but that really isn't going to help much code. It would be useless for
 ref-counting const objects, it wouldn't allow you to put a mutex in a const
 object, and you couldn't do anything with caching calculated properties in a
 const object.


Embed a headconst pointer to the thing you want to mutate.
Feb 17 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 18.02.2016 08:51, Walter Bright wrote:

 On 2/17/2016 4:41 PM, Jonathan M Davis wrote:

 Yes, but that really isn't going to help much code. It would be
 useless for
 ref-counting const objects, it wouldn't allow you to put a mutex in a
 const
 object, and you couldn't do anything with caching calculated
 properties in a
 const object.


 Embed a headconst pointer to the thing you want to mutate.


const(headconst(T)) is the same as const(T), no?
Feb 17 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2016 11:58 PM, Timon Gehr wrote:

 const(headconst(T)) is the same as const(T), no?


Er, yes?
Feb 18 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 18.02.2016 10:24, Walter Bright wrote:

 On 2/17/2016 11:58 PM, Timon Gehr wrote:

 const(headconst(T)) is the same as const(T), no?


 Er, yes?


Jonathan wanted to embed a mutable reference count within a const 
object. Contrary to your suggestion, headconst won't help with that.
Feb 18 2016
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, 18 February 2016 at 10:47:57 UTC, Timon Gehr wrote:

 On 18.02.2016 10:24, Walter Bright wrote:

 On 2/17/2016 11:58 PM, Timon Gehr wrote:

 const(headconst(T)) is the same as const(T), no?


 Er, yes?


 Jonathan wanted to embed a mutable reference count within a 
 const object. Contrary to your suggestion, headconst won't help 
 with that.


Exactly. The problem that folks frequently want to be able to 
solve that they simply cannot solve with D's const (and headconst 
wouldn't help) is that they want to be able to pass an object to 
a function that takes it as const or return a const object from a 
const member function and have stuff like reference counting, 
caching, mutexes, etc. work - stuff that has to be part of the 
object (at least in pure code) but which isn't part of its 
logical state and cannot be const. That's why people keep asking 
for logical const. D's const simply does not work with many use 
cases (especially in the more performance-driven,  system cases), 
which tends to mean that you just can't use it. Generic code in 
particular can't afford to use it, because it immediately shuts 
out a whole set of types (just like marking a templated function 
with  safe or pure is a bad idea). So, const ends up being useful 
with built-in stuff like integers and arrays, but it generally 
fails once user-defined types get involved.

And remember that headconst is essentially cppconst, and C++ has 
to have the mutable keyword and allow casting away const and 
mutating in order to solve these sorts of problems. From what I 
can tell, fundamentally, there are some common use cases that 
will not work with const unless it has backdoors. Mutexes and 
reference counts are great examples of that. They need to be 
associated with the object, but they cannot be treated as const 
in order to work even if the actual data is treated as const. 
Aside from C++ interoperability, headconst really doesn't seem 
like it's going to solve much.

I love the fact that D treats it as undefined behavior to cast 
away const and mutate. That's a huge win with regards to compiler 
guarantees and being able to trust const not to mutate stuff on 
its own (though other, mutable references to the same data 
obviously can). And it's required for const to interact well with 
immutable. But having no backdoors at all keeps popping up as a 
problem - and one that headconst clearly can't solve, because it 
doesn't solve it in C++. Andrei has run into problems with this 
as he's being working on the new containers and RCString. What he 
needed was the mutable keyword, and he was casting away const and 
mutating to do it, and I had to point out to him that that wasn't 
defined behavior. And if he's forgetting that, what's the lay 
user doing? Some things need backdoors from const, or they simply 
can't use const, and if you use those in your code, it quickly 
cascades such that you're not using const much of anywhere, 
because you can't. A number of folks have stated in the newsgroup 
that they generally avoid const in D, because it's too 
restrictive to be useful.

We can decide that we just don't care about those problems and 
that you simply don't get to use const in those cases, but that 
means that you lose out on all of the benefits that you get with 
const preventing you from mutating objects that aren't supposed 
to be mutated, and if you have to strip out const from enough of 
your code base, then you lose out on its benefits entirely, 
meaning that C++'s const with its backdoors would have been a win 
in comparison, sad as that may be.

The more I look at it, the more I'm inclined to think that 
introducing  mutable for member variables with a corresponding, 
required attribute on the struct or class it's in (e.g. 
 has_mutable) is really what we need to be able to solve this 
problem and make D usable in some of these high performance cases 
that would be using the mutable keyword in C++. It solves the 
logical const problem without totally throwing away the compiler 
guarantees. Any type without  has_mutable functions as it always 
has, and the cases where  mutable/ has_mutable would be required 
would then work with const, gaining all of its benefits for the 
non- mutable members, and it would allow the compiler to prevent 
you from doing stupid stuff like mutating immutable objects, 
because an object with  has_mutable couldn't be immutable.

Regardless, I really don't think that adding headconst is worth 
the complication. It doesn't solve enough to be worth it. I'd 
much rather see something specific to extern(C++) that's used for 
mangling and which doesn't allow you to pass const objects to it 
unless the C++ declaration is equivalent to transitive const (and 
probably which treats return types as transitive const if they're 
partially const). Remember that we ditched having headconst, 
tailconst, etc. in D ages ago, because it was deemed 
overcomplicated. And I don't think that that's changed. 
Personally, I think that it's too complicated in C++, with 
declarations like const T*, T const*, T* const, etc. Most 
non-experts are going to fall flat on their face trying to 
decipher C++ declarations with const in them once you start doing 
much beyond const T or const T*. Let's please not go anywhere 
near there with D.

- Jonathan M Davis
Feb 18 2016
next sibling parent reply ixid <nuaccount gmail.com> writes:
On Thursday, 18 February 2016 at 11:57:59 UTC, Jonathan M Davis 
wrote:

The problem that folks frequently want to be able to solve that 
they simply cannot solve >with D's const (and headconst wouldn't 
help) is that they want to be able to pass an object >to a 
function that takes it as const or return a const object from a 
const member function >and have stuff like reference counting, 
caching, mutexes, etc. work - stuff that has to be >part of the 
object (at least in pure code) but which isn't part of its 
logical state and >cannot be const.


Is it not possible to have two objects, the data and the 
information about the data? It seems like a mistake to try to 
treat metadata as the data. I just ask out of interest as I lack 
the experience to have a meaningful view, the category confusion 
between data and metadata seems like a path to excessive 
complexity.
Feb 18 2016
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Thursday, 18 February 2016 at 14:58:00 UTC, ixid wrote:

 Is it not possible to have two objects, the data and the 
 information about the data? It seems like a mistake to try to


Not if the immutability is transitive. But "struct" isn't really 
an object, it is a chunk of memory. Often an object is 
represented as a chunk of memory... sometimes multiple objects 
are represented in the same chunk of memory for efficiency. 
Sometimes an object is distributed over many chunks of memory...

Say, if you model a person using a chat service. You might want 
only one context to have full write access, but all context to 
have access to an "is busy" field.

No point in putting "is busy" in a different memory chunk.
Feb 18 2016
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, 18 February 2016 at 14:58:00 UTC, ixid wrote:

 On Thursday, 18 February 2016 at 11:57:59 UTC, Jonathan M Davis 
 wrote:

The problem that folks frequently want to be able to solve that 
they simply cannot solve >with D's const (and headconst 
wouldn't help) is that they want to be able to pass an object

to a function that takes it as const or return a const object


from a const member function >and have stuff like reference 
counting, caching, mutexes, etc. work - stuff that has to be

part of the object (at least in pure code) but which isn't


part of its logical state and >cannot be const.


 Is it not possible to have two objects, the data and the 
 information about the data? It seems like a mistake to try to 
 treat metadata as the data. I just ask out of interest as I 
 lack the experience to have a meaningful view, the category 
 confusion between data and metadata seems like a path to 
 excessive complexity.


To some extent yes, to some extent no. Lets' say that we have a 
RefCounted wrapper which does reference counting, e.g.

     RefCounted!Foo foo;

The ref count is in RefCounted, not Foo, so you can do something 
like

     RefCounted!(const Foo) foo;

and the ref-count will still work, but if you do

     const RefCounted!Foo foo;

or

     const RefCounted!(const Foo) foo;

RefCounted can't actually mutate its ref-count anymore, even if 
it's copied, because the copy would have a const reference to the 
ref-count, because the copy was made from a const RefCounted!Foo 
and not a RefCounted!Foo. So, it really only helps you for one 
level of const. The only way to get around the problem would be 
if the ref-count were in a global or static variable somewhere, 
in which case, it can't be used in pure functions, which is 
definitely negative.

Similarly, what do you do with cached data? In C++, cached data 
would normally go in a mutable member variable so that the 
calculation could be saved, even if a const function called it - 
but since the calculation was cached and wouldn't change, the 
const function would still be logically const. The only way to do 
that in D is to either have the cache set only in mutable 
functions or to make the function not pure and put the data out 
in a global or static variable somewhere, which again, is 
definitely bad.

Similarly, mutexes really should be associated with the data that 
they're protecting, which often means being passed around with 
the object that contains that data. And that won't work with 
const. You'd once again be forced to put the mutex it a global or 
static variable and forego pure. synchronized works around this 
for some cases by having the type system put the data somewhere 
that's not treated as const, but it's a special case added by the 
language. It wouldn't work at all with explicit mutexes that 
didn't have special language support like that.

In C++, these sorts of problems are solved by marking the 
ref-count, cached variable, mutex, etc. as mutable. Then that 
variable can be mutated in order to do what it needs to do even 
though it's being used in a context where the object that it's in 
is const. Obviously, that can be abused by marking everything as 
mutable and effectively making const meaningless (which is part 
of why Walter isn't a big fan of C++'s const), but in practice, 
that doesn't normally happen, and without mutable or casting away 
const and mutating (neither of which are valid D), you just can't 
do those things with const - especially when you add pure into 
the mix.

- Jonathan M Davis
Feb 18 2016
prev sibling parent reply Doc <x x.com> writes:
On Thursday, 18 February 2016 at 11:57:59 UTC, Jonathan M Davis 
wrote:


 The more I look at it, the more I'm inclined to think that 
 introducing  mutable for member variables with a corresponding, 
 required attribute on the struct or class it's in (e.g. 
  has_mutable) is really what we need to be able to solve this 
 problem and make D usable in some of these high performance 
 cases that would be using the mutable keyword in C++. It solves 
 the logical const problem without totally throwing away the 
 compiler guarantees. Any type without  has_mutable functions as 
 it always has, and the cases where  mutable/ has_mutable would 
 be required would then work with const, gaining all of its 
 benefits for the non- mutable members, and it would allow the 
 compiler to prevent you from doing stupid stuff like mutating 
 immutable objects, because an object with  has_mutable couldn't 
 be immutable.


 - Jonathan M Davis


Could we use a special class Interface for this to limit the 
widespread use of a new keyword or attribute?

I.e. classes could implement a special mutable RefCount (as an 
example) interface. Only code that refers to the object by it's 
mutable interface would be allowed to jailbreak its overall 
constness, and only for those members defined in the mutable 
interface.

Maybe add a MutableInterface keyword or an attribute strictly 
valid in Interface declarations.

Just a late night brainstorm.

-Doc
Feb 18 2016
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, 19 February 2016 at 05:33:54 UTC, Doc wrote:

 On Thursday, 18 February 2016 at 11:57:59 UTC, Jonathan M Davis 
 wrote:


 The more I look at it, the more I'm inclined to think that 
 introducing  mutable for member variables with a 
 corresponding, required attribute on the struct or class it's 
 in (e.g.  has_mutable) is really what we need to be able to 
 solve this problem and make D usable in some of these high 
 performance cases that would be using the mutable keyword in 
 C++. It solves the logical const problem without totally 
 throwing away the compiler guarantees. Any type without 
  has_mutable functions as it always has, and the cases where 
  mutable/ has_mutable would be required would then work with 
 const, gaining all of its benefits for the non- mutable 
 members, and it would allow the compiler to prevent you from 
 doing stupid stuff like mutating immutable objects, because an 
 object with  has_mutable couldn't be immutable.


 - Jonathan M Davis


 Could we use a special class Interface for this to limit the 
 widespread use of a new keyword or attribute?

 I.e. classes could implement a special mutable RefCount (as an 
 example) interface. Only code that refers to the object by it's 
 mutable interface would be allowed to jailbreak its overall 
 constness, and only for those members defined in the mutable 
 interface.

 Maybe add a MutableInterface keyword or an attribute strictly 
 valid in Interface declarations.

 Just a late night brainstorm.


The mutable keyword is used for stuff that frequently _isn't_ 
part of the API of a type and thus doing something with 
interfaces doesn't make a lot of sense to me. And it's really the 
type that's constructed that needs to clearly have mutable 
members, not the interface, because the key thing is to avoid 
constructing immutable variables of that type. In addition, 
interfaces only apply to classes, which is far too limiting, even 
if it's otherwise a good idea.

Having  mutable would not result in  mutable being plastered 
everywhere in D anymore than mutable is in C++. If someone used 
it that much, then there isn't much point in using const in the 
first place. Rather, certain types will have  mutable on at most 
a few of their members, and they'll have to have  mutable on the 
type itself (or something like  has_mutable if it's problematic 
to reuse the same attribute on the type), so it'll be obvious 
when it's used and will prevent the compiler from constructing 
such objects as immutable, since that would break immutability. I 
wouldn't expect it to be intrusive at all, but it's critical for 
certain types of objects to work with const at all, and if those 
objects don't work with const, then the code bases that they're 
in will avoid const, and generic code will be forced to avoid 
const regardless, because it could be used with most anything - 
including stuff that can't be const, whereas if we had  mutable, 
that problem would be reduced considerably and maybe even 
eliminated.

- Jonathan M Davis
Feb 18 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/18/2016 2:47 AM, Timon Gehr wrote:

 On 18.02.2016 10:24, Walter Bright wrote:

 On 2/17/2016 11:58 PM, Timon Gehr wrote:

 const(headconst(T)) is the same as const(T), no?


 Er, yes?


 Jonathan wanted to embed a mutable reference count within a const object.
 Contrary to your suggestion, headconst won't help with that.



Of course it will:

headconst pointer -> pointer -> refcount
                   -> const pointer -> ... constant stuff ...
Feb 18 2016
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, 18 February 2016 at 12:03:00 UTC, Walter Bright 
wrote:

 On 2/18/2016 2:47 AM, Timon Gehr wrote:

 On 18.02.2016 10:24, Walter Bright wrote:

 On 2/17/2016 11:58 PM, Timon Gehr wrote:

 const(headconst(T)) is the same as const(T), no?


 Er, yes?


 Jonathan wanted to embed a mutable reference count within a 
 const object.
 Contrary to your suggestion, headconst won't help with that.



 Of course it will:

 headconst pointer -> pointer -> refcount
                   -> const pointer -> ... constant stuff ...


Sure, until you need to put that headconst pointer in a const 
object or pass it to a function that takes const. Then the whole 
thing is const, and the refcount doesn't work. Remember that C++ 
has headconst, and it still needs the mutable keyword in order to 
solve these problems. Also, adding headconst is suddenly going to 
create this dichotomy between const and headconst where you have 
to figure out which to use and where they really don't interact 
nicely, because you can't pass a const object to something that 
takes headconst, and if you do pass a headconst object to 
something that takes const, then all of the stuff like 
ref-counting stops working. In a way, we'd be forking D between 
const and headconst. Some code would use one, and some code would 
use the other. And neither of them really solves the problem that 
the mutable keyword solves. Neither of them allows for any kind 
of logical const, and _that_ really is the problem that people 
keep complaining about and want solved.

headconst may solve the extern(C++) problem, but it really 
doesn't solve the problems we have with const. At best, it 
reduces them slightly while introducing a whole extra set of 
complications to an already complicated language.

- Jonathan M Davis
Feb 18 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/18/2016 4:16 AM, Jonathan M Davis wrote:

 headconst may solve the extern(C++) problem, but it really doesn't solve the
 problems we have with const.


'mutable' doesn't really solve a problem, it just means that C++ 'const' is a 
documentation aid, not a guarantee.
Feb 18 2016
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, 18 February 2016 at 22:40:32 UTC, Walter Bright 
wrote:

 On 2/18/2016 4:16 AM, Jonathan M Davis wrote:

 headconst may solve the extern(C++) problem, but it really 
 doesn't solve the
 problems we have with const.


 'mutable' doesn't really solve a problem, it just means that 
 C++ 'const' is a documentation aid, not a guarantee.


It's still a guarantee for those members that aren't mutable. 
It's allowing casting away const and mutating that totally blows 
the guarantees out of the water. You can actually look at the 
type to see which members are mutable and know what might change, 
and usually it's stuff like mutexes that are perfectly okay to 
change, whereas with casting, you'd have to look at every spec of 
code that uses the object to see what it does if you want to be 
sure that a const object or particular member isn't mutated.

And while I very much like the fact that D doesn't consider 
casting away const and mutating to be defined behavior and that 
it has stronger guarantees than C++, the reality of the matter is 
that in many cases, it ultimately provides far worse guarantees 
than C++. While C++'s const does have too many backdoors, at 
least when you use it, it catches accidental mutation (e.g. 
getting the arguments reversed when calling copy from the 
algorithm header will result in an error). But if you're forced 
to drop const entirely as tends to happen in D, then you don't 
get that. So, for some code, the stricter const in D is great, 
but for a lot of it, it makes things worse. In spite of the 
stronger guarantees, you ultimately end up with less protection. 
And since the compiler actually lets you cast away const and 
mutate in D to your heart's content without complaining at all 
(even if it's technically undefined behavior), there are plenty 
of folks that do it, because D doesn't provide mutable, and they 
assume that because you can cast away const, mutating it must be 
fine, just like with C++. So, while in principle, D's const 
provides better guarantees, in practice, it really doesn't. 
Casting away const would have to be illegal for it really provide 
those guarantees. And because it's so restrictive, it frequently 
gets avoided anyway. So, even if it doesn't get circumvented, it 
still fails to provide good guarantees, because it's not used.

At least if we added  mutable, we'd be providing a controlled 
means of escaping const in the cases where it's needed, avoiding 
issues with immutable like you'd potentially get if you cast away 
const and mutated. And any code which didn't use  mutable would 
be able to rely on the full guarantees that we have now (the only 
problem being folks that cast away const and mutated, thinking 
that it was okay, but they'd have less incentive to if they had 
 mutable).

In principle, I agree that having something like  mutable does 
weaken const, but in practice, it does a much better job of 
catching bugs, because then you're actually able to use const 
with more complicated objects. And unlike casting away const, 
 mutable is easily detected and accounted for.

D's const simply doesn't work once you start doing stuff like 
putting allocators or reference counts into the mix. And while 
maybe we could not care about that when we thought that it was 
fine to use the GC with everything, I think that we have a much 
harder time holding that position now. As principled as D's 
current position on const may be, it's highly impractical.

- Jonathan M Davis
Feb 18 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/18/2016 9:46 PM, Jonathan M Davis wrote:

 On Thursday, 18 February 2016 at 22:40:32 UTC, Walter Bright wrote:

 On 2/18/2016 4:16 AM, Jonathan M Davis wrote:

 headconst may solve the extern(C++) problem, but it really doesn't solve the
 problems we have with const.


 'mutable' doesn't really solve a problem, it just means that C++ 'const' is a
 documentation aid, not a guarantee.


 It's still a guarantee for those members that aren't mutable. It's allowing
 casting away const and mutating that totally blows the guarantees out of the
 water.


That's why such casts are not allowed in D  safe code.

C++ const does not come with mechanically checkable guarantees, and D's does. 
This makes all the difference.

Allow  mutable, and no more mechanical checking in D. Recall that D supports 
opaque types, meaning types are not fully known to the compiler.
Feb 18 2016
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, 19 February 2016 at 06:39:53 UTC, Walter Bright wrote:

 On 2/18/2016 9:46 PM, Jonathan M Davis wrote:

 On Thursday, 18 February 2016 at 22:40:32 UTC, Walter Bright 
 wrote:

 On 2/18/2016 4:16 AM, Jonathan M Davis wrote:

 headconst may solve the extern(C++) problem, but it really 
 doesn't solve the
 problems we have with const.


 'mutable' doesn't really solve a problem, it just means that 
 C++ 'const' is a
 documentation aid, not a guarantee.


 It's still a guarantee for those members that aren't mutable. 
 It's allowing
 casting away const and mutating that totally blows the 
 guarantees out of the
 water.


 That's why such casts are not allowed in D  safe code.

 C++ const does not come with mechanically checkable guarantees, 
 and D's does. This makes all the difference.


Except that if you're calling 3rd party code, it's free to cast 
away const and mutate and slap an  trusted on there, and if it's 
more than one level deep in the call stack, they could slap an 
 safe on the function you're actually calling, and you wouldn't 
have a clue that they're violating the type system - and it could 
be with something you passed to them. Yes,  safe helps some, but 
ultimately, you still rely on the programmer who wrote the code 
you're using to behave responsibly. Ultimately, the D compiler 
doesn't prevent code from mutating const any more than the C++ 
compiler does. The primary differences are that C++ considers it 
defined behaved, whereas D does not, and D has immutable to worry 
about, whereas C++ does not. And unless the compiler is 
optimizing based on const, as long as the data in question was 
not constructed as immutable, casting away const and mutating 
will work in D just as well as it does in C++. So, it's easy for 
a programmer to think that it's perfectly legitimate to cast away 
const and mutate and then think that it's  system purely because 
of the possibility of the data actually being immutable and not 
even realize that it's undefined behavior even when the data was 
constructed as mutable. Even Andrei made that mistake fairly 
recently:

http://forum.dlang.org/post/n25qkc$2il8$1 digitalmars.com

He cast away const from the allocator member inside of a const 
member function so that he could provide access to it, since it 
was not part of the logical state of the container and really 
shouldn't have been const but had to be, because the container 
was const.  mutable is exactly what was needed in this case. But 
he (and Dicebot) thought that casting away const and mutating was 
defined behaved as long as the underlying data was mutable until 
I pointed out otherwise.

http://forum.dlang.org/post/resphkhblryhrlznxtbq forum.dlang.org

While in theory, you can't cast away D's const and mutate, all 
that's preventing you is  safe, and it's clear that even expert D 
programmers who should arguably know better don't know better. 
I've heard (though haven't verified) that vibe.d casts away const 
to do reference counting, and it's clear from some of the 
stackoverflow answers that a number of D programmers think that 
casting away const and mutating is fine, because D is a systems 
language.

Programmers are casting away const and mutating in practice, and 
the compiler is not preventing it. We're in exactly the same boat 
as C++ in that we rely on the programmer to be smart about 
casting away const in order for const to provide any guarantees. 
We just made it undefined behavior when they do cast away const 
and made it warn about it slightly better by making it  system. 
But it still works in practice, and folks are still doing it. So, 
ultimately, I don't think that the idea that D's const guarantees 
that the object isn't mutated via that reference holds water much 
better than it does with C++'s const, even if in theory it 
should. In both cases, it relies on the programmer to do the 
right thing, and the compiler can't do much if you insist on 
doing the wrong thing, thinking that you know what you're doing. 
And the fact that it actually works in practice to cast away 
const and mutate doesn't help matters any in preventing it.


 Allow  mutable, and no more mechanical checking in D. Recall 
 that D supports opaque types, meaning types are not fully known 
 to the compiler.


Yes, which is why I said that a type with an  mutable member 
would be forced to be marked with  mutable as well (or something 
like  has_mutable if it's problematic to reuse the attribute on 
the class/struct). It would be the same as with an abstract 
class. So, then anything that wouldn't work with  mutable (e.g. 
constructing the type as immutable) could be statically 
prevented, even if the type were opaque, just like you can't 
construct an abstract class. it would obviously have to bubble up 
such that a type that contains an  mutable type would also have 
to be an  mutable type, but it would solve the opaque type 
problem.

- Jonathan M Davis
Feb 18 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/18/2016 11:46 PM, Jonathan M Davis wrote:

 We're in exactly the same boat as C++ in that we rely on the
 programmer to be smart about casting away const in order for const to provide
 any guarantees.


No, we're not.  trusted is specifically designed to be greppable (so are casts, 
by the way). This is quite unlike C++. I defy you (or anyone) to look at a
piece 
of non-trivial C++ code and verify there are no deviated preversions going on.

D is simply not in the same boat at all.

We can debate about the holes in  safe that get bug reports filed on them, but 
these are bugs and we intent to fix all of them. There is no plan nor proposal 
to fix C++ unsafety.


 I said that a type with an  mutable member would be forced to be marked with 


 mutable as well

You're proposing ' mutable const' ?
Feb 19 2016
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, 19 February 2016 at 10:03:44 UTC, Walter Bright wrote:

 On 2/18/2016 11:46 PM, Jonathan M Davis wrote:

 We're in exactly the same boat as C++ in that we rely on the
 programmer to be smart about casting away const in order for 
 const to provide
 any guarantees.


 No, we're not.  trusted is specifically designed to be 
 greppable (so are casts, by the way). This is quite unlike C++. 
 I defy you (or anyone) to look at a piece of non-trivial C++ 
 code and verify there are no deviated preversions going on.

 D is simply not in the same boat at all.


D is definitely in a better boat in that it's easier to find and 
catch problems with const, but it really doesn't do much more to 
actually guarantee that a const object isn't being violated. 
Programmers are free to do horrible things and mark it as 
 trusted, possibly thinking that what they're doing is fine. And 
in D, programmers are frequently tempted to cast away const 
thinking that it's a legitimate backdoor like it is in C++ just 
so long as the object isn't actually immutable, especially 
because there is no mutable keyword in D.


 We can debate about the holes in  safe that get bug reports 
 filed on them, but these are bugs and we intent to fix all of 
 them. There is no plan nor proposal to fix C++ unsafety.


Sure, but the main problem here is with code that is marked as 
 trusted. If you want to be 100% sure that an object isn't 
mutated via a const reference, you're still ultimately forced to 
dig through the code to verify it. That attack space is much 
reduced in comparison to C++, and it's much more greppable, so 
finding and fixing the problem is much easier, but the compiler 
still ultimately fails to actually guarantee that an object is 
not mutated via a const reference. To do that, it would have to 
be outright illegal to cast away const.

So, are we in a much better position than C++ with regards to 
const protecting against mutation? Yes. But we're not actually 
preventing it. And because of how restrictive D's const is, folks 
either end up casting away const to mutate, or they abandon const 
altogether. And if they abandon const altogether, then they 
actually end up with more error-prone code, because they don't 
have it preventing any mutation at all. Even protection with 
holes is better than no protection. But we don't even need to add 
holes on the same level as C++ in order to make D's const 
considerably more usable. I completely agree that we shouldn't 
make casting away const and mutating defined behavior (much as we 
can't actually prevent it), but having an equivalent to C++'s 
mutable would help considerably.


 I said that a type with an  mutable member would be forced to


 be marked with  mutable as well

 You're proposing ' mutable const' ?


No, whether a type has  mutable members is part of its 
definition, not a type qualifier.  I'm proposing that we have 
something like

 mutable struct S
{
     int i;
      mutable int* refCount;
     ...
}

or

 mutable class C
{
public:
      property int i() shared const { Guard guard(m); return i; }
      property void i(int val) shared { Guard guard(m); i = val; }
private:
     shared int i;
     shared  mutable Mutex m;
}

Then if you had an instance of C which was const, you could still 
mutate the Mutex member inside of a const function, and if it had 
a const instance of S, it could still mutate its ref-count 
appropriately in its postblit constructor and assignment operator 
and whatnot. And code which used S or C, would just use it like 
it would now. e.g.

     const S s;
     auto r1 = s.foo();

     const c = new C;
     auto r2 = c.i;

So, functionally, it should act the same as mutable in C++. 
However, it would act like abstract in that it would go with the 
members that it affects and with the type that contains such 
members. Any type that then contained such a type would also have 
to be marked with  mutable. e.g.

 mutable struct Foo
{
     // A member whose type declaration is marked with  mutable,
     // but the member variable itself is not  mutable
     C c;
}

Then if someone needed a mutable member, they could use  mutable 
just like they'd use the mutable keyword in C++, except that 
they'd also have to put it on the struct/class itself so that it 
works with opaque types and always makes it clear to the compiler 
that a type contains  mutable members and that it cannot do 
things like construct an immutable instance of it. If we had 
that, then all of the common uses cases for mutable in C++ that 
we currently cannot have in D (e.g. ref counting, mutexes, 
caching, etc.) could be used. And the code base wouldn't have to 
be littered with extra modifiers - just on the type declarations 
themselves for the types that have  mutable members and on the 
 mutable members themselves. Yes, it would be creating a backdoor 
in const, but it would be much safer and much more restricted 
than casting away const and mutating (assuming that that were 
legit), and it would take away most of the incentive to cast away 
const. It also wouldn't muck up const for code that doesn't use 
 mutable, and just like  trusted or cast, it would be highly 
greppable.

Certainly, if we're going to add a backdoor, I think that 
something well-controlled like this is the way to go, and if we 
don't add any backdoors, I expect that a lot of D code simply 
won't use const, and it'll lose out completely on the guarantees 
that const provides, leading to more bugs. Adding  mutable will 
increase type safety for code that can't currently use const 
while not reducing type safety for code that already uses const.

- Jonathan M Davis
Feb 19 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/19/2016 3:21 AM, Jonathan M Davis wrote:

 D is definitely in a better boat in that it's easier to find and catch problems
 with const, but it really doesn't do much more to actually guarantee that a
 const object isn't being violated. Programmers are free to do horrible things
 and mark it as  trusted,


The point of  trusted is it can be grepped for, and undergo extra scrutiny. The 
idea is to minimize  trusted sections. With C++, it's the WHOLE PROGRAM that
has 
to be scrutinized.

This is ORDERS OF MAGNITUDE different.



 I'm proposing that we have something like

  mutable struct S
 {
      int i;
       mutable int* refCount;
      ...
 }


This does not work for opaque types.
Feb 19 2016
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, 19 February 2016 at 12:03:01 UTC, Walter Bright wrote:

 On 2/19/2016 3:21 AM, Jonathan M Davis wrote:

 D is definitely in a better boat in that it's easier to find 
 and catch problems
 with const, but it really doesn't do much more to actually 
 guarantee that a
 const object isn't being violated. Programmers are free to do 
 horrible things
 and mark it as  trusted,


 The point of  trusted is it can be grepped for, and undergo 
 extra scrutiny. The idea is to minimize  trusted sections. With 
 C++, it's the WHOLE PROGRAM that has to be scrutinized.

 This is ORDERS OF MAGNITUDE different.


Yes, as long as you have the source code, finding  trusted 
violations is _way_ easier in D than it is in C++, but the fact 
that it's possible to cast away const and mutate still means that 
the compiler can't actually guarantee that an object is not 
mutated via a const reference to it. It can just guarantee that 
so long as  trusted code behaves itself. So, that's an enormous 
improvement, but it still means that it's possible for you to 
pass an object to a function that takes its argument by const and 
end up with that object being mutated by that function - and 
without you knowing about it unless you go digging through the 
code for  trusted violations.

But even if the compiler completely succeeded at guaranteeing 
that an object is not mutated via a const reference, there's the 
problem that because D's const is so restrictive, a lot of D code 
ends up not using it, which means that it gets _zero_ benefit 
from const, whereas in C++, it would have been const, and const 
would have caught some bugs even if it didn't provide strong 
guarantees. So, the code that could have used const in C++ but 
can't in D suffers because it doesn't even have partial 
protection by const. It's only the code that doesn't need 
something like the mutable keyword and which doesn't need to 
interact with code that needs something like the mutable keyword 
that can benefit from D's const. For instance, very little 
generic code in D is able to mark its parameters as const even if 
it doesn't mutate it, because there's a high chance that someone 
is going to want to be able to use that function with a type that 
won't work with const.

So, I think that even if D's const is far better than C++'s where 
it does work, we're actually worse off than C++ in a lot of code, 
because there are a lot of places in D where it doesn't work, and 
we can't use const, whereas the same code in C++ could have. So, 
we end up with no protection instead of partial protection, which 
is not an improvement. It's like we've come up with tank armor to 
protect people from bullets, and we won't let anyone wear bullet 
proof vests anymore, because they're not as good as tank armor, 
and so the folks who can't do their jobs behind tank armor are 
screwed when the bullets come.


 I'm proposing that we have something like

  mutable struct S
 {
      int i;
       mutable int* refCount;
      ...
 }


 This does not work for opaque types.


Why not? I would expect the opaque type to have to have it too, 
e.g.

      mutable struct S;

and that if it didn't, it wouldn't be compatible with the full 
type definition. But maybe I'm missing something with opaque 
types, since I don't use them much, and I don't think that I've 
ever used them in D. But the whole idea of putting  mutable on 
the type definition itself instead of just on the member variable 
was that opaque types would have to have it too and that seeing 
the  mutable members would not be required to know that a type 
contained  mutable members.

- Jonathan M Davis
Feb 19 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/19/2016 4:38 AM, Jonathan M Davis wrote:

 Yes, as long as you have the source code, finding  trusted violations is _way_
 easier in D than it is in C++, but the fact that it's possible to cast away
 const and mutate still means that the compiler can't actually guarantee that an
 object is not mutated via a const reference to it.



blocks, and Java has the JNI C interface. Even Haskell has its Foreign Function 
Interface.

The idea is to encapsulate such, which D does as well as any other language. 
This is not a defect of D.



 This does not work for opaque types.


 Why not? I would expect the opaque type to have to have it too, e.g.

       mutable struct S;


That would mean you're proposing ' mutable const' as a type constructor, which 
you'd earlier said otherwise.
Feb 19 2016
next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, 19 February 2016 at 21:53:16 UTC, Walter Bright wrote:

 On 2/19/2016 4:38 AM, Jonathan M Davis wrote:

 Yes, as long as you have the source code, finding  trusted 
 violations is _way_
 easier in D than it is in C++, but the fact that it's possible 
 to cast away
 const and mutate still means that the compiler can't actually 
 guarantee that an
 object is not mutated via a const reference to it.


 All languages that guarantee safety have this issue - Rust and 

 Haskell has its Foreign Function Interface.

 The idea is to encapsulate such, which D does as well as any 
 other language. This is not a defect of D.


I don't think that we should change D in that regard. It _is_ a 
problem with having "unsafe" blocks, and that's life with a 
systems language. I was just trying to make the point that the 
result is that the compiler can't actually guarantee that const 
isn't mutated. It can just reduce how much code could violate 
that guarantee and guarantee that as long as the programmer did 
the right thing in the  trusted code (or there is no  trusted 
code), then the guarantee about const not mutating will hold. So, 
it's making guarantees but with caveats where the caveats are 
fairly easy to track down (at least as long as you have the 
source), whereas C++'s guarantees have barn-sized caveats that 
are  _not_ easy to track down.


 This does not work for opaque types.


 Why not? I would expect the opaque type to have to have it 
 too, e.g.

       mutable struct S;


 That would mean you're proposing ' mutable const' as a type 
 constructor, which you'd earlier said otherwise.


Sorry about that. I think that I missed the "struct" in the 
declaration that you wrote earlier and misunderstood what you 
were talking about, but the idea is that  mutable would be part 
of the type such that you wouldn't be slapping  mutable on every 
declaration that uses the type - e.g.

auto foo( mutable S bar) {...}

wouldn't make sense (which is what I mistakenly thought you were 
referring to). But both when you have the full definition and an 
opaque declaration for the type, the  mutable has to be there, 
e.g.

 mutable struct S
{
    ...
}

or

 mutable struct S;

Then the fact that S has  mutable members is part of the type 
regardless of whether you see the full definition or an opaque 
one, and the compiler can treat it appropriately (also, any 
programmer trying to verify whether a type has  mutable members 
or not can look at either the full definition or the opaque one 
and see it at a glance). AFAIK, the only place that it would end 
up being hidden would be references to base classes, since the 
derived class could be  mutable, but the base class wouldn't be 
(since it has no  mutable members). However, since the  mutable 
would be visible when the type is constructed (since it would be 
constructing the derived type), the compiler could see that it 
wouldn't be legitimate to construct it as immutable (unless we 
figured out how to make  mutable work with immutable, but that's 
a whole other layer of complication of questionable value), which 
might introduce problems with pure functions implicitly casting 
to immutable if we're not careful, but I think that that's 
resolvable. Certainly, it would be impossible to construct an 
 mutable type as immutable.

But ultimately, any code that does not use  mutable should be the 
same as it is now, with the full set of guarantees that it 
currently has. It's just the new code that uses  mutable that 
would have reduced guarantees (but would then still have the full 
guarantees for the non- mutable members), and it would give us a 
backdoor that would not run afoul of immutable. It would solve 
the major use cases that are causing folks to cast away const and 
mutate, thinking that it was legit as long as the data wasn't 
actually immutable - including the cases that Andrei has had 
recently with allocators and reference counts. So, we essentially 
get a type safe logical const, though obviously, as with 
 trusted, it'll still be up to the programmer to not be stupid 
about what they mark as  mutable. But it'll also be easily 
greppable like  trusted is (and actually, as ugly as those   
symbols arguably are, they really do make grepping easier by 
significantly reducing the risk of false positives).

- Jonathan M Davis
Feb 20 2016
prev sibling parent Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Friday, 19 February 2016 at 21:53:16 UTC, Walter Bright wrote:

 On 2/19/2016 4:38 AM, Jonathan M Davis wrote:

 Why not? I would expect the opaque type to have to have it 
 too, e.g.

       mutable struct S;


 That would mean you're proposing ' mutable const' as a type 
 constructor, which you'd earlier said otherwise.


That's not a type constructor though, it's a type annotation. A 
declaration like

      mutable(S) s;

would not be allowed. Neither as a storage class in function 
signatures or for locals:

     void foo( mutable S s) {   // ERROR
          mutable const int i;  // ERROR
     }

What _is_ allowed is  mutable as a storage class (?) for members 
only:

     struct S {
          mutable int x;
     }
Feb 21 2016
prev sibling parent Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Friday, 19 February 2016 at 06:39:53 UTC, Walter Bright wrote:

 Allow  mutable, and no more mechanical checking in D. Recall 
 that D supports opaque types, meaning types are not fully known 
 to the compiler.


Allow  trusted, an no more mechanical checking in D. Recall that 
D supports extern functions, meaning function bodies are not 
fully known to the compiler.

Seriously, yes,  mutable is not mechanically checkable, which is 
why it is  system. That's intended, and not a flaw. It is 
supposed to make it possible to express things that don't 
otherwise fit the type system, and still be reasonably safe.

Jonathan's idea of attaching  mutable to the declaration handles 
the problem of opaque data structures really well. We can debate 
whether we require that annotation every time a struct contains 
 mutable members (even indirectly) to make it more obvious (if 
so, it still needs to be inferred for templated types), or if it 
should only be required for opaque types.
Feb 21 2016
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 18.02.2016 13:03, Walter Bright wrote:

 On 2/18/2016 2:47 AM, Timon Gehr wrote:

 On 18.02.2016 10:24, Walter Bright wrote:

 On 2/17/2016 11:58 PM, Timon Gehr wrote:

 const(headconst(T)) is the same as const(T), no?


 Er, yes?


 Jonathan wanted to embed a mutable reference count within a const object.
 Contrary to your suggestion, headconst won't help with that.



 Of course it will:

 headconst pointer -> pointer -> refcount
                    -> const pointer -> ... constant stuff ...


He wanted to embed a mutable reference count literally within a const 
object. Not a headconst object.
Feb 18 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/18/2016 10:22 AM, Timon Gehr wrote:

 He wanted to embed a mutable reference count literally within a const object.
 Not a headconst object.


I know. I pointed out how it could be done in a way to achieve the same effect.

BTW, shared_ptr<> uses a pointer to the ref count.
Feb 18 2016
parent reply Iakh <iaktakh gmail.com> writes:
On Thursday, 18 February 2016 at 22:46:04 UTC, Walter Bright 
wrote:

 On 2/18/2016 10:22 AM, Timon Gehr wrote:

 He wanted to embed a mutable reference count literally within 
 a const object.
 Not a headconst object.


 I know. I pointed out how it could be done in a way to achieve 
 the same effect.

 BTW, shared_ptr<> uses a pointer to the ref count.


Could D RefCounted!T be splitted into data (possibly immutable) 
and mutable metadata?

struct MutablePart
{
     int rc;
     T cache;
     Mutex mutex;
}

class A
{
    int data;
    // Consider mutable(T)* works like const(T)*
    private mutable(MutablePart)* metadata;
    this() {metadata = new MutablePart}
}

immutable A a;
a.metadata.rc++;

This way instance of A could be in ROM and ref counter is mutable.

If A is const (and this way possibly immutable and shared) 
situation as bad as in case of storing metadata in an allocator. 
Field A.metadata should be shared in some cases and thread local 
in others. In fact D have to add possibly_shared type constructor 
to represent type of metadata for const data.

To make mutable field friendly with  pure, mutable field should be
1) unique. Mutual mutable treat like global.
2) private
3) allowed to update only with expression like:
     this.updateCache(mutable.cache);
     where:
         updateCache() is const and pure;
         updateCache depends only on "this";
         Mutable parameter marked as out;
     So for two identical objects cache will always be the same.
     It is impossible to do incremental caching or even reuse 
resources.
4) accessible only by property like this:
    ref const(Data) getData() pure

Ofc metadata of immutable should be shared.

So what other things breaks mutable like this:
muttable(MutablePart)* metadata;
?
Feb 19 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 02/19/2016 03:46 PM, Iakh wrote:

 On Thursday, 18 February 2016 at 22:46:04 UTC, Walter Bright wrote:

 On 2/18/2016 10:22 AM, Timon Gehr wrote:

 He wanted to embed a mutable reference count literally within a const
 object.
 Not a headconst object.


 I know. I pointed out how it could be done in a way to achieve the
 same effect.

 BTW, shared_ptr<> uses a pointer to the ref count.


 Could D RefCounted!T be splitted into data (possibly immutable) and
 mutable metadata?


This is becoming an FAQ. I think someone should write a blog post about 
it. The basic problem here is composability is not possible with this 
approach. -- Andrei
Feb 19 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
On Fri, 19 Feb 2016 16:09:21 -0500, Andrei Alexandrescu wrote:


 On 02/19/2016 03:46 PM, Iakh wrote:

 On Thursday, 18 February 2016 at 22:46:04 UTC, Walter Bright wrote:

 On 2/18/2016 10:22 AM, Timon Gehr wrote:

 He wanted to embed a mutable reference count literally within a const
 object.
 Not a headconst object.


 I know. I pointed out how it could be done in a way to achieve the
 same effect.

 BTW, shared_ptr<> uses a pointer to the ref count.


 Could D RefCounted!T be splitted into data (possibly immutable) and
 mutable metadata?


 
 This is becoming an FAQ. I think someone should write a blog post about
 it. The basic problem here is composability is not possible with this
 approach. -- Andrei


In other words, it works fine if you have a ref-counted string. It works 
fine if you have a ref-counted list of ref-counted strings. But that list 
*must* be mutable.

You could work around this by creating different types for a logically 
immutable / const refcounted list of ref-counted items than for a mutable 
one. But that's ugly, awkward to work with, and annoying to create.
Feb 19 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 02/19/2016 05:04 PM, Chris Wright wrote:

 In other words, it works fine if you have a ref-counted string. It works
 fine if you have a ref-counted list of ref-counted strings. But that list
 *must*  be mutable.


Much simpler - direct membership. The first thing people want to do when 
you give them a type T is to put it as a member in a struct or class. 
Then they want to make that struct/class immutable. And they can't.

Andrei
Feb 19 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 02/17/2016 06:47 PM, Walter Bright wrote:

 If the headconst was one level up, the struct can have mutating members.


That's not headconst. It's hairconst. -- Andrei
Feb 18 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 02/17/2016 05:44 PM, Walter Bright wrote:

 It would seem that implementing headconst as a type constructor would
 let people who wanted mutable members have their way, without
 introducing backdoors in const.


Doesn't seem that way to me, viz:

struct A
{
     int i;
}

A __const(A) will have a __const(int) member.


Andrei
Feb 18 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/18/2016 4:18 AM, Andrei Alexandrescu wrote:

 On 02/17/2016 05:44 PM, Walter Bright wrote:

 It would seem that implementing headconst as a type constructor would
 let people who wanted mutable members have their way, without
 introducing backdoors in const.


 Doesn't seem that way to me, viz:

 struct A
 {
      int i;
 }

 A __const(A) will have a __const(int) member.


struct A { int* pi; }

and *pi will be mutable even though pi is __const.
Feb 18 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Thursday, 18 February 2016 at 22:48:01 UTC, Walter Bright 
wrote:

 struct A { int* pi; }

 and *pi will be mutable even though pi is __const.


Here's the big deal: when you have started on an implementation 
with constness in, and then evolve the codebase and have to turn 
some fields mutable or face a major rewrite, guess what you will 
do?

You will look for ways to save time and cast away constness, 
write some comment that it probably should be fixed later, but if 
it looks like it is working it will never be fixed...

So you effectively cannot avoid "mutable". You can claim they 
shouldn't do it, but you cannot enforce it.
Feb 18 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/16/2016 5:35 AM, Dicebot wrote:

 In my opinion  mutable would be a disaster of much higher destructive
 impact than head const. I am very opposed to it no matter how it is
 designed. Once you start considering it, you are better at simply
 throwing away existing const system and starting it all from scratch
 with D3. Logical const is harmful as it doesn't give and serious
 guarantees but gives developer a false sense of confidence.



I agree with you on that, and I've argued from that position before.

Note that head const does not introduce any watering down nor destruction of
the 
const/immutable/sharing type system. The main downside of head const would be 
language complexity.
Feb 16 2016
next sibling parent Dicebot <public dicebot.lv> writes:
On 02/17/2016 04:44 AM, Walter Bright wrote:

 Note that head const does not introduce any watering down nor
 destruction of the const/immutable/sharing type system. The main
 downside of head const would be language complexity.


Yes, I agree - it isn't like head const is bad on its own, it simply
feels unnecessary addition outside of C++ bindings for non-trivial
language complication.
Feb 16 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 02/16/2016 09:44 PM, Walter Bright wrote:

 On 2/16/2016 5:35 AM, Dicebot wrote:

 In my opinion  mutable would be a disaster of much higher destructive
 impact than head const. I am very opposed to it no matter how it is
 designed. Once you start considering it, you are better at simply
 throwing away existing const system and starting it all from scratch
 with D3. Logical const is harmful as it doesn't give and serious
 guarantees but gives developer a false sense of confidence.



 I agree with you on that, and I've argued from that position before.

 Note that head const does not introduce any watering down nor
 destruction of the const/immutable/sharing type system. The main
 downside of head const would be language complexity.


I profoundly oppose such an outlook. It has a name - prejudice, pure and 
simple. Rejecting possible future ideas "no matter what" even before 
they exist is extremely damaging.

Consider:

"I oppose implicit narrowing conversions regardless how they are designed"

"static if is fundamentally flawed"

"Variadics cannot be both simple and safe"

etc.


Andrei
Feb 17 2016
parent reply Dicebot <public dicebot.lv> writes:
On 02/17/2016 01:53 PM, Andrei Alexandrescu wrote:

 On 02/16/2016 09:44 PM, Walter Bright wrote:

 On 2/16/2016 5:35 AM, Dicebot wrote:

 In my opinion  mutable would be a disaster of much higher destructive
 impact than head const. I am very opposed to it no matter how it is
 designed. Once you start considering it, you are better at simply
 throwing away existing const system and starting it all from scratch
 with D3. Logical const is harmful as it doesn't give and serious
 guarantees but gives developer a false sense of confidence.



 I agree with you on that, and I've argued from that position before.

 Note that head const does not introduce any watering down nor
 destruction of the const/immutable/sharing type system. The main
 downside of head const would be language complexity.


 
 I profoundly oppose such an outlook. It has a name - prejudice, pure and
 simple. Rejecting possible future ideas "no matter what" even before
 they exist is extremely damaging.
 
 Consider:
 
 "I oppose implicit narrowing conversions regardless how they are designed"
 
 "static if is fundamentally flawed"
 
 "Variadics cannot be both simple and safe"
 
 etc.


It isn't that the concept of logical const /  mutable is inherently
flawed and I am rejecting any possibility of designing it decently. It
is the fact that it is quite alien to existing system and adding it will
inevitably be a hack of some sort.

Redesigning the whole thing with logical const in mind could be feasible
but is a bit too late. Retro-fitting into existing system is a different
thing though.

Consider "static if is fundamentally flawed" vs "static if is
fundamentally flawed [when applied to existing idiomatic C++ code]".
Former is opinionated prejudice. Latter can quite easily be true (even
if I personally wouldn't agree with it).
Feb 18 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Thursday, 18 February 2016 at 11:20:53 UTC, Dicebot wrote:

 Consider "static if is fundamentally flawed" vs "static if is 
 fundamentally flawed [when applied to existing idiomatic C++ 
 code]". Former is opinionated prejudice. Latter can quite 
 easily be true (even if I personally wouldn't agree with it).


It isn't fundamentally flawed, but it is much weaker than a 
constraint-solver.
Feb 18 2016
prev sibling next sibling parent bitwise <bitwise.pvt gmail.com> writes:
On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote:

 rears its head again :-)

 Head Const is what C++ has for const, i.e. it is not 
 transitive, applies to one level only. D has transitive const.

 What head const will do for us:

 1. make it easy to interface to C++ code that uses const, as 
 currently it is not very practical to do so, you have to resort 
 to pragma(mangle)

 2. supports single assignment style of programming, even if the 
 data is otherwise mutable

 The downside is, of course, language complexity.


plz plz plz!

+1

     Bit
Feb 16 2016
prev sibling next sibling parent reply Jakob Ovrum <jakobovrum gmail.com> writes:
On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote:

 rears its head again :-)

 Head Const is what C++ has for const, i.e. it is not 
 transitive, applies to one level only. D has transitive const.

 What head const will do for us:

 1. make it easy to interface to C++ code that uses const, as 
 currently it is not very practical to do so, you have to resort 
 to pragma(mangle)

 2. supports single assignment style of programming, even if the 
 data is otherwise mutable

 The downside is, of course, language complexity.


How about disallowing immutable data with extern(C++) types?  
With extern(C++) data always mutable, `const` could safely be 
reused to mean C++ const in bindings. Any `mutable`-style code 
would be implemented in C++.
Feb 17 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2016 4:03 AM, Jakob Ovrum wrote:

 How about disallowing immutable data with extern(C++) types? With extern(C++)
 data always mutable, `const` could safely be reused to mean C++ const in
 bindings. Any `mutable`-style code would be implemented in C++.


That doesn't help with trying to match the mangling for:

    mutable pointer to const pointer to mutable pointer to const pointer to 
mutable pointer to void

It's still an interesting idea, but it would precluded passing any D immutable 
data structures to C++ code. I tend to think such needs to be possible, with
the 
proviso that you'd have to verify that the C++ end did not violate the 
immutability rules. It's normal practice to write C++ code as if const were 
transitive.
Feb 17 2016
parent Jakob Ovrum <jakobovrum gmail.com> writes:
On Wednesday, 17 February 2016 at 22:01:45 UTC, Walter Bright 
wrote:

 On 2/17/2016 4:03 AM, Jakob Ovrum wrote:

 How about disallowing immutable data with extern(C++) types? 
 With extern(C++)
 data always mutable, `const` could safely be reused to mean 
 C++ const in
 bindings. Any `mutable`-style code would be implemented in C++.


 That doesn't help with trying to match the mangling for:

    mutable pointer to const pointer to mutable pointer to const 
 pointer to mutable pointer to void


Oh yeah, I totally forgot about that.


 It's still an interesting idea, but it would precluded passing 
 any D immutable data structures to C++ code. I tend to think 
 such needs to be possible, with the proviso that you'd have to 
 verify that the C++ end did not violate the immutability rules. 
 It's normal practice to write C++ code as if const were 
 transitive.


It's a sizeable sacrifice. I thought maybe it could be worth it 
for a simple and consistent implementation, but without 
syntactical support for `char * const` mangling it's 
substantially less appealing.
Feb 18 2016
prev sibling parent reply Shachar Shemesh <shachar weka.io> writes:
On 17/02/16 14:03, Jakob Ovrum wrote:

 On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote:

 rears its head again :-)

 Head Const is what C++ has for const, i.e. it is not transitive,
 applies to one level only. D has transitive const.

 What head const will do for us:

 1. make it easy to interface to C++ code that uses const, as currently
 it is not very practical to do so, you have to resort to pragma(mangle)

 2. supports single assignment style of programming, even if the data
 is otherwise mutable

 The downside is, of course, language complexity.


 How about disallowing immutable data with extern(C++) types? With
 extern(C++) data always mutable, `const` could safely be reused to mean
 C++ const in bindings. Any `mutable`-style code would be implemented in
 C++.


The way I see it, here's how it is with C++:
Get const pointer. Cast away its constness. If the original data was 
mutable, you're gold. If not, undefined behaviour.

This is how it works with D:
You're not allowed to cast away constness (UB). The syntax still allows 
you to do it, and because of the very rigid way in which const works in 
D, you often have no choice but to do so in practice. The compiler tries 
not to break your code, so if the original data was mutable, you should, 
probably, be okay.

I fail to see how the C++ way is any worse than ours.

Shachar
Feb 19 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Friday, 19 February 2016 at 08:51:26 UTC, Shachar Shemesh 
wrote:

 The way I see it, here's how it is with C++:
 Get const pointer. Cast away its constness. If the original 
 data was mutable, you're gold. If not, undefined behaviour.


Yes, and since interpret_cast<> seems to not accept casting away 
const, you have to use const_cast<>, so const_cast<> and mutable 
are at least explicit.


 I fail to see how the C++ way is any worse than ours.


The focus in C++ seems to be to add semantic analysis of 
correctness as a separate stage from compilation. No reason for 
why you cannot have both strict constness and lifetime analysis 
in an individual C++ project. C++ seems to be increasingly a 
platform for "your chosen semantics" rather than a single 
language.


But const is still problematic in C++ as well. Mostly thanks to 
generic programming where types can have a completely different 
implementation if the parameter is const or mutable.

So I am not sure if const should be part of the regular type, but 
more along the lines of behavioural typing / type state / deduced 
quality / constraint.

Current languages sacrifice way too much for separate compilation 
and crude header files/attribute files. They could save both 
code-gen IR + semantic analysis IR instead, and feed that to 
various tools.
Feb 19 2016
prev sibling next sibling parent Kagamin <spam here.lot> writes:
On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote:

 1. make it easy to interface to C++ code that uses const, as 
 currently it is not very practical to do so, you have to resort 
 to pragma(mangle)


If it's for mangling, there can be lightweight pragma for 
individual parameters:

C++:
func1(int* const* a);
D:
func1(pragma(cppmangle,"int* const*") const int** a);
Feb 19 2016
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote:

 rears its head again :-)

 Head Const is what C++ has for const, i.e. it is not 
 transitive, applies to one level only. D has transitive const.

 What head const will do for us:

 1. make it easy to interface to C++ code that uses const, as 
 currently it is not very practical to do so, you have to resort 
 to pragma(mangle)

 2. supports single assignment style of programming, even if the 
 data is otherwise mutable

 The downside is, of course, language complexity.


I think that increasing language complexity for the sake of C++ 
integration is a dubious trade-off, especially since "all" that's 
required is correct name mangling. There's no guarantee of what 
the C++ side can do with any type of constness anyway, I'd say 
that any "extern(C++)" mangles as C++ would and leave it at that.

Atila
Feb 24 2016
parent reply extrawurst <stephan extrawurst.org> writes:
On Wednesday, 24 February 2016 at 09:57:51 UTC, Atila Neves wrote:

 On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright 
 wrote:

 rears its head again :-)

 Head Const is what C++ has for const, i.e. it is not 
 transitive, applies to one level only. D has transitive const.

 What head const will do for us:

 1. make it easy to interface to C++ code that uses const, as 
 currently it is not very practical to do so, you have to 
 resort to pragma(mangle)

 2. supports single assignment style of programming, even if 
 the data is otherwise mutable

 The downside is, of course, language complexity.


 I think that increasing language complexity for the sake of C++ 
 integration is a dubious trade-off, especially since "all" 
 that's required is correct name mangling. There's no guarantee 
 of what the C++ side can do with any type of constness anyway, 
 I'd say that any "extern(C++)" mangles as C++ would and leave 
 it at that.

 Atila


I agree with that concern. I would rather see the effort being 
used to improve D - e.g support for GC-free code than to help us 
integrate the complexity of C++ that D was intended to get rid of.

--Stephan
Feb 24 2016
parent rsw0x <anonymous anonymous.com> writes:
On Wednesday, 24 February 2016 at 19:28:28 UTC, extrawurst wrote:

 On Wednesday, 24 February 2016 at 09:57:51 UTC, Atila Neves 
 wrote:

 On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright 
 wrote:

 [...]


 I think that increasing language complexity for the sake of 
 C++ integration is a dubious trade-off, especially since "all" 
 that's required is correct name mangling. There's no guarantee 
 of what the C++ side can do with any type of constness anyway, 
 I'd say that any "extern(C++)" mangles as C++ would and leave 
 it at that.

 Atila


 I agree with that concern. I would rather see the effort being 
 used to improve D - e.g support for GC-free code than to help 
 us integrate the complexity of C++ that D was intended to get 
 rid of.

 --Stephan


A usable `shared` would even be nice.
Feb 24 2016