www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Static inline field initialization

reply Jonas Mminnberg <sasq64 gmail.com> writes:
Because of D's static initialization of members, this assert 
fails:

class Test {
     ubyte[] buf = new ubyte[1000];
}

void main() {
     auto a = new Test();
     auto b = new Test();
     assert(a.buf.ptr != b.buf.ptr);
}

This is bad, since;
* It is not how C++ works
* It introduces silent sharing of data
* It's usually not what you want

Shouldn't this at least generate a warning, or ideally not be 
allowed?
Aug 22 2017
next sibling parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Tuesday, 22 August 2017 at 11:50:50 UTC, Jonas Mminnberg wrote:

 Because of D's static initialization of members, this assert 
 fails:

 class Test {
     ubyte[] buf = new ubyte[1000];
 }

 void main() {
     auto a = new Test();
     auto b = new Test();
     assert(a.buf.ptr != b.buf.ptr);
 }

 This is bad, since;
 * It is not how C++ works
 * It introduces silent sharing of data
 * It's usually not what you want

 Shouldn't this at least generate a warning, or ideally not be 
 allowed?


I agree that it can be confusing if you try to read it with C++ 
semantics [1]; the solution, however, imho is not to change D 
semantics or throw warnings [2], but to refer to the D spec [3], 
which states that static initialization of class members is used 
instead of default initialization (before any constructors are 
run). To me that means a statically initialized class field shall 
have the same value in all class instances, i.e. it's not silent 
sharing, you explicitly requested the sharing. If that doesn't 
mean the same to others, the D spec wording should be updated to 
be clearer on the subject.
If you don't want instances to share a field value, instead of 
static initialization you can use constructor initialization [4]:

----
class Test
{
     ubyte[] buf;
     this()
     {
         buf = new ubyte[1000];
     }
  }

void main()
{
     auto a = new Test();
     auto b = new Test();
     assert(a.buf.ptr != b.buf.ptr);
}
----

[1] D is not C++ and you shouldn't expect similar looking things 
to behave the same
[2] Compiler warnings are (in my experience) ignored by people, 
anyway
[3] https://dlang.org/spec/class.html#constructors
[4] https://dlang.org/spec/class.html#field-init
Aug 22 2017
next sibling parent reply Jonas Mminnberg <sasq64 gmail.com> writes:
On Tuesday, 22 August 2017 at 12:20:45 UTC, Moritz Maxeiner wrote:


 I agree that it can be confusing if you try to read it with C++ 
 semantics [1]; the solution, however, imho is not to change D 
 semantics or throw warnings [2], but to refer to the D spec 
 [3], which states that static initialization of class members 
 is used instead of default initialization (before any 
 constructors are run). To me that means a statically 
 initialized class field shall have the same value in all class 
 instances, i.e. it's not silent sharing, you explicitly 
 requested the sharing. If that doesn't mean the same to others, 
 the D spec wording should be updated to be clearer on the 
 subject.
 If you don't want instances to share a field value, instead of 
 static initialization you can use constructor initialization 
 [4]:

 ----
 class Test
 {
     ubyte[] buf;
     this()
     {
         buf = new ubyte[1000];
     }
  }

 void main()
 {
     auto a = new Test();
     auto b = new Test();
     assert(a.buf.ptr != b.buf.ptr);
 }


I know that it is according to the standard but since D has gone 
out of it's way to make sure sharing doesn't occur otherwise, by 
defaulting to TLS storage etc, I feel this breaks the "no 
surprises" rule. I took me a long time to find this out and when 
I mentioned it to other casual D programmers they also had no 
idea this was how it worked.
Aug 22 2017
parent Moritz Maxeiner <moritz ucworks.org> writes:
On Tuesday, 22 August 2017 at 12:38:50 UTC, Jonas Mminnberg wrote:

 On Tuesday, 22 August 2017 at 12:20:45 UTC, Moritz Maxeiner 
 wrote:


 I agree that it can be confusing if you try to read it with 
 C++ semantics [1]; the solution, however, imho is not to 
 change D semantics or throw warnings [2], but to refer to the 
 D spec [3], which states that static initialization of class 
 members is used instead of default initialization (before any 
 constructors are run). To me that means a statically 
 initialized class field shall have the same value in all class 
 instances, i.e. it's not silent sharing, you explicitly 
 requested the sharing. If that doesn't mean the same to 
 others, the D spec wording should be updated to be clearer on 
 the subject.
 If you don't want instances to share a field value, instead of 
 static initialization you can use constructor initialization 
 [4]:

 ----
 class Test
 {
     ubyte[] buf;
     this()
     {
         buf = new ubyte[1000];
     }
  }

 void main()
 {
     auto a = new Test();
     auto b = new Test();
     assert(a.buf.ptr != b.buf.ptr);
 }


 I know that it is according to the standard but since D has 
 gone out of it's way to make sure sharing doesn't occur 
 otherwise, by defaulting to TLS storage etc,


While I can understand the sentiment, there is a difference 
between sharing data between threads and sharing data between 
class instances in the same thread, the latter of which is 
occurring here.
There is a bug with static field initializers (as others have 
pointed out), but it's about the fact that the array is stored in 
global storage and not in TLS and doesn't apply in the example 
above.


 I feel this breaks the "no surprises" rule.
 I took me a long time to find this out and when I mentioned it 
 to other casual D programmers they also had no idea this was 
 how it worked.


While I don't find how it works surprising personally (it's 
consistent with how static initialization works everywhere else 
in D) - in contrast to other subtleties in D - it might make be 
sensible to include this in the Dlang tour.
Aug 22 2017
prev sibling next sibling parent Daniel Kozak via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, Aug 22, 2017 at 2:20 PM, Moritz Maxeiner via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Tuesday, 22 August 2017 at 11:50:50 UTC, Jonas Mminnberg wrote:


 ...



 I agree that it can be confusing if you try to read it with C++ semantics
 [1]; the solution, however, imho is not to change D semantics or throw
 warnings [2], but to refer to the D spec [3], which states that static
 initialization of class members is used instead of default initialization
 (before any constructors are run). To me that means a statically
 initialized class field shall have the same value in all class instances,
 i.e. it's not silent sharing, you explicitly requested the sharing.


What? D spec does say nothing about sharing, its only speak about order,
nothing else. So it is a buf from my POV.
Aug 22 2017
prev sibling parent reply Daniel Kozak via Digitalmars-d <digitalmars-d puremagic.com> writes:
s/buf/bug/

On Tue, Aug 22, 2017 at 3:52 PM, Daniel Kozak <kozzi11 gmail.com> wrote:


 On Tue, Aug 22, 2017 at 2:20 PM, Moritz Maxeiner via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:


 On Tuesday, 22 August 2017 at 11:50:50 UTC, Jonas Mminnberg wrote:


 ...



 I agree that it can be confusing if you try to read it with C++ semantics
 [1]; the solution, however, imho is not to change D semantics or throw
 warnings [2], but to refer to the D spec [3], which states that static
 initialization of class members is used instead of default initialization
 (before any constructors are run). To me that means a statically
 initialized class field shall have the same value in all class instances,
 i.e. it's not silent sharing, you explicitly requested the sharing.


 What? D spec does say nothing about sharing, its only speak about order,
 nothing else. So it is a buf from my POV.
Aug 22 2017
parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Tuesday, 22 August 2017 at 13:53:05 UTC, Daniel Kozak wrote:

 s/buf/bug/

 On Tue, Aug 22, 2017 at 3:52 PM, Daniel Kozak 
 <kozzi11 gmail.com> wrote:


 On Tue, Aug 22, 2017 at 2:20 PM, Moritz Maxeiner via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:


 On Tuesday, 22 August 2017 at 11:50:50 UTC, Jonas Mminnberg 
 wrote:


 ...



 I agree that it can be confusing if you try to read it with 
 C++ semantics [1]; the solution, however, imho is not to 
 change D semantics or throw warnings [2], but to refer to the 
 D spec [3], which states that static initialization of class 
 members is used instead of default initialization (before any 
 constructors are run). To me that means a statically 
 initialized class field shall have the same value in all 
 class instances, i.e. it's not silent sharing, you explicitly 
 requested the sharing.


 What? D spec does say nothing about sharing, its only speak 
 about order, nothing else. So it is a buf from my POV.




It doesn't have to and it's not a bug (though considering the 
replies in this thread the wording should be changed to include 
that): D's builtin arrays consist of a pointer and a length; the 
example thus *explicitly* initializes both the pointer and the 
length to the same (static) value for all instances (as per 
spec), sharing them. There is a bug [1] - as others have pointed 
out - that the static array isn't stored in TLS, but in global 
storage, however, but that doesn't apply in this single threaded 
case.

[1] https://issues.dlang.org/show_bug.cgi?id=2947
Aug 22 2017
parent reply Kagamin <spam here.lot> writes:
On Tuesday, 22 August 2017 at 14:53:21 UTC, Moritz Maxeiner wrote:

 There is a bug [1] - as others have pointed out - that the 
 static array isn't stored in TLS, but in global storage, 
 however, but that doesn't apply in this single threaded case.


The initializer is copied from typeinfo, that can't refer to TLS 
data.
Aug 22 2017
parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Tuesday, 22 August 2017 at 15:52:48 UTC, Kagamin wrote:

 On Tuesday, 22 August 2017 at 14:53:21 UTC, Moritz Maxeiner 
 wrote:

 There is a bug [1] - as others have pointed out - that the 
 static array isn't stored in TLS, but in global storage, 
 however, but that doesn't apply in this single threaded case.


 The initializer is copied from typeinfo, that can't refer to 
 TLS data.


Which means it isn't easily fixable (or even feasible). I'd still 
consider it a loophole in the type system right now as it allows 
declaring global data mutably shared between threads without 
`shared` or `__gshared` (the latter of which couldn't be applied 
here, though).
I'd argue that - as new in this case doesn't allocate on the 
heap, but in the resulting application's appropriate segments) - 
it should work like this (from a language semantics standpoint):

---

class Test
{
     shared(ubyte)[] buf = new shared(ubyte)[1000]; // classic 
global storage, instances in all threads refer to the same static 
array
}

class Test
{
     ubyte[] buf = new ubyte[1000]; // thread local storage, 
instances in the same thread refer to the same static array
}
---
Aug 22 2017
next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Aug 22, 2017 at 04:28:43PM +0000, Moritz Maxeiner via Digitalmars-d
wrote:
[...]

 I'd argue that - as new in this case doesn't allocate on the heap, but
 in the resulting application's appropriate segments) - it should work
 like this (from a language semantics standpoint):
 
 ---
 
 class Test
 {
     shared(ubyte)[] buf = new shared(ubyte)[1000]; // classic global
 storage, instances in all threads refer to the same static array
 }
 
 class Test
 {
     ubyte[] buf = new ubyte[1000]; // thread local storage, instances in the
 same thread refer to the same static array
 }
 ---


Sounds like a good idea.  Please file this in bugzilla (if it isn't
already) so that it doesn't get lost in the ether.


T

-- 
Two wrongs don't make a right; but three rights do make a left...
Aug 22 2017
prev sibling parent reply Kagamin <spam here.lot> writes:
On Tuesday, 22 August 2017 at 16:28:43 UTC, Moritz Maxeiner wrote:

 class Test
 {
     ubyte[] buf = new ubyte[1000]; // thread local storage, 
 instances in the same thread refer to the same static array
 }


Dynamic initialization is done by constructor:

class Test
{
     static ubyte[1000] s;
     ubyte[] buf;
     this()
     {
         buf=s;
     }
}

It's also unambiguous as to how it works.
Aug 23 2017
parent Moritz Maxeiner <moritz ucworks.org> writes:
On Wednesday, 23 August 2017 at 09:12:19 UTC, Kagamin wrote:

 On Tuesday, 22 August 2017 at 16:28:43 UTC, Moritz Maxeiner 
 wrote:

 class Test
 {
     ubyte[] buf = new ubyte[1000]; // thread local storage, 
 instances in the same thread refer to the same static array
 }


 Dynamic initialization is done by constructor:

 [...]

 It's also unambiguous as to how it works.


I am aware, as I have pointed out the same in the above, but the 
post you quote is explicitly not about dynamic initialization 
(i.e. the result of new points into the heap), but about static 
initialization (the result of new points into sections of the 
binary format).
Specifically, it's about static initialization done for variables 
put into classic global storage (e.g. ELF sections .data/.bss) 
and variables put into thread local storage (e.g. ELF sections 
.tdata/.tbss).
Aug 23 2017
prev sibling next sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 8/22/17 7:50 AM, Jonas Mminnberg wrote:

 Because of D's static initialization of members, this assert fails:
 
 class Test {
      ubyte[] buf = new ubyte[1000];
 }
 
 void main() {
      auto a = new Test();
      auto b = new Test();
      assert(a.buf.ptr != b.buf.ptr);
 }
 
 This is bad, since;
 * It is not how C++ works
 * It introduces silent sharing of data
 * It's usually not what you want
 
 Shouldn't this at least generate a warning, or ideally not be allowed?
 


https://issues.dlang.org/show_bug.cgi?id=2947

-Steve
Aug 22 2017
prev sibling parent Kagamin <spam here.lot> writes:
https://issues.dlang.org/show_bug.cgi?id=2947
Aug 22 2017