www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Casting away immutability

reply "Sergei Degtiarev" <sdegtiarev yahoo.com> writes:
I can't understand how cast coexist with immutability. Consider 
the code:
	immutable immutable(int)[4] buf;
	auto x=buf[0];
	auto p=buf.ptr;
	
	auto i=cast(int[]) buf;
	i[]=1;
	
	assert(buf.ptr == p);
	assert(buf[0] != x);

I've just modified immutable data, is it normal?
Another, ever simpler, code:
	immutable a=0;
	auto b=cast(int*) &a;
	*b=1;
	assert(&a == b);
	assert(a != *b);
Last two assertions both succeed and really puzzle me. What is 
pointer in D at all?
Sep 01 2015
next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 2 September 2015 at 02:05:00 UTC, Sergei Degtiarev 
wrote:

 I can't understand how cast coexist with immutability.



Cast bypasses immutability, triggering implementation-defined 
behavior. You might modify immutable data, you might cause the 
program to crash, it might just not do anything, depending on 
exactly what is going on in the implementation.

When you use cast, you basically take matters into your own hands.
Sep 01 2015
prev sibling parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Wednesday, September 02, 2015 02:04:58 Sergei Degtiarev via
Digitalmars-d-learn wrote:

 I can't understand how cast coexist with immutability. Consider
 the code:
   immutable immutable(int)[4] buf;
   auto x=buf[0];
   auto p=buf.ptr;

   auto i=cast(int[]) buf;
   i[]=1;

   assert(buf.ptr == p);
   assert(buf[0] != x);

 I've just modified immutable data, is it normal?
 Another, ever simpler, code:
   immutable a=0;
   auto b=cast(int*) &a;
   *b=1;
   assert(&a == b);
   assert(a != *b);
 Last two assertions both succeed and really puzzle me. What is
 pointer in D at all?


You can cast away immutable, just like you can cast away const, but in
either case, mutating the data after casting it to mutable is undefined
behavior. The compiler is free to assume that you won't do that, and in the
case of immutable, if the data is actually in ROM, it could crash your
program.

The two main places that casting away const or immutable would be used at
this point would be to pass it to a C function which isn't properly
annotated (but which you know won't mutate the data) or when you cast to
immutable to pass something via std.concurrency and then to cast back to
mutable on the receiving thread, since we don't currently have a way to
indicate ownership in a manner that would allow us to pass mutable data
across threads. But if you do that, you need to be sure that you just passed
across the only reference to that data, or you're going to have bugs,
because anything which isn't immutable or shared is considered thread-local
by the compiler.

So, there are cases where it makes sense if you're very careful and know
what you're doing, but those cases are few and far between. But D allows the
cast if nothing else, because it's a systems language and will let you shoot
yourself in the foot if you really try to. And when you cast, you're telling
the compiler that you know better than it does. So, you'd better be right,
or you're going to blow your foot off (figuratively speaking).

Regardless, casting away const or immutable and then mutating is undefined
behavior. So, don't do it.

- Jonathan M Davis
Sep 01 2015
parent reply "Sergei Degtiarev" <sdegtiarev yahoo.com> writes:
On Wednesday, 2 September 2015 at 02:50:30 UTC, Jonathan M Davis 
wrote:

 is undefined behavior. So, don't do it.


I don't. Actually, I'm looking for opposite - to protect data, 
like it is a class with two methods, returning void[] and 
immutable(void)[] as memory buffer, which user of the class 
supposed to cast to desired type. I naively assumed that void[] 
to be cast to, say, int[]; and immutable(void)[] to 
immutable(int)[] respectively. Unfortunately, D cast notices no 
difference between these two types.
I seems a little bit too easy to to shoot yourself in the foot.


And still,

   assert(&a == b);
   assert(a != *b);
 


How these two lines would look in assembler? Defined behavior or 
undefined behavior, I can't imagine that.
Sep 01 2015
parent reply "lobo" <swaplobo gmail.com> writes:
On Wednesday, 2 September 2015 at 04:04:54 UTC, Sergei Degtiarev 
wrote:

 On Wednesday, 2 September 2015 at 02:50:30 UTC, Jonathan M 
 Davis wrote:

 is undefined behavior. So, don't do it.


 I don't. Actually, I'm looking for opposite - to protect data, 
 like it is a class with two methods, returning void[] and 
 immutable(void)[] as memory buffer, which user of the class 
 supposed to cast to desired type. I naively assumed that void[] 
 to be cast to, say, int[]; and immutable(void)[] to 
 immutable(int)[] respectively. Unfortunately, D cast notices no 
 difference between these two types.
 I seems a little bit too easy to to shoot yourself in the foot.


No, I think your design is unsafe because you're throwing away 
type information and returning void[], then telling the compiler 
not to worry about it.


 And still,

   assert(&a == b);
   assert(a != *b);
 


 How these two lines would look in assembler? Defined behavior 
 or undefined behavior, I can't imagine that.


why don't you have a look?

bye,
lobo
Sep 01 2015
parent reply "Sergei Degtiarev" <sdegtiarev yahoo.com> writes:
On Wednesday, 2 September 2015 at 04:19:24 UTC, lobo wrote:

 No, I think your design is unsafe because you're throwing away 
 type information and returning void[], then telling the 
 compiler not to worry about it.


It is unsafe, but this not my design but std.mmfile module in 
phobos. This is what I'm trying to improve. And there was no type 
information initially, this is raw allocated memory returned to 
user.
Sep 02 2015
parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Wednesday, September 02, 2015 14:00:07 Sergei Degtiarev via
Digitalmars-d-learn wrote:

 On Wednesday, 2 September 2015 at 04:19:24 UTC, lobo wrote:

 No, I think your design is unsafe because you're throwing away
 type information and returning void[], then telling the
 compiler not to worry about it.


 It is unsafe, but this not my design but std.mmfile module in
 phobos. This is what I'm trying to improve. And there was no type
 information initially, this is raw allocated memory returned to
 user.


Well, that's how mmap works. You're just getting raw bytes as void*, and
the D code converts that to void[], which is slightly safer. It doesn't
inherently have a type, and often, the data referred to by mmap never did
have a type. It's just bytes in a file. So, if you want to use it as a type,
you have to tell the compiler what it's type is - and get it right - via a
cast. And yes, you can cast to immutable as part of that, because casts let
you shoot yourself in the foot, because you're telling the compiler that you
know better than it does, but you shouldn't be casting anything you get from
C to immutable, because it's not going to be immutable. Most code shouldn't
be casting to immutable any more than it should be casting away immutable.

immutable data is treated by the compiler as immutable - it will _never_
change over the course of the program - so if it does change, you're going
to have fun bugs, and if it really refers to mutable data (as would be the
case with an mmapped file), then it could change. Additionally, immutable
is treated as implicitly shared, so the compiler could do different
optimizations based on that (though that probably won't affect anything if
you only ever have it one a single thread), and if it's not really
immutable, you could have fun bugs caused by that.

Don't cast to or from immutable unless you know what you're doing, and in
most of those cases, there's a decent chance that that's not the best way to
do what you're trying to do anyway.

But at the end of the day, the cast operator is a blunt instrument which
inloves you telling the compiler that you know better than it does, so you
had better know better, or you're going to shoot yourself in the foot.

- Jonathan M Davis
Sep 02 2015
parent reply "Sergei Degtiarev" <sdegtiarev yahoo.com> writes:
On Thursday, 3 September 2015 at 05:15:35 UTC, Jonathan M Davis 
wrote:

 On Wednesday, September 02, 2015 14:00:07 Sergei Degtiarev via 
 Digitalmars-d-learn wrote:
 Well, that's how mmap works. You're just getting raw bytes as 
 void*, and the D code converts that to void[], which is 
 slightly safer. It doesn't inherently have a type, and often, 
 the data referred to by mmap never did have a type. It's just 
 bytes in a file. So, if you want to use it as a type, you have 
 to tell the compiler what it's type is - and get it right - via 
 a cast. And yes, you can cast to immutable as part of that, 
 because casts let you shoot yourself in the foot, because 
 you're telling the compiler that you know better than it does, 
 but you shouldn't be casting anything you get from C to 
 immutable, because it's not going to be immutable. Most code 
 shouldn't be casting to immutable any more than it should be 
 casting away immutable.

 immutable data is treated by the compiler as immutable - it 
 will _never_ change over the course of the program - so if it 
 does change, you're going to have fun bugs, and if it really 
 refers to mutable data (as would be the case with an mmapped 
 file), then it could change. Additionally, immutable is treated 
 as implicitly shared, so the compiler could do different 
 optimizations based on that (though that probably won't affect 
 anything if you only ever have it one a single thread), and if 
 it's not really immutable, you could have fun bugs caused by 
 that.

 Don't cast to or from immutable unless you know what you're 
 doing, and in most of those cases, there's a decent chance that 
 that's not the best way to do what you're trying to do anyway.


Agree, however, memory obtained with mmap(..., PROT_READ, ..); is 
essentially read-only and any attempt to write to it will cause 
immediate crash with segmentation violation. It would be very 
desirable in this case to return something that could be cast to 
immutable(type)[] but not type[].
This is what I tried to find out. Thank you for the help.
Sep 03 2015
next sibling parent reply "Mike Parker" <aldacron gmail.com> writes:
On Thursday, 3 September 2015 at 13:28:54 UTC, Sergei Degtiarev 
wrote:


 Agree, however, memory obtained with mmap(..., PROT_READ, ..); 
 is essentially read-only and any attempt to write to it will 
 cause immediate crash with segmentation violation. It would be 
 very desirable in this case to return something that could be 
 cast to immutable(type)[] but not type[].
 This is what I tried to find out. Thank you for the help.


immutable(T)[] getGetData(T)() {
     return cast(immutable(T)[])data;
}

...

auto ints = obj.getData!int;
Sep 03 2015
parent reply "Sergei Degtiarev" <sdegtiarev yahoo.com> writes:
On Thursday, 3 September 2015 at 14:36:12 UTC, Mike Parker wrote:

 immutable(T)[] getGetData(T)() {
     return cast(immutable(T)[])data;
 }


Absolutely, instead of returning raw void[] and allow user to 
cast it, std.mmfile should implement template function to return 
desired type.
This would allow all necessary checks inside the module, 
returning pure result.
Sep 03 2015
parent reply "Kagamin" <spam here.lot> writes:
On Wednesday, 2 September 2015 at 04:04:54 UTC, Sergei Degtiarev 
wrote:

 I seems a little bit too easy to to shoot yourself in the foot.


Yes, cast is a little not fool-proof in this scenario.

On Thursday, 3 September 2015 at 13:28:54 UTC, Sergei Degtiarev 
wrote:

 Agree, however, memory obtained with mmap(..., PROT_READ, ..); 
 is essentially read-only and any attempt to write to it will 
 cause immediate crash with segmentation violation. It would be 
 very desirable in this case to return something that could be 
 cast to immutable(type)[] but not type[].


If you can't write to it doesn't mean it won't change. Use const 
for memory that you can't write to.

On Thursday, 3 September 2015 at 16:05:33 UTC, Sergei Degtiarev 
wrote:

 Absolutely, instead of returning raw void[] and allow user to 
 cast it, std.mmfile should implement template function to 
 return desired type.


Well, interpretation of data can be arbitrarily complex, I'd 
suggest that the raw data should be passed to your specific 
parser, that will do the right thing and provide data in expected 
format.
Sep 04 2015
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Friday, 4 September 2015 at 08:37:26 UTC, Kagamin wrote:

 On Thursday, 3 September 2015 at 13:28:54 UTC, Sergei Degtiarev 
 wrote:

 Agree, however, memory obtained with mmap(..., PROT_READ, ..); 
 is essentially read-only and any attempt to write to it will 
 cause immediate crash with segmentation violation. It would be 
 very desirable in this case to return something that could be 
 cast to immutable(type)[] but not type[].


 If you can't write to it doesn't mean it won't change. Use 
 const for memory that you can't write to.


Note that you can however achieve immutability by using a 
_private_ read-only mapping.
Sep 04 2015
parent reply "Kagamin" <spam here.lot> writes:
On Friday, 4 September 2015 at 09:05:07 UTC, Marc Schütz wrote:

 Note that you can however achieve immutability by using a 
 _private_ read-only mapping.


man pages say the behavior is unspecified.
Sep 04 2015
parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Friday, 4 September 2015 at 13:29:42 UTC, Kagamin wrote:

 On Friday, 4 September 2015 at 09:05:07 UTC, Marc Schütz wrote:

 Note that you can however achieve immutability by using a 
 _private_ read-only mapping.


 man pages say the behavior is unspecified.


The Linux man page says:

"It is unspecified whether changes made to the file after the 
mmap() call are visible in the mapped region."

I.e., you're right. I interpreted this to refer only to portions 
of the file that have never been accessed, but reading it again, 
it doesn't say that... And it makes sense, as this makes it 
possible to drop clean pages on memory pressure, instead of 
swapping them out.
Sep 04 2015
prev sibling parent "jqb" <jqb somewhere.net> writes:
On Thursday, 3 September 2015 at 13:28:54 UTC, Sergei Degtiarev 
wrote:
[...]

 Agree, however, memory obtained with mmap(..., PROT_READ, ..); 
 is essentially read-only and any attempt to write to it will 
 cause immediate crash with segmentation violation. It would be 
 very desirable in this case to return something that could be 
 cast to immutable(type)[] but not type[].
 This is what I tried to find out. Thank you for the help.


It can still change by writing (in the same or another process) 
to the file via another file descriptor. Of course the results 
are non-deterministic in that case, so it doesn't much matter 
whether the compiler assumes the data to be immutable or not.
Sep 04 2015