www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Function argument that is a pointer to memory which the function is

reply Cecil Ward <d cecilward.com> writes:
say in C I have a function with a pointer argument
     foo( const sometype_t * p )

I have asked about this D nightmare before. Using the same 
pattern in D or the in argument qualifier as far as I can see the 
value of the pointer is then itself effectively locked made 
constant. Without dangerous and ugly casts you are stuck.

q1. If you want a pointer to memory that is not to be modified 
then you can't walk the pointer through that memory.  So what are 
my options? I need a pointer that I can increment. (I could avoid 
the whole issue by using an index instead, but that seems to be 
giving in to madness.)

It seems to me that this is the worst thing I have seen about D. 
Perhaps trying to make pointers unusable is a surreptious 
strategt]y for encouraging designers to phase them out. Making 
code unsafe just to get out of this nightmare (by casting or 
giving up and dropping important const protection) is not the way.

q2. If you want a pointer to modifiable memory but wish to ensure 
that the value of that address stays fixed, stays where it's put, 
then what on earth do you do. What are my options?

Is there any way at all to campaign for a change to this 
craziness? I doubt this is a democracy. It's also rather more 
than a bit late.

q3. The in keyword seems to be mixed up concerning the 
distinction between modifiable arguments and modifiable memory. 
Is there any way of making in usable for the purposes of 
documenting the calling convention, showin which arguments are 
inputs only, which are outputs and which are modified - 
read-modified-returned?

Apologies for my lack for my lack of familiarity with the 
possible ways out of this.

q4. If my understanding is correct, it seems difficult to create 
a non const copy of (an address that is fixed) either; that is, 
making a modifiable copy of an address, one which can be 
incremented, moved upwards starting from a locked base address. 
It seems that declaring a pointer argument with const or even 
using the keyword in triggers this problem, the latter being 
particularly nightmarish because I would want in to mean that 
that argument (the address, which is what I am declaring) is 
merely an input-only parameter to the routine, or alternatively a 
locked /fixed address value which stay s put, and nothing.

I'm interested in the cleanest safest techniques for digging 
myself out of this while always preserving const correctness, 
preventing possibility of writing to memory and preventing evil 
type changes where pointers end up pointing to some different 
kind of objects because if evil casting. I really don't want to 
use casts that have to much power, where they could allow 
overrides to any kind of bugs in or even create a new bug, 
including cases when things break because of duplication of types 
so later changes of types cause a bug because kludge contain 
duplicate type specifiers that do not get updated.

There probably is a tool somewhere to safely create a modifiable 
object based on a const object but I'm not sure where to look.

Any wise guidance appreciated mucky.
Mar 14 2018
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, March 14, 2018 22:23:47 Cecil Ward via Digitalmars-d-learn 
wrote:

 say in C I have a function with a pointer argument
      foo( const sometype_t * p )

 I have asked about this D nightmare before. Using the same
 pattern in D or the in argument qualifier as far as I can see the
 value of the pointer is then itself effectively locked made
 constant. Without dangerous and ugly casts you are stuck.

 q1. If you want a pointer to memory that is not to be modified
 then you can't walk the pointer through that memory.  So what are
 my options? I need a pointer that I can increment. (I could avoid
 the whole issue by using an index instead, but that seems to be
 giving in to madness.)


Once something is const, it can't be modified without violating the type
system, so if you need to mutate something, it can't be const. However, with
a pointer, you can have a mutable pointer to a const object (i.e.
tail-const), e.g.

const(sometype_t)* p;

and in that case, the pointer can be mutated while still not being able to
mutate what it points to. However, once the pointer itself is const, it
can't be mutated, and you'd be forced to copy the pointer to get a
tail-const pointer.


 It seems to me that this is the worst thing I have seen about D.
 Perhaps trying to make pointers unusable is a surreptious
 strategt]y for encouraging designers to phase them out. Making
 code unsafe just to get out of this nightmare (by casting or
 giving up and dropping important const protection) is not the way.


Code with pointers is  system where the compiler cannot guarantee that the
operation is memory safe. Given the nature of pointers, that means that
stuff like pointer arithmetic can't be treated as  safe. It works just fine,
but it's then up to the programmer to verify the  safety of what's going on.
D is only trying to discourage the use of pointers in the sense that any
time that it's trying to proved memory safety, it tends to have to restrict
code thoroughly enough that it can't do much with pointers. DIP 1000 will
improve that, since it improves scope sufficiently that a lot more code
using pointers will be able to be  safe, but fundamentally, pointers are
sufficiently unrestricted that they quickly can't be proven to be memory
safe. So, using them isn't forbidden, but their use does end up being
restricted in any code that's trying to guarantee memory safety. However,
there's no requirement that  safe be used, and you can use pointers pretty
much as freely in D as you can in C/C++.

As far as const goes, the way it works really has nothing to do with
pointers one way or the other beyond the fact that it allows for tail-const
so that pointers can be mutated while leaving the data const. const works
the way that it does in D, because it has to in order to not violate
immutable (since a pointer to const data could actually be pointing to
immutable data), and because Walter believes that const is pointless if it
doesn't provide strong compiler guarantees. So, unlike C++, D's const
provides strong compiler guarantees, but it does make it restrictive enough
that it's often unusable. The reasons behind it really have nothing to do
with trying to discourage pointers though.


 q2. If you want a pointer to modifiable memory but wish to ensure
 that the value of that address stays fixed, stays where it's put,
 then what on earth do you do. What are my options?

 Is there any way at all to campaign for a change to this
 craziness? I doubt this is a democracy. It's also rather more
 than a bit late.


Early on in D2, const actually supported the notion of head-const, but it
was deemed too complicated, and it really doesn't play well with immutable.
As such, while you can use parens with const to have tail-const (thereby
having a mutable pointer to const data), you can't have a const pointer to
mutable data. Once a part of a type is const, everything inside it is const.

However, you can wrap a pointer in a struct which allows you access to the
data that the pointer points to without allowing access to the pointer
itself, thereby effectively making the pointer read-only. It can't actually
be const, and it's extra work to create such a wrapper struct, but it's
quite possible if that's what you really want.


 q3. The in keyword seems to be mixed up concerning the
 distinction between modifiable arguments and modifiable memory.
 Is there any way of making in usable for the purposes of
 documenting the calling convention, showin which arguments are
 inputs only, which are outputs and which are modified -
 read-modified-returned?

 Apologies for my lack for my lack of familiarity with the
 possible ways out of this.


Originally in D2, in was a synonym for const scope (which, since scope has
mostly been unimplemented effectively was the same as const). IIRC, it was
an attempt to make porting D1 code to D2 easier, since D1 used in for
something similar. Folks have often used in in D2 mainly because they
thought that it went well conceptually with out, without really taking into
account what it really means - particularly that part about scope. So, it's
gotten used a fair bit when const could have and really should have been
used instead. If scope were properly implemented, and in then really meant
const scope as intended, a ton of code would break. However, now that scope
is finally being properly fleshed out and implemented as part of DIP 1000,
Walter Bright officially made in just a synonym for const and not const
scope, because he wanted to avoid the breakage that would be caused by
actually treating it as const scope now that scope actually does something
(when compiling with -dip1000 anyway).

So, really, you can ignore in entirely. And if/when you do see it, realize
that it's just const. There's nothing special about it. Using it means all
of the normal pros and cons that go with const.


 q4. If my understanding is correct, it seems difficult to create
 a non const copy of (an address that is fixed) either; that is,
 making a modifiable copy of an address, one which can be
 incremented, moved upwards starting from a locked base address.
 It seems that declaring a pointer argument with const or even
 using the keyword in triggers this problem, the latter being
 particularly nightmarish because I would want in to mean that
 that argument (the address, which is what I am declaring) is
 merely an input-only parameter to the routine, or alternatively a
 locked /fixed address value which stay s put, and nothing.


If you're just talking about a pointer, getting a mutable copy is trivial.

auto foo(const T* ptr)
{
    const(T)* local = ptr;
    ...
}

But in generaly, if you're using const, you have to be willing to not ever
mutate the data involved, and that often means that const simply is not a
good fit for a particular piece of code.


 I'm interested in the cleanest safest techniques for digging
 myself out of this while always preserving const correctness,
 preventing possibility of writing to memory and preventing evil
 type changes where pointers end up pointing to some different
 kind of objects because if evil casting. I really don't want to
 use casts that have to much power, where they could allow
 overrides to any kind of bugs in or even create a new bug,
 including cases when things break because of duplication of types
 so later changes of types cause a bug because kludge contain
 duplicate type specifiers that do not get updated.

 There probably is a tool somewhere to safely create a modifiable
 object based on a const object but I'm not sure where to look.

 Any wise guidance appreciated mucky.


Basically, you should never, ever cast away const. Casting it away is fine
so long as you can then guarantee that the data is never mutated, but
mutating data after casting away const is undefined behavior and can result
in subtle bugs. D does not have _any_ backdoors for getting around const.
When something is const in D, it's const. It can be mutated via a mutable
reference to the same data, but you can't legally get a mutable reference
from a const reference. So, if you have const, you're stuck. This provides
great guarantees (unlike C/C++) but often means that const is unusable in D.

If you're dealing with pointers or dynamic arrays, you can get a tail-const
pointer or slice to the data - e.g.

const(T*) ptr1;
const(T)* ptr2 = ptr1;

const(T[]) arr1;
const(T)[] arr2 = arr1;

and if the type is a value type, you can get a mutable copy, but once you're
dealing with const references or const data that you can't (or don't want
to) copy, you're stuck. Once something is const in D, it really is const.

That's why many D programmers do almost nothing with const. The guarantees
are great, but they're so restrictive as to make const borderline useless -
especially once user-defined types get involved. I would strongly advise you
to forget about "const-correctness" when writing D code. You can get it on
some level if you really work at it, but it's trivial to back yourself into
a corner where you're screwed because you have something that's const where
you need at least some portion of it to be mutable, and you can't have it.

I recently wrote up an article on the subject:

http://jmdavisprog.com/articles/why-const-sucks.html

- Jonathan M Davis
Mar 14 2018
prev sibling next sibling parent ag0aep6g <anonymous example.com> writes:
On 03/14/2018 11:23 PM, Cecil Ward wrote:

 say in C I have a function with a pointer argument
      foo( const sometype_t * p )
 
 I have asked about this D nightmare before. Using the same pattern in D 
 or the in argument qualifier as far as I can see the value of the 
 pointer is then itself effectively locked made constant. Without 
 dangerous and ugly casts you are stuck.
 
 q1. If you want a pointer to memory that is not to be modified then you 
 can't walk the pointer through that memory.  So what are my options? I 
 need a pointer that I can increment. (I could avoid the whole issue by 
 using an index instead, but that seems to be giving in to madness.)


I think you're looking for this: `const(sometype_t)* p`. Here, the 
pointer itself is mutable.

[...]

 q2. If you want a pointer to modifiable memory but wish to ensure that 
 the value of that address stays fixed, stays where it's put, then what 
 on earth do you do. What are my options?


Can't be done directly with type qualifiers. You could maybe write a 
wrapper struct that acts like a pointer while disallowing modification 
of the pointer itself. I don't know how feasible this could be. I've 
never felt a need for it.

[...]

 q3. The in keyword seems to be mixed up concerning the distinction 
 between modifiable arguments and modifiable memory. Is there any way of 
 making in usable for the purposes of documenting the calling convention, 
 showin which arguments are inputs only, which are outputs and which are 
 modified - read-modified-returned?


`in` was meant to mean `scope const`. But I think it's effectively just 
`const` at the moment. Redefining `in` to something more useful probably 
means it has to be deprecated and removed first. Then brought back later 
with a new meaning. If that happens, it's going to take years.

[...]

 q4. If my understanding is correct, it seems difficult to create a non 
 const copy of (an address that is fixed) either; that is, making a 
 modifiable copy of an address, one which can be incremented, moved 
 upwards starting from a locked base address.


You can easily make a mutable pointer from a const one:

     const int* c;
     const(int)* m = c; /* no problem */

The target of the pointer just has to remain `const`.

[...]

 There probably is a tool somewhere to safely create a modifiable object 
 based on a const object but I'm not sure where to look.


Generally, that means a deep copy, right? I don't think that's in the 
standard library.

For arrays, there's `.dup`:

     const int[] c = [1, 2, 3];
     int[] m = c.dup;
Mar 14 2018
prev sibling parent Cecil Ward <d cecilward.com> writes:
On Wednesday, 14 March 2018 at 22:23:47 UTC, Cecil Ward wrote:

 say in C I have a function with a pointer argument
     foo( const sometype_t * p )

 [...]



That's the secret - I didn't know about the const (T) * thing - I 
would never have discovered that ! Many thanks, the missing piece 
to the puzzle.

Many generous replies, thanks to all for their extremely helpful 
contributions. There is a wealth of precious explanation in them, 
and apologies for not thanking the contributors individually.
Mar 15 2018