www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - const debacle

reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
This issue was just brought to my attention, and I wonder if the other 
const-proponents have a good solution for this.  This is going to be very 
tough to deal with.

How do you declare a function that takes an array, is not allowed to change 
the array, but returns a slice into the argument array, and the return type 
matches the argument type.

For example, if I pass a mutable array to this function, I want it to pass 
me back a mutable slice into the array, but I also want a guarantee that the 
function will not modify the array.  I can't declare the argument is const 
because then the return value must also be const, as it is a slice of the 
original array.

How does one solve this problem?

-Steve
Mar 21 2008
next sibling parent reply BCS <BCS pathlink.com> writes:
Steven Schveighoffer wrote:

 This issue was just brought to my attention, and I wonder if the other 
 const-proponents have a good solution for this.  This is going to be very 
 tough to deal with.
 
 How do you declare a function that takes an array, is not allowed to change 
 the array, but returns a slice into the argument array, and the return type 
 matches the argument type.
 
 For example, if I pass a mutable array to this function, I want it to pass 
 me back a mutable slice into the array, but I also want a guarantee that the 
 function will not modify the array.  I can't declare the argument is const 
 because then the return value must also be const, as it is a slice of the 
 original array.
 
 How does one solve this problem?
 
 -Steve 
 
 


IIRC Walter has something planed for this. Something to do with a 
"return" const type. It would do just that; "the function's return type 
has the same constness as one of it's arguments."

In short: Not yet.
Mar 21 2008
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"BCS" wrote

 Steven Schveighoffer wrote:

 This issue was just brought to my attention, and I wonder if the other 
 const-proponents have a good solution for this.  This is going to be very 
 tough to deal with.

 How do you declare a function that takes an array, is not allowed to 
 change the array, but returns a slice into the argument array, and the 
 return type matches the argument type.

 For example, if I pass a mutable array to this function, I want it to 
 pass me back a mutable slice into the array, but I also want a guarantee 
 that the function will not modify the array.  I can't declare the 
 argument is const because then the return value must also be const, as it 
 is a slice of the original array.

 How does one solve this problem?

 -Steve


 IIRC Walter has something planed for this. Something to do with a "return" 
 const type. It would do just that; "the function's return type has the 
 same constness as one of it's arguments."

 In short: Not yet.


I remember that idea.  I'm concerned however that in this mode, the compiler 
will not check the function itself for const-correctness (i.e. ensure it 
doesn't modify the input) because the argument will not be const.  See my 
reply to Walter for my example.  I want a way to specify "the function's 
return type has the same constness as one of it's arguments, and it promises 
not to modify that argument, and the compiler made sure of that."

-Steve
Mar 21 2008
next sibling parent Sean Kelly <sean invisibleduck.org> writes:
== Quote from Steven Schveighoffer (schveiguy yahoo.com)'s article

 "BCS" wrote

 IIRC Walter has something planed for this. Something to do with a "return"
 const type. It would do just that; "the function's return type has the
 same constness as one of it's arguments."

 In short: Not yet.


 I remember that idea.  I'm concerned however that in this mode, the compiler
 will not check the function itself for const-correctness (i.e. ensure it
 doesn't modify the input) because the argument will not be const.  See my
 reply to Walter for my example.  I want a way to specify "the function's
 return type has the same constness as one of it's arguments, and it promises
 not to modify that argument, and the compiler made sure of that."


Oops, I'd forgotten about this while writing my post.  In short, I don't think
there's any way to do this
automatically.  You'd likely either have to supply the type manually or resort
to some sort of proxy
function to do this for you.


Sean
Mar 21 2008
prev sibling parent BCS <ao pathlink.com> writes:
Reply to Steven,


 I remember that idea.  I'm concerned however that in this mode, the
 compiler will not check the function itself for const-correctness
 (i.e. ensure it doesn't modify the input) because the argument will
 not be const.  See my reply to Walter for my example.  I want a way to
 specify "the function's return type has the same constness as one of
 it's arguments, and it promises not to modify that argument, and the
 compiler made sure of that."
 
 -Steve
 


it should be possible to have the compiler work the way you want. The function 
would be built assuming some level of constness and then, at the calling 
site the arg would be checked to see if it is no more restrictive. If so, 
then the call is allowed and the return type is generated from the arg's. 
(all at compile time of course)
Mar 21 2008
prev sibling next sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Steven Schveighoffer wrote:

 How do you declare a function that takes an array, is not allowed to change 
 the array, but returns a slice into the argument array, and the return type 
 matches the argument type.



The way to do it is with a function template, parameterizing the type in 
question.
Mar 21 2008
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Walter Bright" wrote

 Steven Schveighoffer wrote:

 How do you declare a function that takes an array, is not allowed to 
 change the array, but returns a slice into the argument array, and the 
 return type matches the argument type.



 The way to do it is with a function template, parameterizing the type in 
 question.


So your answer essentially is, there is no way to specify that the function 
will not modify the parameter but returns the same type passed in?

Basically, I want a function like:

T[] foo(T)(in T[] source, in T[] pattern);

Where foo returns a slice of source that matches pattern, and promises not 
to modify source or pattern, but returns the same type as was passed in. 
Then the caller could modify the slice as necessary.  It's kind of like a 
'scoped' const, where the argument is only const within the function, but 
reverts to what it was passed in as once the function is exited.

What you're saying is, specify it like:

T[] foo(T)(T[] source, in T[] pattern);

Which means there is no check by the compiler to guarantee that source is 
not modified (unless T is const), and basically, the coder has to rely on 
the documentation to determine whether source will remain intact?  I think 
this degrades the benefits of const in this context.

-Steve
Mar 21 2008
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Steven Schveighoffer wrote:

 So your answer essentially is, there is no way to specify that the function 
 will not modify the parameter but returns the same type passed in?


Right.


 Which means there is no check by the compiler to guarantee that source is 
 not modified (unless T is const), and basically, the coder has to rely on 
 the documentation to determine whether source will remain intact?  I think 
 this degrades the benefits of const in this context.


There's been some discussion about this, but no good resolution. (C++ 
has the same issue, btw.)
Mar 21 2008
next sibling parent Jason House <jason.james.house gmail.com> writes:
Maybe I'm not thinking straight, but wouldn't use of the return keyword in a
function imply const access inside the function? The code must work if the
input argument is const or not...


Walter Bright Wrote:


 Steven Schveighoffer wrote:

 So your answer essentially is, there is no way to specify that the function 
 will not modify the parameter but returns the same type passed in?


 
 Right.
 

 Which means there is no check by the compiler to guarantee that source is 
 not modified (unless T is const), and basically, the coder has to rely on 
 the documentation to determine whether source will remain intact?  I think 
 this degrades the benefits of const in this context.


 
 There's been some discussion about this, but no good resolution. (C++ 
 has the same issue, btw.)
Mar 21 2008
prev sibling parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 21/03/2008, Walter Bright <newshound1 digitalmars.com> wrote:

 There's been some discussion about this, but no good resolution.


Allow me to offer a suggestion.

There are several stages to this reasoning, but I think you're going
to like the final deduction at the end. Stage one: allow the word
"auto" wherever "const" and "invariant" are currently allowed. The
meaning of "auto" is always "do nothing". This is consistent with the
documented meaning: "The auto attribute is used when there are no
other attributes and type inference is desired." - we're just
extending it beyond type inference. So in particular:

    auto(T)

becomes an allowable do-nothing type constructor, so that auto(T) is
just an alias for T. And also, since we can declare functions like

    void f(const int x)

then we must also allow

    void f(auto int x)

again, with auto being a "do nothing" keyword, so the latter is
completely equivalent to

    void f(int x)

(Note: I am /not/ suggesting that "void f(auto x)" be allowed as a
function signature. In fact, it won't be, since "void f(const x)" is
not allowed.)

Why am I suggesting this? Well, bear with me. All will become clear
shortly. The next step is to introduce a new kind of template
parameter:

    template T(const K)
    {
        ...
    }

The idea is that the placeholder K may be substituted with one of
exactly three symbols: "auto", "const" and "invariant". Thus, we may
now write:

    template T(const K)
    {
        K char[] f(K char[] buf ) { ... }
    }

and instantiate it with each of:

    T!(auto).f(buf)
    T!(const).f(buf)
    T!(invariant).f(buf)

This expands out to the three functions

    auto char[] f(auto char[] buf ) { ... }
    const char[] f(const char[] buf ) { ... }
    invariant char[] f(invariant char[] buf ) { ... }

and since "auto" means "do nothing", the first of these reduces to

    char[] f(char[] buf ) { ... }

Next, we use the simplified template declaration syntax for functions,
to do exactly the same thing:

    K char[] f(const K)(K char[] buf ) { ... }

and instantiate with

    f!(auto)(buf)
    f!(const)(buf)
    f!(invariant)(buf)

And the final step - allow type deduction, so that we now only have to write

    f(buf)

and have the compiler choose the right instantiation based on the
constancy of buf.

This, I think, would be problem solved.
Mar 21 2008
next sibling parent reply e-t172 <e-t172 akegroup.org> writes:
Janice Caron a écrit :

 Why am I suggesting this? Well, bear with me. All will become clear
 shortly. The next step is to introduce a new kind of template
 parameter:
 
     template T(const K)
     {
         ...
     }


The problem of this solution is that you're basically using templates to 
solve a problem which will probably affect a LOT of functions 
(especially text processing functions).

I think we should keep in mind that :
  - most of libraries are distributed in shared form (at least on Unix), 
because shared libraries are far more practical;
  - templates themselves cannot be compiled, so they cannot be included 
in a shared library, they are "compiled" when the code that instanciates 
them is compiled (which basically is the same as using static libraries)

This means that when you are over-using templates, you lose all the 
advantages of shared libraries. I think a lot of people easily forget 
about this; for example, in Tango text.Util:

T[] trim(T) (T[] source)
T[] strip(T) (T[] source, T match)
T[][] split(T) (T[] src, T[] pattern)

etc... nearly all functions in this module are templates, in order to 
support char, dchar and wchar without duplicating code. This basically 
means that when we'll be able to compile Tango in a shared library, this 
module will be compiled to... nothing. This means that when the code of 
this module is modified, ALL projects using it must be recompiled to use 
the new module. Now imagine that D become very popular (which I hope 
will happen), this will become very hard to manage (if you don't 
understand why, imagine what would happen if the libc was linked 
statically in all executables of your system...)

In the case of Tango, we can address this issue by writing this in 
text.Util:

char[] trim(char[] source) { return trim!(char)(source); }
dchar[] trim(dchar[] source) { return trim!(dchar)(source); }
wchar[] trim(wchar[] source) { return trim!(wchar)(source); }

And so on for all functions. However, it is not very elegant. A better 
idea would be to include a new "feature" in D 2.0, i.e. "compiled 
templates", with a new syntax like this :



This will behave exactly like the original syntax, except that when you 
compile text.Util, trim() will be included in the object file in three 
versions: char, dchar and wchar. Now when the caller writes 
trim!(char)(foo), the compiler WILL automatically use the already 
compiled "char" version of trim(). And if you want to do something 
exotic, like trim!(mytype)(foo), it will still be possible, but it will 
behave like a "classic" template, that is, the template code will be 
compiled into the code that instanciates it.

The point here is to allow the compiler to generate object code for 
templates which get only used with a small subset of types 99% of the 
time (like these functions in Tango, which are only used with char, 
dchar and wchar), in order to render shared libraries more useful and 
practical.

Of course, "pure templates", that is, templates which are meant to be 
instanciated and used with any given type as parameter, cannot benefit 
from this "feature".

So, back to the point, in your case :


     template T(const K)
     {
         K char[] f(K char[] buf ) { ... }
     }


Would become :


      {
          K char[] f(K char[] buf ) { ... }
      }

A better solution: each "const K" template construct is always compiled 
in the three versions (or rather in one version, because the object code 
for the three is identical).
Mar 22 2008
next sibling parent "Janice Caron" <caron800 googlemail.com> writes:
On 22/03/2008, e-t172 <e-t172 akegroup.org> wrote:

 The problem of this solution is that you're basically using templates to
  solve a problem which will probably affect a LOT of functions


True, but I don't see that as a problem.



 A better solution: each "const K" template construct is always compiled
  in the three versions (or rather in one version, because the object code
  for the three is identical).


I completely agree. The "const K" template construct could (should?)
be instantiated for all three cases.

Note, however, that it's not actually /guaranteed/ that all three
cases will be identical, because the function might have "static if"
in it. e.g.

    K(char)[] f(const K)(K(char)[] buffer)
    {
        static if (K == invariant)
        {
            return buffer;
        }
        else
        {
            return buffer.dup;
        }
    }

But even in these edge cases, I'd be happy for all three versions to
auto-instantiate. There are only three cases, after all, and the
linker can probably throw away any function that's not used.
Mar 22 2008
prev sibling next sibling parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 22/03/2008, e-t172 <e-t172 akegroup.org> wrote:

  A better
  idea would be to include a new "feature" in D 2.0, i.e. "compiled
  templates", with a new syntax like this :




I believe you can already do that - albeit with a different syntax.

    T[] trim(T) (T[] source)
    {
        /*...*/
    }

    private
    {
        alias trim!(char) dummy1;
        alias trim!(wchar) dummy2;
        alias trim!(dchar) dummy3;
    }

I think those aliases will force an instantiation.
Mar 22 2008
parent reply e-t172 <e-t172 akegroup.org> writes:
Janice Caron a écrit :

 On 22/03/2008, e-t172 <e-t172 akegroup.org> wrote:

  A better
  idea would be to include a new "feature" in D 2.0, i.e. "compiled
  templates", with a new syntax like this :




 
 I believe you can already do that - albeit with a different syntax.
 
     T[] trim(T) (T[] source)
     {
         /*...*/
     }
 
     private
     {
         alias trim!(char) dummy1;
         alias trim!(wchar) dummy2;
         alias trim!(dchar) dummy3;
     }
 
 I think those aliases will force an instantiation.


As a matter of fact, it does, I just tried it. I learned something today :)

Now library developers just have to begin using this kind of aliases for 
templates which get only used with a few, known types, like in 
tango.text.Util.
Mar 22 2008
parent reply Lars Ivar Igesund <larsivar igesund.net> writes:
e-t172 wrote:


 Janice Caron a écrit :

 On 22/03/2008, e-t172 <e-t172 akegroup.org> wrote:

  A better
  idea would be to include a new "feature" in D 2.0, i.e. "compiled
  templates", with a new syntax like this :




 
 I believe you can already do that - albeit with a different syntax.
 
     T[] trim(T) (T[] source)
     {
         /*...*/
     }
 
     private
     {
         alias trim!(char) dummy1;
         alias trim!(wchar) dummy2;
         alias trim!(dchar) dummy3;
     }
 
 I think those aliases will force an instantiation.


 
 As a matter of fact, it does, I just tried it. I learned something today
 :)
 
 Now library developers just have to begin using this kind of aliases for
 templates which get only used with a few, known types, like in
 tango.text.Util.


They're not because current tool chain is not able to remove the unused
variations from the executables, thus you're likely to get 3 times the
codegen that you need, also called bloat.

-- 
Lars Ivar Igesund
blog at http://larsivi.net
DSource, #d.tango & #D: larsivi
Dancing the Tango
Mar 22 2008
next sibling parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 22/03/2008, Lars Ivar Igesund <larsivar igesund.net> wrote:

  > Now library developers just have to begin using this kind of aliases for
  > templates which get only used with a few, known types

 They're not because current tool chain is not able to remove the unused
  variations from the executables, thus you're likely to get 3 times the
  codegen that you need, also called bloat.


If that's true then the solution is to fix the linker.

This thread is about looking ahead to what /could/ be possible, not
about what is possible now. We /could/ implement return types whose
constancy is dependent on input types, if, in the future, if (1) the
suggestions I made are implemented, and (2) the linker is optimised to
remove that which is unused. As a side-effect, Sean's
three-types-of-string example would then also benefit from that linker
optimisation.

That said, I'm /very/ surprised to hear that OptLink can't do that already.
Mar 22 2008
parent reply Lars Ivar Igesund <larsivar igesund.net> writes:
Janice Caron wrote:


 On 22/03/2008, Lars Ivar Igesund <larsivar igesund.net> wrote:

  > Now library developers just have to begin using this kind of aliases
  > for templates which get only used with a few, known types

 They're not because current tool chain is not able to remove the unused
  variations from the executables, thus you're likely to get 3 times the
  codegen that you need, also called bloat.


 
 If that's true then the solution is to fix the linker.
 
 This thread is about looking ahead to what /could/ be possible, not
 about what is possible now. We /could/ implement return types whose
 constancy is dependent on input types, if, in the future, if (1) the
 suggestions I made are implemented, and (2) the linker is optimised to
 remove that which is unused. As a side-effect, Sean's
 three-types-of-string example would then also benefit from that linker
 optimisation.
 
 That said, I'm /very/ surprised to hear that OptLink can't do that
 already.


Optlink doesn't and Walter has stated in the past that optlink will not be
developed further. Note that ld isn't any better, but that llvmdc should be
capable to elide unused instantiatons.

-- 
Lars Ivar Igesund
blog at http://larsivi.net
DSource, #d.tango & #D: larsivi
Dancing the Tango
Mar 22 2008
parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 22/03/2008, Lars Ivar Igesund <larsivar igesund.net> wrote:

 Optlink doesn't and Walter has stated in the past that optlink will not be
  developed further.


Then Walter has either contradicted himself or changed his mind! :-)
No biggie - happens to us all.

I can't be bothered to hunt it down right now, but I can distinctly
remember Walter saying that the solution to one particular problem
(template bloat caused by e.g. int, const(int) and invariant(int)
being distinct) would be to improve the linker. So I suspect this
avenue is not so closed as you might have thought.
Mar 22 2008
parent Bill Baxter <dnewsgroup billbaxter.com> writes:
Janice Caron wrote:

 On 22/03/2008, Lars Ivar Igesund <larsivar igesund.net> wrote:

 Optlink doesn't and Walter has stated in the past that optlink will not be
  developed further.


 
 Then Walter has either contradicted himself or changed his mind! :-)
 No biggie - happens to us all.
 
 I can't be bothered to hunt it down right now, but I can distinctly
 remember Walter saying that the solution to one particular problem
 (template bloat caused by e.g. int, const(int) and invariant(int)
 being distinct) would be to improve the linker. So I suspect this
 avenue is not so closed as you might have thought.


Yeh, but saying the solution "would be to improve the linker" is 
different from saying "I'm going to fix the linker".

What I recall Walter saying in the past is that he would love to 
_rewrite_ the linker, but it is just not practical because of the vast 
amounts of time it would take to do so.

--bb
Mar 22 2008
prev sibling parent e-t172 <e-t172 akegroup.org> writes:
Lars Ivar Igesund a écrit :

 They're not because current tool chain is not able to remove the unused
 variations from the executables, thus you're likely to get 3 times the
 codegen that you need, also called bloat.


I was talking about shared libraries. When you're building a shared 
library, it makes sense to include the three versions in it.
Mar 22 2008
prev sibling parent "Janice Caron" <caron800 googlemail.com> writes:
On 22/03/2008, e-t172 <e-t172 akegroup.org> wrote:

  A better
  idea would be to include a new "feature" in D 2.0, i.e. "compiled
  templates", with a new syntax like this :




I believe you can already do that - albeit with a different syntax.

    T[] trim(T) (T[] source)
    {
        /*...*/
    }

    private
    {
        alias trim!(char) dummy1;
        alias trim!(wchar) dummy2;
        alias trim!(dchar) dummy3;
    }

I think those aliases will force an instantiation.
Mar 22 2008
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Janice Caron" wrote

 On 21/03/2008, Walter Bright wrote:

 There's been some discussion about this, but no good resolution.


 Allow me to offer a suggestion.
 ...


Sorry to say this, but this idea adds nothing to templates.  Since const T, 
invariant T, and just T are all distinct types, I can just write the 
template:

T[] f(T)(T[] buf)

And it accomplishes what you are trying to do (instantiates a version based 
on the constness of T).  If I want to guarantee T is a char, I think I can 
do:

T[] f(T : char)(T[] buf)

or something similar (I don't work with D2 daily, so I'm not sure how this 
syntax would work), or if this doesn't work, use a static if.

This does not solve the problem that I had originally queried about.  The 
problem with your method is still that the non-const version does not 
guarantee that f does not change the input buffer.  I want to specify that 
the function does not change the input buffer, and returns the same type as 
passed in (without duping the input).

I have been thinking about this a lot, and I think actually in the grand 
scheme of things, this is not a blocker, as even though the compiler does 
not guarantee constness for the non-const version, one can document that a 
function is const (and even test this by compiling a const version in a 
unittest).  However, it is a hole in the const scheme, which should probably 
be fixed at some point.

-Steve
Mar 22 2008
next sibling parent "Janice Caron" <caron800 googlemail.com> writes:
On 22/03/2008, Steven Schveighoffer <schveiguy yahoo.com> wrote:

  T[] f(T)(T[] buf)

  And it accomplishes what you are trying to do (instantiates a version based
  on the constness of T).


There is a difference. Your template has an infinite number of
possibilities for T, whereas my suggestion has exactly three
possibilities for K.



  This does not solve the problem that I had originally queried about.  The
  problem with your method is still that the non-const version does not
  guarantee that f does not change the input buffer.


I think it does actually, because if the template compiles for K ==
const, then the code does not modify the buffer - and /the same code/
is generated in all three cases, remember.

(OK, there could be exceptions to that rule if you've got some static
iffiness going on, but I think that's rare enough that it shouldn't
stomp on the idea).

But if you /really/ want to be sure, you only have to write your
function slightly differently, as below.



 I want to specify that
  the function does not change the input buffer, and returns the same type as
  passed in (without duping the input).


I think you can still solve your particular problem with my scheme by doing

    K(char)[] f(const K)(K(char)[] buf_)
    {
        K(char)[] ret;
        const(char)[] buf = buf_;
        // never refer to buf_ again.

       return ret;
    }

If you don't like the notion of having to remember not to refer to
buf_ again within the function, more exotic possibilities remain, such
as:

    K(char)[] f(const K)(K(char)[] buf)
    {
        K(char)[] f_(const(char)[] buf)
        {
            /*...*/
        }
        return f_(buf);
    }

So it /can/ be done.
Mar 22 2008
prev sibling next sibling parent Bill Baxter <dnewsgroup billbaxter.com> writes:
Steven Schveighoffer wrote:

 "Janice Caron" wrote



 I have been thinking about this a lot, and I think actually in the grand 
 scheme of things, this is not a blocker, as even though the compiler does 
 not guarantee constness for the non-const version, one can document that a 
 function is const (and even test this by compiling a const version in a 
 unittest).  However, it is a hole in the const scheme, which should probably 
 be fixed at some point.


So the solution to your problem is to use "documentation" const?  The 
good thing about that is that it works for D1.0 too!  :-)

--bb
Mar 22 2008
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Steven Schveighoffer wrote:

 I want to specify that 
 the function does not change the input buffer, and returns the same type as 
 passed in (without duping the input).


Look at it another way. You want to declare a filter that does not 
change the input, but can pass it along to another function that will. I 
  suspect this might be a fundamentally broken concept:

    T[] f(const T[]);
    void g(T[]);

    ...

    T[] t;
    g(f(t));

What's happened here? Although f() promised it wouldn't modify t, it got 
modified anyway, because f() passed t to g() which modified it. Far from 
being a weakness in D's const regime, I think the fact that such cannot 
be implemented without casting away const at some point shows that the 
const system is sound, and trying to implement such a filter is unsound.
Mar 22 2008
next sibling parent reply BCS <ao pathlink.com> writes:
Reply to Walter,


 Steven Schveighoffer wrote:
 

 I want to specify that the function does not change the input buffer,
 and returns the same type as passed in (without duping the input).
 


 Look at it another way. You want to declare a filter that does not
 change the input, but can pass it along to another function that will.
 I
 suspect this might be a fundamentally broken concept:
 T[] f(const T[]);
 void g(T[]);
 ...
 
 T[] t;
 g(f(t));

 What's happened here? Although f() promised it wouldn't modify t, it
 got modified anyway, because f() passed t to g() which modified it.
 Far from being a weakness in D's const regime, I think the fact that
 such cannot be implemented without casting away const at some point
 shows that the const system is sound, and trying to implement such a
 filter is unsound.
 


I think what is being asked for is a function that asserts that it won't 
change what it is given but, you can do anything with the output that you 
could do to the input. Inside it would not be allowed to change it's arg 
but can return it. This should be safe because anything you do the the return 
value could be done to the argument giving the same end effect.
Mar 22 2008
parent reply Walter Bright <newshound1 digitalmars.com> writes:
BCS wrote:

 Look at it another way. You want to declare a filter that does not
 change the input, but can pass it along to another function that will.
 I
 suspect this might be a fundamentally broken concept:
 T[] f(const T[]);
 void g(T[]);
 ...

 T[] t;
 g(f(t));

 What's happened here? Although f() promised it wouldn't modify t, it
 got modified anyway, because f() passed t to g() which modified it.
 Far from being a weakness in D's const regime, I think the fact that
 such cannot be implemented without casting away const at some point
 shows that the const system is sound, and trying to implement such a
 filter is unsound.


 
 I think what is being asked for is a function that asserts that it won't 
 change what it is given but, you can do anything with the output that 
 you could do to the input. Inside it would not be allowed to change it's 
 arg but can return it.


That's how I interpreted it, too.


 This should be safe because anything you do the 
 the return value could be done to the argument giving the same end effect.


I don't believe it's safe because, as my example shows, it violates its 
contract with the caller.
Mar 22 2008
parent BCS <BCS pathlink.com> writes:
Walter Bright wrote:

 BCS wrote:
 

 Look at it another way. You want to declare a filter that does not
 change the input, but can pass it along to another function that will.
 I
 suspect this might be a fundamentally broken concept:
 T[] f(const T[]);
 void g(T[]);
 ...

 T[] t;
 g(f(t));

 What's happened here? Although f() promised it wouldn't modify t, it
 got modified anyway, because f() passed t to g() which modified it.
 Far from being a weakness in D's const regime, I think the fact that
 such cannot be implemented without casting away const at some point
 shows that the const system is sound, and trying to implement such a
 filter is unsound.




 
 That's how I interpreted it, too.
 


OK


 This should be safe because anything you do the the return value could 
 be done to the argument giving the same end effect.


 
 
 I don't believe it's safe because, as my example shows, it violates its 
 contract with the caller.


I see your point but I'm not sure I agree.

It depends on the small print. Is the contract that the function will 
not modify the data or that calling the function will not result in data 
being modified? (I know that's not to clear but I can't think of a 
clearer way to say that)

In the first case there is no problem because the f function does not 
modify the data, the g function does the modification. In the second 
there is a problem. However, unless I'm missing the point of the request 
(a real possibility) this is all moot as what is requested is not that 
the example above be allowed to work but that something like this be 
allowed:

T[] f(return_const T[]);
void g(T[]);

T[] t;
g(f(t));

for this case the contract can be anything /you/ want.
Mar 24 2008
prev sibling next sibling parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 22/03/2008, Walter Bright <newshound1 digitalmars.com> wrote:

 Look at it another way. You want to declare a filter that does not
  change the input, but can pass it along to another function that will. I
   suspect this might be a fundamentally broken concept:


I don't think it's a broken concept. In fact, I think that what
Stephen wants is completely reasonable and useful.

Taking what Stephen wants literally, here's how you do it

    const(char)[] f(const(char)[] a)
    {
        /* Stephen's function body */
        return a[1..$];
    }

    char[] f(char[] a)
    {
        return cast(char[]) f(cast(const(char)[])a);
    }

    invariant(char)[] f(invariant(char)[] a)
    {
        return cast(invariant(char)[]) f(cast(const(char)[])a);
    }

While /in general/ it is not safe to cast away constancy, in this
particular case it is obviously safe since all you're doing is
returning a slice of the input.

However, I also argue with Stephen, because I believe that the
following would /also/ work:

    T[] f(T)(T[] a)
    {
        /* Stephen's function body */
        return a[1..$];
    }

or, using the suggestion I made further up the thread

    K(char)[] f(const K)(K(char)[] a)
    {
        /* Stephen's function body */
        return a[1..$];
    }

I know that Stephen will argue with me that the mutable version makes
no promise not to modify the input, but so what? It /doesn't/ modify
the input, and that promise is still there implicitly, because if it
did modify the input then the const version wouldn't compile!

I suppose then that the trick is to prove that the const version
compiles. Well that's easy - just force the const version to
instantiate.
Mar 23 2008
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Janice Caron wrote:

 It /doesn't/ modify the input,


By returning the input as a modifiable aggregate, it breaks its promise 
not to modify it. The cast to remove const-ness is where the problem is. 
Consider again:

    T[] f(const T[]);
    void g(T[]);

    ...

    T[] t;
    g(f(t));
    ... (*) ...

Let's say g() modifies its argument. Then, at point (*), t has changed, 
even though t was passed to a function that promised not to mutate it. 
This is fundamentally a broken program.
Mar 23 2008
parent reply "Janice Caron" <caron800 googlemail.com> writes:
Look, let's come up with a real life use case. Let's say we want to
implement a D version of strchr(). How do we declare it? How about

    char[] strchr(char[] s, char c)
    {
        int n = s.find(c);
        return n == -1 ? null : s[n..$];
    }

Trouble is, that declaration suggests that s might be modified. This
means, if there is a bug within the function body which inadvertently
modifies s, the compiler won't catch it.

Fortunately, I think I've already solved the problem, a few posts up.
The solution in this case is:

   T strchr(T : const(char)[])(T s, char c)
   {
        int n = s.find(c);
        return n == -1 ? null : s[n..$];
   }

I don't see that there is anything fundamentally broken about that. T
is any type which will implicitly cast to const(char)[]; you have
compile-time checking that s doesn't get modified within the function;
s /can/ be modified via it's return value /if s is not const/. This is
exactly what's being requested, is it not? Where's the problem with
it?
Mar 23 2008
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Janice Caron wrote:

 Fortunately, I think I've already solved the problem, a few posts up.
 The solution in this case is:
 
    T strchr(T : const(char)[])(T s, char c)
    {
         int n = s.find(c);
         return n == -1 ? null : s[n..$];
    }
 
 I don't see that there is anything fundamentally broken about that.


It isn't broken because the return type T is const.


 T
 is any type which will implicitly cast to const(char)[]; you have
 compile-time checking that s doesn't get modified within the function;
 s /can/ be modified via it's return value /if s is not const/. This is
 exactly what's being requested, is it not? Where's the problem with
 it?


The problem with that strchr is it doesn't address Steven's issue - if 
the input is const, the output is, too. strchr is otherwise correct.
Mar 23 2008
next sibling parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 23/03/2008, Walter Bright <newshound1 digitalmars.com> wrote:

  >    T strchr(T : const(char)[])(T s, char c)

 It isn't broken because the return type T is const.


Is it? In that case, I've misunderstood template syntax. I thought that

    (T : U)

meant any kind of T which could be implicitly cast to U, and that, therefore:

    (T : const(char)[])

would mean any kind of T which could be implicitly cast to
const(char)[] - which clearly includes char[]. If T can be char[] (as
I assumed) then the return type isn't necessarily const; if T cannot
be char[] (as you seem to be saying) then does that mean (T : U)
doesn't work as I thought it did?

I guess, if I've misunderstood template syntax that badly then
template syntax is /definitely/ too confusing.
Mar 23 2008
next sibling parent Christopher Wright <dhasenan gmail.com> writes:
Janice Caron wrote:

 On 23/03/2008, Walter Bright <newshound1 digitalmars.com> wrote:

  >    T strchr(T : const(char)[])(T s, char c)

 It isn't broken because the return type T is const.


 
 Is it? In that case, I've misunderstood template syntax. I thought that
 
     (T : U)
 
 meant any kind of T which could be implicitly cast to U, and that, therefore:
 
     (T : const(char)[])
 
 would mean any kind of T which could be implicitly cast to
 const(char)[] - which clearly includes char[]. If T can be char[] (as
 I assumed) then the return type isn't necessarily const; if T cannot
 be char[] (as you seem to be saying) then does that mean (T : U)
 doesn't work as I thought it did?
 
 I guess, if I've misunderstood template syntax that badly then
 template syntax is /definitely/ too confusing.


In that case, the following should, in theory, work:

class A {}
class B
{
    A opImplicitCast()
    {
       // ...
    }
}

template Template(T : A){}
Template!(B);

Of course, that fails.
Mar 23 2008
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Ok, I see what you mean. But what the template gives us is:

const(T)[] f(const(T)[] t) { ... }
T[] f(T[] t) { ... }

whereas what Steven is asking for is:

T[] f(const(T)[] t) { ... }
Mar 23 2008
parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 24/03/2008, Walter Bright <newshound1 digitalmars.com> wrote:

 Ok, I see what you mean. But what the template gives us is:

  const(T)[] f(const(T)[] t) { ... }
  T[] f(T[] t) { ... }

  whereas what Steven is asking for is:

  T[] f(const(T)[] t) { ... }


I've thought this through pretty deeply in the last few hours, and it
seems to me that everyone's right, in a sense. But I do believe you
are wrong in one aspect, which is that the concept is /not/
fundamentally broken, and I believe I can prove it.

First off, nobody is asking for

    T[] f(const(T)[] t) { ... }

That really would be fundamentally broken, as you say. In fact, lets
be explicit here - we want to return a slice of the original, so let's
write it as

    T[] f(T)(const(T)[] a) { return a[X..Y]; }

Broken, as you say, but *it's still not what's being asked for*.
What's being asked for is

    K(T)[] f(T)(const(T)[] a) { return cast(K(T)[]) a[X..Y]; }

where K is the constancy of a at the caller site. I claim that I can
prove that this is not fundamentally broken, and specifically that it
does not violate const correctness.

As the first step of the proof, let us rewrite the function to return
a pair of indeces. I don't think D can return multiple values yet, so
we'll assume the existence of a Slice struct:

    struct Indeces
    {
        uint begin;
        uint end;
    }

So now we rewrite the function as

    Indeces f(T)(const(T)[] a) { return Indeces(X,Y); }

Hopefully, you can clearly see that nothing is amiss here. The
function gets a read-only view of a, and does not modify it. All is
well. What it returns is two integers. Next, at the call-site, the
caller does:

    char[] a,b;
    Indeces t;

    t = f(a);
    b = a[t.begin..t.end];

This does exactly what is required, and again, I hope you'll agree
that const correctness has not been violated. Now recall Stephen's
example code from earlier:

    auto tmp = f(t);
    g(tmp);

Observe that I can now easily rewrite this as:

    auto tmp = f(t);
    g(t[tmp.begin..tmp.end]);

And again, no const correctness has been violated.

The reason that everything works is that "const" means "I have a
read-only view". In particular, the function f has a read-only view of
the array a - however, the /caller/ has a read/write view of that
array. It follows logically that the caller (who has a read/write
view) is always going to be able to modify the array through that
read/write view, even though the callee (who only has a read-only
view) cannot.

With that understanding, it follows that it /must/ be safe to pass a
read/write view of a slice of an array to someone *who already has* a
read-write view of the whole array.

QED

Now, with all that said, returning indeces is simpler and cleaner than
having to generate three versions of the function, so I believe that
returning indeces is the way to go.

We are told that in the future, D will have macros. When that day
comes, Stephen will be able to create his desired signature as a macro
instead of a function (because the slicing happens at the call-site,
not within the function body).

So I would like to suggest that the struct Indeces be added to the
standard library. (It's just too simple to expect functions to have to
reinvent the wheel over and over again.) Then everybody wins.
Mar 24 2008
parent reply "Neil Vice" <psgdg swiftdsl.com.au> writes:
My apologies if I misunderstand as I have not been following the discussion 
at large but could you not implemented the desired template method using two 
overloaded templates, differing only by the constness of the parameter and 
return type?

i.e.

const(T)[] f(const(T)[] t) { ... }


"Janice Caron" <caron800 googlemail.com> wrote in message 
news:mailman.188.1206343793.2351.digitalmars-d puremagic.com...

 On 24/03/2008, Walter Bright <newshound1 digitalmars.com> wrote:

 Ok, I see what you mean. But what the template gives us is:

  const(T)[] f(const(T)[] t) { ... }
  T[] f(T[] t) { ... }

  whereas what Steven is asking for is:

  T[] f(const(T)[] t) { ... }


 I've thought this through pretty deeply in the last few hours, and it
 seems to me that everyone's right, in a sense. But I do believe you
 are wrong in one aspect, which is that the concept is /not/
 fundamentally broken, and I believe I can prove it.

 First off, nobody is asking for

    T[] f(const(T)[] t) { ... }

 That really would be fundamentally broken, as you say. In fact, lets
 be explicit here - we want to return a slice of the original, so let's
 write it as

    T[] f(T)(const(T)[] a) { return a[X..Y]; }

 Broken, as you say, but *it's still not what's being asked for*.
 What's being asked for is

    K(T)[] f(T)(const(T)[] a) { return cast(K(T)[]) a[X..Y]; }

 where K is the constancy of a at the caller site. I claim that I can
 prove that this is not fundamentally broken, and specifically that it
 does not violate const correctness.

 As the first step of the proof, let us rewrite the function to return
 a pair of indeces. I don't think D can return multiple values yet, so
 we'll assume the existence of a Slice struct:

    struct Indeces
    {
        uint begin;
        uint end;
    }

 So now we rewrite the function as

    Indeces f(T)(const(T)[] a) { return Indeces(X,Y); }

 Hopefully, you can clearly see that nothing is amiss here. The
 function gets a read-only view of a, and does not modify it. All is
 well. What it returns is two integers. Next, at the call-site, the
 caller does:

    char[] a,b;
    Indeces t;

    t = f(a);
    b = a[t.begin..t.end];

 This does exactly what is required, and again, I hope you'll agree
 that const correctness has not been violated. Now recall Stephen's
 example code from earlier:

    auto tmp = f(t);
    g(tmp);

 Observe that I can now easily rewrite this as:

    auto tmp = f(t);
    g(t[tmp.begin..tmp.end]);

 And again, no const correctness has been violated.

 The reason that everything works is that "const" means "I have a
 read-only view". In particular, the function f has a read-only view of
 the array a - however, the /caller/ has a read/write view of that
 array. It follows logically that the caller (who has a read/write
 view) is always going to be able to modify the array through that
 read/write view, even though the callee (who only has a read-only
 view) cannot.

 With that understanding, it follows that it /must/ be safe to pass a
 read/write view of a slice of an array to someone *who already has* a
 read-write view of the whole array.

 QED

 Now, with all that said, returning indeces is simpler and cleaner than
 having to generate three versions of the function, so I believe that
 returning indeces is the way to go.

 We are told that in the future, D will have macros. When that day
 comes, Stephen will be able to create his desired signature as a macro
 instead of a function (because the slicing happens at the call-site,
 not within the function body).

 So I would like to suggest that the struct Indeces be added to the
 standard library. (It's just too simple to expect functions to have to
 reinvent the wheel over and over again.) Then everybody wins.
Mar 24 2008
parent "Neil Vice" <psgdg swiftdsl.com.au> writes:
 "Janice Caron" <caron800 googlemail.com> wrote:
 Broken, as you say, but *it's still not what's being asked for*.
 What's being asked for is

    K(T)[] f(T)(const(T)[] a) { return cast(K(T)[]) a[X..Y]; }

 where K is the constancy of a at the caller site. I claim that I can
 prove that this is not fundamentally broken, and specifically that it
 does not violate const correctness.


Apologies... apparently Ctrl-Enter sends in my client O_o

My apologies if I misunderstand as I have not been following the discussion
at large but could you not implemented the desired template method using two
overloaded templates, differing only by the constness of the parameter and
return type?

i.e.

const(T)[] f(const(T)[] t) { ... }
T[] f(T[] t) { ... }

where you could share implementation if desired using something like:

T[] f(T[] t)
{
    return cast(T[]) f(const(T) t);
}

I should probably have attempted this before posting but it's late and after 
my previous half-post I want to get this out there =)
Mar 24 2008
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Walter Bright" wrote

 Janice Caron wrote:

 Fortunately, I think I've already solved the problem, a few posts up.
 The solution in this case is:

    T strchr(T : const(char)[])(T s, char c)
    {
         int n = s.find(c);
         return n == -1 ? null : s[n..$];
    }

 I don't see that there is anything fundamentally broken about that.


 It isn't broken because the return type T is const.


 T
 is any type which will implicitly cast to const(char)[]; you have
 compile-time checking that s doesn't get modified within the function;
 s /can/ be modified via it's return value /if s is not const/. This is
 exactly what's being requested, is it not? Where's the problem with
 it?


 The problem with that strchr is it doesn't address Steven's issue - if the 
 input is const, the output is, too. strchr is otherwise correct.


This is a slight misinterpretation.  If the input is const, I DO want it to 
return const.  The issue is if the input is mutable, I want it to return 
mutable BUT I also want to be able to specify that the compiler should 
complain if the function modifies the mutable input.

-Steve
Mar 24 2008
parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
Steven Schveighoffer wrote:

 I also want to be able to specify that the compiler should
 complain if the function modifies the mutable input.


I think that right there is basically the *only* think I really want out 
of a const system.  And even for that "best effort" is good enough for 
me.  Just don't make my life complicated in the name of correctness.  If 
I want that I'll go use Haskell.

--bb
Mar 24 2008
parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 24/03/2008, Bill Baxter <dnewsgroup billbaxter.com> wrote:

 Steven Schveighoffer wrote:
  > I also want to be able to specify that the compiler should
  > complain if the function modifies the mutable input.

 I think that right there is basically the *only* think I really want out
  of a const system.  And even for that "best effort" is good enough for
  me.


No less than three working solutions have been mentioned on this
thread: a template solution, suggested by me; a solution using an
explicit cast suggested by Daniel919; and the notion of returning
indeces suggested by me. How many more working solutions do we need?
This is a solved problem.
Mar 24 2008
next sibling parent Bill Baxter <dnewsgroup billbaxter.com> writes:
Janice Caron wrote:

 On 24/03/2008, Bill Baxter <dnewsgroup billbaxter.com> wrote:

 Steven Schveighoffer wrote:
  > I also want to be able to specify that the compiler should
  > complain if the function modifies the mutable input.

 I think that right there is basically the *only* think I really want out
  of a const system.  And even for that "best effort" is good enough for
  me.


 
 No less than three working solutions have been mentioned on this
 thread: a template solution, suggested by me; a solution using an
 explicit cast suggested by Daniel919; and the notion of returning
 indeces suggested by me. How many more working solutions do we need?
 This is a solved problem.


It's not the fact that it can be done or cannot be done so much as it 
requiring 3 people and a dozen posts to piece together first what the 
actual objective was and second how to do it.  Just give me a simple way 
to avoid shooting myself in the foot in obvious ways and I think I will 
be satisfied, even if the result does not offer a 100% guarantee.

--bb
Mar 24 2008
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Janice Caron" wrote

 On 24/03/2008, Bill Baxter wrote:

 Steven Schveighoffer wrote:
  > I also want to be able to specify that the compiler should
  > complain if the function modifies the mutable input.

 I think that right there is basically the *only* think I really want out
  of a const system.  And even for that "best effort" is good enough for
  me.


 No less than three working solutions have been mentioned on this
 thread: a template solution, suggested by me; a solution using an
 explicit cast suggested by Daniel919; and the notion of returning
 indeces suggested by me. How many more working solutions do we need?
 This is a solved problem.


No, it is not a solved problem.  As a caller of a function, you still cannot 
rely on the compiler verifying that a function does not modify it's 
arguments for mutable arguments.  The only way to be sure is to read the 
source code yourself.  I'd much rather have the compiler do it.

-Steve
Mar 24 2008
parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 24/03/2008, Steven Schveighoffer <schveiguy yahoo.com> wrote:

 No, it is not a solved problem.  As a caller of a function, you still cannot
  rely on the compiler verifying that a function does not modify it's
  arguments for mutable arguments.


I don't follow that. I believe that at least two of the three
solutions do indeed make that guarantee.

Solution 2: Explicit cast

    T strchr(T)(const T s, char c)
    {
        return cast(T)(whatever);
    }

The declaration of s as "const T" guarantees that the function body
will not modify s.

Solution 3: Return a range

    Range!(uint) strchr(const(char)[] s, char c)

The declaration of s as "const(char)[]" guarantees that the function
body will not modify the elements of s.
Mar 24 2008
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Janice Caron" wrote

 On 24/03/2008, Steven Schveighoffer wrote:

 No, it is not a solved problem.  As a caller of a function, you still 
 cannot
  rely on the compiler verifying that a function does not modify it's
  arguments for mutable arguments.


 I don't follow that. I believe that at least two of the three
 solutions do indeed make that guarantee.

 Solution 2: Explicit cast

    T strchr(T)(const T s, char c)
    {
        return cast(T)(whatever);
    }

 The declaration of s as "const T" guarantees that the function body
 will not modify s.


This does not solve the problem, as if I pass a const T to the function, it 
violates const correctness, just like the C version did.


 Solution 3: Return a range

    Range!(uint) strchr(const(char)[] s, char c)

 The declaration of s as "const(char)[]" guarantees that the function
 body will not modify the elements of s.


I don't consider this to be a solution because you must implement part of 
the function every time you call it.  Here is a better case:

T min(T)(T firstval, T secondval)
{
  return firstval < secondval ? firstval : secondval;
}

This should have const on firstval and secondval, even if T is a mutable 
type, but because of the current const rules, it can't.  As a user of min, I 
am still forced to read the implementation of the function, or trust the 
implementor of the function to have done what he says it does (and for 
him/her to have checked all the functions he might call).  I'd much rather 
trust the compiler, and not worry about it.

Yes, you could rewrite the function as:

bool less(T)(const(T) firstval, const(T) secondval)
{
   return firstval < secondval;
}

// x = min(x, y);
x = less(x, y) ? x : y;

Now this function is totally useless as I can rewrite it as:

x = x < y ? x : y;

I don't like this general design idea because the caller must know how to 
build the 'real' result based on the parameters given back, and basically 
must re-implement some or all of the function every time he/she calls it. 
There are more abstract cases too, that you would almost have to return a 
delegate, and have the user call the delegate on the parameter again!  Think 
of how awful implementing such a function would be.  I'd much rather have 
the function itself construct the result in the way it wants to, then tell 
the caller how to construct it.

-Steve
Mar 24 2008
next sibling parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 24/03/2008, Steven Schveighoffer <schveiguy yahoo.com> wrote:

  > Solution 2: Explicit cast
  >
  >    T strchr(T)(const T s, char c)
  >    {
  >        return cast(T)(whatever);
  >    }
  >
  > The declaration of s as "const T" guarantees that the function body
  > will not modify s.

 This does not solve the problem, as if I pass a const T to the function, it
  violates const correctness


Not according to Daniel's code, it doesn't.

       const(char[]) s2b = "0123456789";
       assert (is(typeof(slice(s2b,1,4)) == typeof(s2b)));



  > Solution 3: Return a range

 I don't consider this to be a solution because you must implement part of
  the function every time you call it.


Trivially solved with macros. (Or at least, it will be, when D has macros).




  Here is a better case:

  T min(T)(T firstval, T secondval)
  {
   return firstval < secondval ? firstval : secondval;
  }


Ooh - now you're changing the ground rules! In your first post, it was
all about returning a slice into an array. This is a new problem.

It's a good one though! I'm gonna have to think about that for a bit!
Mar 24 2008
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Janice Caron" wrote

 On 24/03/2008, Steven Schveighoffer wrote:

  > Solution 2: Explicit cast
  >
  >    T strchr(T)(const T s, char c)
  >    {
  >        return cast(T)(whatever);
  >    }
  >
  > The declaration of s as "const T" guarantees that the function body
  > will not modify s.

 This does not solve the problem, as if I pass a const T to the function, 
 it
  violates const correctness


 Not according to Daniel's code, it doesn't.

       const(char[]) s2b = "0123456789";
       assert (is(typeof(slice(s2b,1,4)) == typeof(s2b)));


What about:

assert (is(typeof(slice!(char[])(s2b,1,4)) == typeof(s2b)));

It's not a solution if abuse is allowed by the compiler without complaint :)


  > Solution 3: Return a range

 I don't consider this to be a solution because you must implement part of
  the function every time you call it.


 Trivially solved with macros. (Or at least, it will be, when D has 
 macros).


I hope never to have to use macros, as I hated them in C/C++, and I still 
hate them :)  In any case, having to implement macros to workaround a hole 
in the const system is not going to be a good promoter of using const.


  Here is a better case:

  T min(T)(T firstval, T secondval)
  {
   return firstval < secondval ? firstval : secondval;
  }


 Ooh - now you're changing the ground rules! In your first post, it was
 all about returning a slice into an array. This is a new problem.

 It's a good one though! I'm gonna have to think about that for a bit!


Technically, it's the same problem, just a different example :)

-Steve
Mar 24 2008
prev sibling parent reply "Craig Black" <cblack ara.com> writes:
 T min(T)(T firstval, T secondval)
 {
  return firstval < secondval ? firstval : secondval;
 }

 This should have const on firstval and secondval, even if T is a mutable 
 type, but because of the current const rules, it can't.


I must have missed something.  What's wrong with this:

T min(T)(const T a, const T b) { return a < b ? a : b; }
Mar 24 2008
parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 24/03/2008, Craig Black <cblack ara.com> wrote:

 I must have missed something.  What's wrong with this:

  T min(T)(const T a, const T b) { return a < b ? a : b; }


Surely, that won't compile, because the type of (a < b ? a : b) is
const(T), and const(T) won't implicitly convert to T.
Mar 24 2008
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Janice Caron" wrote

 On 24/03/2008, Craig Black wrote:

 I must have missed something.  What's wrong with this:

  T min(T)(const T a, const T b) { return a < b ? a : b; }


 Surely, that won't compile, because the type of (a < b ? a : b) is
 const(T), and const(T) won't implicitly convert to T.


Well, with the latest version of DMD, it will for value types that are not 
pointers, but not for ones that are reference or pointer types :)

i.e. compiles for int, not for class C

Of course, your point is correct, it doesn't work how you want it to.

-Steve
Mar 24 2008
prev sibling next sibling parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 23/03/2008, Janice Caron <caron800 googlemail.com> wrote:

 I don't think it's a broken concept. In fact, I think that what
  Stephen wants is completely reasonable and useful.


Stephen, I think this does the trick.

    T f(T : const(char)[])(T a)
    {
        return a[1..$];
    }

    private alias f!(const(char)[]) dummy;

The alias forces an instantiation for T == const(char)[], and if that
compiles, then the function does not modify its input.
Mar 23 2008
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Janice Caron" wrote in message

 On 23/03/2008, Janice Caron wrote:

 I don't think it's a broken concept. In fact, I think that what
  Stephen wants is completely reasonable and useful.


 Stephen, I think this does the trick.

    T f(T : const(char)[])(T a)
    {
        return a[1..$];
    }

    private alias f!(const(char)[]) dummy;

 The alias forces an instantiation for T == const(char)[], and if that
 compiles, then the function does not modify its input.


This is similar to what I was saying, but I appreciate the actual method of 
ensuring to the developer of f that the compiler has checked for 
const-correctness.  This, however, does nothing for the user of f, who must 
still check the source code, or instantiate a const version himself (which 
even then does not guarantee as you have said, static if could get around 
that).

However, I have lessened my belief that this ommission is bad, as there are 
solutions that get very close to optimal.  I still think it would be nice to 
have a way to specify this kind of contract, but if it never happens, it's 
not going to block my work.

-Steve
Mar 23 2008
next sibling parent "Janice Caron" <caron800 googlemail.com> writes:
On 23/03/2008, Steven Schveighoffer <schveiguy yahoo.com> wrote:

  This, however, does nothing for the user of f, who must
  still check the source code, or instantiate a const version himself


In that case, I haven't understood your need.

Who needs to check what, and why? What would be a good example use case?
Mar 23 2008
prev sibling parent "Janice Caron" <caron800 googlemail.com> writes:
Stephen, there is another way to achieve the same thing. Instead of doing

    newArray = f(oldArray);

you can do

    auto t = f(oldArray);
    newArray = oldArray[t.first..t.second];

where t is a Pair!(int)

    struct Pair(T)
    {
        T first;
        T second;
    }

The idea here is that instead of returning an actual slice, f just
returns the indeces. Then the callee can use those indeces to take the
slice from the original array. So f would then be written something
like

    Pair!(int) f(const(char)[] array)
    {
        return Pair!(int)(1,array.length);
    }

See, now the type of the caller's array is completely unknown to the
callee, but that doesn't matter, because now it only has to return two
ints. The caller, on the other hand, gets a guarantee that f won't
modify the array, /and/ can use the returned pair to make a slice that
is guaranteed to be of exactly the right type.

(I'm not sure if Pair! exists in a standard library anywhere. If it
doesn't, it probably should).
Mar 23 2008
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Walter Bright" wrote

 Steven Schveighoffer wrote:

 I want to specify that the function does not change the input buffer, and 
 returns the same type as passed in (without duping the input).


 Look at it another way. You want to declare a filter that does not change 
 the input, but can pass it along to another function that will. I suspect 
 this might be a fundamentally broken concept:

    T[] f(const T[]);
    void g(T[]);

    ...

    T[] t;
    g(f(t));

 What's happened here? Although f() promised it wouldn't modify t, it got 
 modified anyway, because f() passed t to g() which modified it. Far from 
 being a weakness in D's const regime, I think the fact that such cannot be 
 implemented without casting away const at some point shows that the const 
 system is sound, and trying to implement such a filter is unsound.


I wholeheartedly disagree :)

g(f(t)) is a shortcut for:

auto tmp = f(t);
g(tmp);

and as you can see, using g on the output of f is not f's fault.  Even if 
you wanted a filter function, I would expect the filter to return exactly 
the same type as it was given (as it is meant to be injected in place as an 
argument to another function which would normally take the same input).

Actually, your example demonstrates this point exactly.  If the code looks 
like:

g(t);

and I say "oh, I want to filter that input with f":

g(f(t));

If f now returns const T[], then this will not compile, and the filter is 
useless, but here comes the worst problem.  If someone else is responsible 
for f, they might say "hey, f doesn't modify the input, so I can re-label it 
const.  Oops, it won't let me return a mutable array, I'll just change that" 
without realizing the implications it has on its usage.

I think people are going to have to leave out const for such fucntions and 
templatize them, even if they are correctly const, and therefore, const 
loses some of its value (not to mention, if you templatize f, it can 
generate 2 or 3 identical functions for no good reason).

I have been thinking of a way to specify this type of function, but it's not 
completely cooked, when it is, I'll post my idea.

-Steve
Mar 23 2008
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Steven Schveighoffer wrote:

 "Walter Bright" wrote

 Steven Schveighoffer wrote:

 I want to specify that the function does not change the input buffer, and 
 returns the same type as passed in (without duping the input).


 Look at it another way. You want to declare a filter that does not change 
 the input, but can pass it along to another function that will. I suspect 
 this might be a fundamentally broken concept:

    T[] f(const T[]);
    void g(T[]);

    ...

    T[] t;
    g(f(t));

 What's happened here? Although f() promised it wouldn't modify t, it got 
 modified anyway, because f() passed t to g() which modified it. Far from 
 being a weakness in D's const regime, I think the fact that such cannot be 
 implemented without casting away const at some point shows that the const 
 system is sound, and trying to implement such a filter is unsound.


 
 I wholeheartedly disagree :)
 
 g(f(t)) is a shortcut for:
 
 auto tmp = f(t);
 g(tmp);
 
 and as you can see, using g on the output of f is not f's fault.


Yes, it is, because f returned a mutable reference to const data. This 
is *fundamentally* unsound because f promised not to change its input, 
and yet by returning a mutable reference to it it allows others to 
change it.


 Even if 
 you wanted a filter function, I would expect the filter to return exactly 
 the same type as it was given (as it is meant to be injected in place as an 
 argument to another function which would normally take the same input).


Splitting g(f()) into two statements with a temporary does not 
fundamentally alter anything, and if it did, that should be a huge red 
flag that something is wrong.


 Actually, your example demonstrates this point exactly.  If the code looks 
 like:
 
 g(t);
 
 and I say "oh, I want to filter that input with f":
 
 g(f(t));
 
 If f now returns const T[], then this will not compile,


right


 and the filter is 
 useless, but here comes the worst problem.  If someone else is responsible 
 for f, they might say "hey, f doesn't modify the input, so I can re-label it 
 const.  Oops, it won't let me return a mutable array, I'll just change that" 
 without realizing the implications it has on its usage.


That's right - that's what const-correctness is all about. 
Const-correctness is useless otherwise <g>.



 I think people are going to have to leave out const for such fucntions and 
 templatize them, even if they are correctly const, and therefore, const 
 loses some of its value (not to mention, if you templatize f, it can 
 generate 2 or 3 identical functions for no good reason).


Templatizing is the language solution. I know about the bloat problem, 
but that is really an implementation issue. I know the linker doesn't do 
it now, but identical blocks of generated code with different names 
should be merged.
Mar 23 2008
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Walter Bright" wrote

 Steven Schveighoffer wrote:

 "Walter Bright" wrote

 Steven Schveighoffer wrote:

 I want to specify that the function does not change the input buffer, 
 and returns the same type as passed in (without duping the input).


 Look at it another way. You want to declare a filter that does not 
 change the input, but can pass it along to another function that will. I 
 suspect this might be a fundamentally broken concept:

    T[] f(const T[]);
    void g(T[]);

    ...

    T[] t;
    g(f(t));

 What's happened here? Although f() promised it wouldn't modify t, it got 
 modified anyway, because f() passed t to g() which modified it. Far from 
 being a weakness in D's const regime, I think the fact that such cannot 
 be implemented without casting away const at some point shows that the 
 const system is sound, and trying to implement such a filter is unsound.


 I wholeheartedly disagree :)

 g(f(t)) is a shortcut for:

 auto tmp = f(t);
 g(tmp);

 and as you can see, using g on the output of f is not f's fault.


 Yes, it is, because f returned a mutable reference to const data. This is 
 *fundamentally* unsound because f promised not to change its input, and 
 yet by returning a mutable reference to it it allows others to change it.


The block to your understanding might be because the language does not allow 
me to express my point :)

your original example had

T[] f(const T[] arg)

Which, I agree, if specified this way cannot return a slice of the argument, 
because that would violate const rules.  What I'm talking about is:

??? T[] f(??? T[] arg)

Where the ??? is replaced by some yet uncreated construct that says "you 
cannot modify arg while in f, but f can return a subset of arg".  The return 
type is dependent on the argument type, similar to a template.  If the 
argument passed is mutable, the return value is mutable.  If the argument 
passed is const, the return value is const.  However, even if the argument 
passed is mutable, it cannot be changed while inside f, allowing the coder 
to specify that f does not modify the arg, without restricting the caller 
from modifying the arg (if the arg is mutable to begin with).

Two great examples are Janice's example of strchr, and a min/max template. 
None of these should modify the arguments, but if specified as const 
arguments, the results must then also be const (with the current regime), 
thereby forcing you not to specify const in the signature, but rather in the 
documentation.


 Even if you wanted a filter function, I would expect the filter to return 
 exactly the same type as it was given (as it is meant to be injected in 
 place as an argument to another function which would normally take the 
 same input).


 Splitting g(f()) into two statements with a temporary does not 
 fundamentally alter anything, and if it did, that should be a huge red 
 flag that something is wrong.


I agree :)  But if the arg is mutable, f didn't violate the rule I stated 
above, it did not change the arg.  It doesn't even violate const rules that 
you have specified, since the arg is mutable.  However, it also did not 
restrict the caller from changing the arg.


 Actually, your example demonstrates this point exactly.  If the code 
 looks like:

 g(t);

 and I say "oh, I want to filter that input with f":

 g(f(t));

 If f now returns const T[], then this will not compile,


 right


 and the filter is useless, but here comes the worst problem.  If someone 
 else is responsible for f, they might say "hey, f doesn't modify the 
 input, so I can re-label it const.  Oops, it won't let me return a 
 mutable array, I'll just change that" without realizing the implications 
 it has on its usage.


 That's right - that's what const-correctness is all about. 
 Const-correctness is useless otherwise <g>.


Hm... I would argue that const-correctness is then less useful, and most 
likely will not be used, thereby rendering it useless to all except 
functional programmers.  The problem is that you are peppering const all 
over phobos, and so this will be a huge barrier to people who just want to 
get things done and don't care about functional programming.  Without const 
being fully specified, it will go the way of C++ const, where it's a useful 
concept in theory, but fails in practice.  I'm not saying that all of const 
is bad, I think you are on the right track, but in order to be used in 
practice, you need to give everyone the tools that make sense.


 I think people are going to have to leave out const for such fucntions 
 and templatize them, even if they are correctly const, and therefore, 
 const loses some of its value (not to mention, if you templatize f, it 
 can generate 2 or 3 identical functions for no good reason).


 Templatizing is the language solution. I know about the bloat problem, but 
 that is really an implementation issue. I know the linker doesn't do it 
 now, but identical blocks of generated code with different names should be 
 merged.


That is good to hear :)

-Steve
Mar 24 2008
parent renoX <renosky free.fr> writes:
The issue with your "function which returns mutable data" is that you 
have no way to know (except looking at the code) whether the return 
value is a reference to the input value or if it's another buffer..

The COW style of D seems better implemented with const than without 
const now that we have it..

Regards,
renoX
Mar 24 2008
prev sibling next sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
== Quote from Steven Schveighoffer (schveiguy yahoo.com)'s article

 This issue was just brought to my attention, and I wonder if the other
 const-proponents have a good solution for this.  This is going to be very
 tough to deal with.
 How do you declare a function that takes an array, is not allowed to change
 the array, but returns a slice into the argument array, and the return type
 matches the argument type.
 For example, if I pass a mutable array to this function, I want it to pass
 me back a mutable slice into the array, but I also want a guarantee that the
 function will not modify the array.  I can't declare the argument is const
 because then the return value must also be const, as it is a slice of the
 original array.
 How does one solve this problem?


Use overloaded functions.  In theory, I think this should work:

T[] func(T)( T[] buf ) { ... }
const T[] func(T)( const T[] buf ) { ... }
invariant T[] func(T)( invariant T[] buf ) { ... }

Even if D doesn't overload on const-ness (I seem to recall that it doesn't
right now), I think the above
set of functions won't conflict because one will be more specialized than the
other, based on the
supplied type.  However, this aspect of D is fairly primitive so it's somewhat
of a shot in the dark.
Weren't templates in 2.0 eventually supposed to get some way to learn the
const-ness of a parameter?  I
seem to recall that from Walter and Andrei's conference presentation.


Sean
Mar 21 2008
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Sean Kelly" wrote in message

 == Quote from Steven Schveighoffer

 This issue was just brought to my attention, and I wonder if the other
 const-proponents have a good solution for this.  This is going to be very
 tough to deal with.
 How do you declare a function that takes an array, is not allowed to 
 change
 the array, but returns a slice into the argument array, and the return 
 type
 matches the argument type.
 For example, if I pass a mutable array to this function, I want it to 
 pass
 me back a mutable slice into the array, but I also want a guarantee that 
 the
 function will not modify the array.  I can't declare the argument is 
 const
 because then the return value must also be const, as it is a slice of the
 original array.
 How does one solve this problem?


 Use overloaded functions.  In theory, I think this should work:

 T[] func(T)( T[] buf ) { ... }
 const T[] func(T)( const T[] buf ) { ... }
 invariant T[] func(T)( invariant T[] buf ) { ... }

 Even if D doesn't overload on const-ness (I seem to recall that it doesn't 
 right now), I think the above
 set of functions won't conflict because one will be more specialized than 
 the other, based on the
 supplied type.  However, this aspect of D is fairly primitive so it's 
 somewhat of a shot in the dark.
 Weren't templates in 2.0 eventually supposed to get some way to learn the 
 const-ness of a parameter?  I
 seem to recall that from Walter and Andrei's conference presentation.


 Sean


Hm... can't you just do this like:

T[] func(T)(T[] buf) {...}

and T will be either const(T), invariant(T), or just T?

In any case, the crux of the problem is still there.  For the version with 
just T, the compiler is not guaranteeing that func doesn't modify the input.

-Steve
Mar 21 2008
next sibling parent Sean Kelly <sean invisibleduck.org> writes:
== Quote from Steven Schveighoffer (schveiguy yahoo.com)'s article

 Hm... can't you just do this like:
 T[] func(T)(T[] buf) {...}
 and T will be either const(T), invariant(T), or just T?
 In any case, the crux of the problem is still there.  For the version with
 just T, the compiler is not guaranteeing that func doesn't modify the input.


You're right of course.  Darn mornings.


Sean
Mar 21 2008
prev sibling parent reply Jarrod <qwerty ytre.wq> writes:
On Fri, 21 Mar 2008 14:15:10 -0400, Steven Schveighoffer wrote:


 In any case, the crux of the problem is still there.  For the version
 with just T, the compiler is not guaranteeing that func doesn't modify
 the input.
 
 -Steve


One of the biggest reasons for const in the first place was to have this 
type of guarantee available.. And now, you can't even use it properly?

I've always thought that using const function parameters was about making 
a contract that exists solely for the scope of that function.. Basically, 
const in the parameter would assure you that the function would not 
modify the data you pass it. The contract should not exist after the 
scope is left.

I do understand the problem behind this: you can pass a const array to a 
function that takes const and then get a mutable reference to the data 
returned, which is bad. But at the same time I think it's worse how it 
currently is.

I know everyone is throwing in some kind of template or overload idea, 
but mine is a little more simple: Change 'in'. Use 'in' to define a 
function that will take a value and not change it, but make no guarantees 
about the return type (it makes sense too. 'in' should pertain to inside 
the function only). So if you want your function to take a const 
reference and not change the data, use "const in" or just "const". That 
way you can make an overload or a template for each type, and guarantee 
for all three functions that you will not change the data inside the 
function.
Mar 24 2008
parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 24/03/2008, Jarrod <qwerty ytre.wq> wrote:

  I know everyone is throwing in some kind of template or overload idea,
  but mine is a little more simple: Change 'in'. Use 'in' to define a
  function that will take a value and not change it, but make no guarantees
  about the return type


I'm going to have to join forces with Walter on this one. It doesn't
matter what syntax you use to declare function parameters - any such
declaration, whatever the syntax, /will break const correctness/.

What if I declare

    T[] f(in T[] a, const(T)[] b)
    {
        auto c = someTest ? a : b;
        // what type is c?

        return c;
        // can we return c?
    }

It's just too messy. If you really, really, really want to do this,
you want the special tag to be on the return type, not on one of the
input parameters. After all, what would

    T[] f(in T[] a, in T[] b)

mean, in your scheme? Does it mean we can return either a slice of a
or a slice of b? I don't believe that that is in any way desirable.

The only way that I can think of doing it would be to tag the return
type, with something like:

    sliceof(a) f(const(T)[] a, const(T)[] b);

or

    sliceof(b) f(const(T)[] a, const(T)[] b);

That way, the compiler would know which input was allowed to be sliced
- and /there can be only one!/

More to the point, the compiler would have to /prove/ that the return
value was, in all cases, a slice of the declared variable, so that it
could emit a compile error if it wasn't. I don't know much about
implementing compilers, but it seems to me that this would be a very,
very hard thing to prove.

The simpler solution, I believe, is to have the function return a pair
of indeces, and then take the slice at the call site.
Mar 24 2008
next sibling parent reply Christopher Wright <dhasenan gmail.com> writes:
Janice Caron wrote:

 On 24/03/2008, Jarrod <qwerty ytre.wq> wrote:

  I know everyone is throwing in some kind of template or overload idea,
  but mine is a little more simple: Change 'in'. Use 'in' to define a
  function that will take a value and not change it, but make no guarantees
  about the return type


 
 I'm going to have to join forces with Walter on this one. It doesn't
 matter what syntax you use to declare function parameters - any such
 declaration, whatever the syntax, /will break const correctness/.


I think your suggestions are fundamentally broken. If I want a contract 
with a method, I shouldn't have to change my representation of my data.

This method should be const-correct by having parameters that are const 
for however long it's running. After that, everything goes back to the 
caller, which has different contracts. The caller can break contracts 
that the callee is bound to because they are different entities with 
different contracts.
Mar 24 2008
parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 24/03/2008, Christopher Wright <dhasenan gmail.com> wrote:

 Janice Caron wrote:
  > On 24/03/2008, Jarrod <qwerty ytre.wq> wrote:


  I know everyone is throwing in some kind of template or overload idea,




  >>  but mine is a little more simple: Change 'in'. Use 'in' to define a
  >>  function that will take a value and not change it, but make no guarantees
  >>  about the return type


 I'm going to have to join forces with Walter on this one. It doesn't


  > matter what syntax you use to declare function parameters - any such
  > declaration, whatever the syntax, /will break const correctness/.


 I think your suggestions are fundamentally broken.


Whoa - hang on here. You're going to have to define "fundamentally
broken". When Walter says "fundamentally broken" he usually means "not
const correct". What do you mean?

Though various options have been discussed, and various suggestions
have been floated, including by me, I am now recommending one, and
only one, solution, which is for the function to return a pair of
indeces. This suggestion supercedes any previous suggestions I may
have made. And I guarantee you that returning a pair of indeces is not
broken in any respect.



 If I want a contract
 with a method, I shouldn't have to change my representation of my data.


I wasn't aware that I had suggested changing representation of data.



  This method should be const-correct by having parameters that are const
  for however long it's running. After that, everything goes back to the
  caller, which has different contracts. The caller can break contracts
  that the callee is bound to because they are different entities with
  different contracts.


In general, you /cannot/ convert const to mutable in a return
statement. For example:

    const(char)[] g = "hello".dup; // A global variable

    char[] f()
    {
        return g; // NOT acceptable
    }

We can argue that a slice of an input parameter should be a special
case, but why bother? It's hard for the compiler to prove, and
returning indeces instead is trivially easy.
Mar 24 2008
parent reply Christopher Wright <dhasenan gmail.com> writes:
Janice Caron wrote:

 On 24/03/2008, Christopher Wright <dhasenan gmail.com> wrote:

 Janice Caron wrote:
  > On 24/03/2008, Jarrod <qwerty ytre.wq> wrote:


  I know everyone is throwing in some kind of template or overload idea,




  >>  but mine is a little more simple: Change 'in'. Use 'in' to define a
  >>  function that will take a value and not change it, but make no guarantees
  >>  about the return type


 I'm going to have to join forces with Walter on this one. It doesn't


  > matter what syntax you use to declare function parameters - any such
  > declaration, whatever the syntax, /will break const correctness/.


 I think your suggestions are fundamentally broken.


 
 Whoa - hang on here. You're going to have to define "fundamentally
 broken". When Walter says "fundamentally broken" he usually means "not
 const correct". What do you mean?


I mean a lot of people are saying "fundamentally broken" and I want to 
be like the cool people.

Though in this case, you solved a very particular instance of a very 
general problem, introducing a new language construct to do so.


 Though various options have been discussed, and various suggestions
 have been floated, including by me, I am now recommending one, and
 only one, solution, which is for the function to return a pair of
 indeces. This suggestion supercedes any previous suggestions I may
 have made. And I guarantee you that returning a pair of indeces is not
 broken in any respect.


Agreed, but it doesn't solve the problem if I'm not using a list-like 
data structure.


 If I want a contract
 with a method, I shouldn't have to change my representation of my data.


 
 I wasn't aware that I had suggested changing representation of data.


Const data is not mutable data. If I want to call a method with const 
parameters whose return type is based on its input type, then I have to 
make my data const.


  This method should be const-correct by having parameters that are const
  for however long it's running. After that, everything goes back to the
  caller, which has different contracts. The caller can break contracts
  that the callee is bound to because they are different entities with
  different contracts.


 
 In general, you /cannot/ convert const to mutable in a return
 statement. For example:
 
     const(char)[] g = "hello".dup; // A global variable
 
     char[] f()
     {
         return g; // NOT acceptable
     }
 
 We can argue that a slice of an input parameter should be a special
 case, but why bother? It's hard for the compiler to prove, and
 returning indeces instead is trivially easy.


I just want data that I can mutate and functions that can promise not to 
mutate them.

Originally, at the start of this const foray, everyone clamored for all 
function arguments being immutable by default. If we had that, I'm 
absolutely certain that this issue would have been solved several 
revisions ago.
Mar 24 2008
parent Jarrod <qwerty ytre.wq> writes:
On Mon, 24 Mar 2008 19:08:58 -0400, Christopher Wright wrote:
 

 I just want data that I can mutate and functions that can promise not to
 mutate them.


That's what I meant by redefining 'in'



 Originally, at the start of this const foray, everyone clamored for all
 function arguments being immutable by default. If we had that, I'm
 absolutely certain that this issue would have been solved several
 revisions ago.


Seconded.
Mar 24 2008
prev sibling next sibling parent reply Jarrod <qwerty ytre.wq> writes:
On Mon, 24 Mar 2008 10:50:16 +0000, Janice Caron wrote:


 On 24/03/2008, Jarrod <qwerty ytre.wq> wrote:

  I know everyone is throwing in some kind of template or overload idea,
  but mine is a little more simple: Change 'in'. Use 'in' to define a
  function that will take a value and not change it, but make no
  guarantees about the return type


 
 I'm going to have to join forces with Walter on this one. It doesn't
 matter what syntax you use to declare function parameters - any such
 declaration, whatever the syntax, /will break const correctness/.


Well I'm not really 'against' Walter here myself. I'm merely pointing out 
that attempting to stick to const-correctness ends up putting a hole in 
one of the key uses of const.
I provided a suggestion that both keeps const-correctness yet also allows 
for a simpler contract to merely show your data is safe at least when 
being used by the function you are calling.


 What if I declare
 
     T[] f(in T[] a, const(T)[] b)
     {
         auto c = someTest ? a : b;
         // what type is c?
 
         return c;
         // can we return c?
     }


I would like to know what someone is thinking if they make this function? 
If you were going to return anything that is going to potentially be 
const, then wouldn't you make 'a' const as well?
Why would you potentially screw yourself over like that? Sometimes the 
programmer needs to do the sanity checks.



 It's just too messy. If you really, really, really want to do this, you
 want the special tag to be on the return type, not on one of the input
 parameters. After all, what would
 
     T[] f(in T[] a, in T[] b)
 
 mean, in your scheme? Does it mean we can return either a slice of a or
 a slice of b? I don't believe that that is in any way desirable.


Again, not at all sure why you would do that, but yes it does mean you 
can return a slice of either. Does the current system have a better way 
to deal with this that my suggestion would break?



 The only way that I can think of doing it would be to tag the return
 type, with something like:
 
     sliceof(a) f(const(T)[] a, const(T)[] b);
 
 or
 
     sliceof(b) f(const(T)[] a, const(T)[] b);
 
 That way, the compiler would know which input was allowed to be sliced -
 and /there can be only one!/
 
 More to the point, the compiler would have to /prove/ that the return
 value was, in all cases, a slice of the declared variable, so that it
 could emit a compile error if it wasn't. I don't know much about
 implementing compilers, but it seems to me that this would be a very,
 very hard thing to prove.
 
 The simpler solution, I believe, is to have the function return a pair
 of indeces, and then take the slice at the call site.


Perhaps I have misunderstood the original issue, but the problem as far 
as I know is not that you can't return a slice of a const, nor is the 
issue that you cannot prove which array a returned slice belongs to. The 
issue is that you can only return a const slice of a const, even if it is 
only a temporarily const piece of data due to the function scoping it as 
such. Maybe I'm tired, but how does sliceof() fix this? =/
Mar 24 2008
parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 24/03/2008, Jarrod <qwerty ytre.wq> wrote:

  Maybe I'm tired, but how does sliceof() fix this? =/


It indicates to the compiler exactly what you want to do. Knowing
this, the compiler is then able to syntax-check the function for
errors, on the assumption that the only valid return values are slices
of the specified variable. Attempting to return anything else becomes
a compile-error. In effect, it allows the compiler to infer the
explicit cast which Daniel919 wrote in explicitly. Remember, if the
compiler cannot prove that it is safe, then the compiler must insist
on the presence of an explicit cast. By telling the compiler exactly
what you want to do, you allow the compiler to deduce that such a cast
would be safe.

But returning a range would fix it better! :-) It's more powerful, and
/way/ less confusing.
Mar 24 2008
parent Jarrod <qwerty ytre.wq> writes:
On Mon, 24 Mar 2008 13:35:14 +0000, Janice Caron wrote:


 On 24/03/2008, Jarrod <qwerty ytre.wq> wrote:

  Maybe I'm tired, but how does sliceof() fix this? =/


 
 It indicates to the compiler exactly what you want to do. Knowing this,
 the compiler is then able to syntax-check the function for errors, on
 the assumption that the only valid return values are slices of the
 specified variable. Attempting to return anything else becomes a
 compile-error. In effect, it allows the compiler to infer the explicit
 cast which Daniel919 wrote in explicitly. Remember, if the compiler
 cannot prove that it is safe, then the compiler must insist on the
 presence of an explicit cast. By telling the compiler exactly what you
 want to do, you allow the compiler to deduce that such a cast would be
 safe.
 
 But returning a range would fix it better! :-) It's more powerful, and
 /way/ less confusing.


I would say it's less powerful. 
It only pertains to returning slices of arrays. What about returning 
elements from a struct or class ref, or subelements inside arrays? Const 
is transitive to my knowledge, so you couldn't do this normally. sliceof 
won't fix this either. The best you could do in the same vein as your 
solution is invent a keyword that takes an input's storage class and can 
use it for a return value:
storagetype(a) T[] (const T[] a, const T[] b);
But that would probably get problematic for returning nested things..

It's kind of a tough problem.
Mar 24 2008
prev sibling parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
Janice Caron wrote:

 On 24/03/2008, Jarrod <qwerty ytre.wq> wrote:

  I know everyone is throwing in some kind of template or overload idea,
  but mine is a little more simple: Change 'in'. Use 'in' to define a
  function that will take a value and not change it, but make no guarantees
  about the return type


 
 I'm going to have to join forces with Walter on this one. It doesn't
 matter what syntax you use to declare function parameters - any such
 declaration, whatever the syntax, /will break const correctness/.
 
 What if I declare
 ...


Or what if we just say const correctness doesn't really belong in a 
language designed to be as easy to use as Python and Ruby?

As time wears on I find myself not becoming any more enthusiastic about 
const in D...
All I wanted was a simple way to avoid simple mistakes in my code.  Not 
a complicated way to avoid complicated mistakes.  And certainly not a 
complicated way to avoid simple mistakes.

--bb
Mar 24 2008
parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 24/03/2008, Bill Baxter <dnewsgroup billbaxter.com> wrote:

  As time wears on I find myself not becoming any more enthusiastic about
  const in D...
  All I wanted was a simple way to avoid simple mistakes in my code.  Not
  a complicated way to avoid complicated mistakes.  And certainly not a
  complicated way to avoid simple mistakes.


The feature which is being requested, and rejected, is *not available
in C, or C++, or any other language of which I am aware*.

Therefore, the fact that it's also not available in D is hardly a criticism!
Mar 24 2008
next sibling parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
Janice Caron wrote:

 On 24/03/2008, Bill Baxter <dnewsgroup billbaxter.com> wrote:

  As time wears on I find myself not becoming any more enthusiastic about
  const in D...
  All I wanted was a simple way to avoid simple mistakes in my code.  Not
  a complicated way to avoid complicated mistakes.  And certainly not a
  complicated way to avoid simple mistakes.


 
 The feature which is being requested, and rejected, is *not available
 in C, or C++, or any other language of which I am aware*.
 
 Therefore, the fact that it's also not available in D is hardly a criticism!


I was hoping for something simpler and less redundant than C++'s const. 
  Doesn't seem like it's going to be the case.  The more I think I about 
it the more I think I don't really care about transitivity and rock 
solid assurances.  I just want a simple way to prevent shooting myself 
in the foot in really dumb obvious ways.

...And this:

#include <stdio.h>

char* substring(int begin, const char* string)
{
     //string[0] = 'b';
     return string+begin;
}


int main(int argc, char* argv[])
{
     char * foo = "hi there\n";
     char * bar = substring(3, foo);

     printf(bar);
}

compiles in C with the line commented; doesn't compile if you uncomment 
it.  (Though gcc does give a warning in the former case.)

--bb
Mar 24 2008
next sibling parent "Janice Caron" <caron800 googlemail.com> writes:
On 24/03/2008, Bill Baxter <dnewsgroup billbaxter.com> wrote:

  ...And this:

  char* substring(int begin, const char* string)
  {
      return string+begin;
  }

  char * foo = "hi there\n";
  char * bar = substring(3, foo);

  compiles in C


I'd call that a bug in your C compiler, because if that compiles, so
will the following:

    void f(char const * s)
    {
        char * t = substring(0, f);
        t[0] = 'x';
    }
Mar 24 2008
prev sibling parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 24/03/2008, Janice Caron <caron800 googlemail.com> wrote:

  I'd call that a bug in your C compiler, because if that compiles, so
  will the following:


Ah rats! Ignore the typo. I meant:

    void f(char const * s)
    {
        char * t = substring(0, s);
        t[0] = 'x';
    }
Mar 24 2008
parent Bill Baxter <dnewsgroup billbaxter.com> writes:
Janice Caron wrote:

 On 24/03/2008, Janice Caron <caron800 googlemail.com> wrote:

  I'd call that a bug in your C compiler, because if that compiles, so
  will the following:


 
 Ah rats! Ignore the typo. I meant:
 
     void f(char const * s)
     {
         char * t = substring(0, s);
         t[0] = 'x';
     }


Probably "my compiler" will let that through.  That's what I mean by not 
caring if the detection of const violations is 100%.  It's still more 
error checking than what Python and Java developers get.  I can 
understand if some folks want to argue that's not enough.  I'm just 
saying I think it would be enough for me.

--bb
Mar 24 2008
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Janice Caron" wrote

 On 24/03/2008, Bill Baxter wrote:

  As time wears on I find myself not becoming any more enthusiastic about
  const in D...
  All I wanted was a simple way to avoid simple mistakes in my code.  Not
  a complicated way to avoid complicated mistakes.  And certainly not a
  complicated way to avoid simple mistakes.


 The feature which is being requested, and rejected, is *not available
 in C, or C++, or any other language of which I am aware*.

 Therefore, the fact that it's also not available in D is hardly a 
 criticism!


I don't know that anyone has said it's rejected.  It's just not accepted yet 
:)  Nobody has proven to anyone that this functionality is incorrect or can 
be implemented using the current const regime.

That it is not available in C or C++, or "C++ has the same problem" is not 
an argument against anything.  D has plenty of things that C++ does not 
have, and all for the better IMO.  This is my attempt to plug what I see is 
a hole in the current const implementation.  With this hole, const becomes 
less useful, and therefore will be used less, which makes it more of a 
burden on the coder instead of a benefit.

-Steve
Mar 24 2008
parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 24/03/2008, Steven Schveighoffer <schveiguy yahoo.com> wrote:

  That it is not available in C or C++, or "C++ has the same problem" is not
  an argument against anything.


Fear not, Steve, I was merely responding to Bill, who said:

  As time wears on I find myself not becoming any more enthusiastic about
  const in D...




In the light of Bill's comment (he specifically said "const in D", not
"const"), it seemed reasonable to mention that other languages can't
do this either.
Mar 24 2008
parent Bill Baxter <dnewsgroup billbaxter.com> writes:
Janice Caron wrote:

 On 24/03/2008, Steven Schveighoffer <schveiguy yahoo.com> wrote:

  That it is not available in C or C++, or "C++ has the same problem" is not
  an argument against anything.


 
 Fear not, Steve, I was merely responding to Bill, who said:

  As time wears on I find myself not becoming any more enthusiastic about
  const in D...




 
 In the light of Bill's comment (he specifically said "const in D", not
 "const"), it seemed reasonable to mention that other languages can't
 do this either.


Yes you are correct.  I was talking specifically about D.  (But I'm not 
happy with C++ const these days either.)

--bb
Mar 24 2008
prev sibling next sibling parent "Janice Caron" <caron800 googlemail.com> writes:
On 21/03/2008, Janice Caron <caron800 googlemail.com> wrote:

 I really wish there was a better way of doing it though. Essentially,
  we want a way to auto-generate all three versions of

     K(T)[] func(T)(K(T)[] buf ) { ... }


Oh, and - as has been noted previously - the object code for all three
versions could be byte-for-byte identical, and therefore the function
only needs to be compiled once. The trick is to /allow/ it to compile,
for each of the three cases.
Mar 21 2008
prev sibling next sibling parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 21/03/2008, Sean Kelly <sean invisibleduck.org> wrote:

 Use overloaded functions.  In theory, I think this should work:

  T[] func(T)( T[] buf ) { ... }
  const T[] func(T)( const T[] buf ) { ... }
  invariant T[] func(T)( invariant T[] buf ) { ... }


I really wish there was a better way of doing it though. Essentially,
we want a way to auto-generate all three versions of

    K(T)[] func(T)(K(T)[] buf ) { ... }

where K() can be used anywhere within scope, and where K(U) expands to
each of U, const(U) and invariant(U) in succession. That would be /so/
much nicer than all that messing about with TransferConst!

Obviously, K is not the best choice of keyword! :-)
Mar 21 2008
parent reply Davidson Corry <davidsoncorry comcasst.net> writes:
Janice Caron wrote:

 On 21/03/2008, Sean Kelly <sean invisibleduck.org> wrote:

 Use overloaded functions.  In theory, I think this should work:

  T[] func(T)( T[] buf ) { ... }
  const T[] func(T)( const T[] buf ) { ... }
  invariant T[] func(T)( invariant T[] buf ) { ... }


 
 I really wish there was a better way of doing it though. Essentially,
 we want a way to auto-generate all three versions of
 
     K(T)[] func(T)(K(T)[] buf ) { ... }
 


If you will permit a newcomer to comment... ( Hi, folks. I'm new to D 
but have been lurking here on and off for a few months. I've used C++ 
for many years...)

It seems to me that you are conflating "the type (including const-ness) 
by which the function will internally refer to its inputs" with "the 
type that the function can return". Those will often be the same and 
usually related, but they don't have to be identical.

Put another way, the function signature makes one of two promises:

(1) you pass me X, you know it's const, I know it's const, and I cannot 
in good conscience return a reference to it or any subcomponent of it 
that is NOT const.

(2) you pass me X, it may or may not be const, I don't really care, all 
I guarantee you is that *I* will treat it as const, but I'm not going to 
impose that restriction on you when I hand it back.

Those are DIFFERENT promises, but I think both might be useful. So we 
need different ways to express them. Inspired by C++
	void foo(A) const;
as the way to say "function foo() won't touch your A object", I suggest

idem(x) foo(const T x, const U y);	// first promise
idem(y) bar(T x const, U y const);	// second promise

The first signature RIGOROUSLY protects the constness of the data 
referred to by x (or any reference derived from it). Even if you pass it 
a non-const T that gets automatically cast to const, it will not return 
a non-const reference to the underlying data.

The second signature, in contrast, DOESN'T make that same guarantee. It 
doesn't cast the data referenced to the first parameter to const. The 
compiler gets a hint that the function is SUPPOSED TO treat the 
underlying data as const, and may throw warnings or errors if it sees 
the function doing things that wouldn't be const-safe (including calls 
to other functions that don't make the necessary guarantees in THEIR 
signatures...). But bar() is not constrained to return a reference to 
const data.

[note:  idem(y) means "the same type and const-ness as parameter y".
(Latin "the same"). I regret that I'm a bit confused about  auto(y)
and whether it would work here. If it would do so unambiguously, 
excellent and certainly don't introduce an unnecessary keyword.]

[and substitute "invariant" above wherever you see "const" and do the 
same analysis.]

[I also can't yet see a clean syntax to do the TransferConst! thing...]

Does this make any sense? Or have I said something silly, or resurrected 
an old and discarded idea, or otherwise taken a pratfall on my maiden sally?

Hi, all! Nice to meet you!
                                         -- Davidson Corry
Mar 25 2008
next sibling parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 26/03/2008, Davidson Corry <davidsoncorry comcasst.net> wrote:

  idem(x) foo(const T x, const U y);      // first promise
  idem(y) bar(T x const, U y const);      // second promise

  Does this make any sense? Or have I said something silly, or resurrected
  an old and discarded idea, or otherwise taken a pratfall on my maiden sally?


It makes sense, and you're spot on. I think the problem is well
understood now. All we're lacking a good syntax to recommend. And it
seems to me that no one's come up with anything better than Walter's

    U bar(const(T) x, return const(U) y)

although actually I still think it should be

    const(U) bar(const(T) x, return const(U) y)

because inside the function body, every return statement is going to
be returning a const(U), not a U.

Your "bar" example doesn't make complete sense to me. If the function
can only return a U (as indicated by your idem(y) - which incidently
is similar to my earlier sliceof(y) suggestion), then why does x also
have the "returnable" parameter declaration syntax? Was that a typo or
am I missing something?

Walter's syntax is not as expressive as we might like, but it does
have the benefit of being simple. By contrast, making a distinction
between const-on-the-left and const-on-the-right is potentially
confusing - and also, no more expressive that Walter's idea, so
there's nothing to be gained by it.

For what it's worth, I have a new idea, largely inspired by Stephen,
which is to allow the keyword inout as a type constructor. inout would
basically make promise 2.

    inout(U) bar(const(T) x, inout(U) y)

I think it's fully expressive, and not too confusing, but on the down
side, it would mean that we'd then have three types of constancy
instead of just two
Mar 26 2008
parent reply Davidson Corry <davidsoncorry comcasst.net> writes:
Janice Caron wrote:

 On 26/03/2008, Davidson Corry <davidsoncorry comcasst.net> wrote:

  idem(x) foo(const T x, const U y);      // first promise
  idem(y) bar(T x const, U y const);      // second promise

  Does this make any sense? Or have I said something silly, or resurrected
  an old and discarded idea, or otherwise taken a pratfall on my maiden sally?


 
 It makes sense, and you're spot on. I think the problem is well
 understood now. All we're lacking a good syntax to recommend. And it
 seems to me that no one's come up with anything better than Walter's
 
     U bar(const(T) x, return const(U) y)
 
 although actually I still think it should be
 
     const(U) bar(const(T) x, return const(U) y)
 
 because inside the function body, every return statement is going to
 be returning a const(U), not a U.
 
 Your "bar" example doesn't make complete sense to me. If the function
 can only return a U (as indicated by your idem(y) - which incidently
 is similar to my earlier sliceof(y) suggestion), then why does x also
 have the "returnable" parameter declaration syntax? Was that a typo or
 am I missing something?
 
 Walter's syntax is not as expressive as we might like, but it does
 have the benefit of being simple. By contrast, making a distinction
 between const-on-the-left and const-on-the-right is potentially
 confusing - and also, no more expressive that Walter's idea, so
 there's nothing to be gained by it.


Thank you. I was trying to suggest a few more things without burdening 
the message with too much extra explanation; in doing so I was unclear. 
I apologise.

There are two things going on here. First, I wanted to show that you 
could choose ANY of the function parameters as the basis for the 
returned type. You're not constrained to use the first parameter, nor 
the "first parameter that has one of these tricky annotations". <grin>

Second, these annotations are not necessarily tied to returns. They can 
be applied to any parameter(s) in the function, and each one makes a 
separate "promise".

The "promise" is actually two parts:
"I promise not to mutate the data underlying the X that you passed me, 
whether you think it's mutable or not. And IF I RETURN SOMETHING TO YOU 
BASED ON THOSE DATA, I won't constrain you to treat it as const even 
though I did. (If the return data are not referenced through X, the 
second part of the promise is irrelevant.)"

Imagine a signature like
	void foo(T t const, U u invariant, const V v)

Since the "return" is void, obviously NONE of the input parameters are 
participating in it. Nevertheless, we might have good reason to make the 
promises "I will treat your t as if it were const, your u as if it were 
invariant, and if you try to pass me a non-const v, I will cast it to 
const". In THIS example, there's not much difference between
"const V v" and "V v const" for the third parameter -- but if the return 
type were derived from v, the compiler would reject an attempt to return 
a NON-const reference to it. (Whereas the compiler would ALLOW non-const 
return types based on t or u.)

That is, the annotations participate in matching signatures for 
overloaded function names. As a secondary issue, they MAY also support 
automatic inference of a return type as well, but that's not part of 
signature matching. (In C++, you can't overload on return type. Correct 
me if I'm wrong, but I believe that this is true in D as well.)

Since what I'm talking about does not (necessarily) affect return, the 
"return (type) x" syntax is inappropriate, IMHO. Particularly since you 
could only mark ONE parameter as "return (Type) x", whereas here you 
could mark any or all parameters as "as-if-const".

I agree with you that 
const-on-the-left-const-on-the-right-stand-up-sit-down-fight-fight-fight 
<grin> is potentially confusing. I just haven't thought of a better way 
to express the notion "I will treat this parameter as if it were const, 
but I won't require that YOU promise to treat it the same way".

I think of "const on the left" as being part of the type: (const T)
whereas "const on the right" is an additional assurance from the 
function, part of its contract if you will: (plain T, possibly mutable, 
but I will not change it).

Does that make things clearer?

-- Dai
Mar 26 2008
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Davidson Corry" wrote

 Does that make things clearer?


I think that we're all arguing the same point here :)

All of our suggestions accomplish the same thing, but I think yours is a tad 
less expressive than Janices and mine.

For example, if I want a mutable array of const characters, I use:

const(char)[]

I can pass this into a function no problem.

However, there is no way to put a trailing const in your scheme, unless you 
did something like:

(char const)[]

But I think this is really confusing to most people.  It's one of the things 
I didn't like about C++'s const, I never knew where to put the const because 
I could never remember which was which :)

The statement:

const(char)

is clear, it means whatever inside the parentheses is const.

Regarding using 'const' as the keyword for 'const during the function, but I 
can return the same type as you gave me', I think in order to use the 
parentheses scheme, we'd have to use another keyword.  But I think that the 
scheme should be just as expressive as const is now, which I don't see 
happening with the trailing const notation.

-Steve
Mar 26 2008
parent reply Davidson Corry <davidsoncorry comcasst.net> writes:
Steven Schveighoffer wrote:

 I think that we're all arguing the same point here :)


Of course -- that's half the fun! <grin>


 All of our suggestions accomplish the same thing, but I think yours is a tad 
 less expressive than Janice's and mine.
            [ some deleted ]
 Regarding using 'const' as the keyword for 'const during the function, but I 
 can return the same type as you gave me', I think in order to use the 
 parentheses scheme, we'd have to use another keyword.  But I think that the 
 scheme should be just as expressive as const is now, which I don't see 
 happening with the trailing const notation.


Point well taken. I think language designers -- and wannabes like me -- 
will stand on their heads to avoid adding a new keyword, both to avoid 
pilfering an identifier from the namespace, and because, when it turns 
out to have been a mistake, removing a keyword from a language is 
terribly difficult. But if standing the syntax on its head to avoid a 
new keyword turns out to restrict expressiveness and clarity, it may be 
justified to add one.

A keyword approach that occurs to me is
	asif const(T)
or perhaps expressed even better
	locally const(T)

But I dislike this because the new keyword can ONLY be used as an 
annotation to a function parameter. I can't think of a place in regular 
code where it would be useful. And it seems wrong to "waste" a keyword 
on so specialized a purpose.

You can always express the "locally const" notion in your own code by 
making a copy of a (possibly non-const) parameter reference to a local 
reference that IS const, and using that in the body of your function, 
then return a value based on the original parameter reference. And the 
compiler will catch it when you mis-use the "consticated" reference. 
What this does NOT do is to PUBLISH that "promise" to callers of your 
function so that they can rely on it and possibly optimize against it.

But it's a legitimate question to ask, is this facility useful enough to 
be worth bloating the syntax of the language? And if so, is this 
necessarily the best way to publish that information?

I noted in an earlier post that "locally const" for a parameter becomes 
in essence a part of the function's contract. Perhaps this annotation 
could be expressed, not in the function's parameter list, but in an in{} 
or out{} block instead. That would also allow it to be optimized away in 
a release build, perhaps.

In Eiffel, the contract conditions are considered as much a part of a 
method's "signature" as the parameter list -- see the short/flat views 
of a class. I don't know if the same attitude prevails in the D world. I 
know that few users of C-derived contracting languages (iContract and 

concept.

Likewise I don't know of any Eiffel compilers that inspect the pre- 
and/or postconditions and optimize callers against them. But I don't see 
any theoretical reason why they couldn't...

-- Dai
Mar 26 2008
parent Davidson Corry <davidsoncorry comcasst.net> writes:
Davidson Corry wrote:

     asif const(T)
 or perhaps expressed even better
     locally const(T)


Gaah. I hate it when I say something, and then learn that I didn't know 
what I was talking about. (By reading the Web page "Here a const, there 
a const".)

I told you that I come from many years of C++. Well, I have been guilty 
of C++-think. 'const' in D doesn't mean what it means in C/C++. What 
I've been describing as "locally const" IS what D means by 'const'. What 
I've been thinking of as "(globally) const", D has a better name for, 
which is 'invariant'. And all of what I was asking for, D already gives me.

Which leaves what you folks were already talking about, to which I 
contributed only confusion. My apologies.

<sigh> RTFM. Then re-RTFM. Then THINK about what you R in TFM. Then 
UNDERSTAND TFM.

THEN speak. <sigh>

-- Dai
Mar 27 2008
prev sibling parent reply "Janice Caron" <caron800 googlemail.com> writes:
I think I've got it.

Imagine, if you will, a third flavor of constancy, called inout(T),
with the following rules:

(1) T will NOT implicitly cast to inout(T)
(2) const(T) will NOT implicitly cast to inout(T)
(3) invariant(T) will NOT implicitly cast to inout(T)

and
(4) inout(T) will implicitly cast to const(T)

When compiling a function body, the compiler will act as though
inout(T) means "I promise that I will not modify T, but someone else
might". (i.e. the same as const(T)).

When /calling/ a function which contains at least one inout(T) in its
signature, then the call shall be compiled /as if/ there had been
three separate declarations - one for mutable, one for const, and one
for invariant.This is probably better explained with an example - so
let's start with strstr. Here's the function definition:

    inout(char)[] strstr(inout(char)[] s, const(char)[] pattern)
    {
        int n = s.find(pattern);
        return n == -1 ? null : s[n..$];
    }

The compiler generates exactly one function from this body - this is
not a template. It compiles, because rules (1) to (4) above are
adhered to, and because, within the function body, the inout()
contract is adhered to.

Now let's see how things look from the call site:

    string s;
    string pattern;
    auto r = strstr(s,pattern);

In order to resolve this, the compiler looks at the function
signature, and sees:

    inout(char)[] strstr(inout(char)[] s, const(char)[] pattern)

So (and here's the clever part) it substitutes "inout" with all three
possibilities, and then tries to find the best match from among the
three. In other words, it acts /as if/ there had been three separate
function declarations:

    char[] strstr(char[] s, const(char)[] pattern);
    const(char)[] strstr(const(char)[] s, const(char)[] pattern);
    invariant(char)[] strstr(invariant(char)[] s, const(char)[] pattern);

It tries to find the best match from among those three declarations,
using the normal matching rules. It finds that the best match is:

    invariant(char)[] strstr(invariant(char)[] s, const(char)[] pattern);

and so that's the one it calls. (Even though there's only one
function, the caller behaves as though there were three). And since r
was declared with auto, r gets a type of invariant(char)[].

Let's try another example:

    inout(T) min(T)(inout(T) x, inout(T) y)
    {
    	return x < y ? x : y;
    }

Again, we use rules (1) to (4) to compile this into a function, this
time templatised on T. Now let's have a more interesting call:

    // C is a class
    C x;
    const(C) y;
    auto z = min(x,y);

What type is z?

The procedure outlined above makes this very easy to figure out.
Again, the compiler acts /as if/ it has seen three separate
definitions, these being:

    T min(T)(T x, T y);
    const(T) min(T)(const(T) x, const(T) y);
    invariant(T) min(T)(invariant(T) x, invariant(T) y);

It finds the best match, using the normal matching rules. It finds
that the best match is with:

    const(T) min(T)(const(T) x, const(T) y);

and so that's the one it calls. The template paramter T matches the
class C, and so the return type must be const(C). Therefore z has a
type of const(C).

So far as I can, this solves all problems, and just works.

It does, however, require one additional caveat. For member functions,
it must be possible to declare

    class C
    {
        inout f(...);
    }

which, again, generates exactly one function body. By similar rules,
at the point of the function call, the compiler must act as though it
had seen three separate declarations:

    class C
    {
        f(...);
        const f(...);
        invariant f(...);
    }

and match the right one, using the normal rules.

I honestly think this might be problem solved. But please, please,
please chime in if I've got anything wrong.
Mar 26 2008
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Janice Caron" wrote

I think I've got it.
 ...


Janice, if you look closely, this follows EXACTLY my proposition in 
"Proposal for scoped const contracts", just substitute foo for inout :)  But 
I'm glad we came to the same conclusion...

BTW, I think I like 'inout' better than 'in' and 'out' separately, and inout 
is DEFINITELY fair game as ref has completely superceded it :)

Your way of explaining how the resolution of inout works as 3 separate 
function bindings is good for understanding purposes.  I'd amend my rules to 
be similar:

1) if all inouts are all homogeneous (i.e. all are mutable, all are const, 
or all are invariant), then the version of the function with that same 
constancy is called.
2) otherwise, the const version is called.

Other than that, this is exactly what I proposed and what I think should be 
implemented.

-Steve
Mar 26 2008
next sibling parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 26/03/2008, Steven Schveighoffer <schveiguy yahoo.com> wrote:

 "Janice Caron" wrote


I think I've got it.


  Janice, if you look closely, this follows EXACTLY my proposition in
  "Proposal for scoped const contracts", just substitute foo for inout :)  But
  I'm glad we came to the same conclusion...


You're right!

I guess, as with all these things, it's not so much the syntax itself
that matters, as grokking what it all means. Sorry it took me so long.



  Other than that, this is exactly what I proposed and what I think should be
  implemented.


The question is, is our scheme simple enough. It won't fly if it
complicates the const system too much.

That said, there's not a great deal of difference in appearance,
between our scheme, and Walter's. For example, by omitting the
brackets, compare:

    inout(T) f(inout T x);

with

    T f(return T x);

They don't look that dissimilar. I think it might fly. Walter's scheme
isn't sufficiently expressive, and the declared return type is
counterintuitive. I think our scheme is better (though if Walter
wanted to call it return(T) instead of inout(T), I guess I could live
with that - as long as we get to keep everything else).

Another advantage of our scheme is, we get to use temporary values!

    alias inout(char)[] iostring;

    iostring f(iostring s)
    {
        iostring tmp = s[1..$];
        return tmp;
    }



So, what problem would you like to tackle next? :-)
(Joking)
Mar 26 2008
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Janice Caron" wrote

 The question is, is our scheme simple enough. It won't fly if it
 complicates the const system too much.


I agree.  But I think it actually makes the scheme more flexible in a way 
that makes it simpler.  For example, instead of 3 functions for constancy, 
there can be just one, making the code simpler, and the binary smaller even. 
Writing:

inout(T) f(inout(T));

is just as easy as:

const(T) f(const(T));

So from a usability standpoint, it is just as simple as const or invariant. 
And as you say, it's the same as any other const type, so you can declare 
aliases and temporary variables just as easily.

The complication comes with "Oh God, I now have 3 const types, which one do 
I choose?!"  But I think this is answered in one paragraph (inout is only 
available as a function parameter, so that is the only time where this 
becomes an issue):

If you wish to have a function parameter that supports mutable, const, and 
invariant at the call site, but is not mutable inside the function, and 
which dictates the return value constancy based on those arguments, use 
inout.  If you want a parameter to be required to be const or invariant, 
declare it that way instead.

Really, the most complicated part of this is the implementation of it in the 
compiler.  I think using it will be really simple and intuitive.

-Steve
Mar 26 2008
parent "Janice Caron" <caron800 googlemail.com> writes:
On 26/03/2008, Steven Schveighoffer <schveiguy yahoo.com> wrote:

  The complication comes with "Oh God, I now have 3 const types, which one do
  I choose?!"  But I think this is answered in one paragraph (inout is only
  available as a function parameter, so that is the only time where this
  becomes an issue):


Well, technically speaking, you /could/ declare a variable with inout
constancy - even at global scope. (Forbidding it would be more
complicated than allowing it).

    inout T x;

However, doing so would be pointless, since nothing implicitly casts
to inout, so you couldn't even assign it (except from another inout)
without an explicit cast!

    inout T x = anything; /*error*/

(I'm just being pedantic for the hell of it now! It's just so I can be
sure I have a good grasp of the basics).




  Really, the most complicated part of this is the implementation of it in the
  compiler.


Oddly, I think that will be relatively easy, since the rules are plain
and straightforward. By and large, inout(T) behaves just like
const(T), except that it is a distinct type and there are rules about
what will implicitly cast into what. The triple-lookup at call-time
can't be /that/ hard, I wouldn't have thought.

I think we're onto a winner here.
Mar 26 2008
prev sibling parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 26/03/2008, Steven Schveighoffer <schveiguy yahoo.com> wrote:

 "Janice Caron" wrote

  Your way of explaining how the resolution of inout works as 3 separate
  function bindings is good for understanding purposes.  I'd amend my rules to
  be similar:


I've just figured out an /even easier/ way of describing this. I've
got it down to just two rules now.

Rule 1: inout(T) is a typedef for const(T)

This automatically implies that inout(T) is a distinct type from
const(T), that nothing can implicitly cast to inout(T), and that
inout(T) will implicitly cast to const(T).

Rule 2: When a function declaration is compiled, in which at least one
of the function parameters (including this) contains an inout(T) type,
three additional functions are generated by the compiler, which apply
casts to parameters and return type as appropriate.

e.g. If you write

    inout(T) f(inout(T) x, inout(T) y)
    {
        whatever;
    }

then the compiler adds

    T f(T x, T y)
    {
        return cast(T) f(cast(inout)T)x, cast(inout)T)y);
    }

    const(T) f(const(T) x, const(T) y)
    {
        return cast(const(T)) f(cast(inout)T)x, cast(inout)T)y);
    }

    invariant(T) f(invariant(T) x, invariant(T) y)
    {
        return cast(invariant(T)) f(cast(inout)T)x, cast(inout)T)y);
    }

Those are the /only/ rules you need, because the existing function
signature matching algorithm will always pick the right function, and
the extra three functions will all be inlined so there's no code
bloat.

I like keeping the explanations simple! :-)
Mar 27 2008
parent Davidson Corry <davidsoncorry comcasst.net> writes:
Janice Caron wrote:

 On 26/03/2008, Steven Schveighoffer <schveiguy yahoo.com> wrote:

 "Janice Caron" wrote

  Your way of explaining how the resolution of inout works as 3 separate
  function bindings is good for understanding purposes.  I'd amend my rules to
  be similar:


 
 I've just figured out an /even easier/ way of describing this. I've
 got it down to just two rules now.
 	[elided for brevity, go read Janice's original message]
 I like keeping the explanations simple! :-)


That is lovely, Janice. Nice work!

-- Dai
Mar 27 2008
prev sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
Janice Caron wrote:

 I think I've got it.
 


...

 
 I honestly think this might be problem solved. But please, please,
 please chime in if I've got anything wrong.


Well, *DUH*!
Yes, your proposal looks safe and sound, and you managed to follow the 
whole logical progression to reach this conclusion. But all of it was a 
trail that had been walked before! Let me explain.

A mechanism exactly like yours has already been described in an academic 
paper called Javari, about adding a transitive const system to Java 
(http://groups.csail.mit.edu/pag/pubs/tschantz-refimmut-mengthesis.pdf). 
The mechanism there is called "romaybe" (short for read-only maybe), and 
is pretty much the same as yours. Quotes:
  "romaybe, a syntactic convenience that reduces code duplication;"
  "Javari provides the keyword romaybe to declare that a method should 
be templated over the mutability of one or more formal parameters—that 
is, the method should have different signatures in the mutable and 
read-only types."


Nearly a year ago, someone posted the link to the Javari paper. A few 
months later, when the const discussion was starting up and gaining 
prominence, I mentioned, quote: "I would say reading it [the Javari 
paper] should likely be  mandatory for anyone who wants to venture 
thinking and posting how the const system should work." 
(http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D.learn&article_id=3880)
That's because the paper and the proposal genuinely look to be very well 
written and thought out, well detailed, and having all the issues 
considered. (unlike this other proposal: 
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4211070).

Hence my comments like, "I wish I had an euro every time someone posted 
about something that had already been discussed and concluded in Javari."

Now, you weren't yet in the D NG when Javari was mentioned, so I'm not 
disparaging you about all of this. In fact, it's a good sign that you 
reached the same conclusion as the one mentioned in the paper.
What is really annoying me, is that Walter (and Andrei), the masters 
behind D's language design, seem to be utterly ignoring such a good 
repository of knowledge and technical wisdom. They seem to be designing 
const with more of a gun-ho attitude, without carefully considering all 
aspects. That's why we had like 3 or 4 versions of const so far, and 
frankly, unlike Walter, I don't think the current system has "mostly 
been worked out". The 'inout'/'romaybe' is just one example of a kink, 
but are many other aspects at fault. (more on that on another thread).


(BTW: the Javari paper also mentions logical const, and offers a 
proposal very similar to your's 'unpaintable')

-- 
Bruno Medeiros - Software Developer, MSc. in CS/E graduate
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Apr 27 2008
prev sibling parent "Janice Caron" <caron800 googlemail.com> writes:
We've had this discussion before! :-)

Google "TransferConst!"
Mar 21 2008
prev sibling parent reply Daniel919 <Daniel919 web.de> writes:
 How do you declare a function that takes an array, is not allowed to change 
 the array, but returns a slice into the argument array, and the return type 
 matches the argument type.


What about:

T slice(T)(const T s, int start, int end)
{
	return cast(T) s[start .. end];
}

void main()
{
	char[] s1 = "0123456789".dup;
	const(char)[] s2 = "0123456789";
	const(char[]) s2b = "0123456789";
	invariant(char)[] s3 = "0123456789";
	invariant(char[]) s3b = "0123456789";
	assert (is(typeof(slice(s1,1,4)) == typeof(s1)));
	assert (is(typeof(slice(s2,1,4)) == typeof(s2)));
	assert (is(typeof(slice(s2b,1,4)) == typeof(s2b)));
	assert (is(typeof(slice(s3,1,4)) == typeof(s3)));
	assert (is(typeof(slice(s3b,1,4)) == typeof(s3b)));
}
Mar 24 2008
parent reply "Janice Caron" <caron800 googlemail.com> writes:
On 24/03/2008, Daniel919 <Daniel919 web.de> wrote:

 How do you declare a function that takes an array, is not allowed to change


  > the array, but returns a slice into the argument array, and the return type
  > matches the argument type.

         return cast(T) s[start .. end];


That's fine, but you've got an explicit cast in there. Walter said it
"cannot be implemented without casting away const at some point", so
with that cast, you've just kinda demonstrated that he's right.

The explicit cast basically says "I, the programmer, know that this is
safe, so compiler, please ignore your usual rules". And that's fine,
because in this case, it /is/ safe.

The problem is to do it without the explicit cast. The problem is how
to let the compiler prove that it's safe. And that's why, if you don't
want an explicit cast in there, I believe that returning indeces is
the best way to go.
Mar 24 2008
parent Daniel919 <Daniel919 web.de> writes:
Janice Caron wrote:

 The problem is to do it without the explicit cast. The problem is how
 to let the compiler prove that it's safe. And that's why, if you don't
 want an explicit cast in there, I believe that returning indeces is
 the best way to go.


I see, you propose a "Range Type".
And indeed, this looks like a nice solution for this problem.
Mar 24 2008