www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Generic const - a non-functional view

reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
Before I begin, I want to say that a lot of this has come from reading other 
ideas that other people have had.  I know for instance that Janice has 
proposed something along these same lines using templates.  I think this 
proposal has not been posted quite exactly the same by anyone else, and the 
underlying themes are different than other proposals I or others have had 
regarding const and logical const.  In any case, credit should go to all 
those ideas posted that led me to this proposal.

D2's const system is centered around functional programming.  Without it, 
the grand view of Walter and Andrei to have a compiler that optimizes for 
multi-core systems with little effort from developers cannot be realized. 
However, there is another reason to have const, which Walter at least 
acknowledges.  That reason is for specifying contracts for functions.  That 
goal is partially realized, but since the const system is more focused on 
FP, it does not completely provide ways to specify all possible functionally 
sound constructs.  This post is aimed at outlining a way that D2's const 
system could be extended to provide const specification for both pure 
functions and function contracts, in as easy to use fashion as possible.

We begin by thinking about defining const.  Const is an attribute applied to 
a type.  The attribute doesn't change the data that is described by the 
type, it only changes the type, and rules for accessing the data.  We have 
four rules for const:

1. A variable that is typed as const or invariant cannot be changed.

2. A variable that is typed as const or invariant that has references to 
data cannot change that data through the references. (const is transitive)

3. A variable that is typed as invariant is guaranteed that all data that 
can be referenced through the variable will not change by any other 
references to the same data.

4. One can implicitly cast a data type to the same data type with different 
constancy if the newly modified type provides at least all the same 
protections to the same data that the original type provided.

In the current D2 scheme, we have the following two rules which implement 
rule 4:
A. mutable or invariant data can implicitly be casted to const.
B. No other implicit casts are possible.

But let's think about how constancy is applied to an aggregate data type. 
The current granularity of applying const is to an entire aggregate.  One 
cannot selectively apply const to pieces of an aggregate.  However, if one 
could do so, one does not violate any of the rules.  Here is an example, 
keep in mind that X and X' are meant to be the same data type but with a 
different constancy applied:

class X
{
   int m;
   int m2;
}

class X'
{
   int m;
   const int m2;
}

Since X and X' are the same data type, but with different constancy, one can 
implicitly cast a variable of type X to a variable of type X' without 
violating rules 1-4.

So the question now becomes, how does one go about specifying a syntax for 
providing this functionality such that one has as much expressiveness as 
possible without violating the rules, and without making the syntax or 
specification cumbersome?

First, we specify a way to tag members of an aggregate such that they can be 
referred to in groups.  This one piece of grammar is the only crufty part of 
the solution, but could be easily alleviated by introducing a new keyword. 
In any case, using existing keywords, you create a 'tag group', which is 
scoped like any other symbol.  Using existing syntax, this looks like:

const alias A;

If this appears in a module or class, or even function, it only exists 
within that scope, just like a variable symbol.  How does one use such a 
tag?

class X
{
   int m;
   A int m2;
}

Now, m2 is in the tag group identified as A.  Now we need a syntax in order 
to affect just the constancy of the A tag group of X.  If we think of 
casting only the A group to const as parameterizing const's scope, it's 
similar to a parameter to a template.  So let's use the same syntax:

const!(A)(X) x;

So const!(A) means 'Apply const only to the group A', and X is the type to 
apply this rule to.  So now, x.m is mutable, x.m2 is const.  Now, we can 
cast a plain X to a const!(A)(X) implicitly, because this does not break any 
rules 1-4 above.  However, we cannot implicitly cast a const!(A)(X) back to 
an X because it would break rule 4.

Let us also define a special compiler-builtin tag group called __const_ALL. 
This tag group is special in that all members of any aggregate, regardless 
of the tag that is applied to them, also are tagged with __const_ALL.  This 
special group is used when applying const or invariant without 
parameterization, this group is implied.  i.e., const(X) is equivalent to 
const!(__const_ALL)(X), and invariant(X) is equivalent to 
invariant!(__const_ALL)(X).  Using class X above, a const!(__const_ALL)(X) 
has all const members, even though m2 is tagged with A.

This allows us to implement a logical const scheme, and also allow data to 
be transitively invariant in order to be used in pure functions.  We get the 
best of all worlds.

However, there are some more problems to solve.

First, it should be explicitly stated that a parameterized const is just as 
powerful as an unparameterized const with regards to pointers and arrays. 
i.e., you can parameterize only what a pointer points to, or only the 
elements of an array:

const!(A)(X) *x;
const!(A)(X)[] x2;

x and x2 are mutable, but the data pointed to is typed as const!(A)(X).

It is very unlikely that tag groups will have short names like 'A', they 
will be something more descriptive.  Utilizing these tag group names 
everywhere would be very cumbersome:

const alias logic_const;

const!(logic_const)(X) x2 = x;

So we allow aliases for these parameterized type modifiers:

const alias logic_const;

alias const!(logic_const) lconst;

lconst(X) x2 = x;

This syntax is more readable, easier to use and understand.

The next problem is being able to tag members as NOT being affected by a 
parameterized const statement.  For example, C++, through the mutable 
keyword, allows you to define members that are not casted to const when the 
aggregate is casted to const.  This is like a 'reverse' tag group.  Without 
something similar, one must tag all the members that SHOULD be casted to 
const when logical const is applied.  So we borrow the bitwise not operator 
for this purpose:

const alias mutable;

alias const!(~mutable) lconst;

class X
{
   mutable int m;
   int m2;
}

auto x = new X;
lconst(X) lx = x; // ok, not weakening constancy
lx.m = 2; // ok
lx.m2 = 3; // error, m2 was not tagged by mutable, so was cast to const

Next, we try to reduce the code for parameterizing const on multiple 
parameters.  For example, if you have two tag groups A and B in an 
aggregate, and you want to cast both of them to const, it would be:

const!(A)(const!(B)(X)) x;

Let's allow this kind of multiple specification to be shorthanded as:

const!(A, B)(X);

Next we tackle the chaining problem.  Let's say you have a linked list, and 
you want to specify a sorting function for that linked list.  The sorting 
function will not change any values pointed to by that linked list, but can 
change the order of the nodes in the linked list.  How does one specify this 
contract?  Using the rules we have so far:

const alias linkdata;

class Link(T)
{
    Link!(T) next;
    linkdata T data;
}

If we pass the head of the link list as const!(linkdata)(Link!(T)) to the 
function, what happens?  Upon traversing to the next link, the function is 
able to modify the data in the next link.  We need a way to specify a 
recursive linkdata constancy.  In order to do this, we need a second tag 
group, and recursive definition:

const alias linkdata;
const alias link;

class Link(T)
{
   link Link!(T) next;
   linkdata T data;
}

alias const!(linkdata)(dataconst(link)) dataconst;

Note that we use dataconst inside the definition for dataconst, just like we 
could have a pointer to a type within the definition of the type.  Note also 
that we need to define this as an alias so we can name the parameterized 
const definition in order to use it within the definition.  Therefore, you 
cannot specify this type of constancy without an alias statement.

dataconst(Link!(T)) x;

Now, x.next is of type dataconst(Link!(T)), and x.data is of type const(T).

The one problem we cannot solve is head-const.  For example, if you wanted 
to allow someone to traverse a link list and not change the structure of the 
links, but allow them to change the data, it is not possible without 
breaking rule 2, as you cannot have a constant pointer that points to 
mutable data.  However, this kind of thing could be solved with generic 
tagging (not just for const) combined with function overloading, or by using 
interfaces that enforce these requirements.

There is one final bonus problem that can be solved using these ideas.  If 
you think of the data that a class reference points to being of type 
<classname>_data, then you can think of a class reference as a pointer:

class X {};

alias X_data * X; // implicitly defined

Let's define another builtin tag group called __class_data_ref;  We can now 
redefine a class reference to X to be:

alias __class_data_ref(X_data) * X;

Now, we can define an alias to make x1 tail-const:

alias const!(__class_data_ref) tconst;

tconst(X) x1;

Now, x1 is rebindable, but the data x1 points to is const.

So in conclusion, I think this extension has the following benefits:

1. The existing ideas for pure functions requiring invariant are not broken
2. Logical const is possible for people who want const-contract 
specification
3. Logical const can be recursive, allowing for contract specifications that 
are not available in any other language AFAIK.
4. The syntax is powerful enough to allow specification of any const 
casting, but easy enough to be usable and readable.

I understand that this proposal might be difficult to implement in the 
compiler, and it might just be not worth the effort.  There are a lot of 
problems that can be solved by not using const or using interfaces, instead 
of these constructs.  However, this proposal represents what I think is the 
most lenient and flexible proposal that is also const-correct from those I 
have read, and allows for almost any const contract specification (excluding 
head-const).  Whether it gets implemented or not, it was interesting to 
think about how to solve all these problems, and I enjoyed the challenge.

I would really like to hear people's opinions, especially Walter's.

-Steve
Jun 24 2008
next sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
This is a hard problem. But if there's anything that I've learned from 
this whole const thing, it's that const is a hard concept for people to 
get. In order for a const system to be usable, it has to be very simple.

With functional languages, everything is const, so it is trivially 
understandable. Perl has invariant strings, but they are implicitly 
invariant and so nobody notices it, they just work.

C++ manages to hide the complexity, at the cost of const being ineffective.

D has a minimal system that is effective - const and invariant. The fact 
that it still seems to cause such misunderstanding is enough evidence 
for me to be very, very wary of adding more complexity to it.
Jun 24 2008
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Walter Bright" wrote

 D has a minimal system that is effective - const and invariant. The fact 
 that it still seems to cause such misunderstanding is enough evidence for 
 me to be very, very wary of adding more complexity to it.


Effective only for a feature that doesn't exist yet (pure functions).

From my experience, the D const system as it stands now does not cause such 
misunderstanding.  Everything is quite clear.  What seems to cause some 
misunderstanding:


complaint on NG for newbies trying out D2)
- The concept of tail-const pointers and arrays
- Why you cannot have tail-const class references
- Why you cannot have logical const structures.

Explaining const and invariant by itself is easy.  And understandable.  It's 
the usages of them which are confusing.

But I look at this extension as a sort of 'meta' const.  It's like templates 
and mixins.  Some people (not me!) can do magic with these that would not be 
possible if templates weren't so wonderfully complex.  But to use a template 
that someone wrote isn't even close to as difficult.  For example, the 
conversion template to(T).  To use it I just do:

auto str = to!(string)(5);

and I don't have to know how or why it works.

Similarly, if a library implements const groups such as lconst like:

const alias mutable;
alias const!(~mutable) lconst;
alias invariant!(~mutable) linvariant;

One does not have to understand exactly what they mean.  All they have to 
know is 'declare a member mutable if you want it to be mutable when the 
class is lconst or linvariant.'  This kind of thing is easy to document and 
explain.  One does not HAVE to know all the details of how these magic 
generic const structures work, all they have to know is how to use them.  In 
fact, they do not need to know much more than what they already must learn 
for the current const.  Except that the last two points of confusion I 
identified above are removed.

In any case, like I said, if you find that it would be difficult to 
implement in the compiler, then so be it.  But assuming that you can't add a 
feature because it would complicate the language, in light of the fact that 
you already complicated the language too much, seems to be a contradictory 
argument.  We already have a complicated language by your measure.  Why not 
make it a complete language?

-Steve
Jun 24 2008
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Steven Schveighoffer wrote:

 "Walter Bright" wrote

 D has a minimal system that is effective - const and invariant. The fact 
 that it still seems to cause such misunderstanding is enough evidence for 
 me to be very, very wary of adding more complexity to it.


 
 Effective only for a feature that doesn't exist yet (pure functions).
 
 From my experience, the D const system as it stands now does not cause such 
 misunderstanding.  Everything is quite clear.  What seems to cause some 
 misunderstanding:
 

 complaint on NG for newbies trying out D2)
 - The concept of tail-const pointers and arrays
 - Why you cannot have tail-const class references
 - Why you cannot have logical const structures.
 
 Explaining const and invariant by itself is easy.  And understandable.  It's 
 the usages of them which are confusing.


If one doesn't understand the usages (and the consequences of) const, 
then I don't agree that one understands it.



 But I look at this extension as a sort of 'meta' const.  It's like templates 
 and mixins.  Some people (not me!) can do magic with these that would not be 
 possible if templates weren't so wonderfully complex.  But to use a template 
 that someone wrote isn't even close to as difficult.  For example, the 
 conversion template to(T).  To use it I just do:
 
 auto str = to!(string)(5);
 
 and I don't have to know how or why it works.


I don't agree that it's ok for library features to be complicated 
because only advanced users will write libraries. For one thing, it 
impedes writing libraries (and we all know that proliferation of 
libraries is essential for D's success) and it presumes that application 
code is somehow at a level that doesn't need those features.

For example, it's really hard to write reusable libraries in C++, and 
look at the consequences - few reusable libraries exist compared with 
other languages.



 Similarly, if a library implements const groups such as lconst like:
 
 const alias mutable;
 alias const!(~mutable) lconst;
 alias invariant!(~mutable) linvariant;
 
 One does not have to understand exactly what they mean.


As an engineer, I prefer to understand my tools completely :-)
If they aren't understood, and there's an error message, one has no idea 
what to do.


 All they have to 
 know is 'declare a member mutable if you want it to be mutable when the 
 class is lconst or linvariant.'  This kind of thing is easy to document and 
 explain.  One does not HAVE to know all the details of how these magic 
 generic const structures work, all they have to know is how to use them.  In 
 fact, they do not need to know much more than what they already must learn 
 for the current const.  Except that the last two points of confusion I 
 identified above are removed.
 
 In any case, like I said, if you find that it would be difficult to 
 implement in the compiler, then so be it.  But assuming that you can't add a 
 feature because it would complicate the language, in light of the fact that 
 you already complicated the language too much, seems to be a contradictory 
 argument.  We already have a complicated language by your measure.  Why not 
 make it a complete language?


I worry about the complexity all the time. The idea is not to make a 
complicated language, but to find a way to make a simpler language that 
can do everything we need done. Complexity should only come with a 
benefit that outweighs the cost of it.
Jun 24 2008
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Walter Bright" wrote

 Steven Schveighoffer wrote:

 "Walter Bright" wrote

 D has a minimal system that is effective - const and invariant. The fact 
 that it still seems to cause such misunderstanding is enough evidence 
 for me to be very, very wary of adding more complexity to it.


 Effective only for a feature that doesn't exist yet (pure functions).

 From my experience, the D const system as it stands now does not cause 
 such misunderstanding.  Everything is quite clear.  What seems to cause 
 some misunderstanding:

 - Why all string library functions require invariant arguments (probably 

 - The concept of tail-const pointers and arrays
 - Why you cannot have tail-const class references
 - Why you cannot have logical const structures.

 Explaining const and invariant by itself is easy.  And understandable. 
 It's the usages of them which are confusing.


 If one doesn't understand the usages (and the consequences of) const, then 
 I don't agree that one understands it.


That is a fair point.  But still, there are plenty of features in D already 
that one does not need to understand the syntax of fully in order to use.


 But I look at this extension as a sort of 'meta' const.  It's like 
 templates and mixins.  Some people (not me!) can do magic with these that 
 would not be possible if templates weren't so wonderfully complex.  But 
 to use a template that someone wrote isn't even close to as difficult. 
 For example, the conversion template to(T).  To use it I just do:

 auto str = to!(string)(5);

 and I don't have to know how or why it works.


 I don't agree that it's ok for library features to be complicated because 
 only advanced users will write libraries. For one thing, it impedes 
 writing libraries (and we all know that proliferation of libraries is 
 essential for D's success) and it presumes that application code is 
 somehow at a level that doesn't need those features.


I think you may have missed my point.  Yes, library proliferation is 
essential for success.  But libraries can be complicated without the need 
for the user of the library to understand them.  Libraries don't have to be 
complicated.  Libraries don't have to use parameterized const for anything. 
But the ability is there if you need it.  I view this no differently than 
templates.  Templates can be complex and very abstract, but very powerful 
and useful.  They also do not need to be fully understood to be used.


 For example, it's really hard to write reusable libraries in C++, and look 
 at the consequences - few reusable libraries exist compared with other 
 languages.


I'd say you are wrong here.


 Similarly, if a library implements const groups such as lconst like:

 const alias mutable;
 alias const!(~mutable) lconst;
 alias invariant!(~mutable) linvariant;

 One does not have to understand exactly what they mean.


 As an engineer, I prefer to understand my tools completely :-)
 If they aren't understood, and there's an error message, one has no idea 
 what to do.


Sure, so learn the concepts.  I don't think there is any way to create a 
const system that does not require some learning.  Not everyone has read the 
dmd source code, or phobos source code, to find out exactly how it works, so 
not everyone needs to understand the tools they use completely.  In fact, I 
think you would be in the minority.

But my point above is, if you wish to not worry about what const!(X) means, 
you can read the library documentation and only use mutable, lconst, and 
linvariant.


 All they have to know is 'declare a member mutable if you want it to be 
 mutable when the class is lconst or linvariant.'  This kind of thing is 
 easy to document and explain.  One does not HAVE to know all the details 
 of how these magic generic const structures work, all they have to know 
 is how to use them.  In fact, they do not need to know much more than 
 what they already must learn for the current const.  Except that the last 
 two points of confusion I identified above are removed.

 In any case, like I said, if you find that it would be difficult to 
 implement in the compiler, then so be it.  But assuming that you can't 
 add a feature because it would complicate the language, in light of the 
 fact that you already complicated the language too much, seems to be a 
 contradictory argument.  We already have a complicated language by your 
 measure.  Why not make it a complete language?


 I worry about the complexity all the time. The idea is not to make a 
 complicated language, but to find a way to make a simpler language that 
 can do everything we need done. Complexity should only come with a benefit 
 that outweighs the cost of it.


And that is a judgement call you will have to make.  I already get the 
feeling you believe it is not worth it, and that's ok.  At least I put the 
idea out there :)

-Steve
Jun 25 2008
prev sibling parent reply "Me Here" <p9e883002 sneakemail.com> writes:
Walter Bright wrote:

Perl has invariant strings, but they are implicitly invariant

 and so nobody notices it, they just work.
 


Sorry Walter, but thta is simply not the case: Vis:

[0] Perl> $x = 'x' x 500e6;
[0] Perl> print length $x;;
500000000
[0] Perl> substr $x, 250e6, 1, 'y';;
[0] Perl> print length $x;;
500000000
[0] Perl> print substr $x, 250e6-5, 10;;
xxxxxyxxxx

b.
--
Jun 25 2008
parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
Me Here wrote:

 Walter Bright wrote:
 
 Perl has invariant strings, but they are implicitly invariant

 and so nobody notices it, they just work.


 Sorry Walter, but thta is simply not the case: Vis:
 
 [0] Perl> $x = 'x' x 500e6;
 [0] Perl> print length $x;;
 500000000
 [0] Perl> substr $x, 250e6, 1, 'y';;
 [0] Perl> print length $x;;
 500000000
 [0] Perl> print substr $x, 250e6-5, 10;;
 xxxxxyxxxx
 
 b.


What are you disagreeing with?

The fact that they're invariant?
Or the fact that nobody notices?

I have no idea if they're invariant in perl or not.  But I don't think 
your test above is conclusive proof that they're mutable.

--bb
Jun 25 2008
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Bill Baxter" wrote

 Me Here wrote:

 Walter Bright wrote:

 Perl has invariant strings, but they are implicitly invariant

 and so nobody notices it, they just work.


 Sorry Walter, but thta is simply not the case: Vis:

 [0] Perl> $x = 'x' x 500e6;
 [0] Perl> print length $x;;
 500000000
 [0] Perl> substr $x, 250e6, 1, 'y';;
 [0] Perl> print length $x;;
 500000000
 [0] Perl> print substr $x, 250e6-5, 10;;
 xxxxxyxxxx

 b.


 What are you disagreeing with?

 The fact that they're invariant?
 Or the fact that nobody notices?

 I have no idea if they're invariant in perl or not.  But I don't think 
 your test above is conclusive proof that they're mutable.


No it's not.  The only conclusive proof comes from observing what happens 
when you copy strings from one to the other:



print "before x\n";
sleep 20;
$x = 'x' x 100000000;
print "initialized x\n";
sleep 5;
$y = $x;
print "copied to y\n";
sleep 5;
substr $x, 3, 1, 'y';
print "did substring\n";
print substr $y, 0, 5;
print "\n";
sleep 5;

OK, so what does this do?  I set x to a string of 100 million x's, then 
assign x to y, then replace the 4th character in x with a 'y', then print 
the first 5 characters of y to see if they changed too (see if x and y 
reference the same data)

So what does this output?
before x
initialized x
copied to y
did substring
xxxxx

But this is not yet conclusive proof, we need to watch what happens with 
memory usage when each step occurs (hence the sleeps).  So using 'top', I 
observed this:

before x => mem usage 3k
initialized x => 191MB (!)
copied to y => 291MB
did substring => 291MB
xxxxx

So, what it looks like is on assignment, the string is copied, and the 
editing edits the string in-place.   But I can't really explain why it takes 
191MB to store x, where it only takes 100MB to store y.

So I'd say perl does not have invariant strings.  'course, I'm not a perl 
hacker, so I don't know if I did this correctly :)

-Steve
Jun 26 2008
next sibling parent reply Dee Girl <deegirl noreply.com> writes:
Steven Schveighoffer Wrote:


 
 "Bill Baxter" wrote

 Me Here wrote:

 Walter Bright wrote:

 Perl has invariant strings, but they are implicitly invariant

 and so nobody notices it, they just work.


 Sorry Walter, but thta is simply not the case: Vis:

 [0] Perl> $x = 'x' x 500e6;
 [0] Perl> print length $x;;
 500000000
 [0] Perl> substr $x, 250e6, 1, 'y';;
 [0] Perl> print length $x;;
 500000000
 [0] Perl> print substr $x, 250e6-5, 10;;
 xxxxxyxxxx

 b.


 What are you disagreeing with?

 The fact that they're invariant?
 Or the fact that nobody notices?

 I have no idea if they're invariant in perl or not.  But I don't think 
 your test above is conclusive proof that they're mutable.


 
 No it's not.  The only conclusive proof comes from observing what happens 
 when you copy strings from one to the other:
 

 
 print "before x\n";
 sleep 20;
 $x = 'x' x 100000000;
 print "initialized x\n";
 sleep 5;
 $y = $x;
 print "copied to y\n";
 sleep 5;
 substr $x, 3, 1, 'y';
 print "did substring\n";
 print substr $y, 0, 5;
 print "\n";
 sleep 5;
 
 OK, so what does this do?  I set x to a string of 100 million x's, then 
 assign x to y, then replace the 4th character in x with a 'y', then print 
 the first 5 characters of y to see if they changed too (see if x and y 
 reference the same data)
 
 So what does this output?
 before x
 initialized x
 copied to y
 did substring
 xxxxx
 
 But this is not yet conclusive proof, we need to watch what happens with 
 memory usage when each step occurs (hence the sleeps).  So using 'top', I 
 observed this:
 
 before x => mem usage 3k
 initialized x => 191MB (!)
 copied to y => 291MB
 did substring => 291MB
 xxxxx
 
 So, what it looks like is on assignment, the string is copied, and the 
 editing edits the string in-place.   But I can't really explain why it takes 
 191MB to store x, where it only takes 100MB to store y.
 
 So I'd say perl does not have invariant strings.  'course, I'm not a perl 
 hacker, so I don't know if I did this correctly :)
 
 -Steve 


Hello! I think the first part of your message is correct. the second is maybe
mis guided. Walter is correct. Perl strings do not have mutable chars. They can
be think as similar to D strings. Your example with $x and $y shows that.

Perl can optimize copies sometimes. But it does not matter. Semantics is all
that matters. And Perl strings can not mutate individual chars ever. Thanks,
Dee Girl
Jun 26 2008
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Dee Girl" wrote

 Steven Schveighoffer Wrote:

 So, what it looks like is on assignment, the string is copied, and the
 editing edits the string in-place.   But I can't really explain why it 
 takes
 191MB to store x, where it only takes 100MB to store y.

 So I'd say perl does not have invariant strings.  'course, I'm not a perl
 hacker, so I don't know if I did this correctly :)

 -Steve


 Hello! I think the first part of your message is correct. the second is 
 maybe mis guided. Walter is correct. Perl strings do not have mutable 
 chars. They can be think as similar to D strings. Your example with $x and 
 $y shows that.

 Perl can optimize copies sometimes. But it does not matter. Semantics is 
 all that matters. And Perl strings can not mutate individual chars ever. 
 Thanks, Dee Girl


I am not super-knowledgable about perl, but I understand the workings of 
invariant strings and what they mean for memory usage.  The memory usage 
exhibited by perl when copying one string to another suggests an entire copy 
of the data, not just copying a reference.  If strings were immutable (like 
they are in D or Java), then memory usage should not go up by 100MB when 
simply assinging two variables to point to the same data.  It actually 
appears to me that perl has mutable strings but only allows one reference to 
the data at a time.  i.e. they are more like C++ std::strings (when not used 
with references).

Perhaps you can explain how my example shows they are immutable?  Maybe I am 
not getting something.

-Steve
Jun 26 2008
parent reply Dee Girl <deegirl noreply.com> writes:
Steven Schveighoffer Wrote:


 I am not super-knowledgable about perl, but I understand the workings of 
 invariant strings and what they mean for memory usage.  The memory usage 
 exhibited by perl when copying one string to another suggests an entire copy 
 of the data, not just copying a reference.  If strings were immutable (like 
 they are in D or Java), then memory usage should not go up by 100MB when 
 simply assinging two variables to point to the same data.  It actually 
 appears to me that perl has mutable strings but only allows one reference to 
 the data at a time.  i.e. they are more like C++ std::strings (when not used 
 with references).
 
 Perhaps you can explain how my example shows they are immutable?  Maybe I am 
 not getting something.


Perl has mix of reference counting with duplication and many optimization
depending on code. It is mis guided to judge only by memory use. That is not
relevant, just implementation detail. Tomorrow Perl is better, yesterday is
bad. It is not relevant.

What is needed to look is semantics. Strings in Perl never alias, have strict
value semantics. It is same as saying they have immutable characters because
you can not distinguish.

You assign $y = $x. Two things could happen, a refcount is done or a full copy
is done. You do not know. But you do not care! You care if changing one
character in $x changes one character in $y. That never is happen. Which means
semantically Perl strings are as good as strings of invariant characters. They
never alias mutable data. This is the important thing. Thank you, Dee Girl
Jun 26 2008
next sibling parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
Dee Girl wrote:

 Steven Schveighoffer Wrote:
 
 You assign $y = $x. Two things could happen, a refcount is done or a full copy
is done. You do not know. But you do not care! You care if changing one
character in $x changes one character in $y. That never is happen. Which means
semantically Perl strings are as good as strings of invariant characters. They
never alias mutable data. This is the important thing. 


I certainly care if a 100MB string is getting duplicated.  It's most 
definitely going to change how I write the algorithm to manipulate that 
string.

"Act like value type" and "are immutable" are two categories which have 
overlap, but they are not identical.  Walter keeps saying strings in 
perl are immutable, but Steven's test demonstrates that (at least for 
100MB strings) they are not immutable, but they do act like value types.

This difference is relevant since Walter has often argued that invariant 
strings are the way to go based on the "fact" that they have been such a 
success in languages like Perl.  And the implication is clear there (to 
me any way) that by 'invariant' he means invariant in the sense that D2 
strings are invariant.  If he wants to include Perl in his argument he 
should be saying "value type strings" rather than "invariant strings". 
Or he should just stick to using Java as his example.  :-)

--bb
Jun 26 2008
parent reply Dee Girl <deegirl noreply.com> writes:
Bill Baxter Wrote:


 Dee Girl wrote:

 Steven Schveighoffer Wrote:
 
 You assign $y = $x. Two things could happen, a refcount is done or a full copy
is done. You do not know. But you do not care! You care if changing one
character in $x changes one character in $y. That never is happen. Which means
semantically Perl strings are as good as strings of invariant characters. They
never alias mutable data. This is the important thing. 


 
 I certainly care if a 100MB string is getting duplicated.  It's most 
 definitely going to change how I write the algorithm to manipulate that 
 string.


Perl uses many strategy. Focus on copy or no copy is missing my point and
Walter point.


 "Act like value type" and "are immutable" are two categories which have 
 overlap, but they are not identical.  Walter keeps saying strings in 
 perl are immutable, but Steven's test demonstrates that (at least for 
 100MB strings) they are not immutable, but they do act like value types.

 This difference is relevant since Walter has often argued that invariant 
 strings are the way to go based on the "fact" that they have been such a 
 success in languages like Perl.  And the implication is clear there (to 
 me any way) that by 'invariant' he means invariant in the sense that D2 
 strings are invariant.  If he wants to include Perl in his argument he 
 should be saying "value type strings" rather than "invariant strings". 
 Or he should just stick to using Java as his example.  :-)


I am sorry, again missing the point. Walter argument is very good. In Perl may
get copy or just reference. In D always get a reference. You can create new
copy. So even D has more control than Perl. In Perl you don't control when copy
is.

Walter argument was that value-like types are better because no mutable
aliasing. Mutable aliasing makes things difficult because for non local
dependencies. This why scripting languages so easy to play with strings. D does
that in a different way from Perl which is as powerful or even more. I think
his point is perfect valid, even when terminology is imprecise this time. And I
am happy he does not fall again in his C++ comparison trap where argument
becomes childish ^_^. Thank you, Dee Girl
Jun 26 2008
parent Bill Baxter <dnewsgroup billbaxter.com> writes:
Dee Girl wrote:

 Bill Baxter Wrote:
 

 Dee Girl wrote:

 Steven Schveighoffer Wrote:

 You assign $y = $x. Two things could happen, a refcount is done or a full copy
is done. You do not know. But you do not care! You care if changing one
character in $x changes one character in $y. That never is happen. Which means
semantically Perl strings are as good as strings of invariant characters. They
never alias mutable data. This is the important thing. 


 I certainly care if a 100MB string is getting duplicated.  It's most 
 definitely going to change how I write the algorithm to manipulate that 
 string.


 
 Perl uses many strategy. Focus on copy or no copy is missing my point and
Walter point.
 

 "Act like value type" and "are immutable" are two categories which have 
 overlap, but they are not identical.  Walter keeps saying strings in 
 perl are immutable, but Steven's test demonstrates that (at least for 
 100MB strings) they are not immutable, but they do act like value types.

 This difference is relevant since Walter has often argued that invariant 
 strings are the way to go based on the "fact" that they have been such a 
 success in languages like Perl.  And the implication is clear there (to 
 me any way) that by 'invariant' he means invariant in the sense that D2 
 strings are invariant.  If he wants to include Perl in his argument he 
 should be saying "value type strings" rather than "invariant strings". 
 Or he should just stick to using Java as his example.  :-)


 
 I am sorry, again missing the point. Walter argument is very good. In Perl may
get copy or just reference. In D always get a reference. You can create new
copy. So even D has more control than Perl. In Perl you don't control when copy
is.
 
 Walter argument was that value-like types are better because no mutable
aliasing. Mutable aliasing makes things difficult because for non local
dependencies. This why scripting languages so easy to play with strings. D does
that in a different way from Perl which is as powerful or even more. I think
his point is perfect valid, even when terminology is imprecise this time. And I
am happy he does not fall again in his C++ comparison trap where argument
becomes childish ^_^. Thank you, Dee Girl
 


I think we're all on the same page here.
My point is that Walter is using the wrong words to make his argument. 
He means "value type" but he has on several occasions stated that Perl 
strings are good because they are "invariant".

--bb
Jun 26 2008
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Dee Girl" wrote

 Steven Schveighoffer Wrote:


 I am not super-knowledgable about perl, but I understand the workings of
 invariant strings and what they mean for memory usage.  The memory usage
 exhibited by perl when copying one string to another suggests an entire 
 copy
 of the data, not just copying a reference.  If strings were immutable 
 (like
 they are in D or Java), then memory usage should not go up by 100MB when
 simply assinging two variables to point to the same data.  It actually
 appears to me that perl has mutable strings but only allows one reference 
 to
 the data at a time.  i.e. they are more like C++ std::strings (when not 
 used
 with references).

 Perhaps you can explain how my example shows they are immutable?  Maybe I 
 am
 not getting something.


 Perl has mix of reference counting with duplication and many optimization 
 depending on code. It is mis guided to judge only by memory use. That is 
 not relevant, just implementation detail. Tomorrow Perl is better, 
 yesterday is bad. It is not relevant.

 What is needed to look is semantics. Strings in Perl never alias, have 
 strict value semantics. It is same as saying they have immutable 
 characters because you can not distinguish.


OK, I get what you are saying.  Immutability is not the important 
characteristic, it's value-semantics.  That still invalidates Walter's 
argument that immutability is essential to how perl strings 'just work'.


 You assign $y = $x. Two things could happen, a refcount is done or a full 
 copy is done. You do not know. But you do not care!


I might care about whether perl decides to consume half my memory or not :) 
But technically I don't care since I don't use perl ;)

Thanks for the info!

-Steve
Jun 26 2008
parent Dee Girl <deegirl noreply.com> writes:
Steven Schveighoffer Wrote:


 
 "Dee Girl" wrote

 Steven Schveighoffer Wrote:


 I am not super-knowledgable about perl, but I understand the workings of
 invariant strings and what they mean for memory usage.  The memory usage
 exhibited by perl when copying one string to another suggests an entire 
 copy
 of the data, not just copying a reference.  If strings were immutable 
 (like
 they are in D or Java), then memory usage should not go up by 100MB when
 simply assinging two variables to point to the same data.  It actually
 appears to me that perl has mutable strings but only allows one reference 
 to
 the data at a time.  i.e. they are more like C++ std::strings (when not 
 used
 with references).

 Perhaps you can explain how my example shows they are immutable?  Maybe I 
 am
 not getting something.


 Perl has mix of reference counting with duplication and many optimization 
 depending on code. It is mis guided to judge only by memory use. That is 
 not relevant, just implementation detail. Tomorrow Perl is better, 
 yesterday is bad. It is not relevant.

 What is needed to look is semantics. Strings in Perl never alias, have 
 strict value semantics. It is same as saying they have immutable 
 characters because you can not distinguish.


 
 OK, I get what you are saying.  Immutability is not the important 
 characteristic, it's value-semantics.  That still invalidates Walter's 
 argument that immutability is essential to how perl strings 'just work'.


Walter had good argument with wrong words. Perl strings are good because they
act like values. So are D strings. The argument is valid. 

I work more with D strings now and I never found better idea for string
implementation in all language I know. It is amazing how things stay together
in type system so thin.


 You assign $y = $x. Two things could happen, a refcount is done or a full 
 copy is done. You do not know. But you do not care!


 
 I might care about whether perl decides to consume half my memory or not :) 
 But technically I don't care since I don't use perl ;)


Me not too since I use D ^_^. In general scripting language has less control of
allocation. But D regex veeeery sloooow... I wish some body optimizes
std.regex. Also the API of regex is very (do not know the word...) scrambled or
disordered or inconsistent. When ever I use regex I must look the manual page.
API is terrible and you never know what function you must call and they are not
orthogonal and the names are weird. Andrei please fix ^_^. Thank you, Dee Girl
Jun 26 2008
prev sibling parent "Me Here" <p9e883002 sneakemail.com> writes:
Dee Girl wrote:


 Steven Schveighoffer Wrote:
 

 
 "Bill Baxter" wrote

 Me Here wrote:

 Walter Bright wrote:
 
 Perl has invariant strings, but they are implicitly invariant

 and so nobody notices it, they just work.
 


 Sorry Walter, but thta is simply not the case: Vis:
 
 [0] Perl> $x = 'x' x 500e6;
 [0] Perl> print length $x;;
 500000000
 [0] Perl> substr $x, 250e6, 1, 'y';;
 [0] Perl> print length $x;;
 500000000
 [0] Perl> print substr $x, 250e6-5, 10;;
 xxxxxyxxxx
 
 b.


 
 What are you disagreeing with?
 
 The fact that they're invariant?
 Or the fact that nobody notices?
 
 I have no idea if they're invariant in perl or not.  But I don't think 
 your test above is conclusive proof that they're mutable.


 
 No it's not.  The only conclusive proof comes from observing what happens 
 when you copy strings from one to the other:
 

 
 print "before x\n";
 sleep 20;
 $x = 'x' x 100000000;
 print "initialized x\n";
 sleep 5;
 $y = $x;
 print "copied to y\n";
 sleep 5;
 substr $x, 3, 1, 'y';
 print "did substring\n";
 print substr $y, 0, 5;
 print "\n";
 sleep 5;
 
 OK, so what does this do?  I set x to a string of 100 million x's, then 
 assign x to y, then replace the 4th character in x with a 'y', then print 
 the first 5 characters of y to see if they changed too (see if x and y 
 reference the same data)
 
 So what does this output?
 before x
 initialized x
 copied to y
 did substring
 xxxxx
 
 But this is not yet conclusive proof, we need to watch what happens with 
 memory usage when each step occurs (hence the sleeps).  So using 'top', I 
 observed this:
 
 before x => mem usage 3k
 initialized x => 191MB (!)
 copied to y => 291MB
 did substring => 291MB
 xxxxx
 
 So, what it looks like is on assignment, the string is copied, and the 
 editing edits the string in-place.   But I can't really explain why it
 takes 191MB to store x, where it only takes 100MB to store y.
 
 So I'd say perl does not have invariant strings.  'course, I'm not a perl 
 hacker, so I don't know if I did this correctly :)
 
 -Steve 


 
 Hello! I think the first part of your message is correct. the second is maybe
 mis guided. Walter is correct. Perl strings do not have mutable chars. They
 can be think as similar to D strings. Your example with $x and $y shows that.
 
 Perl can optimize copies sometimes. But it does not matter. Semantics is all
 that matters. And Perl strings can not mutate individual chars ever. Thanks,
 Dee Girl


Utter baloney.

b.

--
Jun 26 2008
prev sibling parent Bill Baxter <dnewsgroup billbaxter.com> writes:
Steven Schveighoffer wrote:

 "Bill Baxter" wrote

 Me Here wrote:

 Walter Bright wrote:

 Perl has invariant strings, but they are implicitly invariant

 and so nobody notices it, they just work.


 Sorry Walter, but thta is simply not the case: Vis:

 [0] Perl> $x = 'x' x 500e6;
 [0] Perl> print length $x;;
 500000000
 [0] Perl> substr $x, 250e6, 1, 'y';;
 [0] Perl> print length $x;;
 500000000
 [0] Perl> print substr $x, 250e6-5, 10;;
 xxxxxyxxxx

 b.


 What are you disagreeing with?

 The fact that they're invariant?
 Or the fact that nobody notices?

 I have no idea if they're invariant in perl or not.  But I don't think 
 your test above is conclusive proof that they're mutable.


 
 No it's not.  The only conclusive proof comes from observing what happens 
 when you copy strings from one to the other:
 

 
 print "before x\n";
 sleep 20;
 $x = 'x' x 100000000;
 print "initialized x\n";
 sleep 5;
 $y = $x;
 print "copied to y\n";
 sleep 5;
 substr $x, 3, 1, 'y';
 print "did substring\n";
 print substr $y, 0, 5;
 print "\n";
 sleep 5;
 
 OK, so what does this do?  I set x to a string of 100 million x's, then 
 assign x to y, then replace the 4th character in x with a 'y', then print 
 the first 5 characters of y to see if they changed too (see if x and y 
 reference the same data)
 
 So what does this output?
 before x
 initialized x
 copied to y
 did substring
 xxxxx
 
 But this is not yet conclusive proof, we need to watch what happens with 
 memory usage when each step occurs (hence the sleeps).  So using 'top', I 
 observed this:
 
 before x => mem usage 3k
 initialized x => 191MB (!)
 copied to y => 291MB
 did substring => 291MB
 xxxxx
 
 So, what it looks like is on assignment, the string is copied, and the 
 editing edits the string in-place.   But I can't really explain why it takes 
 191MB to store x, where it only takes 100MB to store y.
 
 So I'd say perl does not have invariant strings.  'course, I'm not a perl 
 hacker, so I don't know if I did this correctly :)


That is a bit more conclusive.  So it appears that what Perl has is not 
invariant strings, but rather mutable strings that are copied on 
assignment so that they can be thought of more or less as value types.

So what happens with function calls?  To keep the illusion of "string is 
a value type" those would need to dup string arguments too.

--bb
Jun 26 2008
prev sibling parent reply "Me Here" <p9e883002 sneakemail.com> writes:
Bill Baxter wrote:


 Me Here wrote:

 Walter Bright wrote:
 
 Perl has invariant strings, but they are implicitly invariant

 and so nobody notices it, they just work.
 


 Sorry Walter, but thta is simply not the case: Vis:
 
[0] Perl> $x = 'x' x 500e6;
[0] Perl> print length $x;;
 500000000
[0] Perl> substr $x, 250e6, 1, 'y';;
[0] Perl> print length $x;;
 500000000
[0] Perl> print substr $x, 250e6-5, 10;;
 xxxxxyxxxx
 
 b.


 
 What are you disagreeing with?


Simple. Walter's statement that Perl's strings are invarient. 


 
 The fact that they're invariant?
 Or the fact that nobody notices?
 
 I have no idea if they're invariant in perl or not.  But I don't think your
 test above is conclusive proof that they're mutable.
 


True. It is not conclusive proof. That's very hard to provide in a forum such
as this.

I *know* the above proves it, because I can monitor the memory usage and
addresses.
I used a very large string and the mutated a character in the middle of it. If
the original string was mutated, the memory consumption of the process would
have to (breifly) double. It does not.

For those familiar with Perl, a simple 

    $scalar =~ s[(.)(.)][\2\2]g;

would be sufficient to demonstrate it. This will swap every pair of characters
in the string, in place.

Or 
    $scalar =~ tr[A-Za-z][a-zA-Z];

which will switch the case of all characters in the string *in-place*.

So, let me make this statement: Perl's SVs are *absolutely not* immutable!


 --bb


What does this mean for the wider discussion? Nothing. But that particular part
of Walter's post is incorrect.

b.
--
Jun 26 2008
parent reply "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
Me Here <p9e883002 sneakemail.com> wrote:

 I *know* the above proves it, because I can monitor the memory usage and
 addresses.
 I used a very large string and the mutated a character in the middle of  
 it. If
 the original string was mutated, the memory consumption of the process  
 would
 have to (breifly) double. It does not.


Could not the garbage collector theoretically be intelligent enough to see
that there's only one reference to the string, and thus not do CoW?

-- Simen
Jun 26 2008
next sibling parent Bill Baxter <dnewsgroup billbaxter.com> writes:
Simen Kjaeraas wrote:

 Me Here <p9e883002 sneakemail.com> wrote:

 I *know* the above proves it, because I can monitor the memory usage and
 addresses.
 I used a very large string and the mutated a character in the middle 
 of it. If
 the original string was mutated, the memory consumption of the process 
 would
 have to (breifly) double. It does not.


 
 Could not the garbage collector theoretically be intelligent enough to see
 that there's only one reference to the string, and thus not do CoW?
 
 -- Simen


Right.  That's what I was thinking too.  But Steven's example with 
memory info rules out that possibility.

--bb
Jun 26 2008
prev sibling parent reply "Me Here" <p9e883002 sneakemail.com> writes:
Simen Kjaeraas wrote:


 Me Here <p9e883002 sneakemail.com> wrote:

 I know the above proves it, because I can monitor the memory usage and
 addresses.
 I used a very large string and the mutated a character in the middle of
 it. If the original string was mutated, the memory consumption of the
 process   would have to (breifly) double. It does not.


 
 Could not the garbage collector theoretically be intelligent enough to see
 that there's only one reference to the string, and thus not do CoW?
 
 -- Simen


Perhaps you will find this a more convincing demonstration:

    [0] Perl> $s = 'the quick brown fox';;
    [0] Perl> $r = \substr $s, 10, 5;;
    [0] Perl> $$r = 'green';;
    [0] Perl> print $s;;
    the quick green fox

Now the description.

1) Assign a string to the scalar $s
2) Take a reference $r to a portion of that scalar
3) Replace that portion in place by assigning through the reference.
4) Print the modified original string.

Besides which, I don't think I know this. I know I know it.

As for the GC deciding not to do COW, Your way off base here, in Perl at least.

b.

--
Jun 26 2008
next sibling parent Dee Girl <deegirl noreply.com> writes:
Me Here Wrote:


 Simen Kjaeraas wrote:
 

 Me Here <p9e883002 sneakemail.com> wrote:

 I know the above proves it, because I can monitor the memory usage and
 addresses.
 I used a very large string and the mutated a character in the middle of
 it. If the original string was mutated, the memory consumption of the
 process   would have to (breifly) double. It does not.


 
 Could not the garbage collector theoretically be intelligent enough to see
 that there's only one reference to the string, and thus not do CoW?
 
 -- Simen


 
 Perhaps you will find this a more convincing demonstration:
 
     [0] Perl> $s = 'the quick brown fox';;
     [0] Perl> $r = \substr $s, 10, 5;;
     [0] Perl> $$r = 'green';;
     [0] Perl> print $s;;
     the quick green fox


I stand corrected. As could your manners ^_^.

This is good example. Trick of taking a reference to result of substr is new to
me. Maybe now D is more better compared to Perl because in D you can not make
aliased changes to string! Thanks for teaching me. Dee Girl
Jun 26 2008
prev sibling parent reply Dee Girl <deegirl noreply.com> writes:
Me Here Wrote:


 Simen Kjaeraas wrote:
 

 Me Here <p9e883002 sneakemail.com> wrote:

 I know the above proves it, because I can monitor the memory usage and
 addresses.
 I used a very large string and the mutated a character in the middle of
 it. If the original string was mutated, the memory consumption of the
 process   would have to (breifly) double. It does not.


 
 Could not the garbage collector theoretically be intelligent enough to see
 that there's only one reference to the string, and thus not do CoW?
 
 -- Simen


 
 Perhaps you will find this a more convincing demonstration:
 
     [0] Perl> $s = 'the quick brown fox';;
     [0] Perl> $r = \substr $s, 10, 5;;
     [0] Perl> $$r = 'green';;
     [0] Perl> print $s;;
     the quick green fox


I stand corrected. As could your manners ^_^.

This is good example. Trick of taking a reference to result of substr is new to
me. Very interesting even when I made $$r = "very long string" x 100 it still
works. The number of characters can be different. How they do it?

Maybe now D is more better compared to Perl because in D you can not make
aliased changes to string! Thanks for teaching me. Dee Girl
Jun 26 2008
parent reply "Me Here" <p9e883002 sneakemail.com> writes:
Dee Girl wrote:


 Me Here Wrote:

 Perhaps you will find this a more convincing demonstration:
 
     [0] Perl> $s = 'the quick brown fox';;
     [0] Perl> $r = \substr $s, 10, 5;;
     [0] Perl> $$r = 'green';;
     [0] Perl> print $s;;
     the quick green fox


 
 I stand corrected. As could your manners ^_^.


Sorry if you find my directness to be 'rude', but making assumptions based on
your guesswork about trivial eveidence and then propounding conclusions as if
they were facts, does not engender me to de indirect.


 
 This is good example. Trick of taking a reference to result of substr is new
 to me. Very interesting even when I made $$r = "very long string" x 100 it
 still works. The number of characters can be different. How they do it?


It wasn't easy. But, if you look back to the earlier thread where I
demonstrated the in-place insertions and deletions are achieved through the use
of a pointer, length and offset, it should be clearer.


 
 Maybe now D is more better compared to Perl because in D you can not make
 aliased changes to string! Thanks for teaching me. Dee Girl


Welcome, b.

--
Jun 26 2008
parent reply Dee Girl <deegirl noreply.com> writes:
Me Here Wrote:


 Dee Girl wrote:
 

 Me Here Wrote:

 Perhaps you will find this a more convincing demonstration:
 
     [0] Perl> $s = 'the quick brown fox';;
     [0] Perl> $r = \substr $s, 10, 5;;
     [0] Perl> $$r = 'green';;
     [0] Perl> print $s;;
     the quick green fox


 
 I stand corrected. As could your manners ^_^.


 
 Sorry if you find my directness to be 'rude', but making assumptions based on
 your guesswork about trivial eveidence and then propounding conclusions as if
 they were facts, does not engender me to de indirect.


I do not like to do that. It was from the Perl book which I read from cover to
back cover. But it has been some time and clearly I do not remember all
correct. Thank you, Dee Girl
Jun 26 2008
parent "Me Here" <p9e883002 sneakemail.com> writes:
Dee Girl wrote:


 Me Here Wrote:
 

 Dee Girl wrote:
 
 But it has been some time and clearly I do not remember all


 correct. Thank you, Dee Girl


There are a lot of perl books and much that is not written about in most of
them. It is easy to infer the wrong thing based upon what you see.

I know, because I make that mistake all the time :)

Here's another little snippet that might amuse you:

	$s = '1234'; 
	$r1 = \vec( $s, 1, 8 ); 
	$r2 = \vec( $s, 2, 8 ); 
	$$r1 ^= $$r2 ^= $$r1 ^= $$r2; 
	
	print $s;;
	1324

Cheers, b.

--
Jun 26 2008
prev sibling next sibling parent reply Leandro Lucarella <llucax gmail.com> writes:
Steven Schveighoffer, el 24 de junio a las 13:30 me escribiste:

 I understand that this proposal might be difficult to implement in the 
 compiler, and it might just be not worth the effort.  There are a lot of 
 problems that can be solved by not using const or using interfaces, instead 
 of these constructs.  However, this proposal represents what I think is the 
 most lenient and flexible proposal that is also const-correct from those I 
 have read, and allows for almost any const contract specification (excluding 
 head-const).  Whether it gets implemented or not, it was interesting to 
 think about how to solve all these problems, and I enjoyed the challenge.


I agree with this. I think it´s wonderful, well thought proposal, which
seems to address almost all the const issues in an elegant way, but I find
it a little too complex and hard to understand/explain to worth it
(besides the possible complex implementation).

But I think this proposal is very rich in terms of understanding the "const
problem" and things that can or can´t be done using const.

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
SE CUMPLEN 1345 DIAS DESDE QUE NOS ROBARON NUESTRA CAMARA
	-- Crónica TV
Jun 24 2008
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Leandro Lucarella" wrote

 Steven Schveighoffer, el 24 de junio a las 13:30 me escribiste:

 I understand that this proposal might be difficult to implement in the
 compiler, and it might just be not worth the effort.  There are a lot of
 problems that can be solved by not using const or using interfaces, 
 instead
 of these constructs.  However, this proposal represents what I think is 
 the
 most lenient and flexible proposal that is also const-correct from those 
 I
 have read, and allows for almost any const contract specification 
 (excluding
 head-const).  Whether it gets implemented or not, it was interesting to
 think about how to solve all these problems, and I enjoyed the challenge.


 I agree with this. I think it´s wonderful, well thought proposal, which
 seems to address almost all the const issues in an elegant way, but I find
 it a little too complex and hard to understand/explain to worth it
 (besides the possible complex implementation).


Well, at least you agree :)  I also agree it is complex to explain, but as I 
replied to Walter, an average developer doesn't have to know about how it 
works to use it, if the library writer has already set up the appropriate 
const groups.

Thanks for the reply!

-Steve
Jun 24 2008
parent "Craig Black" <craigblack2 cox.net> writes:
 Well, at least you agree :)  I also agree it is complex to explain, but as 
 I replied to Walter, an average developer doesn't have to know about how 
 it works to use it, if the library writer has already set up the 
 appropriate const groups.


I write libraries in C++ and I strongly agree.  I would rather have a 
complicated way of doing something than to not be able to do it at all. 
Even if it is a pain in the ass to get working, it works, and it's usable.

-Craig
Jun 24 2008
prev sibling parent reply Jason House <jason.james.house gmail.com> writes:
Steven Schveighoffer wrote: 

 I would really like to hear people's opinions, especially Walter's.


What I like:
* Sticking to fundamental principles
* The const!(A,B)(T) syntax looks quite clean
* The ability for mixed const (AKA restricted mutability)

What I don't like:
* const!(~mutable)(T) syntax.  While a mutable keyword may be kind of nice,
the syntax is starting to get clunky and also become the only syntax
for "not const"
* No discussion of how to define const/invariant functions


A lot of times, when I think about tail const/mixed const/mutable/..., I
start thinking about arrays.
AKA:
  T[] vs. array!(T)
  const(T)[] vs. array!(const(T)) vs. const!(element)(array!(T))
Jun 24 2008
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Jason House" wrote

 Steven Schveighoffer wrote:

 I would really like to hear people's opinions, especially Walter's.


 What I like:
 * Sticking to fundamental principles
 * The const!(A,B)(T) syntax looks quite clean
 * The ability for mixed const (AKA restricted mutability)

 What I don't like:
 * const!(~mutable)(T) syntax.  While a mutable keyword may be kind of 
 nice,
 the syntax is starting to get clunky and also become the only syntax
 for "not const"


Yes, it is clunky, but also hidden :)  All you ever have to do is use 
'mutable', 'lconst', and 'linvariant.'  Compare it to mixins or templates.

import std.mutable;

class myclass
{
   mutable int x;
   int y;
}

void f(lconst(myclass) input)
{
    input.y = 2; // error, input.y is const
    input.x = 3; // ok.
}


 * No discussion of how to define const/invariant functions


I did leave this out.  Since const!(T) is analogous to const, it would be 
the same:

class myclass
{
   lconst void f() { /* 'this' is lconst */}
   void f2() lconst {/* same thing */}
}


 A lot of times, when I think about tail const/mixed const/mutable/..., I
 start thinking about arrays.
 AKA:
  T[] vs. array!(T)
  const(T)[] vs. array!(const(T)) vs. const!(element)(array!(T))


or const!(element)(T)[]

or

alias const!(element) eleconst;

eleconst(T)[]

-Steve
Jun 25 2008
parent reply Jason House <jason.james.house gmail.com> writes:
Steven Schveighoffer Wrote:


 "Jason House" wrote

 A lot of times, when I think about tail const/mixed const/mutable/..., I
 start thinking about arrays.
 AKA:
  T[] vs. array!(T)
  const(T)[] vs. array!(const(T)) vs. const!(element)(array!(T))




I probably should have stated that arrays have special status in D.  As built
in containers, they have support for for tail const while other objects don't
have that luxury.  I implicitly measure the quality of a const proposal by how
it can implement an alternate to arrays without the syntactic sugar arrays
provide.



 or const!(element)(T)[]
 
 or
 
 alias const!(element) eleconst;
 
 eleconst(T)[]


What's the difference between const!(element)(T)[] and const(T)[]?  I believe
this is more a misunderstanding of what I was driving at.  Consider this
template:

class array(T){
  typedef element T elem;
  elem contents[];
}

Maybe I got the syntax wrong, but the basic idea is that T and element are
really the same kind of thing.  array!(const T) and const!(element)(array!(T))
are different types.  Should they be?  I don't think so.  There's still
something buried in a tail const scheme that still needs to get fleshed out,
somehow...
Jun 25 2008
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Jason House" wrote

 Steven Schveighoffer Wrote:


 "Jason House" wrote

 A lot of times, when I think about tail const/mixed const/mutable/..., 
 I
 start thinking about arrays.
 AKA:
  T[] vs. array!(T)
  const(T)[] vs. array!(const(T)) vs. const!(element)(array!(T))




 I probably should have stated that arrays have special status in D.  As 
 built in containers, they have support for for tail const while other 
 objects don't have that luxury.  I implicitly measure the quality of a 
 const proposal by how it can implement an alternate to arrays without the 
 syntactic sugar arrays provide.


Ah, I now understand what you meant.  Yes, it can be done, just like you 
say.

const alias element;

class array(T){
   typedef element(T) elem;
   elem[] contents; // same as elem contents[] I think
}

the tag is not in itself const.  It is a noop when not selected for 
const/invariant.

So array!(T) becomes:

{
   typedef T elem;
   elem[] contents;
}

And const!(element)(array!(T)) becomes:

{
   typedef const(T) elem;
   elem[] contents;  // which is now const(T)[]
}


 or const!(element)(T)[]

 or

 alias const!(element) eleconst;

 eleconst(T)[]


 What's the difference between const!(element)(T)[] and const(T)[]?  I 
 believe this is more a misunderstanding of what I was driving at.


Yes, I misunderstood you.  const!(element)(T)[] and const(T)[] are different 
depending on what T is.  If T is a struct:

struct T
{
   element int x;
   int y;
}

Then const(T) yields:

{
   const int x;
   const int y;
}

and const!(element)(T) yields:

{
   const int x;
   int y;
}


  Consider this template:

 class array(T){
  typedef element T elem;
  elem contents[];
 }

 Maybe I got the syntax wrong, but the basic idea is that T and element are 
 really the same kind of thing.  array!(const T) and 
 const!(element)(array!(T)) are different types.  Should they be?  I don't 
 think so.  There's still something buried in a tail const scheme that 
 still needs to get fleshed out, somehow...


Tail const is much more possible with the const scheme I outlined.  Your 
example is not really relevant because the entire set of members is marked 
by a tag.  This is no different than today's const.

Basically, as a simple rough explanation, what I propose allows you to 
direct const to apply only to certain pieces of a struct or class, instead 
of always affecting the whole thing.  Tail const is possible because you can 
tag only the 'pointed to' data and const-ify only that part.

Another aspect is this:

array!(T) and array!(const T) are not implicitly castable, because they are 
two separate types.  e.g.:

array!(T) t = new array!(T)(5); // array of 5 Ts

const(array!(T)) t2 = t; // ok, t2 is the same type, but with a const 
modifier.

array!(const T) t3 = t; // error, incompatible types

const!(element)(array!(T)) t4 = t; // ok, because t4 is the same type, but 
with a const modifier.

-Steve
Jun 25 2008
parent reply Jason House <jason.james.house gmail.com> writes:
Steven Schveighoffer Wrote:


 "Jason House" wrote

  Consider this template:

 class array(T){
  typedef element T elem;
  elem contents[];
 }

 Maybe I got the syntax wrong, but the basic idea is that T and element are 
 really the same kind of thing.  array!(const T) and 
 const!(element)(array!(T)) are different types.  Should they be?  I don't 
 think so.  There's still something buried in a tail const scheme that 
 still needs to get fleshed out, somehow...


 
 Tail const is much more possible with the const scheme I outlined.  Your 
 example is not really relevant because the entire set of members is marked 
 by a tag.  This is no different than today's const.


Actually, it is different.  The array is resizable, so not everything inside is
constant.  I'm sure using an array inside a class is far less clear for const
examples, but I was trying to show the parallels with a simple array.



 Basically, as a simple rough explanation, what I propose allows you to 
 direct const to apply only to certain pieces of a struct or class, instead 
 of always affecting the whole thing.  Tail const is possible because you can 
 tag only the 'pointed to' data and const-ify only that part.
 
 Another aspect is this:
 
 array!(T) and array!(const T) are not implicitly castable, because they are 
 two separate types.  e.g.:
 
 array!(T) t = new array!(T)(5); // array of 5 Ts
 
 const(array!(T)) t2 = t; // ok, t2 is the same type, but with a const 
 modifier.
 
 array!(const T) t3 = t; // error, incompatible types
 
 const!(element)(array!(T)) t4 = t; // ok, because t4 is the same type, but 
 with a const modifier.


That's all true as proposed.  The deeper question is should it be that way? 
When should having a const argument as a template parameter result in a whole
new type and when should implicit casts to const template parameters be allowed?
Jun 25 2008
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Jason House" wrote

 Steven Schveighoffer Wrote:


 "Jason House" wrote

  Consider this template:

 class array(T){
  typedef element T elem;
  elem contents[];
 }

 Maybe I got the syntax wrong, but the basic idea is that T and element 
 are
 really the same kind of thing.  array!(const T) and
 const!(element)(array!(T)) are different types.  Should they be?  I 
 don't
 think so.  There's still something buried in a tail const scheme that
 still needs to get fleshed out, somehow...


 Tail const is much more possible with the const scheme I outlined.  Your
 example is not really relevant because the entire set of members is 
 marked
 by a tag.  This is no different than today's const.


 Actually, it is different.  The array is resizable, so not everything 
 inside is constant.  I'm sure using an array inside a class is far less 
 clear for const examples, but I was trying to show the parallels with a 
 simple array.


Yes, I stand corrected.  For your example const(array!(T)) is different than 
array!(const T).  My mistake.


 Basically, as a simple rough explanation, what I propose allows you to
 direct const to apply only to certain pieces of a struct or class, 
 instead
 of always affecting the whole thing.  Tail const is possible because you 
 can
 tag only the 'pointed to' data and const-ify only that part.

 Another aspect is this:

 array!(T) and array!(const T) are not implicitly castable, because they 
 are
 two separate types.  e.g.:

 array!(T) t = new array!(T)(5); // array of 5 Ts

 const(array!(T)) t2 = t; // ok, t2 is the same type, but with a const
 modifier.

 array!(const T) t3 = t; // error, incompatible types

 const!(element)(array!(T)) t4 = t; // ok, because t4 is the same type, 
 but
 with a const modifier.


 That's all true as proposed.  The deeper question is should it be that 
 way?  When should having a const argument as a template parameter result 
 in a whole new type and when should implicit casts to const template 
 parameters be allowed?


The problem is that with template compile-time evaluation, one can construct 
a completely different template for const T than for T.  With the generic 
const proposal, you cannot do this, as there is no generated type based on 
whether something is const or not.  The compiler would have to disallow this 
kind of thing, or else have a special way to identify the constancy as not 
varying the template generated.  But I much prefer not dealing with 
templates at all.

If there is a simpler way to do it, then I'm all for that.

-Steve
Jun 25 2008