digitalmars.D.learn - why won't byPair work with a const AA?
- Matthew Gamble (19/19) Jul 29 2017 I have a class member function from which I'm trying to return a
- =?UTF-8?Q?Ali_=c3=87ehreli?= (6/23) Jul 29 2017 I think it should work. I think a cast to unqualified is a safe
- Matthew Gamble (8/41) Jul 29 2017 Thanks Ali,
- =?UTF-8?Q?Ali_=c3=87ehreli?= (29/37) Jul 29 2017 As your question reveals, casting away const is not safe in general and
- Steven Schveighoffer (11/32) Aug 01 2017 byPair must store a pointer to the data in the AA. If you mark the AA
- H. S. Teoh via Digitalmars-d-learn (13/32) Aug 01 2017 [...]
- Steven Schveighoffer (9/42) Aug 01 2017 It works, because the byKeyValue implementation is so... ugly.
- H. S. Teoh via Digitalmars-d-learn (27/51) Aug 01 2017 [...]
- Steven Schveighoffer (6/34) Aug 01 2017 No, you can't const the Slot, because if the value type is a pointer,
- H. S. Teoh via Digitalmars-d-learn (25/59) Aug 01 2017 [...]
- Steven Schveighoffer (9/69) Aug 02 2017 You can iterate a const AA, but if you want to iterate a non-const AA,
- H. S. Teoh via Digitalmars-d-learn (20/31) Aug 02 2017 [...]
- Steven Schveighoffer (13/44) Aug 02 2017 It's not currently legal, you can't have inout members of a struct. This...
- H. S. Teoh via Digitalmars-d-learn (26/39) Aug 02 2017 [...]
- Steven Schveighoffer (8/49) Aug 02 2017 I have ideas :) I just haven't fleshed them out enough to present the
- Olivier FAURE (4/19) Aug 03 2017 I understand the general concept you're describing, but what
- Steven Schveighoffer (44/47) Aug 03 2017 tail modifiers are modifiers that only apply to the "tail" of the type.
- H. S. Teoh via Digitalmars-d-learn (47/59) Aug 02 2017 [...]
I have a class member function from which I'm trying to return a sorted array of key, value tuples stored in an associative array as a private member. The member function should be able to be marked const to prevent the AA from being modified. I have reduced the problem to the simple case below which won't compile with DMD v2.072.2. import std.array; import std.algorithm; class A { this() { aa = ["a":1, "b" : 2, "c" : 3]; } auto pairs() property const { return aa.byPair.array.sort().release; } private: int[string] aa; } If I remove const from the pairs function it compiles fine. I'm just not sure this is a behavior I want. Any help/recommendation would be appreciated.
Jul 29 2017
On 07/29/2017 09:19 PM, Matthew Gamble wrote:I have a class member function from which I'm trying to return a sorted array of key, value tuples stored in an associative array as a private member. The member function should be able to be marked const to prevent the AA from being modified. I have reduced the problem to the simple case below which won't compile with DMD v2.072.2. import std.array; import std.algorithm; class A { this() { aa = ["a":1, "b" : 2, "c" : 3]; } auto pairs() property const { return aa.byPair.array.sort().release; } private: int[string] aa; } If I remove const from the pairs function it compiles fine. I'm just not sure this is a behavior I want. Any help/recommendation would be appreciated.I think it should work. I think a cast to unqualified is a safe workaround in this case: auto pairs() property const { return (cast(int[string])aa).byPair.array.sort().release; } Ali
Jul 29 2017
On Sunday, 30 July 2017 at 04:36:19 UTC, Ali Çehreli wrote:On 07/29/2017 09:19 PM, Matthew Gamble wrote:Thanks Ali, That works to solve the compile problem. Seems the data in aa was safe from modification without the const or casting, even if the values are reference types (i.e. ClassB[string]). Is that expected? Best, MattI have a class member function from which I'm trying to return a sorted array of key, value tuples stored in an associative array as a private member. The member function should be able to be marked const to prevent the AA from being modified. I have reduced the problem to the simple case below which won't compile with DMD v2.072.2. import std.array; import std.algorithm; class A { this() { aa = ["a":1, "b" : 2, "c" : 3]; } auto pairs() property const { return aa.byPair.array.sort().release; } private: int[string] aa; } If I remove const from the pairs function it compiles fine. I'm just not sure this is a behavior I want. Any help/recommendation would be appreciated.I think it should work. I think a cast to unqualified is a safe workaround in this case: auto pairs() property const { return (cast(int[string])aa).byPair.array.sort().release; } Ali
Jul 29 2017
On 07/29/2017 10:15 PM, Matthew Gamble wrote:As your question reveals, casting away const is not safe in general and not for this code. :-/I think it should work. I think a cast to unqualified is a safe workaround in this case: auto pairs() property const { return (cast(int[string])aa).byPair.array.sort().release; }That works to solve the compile problem. Seems the data in aa was safe from modification without the const or casting, even if the values are reference types (i.e. ClassB[string]). Is that expected?No, they are not safe as you can mutate values in the AA: import std.stdio; import std.array; import std.algorithm; class B { int i; void mutate() { ++i; } } class A { this() { aa = ["a" : new B(), "b" : new B(), "c" : new B()]; } auto pairs() property const { return (cast(B[string])aa).byPair.array.sort().release; } private: B[string] aa; } void main() { auto a = new A(); auto p = a.pairs(); p.front[1].mutate(); assert(p.front[1].i == 1); // <-- Mutated :( } Of course, the problem is the same without const and the cast. Ali
Jul 29 2017
On 7/30/17 12:19 AM, Matthew Gamble wrote:I have a class member function from which I'm trying to return a sorted array of key, value tuples stored in an associative array as a private member. The member function should be able to be marked const to prevent the AA from being modified. I have reduced the problem to the simple case below which won't compile with DMD v2.072.2. import std.array; import std.algorithm; class A { this() { aa = ["a":1, "b" : 2, "c" : 3]; } auto pairs() property const { return aa.byPair.array.sort().release; } private: int[string] aa; } If I remove const from the pairs function it compiles fine. I'm just not sure this is a behavior I want. Any help/recommendation would be appreciated.byPair must store a pointer to the data in the AA. If you mark the AA const, then it must store a const pointer to AA data. However, because we don't have tail modifiers, you can't construct just one range that would make this work. We would need many different ranges, one for each flavor of mutability. So you are stuck doing what Ali recommended -- cast away the const. In this case, no harm comes if you don't cast back to const (since the value of the AA is `int`), but in general this solution isn't valid if the value type contains references. -Steve
Aug 01 2017
On Tue, Aug 01, 2017 at 10:04:18AM -0400, Steven Schveighoffer via Digitalmars-d-learn wrote:On 7/30/17 12:19 AM, Matthew Gamble wrote:[...][...] Actually, there's nothing about the implementation of both byKeyValue (the underlying implementation in druntime) and byPair in std.array that would preclude them from being used with const AA's. The only flaw is that the declaration of byPair doesn't match const AA's: https://issues.dlang.org/show_bug.cgi?id=17711 Here's the fix: https://github.com/dlang/phobos/pull/5668 T -- Sometimes the best solution to morale problems is just to fire all of the unhappy people. -- despair.comimport std.array; import std.algorithm; class A { this() { aa = ["a":1, "b" : 2, "c" : 3]; } auto pairs() property const { return aa.byPair.array.sort().release; } private: int[string] aa; } If I remove const from the pairs function it compiles fine. I'm just not sure this is a behavior I want. Any help/recommendation would be appreciated.byPair must store a pointer to the data in the AA. If you mark the AA const, then it must store a const pointer to AA data.
Aug 01 2017
On 8/1/17 6:50 PM, H. S. Teoh via Digitalmars-d-learn wrote:On Tue, Aug 01, 2017 at 10:04:18AM -0400, Steven Schveighoffer via Digitalmars-d-learn wrote:It works, because the byKeyValue implementation is so... ugly. For instance this: return Result(_aaRange(cast(void*)aa)); Just throws away all const/mutability. However, the Pair struct inside restores the correct modifiers. I hope... If this were a true implementation without the opaqueness, it would not work properly. -SteveOn 7/30/17 12:19 AM, Matthew Gamble wrote:[...][...] Actually, there's nothing about the implementation of both byKeyValue (the underlying implementation in druntime) and byPair in std.array that would preclude them from being used with const AA's. The only flaw is that the declaration of byPair doesn't match const AA's: https://issues.dlang.org/show_bug.cgi?id=17711 Here's the fix: https://github.com/dlang/phobos/pull/5668import std.array; import std.algorithm; class A { this() { aa = ["a":1, "b" : 2, "c" : 3]; } auto pairs() property const { return aa.byPair.array.sort().release; } private: int[string] aa; } If I remove const from the pairs function it compiles fine. I'm just not sure this is a behavior I want. Any help/recommendation would be appreciated.byPair must store a pointer to the data in the AA. If you mark the AA const, then it must store a const pointer to AA data.
Aug 01 2017
On Tue, Aug 01, 2017 at 07:09:45PM -0400, Steven Schveighoffer via Digitalmars-d-learn wrote:On 8/1/17 6:50 PM, H. S. Teoh via Digitalmars-d-learn wrote:[...][...] Actually, a proper implementation would still work, provided you declare your pointer types carefully. Sketch of idea: auto byKeyValue(AA)(AA aa) { struct Result { const(Slot)* current; // N.B.: proper type bool empty() { ... } auto front() { return Pair(*current); } void popFront() { current = current.next; ... } } return Result(aa); } Basically, the type of `current` must be const(Slot)* rather than const(Slot*), which would be the default inferred type. But since it's legal to assign a const pointer to a pointer to const (you can't modify the original pointer, nor what it points to, but it's valid to copy the pointer to a mutable pointer variable, as long as what is pointed to is still const), this actually will work without breaking / bypassing the type system. T -- People tell me that I'm skeptical, but I don't believe them.Actually, there's nothing about the implementation of both byKeyValue (the underlying implementation in druntime) and byPair in std.array that would preclude them from being used with const AA's. The only flaw is that the declaration of byPair doesn't match const AA's: https://issues.dlang.org/show_bug.cgi?id=17711 Here's the fix: https://github.com/dlang/phobos/pull/5668It works, because the byKeyValue implementation is so... ugly. For instance this: return Result(_aaRange(cast(void*)aa)); Just throws away all const/mutability. However, the Pair struct inside restores the correct modifiers. I hope... If this were a true implementation without the opaqueness, it would not work properly.
Aug 01 2017
On 8/1/17 7:15 PM, H. S. Teoh via Digitalmars-d-learn wrote:On Tue, Aug 01, 2017 at 07:09:45PM -0400, Steven Schveighoffer via Digitalmars-d-learn wrote:No, you can't const the Slot, because if the value type is a pointer, you can't then cast away the const-ness of it legally. There are ways to get it right, it involves templating for mutability. The current code is pretty horrific though, any way you slice it. -SteveIf this were a true implementation without the opaqueness, it would not work properly.[...] Actually, a proper implementation would still work, provided you declare your pointer types carefully. Sketch of idea: auto byKeyValue(AA)(AA aa) { struct Result { const(Slot)* current; // N.B.: proper type bool empty() { ... } auto front() { return Pair(*current); } void popFront() { current = current.next; ... } } return Result(aa); } Basically, the type of `current` must be const(Slot)* rather than const(Slot*), which would be the default inferred type. But since it's legal to assign a const pointer to a pointer to const (you can't modify the original pointer, nor what it points to, but it's valid to copy the pointer to a mutable pointer variable, as long as what is pointed to is still const), this actually will work without breaking / bypassing the type system.
Aug 01 2017
On Tue, Aug 01, 2017 at 07:31:41PM -0400, Steven Schveighoffer via Digitalmars-d-learn wrote:On 8/1/17 7:15 PM, H. S. Teoh via Digitalmars-d-learn wrote:[...] Counter-proof: struct Slot { Slot* next; const(string) key; const(int) value; } struct AA { Slot*[] slots; } unittest { const(AA) aa; static assert(is(typeof(aa.slots[0]) == const(Slot*))); const(Slot)* p = aa.slots[0]; // N.B.: legal p = p.next; // N.B.: legal } Note especially the type checked for in the static assert: you cannot modify any of the Slot pointers in the AA, but you *can* assign them to mutable (but tail-const) pointers. Therefore you totally can iterate a const AA without any casts or any breaking of the type system. And there's no need to template for mutability either. T -- One Word to write them all, One Access to find them, One Excel to count them all, And thus to Windows bind them. -- Mike ChampionOn Tue, Aug 01, 2017 at 07:09:45PM -0400, Steven Schveighoffer via Digitalmars-d-learn wrote:No, you can't const the Slot, because if the value type is a pointer, you can't then cast away the const-ness of it legally. There are ways to get it right, it involves templating for mutability.If this were a true implementation without the opaqueness, it would not work properly.[...] Actually, a proper implementation would still work, provided you declare your pointer types carefully. Sketch of idea: auto byKeyValue(AA)(AA aa) { struct Result { const(Slot)* current; // N.B.: proper type bool empty() { ... } auto front() { return Pair(*current); } void popFront() { current = current.next; ... } } return Result(aa); } Basically, the type of `current` must be const(Slot)* rather than const(Slot*), which would be the default inferred type. But since it's legal to assign a const pointer to a pointer to const (you can't modify the original pointer, nor what it points to, but it's valid to copy the pointer to a mutable pointer variable, as long as what is pointed to is still const), this actually will work without breaking / bypassing the type system.
Aug 01 2017
On 8/1/17 7:44 PM, H. S. Teoh via Digitalmars-d-learn wrote:On Tue, Aug 01, 2017 at 07:31:41PM -0400, Steven Schveighoffer via Digitalmars-d-learn wrote:You can iterate a const AA, but if you want to iterate a non-const AA, you need a different type. For instance, if your AA is int*[string], you will get const(int*) out of it, which may not be what you want. Unless your range is a slice or a pointer, then you can't do it properly without having separate types for const, immutable, mutable ranges. You can use templates to build them, but it's not straightforward or pleasant. -SteveOn 8/1/17 7:15 PM, H. S. Teoh via Digitalmars-d-learn wrote:[...] Counter-proof: struct Slot { Slot* next; const(string) key; const(int) value; } struct AA { Slot*[] slots; } unittest { const(AA) aa; static assert(is(typeof(aa.slots[0]) == const(Slot*))); const(Slot)* p = aa.slots[0]; // N.B.: legal p = p.next; // N.B.: legal } Note especially the type checked for in the static assert: you cannot modify any of the Slot pointers in the AA, but you *can* assign them to mutable (but tail-const) pointers. Therefore you totally can iterate a const AA without any casts or any breaking of the type system. And there's no need to template for mutability either.On Tue, Aug 01, 2017 at 07:09:45PM -0400, Steven Schveighoffer via Digitalmars-d-learn wrote:No, you can't const the Slot, because if the value type is a pointer, you can't then cast away the const-ness of it legally. There are ways to get it right, it involves templating for mutability.If this were a true implementation without the opaqueness, it would not work properly.[...] Actually, a proper implementation would still work, provided you declare your pointer types carefully. Sketch of idea: auto byKeyValue(AA)(AA aa) { struct Result { const(Slot)* current; // N.B.: proper type bool empty() { ... } auto front() { return Pair(*current); } void popFront() { current = current.next; ... } } return Result(aa); } Basically, the type of `current` must be const(Slot)* rather than const(Slot*), which would be the default inferred type. But since it's legal to assign a const pointer to a pointer to const (you can't modify the original pointer, nor what it points to, but it's valid to copy the pointer to a mutable pointer variable, as long as what is pointed to is still const), this actually will work without breaking / bypassing the type system.
Aug 02 2017
On Wed, Aug 02, 2017 at 08:20:23AM -0400, Steven Schveighoffer via Digitalmars-d-learn wrote:On 8/1/17 7:44 PM, H. S. Teoh via Digitalmars-d-learn wrote:[...]You can iterate a const AA, but if you want to iterate a non-const AA, you need a different type. For instance, if your AA is int*[string], you will get const(int*) out of it, which may not be what you want. Unless your range is a slice or a pointer, then you can't do it properly without having separate types for const, immutable, mutable ranges. You can use templates to build them, but it's not straightforward or pleasant.[...] Hmm. This seems like a perfect use case for inout, though I'm not confident the current implementation of inout can handle this. You'd do something like this: auto byPair(AA)(inout(AA) aa) { struct Result { inout(Slot)* current; ... // range primitives here } return Result(aa); } What I'm unsure of is whether the current implementation of inout can handle the inout inside the definition of Result. T -- Trying to define yourself is like trying to bite your own teeth. -- Alan Watts
Aug 02 2017
On 8/2/17 11:52 AM, H. S. Teoh via Digitalmars-d-learn wrote:On Wed, Aug 02, 2017 at 08:20:23AM -0400, Steven Schveighoffer via Digitalmars-d-learn wrote:It's not currently legal, you can't have inout members of a struct. This could be added, but it still wouldn't work, because you can't "strip off" the inout part upon return. The real answer is to have tail modifiers for structs, so you can do the same thing an array does. Note that if Result is an array, you CAN use inout: auto byPair(AA)(inout(AA) aa) { alias Result = inout(X)[]; return Result(...); } -SteveOn 8/1/17 7:44 PM, H. S. Teoh via Digitalmars-d-learn wrote:[...]You can iterate a const AA, but if you want to iterate a non-const AA, you need a different type. For instance, if your AA is int*[string], you will get const(int*) out of it, which may not be what you want. Unless your range is a slice or a pointer, then you can't do it properly without having separate types for const, immutable, mutable ranges. You can use templates to build them, but it's not straightforward or pleasant.[...] Hmm. This seems like a perfect use case for inout, though I'm not confident the current implementation of inout can handle this. You'd do something like this: auto byPair(AA)(inout(AA) aa) { struct Result { inout(Slot)* current; ... // range primitives here } return Result(aa); } What I'm unsure of is whether the current implementation of inout can handle the inout inside the definition of Result.
Aug 02 2017
On Wed, Aug 02, 2017 at 01:15:44PM -0400, Steven Schveighoffer via Digitalmars-d-learn wrote: [...]It's not currently legal, you can't have inout members of a struct. This could be added, but it still wouldn't work, because you can't "strip off" the inout part upon return. The real answer is to have tail modifiers for structs, so you can do the same thing an array does. Note that if Result is an array, you CAN use inout: auto byPair(AA)(inout(AA) aa) { alias Result = inout(X)[]; return Result(...); }[...] Yeah, this isn't the first time I've run into this. But then the problem becomes, how do you design tail modifiers for structs? Because here the inout (or its equivalent) has to apply to one specific member; the range methods can't also inherit the modifier otherwise in the const case you wouldn't be able to implement popFront(). I suppose, within the current type system, you'd have to template on modifiers, as you said, so that the incoming modifier is properly represented in the resulting type. But since we're already templating on the AA type, perhaps what we could do is something like: auto byPair(AA)(inout(AA) aa) { alias Modifiers = std.traits.getModifiers!AA; struct Result { std.traits.ApplyModifiers!(Slot*, Modifiers) slot; ... // range methods here } return Result(aa); } Of course, getModifiers and ApplyModifiers are fictitious Phobos templates, but you get the idea. T -- Дерево держится корнями, а человек - друзьями.
Aug 02 2017
On 8/2/17 2:06 PM, H. S. Teoh via Digitalmars-d-learn wrote:On Wed, Aug 02, 2017 at 01:15:44PM -0400, Steven Schveighoffer via Digitalmars-d-learn wrote: [...]I have ideas :) I just haven't fleshed them out enough to present the case. I've told Andrei about them and his reaction was not positive. But I think eventually D will need them.It's not currently legal, you can't have inout members of a struct. This could be added, but it still wouldn't work, because you can't "strip off" the inout part upon return. The real answer is to have tail modifiers for structs, so you can do the same thing an array does. Note that if Result is an array, you CAN use inout: auto byPair(AA)(inout(AA) aa) { alias Result = inout(X)[]; return Result(...); }[...] Yeah, this isn't the first time I've run into this. But then the problem becomes, how do you design tail modifiers for structs?Because here the inout (or its equivalent) has to apply to one specific member; the range methods can't also inherit the modifier otherwise in the const case you wouldn't be able to implement popFront(). I suppose, within the current type system, you'd have to template on modifiers, as you said, so that the incoming modifier is properly represented in the resulting type. But since we're already templating on the AA type, perhaps what we could do is something like: auto byPair(AA)(inout(AA) aa) { alias Modifiers = std.traits.getModifiers!AA; struct Result { std.traits.ApplyModifiers!(Slot*, Modifiers) slot; ... // range methods here } return Result(aa); } Of course, getModifiers and ApplyModifiers are fictitious Phobos templates, but you get the idea.Yes, but of course inout doesn't play a role here, as it's not a compile-time construct. Just: auto byPair(AA)(AA aa) -Steve
Aug 02 2017
On Wednesday, 2 August 2017 at 18:06:03 UTC, H. S. Teoh wrote:On Wed, Aug 02, 2017 at 01:15:44PM -0400, Steven Schveighoffer via Digitalmars-d-learn wrote: [...]I understand the general concept you're describing, but what exactly are tail modifiers? It's the first time I see this name, and my google-fu gives me nothing.The real answer is to have tail modifiers for structs, so you can do the same thing an array does. Note that if Result is an array, you CAN use inout: auto byPair(AA)(inout(AA) aa) { alias Result = inout(X)[]; return Result(...); }[...] Yeah, this isn't the first time I've run into this. But then the problem becomes, how do you design tail modifiers for structs?
Aug 03 2017
On 8/3/17 4:30 AM, Olivier FAURE wrote:I understand the general concept you're describing, but what exactly are tail modifiers? It's the first time I see this name, and my google-fu gives me nothing.tail modifiers are modifiers that only apply to the "tail" of the type. For example const(int)* is applying const to the "tail" of the int pointer, that is, the int that it points at. It's not applying to the actual pointer itself (we call that the "head"). Another way to look at it is that when you copy a variable you always make a copy of the head, and you don't make a copy of the tail. For this reason, the fully-modified and tail-modified types are implicitly castable, and this is the important property we need to duplicate. A tail-modified struct would be something like this: struct S { int * x; } const(S) s; This looks like this: struct ConstS { const(int *) x; } A tail-const S would look like this: struct TailConstS { const(int)* x; } That is, everything the struct points at remains const, but the actual data of the struct is now mutable. Note that TailConstS and ConstS can implicitly convert. This is how arrays work. An array T[] is really: struct Array { size_t length; T* ptr; } A tail-const array const(T)[] is really: struct TailConstArray { size_t length; // mutable const(T)* ptr; // mutable, but points at const data } This property of implicit casting is what's needed to fully realize custom ranges and const together. The way to get it is to define tail-modifier syntax for all types, not just arrays and pointers. -Steve
Aug 03 2017
On Wed, Aug 02, 2017 at 11:06:03AM -0700, H. S. Teoh via Digitalmars-d-learn wrote: [...]auto byPair(AA)(inout(AA) aa) { alias Modifiers = std.traits.getModifiers!AA; struct Result { std.traits.ApplyModifiers!(Slot*, Modifiers) slot; ... // range methods here } return Result(aa); } Of course, getModifiers and ApplyModifiers are fictitious Phobos templates, but you get the idea.[...] Hmm, actually, they don't have to be fictitious; here's an actual, compilable example: struct Slot { Slot* next; string key; int value; } struct AA { Slot*[] slots; } auto byPair(AA)(AA aa) { import std.traits : QualifierOf; alias Qual = QualifierOf!AA; struct Result { Qual!(Slot)* slot; bool empty() { return slot is null; } auto front() { struct Front { Qual!string key; Qual!int value; } return Front(slot.key, slot.value); } void popFront() { slot = slot.next; } } return Result(aa.slots[0]); } unittest { AA aa; const(AA) constAa; immutable(AA) immAa; auto mutPair = aa.byPair; static assert(is(typeof(mutPair.front.value) == int)); auto constPair = constAa.byPair; static assert(is(typeof(constPair.front.value) == const(int))); auto immPair = immAa.byPair; static assert(is(typeof(immPair.front.value) == immutable(int))); } T -- If Java had true garbage collection, most programs would delete themselves upon execution. -- Robert Sewell
Aug 02 2017