www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Is this a bug in std.typecons.Tuple.slice?

reply Saurabh Das <saurabh.das gmail.com> writes:
This code:

void main()
{
     import std.typecons;
     auto tp = tuple!("a", "b", "c")(10, false, "hello");

     auto u0 = tp.slice!(0, tp.length);
     auto u1 = tp.slice!(1, tp.length);
     auto u2 = tp.slice!(2, tp.length);

     static assert(is(typeof(u0) == Tuple!(int, "a", bool, "b", 
string, "c")));
     static assert(is(typeof(u1) == Tuple!(bool, "b", string, 
"c")));
     static assert(is(typeof(u2) == Tuple!(string, "c")));

     assert(u2.c == "hello");
     assert(u0.c == "hello");
     assert(u1.c == "hello");    // This assert fails. Why?
}

core.exception.AssertError erasetype.d(16): Assertion failure
----------------
4   erasetype                           0x0000000100ce8128 
_d_assert + 104
5   erasetype                           0x0000000100cd12fe void 
erasetype.__assert(int) + 38
6   erasetype                           0x0000000100cd12aa _Dmain 
+ 250
7   erasetype                           0x0000000100cf7297 
D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZ9__lambda1MFZv + 39
8   erasetype                           0x0000000100cf71cf void 
rt.dmain2._d_run_main(int, char**, extern (C) int 
function(char[][])*).tryExec(scope void delegate()) + 55
9   erasetype                           0x0000000100cf723c void 
rt.dmain2._d_run_main(int, char**, extern (C) int 
function(char[][])*).runAll() + 44
10  erasetype                           0x0000000100cf71cf void 
rt.dmain2._d_run_main(int, char**, extern (C) int 
function(char[][])*).tryExec(scope void delegate()) + 55
11  erasetype                           0x0000000100cf7121 
_d_run_main + 497
12  erasetype                           0x0000000100cd133f main + 
15
13  libdyld.dylib                       0x00007fff8a1345c8 start 
+ 0
14  ???                                 0x0000000000000000 0x0 + 0

Why does 'u1' behave differently? I'm thinking it's a bug, but I 
haven't worked much with tuples, so maybe I'm missing something?

Thanks,
Saurabh
Feb 04 2016
parent reply Saurabh Das <saurabh.das gmail.com> writes:
On Thursday, 4 February 2016 at 12:28:39 UTC, Saurabh Das wrote:

 This code:
 [...]


Update: Simplified, this also doesn't work:

void main()
{
     import std.typecons;
     auto tp = tuple(10, false, "hello");

     auto u0 = tp.slice!(0, tp.length);
     auto u1 = tp.slice!(1, tp.length);
     auto u2 = tp.slice!(2, tp.length);

     static assert(is(typeof(u0) == Tuple!(int, bool, string)));
     static assert(is(typeof(u1) == Tuple!(bool, string)));
     static assert(is(typeof(u2) == Tuple!(string)));

     assert(u2[0] == "hello");
     assert(u0[2] == "hello");
     assert(u1[1] == "hello");    // This fails
}

rdmd erasetype.d
core.exception.AssertError erasetype.d(16): Assertion failure

Any ideas?
Feb 04 2016
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Thu, 04 Feb 2016 15:17:54 +0000
schrieb Saurabh Das <saurabh.das gmail.com>:


 On Thursday, 4 February 2016 at 12:28:39 UTC, Saurabh Das wrote:

 This code:
 [...]


 
 Update: Simplified, this also doesn't work:
 
 void main()
 {
      import std.typecons;
      auto tp = tuple(10, false, "hello");
 
      auto u0 = tp.slice!(0, tp.length);
      auto u1 = tp.slice!(1, tp.length);
      auto u2 = tp.slice!(2, tp.length);
 
      static assert(is(typeof(u0) == Tuple!(int, bool, string)));
      static assert(is(typeof(u1) == Tuple!(bool, string)));
      static assert(is(typeof(u2) == Tuple!(string)));
 
      assert(u2[0] == "hello");
      assert(u0[2] == "hello");
      assert(u1[1] == "hello");    // This fails
 }
 
 rdmd erasetype.d
 core.exception.AssertError erasetype.d(16): Assertion failure
 
 Any ideas?


Yes this is a clear bug, I'll report a bug and post the issue
number later.

-- 
Marco
Feb 04 2016
parent reply Marco Leise <Marco.Leise gmx.de> writes:
https://issues.dlang.org/show_bug.cgi?id=15645
Feb 04 2016
next sibling parent Saurabh Das <saurabh.das gmail.com> writes:
On Thursday, 4 February 2016 at 17:52:16 UTC, Marco Leise wrote:

 https://issues.dlang.org/show_bug.cgi?id=15645


Thank you.

I understood why this is happening from your explanation in the 
bug report.
Feb 04 2016
prev sibling parent reply Saurabh Das <saurabh.das gmail.com> writes:
On Thursday, 4 February 2016 at 17:52:16 UTC, Marco Leise wrote:

 https://issues.dlang.org/show_bug.cgi?id=15645


Is this a possible fixed implementation? :

          property
         Tuple!(sliceSpecs!(from, to)) slice(size_t from, size_t 
to)()  trusted const
         if (from <= to && to <= Types.length)
         {
             auto sliceMixinGenerator()
             {
                 string rv;
                 for(auto i=from; i<to; ++i)
                 {
                     import std.conv : to;
                     rv ~= "returnValue[" ~ (i-from).to!string ~ 
"]=field[" ~ i.to!string ~"];";
                 }
                 return rv;
             }
             alias ReturnType = typeof(return);
             ReturnType returnValue;
             mixin(sliceMixinGenerator());
             return returnValue;
         }

         ///
         unittest
         {
             Tuple!(int, string, float, double) a;
             a[1] = "abc";
             a[2] = 4.5;
             auto s = a.slice!(1, 3);
             static assert(is(typeof(s) == Tuple!(string, float)));
             assert(s[0] == "abc" && s[1] == 4.5);

             Tuple!(int, int, long) b;
             b[1] = 42;
             b[2] = 101;
             auto t = b.slice!(1, 3);
             static assert(is(typeof(t) == Tuple!(int, long)));
             assert(t[0] == 42 && t[1] == 101);
         }

I'm unsure about:
1. Removing 'ref' from the return type
2. Adding 'const' to the function signature
3. Is the new implementation less efficient for correctly aligned 
tuples?
Feb 04 2016
next sibling parent Saurabh Das <saurabh.das gmail.com> writes:
On Friday, 5 February 2016 at 05:18:01 UTC, Saurabh Das wrote:

 [...]


PS: Additionally, ' trusted' can now be substituted with ' safe'.
Feb 04 2016
prev sibling parent reply Saurabh Das <saurabh.das gmail.com> writes:
On Friday, 5 February 2016 at 05:18:01 UTC, Saurabh Das wrote:
[...]

Apologies for spamming. This is an improved implementation:

          property
         Tuple!(sliceSpecs!(from, to)) slice(size_t from, size_t 
to)()  safe const
         if (from <= to && to <= Types.length)
         {
             return typeof(return)(field[from .. to]);
         }

         ///
         unittest
         {
             Tuple!(int, string, float, double) a;
             a[1] = "abc";
             a[2] = 4.5;
             auto s = a.slice!(1, 3);
             static assert(is(typeof(s) == Tuple!(string, float)));
             assert(s[0] == "abc" && s[1] == 4.5);

             Tuple!(int, int, long) b;
             b[1] = 42;
             b[2] = 101;
             auto t = b.slice!(1, 3);
             static assert(is(typeof(t) == Tuple!(int, long)));
             assert(t[0] == 42 && t[1] == 101);
         }

These questions still remain:

 1. Removing 'ref' from the return type
 2. Adding 'const' to the function signature
 3. Is the new implementation less efficient for correctly 
 aligned tuples?


4.  trusted ->  safe?
Feb 04 2016
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Fri, 05 Feb 2016 05:31:15 +0000
schrieb Saurabh Das <saurabh.das gmail.com>:


 On Friday, 5 February 2016 at 05:18:01 UTC, Saurabh Das wrote:
 [...]
 
 Apologies for spamming. This is an improved implementation:
 
           property
          Tuple!(sliceSpecs!(from, to)) slice(size_t from, size_t 
 to)()  safe const
          if (from <= to && to <= Types.length)
          {
              return typeof(return)(field[from .. to]);
          }
 
          ///
          unittest
          {
              Tuple!(int, string, float, double) a;
              a[1] = "abc";
              a[2] = 4.5;
              auto s = a.slice!(1, 3);
              static assert(is(typeof(s) == Tuple!(string, float)));
              assert(s[0] == "abc" && s[1] == 4.5);
 
              Tuple!(int, int, long) b;
              b[1] = 42;
              b[2] = 101;
              auto t = b.slice!(1, 3);
              static assert(is(typeof(t) == Tuple!(int, long)));
              assert(t[0] == 42 && t[1] == 101);
          }


That's quite concise. I like this. Though 'field' is now
called 'expand':
        // backwards compatibility
        alias field = expand;


 These questions still remain:

 1. Removing 'ref' from the return type




Must happen. 'ref' only worked because of the reinterpreting
cast which doesn't work in general. This will change the
semantics. Now the caller of 'slice' will deal with a whole
new copy of everything in the returned slice instead of a
narrower view into the original data. But that's a needed
change to fix the bug.


 2. Adding 'const' to the function signature




Hmm. Since const is transitive this may add const to stuff
that wasn't const, like in a Tuple!(Object). When you call
const slice on that, you would get a Tuple!(const(Object)).
I would use inout, making it so that the tuple's original
constness is propagated to the result.

I.e.:  property inout(Tuple!(sliceSpecs!(from, to)))
slice(size_t from, size_t to)()  safe inout


 3. Is the new implementation less efficient for correctly 
 aligned tuples?




Yes, the previous one just added a compile-time known offset
to the "this"-pointer. That's _one_ assembly instruction after
inlining and optimization.
The new one makes a copy of every field. On struct fields this
can call the copy constructor "this(this)" which is used for
example in reference counting to preform an increment for the
copy.
On "plain old data" it would simply copy the bit patterns. But
that's obviously still less efficient than adding an offset to
the pointer.
You need two methods if you want to offer the best of both
worlds. As soon as your function does not return a pointer or
has 'ref' on it, the compiler will provide memory on the stack
to hold the result and a copy will occur.
That said, sufficiently smart compilers can analyze what's
happening and come to the conclusion that when after the copy,
the original is no longer used, they can be merged.


 4.  trusted ->  safe?


Sounds good, but be aware of the mentioned implications with
"this(this)". Copy constructors often need to do unsafe
things, so  safe here would disallow them in Tuples.
On the other hand recent versions of the front-end should
infer attributes for templates, so you can generally omit them
and "let the Tuple decide". This mechanism also already adds
 nogc, pure, nothrow as possible in both the original and your
implementation.
(The original code only had  trusted on it because the
compiler would always infer the safety as  system due to the
pointer casts and  system is close to intolerable for a
"high-level" functional programming feature such as tuples.
The other attributes should have been inferred correctly.)

-- 
Marco
Feb 05 2016
next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Friday, 5 February 2016 at 19:16:11 UTC, Marco Leise wrote:

 1. Removing 'ref' from the return type




 Must happen. 'ref' only worked because of the reinterpreting 
 cast which doesn't work in general. This will change the 
 semantics. Now the caller of 'slice' will deal with a whole new 
 copy of everything in the returned slice instead of a narrower 
 view into the original data. But that's a needed change to fix 
 the bug.


Actually, it's not: 
https://github.com/D-Programming-Language/phobos/pull/3973

All that is required is to include a little padding at the 
beginning of the slice struct to make the alignments match.
Feb 05 2016
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Sat, 06 Feb 2016 04:28:17 +0000
schrieb tsbockman <thomas.bockman gmail.com>:


 On Friday, 5 February 2016 at 19:16:11 UTC, Marco Leise wrote:

 1. Removing 'ref' from the return type




 Must happen. 'ref' only worked because of the reinterpreting 
 cast which doesn't work in general. This will change the 
 semantics. Now the caller of 'slice' will deal with a whole new 
 copy of everything in the returned slice instead of a narrower 
 view into the original data. But that's a needed change to fix 
 the bug.


 
 Actually, it's not: 
 https://github.com/D-Programming-Language/phobos/pull/3973
 
 All that is required is to include a little padding at the 
 beginning of the slice struct to make the alignments match.


I don't want to sound dismissive, but when that thought came
to my mind I considered it unacceptable that the type of
Tuple!(int, bool, string).slice(1, 3) would be something
different than Tuple!(bool, string). In your case
Tuple!(TuplePad!4LU, bool, string). That's just a matter of
least surprise when comparing the types.

I'll let others decide, since I never used tuple slices.

-- 
Marco
Feb 05 2016
next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Saturday, 6 February 2016 at 06:34:05 UTC, Marco Leise wrote:

 I don't want to sound dismissive, but when that thought came
 to my mind I considered it unacceptable that the type of
 Tuple!(int, bool, string).slice(1, 3) would be something
 different than Tuple!(bool, string). In your case
 Tuple!(TuplePad!4LU, bool, string). That's just a matter of
 least surprise when comparing the types.

 I'll let others decide, since I never used tuple slices.


I'm not sure which approach is ultimately better, but aside from 
the performance implications, your "needed change" could break a 
lot of valid code in the wild - or it might break none; it really 
just depends on whether anyone actually *uses* the `ref`-ness of 
the `Tuple.slice` return type.

(It appears that Phobos, at least, does not. But there is no 
guarantee that the rest of the world is using `Tuple` only in the 
ways that Phobos does.)

Leaving aside bizarre meta-programming stuff (because then 
*anything* is a breaking change), my PR does not break any code, 
except that which was already broken: the type of the slice is 
only different in those cases where it *has* to be, for alignment 
reasons; otherwise it remains the same as it was before.
Feb 05 2016
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Sat, 06 Feb 2016 07:57:08 +0000
schrieb tsbockman <thomas.bockman gmail.com>:


 On Saturday, 6 February 2016 at 06:34:05 UTC, Marco Leise wrote:

 I don't want to sound dismissive, but when that thought came
 to my mind I considered it unacceptable that the type of
 Tuple!(int, bool, string).slice(1, 3) would be something
 different than Tuple!(bool, string). In your case
 Tuple!(TuplePad!4LU, bool, string). That's just a matter of
 least surprise when comparing the types.

 I'll let others decide, since I never used tuple slices.


 
 I'm not sure which approach is ultimately better, but aside from 
 the performance implications, your "needed change" could break a 
 lot of valid code in the wild - or it might break none; it really 
 just depends on whether anyone actually *uses* the `ref`-ness of 
 the `Tuple.slice` return type.


True.
 

 (It appears that Phobos, at least, does not. But there is no 
 guarantee that the rest of the world is using `Tuple` only in the 
 ways that Phobos does.)
 Leaving aside bizarre meta-programming stuff (because then 
 *anything* is a breaking change), my PR does not break any code, 
 except that which was already broken: the type of the slice is 
 only different in those cases where it *has* to be, for alignment 
 reasons; otherwise it remains the same as it was before.


I understand that. We just have a different perspective on the
problem. Your priorities:

- don't break what's not broken
- .slice! lends on opSlice and should return by ref

My priorities:

- type of .slice! should be as if constructing with same
  values from scratch
- keep code additions in Phobos to a minimum

Why do I insist on the return type? Because surprisingly
simple code breaks if it doesn't match. Not everything can be
covered by runtime conversions in D. It still took me a while
to come up with something obvious:

	uint[Tuple!(uint, ulong)] hash;

	auto tup = tuple(1u, 2u, 3UL);

	hash[tup.slice!(1, 3)] = tup[0];

                     compiles?  works?
original Tuple     : yes        no
Saurabh Das changes: yes        yes
your changes       : no         no

What I like most about your proposal is that it doesn't break
any existing code that wasn't broken before. That can't be
emphasized enough.

-- 
Marco
Feb 06 2016
next sibling parent tsbockman <thomas.bockman gmail.com> writes:
On Sunday, 7 February 2016 at 02:11:15 UTC, Marco Leise wrote:

 Why do I insist on the return type? Because surprisingly simple 
 code breaks if it doesn't match. Not everything can be covered 
 by runtime conversions in D. It still took me a while to come 
 up with something obvious:

 	uint[Tuple!(uint, ulong)] hash;

 	auto tup = tuple(1u, 2u, 3UL);

 	hash[tup.slice!(1, 3)] = tup[0];

                      compiles?  works?
 original Tuple     : yes        no
 Saurabh Das changes: yes        yes
 your changes       : no         no


Thank you for the example.

If multiple alias this ever makes it into the language (see 
https://wiki.dlang.org/DIP66 and 
https://github.com/D-Programming-Language/dmd/pull/3998), this 
could be fixed quite easily. But, I do not see any way to fix it 
with the tools currently available.
Feb 06 2016
prev sibling next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Sunday, 7 February 2016 at 02:11:15 UTC, Marco Leise wrote:

 I understand that. We just have a different perspective on the 
 problem. Your priorities:

 - don't break what's not broken
 - .slice! lends on opSlice and should return by ref

 My priorities:

 - type of .slice! should be as if constructing with same
   values from scratch
 - keep code additions in Phobos to a minimum

 Why do I insist on the return type? Because surprisingly simple 
 code breaks if it doesn't match. Not everything can be covered 
 by runtime conversions in D.


I think the key question is, do users care about being able to 
modify the original `Tuple` instance indirectly through `slice`?

If yes, then the only workable solutions I can see are:

1) My current proposal (insert a hidden padding member at the 
beginning of the slice `Tuple`)

2) Don't return a `Tuple` at all - return a dedicated 
`TupleSlice` type that is implicitly convertible to `Tuple`. This 
approach would work with the example you came up with, but 
implementing `TupleSlice` well could be very complex, I think.

If not, then I have no fundamental objection to Saurabh Das' 
approach, although I think the PR needs work.

We should start a new thread in "General" to ask whether people 
care about the `ref`-ness of `Tuple` slices is really the 
deciding factor.
Feb 06 2016
next sibling parent tsbockman <thomas.bockman gmail.com> writes:
On Sunday, 7 February 2016 at 02:51:49 UTC, tsbockman wrote:

 We should start a new thread in "General" to ask whether people 
 care about the `ref`-ness of `Tuple` slices is really the 
 deciding factor.


I made a poll: 
http://forum.dlang.org/post/inswmiscuqirkhfqlhtd forum.dlang.org
Feb 06 2016
prev sibling parent Saurabh Das <saurabh.das gmail.com> writes:
On Sunday, 7 February 2016 at 02:51:49 UTC, tsbockman wrote:

 On Sunday, 7 February 2016 at 02:11:15 UTC, Marco Leise wrote:

 I understand that. We just have a different perspective on the 
 problem. Your priorities:

 - don't break what's not broken
 - .slice! lends on opSlice and should return by ref

 My priorities:

 - type of .slice! should be as if constructing with same
   values from scratch
 - keep code additions in Phobos to a minimum

 Why do I insist on the return type? Because surprisingly 
 simple code breaks if it doesn't match. Not everything can be 
 covered by runtime conversions in D.


 I think the key question is, do users care about being able to 
 modify the original `Tuple` instance indirectly through `slice`?

 If yes, then the only workable solutions I can see are:

 1) My current proposal (insert a hidden padding member at the 
 beginning of the slice `Tuple`)

 2) Don't return a `Tuple` at all - return a dedicated 
 `TupleSlice` type that is implicitly convertible to `Tuple`. 
 This approach would work with the example you came up with, but 
 implementing `TupleSlice` well could be very complex, I think.

 If not, then I have no fundamental objection to Saurabh Das' 
 approach, although I think the PR needs work.


The PR definitely needs work - it was proposed to outline the 
direction. I haven't worked at all on Phobos and I am not yet 
knowledgeable on writing library-quality code in D.

I'm hoping to contribute back to Phobos this year - so pointing 
out as many flaws will help learn faster :) In particular - the 
inout problem in the PR - I'm not sure yet on how to fix that.

Thanks,
Saurabh


 We should start a new thread in "General" to ask whether people 
 care about the `ref`-ness of `Tuple` slices is really the 
 deciding factor.
Feb 06 2016
prev sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Sunday, 7 February 2016 at 02:11:15 UTC, Marco Leise wrote:

 What I like most about your proposal is that it doesn't break 
 any existing code that wasn't broken before. That can't be 
 emphasized enough.


Although I wish more than 3 people had voted in my poll, two of 
them did claim to need the `ref`-ness of `Tuple.slice`, so I 
don't think we can just ditch it. (I did not vote.)

If you guys want to add a return-by-value version, it should be 
treated as an enhancement, not a bug fix in my opinion.
Feb 08 2016
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Tue, 09 Feb 2016 00:38:10 +0000
schrieb tsbockman <thomas.bockman gmail.com>:


 On Sunday, 7 February 2016 at 02:11:15 UTC, Marco Leise wrote:

 What I like most about your proposal is that it doesn't break 
 any existing code that wasn't broken before. That can't be 
 emphasized enough.


 
 Although I wish more than 3 people had voted in my poll, two of 
 them did claim to need the `ref`-ness of `Tuple.slice`, so I 
 don't think we can just ditch it. (I did not vote.)
 
 If you guys want to add a return-by-value version, it should be 
 treated as an enhancement, not a bug fix in my opinion.


As mentioned I never used the feature myself and wont vote
for one or the other. Three people with no source code to
exemplify current use of .slice! is indeed not much to base
decisions on and both fixes yield unexpected results in
different contexts that warrant bug reports.

-- 
Marco
Feb 08 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Tuesday, 9 February 2016 at 06:22:55 UTC, Marco Leise wrote:

 As mentioned I never used the feature myself and wont vote for 
 one or the other. Three people with no source code to exemplify 
 current use of .slice! is indeed not much to base decisions 
 on...


The mere fact that all I had to do to find people who use and 
care about the `ref`-ness of `Tuple.slice` was ask, and then wait 
a few hours, strongly suggests that there are other such people 
among the D user base.

When faced with a judgment call like this, we really ought to err 
on the side of maintaining backwards compatibility - especially 
since this does not preclude adding a separate by-value version 
of `Tuple.slice`, as well. It was going to need a new name anyway.
Feb 09 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 9 February 2016 at 08:35:21 UTC, tsbockman wrote:

 When faced with a judgment call like this, we really ought to 
 err on the side of maintaining backwards compatibility - 
 especially since this does not preclude adding a separate 
 by-value version of `Tuple.slice`, as well. It was going to 
 need a new name anyway.


I suggest lobbying for proper builtin tuple support. IMO one 
shouldn't be able to take the reference of a tuple, to ensure 
that it can be kept in registers. Modern desktop CPUs have maybe 
512 bytes of register space. In most cases a tuple will be within 
8 bytes * 16 or something like that.
Feb 09 2016
next sibling parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Tuesday, 9 February 2016 at 09:05:58 UTC, Ola Fosheim Grøstad 
wrote:

 IMO one shouldn't be able to take the reference of a tuple, to 
 ensure that it can be kept in registers.


No need to restrict the language here, there's nothing stopping a 
decent compiler from storing tuples (actually _anything_) in 
registers, in some cases even if references are taken. I'm pretty 
sure LLVM can handle this.
Feb 09 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 9 February 2016 at 10:54:42 UTC, Marc Schütz wrote:

 No need to restrict the language here, there's nothing stopping 
 a decent compiler from storing tuples (actually _anything_) in 
 registers, in some cases even if references are taken. I'm 
 pretty sure LLVM can handle this.


If you don't restrict the language people will write code that 
the optimizer will struggle with.  LLVM can only handle what goes 
on within a compilation unit, and not if there are stores, 
because those are visible in other threads.

Tuples should be considered immutable constants (think functional 
programming), not in-memory storage.

Tuple's can serve as a good approximation to SIMD registers.
Feb 09 2016
parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Tuesday, 9 February 2016 at 11:38:14 UTC, Ola Fosheim Grøstad 
wrote:

 On Tuesday, 9 February 2016 at 10:54:42 UTC, Marc Schütz wrote:

 No need to restrict the language here, there's nothing 
 stopping a decent compiler from storing tuples (actually 
 _anything_) in registers, in some cases even if references are 
 taken. I'm pretty sure LLVM can handle this.


 If you don't restrict the language people will write code that 
 the optimizer will struggle with.


So what? Using that argument, you could just as well forbid 
taking the address of any variable. What's so special about 
tuples, in contrast to structs and arrays?


 LLVM can only handle what goes on within a compilation unit, 
 and not if there are stores, because those are visible in other 
 threads.

 Tuples should be considered immutable constants (think 
 functional programming), not in-memory storage.


Again, why?


 Tuple's can serve as a good approximation to SIMD registers.


What relation does that have to the above?
Feb 09 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 9 February 2016 at 13:43:16 UTC, Marc Schütz wrote:

 So what? Using that argument, you could just as well forbid 
 taking the address of any variable. What's so special about 
 tuples, in contrast to structs and arrays?


Some key common qualities for a tuple:

1. They are primarily used for multiple return values from 
functions.

2. Tuples use structural typing, not nominal typing.

3. They are identity-less. If you can take reference and compare, 
they no longer are identity-less.



 Tuples should be considered immutable constants (think 
 functional programming), not in-memory storage.


 Again, why?


Because that is how a tuple is commonly defined, for performance 
and semantic reasons.



 Tuple's can serve as a good approximation to SIMD registers.


 What relation does that have to the above?


You don't want to spill out SIMD registers to the stack if you 
can avoid it. You want to do the changes within the CPU pipeline, 
i.e. using copies (and register renaming).
Feb 09 2016
parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Tuesday, 9 February 2016 at 14:28:35 UTC, Ola Fosheim Grøstad 
wrote:

 On Tuesday, 9 February 2016 at 13:43:16 UTC, Marc Schütz wrote:

 So what? Using that argument, you could just as well forbid 
 taking the address of any variable. What's so special about 
 tuples, in contrast to structs and arrays?


 Some key common qualities for a tuple:

 1. They are primarily used for multiple return values from 
 functions.


As you said, primarily. There's no reason not to use them for 
something else.


 2. Tuples use structural typing, not nominal typing.


This has no relevance for the question at hand.


 3. They are identity-less. If you can take reference and 
 compare, they no longer are identity-less.


Like value types in general, nothing special about tuples here.


 Tuples should be considered immutable constants (think 
 functional programming), not in-memory storage.


 Again, why?


 Because that is how a tuple is commonly defined, for 
 performance and semantic reasons.


I believe it's more because the concept is more frequently used 
in functional programming languages, for which immutability is 
not surprising. Other languages do have mutable tuples, e.g. 
Swift and C++11 (std::tuple).


 Tuple's can serve as a good approximation to SIMD registers.


 What relation does that have to the above?


 You don't want to spill out SIMD registers to the stack if you 
 can avoid it. You want to do the changes within the CPU 
 pipeline, i.e. using copies (and register renaming).


As said above, wanting to avoid spilling is not a reason to 
disallow spilling. Besides, fixed-size arrays seem more similar 
to SIMD registers, and they don't have the restrictions you 
tuples to have.
Feb 09 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 9 February 2016 at 16:00:09 UTC, Marc Schütz wrote:

 2. Tuples use structural typing, not nominal typing.


 This has no relevance for the question at hand.


It does if you allow casting and to use tuples a types in 
aggregates. The language becomes less strongly typed.


 3. They are identity-less. If you can take reference and 
 compare, they no longer are identity-less.


 Like value types in general, nothing special about tuples here.


I don't know what you mean by that. D doesn't provide proper 
value types. If you can compare identities (address) then it is 
not a value, but an object.


 I believe it's more because the concept is more frequently used 
 in functional programming languages, for which immutability is 
 not surprising. Other languages do have mutable tuples, e.g. 
 Swift and C++11 (std::tuple).


C++ doesn't have real tuples. I don't know the details of Swift 
regarding tuples, but Swift is an OO language that does not aim 
for high performance or very strong typing.


 As said above, wanting to avoid spilling is not a reason to 
 disallow spilling. Besides, fixed-size arrays seem more similar 
 to SIMD registers, and they don't have the restrictions you 
 tuples to have.


Well, I disagree. There is very little reason to encourage people 
to use tuples for storage, you end up with a weaker typed 
language and less performant code.

(You can do various types of packing in registers too, depends on 
the CPU.)
Feb 09 2016
prev sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Tuesday, 9 February 2016 at 09:05:58 UTC, Ola Fosheim Grøstad 
wrote:

 I suggest lobbying for proper builtin tuple support.


Built-in tuple support would be great (although to my mind, 
mostly because the current syntax is clunky). But that is a 
long-term goal, and `Tuple.slice` is corrupting data *right now*.

Some sort of short-term fix should be merged in the next release 
of D.
Feb 09 2016
parent Saurabh Das <saurabh.das gmail.com> writes:
On Wednesday, 10 February 2016 at 00:24:56 UTC, tsbockman wrote:

 [...]
 `Tuple.slice` is corrupting data *right now*.

 Some sort of short-term fix should be merged in the next 
 release of D.


+1
Feb 09 2016
prev sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Saturday, 6 February 2016 at 06:34:05 UTC, Marco Leise wrote:

 [...]


I should also point out that, since there is no way to actually 
find out whether anyone is using the `ref`-ness of the return 
type in the wild, the approach that you and Saurabh Das are 
taking really ought to include changing the symbol name and 
deprecating the old one.

Otherwise you could introduce subtle bugs into previously valid 
code; not every significant effect of removing `ref` will cause 
an error message at compile time *or* run time - some will just 
silently change the behaviour of the program, which is awful.
Feb 06 2016
parent reply Saurabh Das <saurabh.das gmail.com> writes:
On Saturday, 6 February 2016 at 08:01:20 UTC, tsbockman wrote:

 On Saturday, 6 February 2016 at 06:34:05 UTC, Marco Leise wrote:

 [...]


 I should also point out that, since there is no way to actually 
 find out whether anyone is using the `ref`-ness of the return 
 type in the wild, the approach that you and Saurabh Das are 
 taking really ought to include changing the symbol name and 
 deprecating the old one.

 Otherwise you could introduce subtle bugs into previously valid 
 code; not every significant effect of removing `ref` will cause 
 an error message at compile time *or* run time - some will just 
 silently change the behaviour of the program, which is awful.


I think we should add a static assert to slice to ensure that the 
current implementation is not used in a case where the alignment 
doesn't match. This is better than failing without any warning.

We could add new (differently named) functions for slicing 
non-aligned tuples.

I agree that my approach of removing the ref may break existing 
code, so if introduced, it should be named differently.

I am not comfortable with tuple(42, true, "abc").slice(1, 3) 
being different in type from tuple(true, " abc").
Feb 06 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Saturday, 6 February 2016 at 08:47:01 UTC, Saurabh Das wrote:

 I think we should add a static assert to slice to ensure that 
 the current implementation is not used in a case where the 
 alignment doesn't match. This is better than failing without 
 any warning.


If we pursue the deprecation route, I agree that this is a 
necessary step.


 We could add new (differently named) functions for slicing 
 non-aligned tuples.

 I agree that my approach of removing the ref may break existing 
 code, so if introduced, it should be named differently.

 I am not comfortable with tuple(42, true, "abc").slice(1, 3) 
 being different in type from tuple(true, " abc").


Why? What practical problem does this cause?
Feb 06 2016
parent Marco Leise <Marco.Leise gmx.de> writes:
Am Sat, 06 Feb 2016 11:02:37 +0000
schrieb tsbockman <thomas.bockman gmail.com>:


 On Saturday, 6 February 2016 at 08:47:01 UTC, Saurabh Das wrote:

 I think we should add a static assert to slice to ensure that 
 the current implementation is not used in a case where the 
 alignment doesn't match. This is better than failing without 
 any warning.


 
 If we pursue the deprecation route, I agree that this is a 
 necessary step.


That would hurt the least, yes. It's more like a .dup with
start and end parameter then.


 We could add new (differently named) functions for slicing 
 non-aligned tuples.

 I agree that my approach of removing the ref may break existing 
 code, so if introduced, it should be named differently.

 I am not comfortable with tuple(42, true, "abc").slice(1, 3) 
 being different in type from tuple(true, " abc").


 
 Why? What practical problem does this cause?


For me it is mostly a gut feeling. Historically types
mimicking other types have often evoked trouble for example in
generic functions or concretely - if you look at my other post
- in D's AA implementation.

-- 
Marco
Feb 06 2016
prev sibling parent Saurabh Das <saurabh.das gmail.com> writes:
On Friday, 5 February 2016 at 19:16:11 UTC, Marco Leise wrote:

 Am Fri, 05 Feb 2016 05:31:15 +0000
 schrieb Saurabh Das <saurabh.das gmail.com>:

 [...]


That is enlightening. I have updated the PR at 
https://github.com/D-Programming-Language/phobos/pull/3975 to 
incorporate these changes.
Feb 05 2016