www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - TypeFunction example: ImplictConvTargets

reply Stefan Koch <uplink.coder googlemail.com> writes:
Hi,

I've posted an incomplete version of a semantic translation of 
ImplictConvTargets from a template into a type function.

After a few rather trivial fixes let me show you what the code 
looks like now.

---
alias type = alias;

// needed to avoid deeper changes ... in the future it may be 
unnecessary.
auto makeAliasArray(type[] types ...)
{
     return types;
}

enum basic_types = makeAliasArray(bool, ubyte, char, byte, 
ushort, wchar, short, uint, dchar, int, ulong, long);

type[] convTargets(type T)
{
     if (isBasicType(T))
         return basicTypeConvTargets(T);
     return null;
}

bool isBasicType(type T)
{
     foreach(t;basic_types)
     {
         if (is(T == t))
             return true;
     }
     return false;
}

type[] basicTypeConvTargets(type T)
{
     type[] targets;
     targets.length = basic_types.length;
     assert(isBasicType(T), "You may not call this function when 
you don't have a basic type ... (given: " ~ T.stringof ~ ")");
     size_t n = 0;
     foreach(t;basic_types)
     {
         if (is(T : t))
         {
             targets[n++] = t;
         }
     }
     return targets[0 .. n];
}
// 42 lines including whitespace and comments

pragma(msg, convTargets(long)); // outputs [(ulong), (long)]
---

And again here is the part of the template that we just 
re-implemented:
---
     static if (is(T == bool))
         alias ImplicitConversionTargets =
             AliasSeq!(byte, ubyte, short, ushort, int, uint, 
long, ulong, CentTypeList,
                        float, double, real, char, wchar, dchar);
     else static if (is(T == byte))
         alias ImplicitConversionTargets =
             AliasSeq!(short, ushort, int, uint, long, ulong, 
CentTypeList,
                        float, double, real, char, wchar, dchar);
     else static if (is(T == ubyte))
         alias ImplicitConversionTargets =
             AliasSeq!(short, ushort, int, uint, long, ulong, 
CentTypeList,
                        float, double, real, char, wchar, dchar);
     else static if (is(T == short))
         alias ImplicitConversionTargets =
             AliasSeq!(int, uint, long, ulong, CentTypeList, 
float, double, real);
     else static if (is(T == ushort))
         alias ImplicitConversionTargets =
             AliasSeq!(int, uint, long, ulong, CentTypeList, 
float, double, real);
     else static if (is(T == int))
         alias ImplicitConversionTargets =
             AliasSeq!(long, ulong, CentTypeList, float, double, 
real);
     else static if (is(T == uint))
         alias ImplicitConversionTargets =
             AliasSeq!(long, ulong, CentTypeList, float, double, 
real);
     else static if (is(T == long))
         alias ImplicitConversionTargets = AliasSeq!(float, 
double, real);
     else static if (is(T == ulong))
         alias ImplicitConversionTargets = AliasSeq!(float, 
double, real);
     // part omitted because we don't have ucent and cent in our 
list
     else static if (is(T == char))
         alias ImplicitConversionTargets =
             AliasSeq!(wchar, dchar, byte, ubyte, short, ushort,
                        int, uint, long, ulong, CentTypeList, 
float, double, real);
     else static if (is(T == wchar))
         alias ImplicitConversionTargets =
             AliasSeq!(dchar, short, ushort, int, uint, long, 
ulong, CentTypeList,
                        float, double, real);
     else static if (is(T == dchar))
         alias ImplicitConversionTargets =
             AliasSeq!(int, uint, long, ulong, CentTypeList, 
float, double, real);
     // 41 lines including white-space and comments (only that 
there is no white-space or comments)
---

I leave it up to you to decide which version is more 
understandable and extendable (should we ever get another basic 
type :))

As noted previously please do discuss!
Maybe you like the template version more?
Let me know.
Oct 05 2020
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 06/10/2020 12:44 AM, Stefan Koch wrote:

 I leave it up to you to decide which version is more understandable and 
 extendable (should we ever get another basic type :))


You forgot ucent and cent ;)
Oct 05 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 October 2020 at 11:52:06 UTC, rikki cattermole wrote:

 On 06/10/2020 12:44 AM, Stefan Koch wrote:

 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another 
 basic type :))


 You forgot ucent and cent ;)


You can't write those without making dmd error out.
And complain about ucent and cent not being implemented.
At least that used to be the case.

I also didn't write the floating types since, there's still a bug 
in the type function implementation that prevents those being 
turned into type expressions.
Oct 05 2020
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 October 2020 at 11:58:55 UTC, Stefan Koch wrote:

 On Monday, 5 October 2020 at 11:52:06 UTC, rikki cattermole 
 wrote:

 On 06/10/2020 12:44 AM, Stefan Koch wrote:

 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another 
 basic type :))


 You forgot ucent and cent ;)


 You can't write those without making dmd error out.
 And complain about ucent and cent not being implemented.
 At least that used to be the case.

 I also didn't write the floating types since, there's still a 
 bug in the type function implementation that prevents those 
 being turned into type expressions.


Yep ... it's still a problem.

test_alias_implconv.d(9): Error: cent and ucent types not 
implemented
test_alias_implconv.d(13):        called from here: isBasicType(T)
test_alias_implconv.d(47):        called from here: 
convTargets(cast(alias)(long))
test_alias_implconv.d(47):        called from here: 
isEqual(convTargets(cast(alias)(long)), 
makeAliasArray([cast(alias)(ulong), cast(alias)(long)][]))
test_alias_implconv.d(47):        while evaluating: static 
assert(isEqual(convTargets(cast(alias)(long)), 
makeAliasArray([cast(alias)(ulong), cast(alias)(long)][])))


This shows another advantage of type functions.
You get a call stack ;)
Oct 05 2020
prev sibling next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:

 Hi,

 [snip]

 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another basic 
 type :))

 As noted previously please do discuss!
 Maybe you like the template version more?
 Let me know.



It would be useful if these examples also include some kind of 
performance information as well.

Further, as far as I could tell from the previous thread, there 
were three potential approaches: type functions, templates, and 
reification.
Oct 05 2020
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 October 2020 at 12:46:29 UTC, jmh530 wrote:

 On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:

 Hi,

 [snip]

 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another 
 basic type :))

 As noted previously please do discuss!
 Maybe you like the template version more?
 Let me know.



 It would be useful if these examples also include some kind of 
 performance information as well.

 Further, as far as I could tell from the previous thread, there 
 were three potential approaches: type functions, templates, and 
 reification.


Well type function _are_ transparent type reification. With nice 
syntax, that mirrors the one that's used in the rest of the 
language to query types.

As for the approach of explicitly reifying types using templates 
perhaps Andrei could present the example.
(A complete example please, that compiles standalone without 
importing a library.)
Oct 05 2020
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/5/20 7:44 AM, Stefan Koch wrote:

 Hi,
 
 I've posted an incomplete version of a semantic translation of 
 ImplictConvTargets from a template into a type function.
 
 After a few rather trivial fixes let me show you what the code looks 
 like now.


[snip]

The existing implementation is ancient (e.g. predates std.meta) and 
certainly deserves a redoing with Filter, which turns it into a 3-liner 
(untested):

alias Integrals = AliasSeq!(byte, ubyte, short, ushort, int, uint, long, 
ulong, CentTypeList, float, double, real, char, wchar, dchar);
alias convertsTo(U) = is(T : U);
alias ImplicitConversionTargets(T) = Filter!(convertsTo, Integrals);

This would make for a nice refactoring PR and would provide a better 
baseline for comparison.

Things could be further simplified - I haven't looked in detail but it 
seems to me each integral converts to all integrals that are as large or 
larger. So if we keep a list of integrals sorted by size we could simply 
return slices from it instead of doing filtering.

Anyway, implementation size or difficulty is not the problem with 
ImplicitConversionTargets - it's completeness and correctness. Also the 
appropriate definition in the language, which now is scattered all over 
and probably less precise than it should. ImplicitConversionTargets 
currently doesn't handle alias this, functions and delegates wich 
co/contravariance, probably a few array conversions (I recall Per 
recently added or is about to add one), and some qualified structs.
Oct 05 2020
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 October 2020 at 12:50:39 UTC, Andrei Alexandrescu 
wrote:

 On 10/5/20 7:44 AM, Stefan Koch wrote:

 Hi,
 
 I've posted an incomplete version of a semantic translation of 
 ImplictConvTargets from a template into a type function.
 
 After a few rather trivial fixes let me show you what the code 
 looks like now.


 [snip]

 The existing implementation is ancient (e.g. predates std.meta) 
 and certainly deserves a redoing with Filter, which turns it 
 into a 3-liner (untested):

 alias Integrals = AliasSeq!(byte, ubyte, short, ushort, int, 
 uint, long, ulong, CentTypeList, float, double, real, char, 
 wchar, dchar);
 alias convertsTo(U) = is(T : U);
 alias ImplicitConversionTargets(T) = Filter!(convertsTo, 
 Integrals);



Please post the complete version.
Which does not import the stdlib.
Thanks.
Oct 05 2020
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 October 2020 at 12:50:39 UTC, Andrei Alexandrescu 
wrote:

 Anyway, implementation size or difficulty is not the problem 
 with ImplicitConversionTargets


In the context of this example it is.
If you want to discuss it an another context please start a new 
thread.
Oct 05 2020
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 October 2020 at 12:50:39 UTC, Andrei Alexandrescu 
wrote:


 The existing implementation is ancient (e.g. predates std.meta) 
 and certainly deserves a redoing with Filter, which turns it 
 into a 3-liner (untested):

 alias Integrals = AliasSeq!(byte, ubyte, short, ushort, int, 
 uint, long, ulong, CentTypeList, float, double, real, char, 
 wchar, dchar);
 alias convertsTo(U) = is(T : U);
 alias ImplicitConversionTargets(T) = Filter!(convertsTo, 
 Integrals);


This code does not work.
I don't even need to compile it to see that.
Oct 05 2020
parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 5 October 2020 at 20:57:04 UTC, Stefan Koch wrote:

 On Monday, 5 October 2020 at 12:50:39 UTC, Andrei Alexandrescu 
 wrote:


 The existing implementation is ancient (e.g. predates 
 std.meta) and certainly deserves a redoing with Filter, which 
 turns it into a 3-liner (untested):

 alias Integrals = AliasSeq!(byte, ubyte, short, ushort, int, 
 uint, long, ulong, CentTypeList, float, double, real, char, 
 wchar, dchar);
 alias convertsTo(U) = is(T : U);
 alias ImplicitConversionTargets(T) = Filter!(convertsTo, 
 Integrals);


 This code does not work.
 I don't even need to compile it to see that.


It has some simple mistakes, but the fundamental idea is sound. 
Here's a version that actually compiles:

import std.meta;

alias Numerics = AliasSeq!(byte, ubyte, short, ushort, int, uint, 
long, ulong, float, double, real, char, wchar, dchar);
enum convertsTo(T, U) = is(T : U);
alias ImplicitConversionTargets(T) = 
Filter!(ApplyLeft!(convertsTo, T), Numerics);

// prints: (int, uint, long, ulong, float, double, real, dchar)
pragma(msg, ImplicitConversionTargets!int);
// prints: (float, double, real)
pragma(msg, ImplicitConversionTargets!double);
Oct 05 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 October 2020 at 21:13:09 UTC, Paul Backus wrote:

 On Monday, 5 October 2020 at 20:57:04 UTC, Stefan Koch wrote:

 On Monday, 5 October 2020 at 12:50:39 UTC, Andrei Alexandrescu 
 wrote:


 [...]


 This code does not work.
 I don't even need to compile it to see that.


 It has some simple mistakes, but the fundamental idea is sound. 
 Here's a version that actually compiles:

 import std.meta;

 alias Numerics = AliasSeq!(byte, ubyte, short, ushort, int, 
 uint, long, ulong, float, double, real, char, wchar, dchar);
 enum convertsTo(T, U) = is(T : U);
 alias ImplicitConversionTargets(T) = 
 Filter!(ApplyLeft!(convertsTo, T), Numerics);

 // prints: (int, uint, long, ulong, float, double, real, dchar)
 pragma(msg, ImplicitConversionTargets!int);
 // prints: (float, double, real)
 pragma(msg, ImplicitConversionTargets!double);


Now post it with all transitive dependencies.
And we have a fair comparison.
Oct 05 2020
next sibling parent reply foobar <foo bar.com> writes:
On Monday, 5 October 2020 at 21:20:36 UTC, Stefan Koch wrote:

 On Monday, 5 October 2020 at 21:13:09 UTC, Paul Backus wrote:

 On Monday, 5 October 2020 at 20:57:04 UTC, Stefan Koch wrote:

 On Monday, 5 October 2020 at 12:50:39 UTC, Andrei 
 Alexandrescu wrote:


 [...]


 This code does not work.
 I don't even need to compile it to see that.


 It has some simple mistakes, but the fundamental idea is 
 sound. Here's a version that actually compiles:

 import std.meta;

 alias Numerics = AliasSeq!(byte, ubyte, short, ushort, int, 
 uint, long, ulong, float, double, real, char, wchar, dchar);
 enum convertsTo(T, U) = is(T : U);
 alias ImplicitConversionTargets(T) = 
 Filter!(ApplyLeft!(convertsTo, T), Numerics);

 // prints: (int, uint, long, ulong, float, double, real, dchar)
 pragma(msg, ImplicitConversionTargets!int);
 // prints: (float, double, real)
 pragma(msg, ImplicitConversionTargets!double);


 Now post it with all transitive dependencies.
 And we have a fair comparison.


Post your code with all changes to the language and compiler. 
Then we have a fair comparison.
Oct 05 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 6 October 2020 at 00:25:40 UTC, foobar wrote:

 On Monday, 5 October 2020 at 21:20:36 UTC, Stefan Koch wrote:

 On Monday, 5 October 2020 at 21:13:09 UTC, Paul Backus wrote:

 On Monday, 5 October 2020 at 20:57:04 UTC, Stefan Koch wrote:

 On Monday, 5 October 2020 at 12:50:39 UTC, Andrei 
 Alexandrescu wrote:


 [...]


 This code does not work.
 I don't even need to compile it to see that.


 It has some simple mistakes, but the fundamental idea is 
 sound. Here's a version that actually compiles:

 import std.meta;

 alias Numerics = AliasSeq!(byte, ubyte, short, ushort, int, 
 uint, long, ulong, float, double, real, char, wchar, dchar);
 enum convertsTo(T, U) = is(T : U);
 alias ImplicitConversionTargets(T) = 
 Filter!(ApplyLeft!(convertsTo, T), Numerics);

 // prints: (int, uint, long, ulong, float, double, real, 
 dchar)
 pragma(msg, ImplicitConversionTargets!int);
 // prints: (float, double, real)
 pragma(msg, ImplicitConversionTargets!double);


 Now post it with all transitive dependencies.
 And we have a fair comparison.


 Post your code with all changes to the language and compiler. 
 Then we have a fair comparison.


Actually no.
The compiler changes don't affect the user.
They're for a small number of people to know about and support.
Typefunctions and CTFE are together still much less complicated 
than the template system is.

I have nothing to hide though here it is
https://github.com/dlang/dmd/compare/master...UplinkCoder:talias_master

650 lines of rather clean code which can in the future be 
factored with the respective semantic routines.
Oct 05 2020
parent reply foobar <foo bar.com> writes:
On Tuesday, 6 October 2020 at 03:50:11 UTC, Stefan Koch wrote:

 On Tuesday, 6 October 2020 at 00:25:40 UTC, foobar wrote:

 On Monday, 5 October 2020 at 21:20:36 UTC, Stefan Koch wrote:

 On Monday, 5 October 2020 at 21:13:09 UTC, Paul Backus wrote:

 On Monday, 5 October 2020 at 20:57:04 UTC, Stefan Koch wrote:

 On Monday, 5 October 2020 at 12:50:39 UTC, Andrei 
 Alexandrescu wrote:


 [...]


 This code does not work.
 I don't even need to compile it to see that.


 It has some simple mistakes, but the fundamental idea is 
 sound. Here's a version that actually compiles:

 import std.meta;

 alias Numerics = AliasSeq!(byte, ubyte, short, ushort, int, 
 uint, long, ulong, float, double, real, char, wchar, dchar);
 enum convertsTo(T, U) = is(T : U);
 alias ImplicitConversionTargets(T) = 
 Filter!(ApplyLeft!(convertsTo, T), Numerics);

 // prints: (int, uint, long, ulong, float, double, real, 
 dchar)
 pragma(msg, ImplicitConversionTargets!int);
 // prints: (float, double, real)
 pragma(msg, ImplicitConversionTargets!double);


 Now post it with all transitive dependencies.
 And we have a fair comparison.


 Post your code with all changes to the language and compiler. 
 Then we have a fair comparison.


 Actually no.
 The compiler changes don't affect the user.
 They're for a small number of people to know about and support.
 Typefunctions and CTFE are together still much less complicated 
 than the template system is.

 I have nothing to hide though here it is
 https://github.com/dlang/dmd/compare/master...UplinkCoder:talias_master

 650 lines of rather clean code which can in the future be 
 factored with the respective semantic routines.


Actuallly yes.
The language changes affect the user. This is a large change to 
the language which puts back in the compiler what we do in 
libraries. Whoop-de-do.
Do type functions do anything new?
Oct 06 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 6 October 2020 at 11:34:10 UTC, foobar wrote:

 On Tuesday, 6 October 2020 at 03:50:11 UTC, Stefan Koch wrote:

 On Tuesday, 6 October 2020 at 00:25:40 UTC, foobar wrote:

 On Monday, 5 October 2020 at 21:20:36 UTC, Stefan Koch wrote:

 On Monday, 5 October 2020 at 21:13:09 UTC, Paul Backus wrote:

 On Monday, 5 October 2020 at 20:57:04 UTC, Stefan Koch 
 wrote:

 On Monday, 5 October 2020 at 12:50:39 UTC, Andrei 
 Alexandrescu wrote:


 [...]


 This code does not work.
 I don't even need to compile it to see that.


 It has some simple mistakes, but the fundamental idea is 
 sound. Here's a version that actually compiles:

 import std.meta;

 alias Numerics = AliasSeq!(byte, ubyte, short, ushort, int, 
 uint, long, ulong, float, double, real, char, wchar, dchar);
 enum convertsTo(T, U) = is(T : U);
 alias ImplicitConversionTargets(T) = 
 Filter!(ApplyLeft!(convertsTo, T), Numerics);

 // prints: (int, uint, long, ulong, float, double, real, 
 dchar)
 pragma(msg, ImplicitConversionTargets!int);
 // prints: (float, double, real)
 pragma(msg, ImplicitConversionTargets!double);


 Now post it with all transitive dependencies.
 And we have a fair comparison.


 Post your code with all changes to the language and compiler. 
 Then we have a fair comparison.


 Actually no.
 The compiler changes don't affect the user.
 They're for a small number of people to know about and support.
 Typefunctions and CTFE are together still much less 
 complicated than the template system is.

 I have nothing to hide though here it is
 https://github.com/dlang/dmd/compare/master...UplinkCoder:talias_master

 650 lines of rather clean code which can in the future be 
 factored with the respective semantic routines.


 Actuallly yes.
 The language changes affect the user. This is a large change to 
 the language which puts back in the compiler what we do in 
 libraries. Whoop-de-do.
 Do type functions do anything new?


It only gives access to what the compiler has to do anyway.
It gives you an interface to what must exist within.
You could even say this makes it easier to provide a library 
implementation by using the compiler as a built-in library.
No, type functions don't do anything original they mirror how 
type manipulation works within templates providing a familiar and 
usable interface.
It is an explicit goal of mine to have type functions look just 
like any other D code.
Oct 06 2020
parent reply foobar <foo bar.com> writes:
On Tuesday, 6 October 2020 at 12:35:15 UTC, Stefan Koch wrote:

 On Tuesday, 6 October 2020 at 11:34:10 UTC, foobar wrote:

 On Tuesday, 6 October 2020 at 03:50:11 UTC, Stefan Koch wrote:

 [...]


 Actuallly yes.
 The language changes affect the user. This is a large change 
 to the language which puts back in the compiler what we do in 
 libraries. Whoop-de-do.
 Do type functions do anything new?


 It only gives access to what the compiler has to do anyway.
 It gives you an interface to what must exist within.
 You could even say this makes it easier to provide a library 
 implementation by using the compiler as a built-in library.
 No, type functions don't do anything original they mirror how 
 type manipulation works within templates providing a familiar 
 and usable interface.
 It is an explicit goal of mine to have type functions look just 
 like any other D code.


Last time I looked D was a small language but very powerful. It 
innovated and did things you could not do in C++. Now this forum 
has been taken over by beginners who want a big language which 
does the same as before, only differently. Yawn.
Oct 06 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/6/20 1:51 PM, foobar wrote:

 On Tuesday, 6 October 2020 at 12:35:15 UTC, Stefan Koch wrote:

 On Tuesday, 6 October 2020 at 11:34:10 UTC, foobar wrote:

 On Tuesday, 6 October 2020 at 03:50:11 UTC, Stefan Koch wrote:

 [...]


 Actuallly yes.
 The language changes affect the user. This is a large change to the 
 language which puts back in the compiler what we do in libraries. 
 Whoop-de-do.
 Do type functions do anything new?


 It only gives access to what the compiler has to do anyway.
 It gives you an interface to what must exist within.
 You could even say this makes it easier to provide a library 
 implementation by using the compiler as a built-in library.
 No, type functions don't do anything original they mirror how type 
 manipulation works within templates providing a familiar and usable 
 interface.
 It is an explicit goal of mine to have type functions look just like 
 any other D code.


 
 Last time I looked D was a small language but very powerful. It 
 innovated and did things you could not do in C++. Now this forum has 
 been taken over by beginners who want a big language which does the same 
 as before, only differently. Yawn.


Everything you just said is wrong.

-Steve
Oct 06 2020
parent foobar <foo bar.com> writes:
On Tuesday, 6 October 2020 at 18:17:51 UTC, Steven Schveighoffer 
wrote:

 On 10/6/20 1:51 PM, foobar wrote:

 On Tuesday, 6 October 2020 at 12:35:15 UTC, Stefan Koch wrote:

 [...]


 
 Last time I looked D was a small language but very powerful. 
 It innovated and did things you could not do in C++. Now this 
 forum has been taken over by beginners who want a big language 
 which does the same as before, only differently. Yawn.


 Everything you just said is wrong.

 -Steve


Said Stefan Koch:


No, type functions don't do anything original they mirror how 
type manipulation works within templates providing a familiar 
and usable interface.


Everything YOU just said is wrong.
Oct 06 2020
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/5/2020 2:20 PM, Stefan Koch wrote:

 Now post it with all transitive dependencies.


I'm not sure how transitive dependencies applies here. A illuminating test case 
would be helpful.
Oct 05 2020
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 6 October 2020 at 01:12:01 UTC, Walter Bright wrote:

 On 10/5/2020 2:20 PM, Stefan Koch wrote:

 Now post it with all transitive dependencies.


 I'm not sure how transitive dependencies applies here. A 
 illuminating test case would be helpful.


Transitive dependencies apply, because it shows how much work has 
to be done in the library.
As opposed to the type-function code which works without library 
support (except for the druntime array handling of course)
Oct 05 2020
prev sibling next sibling parent reply claptrap <clap trap.com> writes:
On Tuesday, 6 October 2020 at 01:12:01 UTC, Walter Bright wrote:

 On 10/5/2020 2:20 PM, Stefan Koch wrote:

 Now post it with all transitive dependencies.


 I'm not sure how transitive dependencies applies here. A 
 illuminating test case would be helpful.


Because Stefan didnt rely on external code, he showed *all* the 
code needed, so a counter example should have been the same. 
Since the point is to compare the two *language* features, not 
compare library calls. I mean it's like someone posted a new sort 
algorithm and Andrei replied with....

import std.algorithm;
result = sort!(foo);
Oct 06 2020
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 6 October 2020 at 08:35:20 UTC, claptrap wrote:

 Because Stefan didnt rely on external code, he showed *all* the 
 code needed, so a counter example should have been the same. 
 Since the point is to compare the two *language* features, not 
 compare library calls.


Language features are a means to an end.

If the Phobos version compiles faster, uses less memory, and has 
the same result in the binary, it is a victory, even if it as 
significantly more code.

I'm skeptical of the type functions, but interested. They might 
help with a real problem. But they need to actually win on the 
end results, not against artificial strawmen.
Oct 06 2020
next sibling parent reply Bruce Carneal <bcarneal gmail.com> writes:
On Tuesday, 6 October 2020 at 12:28:55 UTC, Adam D. Ruppe wrote:

 On Tuesday, 6 October 2020 at 08:35:20 UTC, claptrap wrote:

 Because Stefan didnt rely on external code, he showed *all* 
 the code needed, so a counter example should have been the 
 same. Since the point is to compare the two *language* 
 features, not compare library calls.


 Language features are a means to an end.

 If the Phobos version compiles faster, uses less memory, and 
 has the same result in the binary, it is a victory, even if it 
 as significantly more code.


I disagree.  I believe we should aim for the simplest code that 
admits the desired performance.  Simplicity is especially 
important at lower levels where the benefits compound.  We're 
working to factor out complexity for today's programmers and 
tomorrows.


 I'm skeptical of the type functions, but interested. They might 
 help with a real problem.


Well, what is a "real problem"?  If you mean "something that can 
not be done outside the compiler" then nothing is a real problem 
(note CTFE and mixins).  If you mean "something that can not be 
done near optimally outside the compiler" then we're in violent 
agreement.


 But they need to actually win on the end results, not against 
 artificial strawmen.


Agree.

Note that performance is relatively easy to test whereas it's 
harder to compare aggregate debugability over time.  My proxies 
include readability, teachability, independence (fewer layers, 
fewer dependencies), and the amount of the correctness "proof" 
(debugging) handled automatically.

If performance is similar we should opt for better aggregate 
debugability as we, the community, understand it.  My current 
understanding is that type functions are significantly better in 
this regard.

Finally, please note that while I disagree with you on some 
particulars here I am a big fan.  I enjoyed your book, and admire 
the tremendous amount of cheerful quality help that you offer in 
the forums and discord.
Oct 06 2020
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 6 October 2020 at 15:24:31 UTC, Bruce Carneal wrote:

 I believe we should aim for the simplest code that admits the 
 desired performance.


Sure, but the desired performance here, for this stuff, is the 
clear priority. Especially at lower levels where the costs 
compound. Here's some princely advice: it is better to be both 
feared and loved, but if you can be only one or the other, it is 
better to be feared than to be loved. That's the reason why 
Phobos does it the way it does. It'd love to have both, but if it 
is one or the other, fast compiles are far more important than 
pretty code.


 Well, what is a "real problem"?


That code compiles slowly and/or with excessive memory. That's 
why type functions are being investigated.

There's no new functionality gained by the type functions. Their 
whole reason for existing is to make faster, less memory hungry 
builds. (right now anyway, that might change if it gains 
capabilities, but right now an explicit design goal is to make 
them a limited subset of template functionality in the name of 
improved performance).

I frequently take Stefan's examples, copy/paste them into a 
template, and use them. The code doesn't even look that different.

The problem is there's several usages where this version is slow.

type function version:

alias type = alias;
type[] basicTypeConvTargets(type T)
{
     type[] targets;
     targets.length = basic_types.length;
     size_t n = 0;
     foreach(t;basic_types)
     {
         if (is(T : t))
         {
             targets[n++] = t;
         }
     }
     return targets[0 .. n];
}

template version:

alias type = string;
type[] basicTypeConvTargets(T)()
{
     type[] targets;
     targets.length = basic_types.length;
     size_t n = 0;
     foreach(t;basic_types)
     {
         if (is(T : mixin(t)))
         {
             targets[n++] = t;
         }
     }
     return targets[0 .. n];
}


Very little difference! The makeConvMatix was *identical* except 
for the function prototype (and even there, both return 
strings!). Filter's guts can be:

         size_t[Args.length] keep;
         size_t pos = 0;
         foreach(idx, alias arg; Args)
                 if(Pred!arg) keep[pos++] = idx; // note idx, not 
arg.
         return makeResult(keep[0 .. pos]);

In today's D. Again, *almost identical* to the typefunction 
version, just using an index into the list instead of storing the 
alias in the array directly.


What kills this approach is *not* the code being hideous. It is 
the performance aspect - additional CTFE code is necessary to 
convert it back to a type tuple, and secondarily, the foreach 
being unrolled leads to extra work. A type function can simply do 
`returned.tupleof` and keep the foreach how it is. Here, we'd 
have to `mixin(returned.doConversion)`. (Where doConversion is a 
similarly reusable lib function.)

Again, minor syntax difference, but significant performance hit 
because doConversion must rebuild a new string out of stuff the 
compiler already knows. Phobos' current implementation is uglier, 
but also faster and uses less memory than the mixin version. So 
it wins over it.

This is the place the typefunction has a potential win. It might 
combine the nice code AND get a performance improvement. But if 
the TF ends up slower than Phobos has now... it is going to be 
rewritten into the faster version, even if uglier, because it is 
the compile performance driving this evolution.

The Phobos implementation started life with a very simple 
implementation too. It became what it is because it *had to*, 
specifically for performance reasons.
Oct 06 2020
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/6/20 2:09 PM, Adam D. Ruppe wrote:

 Filter's guts can be:
 
          size_t[Args.length] keep;
          size_t pos = 0;
          foreach(idx, alias arg; Args)
                  if(Pred!arg) keep[pos++] = idx; // note idx,
not arg.
          return makeResult(keep[0 .. pos]);
 
 In today's D. Again, *almost identical* to the typefunction version, 
 just using an index into the list instead of storing the alias in the 
 array directly.


I think Filter is a much more apt example than the implicit targets.

Not just because you can make it faster, but because with type 
functions, you should be able to actually *use filter*:

return Args.filter!Pred.array;

I mean, we have filter for a reason. You can easily write filter out 
into a loop, instead of using filter. But there it is, and it makes 
coding so much more pleasant. If the compiler and CTFE can be fast 
enough to make this pleasant (I'm not 100% convinced, but it looks 
promising), then I'm on board.

It comes down to one thing -- arrays vs. tuples. In type functions, a 
tuple is an array, and you can do all the things you can do with a 
normal array: mutate, sort, shrink, grow, loop, use as a range, etc.

With a Tuple, everything is immutable, and each change needs to go 
across a new template boundary. Even a loop is not really a loop.

That being said, I think the only way type functions make a difference 
is if they perform WELL. Even if they perform *as well* as templates 
(though indications are they perform better), I'd rather write code in 
an imperative style than recursive.

-Steve
Oct 06 2020
next sibling parent reply Meta <jared771 gmail.com> writes:
On Tuesday, 6 October 2020 at 18:30:15 UTC, Steven Schveighoffer 
wrote:

 On 10/6/20 2:09 PM, Adam D. Ruppe wrote:

 Filter's guts can be:
 
          size_t[Args.length] keep;
          size_t pos = 0;
          foreach(idx, alias arg; Args)
                  if(Pred!arg) keep[pos++] = idx; // note idx, 
 not arg.
          return makeResult(keep[0 .. pos]);
 
 In today's D. Again, *almost identical* to the typefunction 
 version, just using an index into the list instead of storing 
 the alias in the array directly.


 I think Filter is a much more apt example than the implicit 
 targets.

 Not just because you can make it faster, but because with type 
 functions, you should be able to actually *use filter*:

 return Args.filter!Pred.array;

 I mean, we have filter for a reason. You can easily write 
 filter out into a loop, instead of using filter. But there it 
 is, and it makes coding so much more pleasant. If the compiler 
 and CTFE can be fast enough to make this pleasant (I'm not 100% 
 convinced, but it looks promising), then I'm on board.

 With a Tuple, everything is immutable, and each change needs to 
 go across a new template boundary. Even a loop is not really a 
 loop.


Excellent point; the only reason Filter, staticSort, staticMap, 
Reverse, Repeat, etc., etc. exist is because of the limitations 
of working with AliasSeq. If type functions can allow us to 
replace most of these uses with good old std.range/algorithm 
code, that's a huge win in my book.

We have a big chance to go left when C++ went right and went 
all-in on template metaprogramming. Let's leave templates to do 
what they were designed for - genericizing structures and 
algorithms, and leave the compile-time computation to CTFE + type 
functions.
Oct 06 2020
parent foobar <foo bar.com> writes:
On Tuesday, 6 October 2020 at 18:49:08 UTC, Meta wrote:

 On Tuesday, 6 October 2020 at 18:30:15 UTC, Steven 
 Schveighoffer wrote:

 [...]


 Excellent point; the only reason Filter, staticSort, staticMap, 
 Reverse, Repeat, etc., etc. exist is because of the limitations 
 of working with AliasSeq. If type functions can allow us to 
 replace most of these uses with good old std.range/algorithm 
 code, that's a huge win in my book.

 We have a big chance to go left when C++ went right and went 
 all-in on template metaprogramming. Let's leave templates to do 
 what they were designed for - genericizing structures and 
 algorithms, and leave the compile-time computation to CTFE + 
 type functions.


C++ has constexpr...

We have a big chance to become a language even bigger than C++ 
without doing anything more. Hooray.
Oct 06 2020
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 6 October 2020 at 18:30:15 UTC, Steven Schveighoffer 
wrote:

 It comes down to one thing -- arrays vs. tuples. In type 
 functions, a tuple is an array, and you can do all the things 
 you can do with a normal array: mutate, sort, shrink, grow, 
 loop, use as a range, etc.

 With a Tuple, everything is immutable, and each change needs to 
 go across a new template boundary. Even a loop is not really a 
 loop.


It seems to me like maybe the most obvious way from point A to 
point B is to lift these limitations on tuples (and aliases, and 
manifest constants). Then we could write, for example:

template Filter(alias pred, Args...)
{
     enum pos = 0;
     foreach (Arg; Args) {
         static if (Pred!Arg) {
             Args[pos] := Arg;
             pos := pos + 1;
         }
     }
     alias Filter = Args[0 .. pos];
}

You would still pay for the performance overhead of tuple foreach 
and static if, so maybe it's not that big a win, but it's still 
an improvement over recursive templates and CTFE+mixins.
Oct 06 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 6 October 2020 at 23:36:00 UTC, Paul Backus wrote:

 On Tuesday, 6 October 2020 at 18:30:15 UTC, Steven 
 Schveighoffer wrote:

 It comes down to one thing -- arrays vs. tuples. In type 
 functions, a tuple is an array, and you can do all the things 
 you can do with a normal array: mutate, sort, shrink, grow, 
 loop, use as a range, etc.

 With a Tuple, everything is immutable, and each change needs 
 to go across a new template boundary. Even a loop is not 
 really a loop.


 It seems to me like maybe the most obvious way from point A to 
 point B is to lift these limitations on tuples (and aliases, 
 and manifest constants). Then we could write, for example:

 template Filter(alias pred, Args...)
 {
     enum pos = 0;
     foreach (Arg; Args) {
         static if (Pred!Arg) {
             Args[pos] := Arg;
             pos := pos + 1;
         }
     }
     alias Filter = Args[0 .. pos];
 }

 You would still pay for the performance overhead of tuple 
 foreach and static if, so maybe it's not that big a win, but 
 it's still an improvement over recursive templates and 
 CTFE+mixins.


I think lifting limitations on tuples can't be done in general 
without violation of current language rules.
Type functions are actually just a shell around operations that 
would be illegal in the language as is.
But because the type function provides a boundary I can do these 
things without invalidating the language semantics.

Type functions are something which can be proven to not have 
influence on language semantics outside of their own function 
bodies.
Oct 06 2020
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 6 October 2020 at 23:44:30 UTC, Stefan Koch wrote:

 Type functions are actually just a shell around operations that 
 would be illegal in the language as is.
 But because the type function provides a boundary I can do 
 these things without invalidating the language semantics.

 Type functions are something which can be proven to not have 
 influence on language semantics outside of their own function 
 bodies.


Ugh, why not do it properly and just add type variables? It is 
just a pointer... At runtime it could point to the type's RTTI 
node which contains a pointer to the constructor.
Oct 06 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 7 October 2020 at 00:03:46 UTC, Ola Fosheim Grøstad 
wrote:

 On Tuesday, 6 October 2020 at 23:44:30 UTC, Stefan Koch wrote:

 Type functions are actually just a shell around operations 
 that would be illegal in the language as is.
 But because the type function provides a boundary I can do 
 these things without invalidating the language semantics.

 Type functions are something which can be proven to not have 
 influence on language semantics outside of their own function 
 bodies.


 Ugh, why not do it properly and just add type variables? It is 
 just a pointer... At runtime it could point to the type's RTTI 
 node which contains a pointer to the constructor.


If you know a way to do that cleanly, that does not involve a 
redesign of the compiler,
I am very interested to hear about it.
As things stand type functions are what I can get away with, and 
still be reasonably confident I won't violate language invariants.
Also their syntax blends in fairly nicely with the rest of D.
Oct 06 2020
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 7 October 2020 at 00:11:48 UTC, Stefan Koch wrote:

 If you know a way to do that cleanly, that does not involve a 
 redesign of the compiler,
 I am very interested to hear about it.
 As things stand type functions are what I can get away with, 
 and still be reasonably confident I won't violate language 
 invariants.
 Also their syntax blends in fairly nicely with the rest of D.


Allright, if the syntax is such that it can be made more general 
later then all is good.

(Ive only modified the dmd lexer/parser, so I don't know where 
the limitations are bqeyond ast level...)
Oct 06 2020
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 7 October 2020 at 00:18:10 UTC, Ola Fosheim Grøstad 
wrote:

 On Wednesday, 7 October 2020 at 00:11:48 UTC, Stefan Koch wrote:

 If you know a way to do that cleanly, that does not involve a 
 redesign of the compiler,
 I am very interested to hear about it.
 As things stand type functions are what I can get away with, 
 and still be reasonably confident I won't violate language 
 invariants.
 Also their syntax blends in fairly nicely with the rest of D.


 Allright, if the syntax is such that it can be made more 
 general later then all is good.

 (Ive only modified the dmd lexer/parser, so I don't know where 
 the limitations are bqeyond ast level...)


With more general you mean, not just apply to types but to 
anything?

Yes that can happen. Adam actually wants this functionality as 
well.
Tbh. the syntax is currently blocking me here.
What name do I give to the (pseudo) TOP type?
that can hold anything a variadic template parameter can hold?
Oct 06 2020
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 6 October 2020 at 23:44:30 UTC, Stefan Koch wrote:

 On Tuesday, 6 October 2020 at 23:36:00 UTC, Paul Backus wrote:

 It seems to me like maybe the most obvious way from point A to 
 point B is to lift these limitations on tuples (and aliases, 
 and manifest constants). Then we could write, for example:

 template Filter(alias pred, Args...)
 {
     enum pos = 0;
     foreach (Arg; Args) {
         static if (Pred!Arg) {
             Args[pos] := Arg;
             pos := pos + 1;
         }
     }
     alias Filter = Args[0 .. pos];
 }

 You would still pay for the performance overhead of tuple 
 foreach and static if, so maybe it's not that big a win, but 
 it's still an improvement over recursive templates and 
 CTFE+mixins.


 I think lifting limitations on tuples can't be done in general 
 without violation of current language rules.
 Type functions are actually just a shell around operations that 
 would be illegal in the language as is.
 But because the type function provides a boundary I can do 
 these things without invalidating the language semantics.

 Type functions are something which can be proven to not have 
 influence on language semantics outside of their own function 
 bodies.


I agree that allowing completely unrestricted tuple mutation 
would be problematic, but I think it's still worth asking how far 
we could potentially go in that direction.

For example, maybe you're only allowed to mutate tuples inside 
the scope that declares them. That would let you implement 
templates like Filter and staticMap iteratively, while still 
presenting an immutable "interface" to the rest of the program.
Oct 06 2020
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 7 October 2020 at 00:11:20 UTC, Paul Backus wrote:

 
 I agree that allowing completely unrestricted tuple mutation 
 would be problematic, but I think it's still worth asking how 
 far we could potentially go in that direction.


I would be happy to discuss this over a coffee or beer sometime.
As a thought experiment it's exciting.


 For example, maybe you're only allowed to mutate tuples inside 
 the scope that declares them. That would let you implement 
 templates like Filter and staticMap iteratively, while still 
 presenting an immutable "interface" to the rest of the program.


If you can prove:
  - It never escapes a 'mutable' version into a monophonic scope.
  - Inside a polymorphic scope (template) all transforms are 
derived from the shape defining parameters (template parameters) 
or are invariant.

That is necessary, but I am not sure it's sufficient.
Oct 06 2020
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/6/2020 11:09 AM, Adam D. Ruppe wrote:

 The Phobos implementation started life with a very simple implementation too.
It 
 became what it is because it *had to*, specifically for performance reasons.


Professional C Standard library implementations tend to be hideous code to 
perform objectively simple operations. The reason is speed is so desirable in 
foundational code that it drives out all other considerations. (memcpy() is a 
standout example.)

I remember back in the 80's when Borland came out with Turbo C. The compiler 
didn't generate very good code, but applications built with it were reasonably 
fast. How was this done?

Borland carefully implemented the C library in hand-optimized assembler by some 
very good programmers. Even printf was coded this way. The speedups gained
there 
sped up every Turbo C program. At the time, nobody else had done that.

Much as I hate to admit it, Borland made the smart move there.
Oct 06 2020
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Oct 06, 2020 at 08:47:33PM -0700, Walter Bright via Digitalmars-d wrote:

 On 10/6/2020 11:09 AM, Adam D. Ruppe wrote:

 The Phobos implementation started life with a very simple
 implementation too. It became what it is because it *had to*,
 specifically for performance reasons.


 
 Professional C Standard library implementations tend to be hideous
 code to perform objectively simple operations. The reason is speed is
 so desirable in foundational code that it drives out all other
 considerations. (memcpy() is a standout example.)


A little tangential aside: one time as a little coffee break challenge,
I decided to see how D compares to GNU wc in terms of the speed of
counting lines in a text file.  The core of wc, of course, is in memchr
-- because it's scanning for newline characters in a buffer.  In the
course of ferreting out what made wc so fast, I studied how GNU libc
implemented memchr.  Basically, in order to speed up scanning large
buffers, it uses a series of fancy bit operations on 64-bit words in
order to scan 8 bytes at a time, I suppose the goal being to achieve
close to 8x speedup, and also to reduce the number of branches per
iteration to capitalize on the CPU's pipeline.

In order to do this, however, some not-so-cheap setup was necessary at
the beginning and end of the buffer, which generally are not 8-byte
aligned.  When a particular call to memchr scanned many bytes, of
course, the speed of scanning 8 bytes at a time outweighed this setup /
teardown cost, and wc generally outperformed my D code.

However, when the lines being scanned were on the short side, the
overhead of setting up the 8-byte-at-a-time scanning added up
significantly -- the shorter lines meant less time was spend scanning 8
bytes at a time, and more time spent in the complicated code dealing
with non-aligned start/end of the buffer. In this case, I discovered
that a simplistic for-loop in D outperformed wc.

This led me to think, realistically speaking, how long do lines in a
text file tend to be?  My wild guess is that they tend to be on the
shortish side -- maybe about 60-70 characters on average? For code, a
lot less, since code generally has more whitespace for readability
reasons.  Files that have very long lines tend to be things like XML or
compressed Javascript, which generally you don't really use wc on
anyway, so in my mind, they seem to be more the exception than the norm
when it comes to the applicability of wc.  Furthermore, I venture to
propose that your average string length in C is probably closer to the
80-120 character ballpark, than to large buffers of 1K or 8K which is
where the 8-byte-at-a-time scanning would perform much better. Sure,
large text buffers do get handled in C code routinely; but how many of
them realistically would you find being handed to strchr to scan for
some given byte?  The kind of C strings you'd want to search for a
particular byte in, IME, are usually the shorter kind.

What this means, is that yes memchr may be "optimized" to next week and
back -- but that optimization came with some implicit assumptions about
how long it takes to find a character in a string buffer. These
assumptions unfortunately pessimized the common case with the overhead
of a fancy 8-byte-at-a-time algorithm: one that does better in the
*less* common case of looking for a particular byte in a very large
buffer.

My point behind all this, is that what's considered optimal sometimes
changes depending on what kind of use case you anticipate; and with
different usage patterns, one man's optimal algorithm may be another
man's suboptimal algorithm, and one man's slow, humble for-loop may be
another man's speed demon.  Optimizing for the general case in a
standard library is generally a very tough problem, because how do you
make any assumptions about the general case?  Whichever way you choose
to optimize a primitive like memchr, you're imposing additional
assumptions that may make it worse for some people, even if you think
that you're improving it for your chosen target metric.




 I remember back in the 80's when Borland came out with Turbo C. The
 compiler didn't generate very good code, but applications built with
 it were reasonably fast. How was this done?
 
 Borland carefully implemented the C library in hand-optimized
 assembler by some very good programmers. Even printf was coded this
 way. The speedups gained there sped up every Turbo C program. At the
 time, nobody else had done that.
 
 Much as I hate to admit it, Borland made the smart move there.


So what does this imply in terms of Phobos code in D? ;-)  Should we
uglify Phobos for the sake of performance?  Keeping in mind, of course,
what I said about the difficulty of optimizing for the general case
without pessimizing some legitimate use cases -- or maybe even the
*common* case, if we lose sight of the forest of common use cases while
trying to optimize our chosen benchmark trees.


T

-- 
Life begins when you can spend your spare time programming instead of watching
television. -- Cal Keegan
Oct 06 2020
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/6/2020 9:17 PM, H. S. Teoh wrote:

 So what does this imply in terms of Phobos code in D? ;-)  Should we
 uglify Phobos for the sake of performance?


I'd phrase it more as do what we gotta do for performance, yes.


 Keeping in mind, of course,
 what I said about the difficulty of optimizing for the general case
 without pessimizing some legitimate use cases -- or maybe even the
 *common* case, if we lose sight of the forest of common use cases while
 trying to optimize our chosen benchmark trees.


Of course, we have to use our brains instead of doing things blindly :-)
Oct 06 2020
prev sibling next sibling parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Wednesday, 7 October 2020 at 04:17:59 UTC, H. S. Teoh wrote:

 On Tue, Oct 06, 2020 at 08:47:33PM -0700, Walter Bright via 
 Digitalmars-d wrote:

 On 10/6/2020 11:09 AM, Adam D. Ruppe wrote:

 The Phobos implementation started life with a very simple 
 implementation too. It became what it is because it *had 
 to*, specifically for performance reasons.


 
 Professional C Standard library implementations tend to be 
 hideous code to perform objectively simple operations. The 
 reason is speed is so desirable in foundational code that it 
 drives out all other considerations. (memcpy() is a standout 
 example.)


 A little tangential aside: one time as a little coffee break 
 challenge, I decided to see how D compares to GNU wc in terms 
 of the speed of counting lines in a text file.  The core of wc, 
 of course, is in memchr -- because it's scanning for newline 
 characters in a buffer.  In the course of ferreting out what 
 made wc so fast, I studied how GNU libc implemented memchr.  
 Basically, in order to speed up scanning large buffers, it uses 
 a series of fancy bit operations on 64-bit words in order to 
 scan 8 bytes at a time, I suppose the goal being to achieve 
 close to 8x speedup, and also to reduce the number of branches 
 per iteration to capitalize on the CPU's pipeline.

 In order to do this, however, some not-so-cheap setup was 
 necessary at the beginning and end of the buffer, which 
 generally are not 8-byte aligned.  When a particular call to 
 memchr scanned many bytes, of course, the speed of scanning 8 
 bytes at a time outweighed this setup / teardown cost, and wc 
 generally outperformed my D code.

 However, when the lines being scanned were on the short side, 
 the overhead of setting up the 8-byte-at-a-time scanning added 
 up significantly -- the shorter lines meant less time was spend 
 scanning 8 bytes at a time, and more time spent in the 
 complicated code dealing with non-aligned start/end of the 
 buffer. In this case, I discovered that a simplistic for-loop 
 in D outperformed wc.

 This led me to think, realistically speaking, how long do lines 
 in a text file tend to be?  My wild guess is that they tend to 
 be on the shortish side -- maybe about 60-70 characters on 
 average? For code, a lot less, since code generally has more 
 whitespace for readability reasons.  Files that have very long 
 lines tend to be things like XML or compressed Javascript, 
 which generally you don't really use wc on anyway, so in my 
 mind, they seem to be more the exception than the norm when it 
 comes to the applicability of wc.  Furthermore, I venture to 
 propose that your average string length in C is probably closer 
 to the 80-120 character ballpark, than to large buffers of 1K 
 or 8K which is where the 8-byte-at-a-time scanning would 
 perform much better. Sure, large text buffers do get handled in 
 C code routinely; but how many of them realistically would you 
 find being handed to strchr to scan for some given byte?  The 
 kind of C strings you'd want to search for a particular byte 
 in, IME, are usually the shorter kind.

 What this means, is that yes memchr may be "optimized" to next 
 week and back -- but that optimization came with some implicit 
 assumptions about how long it takes to find a character in a 
 string buffer. These assumptions unfortunately pessimized the 
 common case with the overhead of a fancy 8-byte-at-a-time 
 algorithm: one that does better in the *less* common case of 
 looking for a particular byte in a very large buffer.

 My point behind all this, is that what's considered optimal 
 sometimes changes depending on what kind of use case you 
 anticipate; and with different usage patterns, one man's 
 optimal algorithm may be another man's suboptimal algorithm, 
 and one man's slow, humble for-loop may be another man's speed 
 demon.  Optimizing for the general case in a standard library 
 is generally a very tough problem, because how do you make any 
 assumptions about the general case?  Whichever way you choose 
 to optimize a primitive like memchr, you're imposing additional 
 assumptions that may make it worse for some people, even if you 
 think that you're improving it for your chosen target metric.


There's a counter argument to your example (which doesn't 
invalidate yout point).

The pessimisation for short lines is barely noticeable and is 
drowned in the noise of process launching and even if you can 
measure it consistantly and easily, it doesn't matter much even 
if happening a lot of times (in a batch job f.ex). The 
optimization for long line though, will be noticeable immediately 
because of the inherent processing time of the operation. If an 
operation of 5ms takes now 10ms, it doesn't make a difference. If 
an operation that took 1 minute now takes 30s it is a big deal.



 I remember back in the 80's when Borland came out with Turbo 
 C. The compiler didn't generate very good code, but 
 applications built with it were reasonably fast. How was this 
 done?
 
 Borland carefully implemented the C library in hand-optimized 
 assembler by some very good programmers. Even printf was coded 
 this way. The speedups gained there sped up every Turbo C 
 program. At the time, nobody else had done that.
 
 Much as I hate to admit it, Borland made the smart move there.


 So what does this imply in terms of Phobos code in D? ;-)  
 Should we uglify Phobos for the sake of performance?  Keeping 
 in mind, of course, what I said about the difficulty of 
 optimizing for the general case without pessimizing some 
 legitimate use cases -- or maybe even the *common* case, if we 
 lose sight of the forest of common use cases while trying to 
 optimize our chosen benchmark trees.



My point. Some optimlizations are more about mitigating speed 
degradation on pathological cases than on the normal general 
case, or else the whole O() notation would be of no importance 
(I'm sure the symbol name compression scheme in the D compiler 
pessimized the common case but saved the language from the 
pathological cases of recursive symbols).
Oct 07 2020
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/7/20 12:17 AM, H. S. Teoh wrote:

 On Tue, Oct 06, 2020 at 08:47:33PM -0700, Walter Bright via Digitalmars-d
wrote:

 On 10/6/2020 11:09 AM, Adam D. Ruppe wrote:

 The Phobos implementation started life with a very simple
 implementation too. It became what it is because it *had to*,
 specifically for performance reasons.


 Professional C Standard library implementations tend to be hideous
 code to perform objectively simple operations. The reason is speed is
 so desirable in foundational code that it drives out all other
 considerations. (memcpy() is a standout example.)


 
 A little tangential aside: one time as a little coffee break challenge,
 I decided to see how D compares to GNU wc in terms of the speed of
 counting lines in a text file.  The core of wc, of course, is in memchr
 -- because it's scanning for newline characters in a buffer.  In the
 course of ferreting out what made wc so fast, I studied how GNU libc
 implemented memchr.  Basically, in order to speed up scanning large
 buffers, it uses a series of fancy bit operations on 64-bit words in
 order to scan 8 bytes at a time, I suppose the goal being to achieve
 close to 8x speedup, and also to reduce the number of branches per
 iteration to capitalize on the CPU's pipeline.


memchr is also the secret sauce to how iopipe was able to beat Phobos 
byLine and getline.

I would like to go back to it at some point and see if it can be improved.

I know that part of the difference would be due to the opaque function 
call (this cannot be inlined), though perhaps memchr is an intrinsic?

-Steve
Oct 07 2020
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Oct 07, 2020 at 10:10:56AM -0400, Steven Schveighoffer via
Digitalmars-d wrote:
[...]

 memchr is also the secret sauce to how iopipe was able to beat Phobos
 byLine and getline.
 
 I would like to go back to it at some point and see if it can be
 improved.
 
 I know that part of the difference would be due to the opaque function
 call (this cannot be inlined), though perhaps memchr is an intrinsic?


[...]

I wouldn't be surprised if optimizing compilers like ldc treated it as
an intrinsic.  IIRC, ldc already recognizes memcpy as an intrinsic and
therefore knows how to inline it / substitute it with suitable
instructions in special cases, even though it's supposed to be "opaque".


T

-- 
Жил-был король когда-то, при нём блоха жила.
Oct 07 2020
parent Jon Degenhardt <jond noreply.com> writes:
On Wednesday, 7 October 2020 at 14:41:18 UTC, H. S. Teoh wrote:

 On Wed, Oct 07, 2020 at 10:10:56AM -0400, Steven Schveighoffer 
 via Digitalmars-d wrote: [...]

 memchr is also the secret sauce to how iopipe was able to beat 
 Phobos byLine and getline.
 
 I would like to go back to it at some point and see if it can 
 be improved.
 
 I know that part of the difference would be due to the opaque 
 function call (this cannot be inlined), though perhaps memchr 
 is an intrinsic?


 [...]

 I wouldn't be surprised if optimizing compilers like ldc 
 treated it as an intrinsic.  IIRC, ldc already recognizes 
 memcpy as an intrinsic and therefore knows how to inline it / 
 substitute it with suitable instructions in special cases, even 
 though it's supposed to be "opaque".


It's my understanding that LTO could be used to cross the 
language boundary as well, allowing otherwise opaque function 
calls to be inlined. It would require having the function 
compiled to IR code available at build time.

Related to earlier in this subthread - I regularly work with data 
file 20 bytes per line and others 5000 bytes per line. It's been 
my experience that the two can have quite different performance 
characteristics for the same operation. As pointed out, it is 
sometimes hard to optimize for both.

--Jon
Oct 07 2020
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 7 October 2020 at 14:10:56 UTC, Steven 
Schveighoffer wrote:

 I know that part of the difference would be due to the opaque 
 function call (this cannot be inlined), though perhaps memchr 
 is an intrinsic?


You can test individual bytes in a simd register in a single 
instruction. Will clearly destroy any other implementation...
Oct 07 2020
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 7 October 2020 at 14:43:41 UTC, Ola Fosheim Grøstad 
wrote:

 You can test individual bytes in a simd register in a single 
 instruction. Will clearly destroy any other implementation...


 From 2016:

https://gms.tf/stdfind-and-memchr-optimizations.html
Oct 07 2020
prev sibling parent reply claptrap <clap trap.com> writes:
On Tuesday, 6 October 2020 at 12:28:55 UTC, Adam D. Ruppe wrote:

 On Tuesday, 6 October 2020 at 08:35:20 UTC, claptrap wrote:

 Because Stefan didnt rely on external code, he showed *all* 
 the code needed, so a counter example should have been the 
 same. Since the point is to compare the two *language* 
 features, not compare library calls.


 Language features are a means to an end.

 If the Phobos version compiles faster, uses less memory, and 
 has the same result in the binary, it is a victory, even if it 
 as significantly more code.

 I'm skeptical of the type functions, but interested. They might 
 help with a real problem. But they need to actually win on the 
 end results, not against artificial strawmen.


Im less interested in performance than I am in being able to 
express what I want to do in clear concise code. I find 
template/mixin shenanigans a bit like writing that doesnt have 
any vowels, ys y cn stll ndrstnd t, but it takes a lot more work 
and feels fundamentally unsatisfactory.
Oct 06 2020
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 6 October 2020 at 20:57:10 UTC, claptrap wrote:

 Im less interested in performance than I am in being able to 
 express what I want to do in clear concise code.


Give me an example of what you'd like to use type functions for.

I betcha I can adapt it to current D with very few changes.

Take a look at this for example:

---
import std.algorithm;
template largestType(T...) {
         auto get() {
                 return reified!"a.sizeof".map!T
                     .sort!((a, b) => a > b).front; // or 
maxElement of course
         }
         alias largestType = T[get.idx];
}

pragma(msg, largestType!(long, int, real, byte)); // real
---


Or what about this, without even seeing the template keyword?

import std.algorithm;

pragma(msg,
   reified!"is(a : long)" // compile time lambda we need
     .run!(types => types.filter!(a => a)) // run the code over 
the fetched info
     .over!(string, int, float, Object)); // the list of types
);



Note it used std.algorithm's filter over the mapped compile time 
lambda! But, of course, you'll see it is a string. That is a 
limitation here, pity we don't have some kind of short syntax 
template lambda.




This is the reifiy and dereify implementations:

---

template reified(string mapCode, alias runCode = void) {
         template evaluate(alias a) { enum evaluate = 
mixin(mapCode); }

         template run(alias code) {
                 alias run = reified!(mapCode, code);
         }

         template over(T...) {
                 auto helper() {
                         static struct Result {
                                 size_t idx;
                                 typeof(evaluate!(T[0])) value;
                                 alias value this;
                         }

                         Result[] result;
                         foreach(idx, t; T)
                                 result ~= Result(idx, evaluate!t);

                         return result;
                 }

                 static if(is(runCode == void))
                         enum over = helper;
                 else
                         mixin("alias over = " ~ 
dereifiy(runCode(helper())) ~ ";");
         }
}

import std.meta;

string dereifiy(T)(T t, string localName = "T") {
         import std.conv;
         import std.range;
         static if(isInputRange!T) {
                 string result = "AliasSeq!(";
                 foreach(item; t)
                         result ~= localName ~ "[" ~ 
to!string(item.idx) ~ "],";
                 result ~= ")";
                 return result;
         } else
                 return localName ~ "[" ~ t.idx.to!string ~ "]";
}
---
Oct 06 2020
parent reply claptrap <clap trap.com> writes:
On Tuesday, 6 October 2020 at 23:27:10 UTC, Adam D. Ruppe wrote:

 On Tuesday, 6 October 2020 at 20:57:10 UTC, claptrap wrote:

 Im less interested in performance than I am in being able to 
 express what I want to do in clear concise code.


 Give me an example of what you'd like to use type functions for.

 I betcha I can adapt it to current D with very few changes.


I dont doubt it, but the question is could I do it? could a new D 
user do it?

IE. Whats the learning curve like?



 Take a look at this for example:

 ---
 import std.algorithm;
 template largestType(T...) {
         auto get() {
                 return reified!"a.sizeof".map!T
                     .sort!((a, b) => a > b).front; // or 
 maxElement of course
         }
         alias largestType = T[get.idx];
 }

 pragma(msg, largestType!(long, int, real, byte)); // real


See the point is even even though I understand that, it took me a 
while to grep it, and I couldn't just rattle that off the top of 
my head, it'd take me a fair while to figure out how to do that. 
(Maybe i wouldn't even be able to on my own)

But with Type Functions it'd be something like this...

// assuming type is synonym for alias or whatever.

type convTargets(type[] args)
{
     assert(args.length > 0);
     type result = void; // IIRC void.sizeof == 1?
     foreach(t; args)
         if (t.size > result.sizeof) result = t;
     return result;
}

The point is TF are obvious, if you know regular D code, you can 
use type functions. I've got the gist of it from a handful of 
forums posts. The cognitive load is so much lower.
Oct 06 2020
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 7 October 2020 at 01:27:17 UTC, claptrap wrote:

 IE. Whats the learning curve like?


It is hard to say right now because this is a brand new 
technique. It has been possible in D ... basically forever, these 
language features are all quite aged, but to my knowledge, nobody 
has tried it this way before.


 See the point is even even though I understand that, it took me 
 a while to grep it, and I couldn't just rattle that off the top 
 of my head, it'd take me a fair while to figure out how to do 
 that. (Maybe i wouldn't even be able to on my own)


That might just be because you've never seen a combination of 
patterns like that before... and that's OK, I made up that 
particular thing just a few hours ago. (Though it is based on 
some ideas Andrei and I have been bouncing back and forth.)

But with a little library refinement and some tutorials, maybe it 
would prove to be easy to learn and to use.

The usage of a string lambda reminded me of the early days of 
std.functional and std.algorithm. Back then, a function literal 
looked like `(args) { return thing; }`. They shortened to strings 
since they liked the shortness and that's what we had, but it 
wasn't great.

Right now, a template lambda doesn't exist. We don't even have a 
short form literal like delegates did. So I'm using a string 
there. Maybe we are walking down the same path.


 But with Type Functions it'd be something like this...

 // assuming type is synonym for alias or whatever.

 type convTargets(type[] args)
 {
     assert(args.length > 0);
     type result = void; // IIRC void.sizeof == 1?
     foreach(t; args)
         if (t.size > result.sizeof) result = t;
     return result;
 }


You could have also written that:

// normal boilerplate
template biggest(args...) {
     int helper() {
       // this stuff is really similar between the two!
       assert(args.length > 0);
       size_t result;
       size_t bestSize; // just store the answer and the size 
separate
       foreach(idx, t; args)
          if(t.sizeof > bestSize) {
              result = idx;
              bestSize = t;
          }
       return result;
     }
     // then this part always follows the same pattern again
     alias biggest = args[helper];
}

That pattern btw is very useful to know regardless of if this 
technique works out - it is also the same idea I used for 
user-defined literals in D ages ago.

Anyway, that's where my young library constructs are coming from 
- trying to recognize and abstract this pattern a little. But the 
core of it IS just regular D code: just it returns indexes into 
the array instead of the item itself because compiler tuples are 
weird. (I've been telling Stefan in a separate chat if type 
functions become compiler tuple functions, able to work with that 
`args...` thing to its full potential, we'd have something 
compelling in new functionality, not just speed. That can replace 
some of the super-ugly string mixins in my jsvar and jni that 
transform runtime argument lists! But idk if that will prove 
possible yet. He's exploring it, though.)

The major problem with this pattern so far is it is slow. I was 
actually going to abandon it after Phobos' existing 
implementations and the typefunctions prototype both blew it out 
of the water in speed terms.

But some people say using regular std.algorithm stuff is more 
important to them than speed. So I'm still toying with it for 
that sake.
Oct 06 2020
parent reply claptrap <clap trap.com> writes:
On Wednesday, 7 October 2020 at 03:12:46 UTC, Adam D. Ruppe wrote:

 On Wednesday, 7 October 2020 at 01:27:17 UTC, claptrap wrote:

 IE. Whats the learning curve like?


 It is hard to say right now because


No it's not :)


 See the point is even even though I understand that, it took 
 me a while to grep it, and I couldn't just rattle that off the 
 top of my head, it'd take me a fair while to figure out how to 
 do that. (Maybe i wouldn't even be able to on my own)


 That might just be because you've never seen a combination of 
 patterns like that before... and that's OK, I made up that 
 particular thing just a few hours ago. (Though it is based on 
 some ideas Andrei and I have been bouncing back and forth.)


Yeah Im not that that used to it, but the point is you have too 
work at it to see whats going on, there's two nested functions 
calls, more things passed around, more branches and recursion.

I have no problem with recursion, there's algorithms that make 
more sense as recursion, but this is not one, it's iterating over 
a list, to make it recursive you end up with 2 or 3 times more 
things going on in in order to make it work. Two functions, more 
branches, more things passed backwards and forwards, more 
construction of AliasSeq or whatever.

Just compare how much is going on in the template version vs the 
TF version.




 But with a little library refinement and some tutorials, maybe 
 it would prove to be easy to learn and to use.


I'd rather 600 lines added to the compiler so I can concentrate 
on things I have to learn, I can write less code, my code is more 
intuative and less bug prone, because im not having to bend shit 
out of shape to make it recursive.
Oct 07 2020
parent claptrap <clap trap.com> writes:
On Wednesday, 7 October 2020 at 08:26:21 UTC, claptrap wrote:

 On Wednesday, 7 October 2020 at 03:12:46 UTC, Adam D. Ruppe 
 wrote:


 Yeah Im not that that used to it, but the point is you have too 
 work at it to see whats going on, there's two nested functions 
 calls, more things passed around, more branches and recursion.

 I have no problem with recursion, there's algorithms that make 
 more sense as recursion, but this is not one, it's iterating 
 over a list, to make it recursive you end up with 2 or 3 times 
 more things going on in in order to make it work. Two 
 functions, more branches, more things passed backwards and 
 forwards, more construction of AliasSeq or whatever.

 Just compare how much is going on in the template version vs 
 the TF version.


Appologies, Ive just woke up and I replied here in response to 
Daniel Ks example code, not yours Adam. Hopefully it still kinda 
makes sense.



 Oct 07 2020



prev sibling  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 7 October 2020 at 01:27:17 UTC, claptrap wrote:

 On Tuesday, 6 October 2020 at 23:27:10 UTC, Adam D. Ruppe wrote:

 On Tuesday, 6 October 2020 at 20:57:10 UTC, claptrap wrote:

 Im less interested in performance than I am in being able to 
 express what I want to do in clear concise code.


 Give me an example of what you'd like to use type functions 
 for.

 I betcha I can adapt it to current D with very few changes.


 I dont doubt it, but the question is could I do it? could a new 
 D user do it?

 IE. Whats the learning curve like?



 Take a look at this for example:

 ---
 import std.algorithm;
 template largestType(T...) {
         auto get() {
                 return reified!"a.sizeof".map!T
                     .sort!((a, b) => a > b).front; // or 
 maxElement of course
         }
         alias largestType = T[get.idx];
 }

 pragma(msg, largestType!(long, int, real, byte)); // real


 See the point is even even though I understand that, it took me 
 a while to grep it, and I couldn't just rattle that off the top 
 of my head, it'd take me a fair while to figure out how to do 
 that. (Maybe i wouldn't even be able to on my own)

 But with Type Functions it'd be something like this...

 // assuming type is synonym for alias or whatever.

 type convTargets(type[] args)
 {
     assert(args.length > 0);
     type result = void; // IIRC void.sizeof == 1?
     foreach(t; args)
         if (t.size > result.sizeof) result = t;
     return result;
 }

 The point is TF are obvious, if you know regular D code, you 
 can use type functions. I've got the gist of it from a handful 
 of forums posts. The cognitive load is so much lower.


Your code unfortunately won't work, because `= void`;
has special semantics, it won't assign the type void but leave 
the variable uninitialized.

This is what I wrote and it has shown me a bug.
alias variables should be initialized to an invalid type called, 
TError.
Inside the type-function implementation.
---
alias type = alias;

type biggestType(type[] types ...)
{
     type result;  // should initialize to Terror (The error type 
(the type that is not a type))
                   // for some reason initializes to alias now?
     foreach(t; types)
         if (t.sizeof > result.sizeof) result = t;
     return result;
}
// working around parser issues again.
// is expressions don't like function calls inside them ;)
alias I(alias A) = A;

pragma(msg, is(I!(biggestType()) == alias)); // prints true but 
really should not!
pragma(msg, is(I!(biggestType(int)) == int)); // prints true as 
it should
pragma(msg, is(I!(biggestType(int, ulong)) == int)); // prints 
false as it should
pragma(msg, is(I!(biggestType(int, ulong)) == ulong)); // prints 
true.



 Oct 07 2020



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 7 October 2020 at 22:31:52 UTC, Stefan Koch wrote:

 alias type = alias;

 type biggestType(type[] types ...)
 {
     type result;  // should initialize to Terror (The error 
 type (the type that is not a type))
                   // for some reason initializes to alias now?
     foreach(t; types)
         if (t.sizeof > result.sizeof) result = t;
     return result;
 }
 // working around parser issues again.
 // is expressions don't like function calls inside them ;)
 alias I(alias A) = A;

 pragma(msg, is(I!(biggestType()) == alias)); // prints true but 
 really should not!


I just fixed this bug.
This will now print false.

the initial value of an alias variable is now ∅ The Empty type.
It's even less than void.

alias type = alias;

type TaliasInit()
{
     type x;
     assert(!is(x));
     return x;
}

pragma(msg, TaliasInit()); // prints ∅ (Empty Type)



 Oct 08 2020



  parent reply Dominikus Dittes Scherkl <dominikus scherkl.de>  writes:


On Thursday, 8 October 2020 at 20:52:29 UTC, Stefan Koch wrote:

 the initial value of an alias variable is now ∅ The Empty type.


Just for my understanding:
Is this empty type something you defined?
And would it be equivalent to the planned new bottom type 
(noreturn)?



 Oct 08 2020



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 9 October 2020 at 06:26:04 UTC, Dominikus Dittes 
Scherkl wrote:

 On Thursday, 8 October 2020 at 20:52:29 UTC, Stefan Koch wrote:

 the initial value of an alias variable is now ∅ The Empty type.


 Just for my understanding:
 Is this empty type something you defined?
 And would it be equivalent to the planned new bottom type 
 (noreturn)?


Thanks for asking.
Actually DMD already had it, in there it's known as in there 
Tnone.
It wasn't used for anything, I think it was supposed be the type 
of a not instantiated template ...

It's not bottom, because bottom is a type that a type function 
can return to you.
Tnone or ∅ (in D at least) just means this variable has not been 
assigned a type yet.
No type can implicitly convert to ∅.



 Oct 08 2020



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 9 October 2020 at 06:38:23 UTC, Stefan Koch wrote:

 Thanks for asking.
 Actually DMD already had it, in there it's known as in there 
 Tnone.
 It wasn't used for anything, I think it was supposed be the 
 type of a not instantiated template ...


I don't want to spread misinformation, it is actually used.
In dmd the classes have tag values so one does not have to do 
dynamic casts.
Tnone is used as tag for template parameter types which have been 
deduced, already.

It even has a mangle, but I doubt you should ever see it anywhere.
I will clarify with Walter.



 Oct 08 2020



prev sibling  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 6 October 2020 at 01:12:01 UTC, Walter Bright wrote:

 On 10/5/2020 2:20 PM, Stefan Koch wrote:

 Now post it with all transitive dependencies.


 I'm not sure how transitive dependencies applies here. A 
 illuminating test case would be helpful.


Okay let me show you the transitive dependencies of the "3 liner" 
suggested by Andrei and executed by Paul Backus.

--- phobos dependencies ---

template ApplyLeft(alias Template, args...)
{
     alias ApplyLeft(right...) = SmartAlias!(Template!(args, 
right));
}

private template SmartAlias(T...)
{
     static if (T.length == 1)
     {
         alias SmartAlias = Alias!T;
     }
     else
     {
         alias SmartAlias = T;
     }
}

alias Alias(alias a) = a;
alias Alias(T) = T;

template Filter(alias pred, TList...)
{
     static if (TList.length < filterExpandFactor)
     {
         mixin(FilterShortCode[TList.length]);
     }
     else
     {
         template MaybeNothing(Q...)
         {
             static if (pred!(Q[0]))
                 alias MaybeNothing = AliasSeq!(Q[0]);
             else
                 alias MaybeNothing = Nothing;
         }

         alias Filter = staticMap!(MaybeNothing, TList);
     }
}

private enum staticMapExpandFactor = 150;
private string generateCases()
{
     string[staticMapExpandFactor] chunks;
     chunks[0] = q{};
     static foreach (enum i; 0 .. staticMapExpandFactor - 1)
             chunks[i + 1] = chunks[i] ~ `F!(Args[` ~ i.stringof ~ 
`]),`;
         string ret = `AliasSeq!(`;
         foreach (chunk; chunks)
             ret ~= `q{alias staticMap = AliasSeq!(` ~ chunk ~ 
`);},`;
         return ret ~ `)`;
         }

         private alias staticMapBasicCases = 
AliasSeq!(mixin(generateCases()));

         template staticMap(alias F, Args...)
         {
             static if (Args.length < staticMapExpandFactor)
                 mixin(staticMapBasicCases[Args.length]);
             else
                 alias staticMap = AliasSeq!(staticMap!(F, Args[0 
.. $ / 2]),
                     staticMap!(F, Args[$ / 2 .. $]));
         }


private alias FilterShortCode = AliasSeq!(
     q{
         alias Filter = Nothing;
     },
     q{
         static if (pred!(TList[0]))
             alias Filter = AliasSeq!(TList[0]);
         else
             alias Filter = Nothing;
     },
     q{
         static if (pred!(TList[0]))
         {
             static if (pred!(TList[1]))
                 alias Filter = AliasSeq!(TList[0], TList[1]);
             else
                 alias Filter = AliasSeq!(TList[0]);
         }
         else
         {
             static if (pred!(TList[1]))
                 alias Filter = AliasSeq!(TList[1]);
             else
                 alias Filter = Nothing;
         }
     },
     q{
         static if (pred!(TList[0]))
         {
             static if (pred!(TList[1]))
             {
                 static if (pred!(TList[2]))
                     alias Filter = AliasSeq!(TList[0], TList[1], 
TList[2]);
                 else
                     alias Filter = AliasSeq!(TList[0], TList[1]);
             }
             else
             {
                 static if (pred!(TList[2]))
                     alias Filter = AliasSeq!(TList[0], TList[2]);
                 else
                     alias Filter = AliasSeq!(TList[0]);
             }
         }
         else
         {
             static if (pred!(TList[1]))
             {
                 static if (pred!(TList[2]))
                     alias Filter = AliasSeq!(TList[1], TList[2]);
                 else
                     alias Filter = AliasSeq!(TList[1]);
             }
             else
             {
                 static if (pred!(TList[2]))
                     alias Filter = AliasSeq!(TList[2]);
                 else
                     alias Filter = Nothing;
             }
         }
     },
     q{
         static if (pred!(TList[0]))
         {
             static if (pred!(TList[1]))
             {
                 static if (pred!(TList[2]))
                 {
                     static if (pred!(TList[3]))
                         alias Filter = AliasSeq!(TList[0], 
TList[1], TList[2], TList[3]);
                     else
                         alias Filter = AliasSeq!(TList[0], 
TList[1], TList[2]);
                 }
                 else
                 {
                     static if (pred!(TList[3]))
                         alias Filter = AliasSeq!(TList[0], 
TList[1], TList[3]);
                     else
                         alias Filter = AliasSeq!(TList[0], 
TList[1]);
                 }
             }
             else
             {
                 static if (pred!(TList[2]))
                 {
                     static if (pred!(TList[3]))
                         alias Filter = AliasSeq!(TList[0], 
TList[2], TList[3]);
                     else
                         alias Filter = AliasSeq!(TList[0], 
TList[2]);
                 }
                 else
                 {
                     static if (pred!(TList[3]))
                         alias Filter = AliasSeq!(TList[0], 
TList[3]);
                     else
                         alias Filter = AliasSeq!(TList[0]);
                 }
             }
         }
         else
         {
             static if (pred!(TList[1]))
             {
                 static if (pred!(TList[2]))
                 {
                     static if (pred!(TList[3]))
                         alias Filter = AliasSeq!(TList[1], 
TList[2], TList[3]);
                     else
                         alias Filter = AliasSeq!(TList[1], 
TList[2]);
                 }
                 else
                 {
                     static if (pred!(TList[3]))
                         alias Filter = AliasSeq!(TList[1], 
TList[3]);
                     else
                         alias Filter = AliasSeq!(TList[1]);
                 }
             }
             else
             {
                 static if (pred!(TList[2]))
                 {
                     static if (pred!(TList[3]))
                         alias Filter = AliasSeq!(TList[2], 
TList[3]);
                     else
                         alias Filter = AliasSeq!(TList[2]);
                 }
                 else
                 {
                     static if (pred!(TList[3]))
                         alias Filter = AliasSeq!(TList[3]);
                     else
                         alias Filter = Nothing;
                 }
             }
         }
     }
);

private enum filterExpandFactor = FilterShortCode.length;

package alias Nothing = AliasSeq!(); // yes, this really does 
speed up compilation!

---

--- actual 3 liner ---
    alias Numerics = AliasSeq!(byte, ubyte, short, ushort, int, 
uint, long,
             ulong, float, double, real, char, wchar, dchar);
         enum convertsTo(T, U) = is(T : U);
         alias ImplicitConversionTargets(T) = 
Filter!(ApplyLeft!(convertsTo, T), Numerics);
         pragma(msg, ImplicitConversionTargets!uint);
---

I would consider this illuminating in a way.



 Oct 06 2020



 next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 6 October 2020 at 10:22:44 UTC, Stefan Koch wrote:

 On Tuesday, 6 October 2020 at 01:12:01 UTC, Walter Bright wrote:

 On 10/5/2020 2:20 PM, Stefan Koch wrote:

 Now post it with all transitive dependencies.


 I'm not sure how transitive dependencies applies here. A 
 illuminating test case would be helpful.


 Okay let me show you the transitive dependencies of the "3 
 liner" suggested by Andrei and executed by Paul Backus.

 --- phobos dependencies ---
 ...


Ah dang.
This won't compile ...
I forgot
---
AliasSeq(T...) = T;
---

with that added it will compile.



 Oct 06 2020



prev sibling  parent reply John Colvin <john.loughran.colvin gmail.com>  writes:


On Tuesday, 6 October 2020 at 10:22:44 UTC, Stefan Koch wrote:

 On Tuesday, 6 October 2020 at 01:12:01 UTC, Walter Bright wrote:

 [...]


 Okay let me show you the transitive dependencies of the "3 
 liner" suggested by Andrei and executed by Paul Backus.

 [...]


That implementation was *definitely* (and very obviously) not 
created for simplicity. It can be done in way fewer lines than 
that (and you know that, obviously).

The argument to be made is "look how ridiculous this code has to 
be to avoid some of the worst performance problems, look how 
simple the type functions approach is and how much faster it is", 
yes?



 Oct 06 2020



 next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 6 October 2020 at 12:07:40 UTC, John Colvin wrote:

 On Tuesday, 6 October 2020 at 10:22:44 UTC, Stefan Koch wrote:

 On Tuesday, 6 October 2020 at 01:12:01 UTC, Walter Bright 
 wrote:

 [...]


 Okay let me show you the transitive dependencies of the "3 
 liner" suggested by Andrei and executed by Paul Backus.

 [...]


 That implementation was *definitely* (and very obviously) not 
 created for simplicity. It can be done in way fewer lines than 
 that (and you know that, obviously).

 The argument to be made is "look how ridiculous this code has 
 to be to avoid some of the worst performance problems, look how 
 simple the type functions approach is and how much faster it 
 is", yes?


Yes that sums it up very well actually.
I am not good enough with recursive templates to simplify the 
implementation  without the risk of introducing bugs 
unfortunately.
Which is why I asked for others to provide code to compare this 
to.



 Oct 06 2020



prev sibling  parent reply claptrap <clap trap.com>  writes:


On Tuesday, 6 October 2020 at 12:07:40 UTC, John Colvin wrote:

 On Tuesday, 6 October 2020 at 10:22:44 UTC, Stefan Koch wrote:

 On Tuesday, 6 October 2020 at 01:12:01 UTC, Walter Bright 
 wrote:

 [...]


 Okay let me show you the transitive dependencies of the "3 
 liner" suggested by Andrei and executed by Paul Backus.

 [...]


 That implementation was *definitely* (and very obviously) not 
 created for simplicity. It can be done in way fewer lines than 
 that (and you know that, obviously).


Maybe "simple and concise" takes a lot more work to extract from 
templates. I mean the type function version is so obvious it's 
probably what a newbie would write, the simple concise solution 
is the obvious one. Is the same true of the template / mixin 
versions?  Or do you have to be a D sage to make it palatable?



 Oct 06 2020



  parent reply John Colvin <john.loughran.colvin gmail.com>  writes:


On Tuesday, 6 October 2020 at 12:32:17 UTC, claptrap wrote:

 On Tuesday, 6 October 2020 at 12:07:40 UTC, John Colvin wrote:

 On Tuesday, 6 October 2020 at 10:22:44 UTC, Stefan Koch wrote:

 On Tuesday, 6 October 2020 at 01:12:01 UTC, Walter Bright 
 wrote:

 [...]


 Okay let me show you the transitive dependencies of the "3 
 liner" suggested by Andrei and executed by Paul Backus.

 [...]


 That implementation was *definitely* (and very obviously) not 
 created for simplicity. It can be done in way fewer lines than 
 that (and you know that, obviously).


 Maybe "simple and concise" takes a lot more work to extract 
 from templates. I mean the type function version is so obvious 
 it's probably what a newbie would write, the simple concise 
 solution is the obvious one. Is the same true of the template / 
 mixin versions?  Or do you have to be a D sage to make it 
 palatable?


If you look at the history of std.meta you'll see it was me who 
recently made Filter so ugly, the previous version was simple but 
too slow.



 Oct 06 2020



  parent reply jmh530 <john.michael.hall gmail.com>  writes:


On Tuesday, 6 October 2020 at 13:20:43 UTC, John Colvin wrote:

 [snip]

 If you look at the history of std.meta you'll see it was me who 
 recently made Filter so ugly, the previous version was simple 
 but too slow.


To get Filter/staticMap to have improved performance (and it 
looks to me like Stefan copied the phobos version in his 
example), you were forced to make them uglier. But, isn't 
Stefan's point that his version has even better performance than 
your ugly version? Is your argument that the simple version would 
have better performance in this case?



 Oct 06 2020



 next sibling parent  John Colvin <john.loughran.colvin gmail.com>  writes:


On Tuesday, 6 October 2020 at 13:35:25 UTC, jmh530 wrote:

 On Tuesday, 6 October 2020 at 13:20:43 UTC, John Colvin wrote:

 [snip]

 If you look at the history of std.meta you'll see it was me 
 who recently made Filter so ugly, the previous version was 
 simple but too slow.


 To get Filter/staticMap to have improved performance (and it 
 looks to me like Stefan copied the phobos version in his 
 example), you were forced to make them uglier. But, isn't 
 Stefan's point that his version has even better performance 
 than your ugly version? Is your argument that the simple 
 version would have better performance in this case?


I'm not really trying to argue anything, just trying to 
inform/clarify a little



 Oct 06 2020



prev sibling  parent reply foobar <foo bar.com>  writes:


On Tuesday, 6 October 2020 at 13:35:25 UTC, jmh530 wrote:

 On Tuesday, 6 October 2020 at 13:20:43 UTC, John Colvin wrote:

 [snip]

 If you look at the history of std.meta you'll see it was me 
 who recently made Filter so ugly, the previous version was 
 simple but too slow.


 To get Filter/staticMap to have improved performance (and it 
 looks to me like Stefan copied the phobos version in his 
 example), you were forced to make them uglier. But, isn't 
 Stefan's point that his version has even better performance 
 than your ugly version? Is your argument that the simple 
 version would have better performance in this case?


That is a newbie thing to say. The big complication swept under 
the rug is the language. If you make the language bigger you can 
always makes things look nicer. It doesn't mean you should.
To moderators: Are you seriously rejecting my posts for 
overquoting? What happened to this community?



 Oct 06 2020



  parent  jmh530 <john.michael.hall gmail.com>  writes:


On Tuesday, 6 October 2020 at 17:50:33 UTC, foobar wrote:

 [snip]

 That is a newbie thing to say. The big complication swept under 
 the rug is the language. If you make the language bigger you 
 can always makes things look nicer. It doesn't mean you should.


Honestly, I was just trying to figure out what he was trying to 
say. He subsequently wrote that he was just providing some 
background information (or something to that effect).


 To moderators: Are you seriously rejecting my posts for 
 overquoting? What happened to this community?


It's automated and has been like that as long as I've been here.



 Oct 06 2020



prev sibling next sibling parent  claptrap <clap trap.com>  writes:


On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:

 Hi,


 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another basic 
 type :))

 As noted previously please do discuss!
 Maybe you like the template version more?
 Let me know.


100% the type functions.

It's also worth nothing that the way the TF version is 
implemented you have a bunch of stuff that is reusable, you could 
argue that...

makeAliasArray
isBasicType

shouldn't be in the line count as they would be general library 
functions anyway.

And at the very least the decomposition into smaller more 
understandable and separately useful parts is as a big win as the 
"easy to explain to a newbie" syntax is.



 Oct 05 2020



prev sibling next sibling parent reply Patrick Schluter <Patrick.Schluter bbox.fr>  writes:


On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:
[...]


 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another basic 
 type :))


cent and ucent are waiting ;-)


 As noted previously please do discuss!
 Maybe you like the template version more?
 Let me know.





 Oct 05 2020



  parent  Patrick Schluter <Patrick.Schluter bbox.fr>  writes:


On Monday, 5 October 2020 at 15:07:37 UTC, Patrick Schluter wrote:

 On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:
 [...]


 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another 
 basic type :))


 cent and ucent are waiting ;-)


ooops, should have read the rest of the thread before 
commenting...



 Oct 05 2020



prev sibling next sibling parent reply Adam D. Ruppe <destructionator gmail.com>  writes:


On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:

 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another basic 
 type :))


Also trivially easy with mixins. Normally I don't suggest doing 
things this way but for basic types mixin is guaranteed to work.

```

enum basic_types = ["bool", "ubyte", "char", "byte", "ushort", 
"wchar", "short", "uint", "dchar", "int", "ulong", "long"];

// to not import stdlib
alias AliasSeq(T...) = T;
string join(string[] s, string j) {
         string a;
         foreach(i; s)
                 a ~= i ~ j;
         return a;
}

template basicTypeConvTargets(T) {
    // again, I copy/pasted your code with very slight 
modifications
     string[] helper() {
             string[] targets;
             targets.length = basic_types.length;
             size_t n = 0;
             static foreach(t;basic_types)
             {
                 if (is(T : mixin(t)))
                 {
                     targets[n++] = t;
                 }
             }
             return targets[0 .. n];
     }

     mixin("alias basicTypeConvTargets = AliasSeq!(" ~ 
helper().join(",") ~ ");");
}

pragma(msg, basicTypeConvTargets!ushort);



 Oct 05 2020



  parent  claptrap <clap trap.com>  writes:


On Monday, 5 October 2020 at 21:20:15 UTC, Adam D. Ruppe wrote:

 On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:

 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another 
 basic type :))


 Also trivially easy with mixins. Normally I don't suggest doing 
 things this way but for basic types mixin is guaranteed to work.

 ```

 enum basic_types = ["bool", "ubyte", "char", "byte", "ushort", 
 "wchar", "short", "uint", "dchar", "int", "ulong", "long"];

 // to not import stdlib
 alias AliasSeq(T...) = T;
 string join(string[] s, string j) {
         string a;
         foreach(i; s)
                 a ~= i ~ j;
         return a;
 }

 template basicTypeConvTargets(T) {
    // again, I copy/pasted your code with very slight 
 modifications
     string[] helper() {
             string[] targets;
             targets.length = basic_types.length;
             size_t n = 0;
             static foreach(t;basic_types)
             {
                 if (is(T : mixin(t)))
                 {
                     targets[n++] = t;
                 }
             }
             return targets[0 .. n];
     }

     mixin("alias basicTypeConvTargets = AliasSeq!(" ~ 
 helper().join(",") ~ ");");
 }

 pragma(msg, basicTypeConvTargets!ushort);


Thats still kinda hideous compared to the TF version. I mean you 
can grep the TF version at first glance, you literally barely 
have to think about it. Your mixin version takes some skipping 
back and forth between the code to work out what its doing.

There's that maxim that your code is read many times more that it 
is written, even by yourself never mind by other people. So a 
piece of code that takes 10 seconds to understand vs 30 seconds 
is a huge win. Faster for people to debug modify, and less likely 
they'll introduce new bugs.

I mean if somebody comes along a year or two down the line to 
modify the TF version and the Mixin version, its not hard to 
imagine which one are they more likely to screw up.

Which makes me wonder what is going on with the people steering D 
that this doesnt seem to have any importance anymore?



 Oct 05 2020



prev sibling next sibling parent reply Meta <jared771 gmail.com>  writes:


On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:

 Hi,

 I've posted an incomplete version of a semantic translation of 
 ImplictConvTargets from a template into a type function.

 After a few rather trivial fixes let me show you what the code 
 looks like now.

 ---
 alias type = alias;

 // needed to avoid deeper changes ... in the future it may be 
 unnecessary.
 auto makeAliasArray(type[] types ...)
 {
     return types;
 }

 enum basic_types = makeAliasArray(bool, ubyte, char, byte, 
 ushort, wchar, short, uint, dchar, int, ulong, long);

 type[] convTargets(type T)
 {
     if (isBasicType(T))
         return basicTypeConvTargets(T);
     return null;
 }

 bool isBasicType(type T)
 {
     foreach(t;basic_types)
     {
         if (is(T == t))
             return true;
     }
     return false;
 }

 type[] basicTypeConvTargets(type T)
 {
     type[] targets;
     targets.length = basic_types.length;
     assert(isBasicType(T), "You may not call this function when 
 you don't have a basic type ... (given: " ~ T.stringof ~ ")");
     size_t n = 0;
     foreach(t;basic_types)
     {
         if (is(T : t))
         {
             targets[n++] = t;
         }
     }
     return targets[0 .. n];
 }
 // 42 lines including whitespace and comments

 pragma(msg, convTargets(long)); // outputs [(ulong), (long)]


It would be very impressive if the following works:

type[] basicTypeConvTargets(type T)
{
     assert(isBasicType(T), "You may not call this function when 
you don't have a basic type ... (given: " ~ T.stringof ~ ")");
     return basic_types.filter!((alias U) => is(T: U)).array;
}

Yes, it instantiates a few templates, but it also demonstrates 
that working with type-values is as easy as working with regular 
values.



 Oct 05 2020



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 5 October 2020 at 22:53:31 UTC, Meta wrote:

 type[] basicTypeConvTargets(type T)
 {
     assert(isBasicType(T), "You may not call this function when 
 you don't have a basic type ... (given: " ~ T.stringof ~ ")");
     return basic_types.filter!((alias U) => is(T: U)).array;
 }

 Yes, it instantiates a few templates, but it also demonstrates 
 that working with type-values is as easy as working with 
 regular values.


That should work actually.

as long as you include "alias type = alias;"
which gives you the type type.
And then you can write
basic_types.filter!((type U) => is(T: U)).array;

If that doesn't work, I am going to fix it shortly.



 Oct 05 2020



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:

 Hi,

 I've posted an incomplete version of a semantic translation of 
 ImplictConvTargets from a template into a type function.

 After a few rather trivial fixes let me show you what the code 
 looks like now.

 ---
 alias type = alias;

 // needed to avoid deeper changes ... in the future it may be 
 unnecessary.
 auto makeAliasArray(type[] types ...)
 {
     return types;
 }

 enum basic_types = makeAliasArray(bool, ubyte, char, byte, 
 ushort, wchar, short, uint, dchar, int, ulong, long);

 type[] convTargets(type T)
 {
     if (isBasicType(T))
         return basicTypeConvTargets(T);
     return null;
 }

 bool isBasicType(type T)
 {
     foreach(t;basic_types)
     {
         if (is(T == t))
             return true;
     }
     return false;
 }

 type[] basicTypeConvTargets(type T)
 {
     type[] targets;
     targets.length = basic_types.length;
     assert(isBasicType(T), "You may not call this function when 
 you don't have a basic type ... (given: " ~ T.stringof ~ ")");
     size_t n = 0;
     foreach(t;basic_types)
     {
         if (is(T : t))
         {
             targets[n++] = t;
         }
     }
     return targets[0 .. n];
 }
 // 42 lines including whitespace and comments

 pragma(msg, convTargets(long)); // outputs [(ulong), (long)]
 ---

 And again here is the part of the template that we just 
 re-implemented:
 ---
     static if (is(T == bool))
         alias ImplicitConversionTargets =
             AliasSeq!(byte, ubyte, short, ushort, int, uint, 
 long, ulong, CentTypeList,
                        float, double, real, char, wchar, dchar);
     else static if (is(T == byte))
         alias ImplicitConversionTargets =
             AliasSeq!(short, ushort, int, uint, long, ulong, 
 CentTypeList,
                        float, double, real, char, wchar, dchar);
     else static if (is(T == ubyte))
         alias ImplicitConversionTargets =
             AliasSeq!(short, ushort, int, uint, long, ulong, 
 CentTypeList,
                        float, double, real, char, wchar, dchar);
     else static if (is(T == short))
         alias ImplicitConversionTargets =
             AliasSeq!(int, uint, long, ulong, CentTypeList, 
 float, double, real);
     else static if (is(T == ushort))
         alias ImplicitConversionTargets =
             AliasSeq!(int, uint, long, ulong, CentTypeList, 
 float, double, real);
     else static if (is(T == int))
         alias ImplicitConversionTargets =
             AliasSeq!(long, ulong, CentTypeList, float, double, 
 real);
     else static if (is(T == uint))
         alias ImplicitConversionTargets =
             AliasSeq!(long, ulong, CentTypeList, float, double, 
 real);
     else static if (is(T == long))
         alias ImplicitConversionTargets = AliasSeq!(float, 
 double, real);
     else static if (is(T == ulong))
         alias ImplicitConversionTargets = AliasSeq!(float, 
 double, real);
     // part omitted because we don't have ucent and cent in our 
 list
     else static if (is(T == char))
         alias ImplicitConversionTargets =
             AliasSeq!(wchar, dchar, byte, ubyte, short, ushort,
                        int, uint, long, ulong, CentTypeList, 
 float, double, real);
     else static if (is(T == wchar))
         alias ImplicitConversionTargets =
             AliasSeq!(dchar, short, ushort, int, uint, long, 
 ulong, CentTypeList,
                        float, double, real);
     else static if (is(T == dchar))
         alias ImplicitConversionTargets =
             AliasSeq!(int, uint, long, ulong, CentTypeList, 
 float, double, real);
     // 41 lines including white-space and comments (only that 
 there is no white-space or comments)
 ---

 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another basic 
 type :))

 As noted previously please do discuss!
 Maybe you like the template version more?
 Let me know.


And here is the performance comparison:

uplimk uplimk-virtual-machine:~/d/dmd$ hyperfine -w 10 
"generated/linux/release/64/dmd xx_tf.d -sktf -c" 
"generated/linux/release/64/dmd xx.d -c " 
"generated/linux/release/64/dmd xx_adam.d -c"

   Time (mean ± σ):       9.1 ms ±   0.4 ms    [User: 5.8 ms, 
System: 3.2 ms]
   Range (min … max):     8.5 ms …  10.7 ms    288 runs


   Time (mean ± σ):      17.3 ms ±   0.6 ms    [User: 11.3 ms, 
System: 5.9 ms]
   Range (min … max):    16.5 ms …  19.5 ms    166 runs


   Time (mean ± σ):      20.4 ms ±   0.7 ms    [User: 15.2 ms, 
System: 5.0 ms]
   Range (min … max):    19.5 ms …  22.7 ms    144 runs

Summary
   'generated/linux/release/64/dmd xx_tf.d -sktf -c' ran
     1.91 ± 0.11 times faster than 'generated/linux/release/64/dmd 
xx.d -c '
     2.24 ± 0.13 times faster than 'generated/linux/release/64/dmd 
xx_adam.d -c'


The source code can be found here:

https://gist.github.com/UplinkCoder/d29dd143b352d28390426a3ffedf9521

So can the benchmark results.
Note: not importing std.meta gives the template version (xx.d) a 
speed boost of 3 milliseconds.



 Oct 06 2020



  parent reply jmh530 <john.michael.hall gmail.com>  writes:


On Tuesday, 6 October 2020 at 11:14:05 UTC, Stefan Koch wrote:

 [snip]
 The source code can be found here:

 https://gist.github.com/UplinkCoder/d29dd143b352d28390426a3ffedf9521

 So can the benchmark results.
 Note: not importing std.meta gives the template version (xx.d) 
 a speed boost of 3 milliseconds.


Thanks for providing these results.

You had previously provided some evidence of a significant 
reduction in memory consumption. You might consider adding that 
information (using these examples) to this gist to bolster your 
argument.



 Oct 06 2020



 next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 6 October 2020 at 11:25:41 UTC, jmh530 wrote:

 On Tuesday, 6 October 2020 at 11:14:05 UTC, Stefan Koch wrote:

 [snip]
 The source code can be found here:

 https://gist.github.com/UplinkCoder/d29dd143b352d28390426a3ffedf9521

 So can the benchmark results.
 Note: not importing std.meta gives the template version (xx.d) 
 a speed boost of 3 milliseconds.


 Thanks for providing these results.

 You had previously provided some evidence of a significant 
 reduction in memory consumption. You might consider adding that 
 information (using these examples) to this gist to bolster your 
 argument.


Memory consumption is reduced by 2x roughly.

(adams and pauls version are about the same, with pauls sucking a 
bit more memory, perhaps because there is more to parse?)

I'll add it to the gist.



 Oct 06 2020



prev sibling  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 6 October 2020 at 11:25:41 UTC, jmh530 wrote:

 On Tuesday, 6 October 2020 at 11:14:05 UTC, Stefan Koch wrote:

 [snip]
 The source code can be found here:

 https://gist.github.com/UplinkCoder/d29dd143b352d28390426a3ffedf9521

 So can the benchmark results.
 Note: not importing std.meta gives the template version (xx.d) 
 a speed boost of 3 milliseconds.


 Thanks for providing these results.

 You had previously provided some evidence of a significant 
 reduction in memory consumption. You might consider adding that 
 information (using these examples) to this gist to bolster your 
 argument.


I've just added the gist: 
https://gist.github.com/UplinkCoder/d29dd143b352d28390426a3ffedf9521#file-benchmark-results-txt

Perhaps there is a better way than /usr/bin/time ?



 Oct 06 2020



prev sibling  parent reply Daniel K <dkm4i1 gmail.com>  writes:


On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:

 Maybe you like the template version more?
 Let me know.


I love the premise. Like watching people in TV ads compare the 
knife of the competition with their own. Completely mutilating a 
loaf of bread with the competition.

What if we make an honest comparison?
I spent 15 minutes writing this:

template Tuple(T...)
{
     alias Tuple = T;
}

alias basic_types = Tuple!(bool, ubyte, char, byte, ushort, 
wchar, short, uint, dchar, int, ulong, long);

template ImplicitConvertible(T, Candidate, Candidates...)
{
	static if (is (T : Candidate))
		alias Include = Candidate;
	else
		alias Include = Tuple!();

	static if (Candidates.length > 0)
		alias ImplicitConvertible = Tuple!(Include, 
ImplicitConvertible!(T, Candidates));
	else
		alias ImplicitConvertible = Include;
}

template ImplicitConversionTargets(T)
{
	alias ImplicitConversionTargets = ImplicitConvertible!(T, 
basic_types);
}

pragma (msg, ImplicitConversionTargets!(long));

That's 16 lines of code. Heck it even compiles in D1 if only the 
alias declarations are written in the old style.

Personally I prefer using existing language features.

/Daniel K



 Oct 06 2020



 next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 6 October 2020 at 18:00:29 UTC, Daniel K wrote:

 On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:

 [...]


 I love the premise. Like watching people in TV ads compare the 
 knife of the competition with their own. Completely mutilating 
 a loaf of bread with the competition.

 [...]


As is your right.
I spent 5 minutes on the type-function, because there's nothing 
to think about.
You just loop and that's it :)



 Oct 06 2020



  parent  Daniel K <dkm4i1 gmail.com>  writes:


On Tuesday, 6 October 2020 at 18:43:42 UTC, Stefan Koch wrote:

 On Tuesday, 6 October 2020 at 18:00:29 UTC, Daniel K wrote:

 On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:

 [...]


 I love the premise. Like watching people in TV ads compare the 
 knife of the competition with their own. Completely mutilating 
 a loaf of bread with the competition.

 [...]


 As is your right.
 I spent 5 minutes on the type-function, because there's nothing 
 to think about.
 You just loop and that's it :)


I like how you concisely argued technical aspects. Only people 
with weak arguments would have picked irrelevant points. Wait... 
oh...

/Daniel K



 Oct 06 2020



prev sibling  parent reply claptrap <clap trap.com>  writes:


On Tuesday, 6 October 2020 at 18:00:29 UTC, Daniel K wrote:

 On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:

 template ImplicitConversionTargets(T)
 {
 	alias ImplicitConversionTargets = ImplicitConvertible!(T, 
 basic_types);
 }

 pragma (msg, ImplicitConversionTargets!(long));

 That's 16 lines of code. Heck it even compiles in D1 if only 
 the alias declarations are written in the old style.

 Personally I prefer using existing language features.

 /Daniel K


If someone said to you "I have this list of integers and I want 
to get a new list containing the ones are divisible by a specific 
value", would you write this...

----

int[] vals = [4,7,28,23,585,73,12];

int[] getMultiplesX(int i, int candidate, int[] candidates)
{
     int[] result;
     if ((candidate % i) == 0)
         result ~= candidate;
     if (candidates.length > 0)
         return result ~ getMultiplesX(i, candidates[0], 
candidates[1..$]);
     else
        return result;
}

int[]  getMultiplesOf(int i)
{
     return getMultiplesX(i, vals[0], vals[1..$]);
}

----

Or would you write it like this...

int[] vals = [4,7,28,23,585,73,12];

int[] getMultiplesOf(int i)
{
     int[] result;
     foreach(v; vals)
         if ((v % i) == 0) result ~= v;
     return result;
}

----

Its the same basic algorithm, search a list and make a new list 
from the entries that satisfy a certain condition. It beggars 
belief that anyone would argue that the second version is not 
better on every metric that is important in writing good code. 
It's clearer, more intuitive, more concise, it will almost 
definitely be faster and less wasteful of resources. Newbies will 
grep it far quicker than the other versions. That means faster to 
write and easier to maintain and debug.

I'm honestly losing the will to live here.



 Oct 06 2020



 next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx>  writes:


On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via Digitalmars-d wrote:
[...]

 If someone said to you "I have this list of integers and I want to get
 a new list containing the ones are divisible by a specific value",
 would you write this...
 
 ----
 
 int[] vals = [4,7,28,23,585,73,12];
 
 int[] getMultiplesX(int i, int candidate, int[] candidates)
 {
     int[] result;
     if ((candidate % i) == 0)
         result ~= candidate;
     if (candidates.length > 0)
         return result ~ getMultiplesX(i, candidates[0], candidates[1..$]);
     else
        return result;
 }
 
 int[]  getMultiplesOf(int i)
 {
     return getMultiplesX(i, vals[0], vals[1..$]);
 }
 
 ----
 
 Or would you write it like this...
 
 int[] vals = [4,7,28,23,585,73,12];
 
 int[] getMultiplesOf(int i)
 {
     int[] result;
     foreach(v; vals)
         if ((v % i) == 0) result ~= v;
     return result;
 }


I would write it like this:

	int[] vals = [4,7,28,23,585,73,12];
	
	int[] getMultiplesOf(int i)
	{
	    return vals.filter!(v => (v % i) == 0).array;
	}

One line vs. 4, even more concise. ;-)

Thing is, what someone finds intuitive or not, is a pretty subjective
matter, and depends on what programming style he's familiar with and/or
prefers.  What a C programmer finds readable and obvious may be
needlessly arcane to a Java programmer, and what an APL programmer finds
totally obvious may be completely impenetrable to anyone else. :-P

If we're going to argue over merits, it would really help resolve
matters if we stuck to things that are objectively measurable, since
reasonable people are going to disagree on the subjective points.


T

-- 
Being able to learn is a great learning; being able to unlearn is a greater
learning.



 Oct 06 2020



  parent reply claptrap <clap trap.com>  writes:


On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:

 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via 
 Digitalmars-d wrote: [...]


 I would write it like this:

 	int[] vals = [4,7,28,23,585,73,12];

 	int[] getMultiplesOf(int i)
 	{
 	    return vals.filter!(v => (v % i) == 0).array;
 	}

 One line vs. 4, even more concise. ;-)


The point is to show language not library.



 Thing is, what someone finds intuitive or not, is a pretty 
 subjective matter, and depends on what programming style he's 
 familiar with and/or prefers.  What a C programmer finds 
 readable and obvious may be needlessly arcane to a Java 
 programmer, and what an APL programmer finds totally obvious 
 may be completely impenetrable to anyone else. :-P


We're not looking for "is this intuitive to Java programmers", 
we're asking is this intuitive to D programmers, so if they 
already know D then *you have context* in which to judge whether 
it's intuitive or not. And "It's just like regular D code but 
with types" pretty much hits the nail on the head as fair as 
intuitive goes.



 Oct 06 2020



 next sibling parent reply Daniel K <dkm4i1 gmail.com>  writes:


On Wednesday, 7 October 2020 at 01:07:17 UTC, claptrap wrote:

 On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:

 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via 
 Digitalmars-d wrote: [...]


 I would write it like this:

 	int[] vals = [4,7,28,23,585,73,12];

 	int[] getMultiplesOf(int i)
 	{
 	    return vals.filter!(v => (v % i) == 0).array;
 	}

 One line vs. 4, even more concise. ;-)


 The point is to show language not library.



 Thing is, what someone finds intuitive or not, is a pretty 
 subjective matter, and depends on what programming style he's 
 familiar with and/or prefers.  What a C programmer finds 
 readable and obvious may be needlessly arcane to a Java 
 programmer, and what an APL programmer finds totally obvious 
 may be completely impenetrable to anyone else. :-P


 We're not looking for "is this intuitive to Java programmers", 
 we're asking is this intuitive to D programmers, so if they 
 already know D then *you have context* in which to judge 
 whether it's intuitive or not. And "It's just like regular D 
 code but with types" pretty much hits the nail on the head as 
 fair as intuitive goes.


If recursive templates are not intuitive to you, perhaps you 
still have more D to learn, to become this mythical "D 
programmer".

My 16 lines of template essentially compiled in D for at least 
the past 10 years.
It literally is "regular D code".

If recursion and declarative programming isn't intuitive to you 
in general, then perhaps that's not D's problem at all.

/Daniel K



 Oct 06 2020



  parent  claptrap <clap trap.com>  writes:


On Wednesday, 7 October 2020 at 02:10:03 UTC, Daniel K wrote:

 On Wednesday, 7 October 2020 at 01:07:17 UTC, claptrap wrote:

 On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:

 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via 
 Digitalmars-d wrote: [...]


 I would write it like this:

 	int[] vals = [4,7,28,23,585,73,12];

 	int[] getMultiplesOf(int i)
 	{
 	    return vals.filter!(v => (v % i) == 0).array;
 	}

 One line vs. 4, even more concise. ;-)


 The point is to show language not library.



 Thing is, what someone finds intuitive or not, is a pretty 
 subjective matter, and depends on what programming style he's 
 familiar with and/or prefers.  What a C programmer finds 
 readable and obvious may be needlessly arcane to a Java 
 programmer, and what an APL programmer finds totally obvious 
 may be completely impenetrable to anyone else. :-P


 We're not looking for "is this intuitive to Java programmers", 
 we're asking is this intuitive to D programmers, so if they 
 already know D then *you have context* in which to judge 
 whether it's intuitive or not. And "It's just like regular D 
 code but with types" pretty much hits the nail on the head as 
 fair as intuitive goes.


 If recursive templates are not intuitive to you, perhaps you 
 still have more D to learn, to become this mythical "D 
 programmer".


There's 2 or 3 times more stuff going on in the template version. 
Two function calls, more things passed around, array or alias seq 
slicing, appending, more ifs and elses, etc...

Is less intuitive because it's an algorithm that makes more sense 
as iteration, so you have to bend it all out of shape to make it 
recursive.

I like code as simple as possible, that isnt it.



 My 16 lines of template essentially compiled in D for at least 
 the past 10 years.
 It literally is "regular D code".


By regular D code, I mean non-teplate / meta code. I thought that 
was obvious.



 If recursion and declarative programming isn't intuitive to you 
 in general, then perhaps that's not D's problem at all.


Recursion is fine, there's algorithms that make more sense that 
way, this isnt one of them. Thats the point. You're forcing 
everything in meta land to be recursive.

If all you have is a hammer....

And if all you've had is a hammer for 10 years, you start to 
doubt the existing of screws.



 Oct 07 2020



prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 10/6/20 9:07 PM, claptrap wrote:

 On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:

 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via Digitalmars-d 
 wrote: [...]


 I would write it like this:

     int[] vals = [4,7,28,23,585,73,12];

     int[] getMultiplesOf(int i)
     {
         return vals.filter!(v => (v % i) == 0).array;
     }

 One line vs. 4, even more concise. ;-)


 
 The point is to show language not library.


That's a made-up restriction, and it's odd that it is being discussed 
here as a virtue.

Beginners are attracted to large languages that have everything built 
in. A good language is focused on general primitives that allow writing 
a great deal in libraries.



 Oct 06 2020



 next sibling parent  claptrap <clap trap.com>  writes:


On Wednesday, 7 October 2020 at 02:33:21 UTC, Andrei Alexandrescu 
wrote:

 On 10/6/20 9:07 PM, claptrap wrote:

 On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:

 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via 
 Digitalmars-d wrote: [...]


 I would write it like this:

     int[] vals = [4,7,28,23,585,73,12];

     int[] getMultiplesOf(int i)
     {
         return vals.filter!(v => (v % i) == 0).array;
     }

 One line vs. 4, even more concise. ;-)


 
 The point is to show language not library.


 That's a made-up restriction, and it's odd that it is being 
 discussed here as a virtue.


If you're trying to compare how the two features work in practice 
it's not going to be very illuminating if you hide all the 
workings behind a function call. Ie what good is

int[] getMultiplesOf(int i)
{
     return vals.tf_filter(v => (v % i) == 0);
}

vs...

int[] getMultiplesOf(int i)
{
     return vals.filter!(v => (v % i) == 0).array;
}

Yeah that helps a lot :)



 Oct 07 2020



prev sibling next sibling parent reply Patrick Schluter <Patrick.Schluter bbox.fr>  writes:


On Wednesday, 7 October 2020 at 02:33:21 UTC, Andrei Alexandrescu 
wrote:

 On 10/6/20 9:07 PM, claptrap wrote:

 On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:

 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via 
 Digitalmars-d wrote: [...]


 I would write it like this:

     int[] vals = [4,7,28,23,585,73,12];

     int[] getMultiplesOf(int i)
     {
         return vals.filter!(v => (v % i) == 0).array;
     }

 One line vs. 4, even more concise. ;-)


 
 The point is to show language not library.


 That's a made-up restriction, and it's odd that it is being 
 discussed here as a virtue.


No, it's not. It's central to the argument.


 Beginners are attracted to large languages that have everything 
 built in. A good language is focused on general primitives that 
 allow writing a great deal in libraries.


Then do lisp or forth but not D or C++.



 Oct 07 2020



 next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com>  writes:


On Wednesday, 7 October 2020 at 08:49:21 UTC, Patrick Schluter 
wrote:

 On Wednesday, 7 October 2020 at 02:33:21 UTC, Andrei 
 Alexandrescu wrote:



 That's a made-up restriction, and it's odd that it is being 
 discussed here as a virtue.


 No, it's not. It's central to the argument.


Exactly right. There comes a time when you try to do something 
that isn't covered by libraries,  what do you do then?


 Beginners are attracted to large languages that have 
 everything built in. A good language is focused on general 
 primitives that allow writing a great deal in libraries.


 Then do lisp or forth but not D or C++.


Right again. Funtional programming is only pleasant in a 
dedicated FP language, and even then you need to memorize a large 
set of library constructs to be productive.

D could improve on FP, both language and performance, but adding 
type variables is a better cultural fit... (And Stefan is only 
putting forth a concept, not the end result)



 Oct 07 2020



  parent reply claptrap  <clap trap.com>  writes:


On Wednesday, 7 October 2020 at 09:04:59 UTC, Ola Fosheim Grøstad 
wrote:

 On Wednesday, 7 October 2020 at 08:49:21 UTC, Patrick Schluter 
 wrote:

 On Wednesday, 7 October 2020 at 02:33:21 UTC, Andrei 
 Alexandrescu wrote:



 Right again. Funtional programming is only pleasant in a 
 dedicated FP language, and even then you need to memorize a 
 large set of library constructs to be productive.


How many pure funtional languages are in the tiobe top 10? None 
or maybe 1 im not sure. Theres a reason for that i reckon.



 Oct 07 2020



  parent reply Paolo Invernizzi <paolo.invernizzi gmail.com>  writes:


On Wednesday, 7 October 2020 at 11:19:24 UTC, claptrap wrote:

 On Wednesday, 7 October 2020 at 09:04:59 UTC, Ola Fosheim 
 Grøstad wrote:

 On Wednesday, 7 October 2020 at 08:49:21 UTC, Patrick Schluter 
 wrote:

 On Wednesday, 7 October 2020 at 02:33:21 UTC, Andrei 
 Alexandrescu wrote:



 Right again. Funtional programming is only pleasant in a 
 dedicated FP language, and even then you need to memorize a 
 large set of library constructs to be productive.


 How many pure funtional languages are in the tiobe top 10? None 
 or maybe 1 im not sure. Theres a reason for that i reckon.


We are using a pure functional language for web frontend 
development, and it's a joy ...



 Oct 07 2020



  parent reply claptrap <clap trap.com>  writes:


On Wednesday, 7 October 2020 at 11:55:00 UTC, Paolo Invernizzi 
wrote:

 On Wednesday, 7 October 2020 at 11:19:24 UTC, claptrap wrote:

 On Wednesday, 7 October 2020 at 09:04:59 UTC, Ola Fosheim 
 Grøstad wrote:

 On Wednesday, 7 October 2020 at 08:49:21 UTC, Patrick 
 Schluter wrote:

 On Wednesday, 7 October 2020 at 02:33:21 UTC, Andrei 
 Alexandrescu wrote:



 Right again. Funtional programming is only pleasant in a 
 dedicated FP language, and even then you need to memorize a 
 large set of library constructs to be productive.


 How many pure funtional languages are in the tiobe top 10? 
 None or maybe 1 im not sure. Theres a reason for that i reckon.


 We are using a pure functional language for web frontend 
 development, and it's a joy ...


Maybe they're great in specific circumstances? I dont know what 
the reason is but if FP was so intuitive then you'd expect it'd 
be the norm, or at least on a par, or even in the same ballpark 
as imperative?



 Oct 07 2020



  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 07.10.20 14:32, claptrap wrote:

 On Wednesday, 7 October 2020 at 11:55:00 UTC, Paolo Invernizzi wrote:

 On Wednesday, 7 October 2020 at 11:19:24 UTC, claptrap wrote:

 On Wednesday, 7 October 2020 at 09:04:59 UTC, Ola Fosheim Grøstad wrote:

 On Wednesday, 7 October 2020 at 08:49:21 UTC, Patrick Schluter wrote:

 On Wednesday, 7 October 2020 at 02:33:21 UTC, Andrei Alexandrescu 
 wrote:



 Right again. Funtional programming is only pleasant in a dedicated 
 FP language, and even then you need to memorize a large set of 
 library constructs to be productive.


 How many pure funtional languages are in the tiobe top 10? None or 
 maybe 1 im not sure. Theres a reason for that i reckon.


 We are using a pure functional language for web frontend development, 
 and it's a joy ...


 
 Maybe they're great in specific circumstances? I dont know what the 
 reason is but if FP was so intuitive then you'd expect it'd be the norm, 
 or at least on a par, or even in the same ballpark as imperative?


This particular discussion is not about functional vs imperative at all, 
it is about awkward vs decent.



 Oct 07 2020



prev sibling  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 10/7/20 4:49 AM, Patrick Schluter wrote:

 On Wednesday, 7 October 2020 at 02:33:21 UTC, Andrei Alexandrescu wrote:

 On 10/6/20 9:07 PM, claptrap wrote:

 On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:

 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via Digitalmars-d 
 wrote: [...]


 I would write it like this:

     int[] vals = [4,7,28,23,585,73,12];

     int[] getMultiplesOf(int i)
     {
         return vals.filter!(v => (v % i) == 0).array;
     }

 One line vs. 4, even more concise. ;-)


 The point is to show language not library.


 That's a made-up restriction, and it's odd that it is being discussed 
 here as a virtue.


 
 No, it's not. It's central to the argument.


Then the argument is specious.

I've been also tempted to do this on occasion to tilt a comparison one 
way or another - take C++ without STL or Boost, or Haskell without 
Prelude. The reality is these need to be considered together. (Make a 
hashtable in C++, no standard library allowed...)


 Beginners are attracted to large languages that have everything built 
 in. A good language is focused on general primitives that allow 
 writing a great deal in libraries.


 
 Then do lisp or forth but not D or C++.


Much of the size of C++ and D caters to library writers, and their 
standard libraries grew way faster than the core language - as they should.



 Oct 07 2020



 next sibling parent  Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com>  writes:


On Wednesday, 7 October 2020 at 11:10:15 UTC, Andrei Alexandrescu 
wrote:

 I've been also tempted to do this on occasion to tilt a 
 comparison one way or another - take C++ without STL or Boost, 
 or Haskell without Prelude. The reality is these need to be 
 considered together. (Make a hashtable in C++, no standard


Your argument would work for Go, but not for C++.

Lots of C++ codebases rely only on the most basic stuff like 
malloc and memcpy.

boost is only useful for prototyping, you find better dedicated 
libraries. same for much of STL

Cppcon has a 2 HOURS presentation explaning just type traits... 
That tells me that something went wrong somewhere... Not generic 
enough, too large vocubalary.

Standard libs should be small and composable.

Over 80% of C++ stdlib is useless or outdated. Of course they 
have like a 20+ years deprecation cycle...



 Oct 07 2020



prev sibling  parent reply claptrap <clap trap.com>  writes:


On Wednesday, 7 October 2020 at 11:10:15 UTC, Andrei Alexandrescu 
wrote:

 On 10/7/20 4:49 AM, Patrick Schluter wrote:

 On Wednesday, 7 October 2020 at 02:33:21 UTC, Andrei 
 Alexandrescu wrote:

 On 10/6/20 9:07 PM, claptrap wrote:

 On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:

 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via 
 Digitalmars-d wrote: [...]


 I would write it like this:

     int[] vals = [4,7,28,23,585,73,12];

     int[] getMultiplesOf(int i)
     {
         return vals.filter!(v => (v % i) == 0).array;
     }

 One line vs. 4, even more concise. ;-)


 The point is to show language not library.


 That's a made-up restriction, and it's odd that it is being 
 discussed here as a virtue.


 
 No, it's not. It's central to the argument.


 Then the argument is specious.

 I've been also tempted to do this on occasion to tilt a 
 comparison one way or another - take C++ without STL or Boost, 
 or Haskell without Prelude. The reality is these need to be 
 considered together. (Make a hashtable in C++, no standard 
 library allowed...)


If you're just asking is this easier in this language or that 
then it is unheplful to say no stdlib. But thats not whats going 
on here. You're comparing two language features within one 
language. The incubant feature has 20 years of library support 
behind it. The other does not.

Lets reverse the roles, say TF were invented first, and somebody 
was arguing for templates to be added now. The TP version would 
be a couple of stdlib calls and the template version a whole page.

Its a nonsense to make a comparison like that.

Maybe your "temptation too tilt" is at play here but you havent 
realised it yet?



 Oct 07 2020



 next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 10/7/20 7:58 AM, claptrap wrote:

 Lets reverse the roles, say TF were invented first, and somebody was 
 arguing for templates to be added now. The TP version would be a couple 
 of stdlib calls and the template version a whole page.


Incumbency is a huge matter in programming language design. Of course I 
would not propose another way of doing the same thing.



 Oct 07 2020



 next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 7 October 2020 at 12:30:15 UTC, Andrei Alexandrescu 
wrote:

 On 10/7/20 7:58 AM, claptrap wrote:

 Lets reverse the roles, say TF were invented first, and 
 somebody was arguing for templates to be added now. The TP 
 version would be a couple of stdlib calls and the template 
 version a whole page.


 Incumbency is a huge matter in programming language design. Of 
 course I would not propose another way of doing the same thing.


I just looked up Incumbency, but the German translation was 'Time 
someone has been in office' ...
I am not seeing how these things relate.
Perhaps you can say this in different words?



 Oct 07 2020



 next sibling parent reply Adam D. Ruppe <destructionator gmail.com>  writes:


On Wednesday, 7 October 2020 at 12:37:48 UTC, Stefan Koch wrote:

 On Wednesday, 7 October 2020 at 12:30:15 UTC, Andrei 
 Alexandrescu wrote:

 Incumbency is a huge matter in programming language design. Of 
 course I would not propose another way of doing the same thing.


 Perhaps you can say this in different words?


Basically we just prefer to work with what we have before adding 
new stuff to the language.

So whatever features get added first have an automatic advantage 
in any comparison just because they are already there.



 Oct 07 2020



  parent reply Daniel K <dkm4i1 gmail.com>  writes:


On Wednesday, 7 October 2020 at 12:56:42 UTC, Adam D. Ruppe wrote:

 On Wednesday, 7 October 2020 at 12:37:48 UTC, Stefan Koch wrote:

 On Wednesday, 7 October 2020 at 12:30:15 UTC, Andrei 
 Alexandrescu wrote:

 Incumbency is a huge matter in programming language design. 
 Of course I would not propose another way of doing the same 
 thing.


 Perhaps you can say this in different words?


 Basically we just prefer to work with what we have before 
 adding new stuff to the language.

 So whatever features get added first have an automatic 
 advantage in any comparison just because they are already there.


I agree.

And existing features got added after already having their 
fundamental ideas, advantages, and deficiencies scrutinized.

The fact that we have meta-programming in the form we have, is 
not by chance. It is a super strong and capable design.
I am biased, but I would say "state of the art".

If we boil down the Type Function proposal to its fundamental 
idea, advantages, and deficiencies. What Do we get?

Fundamental idea:
Language level reification of a set of types, making a set of 
types mutable to enable imperative programming style, for what is 
essentially only changing that set of types.
But at that, only in the context of a special kind of CTFE-like 
function.
Other than mutating a set of types, This function cant do 
anything that cant already be achieved rather simply with 
existing language features. As has been demonstrated a number of 
times in this thread.

Advantage:
Mutating a set of types in an imperative style. Something that is 
already quite simple, and requires virtually the same, or even 
less code in practice when written with existing language 
features.

Disadvantage: A new feature people will have to learn and 
understand, document, maintain, bugfix.
No actual substantial future gain.

Is this wrong?
Or where is that striking idea nobody can spot, that we just must 
enable with language level complexity? If the best answer is 
"Imperative type set manipulation"… Geeezzz…

I would love to see an example, of the epitome of what goodness 
TypeFunctions could unlock. What I've seen so far is basically 
nothing, at the cost of far more than nothing. No Deal.

/Daniel K



 Oct 07 2020



 next sibling parent  Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com>  writes:


On Wednesday, 7 October 2020 at 14:01:51 UTC, Daniel K wrote:

 The fact that we have meta-programming in the form we have, is 
 not by chance. It is a super strong and capable design.
 I am biased, but I would say "state of the art".


State of the art requires rewrite-capabilities like Lisp, Pure 
and many other languages have.

C++ chose simple templating, but got burned as users abused it to 
do more. That aspect of C++ was an unintended design flaw!

D chose the same course, but through a deliberate design. So they 
could have a more readable syntax (the semantics differ too, but 
both have advantages/disadvantages).



 Oct 07 2020



prev sibling next sibling parent  Paul Backus <snarwin gmail.com>  writes:


On Wednesday, 7 October 2020 at 14:01:51 UTC, Daniel K wrote:

 If we boil down the Type Function proposal to its fundamental 
 idea, advantages, and deficiencies. What Do we get?

 [...]

 Advantage:
 Mutating a set of types in an imperative style. Something that 
 is already quite simple, and requires virtually the same, or 
 even less code in practice when written with existing language 
 features.

 Disadvantage: A new feature people will have to learn and 
 understand, document, maintain, bugfix.
 No actual substantial future gain.

 Is this wrong?


The big advantage you're missing is that, because type functions 
are so restricted in what they can do, they can be implemented 
much more efficiently in the compiler. For large programs that 
make heavy use of D's metaprogramming features, that can really 
add up.



 Oct 07 2020



prev sibling  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 7 October 2020 at 14:01:51 UTC, Daniel K wrote:

 On Wednesday, 7 October 2020 at 12:56:42 UTC, Adam D. Ruppe 
 wrote:

 On Wednesday, 7 October 2020 at 12:37:48 UTC, Stefan Koch 
 wrote:

 On Wednesday, 7 October 2020 at 12:30:15 UTC, Andrei 
 Alexandrescu wrote:

 Incumbency is a huge matter in programming language design. 
 Of course I would not propose another way of doing the same 
 thing.


 Perhaps you can say this in different words?


 Basically we just prefer to work with what we have before 
 adding new stuff to the language.

 So whatever features get added first have an automatic 
 advantage in any comparison just because they are already 
 there.


 I agree.

 And existing features got added after already having their 
 fundamental ideas, advantages, and deficiencies scrutinized.

 The fact that we have meta-programming in the form we have, is 
 not by chance. It is a super strong and capable design.
 I am biased, but I would say "state of the art".

 If we boil down the Type Function proposal to its fundamental 
 idea, advantages, and deficiencies. What Do we get?

 Fundamental idea:
 Language level reification of a set of types, making a set of 
 types mutable to enable imperative programming style, for what 
 is essentially only changing that set of types.
 But at that, only in the context of a special kind of CTFE-like 
 function.


This is quite close, thanks for your description.
The only thing I would change is "CTFE-like" to CTFE.

The language changes are as follows:
   - Introduce a new basic type "alias" which is the type of 
types. This type does not have a runtime representation, and can 
only exist at compile time/CTFE.
   - Variables of this type emulate the interface that Type 
parameters in templates have.
   - Add an implicit conversion such that any type implicitly 
converts to alias.
   - Allow functions to be called with Types (TypeExps are a 
concept which already exists in the compiler but not in the 
language)
   - Functions which take alias parameters or types drives of 
alias parameters or (such as alias[], or structs which contain 
alias fields) or return such values cannot be called at runtime, 
and will never be generated in the binary (this is because alias 
does not have an ABI representation but helps out with compile 
speed as well).
   - Lastly (and this one is actually one I am not quite happy 
with) but it's needed is the alias[] -> TypeTuple via applying 
.tupleof to a statically determined (compile-time constant) 
alias[].

The CTFE functionality is 100% reused.
(which also means when newCTFE hits complex type functions get a 
speed boost)
The alias type is integrated into the type-system except for the 
special handling of alias[].tupleof  it behaves as expected.
For example you can do use types as keys for associative arrays 
if you wish to.
Which then allows more freedom in algorithm design in the domain 
of introspection/ compile time code-generation in particular.


 Disadvantage: A new feature people will have to learn and 
 understand, document, maintain, bugfix.
 No actual substantial future gain.


As for the disadvantages. Yes it's a new feature but it blends so 
well with the current language that people on here post correct 
type function code, (presumably) without having a compiler that 
is capable of running it. (It's of course possible that claptrap 
cloned my branch and run the code, but I don't think so ...)

As for the feature gain ... well it's akin to comparing c++ 
templates with D templates.
They are equivalent in power, just that one easier to write than 
the other.

 Daniel K: I am thankful for your analysis, would you be opposed 
if I reused your characterization in the DIP?



 Oct 07 2020



 next sibling parent reply claptrap <clap trap.com>  writes:


On Wednesday, 7 October 2020 at 14:40:41 UTC, Stefan Koch wrote:

 On Wednesday, 7 October 2020 at 14:01:51 UTC, Daniel K wrote:

 On Wednesday, 7 October 2020 at 12:56:42 UTC, Adam D. Ruppe 
 wrote:

 On Wednesday, 7 October 2020 at 12:37:48 UTC, Stefan Koch 
 wrote:

 On Wednesday, 7 October 2020 at









 As for the disadvantages. Yes it's a new feature but it blends 
 so well with the current language that people on here post 
 correct type function code, (presumably) without having a 
 compiler that is capable of running it. (It's of course 
 possible that claptrap cloned my branch and run the code, but I 
 don't think so ...)


I just based it off what youve posted in this thread.

Maybe my perspective as a relative newbie to D makes me see that 
as a much bigger payoff than some of the resident greybeards.



 Oct 07 2020



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 7 October 2020 at 14:59:04 UTC, claptrap wrote:

 On Wednesday, 7 October 2020 at 14:40:41 UTC, Stefan Koch wrote:

 On Wednesday, 7 October 2020 at 14:01:51 UTC, Daniel K wrote:

 On Wednesday, 7 October 2020 at 12:56:42 UTC, Adam D. Ruppe 
 wrote:

 On Wednesday, 7 October 2020 at 12:37:48 UTC, Stefan Koch 
 wrote:

 On Wednesday, 7 October 2020 at









 As for the disadvantages. Yes it's a new feature but it blends 
 so well with the current language that people on here post 
 correct type function code, (presumably) without having a 
 compiler that is capable of running it. (It's of course 
 possible that claptrap cloned my branch and run the code, but 
 I don't think so ...)


 I just based it off what youve posted in this thread.

 Maybe my perspective as a relative newbie to D makes me see 
 that as a much bigger payoff than some of the resident 
 greybeards.


You might be happy to see that this exmaple just works with 
talias_master:

---
alias type = alias;

struct KV
{
     string key;
     type value;
}

auto makeAA(KV[] kvs ...)
{
     type[string] result;
     foreach(kv;kvs)
     {
         result[kv.key] = kv.value;
     }
     return result;
}

pragma(msg, makeAA(KV("u32", uint), KV("kv", KV)));
// outputs: ["u32":(uint), "kv":(KV)]
--



 Oct 07 2020



prev sibling  parent reply Daniel K <dkm4i1 gmail.com>  writes:


On Wednesday, 7 October 2020 at 14:40:41 UTC, Stefan Koch wrote:

  Daniel K: I am thankful for your analysis, would you be 
 opposed if I reused your characterization in the DIP?


You are free to use any part of it, if it doesn't reference back 
to me or "the community".
But if you use it to insinuate endorsement from me or "the 
community", you will have to include all of the opposing points. 
No cherry picking.

/Daniel K



 Oct 07 2020



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 7 October 2020 at 15:00:16 UTC, Daniel K wrote:

 On Wednesday, 7 October 2020 at 14:40:41 UTC, Stefan Koch wrote:

  Daniel K: I am thankful for your analysis, would you be 
 opposed if I reused your characterization in the DIP?


 You are free to use any part of it, if it doesn't reference 
 back to me or "the community".
 But if you use it to insinuate endorsement from me or "the 
 community", you will have to include all of the opposing 
 points. No cherry picking.

 /Daniel K


That is fair.
Thanks!



 Oct 07 2020



prev sibling  parent  claptrap <clap trap.com>  writes:


On Wednesday, 7 October 2020 at 12:37:48 UTC, Stefan Koch wrote:

 On Wednesday, 7 October 2020 at 12:30:15 UTC, Andrei 
 Alexandrescu wrote:

 On 10/7/20 7:58 AM, claptrap wrote:

 Lets reverse the roles, say TF were invented first, and 
 somebody was arguing for templates to be added now. The TP 
 version would be a couple of stdlib calls and the template 
 version a whole page.


 Incumbency is a huge matter in programming language design. Of 
 course I would not propose another way of doing the same thing.


 I just looked up Incumbency, but the German translation was 
 'Time someone has been in office' ...
 I am not seeing how these things relate.
 Perhaps you can say this in different words?


incumbent = the established order.

It's usually used in the context of a politician in office who 
defending his position rather than challenging for it. IE. Trump 
is the incumbent, Biden is the challenger.



 Oct 07 2020



prev sibling  parent  Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com>  writes:


On Wednesday, 7 October 2020 at 12:30:15 UTC, Andrei Alexandrescu 
wrote:

 On 10/7/20 7:58 AM, claptrap wrote:

 Lets reverse the roles, say TF were invented first, and 
 somebody was arguing for templates to be added now. The TP 
 version would be a couple of stdlib calls and the template 
 version a whole page.


 Incumbency is a huge matter in programming language design. Of 
 course I would not propose another way of doing the same thing.


c++ concepts is another way of doing the same thing.
So you are basically saying they should not have done it?

c++ modules is another way of doing the same thing.
Etc.

C++ concepts is a prettier and more useful version with compile 
traits qualities.
Probably the most important improvement since RAII...

Nobody that values their time wants to juggle types in c++. D 
isnt on a different plane either.

Improving usability is a survivalstrategy for any language...

Haskell is failing for that reason, it is not going to improve. 
It is primarily a research test bed, just like ML and 
descendants. Academic languages evolve by death and rebirth, not 
by additions. So if you want D3, ok. But if you want D2, then you 
cannot pick the path that academics follow.



 Oct 07 2020



prev sibling  parent reply Adam D. Ruppe <destructionator gmail.com>  writes:


On Wednesday, 7 October 2020 at 11:58:35 UTC, claptrap wrote:

 Lets reverse the roles, say TF were invented first, and 
 somebody was arguing for templates to be added now. The TP 
 version would be a couple of stdlib calls and the template 
 version a whole page.


Templates actually enable all kinds of new things.

A type function cannot actually generate any new code nor create 
new types. Those are explicit limitations in the design right 
now, because by keeping them so strict it enables some 
optimizations that are... at best tricky with templates (many 
attempts to optimize templates end up causing compiler bug errors 
or, perhaps worse yet, random linker errors).

So supposed we lived in a world with only TF. You can make type 
lists and return basic type constants based on limited reflection 
over type (note that TFs will probably *not* support advanced 
`is` expression pattern matching as it is right now!)

Then someone comes along and says let's add templates. Templates 
can do everything a type function can do AND you can generate new 
code based on that information! No more awkwardly returning 
string literals and mixing them in at the usage point, the 
compiler will generate it all directly!

Oh there'd probably be pushback, I'd probably be arguing "but we 
can already mixin!" myself :) (though mixin sucks for name 
generation, you have to use __LINE__ or cookie string hacks, 
whereas templates automatically get unique names!)

But it'd also be a compelling new feature.

That's why I'd be a bit more excited about the type functions if 
they prove capable of doing things we can't really do right now, 
like transform function argument lists through tuple 
manipulation. That'd be a cool application of the same general 
principle of reordering alias[] entities.



 Oct 07 2020



  parent  Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmaill.com>  writes:


On Wednesday, 7 October 2020 at 12:54:09 UTC, Adam D. Ruppe wrote:

 That's why I'd be a bit more excited about the type functions 
 if they prove capable of doing things we can't really do right 
 now, like transform function argument lists through tuple 
 manipulation. That'd be a cool application of the same general 
 principle of reordering alias[] entities.


I think you should just add type variables and improve on other 
mechanisms until you have those capabilities, the you restrict 
the usage of type variables to what Stefan can implement so that 
this feature gets a release date. After that it can be extended 
one step at the time, maybe even to runtime.

The whole has to be specced out before release, but the initial 
release can be constrained to a subset.



 Oct 07 2020



prev sibling  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 07.10.20 04:33, Andrei Alexandrescu wrote:

 On 10/6/20 9:07 PM, claptrap wrote:

 On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:

 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via Digitalmars-d 
 wrote: [...]


 I would write it like this:

     int[] vals = [4,7,28,23,585,73,12];

     int[] getMultiplesOf(int i)
     {
         return vals.filter!(v => (v % i) == 0).array;
     }

 One line vs. 4, even more concise. ;-)


 The point is to show language not library.


 
 That's a made-up restriction, and it's odd that it is being discussed 
 here as a virtue.
 ...


I agree. This is a bad way to market a new language feature. However, I 
don't think "library over language" is necessarily a good justification 
for an inefficient implementation of core language features.


 Beginners are attracted to large languages that have everything built in.


Isn't it rather that beginners can't tell the difference?


 A good language is focused on general primitives that allow writing 
 a great deal in libraries.
 
 


Certainly, but a good language does not require you to write the same 
functionality multiple times.



 Oct 07 2020



prev sibling  parent  foobar <foo bar.com>  writes:


On Wednesday, 7 October 2020 at 01:07:17 UTC, claptrap wrote:

 On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:

 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via 
 Digitalmars-d wrote: [...]


 I would write it like this:

 	int[] vals = [4,7,28,23,585,73,12];

 	int[] getMultiplesOf(int i)
 	{
 	    return vals.filter!(v => (v % i) == 0).array;
 	}

 One line vs. 4, even more concise. ;-)


 The point is to show language not library.


Why?



 Oct 06 2020



prev sibling  parent reply Patrick Schluter <Patrick.Schluter bbox.fr>  writes:


On Tuesday, 6 October 2020 at 23:16:47 UTC, claptrap wrote:
[...]

 Its the same basic algorithm, search a list and make a new list 
 from the entries that satisfy a certain condition. It beggars 
 belief that anyone would argue that the second version is not 
 better on every metric that is important in writing good code. 
 It's clearer, more intuitive, more concise, it will almost 
 definitely be faster and less wasteful of resources. Newbies 
 will grep it far quicker than the other versions. That means 
 faster to write and easier to maintain and debug.

 I'm honestly losing the will to live here.


Thank you. I was myself wondering why this feature was 
encountering such an amount of resistance, especially from the 
top tier of the language specialists. Maybe they are so used to 
the complexity of the template language that they don't perceive 
how challenging for the mere mortal it is in practice.



 Oct 07 2020



  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 07.10.20 10:25, Patrick Schluter wrote:

 I'm honestly losing the will to live here.


 
 Thank you. I was myself wondering why this feature was encountering such 
 an amount of resistance, especially from the top tier of the language 
 specialists.


Fear of losing the edge? :o)



 Oct 07 2020