www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - TypeFunction example: ImplictConvTargets

reply Stefan Koch <uplink.coder googlemail.com> writes:
Hi,

I've posted an incomplete version of a semantic translation of 
ImplictConvTargets from a template into a type function.

After a few rather trivial fixes let me show you what the code 
looks like now.

---
alias type = alias;

// needed to avoid deeper changes ... in the future it may be 
unnecessary.
auto makeAliasArray(type[] types ...)
{
     return types;
}

enum basic_types = makeAliasArray(bool, ubyte, char, byte, 
ushort, wchar, short, uint, dchar, int, ulong, long);

type[] convTargets(type T)
{
     if (isBasicType(T))
         return basicTypeConvTargets(T);
     return null;
}

bool isBasicType(type T)
{
     foreach(t;basic_types)
     {
         if (is(T == t))
             return true;
     }
     return false;
}

type[] basicTypeConvTargets(type T)
{
     type[] targets;
     targets.length = basic_types.length;
     assert(isBasicType(T), "You may not call this function when 
you don't have a basic type ... (given: " ~ T.stringof ~ ")");
     size_t n = 0;
     foreach(t;basic_types)
     {
         if (is(T : t))
         {
             targets[n++] = t;
         }
     }
     return targets[0 .. n];
}
// 42 lines including whitespace and comments

pragma(msg, convTargets(long)); // outputs [(ulong), (long)]
---

And again here is the part of the template that we just 
re-implemented:
---
     static if (is(T == bool))
         alias ImplicitConversionTargets =
             AliasSeq!(byte, ubyte, short, ushort, int, uint, 
long, ulong, CentTypeList,
                        float, double, real, char, wchar, dchar);
     else static if (is(T == byte))
         alias ImplicitConversionTargets =
             AliasSeq!(short, ushort, int, uint, long, ulong, 
CentTypeList,
                        float, double, real, char, wchar, dchar);
     else static if (is(T == ubyte))
         alias ImplicitConversionTargets =
             AliasSeq!(short, ushort, int, uint, long, ulong, 
CentTypeList,
                        float, double, real, char, wchar, dchar);
     else static if (is(T == short))
         alias ImplicitConversionTargets =
             AliasSeq!(int, uint, long, ulong, CentTypeList, 
float, double, real);
     else static if (is(T == ushort))
         alias ImplicitConversionTargets =
             AliasSeq!(int, uint, long, ulong, CentTypeList, 
float, double, real);
     else static if (is(T == int))
         alias ImplicitConversionTargets =
             AliasSeq!(long, ulong, CentTypeList, float, double, 
real);
     else static if (is(T == uint))
         alias ImplicitConversionTargets =
             AliasSeq!(long, ulong, CentTypeList, float, double, 
real);
     else static if (is(T == long))
         alias ImplicitConversionTargets = AliasSeq!(float, 
double, real);
     else static if (is(T == ulong))
         alias ImplicitConversionTargets = AliasSeq!(float, 
double, real);
     // part omitted because we don't have ucent and cent in our 
list
     else static if (is(T == char))
         alias ImplicitConversionTargets =
             AliasSeq!(wchar, dchar, byte, ubyte, short, ushort,
                        int, uint, long, ulong, CentTypeList, 
float, double, real);
     else static if (is(T == wchar))
         alias ImplicitConversionTargets =
             AliasSeq!(dchar, short, ushort, int, uint, long, 
ulong, CentTypeList,
                        float, double, real);
     else static if (is(T == dchar))
         alias ImplicitConversionTargets =
             AliasSeq!(int, uint, long, ulong, CentTypeList, 
float, double, real);
     // 41 lines including white-space and comments (only that 
there is no white-space or comments)
---

I leave it up to you to decide which version is more 
understandable and extendable (should we ever get another basic 
type :))

As noted previously please do discuss!
Maybe you like the template version more?
Let me know.
Oct 05 2020
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 06/10/2020 12:44 AM, Stefan Koch wrote:
 I leave it up to you to decide which version is more understandable and 
 extendable (should we ever get another basic type :))
You forgot ucent and cent ;)
Oct 05 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 October 2020 at 11:52:06 UTC, rikki cattermole wrote:
 On 06/10/2020 12:44 AM, Stefan Koch wrote:
 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another 
 basic type :))
You forgot ucent and cent ;)
You can't write those without making dmd error out. And complain about ucent and cent not being implemented. At least that used to be the case. I also didn't write the floating types since, there's still a bug in the type function implementation that prevents those being turned into type expressions.
Oct 05 2020
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 October 2020 at 11:58:55 UTC, Stefan Koch wrote:
 On Monday, 5 October 2020 at 11:52:06 UTC, rikki cattermole 
 wrote:
 On 06/10/2020 12:44 AM, Stefan Koch wrote:
 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another 
 basic type :))
You forgot ucent and cent ;)
You can't write those without making dmd error out. And complain about ucent and cent not being implemented. At least that used to be the case. I also didn't write the floating types since, there's still a bug in the type function implementation that prevents those being turned into type expressions.
Yep ... it's still a problem. test_alias_implconv.d(9): Error: cent and ucent types not implemented test_alias_implconv.d(13): called from here: isBasicType(T) test_alias_implconv.d(47): called from here: convTargets(cast(alias)(long)) test_alias_implconv.d(47): called from here: isEqual(convTargets(cast(alias)(long)), makeAliasArray([cast(alias)(ulong), cast(alias)(long)][])) test_alias_implconv.d(47): while evaluating: static assert(isEqual(convTargets(cast(alias)(long)), makeAliasArray([cast(alias)(ulong), cast(alias)(long)][]))) This shows another advantage of type functions. You get a call stack ;)
Oct 05 2020
prev sibling next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:
 Hi,

 [snip]

 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another basic 
 type :))

 As noted previously please do discuss!
 Maybe you like the template version more?
 Let me know.
It would be useful if these examples also include some kind of performance information as well. Further, as far as I could tell from the previous thread, there were three potential approaches: type functions, templates, and reification.
Oct 05 2020
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 October 2020 at 12:46:29 UTC, jmh530 wrote:
 On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:
 Hi,

 [snip]

 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another 
 basic type :))

 As noted previously please do discuss!
 Maybe you like the template version more?
 Let me know.
It would be useful if these examples also include some kind of performance information as well. Further, as far as I could tell from the previous thread, there were three potential approaches: type functions, templates, and reification.
Well type function _are_ transparent type reification. With nice syntax, that mirrors the one that's used in the rest of the language to query types. As for the approach of explicitly reifying types using templates perhaps Andrei could present the example. (A complete example please, that compiles standalone without importing a library.)
Oct 05 2020
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/5/20 7:44 AM, Stefan Koch wrote:
 Hi,
 
 I've posted an incomplete version of a semantic translation of 
 ImplictConvTargets from a template into a type function.
 
 After a few rather trivial fixes let me show you what the code looks 
 like now.
[snip] The existing implementation is ancient (e.g. predates std.meta) and certainly deserves a redoing with Filter, which turns it into a 3-liner (untested): alias Integrals = AliasSeq!(byte, ubyte, short, ushort, int, uint, long, ulong, CentTypeList, float, double, real, char, wchar, dchar); alias convertsTo(U) = is(T : U); alias ImplicitConversionTargets(T) = Filter!(convertsTo, Integrals); This would make for a nice refactoring PR and would provide a better baseline for comparison. Things could be further simplified - I haven't looked in detail but it seems to me each integral converts to all integrals that are as large or larger. So if we keep a list of integrals sorted by size we could simply return slices from it instead of doing filtering. Anyway, implementation size or difficulty is not the problem with ImplicitConversionTargets - it's completeness and correctness. Also the appropriate definition in the language, which now is scattered all over and probably less precise than it should. ImplicitConversionTargets currently doesn't handle alias this, functions and delegates wich co/contravariance, probably a few array conversions (I recall Per recently added or is about to add one), and some qualified structs.
Oct 05 2020
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 October 2020 at 12:50:39 UTC, Andrei Alexandrescu 
wrote:
 On 10/5/20 7:44 AM, Stefan Koch wrote:
 Hi,
 
 I've posted an incomplete version of a semantic translation of 
 ImplictConvTargets from a template into a type function.
 
 After a few rather trivial fixes let me show you what the code 
 looks like now.
[snip] The existing implementation is ancient (e.g. predates std.meta) and certainly deserves a redoing with Filter, which turns it into a 3-liner (untested): alias Integrals = AliasSeq!(byte, ubyte, short, ushort, int, uint, long, ulong, CentTypeList, float, double, real, char, wchar, dchar); alias convertsTo(U) = is(T : U); alias ImplicitConversionTargets(T) = Filter!(convertsTo, Integrals);
Please post the complete version. Which does not import the stdlib. Thanks.
Oct 05 2020
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 October 2020 at 12:50:39 UTC, Andrei Alexandrescu 
wrote:
 Anyway, implementation size or difficulty is not the problem 
 with ImplicitConversionTargets
In the context of this example it is. If you want to discuss it an another context please start a new thread.
Oct 05 2020
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 October 2020 at 12:50:39 UTC, Andrei Alexandrescu 
wrote:

 The existing implementation is ancient (e.g. predates std.meta) 
 and certainly deserves a redoing with Filter, which turns it 
 into a 3-liner (untested):

 alias Integrals = AliasSeq!(byte, ubyte, short, ushort, int, 
 uint, long, ulong, CentTypeList, float, double, real, char, 
 wchar, dchar);
 alias convertsTo(U) = is(T : U);
 alias ImplicitConversionTargets(T) = Filter!(convertsTo, 
 Integrals);
This code does not work. I don't even need to compile it to see that.
Oct 05 2020
parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 5 October 2020 at 20:57:04 UTC, Stefan Koch wrote:
 On Monday, 5 October 2020 at 12:50:39 UTC, Andrei Alexandrescu 
 wrote:

 The existing implementation is ancient (e.g. predates 
 std.meta) and certainly deserves a redoing with Filter, which 
 turns it into a 3-liner (untested):

 alias Integrals = AliasSeq!(byte, ubyte, short, ushort, int, 
 uint, long, ulong, CentTypeList, float, double, real, char, 
 wchar, dchar);
 alias convertsTo(U) = is(T : U);
 alias ImplicitConversionTargets(T) = Filter!(convertsTo, 
 Integrals);
This code does not work. I don't even need to compile it to see that.
It has some simple mistakes, but the fundamental idea is sound. Here's a version that actually compiles: import std.meta; alias Numerics = AliasSeq!(byte, ubyte, short, ushort, int, uint, long, ulong, float, double, real, char, wchar, dchar); enum convertsTo(T, U) = is(T : U); alias ImplicitConversionTargets(T) = Filter!(ApplyLeft!(convertsTo, T), Numerics); // prints: (int, uint, long, ulong, float, double, real, dchar) pragma(msg, ImplicitConversionTargets!int); // prints: (float, double, real) pragma(msg, ImplicitConversionTargets!double);
Oct 05 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 October 2020 at 21:13:09 UTC, Paul Backus wrote:
 On Monday, 5 October 2020 at 20:57:04 UTC, Stefan Koch wrote:
 On Monday, 5 October 2020 at 12:50:39 UTC, Andrei Alexandrescu 
 wrote:

 [...]
This code does not work. I don't even need to compile it to see that.
It has some simple mistakes, but the fundamental idea is sound. Here's a version that actually compiles: import std.meta; alias Numerics = AliasSeq!(byte, ubyte, short, ushort, int, uint, long, ulong, float, double, real, char, wchar, dchar); enum convertsTo(T, U) = is(T : U); alias ImplicitConversionTargets(T) = Filter!(ApplyLeft!(convertsTo, T), Numerics); // prints: (int, uint, long, ulong, float, double, real, dchar) pragma(msg, ImplicitConversionTargets!int); // prints: (float, double, real) pragma(msg, ImplicitConversionTargets!double);
Now post it with all transitive dependencies. And we have a fair comparison.
Oct 05 2020
next sibling parent reply foobar <foo bar.com> writes:
On Monday, 5 October 2020 at 21:20:36 UTC, Stefan Koch wrote:
 On Monday, 5 October 2020 at 21:13:09 UTC, Paul Backus wrote:
 On Monday, 5 October 2020 at 20:57:04 UTC, Stefan Koch wrote:
 On Monday, 5 October 2020 at 12:50:39 UTC, Andrei 
 Alexandrescu wrote:

 [...]
This code does not work. I don't even need to compile it to see that.
It has some simple mistakes, but the fundamental idea is sound. Here's a version that actually compiles: import std.meta; alias Numerics = AliasSeq!(byte, ubyte, short, ushort, int, uint, long, ulong, float, double, real, char, wchar, dchar); enum convertsTo(T, U) = is(T : U); alias ImplicitConversionTargets(T) = Filter!(ApplyLeft!(convertsTo, T), Numerics); // prints: (int, uint, long, ulong, float, double, real, dchar) pragma(msg, ImplicitConversionTargets!int); // prints: (float, double, real) pragma(msg, ImplicitConversionTargets!double);
Now post it with all transitive dependencies. And we have a fair comparison.
Post your code with all changes to the language and compiler. Then we have a fair comparison.
Oct 05 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 6 October 2020 at 00:25:40 UTC, foobar wrote:
 On Monday, 5 October 2020 at 21:20:36 UTC, Stefan Koch wrote:
 On Monday, 5 October 2020 at 21:13:09 UTC, Paul Backus wrote:
 On Monday, 5 October 2020 at 20:57:04 UTC, Stefan Koch wrote:
 On Monday, 5 October 2020 at 12:50:39 UTC, Andrei 
 Alexandrescu wrote:

 [...]
This code does not work. I don't even need to compile it to see that.
It has some simple mistakes, but the fundamental idea is sound. Here's a version that actually compiles: import std.meta; alias Numerics = AliasSeq!(byte, ubyte, short, ushort, int, uint, long, ulong, float, double, real, char, wchar, dchar); enum convertsTo(T, U) = is(T : U); alias ImplicitConversionTargets(T) = Filter!(ApplyLeft!(convertsTo, T), Numerics); // prints: (int, uint, long, ulong, float, double, real, dchar) pragma(msg, ImplicitConversionTargets!int); // prints: (float, double, real) pragma(msg, ImplicitConversionTargets!double);
Now post it with all transitive dependencies. And we have a fair comparison.
Post your code with all changes to the language and compiler. Then we have a fair comparison.
Actually no. The compiler changes don't affect the user. They're for a small number of people to know about and support. Typefunctions and CTFE are together still much less complicated than the template system is. I have nothing to hide though here it is https://github.com/dlang/dmd/compare/master...UplinkCoder:talias_master 650 lines of rather clean code which can in the future be factored with the respective semantic routines.
Oct 05 2020
parent reply foobar <foo bar.com> writes:
On Tuesday, 6 October 2020 at 03:50:11 UTC, Stefan Koch wrote:
 On Tuesday, 6 October 2020 at 00:25:40 UTC, foobar wrote:
 On Monday, 5 October 2020 at 21:20:36 UTC, Stefan Koch wrote:
 On Monday, 5 October 2020 at 21:13:09 UTC, Paul Backus wrote:
 On Monday, 5 October 2020 at 20:57:04 UTC, Stefan Koch wrote:
 On Monday, 5 October 2020 at 12:50:39 UTC, Andrei 
 Alexandrescu wrote:

 [...]
This code does not work. I don't even need to compile it to see that.
It has some simple mistakes, but the fundamental idea is sound. Here's a version that actually compiles: import std.meta; alias Numerics = AliasSeq!(byte, ubyte, short, ushort, int, uint, long, ulong, float, double, real, char, wchar, dchar); enum convertsTo(T, U) = is(T : U); alias ImplicitConversionTargets(T) = Filter!(ApplyLeft!(convertsTo, T), Numerics); // prints: (int, uint, long, ulong, float, double, real, dchar) pragma(msg, ImplicitConversionTargets!int); // prints: (float, double, real) pragma(msg, ImplicitConversionTargets!double);
Now post it with all transitive dependencies. And we have a fair comparison.
Post your code with all changes to the language and compiler. Then we have a fair comparison.
Actually no. The compiler changes don't affect the user. They're for a small number of people to know about and support. Typefunctions and CTFE are together still much less complicated than the template system is. I have nothing to hide though here it is https://github.com/dlang/dmd/compare/master...UplinkCoder:talias_master 650 lines of rather clean code which can in the future be factored with the respective semantic routines.
Actuallly yes. The language changes affect the user. This is a large change to the language which puts back in the compiler what we do in libraries. Whoop-de-do. Do type functions do anything new?
Oct 06 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 6 October 2020 at 11:34:10 UTC, foobar wrote:
 On Tuesday, 6 October 2020 at 03:50:11 UTC, Stefan Koch wrote:
 On Tuesday, 6 October 2020 at 00:25:40 UTC, foobar wrote:
 On Monday, 5 October 2020 at 21:20:36 UTC, Stefan Koch wrote:
 On Monday, 5 October 2020 at 21:13:09 UTC, Paul Backus wrote:
 On Monday, 5 October 2020 at 20:57:04 UTC, Stefan Koch 
 wrote:
 On Monday, 5 October 2020 at 12:50:39 UTC, Andrei 
 Alexandrescu wrote:

 [...]
This code does not work. I don't even need to compile it to see that.
It has some simple mistakes, but the fundamental idea is sound. Here's a version that actually compiles: import std.meta; alias Numerics = AliasSeq!(byte, ubyte, short, ushort, int, uint, long, ulong, float, double, real, char, wchar, dchar); enum convertsTo(T, U) = is(T : U); alias ImplicitConversionTargets(T) = Filter!(ApplyLeft!(convertsTo, T), Numerics); // prints: (int, uint, long, ulong, float, double, real, dchar) pragma(msg, ImplicitConversionTargets!int); // prints: (float, double, real) pragma(msg, ImplicitConversionTargets!double);
Now post it with all transitive dependencies. And we have a fair comparison.
Post your code with all changes to the language and compiler. Then we have a fair comparison.
Actually no. The compiler changes don't affect the user. They're for a small number of people to know about and support. Typefunctions and CTFE are together still much less complicated than the template system is. I have nothing to hide though here it is https://github.com/dlang/dmd/compare/master...UplinkCoder:talias_master 650 lines of rather clean code which can in the future be factored with the respective semantic routines.
Actuallly yes. The language changes affect the user. This is a large change to the language which puts back in the compiler what we do in libraries. Whoop-de-do. Do type functions do anything new?
It only gives access to what the compiler has to do anyway. It gives you an interface to what must exist within. You could even say this makes it easier to provide a library implementation by using the compiler as a built-in library. No, type functions don't do anything original they mirror how type manipulation works within templates providing a familiar and usable interface. It is an explicit goal of mine to have type functions look just like any other D code.
Oct 06 2020
parent reply foobar <foo bar.com> writes:
On Tuesday, 6 October 2020 at 12:35:15 UTC, Stefan Koch wrote:
 On Tuesday, 6 October 2020 at 11:34:10 UTC, foobar wrote:
 On Tuesday, 6 October 2020 at 03:50:11 UTC, Stefan Koch wrote:
 [...]
Actuallly yes. The language changes affect the user. This is a large change to the language which puts back in the compiler what we do in libraries. Whoop-de-do. Do type functions do anything new?
It only gives access to what the compiler has to do anyway. It gives you an interface to what must exist within. You could even say this makes it easier to provide a library implementation by using the compiler as a built-in library. No, type functions don't do anything original they mirror how type manipulation works within templates providing a familiar and usable interface. It is an explicit goal of mine to have type functions look just like any other D code.
Last time I looked D was a small language but very powerful. It innovated and did things you could not do in C++. Now this forum has been taken over by beginners who want a big language which does the same as before, only differently. Yawn.
Oct 06 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/6/20 1:51 PM, foobar wrote:
 On Tuesday, 6 October 2020 at 12:35:15 UTC, Stefan Koch wrote:
 On Tuesday, 6 October 2020 at 11:34:10 UTC, foobar wrote:
 On Tuesday, 6 October 2020 at 03:50:11 UTC, Stefan Koch wrote:
 [...]
Actuallly yes. The language changes affect the user. This is a large change to the language which puts back in the compiler what we do in libraries. Whoop-de-do. Do type functions do anything new?
It only gives access to what the compiler has to do anyway. It gives you an interface to what must exist within. You could even say this makes it easier to provide a library implementation by using the compiler as a built-in library. No, type functions don't do anything original they mirror how type manipulation works within templates providing a familiar and usable interface. It is an explicit goal of mine to have type functions look just like any other D code.
Last time I looked D was a small language but very powerful. It innovated and did things you could not do in C++. Now this forum has been taken over by beginners who want a big language which does the same as before, only differently. Yawn.
Everything you just said is wrong. -Steve
Oct 06 2020
parent foobar <foo bar.com> writes:
On Tuesday, 6 October 2020 at 18:17:51 UTC, Steven Schveighoffer 
wrote:
 On 10/6/20 1:51 PM, foobar wrote:
 On Tuesday, 6 October 2020 at 12:35:15 UTC, Stefan Koch wrote:
 [...]
Last time I looked D was a small language but very powerful. It innovated and did things you could not do in C++. Now this forum has been taken over by beginners who want a big language which does the same as before, only differently. Yawn.
Everything you just said is wrong. -Steve
Said Stefan Koch:
No, type functions don't do anything original they mirror how 
type manipulation works within templates providing a familiar 
and usable interface.
Everything YOU just said is wrong.
Oct 06 2020
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/5/2020 2:20 PM, Stefan Koch wrote:
 Now post it with all transitive dependencies.
I'm not sure how transitive dependencies applies here. A illuminating test case would be helpful.
Oct 05 2020
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 6 October 2020 at 01:12:01 UTC, Walter Bright wrote:
 On 10/5/2020 2:20 PM, Stefan Koch wrote:
 Now post it with all transitive dependencies.
I'm not sure how transitive dependencies applies here. A illuminating test case would be helpful.
Transitive dependencies apply, because it shows how much work has to be done in the library. As opposed to the type-function code which works without library support (except for the druntime array handling of course)
Oct 05 2020
prev sibling next sibling parent reply claptrap <clap trap.com> writes:
On Tuesday, 6 October 2020 at 01:12:01 UTC, Walter Bright wrote:
 On 10/5/2020 2:20 PM, Stefan Koch wrote:
 Now post it with all transitive dependencies.
I'm not sure how transitive dependencies applies here. A illuminating test case would be helpful.
Because Stefan didnt rely on external code, he showed *all* the code needed, so a counter example should have been the same. Since the point is to compare the two *language* features, not compare library calls. I mean it's like someone posted a new sort algorithm and Andrei replied with.... import std.algorithm; result = sort!(foo);
Oct 06 2020
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 6 October 2020 at 08:35:20 UTC, claptrap wrote:
 Because Stefan didnt rely on external code, he showed *all* the 
 code needed, so a counter example should have been the same. 
 Since the point is to compare the two *language* features, not 
 compare library calls.
Language features are a means to an end. If the Phobos version compiles faster, uses less memory, and has the same result in the binary, it is a victory, even if it as significantly more code. I'm skeptical of the type functions, but interested. They might help with a real problem. But they need to actually win on the end results, not against artificial strawmen.
Oct 06 2020
next sibling parent reply Bruce Carneal <bcarneal gmail.com> writes:
On Tuesday, 6 October 2020 at 12:28:55 UTC, Adam D. Ruppe wrote:
 On Tuesday, 6 October 2020 at 08:35:20 UTC, claptrap wrote:
 Because Stefan didnt rely on external code, he showed *all* 
 the code needed, so a counter example should have been the 
 same. Since the point is to compare the two *language* 
 features, not compare library calls.
Language features are a means to an end. If the Phobos version compiles faster, uses less memory, and has the same result in the binary, it is a victory, even if it as significantly more code.
I disagree. I believe we should aim for the simplest code that admits the desired performance. Simplicity is especially important at lower levels where the benefits compound. We're working to factor out complexity for today's programmers and tomorrows.
 I'm skeptical of the type functions, but interested. They might 
 help with a real problem.
Well, what is a "real problem"? If you mean "something that can not be done outside the compiler" then nothing is a real problem (note CTFE and mixins). If you mean "something that can not be done near optimally outside the compiler" then we're in violent agreement.
 But they need to actually win on the end results, not against 
 artificial strawmen.
Agree. Note that performance is relatively easy to test whereas it's harder to compare aggregate debugability over time. My proxies include readability, teachability, independence (fewer layers, fewer dependencies), and the amount of the correctness "proof" (debugging) handled automatically. If performance is similar we should opt for better aggregate debugability as we, the community, understand it. My current understanding is that type functions are significantly better in this regard. Finally, please note that while I disagree with you on some particulars here I am a big fan. I enjoyed your book, and admire the tremendous amount of cheerful quality help that you offer in the forums and discord.
Oct 06 2020
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 6 October 2020 at 15:24:31 UTC, Bruce Carneal wrote:
 I believe we should aim for the simplest code that admits the 
 desired performance.
Sure, but the desired performance here, for this stuff, is the clear priority. Especially at lower levels where the costs compound. Here's some princely advice: it is better to be both feared and loved, but if you can be only one or the other, it is better to be feared than to be loved. That's the reason why Phobos does it the way it does. It'd love to have both, but if it is one or the other, fast compiles are far more important than pretty code.
 Well, what is a "real problem"?
That code compiles slowly and/or with excessive memory. That's why type functions are being investigated. There's no new functionality gained by the type functions. Their whole reason for existing is to make faster, less memory hungry builds. (right now anyway, that might change if it gains capabilities, but right now an explicit design goal is to make them a limited subset of template functionality in the name of improved performance). I frequently take Stefan's examples, copy/paste them into a template, and use them. The code doesn't even look that different. The problem is there's several usages where this version is slow. type function version: alias type = alias; type[] basicTypeConvTargets(type T) { type[] targets; targets.length = basic_types.length; size_t n = 0; foreach(t;basic_types) { if (is(T : t)) { targets[n++] = t; } } return targets[0 .. n]; } template version: alias type = string; type[] basicTypeConvTargets(T)() { type[] targets; targets.length = basic_types.length; size_t n = 0; foreach(t;basic_types) { if (is(T : mixin(t))) { targets[n++] = t; } } return targets[0 .. n]; } Very little difference! The makeConvMatix was *identical* except for the function prototype (and even there, both return strings!). Filter's guts can be: size_t[Args.length] keep; size_t pos = 0; foreach(idx, alias arg; Args) if(Pred!arg) keep[pos++] = idx; // note idx, not arg. return makeResult(keep[0 .. pos]); In today's D. Again, *almost identical* to the typefunction version, just using an index into the list instead of storing the alias in the array directly. What kills this approach is *not* the code being hideous. It is the performance aspect - additional CTFE code is necessary to convert it back to a type tuple, and secondarily, the foreach being unrolled leads to extra work. A type function can simply do `returned.tupleof` and keep the foreach how it is. Here, we'd have to `mixin(returned.doConversion)`. (Where doConversion is a similarly reusable lib function.) Again, minor syntax difference, but significant performance hit because doConversion must rebuild a new string out of stuff the compiler already knows. Phobos' current implementation is uglier, but also faster and uses less memory than the mixin version. So it wins over it. This is the place the typefunction has a potential win. It might combine the nice code AND get a performance improvement. But if the TF ends up slower than Phobos has now... it is going to be rewritten into the faster version, even if uglier, because it is the compile performance driving this evolution. The Phobos implementation started life with a very simple implementation too. It became what it is because it *had to*, specifically for performance reasons.
Oct 06 2020
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/6/20 2:09 PM, Adam D. Ruppe wrote:
 Filter's guts can be:
 
          size_t[Args.length] keep;
          size_t pos = 0;
          foreach(idx, alias arg; Args)
                  if(Pred!arg) keep[pos++] = idx; // note idx,
not arg.
          return makeResult(keep[0 .. pos]);
 
 In today's D. Again, *almost identical* to the typefunction version, 
 just using an index into the list instead of storing the alias in the 
 array directly.
I think Filter is a much more apt example than the implicit targets. Not just because you can make it faster, but because with type functions, you should be able to actually *use filter*: return Args.filter!Pred.array; I mean, we have filter for a reason. You can easily write filter out into a loop, instead of using filter. But there it is, and it makes coding so much more pleasant. If the compiler and CTFE can be fast enough to make this pleasant (I'm not 100% convinced, but it looks promising), then I'm on board. It comes down to one thing -- arrays vs. tuples. In type functions, a tuple is an array, and you can do all the things you can do with a normal array: mutate, sort, shrink, grow, loop, use as a range, etc. With a Tuple, everything is immutable, and each change needs to go across a new template boundary. Even a loop is not really a loop. That being said, I think the only way type functions make a difference is if they perform WELL. Even if they perform *as well* as templates (though indications are they perform better), I'd rather write code in an imperative style than recursive. -Steve
Oct 06 2020
next sibling parent reply Meta <jared771 gmail.com> writes:
On Tuesday, 6 October 2020 at 18:30:15 UTC, Steven Schveighoffer 
wrote:
 On 10/6/20 2:09 PM, Adam D. Ruppe wrote:
 Filter's guts can be:
 
          size_t[Args.length] keep;
          size_t pos = 0;
          foreach(idx, alias arg; Args)
                  if(Pred!arg) keep[pos++] = idx; // note idx, 
 not arg.
          return makeResult(keep[0 .. pos]);
 
 In today's D. Again, *almost identical* to the typefunction 
 version, just using an index into the list instead of storing 
 the alias in the array directly.
I think Filter is a much more apt example than the implicit targets. Not just because you can make it faster, but because with type functions, you should be able to actually *use filter*: return Args.filter!Pred.array; I mean, we have filter for a reason. You can easily write filter out into a loop, instead of using filter. But there it is, and it makes coding so much more pleasant. If the compiler and CTFE can be fast enough to make this pleasant (I'm not 100% convinced, but it looks promising), then I'm on board. With a Tuple, everything is immutable, and each change needs to go across a new template boundary. Even a loop is not really a loop.
Excellent point; the only reason Filter, staticSort, staticMap, Reverse, Repeat, etc., etc. exist is because of the limitations of working with AliasSeq. If type functions can allow us to replace most of these uses with good old std.range/algorithm code, that's a huge win in my book. We have a big chance to go left when C++ went right and went all-in on template metaprogramming. Let's leave templates to do what they were designed for - genericizing structures and algorithms, and leave the compile-time computation to CTFE + type functions.
Oct 06 2020
parent foobar <foo bar.com> writes:
On Tuesday, 6 October 2020 at 18:49:08 UTC, Meta wrote:
 On Tuesday, 6 October 2020 at 18:30:15 UTC, Steven 
 Schveighoffer wrote:
 [...]
Excellent point; the only reason Filter, staticSort, staticMap, Reverse, Repeat, etc., etc. exist is because of the limitations of working with AliasSeq. If type functions can allow us to replace most of these uses with good old std.range/algorithm code, that's a huge win in my book. We have a big chance to go left when C++ went right and went all-in on template metaprogramming. Let's leave templates to do what they were designed for - genericizing structures and algorithms, and leave the compile-time computation to CTFE + type functions.
C++ has constexpr... We have a big chance to become a language even bigger than C++ without doing anything more. Hooray.
Oct 06 2020
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 6 October 2020 at 18:30:15 UTC, Steven Schveighoffer 
wrote:
 It comes down to one thing -- arrays vs. tuples. In type 
 functions, a tuple is an array, and you can do all the things 
 you can do with a normal array: mutate, sort, shrink, grow, 
 loop, use as a range, etc.

 With a Tuple, everything is immutable, and each change needs to 
 go across a new template boundary. Even a loop is not really a 
 loop.
It seems to me like maybe the most obvious way from point A to point B is to lift these limitations on tuples (and aliases, and manifest constants). Then we could write, for example: template Filter(alias pred, Args...) { enum pos = 0; foreach (Arg; Args) { static if (Pred!Arg) { Args[pos] := Arg; pos := pos + 1; } } alias Filter = Args[0 .. pos]; } You would still pay for the performance overhead of tuple foreach and static if, so maybe it's not that big a win, but it's still an improvement over recursive templates and CTFE+mixins.
Oct 06 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 6 October 2020 at 23:36:00 UTC, Paul Backus wrote:
 On Tuesday, 6 October 2020 at 18:30:15 UTC, Steven 
 Schveighoffer wrote:
 It comes down to one thing -- arrays vs. tuples. In type 
 functions, a tuple is an array, and you can do all the things 
 you can do with a normal array: mutate, sort, shrink, grow, 
 loop, use as a range, etc.

 With a Tuple, everything is immutable, and each change needs 
 to go across a new template boundary. Even a loop is not 
 really a loop.
It seems to me like maybe the most obvious way from point A to point B is to lift these limitations on tuples (and aliases, and manifest constants). Then we could write, for example: template Filter(alias pred, Args...) { enum pos = 0; foreach (Arg; Args) { static if (Pred!Arg) { Args[pos] := Arg; pos := pos + 1; } } alias Filter = Args[0 .. pos]; } You would still pay for the performance overhead of tuple foreach and static if, so maybe it's not that big a win, but it's still an improvement over recursive templates and CTFE+mixins.
I think lifting limitations on tuples can't be done in general without violation of current language rules. Type functions are actually just a shell around operations that would be illegal in the language as is. But because the type function provides a boundary I can do these things without invalidating the language semantics. Type functions are something which can be proven to not have influence on language semantics outside of their own function bodies.
Oct 06 2020
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 6 October 2020 at 23:44:30 UTC, Stefan Koch wrote:
 Type functions are actually just a shell around operations that 
 would be illegal in the language as is.
 But because the type function provides a boundary I can do 
 these things without invalidating the language semantics.

 Type functions are something which can be proven to not have 
 influence on language semantics outside of their own function 
 bodies.
Ugh, why not do it properly and just add type variables? It is just a pointer... At runtime it could point to the type's RTTI node which contains a pointer to the constructor.
Oct 06 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 7 October 2020 at 00:03:46 UTC, Ola Fosheim Grøstad 
wrote:
 On Tuesday, 6 October 2020 at 23:44:30 UTC, Stefan Koch wrote:
 Type functions are actually just a shell around operations 
 that would be illegal in the language as is.
 But because the type function provides a boundary I can do 
 these things without invalidating the language semantics.

 Type functions are something which can be proven to not have 
 influence on language semantics outside of their own function 
 bodies.
Ugh, why not do it properly and just add type variables? It is just a pointer... At runtime it could point to the type's RTTI node which contains a pointer to the constructor.
If you know a way to do that cleanly, that does not involve a redesign of the compiler, I am very interested to hear about it. As things stand type functions are what I can get away with, and still be reasonably confident I won't violate language invariants. Also their syntax blends in fairly nicely with the rest of D.
Oct 06 2020
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 7 October 2020 at 00:11:48 UTC, Stefan Koch wrote:
 If you know a way to do that cleanly, that does not involve a 
 redesign of the compiler,
 I am very interested to hear about it.
 As things stand type functions are what I can get away with, 
 and still be reasonably confident I won't violate language 
 invariants.
 Also their syntax blends in fairly nicely with the rest of D.
Allright, if the syntax is such that it can be made more general later then all is good. (Ive only modified the dmd lexer/parser, so I don't know where the limitations are bqeyond ast level...)
Oct 06 2020
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 7 October 2020 at 00:18:10 UTC, Ola Fosheim Grøstad 
wrote:
 On Wednesday, 7 October 2020 at 00:11:48 UTC, Stefan Koch wrote:
 If you know a way to do that cleanly, that does not involve a 
 redesign of the compiler,
 I am very interested to hear about it.
 As things stand type functions are what I can get away with, 
 and still be reasonably confident I won't violate language 
 invariants.
 Also their syntax blends in fairly nicely with the rest of D.
Allright, if the syntax is such that it can be made more general later then all is good. (Ive only modified the dmd lexer/parser, so I don't know where the limitations are bqeyond ast level...)
With more general you mean, not just apply to types but to anything? Yes that can happen. Adam actually wants this functionality as well. Tbh. the syntax is currently blocking me here. What name do I give to the (pseudo) TOP type? that can hold anything a variadic template parameter can hold?
Oct 06 2020
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 6 October 2020 at 23:44:30 UTC, Stefan Koch wrote:
 On Tuesday, 6 October 2020 at 23:36:00 UTC, Paul Backus wrote:
 It seems to me like maybe the most obvious way from point A to 
 point B is to lift these limitations on tuples (and aliases, 
 and manifest constants). Then we could write, for example:

 template Filter(alias pred, Args...)
 {
     enum pos = 0;
     foreach (Arg; Args) {
         static if (Pred!Arg) {
             Args[pos] := Arg;
             pos := pos + 1;
         }
     }
     alias Filter = Args[0 .. pos];
 }

 You would still pay for the performance overhead of tuple 
 foreach and static if, so maybe it's not that big a win, but 
 it's still an improvement over recursive templates and 
 CTFE+mixins.
I think lifting limitations on tuples can't be done in general without violation of current language rules. Type functions are actually just a shell around operations that would be illegal in the language as is. But because the type function provides a boundary I can do these things without invalidating the language semantics. Type functions are something which can be proven to not have influence on language semantics outside of their own function bodies.
I agree that allowing completely unrestricted tuple mutation would be problematic, but I think it's still worth asking how far we could potentially go in that direction. For example, maybe you're only allowed to mutate tuples inside the scope that declares them. That would let you implement templates like Filter and staticMap iteratively, while still presenting an immutable "interface" to the rest of the program.
Oct 06 2020
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 7 October 2020 at 00:11:20 UTC, Paul Backus wrote:
 
 I agree that allowing completely unrestricted tuple mutation 
 would be problematic, but I think it's still worth asking how 
 far we could potentially go in that direction.
I would be happy to discuss this over a coffee or beer sometime. As a thought experiment it's exciting.
 For example, maybe you're only allowed to mutate tuples inside 
 the scope that declares them. That would let you implement 
 templates like Filter and staticMap iteratively, while still 
 presenting an immutable "interface" to the rest of the program.
If you can prove: - It never escapes a 'mutable' version into a monophonic scope. - Inside a polymorphic scope (template) all transforms are derived from the shape defining parameters (template parameters) or are invariant. That is necessary, but I am not sure it's sufficient.
Oct 06 2020
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/6/2020 11:09 AM, Adam D. Ruppe wrote:
 The Phobos implementation started life with a very simple implementation too.
It 
 became what it is because it *had to*, specifically for performance reasons.
Professional C Standard library implementations tend to be hideous code to perform objectively simple operations. The reason is speed is so desirable in foundational code that it drives out all other considerations. (memcpy() is a standout example.) I remember back in the 80's when Borland came out with Turbo C. The compiler didn't generate very good code, but applications built with it were reasonably fast. How was this done? Borland carefully implemented the C library in hand-optimized assembler by some very good programmers. Even printf was coded this way. The speedups gained there sped up every Turbo C program. At the time, nobody else had done that. Much as I hate to admit it, Borland made the smart move there.
Oct 06 2020
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Oct 06, 2020 at 08:47:33PM -0700, Walter Bright via Digitalmars-d wrote:
 On 10/6/2020 11:09 AM, Adam D. Ruppe wrote:
 The Phobos implementation started life with a very simple
 implementation too. It became what it is because it *had to*,
 specifically for performance reasons.
Professional C Standard library implementations tend to be hideous code to perform objectively simple operations. The reason is speed is so desirable in foundational code that it drives out all other considerations. (memcpy() is a standout example.)
A little tangential aside: one time as a little coffee break challenge, I decided to see how D compares to GNU wc in terms of the speed of counting lines in a text file. The core of wc, of course, is in memchr -- because it's scanning for newline characters in a buffer. In the course of ferreting out what made wc so fast, I studied how GNU libc implemented memchr. Basically, in order to speed up scanning large buffers, it uses a series of fancy bit operations on 64-bit words in order to scan 8 bytes at a time, I suppose the goal being to achieve close to 8x speedup, and also to reduce the number of branches per iteration to capitalize on the CPU's pipeline. In order to do this, however, some not-so-cheap setup was necessary at the beginning and end of the buffer, which generally are not 8-byte aligned. When a particular call to memchr scanned many bytes, of course, the speed of scanning 8 bytes at a time outweighed this setup / teardown cost, and wc generally outperformed my D code. However, when the lines being scanned were on the short side, the overhead of setting up the 8-byte-at-a-time scanning added up significantly -- the shorter lines meant less time was spend scanning 8 bytes at a time, and more time spent in the complicated code dealing with non-aligned start/end of the buffer. In this case, I discovered that a simplistic for-loop in D outperformed wc. This led me to think, realistically speaking, how long do lines in a text file tend to be? My wild guess is that they tend to be on the shortish side -- maybe about 60-70 characters on average? For code, a lot less, since code generally has more whitespace for readability reasons. Files that have very long lines tend to be things like XML or compressed Javascript, which generally you don't really use wc on anyway, so in my mind, they seem to be more the exception than the norm when it comes to the applicability of wc. Furthermore, I venture to propose that your average string length in C is probably closer to the 80-120 character ballpark, than to large buffers of 1K or 8K which is where the 8-byte-at-a-time scanning would perform much better. Sure, large text buffers do get handled in C code routinely; but how many of them realistically would you find being handed to strchr to scan for some given byte? The kind of C strings you'd want to search for a particular byte in, IME, are usually the shorter kind. What this means, is that yes memchr may be "optimized" to next week and back -- but that optimization came with some implicit assumptions about how long it takes to find a character in a string buffer. These assumptions unfortunately pessimized the common case with the overhead of a fancy 8-byte-at-a-time algorithm: one that does better in the *less* common case of looking for a particular byte in a very large buffer. My point behind all this, is that what's considered optimal sometimes changes depending on what kind of use case you anticipate; and with different usage patterns, one man's optimal algorithm may be another man's suboptimal algorithm, and one man's slow, humble for-loop may be another man's speed demon. Optimizing for the general case in a standard library is generally a very tough problem, because how do you make any assumptions about the general case? Whichever way you choose to optimize a primitive like memchr, you're imposing additional assumptions that may make it worse for some people, even if you think that you're improving it for your chosen target metric.
 I remember back in the 80's when Borland came out with Turbo C. The
 compiler didn't generate very good code, but applications built with
 it were reasonably fast. How was this done?
 
 Borland carefully implemented the C library in hand-optimized
 assembler by some very good programmers. Even printf was coded this
 way. The speedups gained there sped up every Turbo C program. At the
 time, nobody else had done that.
 
 Much as I hate to admit it, Borland made the smart move there.
So what does this imply in terms of Phobos code in D? ;-) Should we uglify Phobos for the sake of performance? Keeping in mind, of course, what I said about the difficulty of optimizing for the general case without pessimizing some legitimate use cases -- or maybe even the *common* case, if we lose sight of the forest of common use cases while trying to optimize our chosen benchmark trees. T -- Life begins when you can spend your spare time programming instead of watching television. -- Cal Keegan
Oct 06 2020
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/6/2020 9:17 PM, H. S. Teoh wrote:
 So what does this imply in terms of Phobos code in D? ;-)  Should we
 uglify Phobos for the sake of performance?
I'd phrase it more as do what we gotta do for performance, yes.
 Keeping in mind, of course,
 what I said about the difficulty of optimizing for the general case
 without pessimizing some legitimate use cases -- or maybe even the
 *common* case, if we lose sight of the forest of common use cases while
 trying to optimize our chosen benchmark trees.
Of course, we have to use our brains instead of doing things blindly :-)
Oct 06 2020
prev sibling next sibling parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Wednesday, 7 October 2020 at 04:17:59 UTC, H. S. Teoh wrote:
 On Tue, Oct 06, 2020 at 08:47:33PM -0700, Walter Bright via 
 Digitalmars-d wrote:
 On 10/6/2020 11:09 AM, Adam D. Ruppe wrote:
 The Phobos implementation started life with a very simple 
 implementation too. It became what it is because it *had 
 to*, specifically for performance reasons.
Professional C Standard library implementations tend to be hideous code to perform objectively simple operations. The reason is speed is so desirable in foundational code that it drives out all other considerations. (memcpy() is a standout example.)
A little tangential aside: one time as a little coffee break challenge, I decided to see how D compares to GNU wc in terms of the speed of counting lines in a text file. The core of wc, of course, is in memchr -- because it's scanning for newline characters in a buffer. In the course of ferreting out what made wc so fast, I studied how GNU libc implemented memchr. Basically, in order to speed up scanning large buffers, it uses a series of fancy bit operations on 64-bit words in order to scan 8 bytes at a time, I suppose the goal being to achieve close to 8x speedup, and also to reduce the number of branches per iteration to capitalize on the CPU's pipeline. In order to do this, however, some not-so-cheap setup was necessary at the beginning and end of the buffer, which generally are not 8-byte aligned. When a particular call to memchr scanned many bytes, of course, the speed of scanning 8 bytes at a time outweighed this setup / teardown cost, and wc generally outperformed my D code. However, when the lines being scanned were on the short side, the overhead of setting up the 8-byte-at-a-time scanning added up significantly -- the shorter lines meant less time was spend scanning 8 bytes at a time, and more time spent in the complicated code dealing with non-aligned start/end of the buffer. In this case, I discovered that a simplistic for-loop in D outperformed wc. This led me to think, realistically speaking, how long do lines in a text file tend to be? My wild guess is that they tend to be on the shortish side -- maybe about 60-70 characters on average? For code, a lot less, since code generally has more whitespace for readability reasons. Files that have very long lines tend to be things like XML or compressed Javascript, which generally you don't really use wc on anyway, so in my mind, they seem to be more the exception than the norm when it comes to the applicability of wc. Furthermore, I venture to propose that your average string length in C is probably closer to the 80-120 character ballpark, than to large buffers of 1K or 8K which is where the 8-byte-at-a-time scanning would perform much better. Sure, large text buffers do get handled in C code routinely; but how many of them realistically would you find being handed to strchr to scan for some given byte? The kind of C strings you'd want to search for a particular byte in, IME, are usually the shorter kind. What this means, is that yes memchr may be "optimized" to next week and back -- but that optimization came with some implicit assumptions about how long it takes to find a character in a string buffer. These assumptions unfortunately pessimized the common case with the overhead of a fancy 8-byte-at-a-time algorithm: one that does better in the *less* common case of looking for a particular byte in a very large buffer. My point behind all this, is that what's considered optimal sometimes changes depending on what kind of use case you anticipate; and with different usage patterns, one man's optimal algorithm may be another man's suboptimal algorithm, and one man's slow, humble for-loop may be another man's speed demon. Optimizing for the general case in a standard library is generally a very tough problem, because how do you make any assumptions about the general case? Whichever way you choose to optimize a primitive like memchr, you're imposing additional assumptions that may make it worse for some people, even if you think that you're improving it for your chosen target metric.
There's a counter argument to your example (which doesn't invalidate yout point). The pessimisation for short lines is barely noticeable and is drowned in the noise of process launching and even if you can measure it consistantly and easily, it doesn't matter much even if happening a lot of times (in a batch job f.ex). The optimization for long line though, will be noticeable immediately because of the inherent processing time of the operation. If an operation of 5ms takes now 10ms, it doesn't make a difference. If an operation that took 1 minute now takes 30s it is a big deal.
 I remember back in the 80's when Borland came out with Turbo 
 C. The compiler didn't generate very good code, but 
 applications built with it were reasonably fast. How was this 
 done?
 
 Borland carefully implemented the C library in hand-optimized 
 assembler by some very good programmers. Even printf was coded 
 this way. The speedups gained there sped up every Turbo C 
 program. At the time, nobody else had done that.
 
 Much as I hate to admit it, Borland made the smart move there.
So what does this imply in terms of Phobos code in D? ;-) Should we uglify Phobos for the sake of performance? Keeping in mind, of course, what I said about the difficulty of optimizing for the general case without pessimizing some legitimate use cases -- or maybe even the *common* case, if we lose sight of the forest of common use cases while trying to optimize our chosen benchmark trees.
My point. Some optimlizations are more about mitigating speed degradation on pathological cases than on the normal general case, or else the whole O() notation would be of no importance (I'm sure the symbol name compression scheme in the D compiler pessimized the common case but saved the language from the pathological cases of recursive symbols).
Oct 07 2020
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/7/20 12:17 AM, H. S. Teoh wrote:
 On Tue, Oct 06, 2020 at 08:47:33PM -0700, Walter Bright via Digitalmars-d
wrote:
 On 10/6/2020 11:09 AM, Adam D. Ruppe wrote:
 The Phobos implementation started life with a very simple
 implementation too. It became what it is because it *had to*,
 specifically for performance reasons.
Professional C Standard library implementations tend to be hideous code to perform objectively simple operations. The reason is speed is so desirable in foundational code that it drives out all other considerations. (memcpy() is a standout example.)
A little tangential aside: one time as a little coffee break challenge, I decided to see how D compares to GNU wc in terms of the speed of counting lines in a text file. The core of wc, of course, is in memchr -- because it's scanning for newline characters in a buffer. In the course of ferreting out what made wc so fast, I studied how GNU libc implemented memchr. Basically, in order to speed up scanning large buffers, it uses a series of fancy bit operations on 64-bit words in order to scan 8 bytes at a time, I suppose the goal being to achieve close to 8x speedup, and also to reduce the number of branches per iteration to capitalize on the CPU's pipeline.
memchr is also the secret sauce to how iopipe was able to beat Phobos byLine and getline. I would like to go back to it at some point and see if it can be improved. I know that part of the difference would be due to the opaque function call (this cannot be inlined), though perhaps memchr is an intrinsic? -Steve
Oct 07 2020
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Oct 07, 2020 at 10:10:56AM -0400, Steven Schveighoffer via
Digitalmars-d wrote:
[...]
 memchr is also the secret sauce to how iopipe was able to beat Phobos
 byLine and getline.
 
 I would like to go back to it at some point and see if it can be
 improved.
 
 I know that part of the difference would be due to the opaque function
 call (this cannot be inlined), though perhaps memchr is an intrinsic?
[...] I wouldn't be surprised if optimizing compilers like ldc treated it as an intrinsic. IIRC, ldc already recognizes memcpy as an intrinsic and therefore knows how to inline it / substitute it with suitable instructions in special cases, even though it's supposed to be "opaque". T -- Жил-был король когда-то, при нём блоха жила.
Oct 07 2020
parent Jon Degenhardt <jond noreply.com> writes:
On Wednesday, 7 October 2020 at 14:41:18 UTC, H. S. Teoh wrote:
 On Wed, Oct 07, 2020 at 10:10:56AM -0400, Steven Schveighoffer 
 via Digitalmars-d wrote: [...]
 memchr is also the secret sauce to how iopipe was able to beat 
 Phobos byLine and getline.
 
 I would like to go back to it at some point and see if it can 
 be improved.
 
 I know that part of the difference would be due to the opaque 
 function call (this cannot be inlined), though perhaps memchr 
 is an intrinsic?
[...] I wouldn't be surprised if optimizing compilers like ldc treated it as an intrinsic. IIRC, ldc already recognizes memcpy as an intrinsic and therefore knows how to inline it / substitute it with suitable instructions in special cases, even though it's supposed to be "opaque".
It's my understanding that LTO could be used to cross the language boundary as well, allowing otherwise opaque function calls to be inlined. It would require having the function compiled to IR code available at build time. Related to earlier in this subthread - I regularly work with data file 20 bytes per line and others 5000 bytes per line. It's been my experience that the two can have quite different performance characteristics for the same operation. As pointed out, it is sometimes hard to optimize for both. --Jon
Oct 07 2020
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 7 October 2020 at 14:10:56 UTC, Steven 
Schveighoffer wrote:
 I know that part of the difference would be due to the opaque 
 function call (this cannot be inlined), though perhaps memchr 
 is an intrinsic?
You can test individual bytes in a simd register in a single instruction. Will clearly destroy any other implementation...
Oct 07 2020
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 7 October 2020 at 14:43:41 UTC, Ola Fosheim Grøstad 
wrote:
 You can test individual bytes in a simd register in a single 
 instruction. Will clearly destroy any other implementation...
From 2016: https://gms.tf/stdfind-and-memchr-optimizations.html
Oct 07 2020
prev sibling parent reply claptrap <clap trap.com> writes:
On Tuesday, 6 October 2020 at 12:28:55 UTC, Adam D. Ruppe wrote:
 On Tuesday, 6 October 2020 at 08:35:20 UTC, claptrap wrote:
 Because Stefan didnt rely on external code, he showed *all* 
 the code needed, so a counter example should have been the 
 same. Since the point is to compare the two *language* 
 features, not compare library calls.
Language features are a means to an end. If the Phobos version compiles faster, uses less memory, and has the same result in the binary, it is a victory, even if it as significantly more code. I'm skeptical of the type functions, but interested. They might help with a real problem. But they need to actually win on the end results, not against artificial strawmen.
Im less interested in performance than I am in being able to express what I want to do in clear concise code. I find template/mixin shenanigans a bit like writing that doesnt have any vowels, ys y cn stll ndrstnd t, but it takes a lot more work and feels fundamentally unsatisfactory.
Oct 06 2020
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 6 October 2020 at 20:57:10 UTC, claptrap wrote:
 Im less interested in performance than I am in being able to 
 express what I want to do in clear concise code.
Give me an example of what you'd like to use type functions for. I betcha I can adapt it to current D with very few changes. Take a look at this for example: --- import std.algorithm; template largestType(T...) { auto get() { return reified!"a.sizeof".map!T .sort!((a, b) => a > b).front; // or maxElement of course } alias largestType = T[get.idx]; } pragma(msg, largestType!(long, int, real, byte)); // real --- Or what about this, without even seeing the template keyword? import std.algorithm; pragma(msg, reified!"is(a : long)" // compile time lambda we need .run!(types => types.filter!(a => a)) // run the code over the fetched info .over!(string, int, float, Object)); // the list of types ); Note it used std.algorithm's filter over the mapped compile time lambda! But, of course, you'll see it is a string. That is a limitation here, pity we don't have some kind of short syntax template lambda. This is the reifiy and dereify implementations: --- template reified(string mapCode, alias runCode = void) { template evaluate(alias a) { enum evaluate = mixin(mapCode); } template run(alias code) { alias run = reified!(mapCode, code); } template over(T...) { auto helper() { static struct Result { size_t idx; typeof(evaluate!(T[0])) value; alias value this; } Result[] result; foreach(idx, t; T) result ~= Result(idx, evaluate!t); return result; } static if(is(runCode == void)) enum over = helper; else mixin("alias over = " ~ dereifiy(runCode(helper())) ~ ";"); } } import std.meta; string dereifiy(T)(T t, string localName = "T") { import std.conv; import std.range; static if(isInputRange!T) { string result = "AliasSeq!("; foreach(item; t) result ~= localName ~ "[" ~ to!string(item.idx) ~ "],"; result ~= ")"; return result; } else return localName ~ "[" ~ t.idx.to!string ~ "]"; } ---
Oct 06 2020
parent reply claptrap <clap trap.com> writes:
On Tuesday, 6 October 2020 at 23:27:10 UTC, Adam D. Ruppe wrote:
 On Tuesday, 6 October 2020 at 20:57:10 UTC, claptrap wrote:
 Im less interested in performance than I am in being able to 
 express what I want to do in clear concise code.
Give me an example of what you'd like to use type functions for. I betcha I can adapt it to current D with very few changes.
I dont doubt it, but the question is could I do it? could a new D user do it? IE. Whats the learning curve like?
 Take a look at this for example:

 ---
 import std.algorithm;
 template largestType(T...) {
         auto get() {
                 return reified!"a.sizeof".map!T
                     .sort!((a, b) => a > b).front; // or 
 maxElement of course
         }
         alias largestType = T[get.idx];
 }

 pragma(msg, largestType!(long, int, real, byte)); // real
See the point is even even though I understand that, it took me a while to grep it, and I couldn't just rattle that off the top of my head, it'd take me a fair while to figure out how to do that. (Maybe i wouldn't even be able to on my own) But with Type Functions it'd be something like this... // assuming type is synonym for alias or whatever. type convTargets(type[] args) { assert(args.length > 0); type result = void; // IIRC void.sizeof == 1? foreach(t; args) if (t.size > result.sizeof) result = t; return result; } The point is TF are obvious, if you know regular D code, you can use type functions. I've got the gist of it from a handful of forums posts. The cognitive load is so much lower.
Oct 06 2020
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 7 October 2020 at 01:27:17 UTC, claptrap wrote:
 IE. Whats the learning curve like?
It is hard to say right now because this is a brand new technique. It has been possible in D ... basically forever, these language features are all quite aged, but to my knowledge, nobody has tried it this way before.
 See the point is even even though I understand that, it took me 
 a while to grep it, and I couldn't just rattle that off the top 
 of my head, it'd take me a fair while to figure out how to do 
 that. (Maybe i wouldn't even be able to on my own)
That might just be because you've never seen a combination of patterns like that before... and that's OK, I made up that particular thing just a few hours ago. (Though it is based on some ideas Andrei and I have been bouncing back and forth.) But with a little library refinement and some tutorials, maybe it would prove to be easy to learn and to use. The usage of a string lambda reminded me of the early days of std.functional and std.algorithm. Back then, a function literal looked like `(args) { return thing; }`. They shortened to strings since they liked the shortness and that's what we had, but it wasn't great. Right now, a template lambda doesn't exist. We don't even have a short form literal like delegates did. So I'm using a string there. Maybe we are walking down the same path.
 But with Type Functions it'd be something like this...

 // assuming type is synonym for alias or whatever.

 type convTargets(type[] args)
 {
     assert(args.length > 0);
     type result = void; // IIRC void.sizeof == 1?
     foreach(t; args)
         if (t.size > result.sizeof) result = t;
     return result;
 }
You could have also written that: // normal boilerplate template biggest(args...) { int helper() { // this stuff is really similar between the two! assert(args.length > 0); size_t result; size_t bestSize; // just store the answer and the size separate foreach(idx, t; args) if(t.sizeof > bestSize) { result = idx; bestSize = t; } return result; } // then this part always follows the same pattern again alias biggest = args[helper]; } That pattern btw is very useful to know regardless of if this technique works out - it is also the same idea I used for user-defined literals in D ages ago. Anyway, that's where my young library constructs are coming from - trying to recognize and abstract this pattern a little. But the core of it IS just regular D code: just it returns indexes into the array instead of the item itself because compiler tuples are weird. (I've been telling Stefan in a separate chat if type functions become compiler tuple functions, able to work with that `args...` thing to its full potential, we'd have something compelling in new functionality, not just speed. That can replace some of the super-ugly string mixins in my jsvar and jni that transform runtime argument lists! But idk if that will prove possible yet. He's exploring it, though.) The major problem with this pattern so far is it is slow. I was actually going to abandon it after Phobos' existing implementations and the typefunctions prototype both blew it out of the water in speed terms. But some people say using regular std.algorithm stuff is more important to them than speed. So I'm still toying with it for that sake.
Oct 06 2020
parent reply claptrap <clap trap.com> writes:
On Wednesday, 7 October 2020 at 03:12:46 UTC, Adam D. Ruppe wrote:
 On Wednesday, 7 October 2020 at 01:27:17 UTC, claptrap wrote:
 IE. Whats the learning curve like?
It is hard to say right now because
No it's not :)
 See the point is even even though I understand that, it took 
 me a while to grep it, and I couldn't just rattle that off the 
 top of my head, it'd take me a fair while to figure out how to 
 do that. (Maybe i wouldn't even be able to on my own)
That might just be because you've never seen a combination of patterns like that before... and that's OK, I made up that particular thing just a few hours ago. (Though it is based on some ideas Andrei and I have been bouncing back and forth.)
Yeah Im not that that used to it, but the point is you have too work at it to see whats going on, there's two nested functions calls, more things passed around, more branches and recursion. I have no problem with recursion, there's algorithms that make more sense as recursion, but this is not one, it's iterating over a list, to make it recursive you end up with 2 or 3 times more things going on in in order to make it work. Two functions, more branches, more things passed backwards and forwards, more construction of AliasSeq or whatever. Just compare how much is going on in the template version vs the TF version.
 But with a little library refinement and some tutorials, maybe 
 it would prove to be easy to learn and to use.
I'd rather 600 lines added to the compiler so I can concentrate on things I have to learn, I can write less code, my code is more intuative and less bug prone, because im not having to bend shit out of shape to make it recursive.
Oct 07 2020
parent claptrap <clap trap.com> writes:
On Wednesday, 7 October 2020 at 08:26:21 UTC, claptrap wrote:
 On Wednesday, 7 October 2020 at 03:12:46 UTC, Adam D. Ruppe 
 wrote:

 Yeah Im not that that used to it, but the point is you have too 
 work at it to see whats going on, there's two nested functions 
 calls, more things passed around, more branches and recursion.

 I have no problem with recursion, there's algorithms that make 
 more sense as recursion, but this is not one, it's iterating 
 over a list, to make it recursive you end up with 2 or 3 times 
 more things going on in in order to make it work. Two 
 functions, more branches, more things passed backwards and 
 forwards, more construction of AliasSeq or whatever.

 Just compare how much is going on in the template version vs 
 the TF version.
Appologies, Ive just woke up and I replied here in response to Daniel Ks example code, not yours Adam. Hopefully it still kinda makes sense.
Oct 07 2020
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 7 October 2020 at 01:27:17 UTC, claptrap wrote:
 On Tuesday, 6 October 2020 at 23:27:10 UTC, Adam D. Ruppe wrote:
 On Tuesday, 6 October 2020 at 20:57:10 UTC, claptrap wrote:
 Im less interested in performance than I am in being able to 
 express what I want to do in clear concise code.
Give me an example of what you'd like to use type functions for. I betcha I can adapt it to current D with very few changes.
I dont doubt it, but the question is could I do it? could a new D user do it? IE. Whats the learning curve like?
 Take a look at this for example:

 ---
 import std.algorithm;
 template largestType(T...) {
         auto get() {
                 return reified!"a.sizeof".map!T
                     .sort!((a, b) => a > b).front; // or 
 maxElement of course
         }
         alias largestType = T[get.idx];
 }

 pragma(msg, largestType!(long, int, real, byte)); // real
See the point is even even though I understand that, it took me a while to grep it, and I couldn't just rattle that off the top of my head, it'd take me a fair while to figure out how to do that. (Maybe i wouldn't even be able to on my own) But with Type Functions it'd be something like this... // assuming type is synonym for alias or whatever. type convTargets(type[] args) { assert(args.length > 0); type result = void; // IIRC void.sizeof == 1? foreach(t; args) if (t.size > result.sizeof) result = t; return result; } The point is TF are obvious, if you know regular D code, you can use type functions. I've got the gist of it from a handful of forums posts. The cognitive load is so much lower.
Your code unfortunately won't work, because `= void`; has special semantics, it won't assign the type void but leave the variable uninitialized. This is what I wrote and it has shown me a bug. alias variables should be initialized to an invalid type called, TError. Inside the type-function implementation. --- alias type = alias; type biggestType(type[] types ...) { type result; // should initialize to Terror (The error type (the type that is not a type)) // for some reason initializes to alias now? foreach(t; types) if (t.sizeof > result.sizeof) result = t; return result; } // working around parser issues again. // is expressions don't like function calls inside them ;) alias I(alias A) = A; pragma(msg, is(I!(biggestType()) == alias)); // prints true but really should not! pragma(msg, is(I!(biggestType(int)) == int)); // prints true as it should pragma(msg, is(I!(biggestType(int, ulong)) == int)); // prints false as it should pragma(msg, is(I!(biggestType(int, ulong)) == ulong)); // prints true.
Oct 07 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 7 October 2020 at 22:31:52 UTC, Stefan Koch wrote:
 alias type = alias;

 type biggestType(type[] types ...)
 {
     type result;  // should initialize to Terror (The error 
 type (the type that is not a type))
                   // for some reason initializes to alias now?
     foreach(t; types)
         if (t.sizeof > result.sizeof) result = t;
     return result;
 }
 // working around parser issues again.
 // is expressions don't like function calls inside them ;)
 alias I(alias A) = A;

 pragma(msg, is(I!(biggestType()) == alias)); // prints true but 
 really should not!
I just fixed this bug. This will now print false. the initial value of an alias variable is now ∅ The Empty type. It's even less than void. alias type = alias; type TaliasInit() { type x; assert(!is(x)); return x; } pragma(msg, TaliasInit()); // prints ∅ (Empty Type)
Oct 08 2020
parent reply Dominikus Dittes Scherkl <dominikus scherkl.de> writes:
On Thursday, 8 October 2020 at 20:52:29 UTC, Stefan Koch wrote:
 the initial value of an alias variable is now ∅ The Empty type.
Just for my understanding: Is this empty type something you defined? And would it be equivalent to the planned new bottom type (noreturn)?
Oct 08 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 9 October 2020 at 06:26:04 UTC, Dominikus Dittes 
Scherkl wrote:
 On Thursday, 8 October 2020 at 20:52:29 UTC, Stefan Koch wrote:
 the initial value of an alias variable is now ∅ The Empty type.
Just for my understanding: Is this empty type something you defined? And would it be equivalent to the planned new bottom type (noreturn)?
Thanks for asking. Actually DMD already had it, in there it's known as in there Tnone. It wasn't used for anything, I think it was supposed be the type of a not instantiated template ... It's not bottom, because bottom is a type that a type function can return to you. Tnone or ∅ (in D at least) just means this variable has not been assigned a type yet. No type can implicitly convert to ∅.
Oct 08 2020
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 9 October 2020 at 06:38:23 UTC, Stefan Koch wrote:
 Thanks for asking.
 Actually DMD already had it, in there it's known as in there 
 Tnone.
 It wasn't used for anything, I think it was supposed be the 
 type of a not instantiated template ...
I don't want to spread misinformation, it is actually used. In dmd the classes have tag values so one does not have to do dynamic casts. Tnone is used as tag for template parameter types which have been deduced, already. It even has a mangle, but I doubt you should ever see it anywhere. I will clarify with Walter.
Oct 08 2020
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 6 October 2020 at 01:12:01 UTC, Walter Bright wrote:
 On 10/5/2020 2:20 PM, Stefan Koch wrote:
 Now post it with all transitive dependencies.
I'm not sure how transitive dependencies applies here. A illuminating test case would be helpful.
Okay let me show you the transitive dependencies of the "3 liner" suggested by Andrei and executed by Paul Backus. --- phobos dependencies --- template ApplyLeft(alias Template, args...) { alias ApplyLeft(right...) = SmartAlias!(Template!(args, right)); } private template SmartAlias(T...) { static if (T.length == 1) { alias SmartAlias = Alias!T; } else { alias SmartAlias = T; } } alias Alias(alias a) = a; alias Alias(T) = T; template Filter(alias pred, TList...) { static if (TList.length < filterExpandFactor) { mixin(FilterShortCode[TList.length]); } else { template MaybeNothing(Q...) { static if (pred!(Q[0])) alias MaybeNothing = AliasSeq!(Q[0]); else alias MaybeNothing = Nothing; } alias Filter = staticMap!(MaybeNothing, TList); } } private enum staticMapExpandFactor = 150; private string generateCases() { string[staticMapExpandFactor] chunks; chunks[0] = q{}; static foreach (enum i; 0 .. staticMapExpandFactor - 1) chunks[i + 1] = chunks[i] ~ `F!(Args[` ~ i.stringof ~ `]),`; string ret = `AliasSeq!(`; foreach (chunk; chunks) ret ~= `q{alias staticMap = AliasSeq!(` ~ chunk ~ `);},`; return ret ~ `)`; } private alias staticMapBasicCases = AliasSeq!(mixin(generateCases())); template staticMap(alias F, Args...) { static if (Args.length < staticMapExpandFactor) mixin(staticMapBasicCases[Args.length]); else alias staticMap = AliasSeq!(staticMap!(F, Args[0 .. $ / 2]), staticMap!(F, Args[$ / 2 .. $])); } private alias FilterShortCode = AliasSeq!( q{ alias Filter = Nothing; }, q{ static if (pred!(TList[0])) alias Filter = AliasSeq!(TList[0]); else alias Filter = Nothing; }, q{ static if (pred!(TList[0])) { static if (pred!(TList[1])) alias Filter = AliasSeq!(TList[0], TList[1]); else alias Filter = AliasSeq!(TList[0]); } else { static if (pred!(TList[1])) alias Filter = AliasSeq!(TList[1]); else alias Filter = Nothing; } }, q{ static if (pred!(TList[0])) { static if (pred!(TList[1])) { static if (pred!(TList[2])) alias Filter = AliasSeq!(TList[0], TList[1], TList[2]); else alias Filter = AliasSeq!(TList[0], TList[1]); } else { static if (pred!(TList[2])) alias Filter = AliasSeq!(TList[0], TList[2]); else alias Filter = AliasSeq!(TList[0]); } } else { static if (pred!(TList[1])) { static if (pred!(TList[2])) alias Filter = AliasSeq!(TList[1], TList[2]); else alias Filter = AliasSeq!(TList[1]); } else { static if (pred!(TList[2])) alias Filter = AliasSeq!(TList[2]); else alias Filter = Nothing; } } }, q{ static if (pred!(TList[0])) { static if (pred!(TList[1])) { static if (pred!(TList[2])) { static if (pred!(TList[3])) alias Filter = AliasSeq!(TList[0], TList[1], TList[2], TList[3]); else alias Filter = AliasSeq!(TList[0], TList[1], TList[2]); } else { static if (pred!(TList[3])) alias Filter = AliasSeq!(TList[0], TList[1], TList[3]); else alias Filter = AliasSeq!(TList[0], TList[1]); } } else { static if (pred!(TList[2])) { static if (pred!(TList[3])) alias Filter = AliasSeq!(TList[0], TList[2], TList[3]); else alias Filter = AliasSeq!(TList[0], TList[2]); } else { static if (pred!(TList[3])) alias Filter = AliasSeq!(TList[0], TList[3]); else alias Filter = AliasSeq!(TList[0]); } } } else { static if (pred!(TList[1])) { static if (pred!(TList[2])) { static if (pred!(TList[3])) alias Filter = AliasSeq!(TList[1], TList[2], TList[3]); else alias Filter = AliasSeq!(TList[1], TList[2]); } else { static if (pred!(TList[3])) alias Filter = AliasSeq!(TList[1], TList[3]); else alias Filter = AliasSeq!(TList[1]); } } else { static if (pred!(TList[2])) { static if (pred!(TList[3])) alias Filter = AliasSeq!(TList[2], TList[3]); else alias Filter = AliasSeq!(TList[2]); } else { static if (pred!(TList[3])) alias Filter = AliasSeq!(TList[3]); else alias Filter = Nothing; } } } } ); private enum filterExpandFactor = FilterShortCode.length; package alias Nothing = AliasSeq!(); // yes, this really does speed up compilation! --- --- actual 3 liner --- alias Numerics = AliasSeq!(byte, ubyte, short, ushort, int, uint, long, ulong, float, double, real, char, wchar, dchar); enum convertsTo(T, U) = is(T : U); alias ImplicitConversionTargets(T) = Filter!(ApplyLeft!(convertsTo, T), Numerics); pragma(msg, ImplicitConversionTargets!uint); --- I would consider this illuminating in a way.
Oct 06 2020
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 6 October 2020 at 10:22:44 UTC, Stefan Koch wrote:
 On Tuesday, 6 October 2020 at 01:12:01 UTC, Walter Bright wrote:
 On 10/5/2020 2:20 PM, Stefan Koch wrote:
 Now post it with all transitive dependencies.
I'm not sure how transitive dependencies applies here. A illuminating test case would be helpful.
Okay let me show you the transitive dependencies of the "3 liner" suggested by Andrei and executed by Paul Backus. --- phobos dependencies --- ...
Ah dang. This won't compile ... I forgot --- AliasSeq(T...) = T; --- with that added it will compile.
Oct 06 2020
prev sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Tuesday, 6 October 2020 at 10:22:44 UTC, Stefan Koch wrote:
 On Tuesday, 6 October 2020 at 01:12:01 UTC, Walter Bright wrote:
 [...]
Okay let me show you the transitive dependencies of the "3 liner" suggested by Andrei and executed by Paul Backus. [...]
That implementation was *definitely* (and very obviously) not created for simplicity. It can be done in way fewer lines than that (and you know that, obviously). The argument to be made is "look how ridiculous this code has to be to avoid some of the worst performance problems, look how simple the type functions approach is and how much faster it is", yes?
Oct 06 2020
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 6 October 2020 at 12:07:40 UTC, John Colvin wrote:
 On Tuesday, 6 October 2020 at 10:22:44 UTC, Stefan Koch wrote:
 On Tuesday, 6 October 2020 at 01:12:01 UTC, Walter Bright 
 wrote:
 [...]
Okay let me show you the transitive dependencies of the "3 liner" suggested by Andrei and executed by Paul Backus. [...]
That implementation was *definitely* (and very obviously) not created for simplicity. It can be done in way fewer lines than that (and you know that, obviously). The argument to be made is "look how ridiculous this code has to be to avoid some of the worst performance problems, look how simple the type functions approach is and how much faster it is", yes?
Yes that sums it up very well actually. I am not good enough with recursive templates to simplify the implementation without the risk of introducing bugs unfortunately. Which is why I asked for others to provide code to compare this to.
Oct 06 2020
prev sibling parent reply claptrap <clap trap.com> writes:
On Tuesday, 6 October 2020 at 12:07:40 UTC, John Colvin wrote:
 On Tuesday, 6 October 2020 at 10:22:44 UTC, Stefan Koch wrote:
 On Tuesday, 6 October 2020 at 01:12:01 UTC, Walter Bright 
 wrote:
 [...]
Okay let me show you the transitive dependencies of the "3 liner" suggested by Andrei and executed by Paul Backus. [...]
That implementation was *definitely* (and very obviously) not created for simplicity. It can be done in way fewer lines than that (and you know that, obviously).
Maybe "simple and concise" takes a lot more work to extract from templates. I mean the type function version is so obvious it's probably what a newbie would write, the simple concise solution is the obvious one. Is the same true of the template / mixin versions? Or do you have to be a D sage to make it palatable?
Oct 06 2020
parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Tuesday, 6 October 2020 at 12:32:17 UTC, claptrap wrote:
 On Tuesday, 6 October 2020 at 12:07:40 UTC, John Colvin wrote:
 On Tuesday, 6 October 2020 at 10:22:44 UTC, Stefan Koch wrote:
 On Tuesday, 6 October 2020 at 01:12:01 UTC, Walter Bright 
 wrote:
 [...]
Okay let me show you the transitive dependencies of the "3 liner" suggested by Andrei and executed by Paul Backus. [...]
That implementation was *definitely* (and very obviously) not created for simplicity. It can be done in way fewer lines than that (and you know that, obviously).
Maybe "simple and concise" takes a lot more work to extract from templates. I mean the type function version is so obvious it's probably what a newbie would write, the simple concise solution is the obvious one. Is the same true of the template / mixin versions? Or do you have to be a D sage to make it palatable?
If you look at the history of std.meta you'll see it was me who recently made Filter so ugly, the previous version was simple but too slow.
Oct 06 2020
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 6 October 2020 at 13:20:43 UTC, John Colvin wrote:
 [snip]

 If you look at the history of std.meta you'll see it was me who 
 recently made Filter so ugly, the previous version was simple 
 but too slow.
To get Filter/staticMap to have improved performance (and it looks to me like Stefan copied the phobos version in his example), you were forced to make them uglier. But, isn't Stefan's point that his version has even better performance than your ugly version? Is your argument that the simple version would have better performance in this case?
Oct 06 2020
next sibling parent John Colvin <john.loughran.colvin gmail.com> writes:
On Tuesday, 6 October 2020 at 13:35:25 UTC, jmh530 wrote:
 On Tuesday, 6 October 2020 at 13:20:43 UTC, John Colvin wrote:
 [snip]

 If you look at the history of std.meta you'll see it was me 
 who recently made Filter so ugly, the previous version was 
 simple but too slow.
To get Filter/staticMap to have improved performance (and it looks to me like Stefan copied the phobos version in his example), you were forced to make them uglier. But, isn't Stefan's point that his version has even better performance than your ugly version? Is your argument that the simple version would have better performance in this case?
I'm not really trying to argue anything, just trying to inform/clarify a little
Oct 06 2020
prev sibling parent reply foobar <foo bar.com> writes:
On Tuesday, 6 October 2020 at 13:35:25 UTC, jmh530 wrote:
 On Tuesday, 6 October 2020 at 13:20:43 UTC, John Colvin wrote:
 [snip]

 If you look at the history of std.meta you'll see it was me 
 who recently made Filter so ugly, the previous version was 
 simple but too slow.
To get Filter/staticMap to have improved performance (and it looks to me like Stefan copied the phobos version in his example), you were forced to make them uglier. But, isn't Stefan's point that his version has even better performance than your ugly version? Is your argument that the simple version would have better performance in this case?
That is a newbie thing to say. The big complication swept under the rug is the language. If you make the language bigger you can always makes things look nicer. It doesn't mean you should. To moderators: Are you seriously rejecting my posts for overquoting? What happened to this community?
Oct 06 2020
parent jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 6 October 2020 at 17:50:33 UTC, foobar wrote:
 [snip]

 That is a newbie thing to say. The big complication swept under 
 the rug is the language. If you make the language bigger you 
 can always makes things look nicer. It doesn't mean you should.
Honestly, I was just trying to figure out what he was trying to say. He subsequently wrote that he was just providing some background information (or something to that effect).
 To moderators: Are you seriously rejecting my posts for 
 overquoting? What happened to this community?
It's automated and has been like that as long as I've been here.
Oct 06 2020
prev sibling next sibling parent claptrap <clap trap.com> writes:
On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:
 Hi,


 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another basic 
 type :))

 As noted previously please do discuss!
 Maybe you like the template version more?
 Let me know.
100% the type functions. It's also worth nothing that the way the TF version is implemented you have a bunch of stuff that is reusable, you could argue that... makeAliasArray isBasicType shouldn't be in the line count as they would be general library functions anyway. And at the very least the decomposition into smaller more understandable and separately useful parts is as a big win as the "easy to explain to a newbie" syntax is.
Oct 05 2020
prev sibling next sibling parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:
[...]

 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another basic 
 type :))
cent and ucent are waiting ;-)
 As noted previously please do discuss!
 Maybe you like the template version more?
 Let me know.
Oct 05 2020
parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Monday, 5 October 2020 at 15:07:37 UTC, Patrick Schluter wrote:
 On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:
 [...]

 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another 
 basic type :))
cent and ucent are waiting ;-)
ooops, should have read the rest of the thread before commenting...
Oct 05 2020
prev sibling next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:
 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another basic 
 type :))
Also trivially easy with mixins. Normally I don't suggest doing things this way but for basic types mixin is guaranteed to work. ``` enum basic_types = ["bool", "ubyte", "char", "byte", "ushort", "wchar", "short", "uint", "dchar", "int", "ulong", "long"]; // to not import stdlib alias AliasSeq(T...) = T; string join(string[] s, string j) { string a; foreach(i; s) a ~= i ~ j; return a; } template basicTypeConvTargets(T) { // again, I copy/pasted your code with very slight modifications string[] helper() { string[] targets; targets.length = basic_types.length; size_t n = 0; static foreach(t;basic_types) { if (is(T : mixin(t))) { targets[n++] = t; } } return targets[0 .. n]; } mixin("alias basicTypeConvTargets = AliasSeq!(" ~ helper().join(",") ~ ");"); } pragma(msg, basicTypeConvTargets!ushort);
Oct 05 2020
parent claptrap <clap trap.com> writes:
On Monday, 5 October 2020 at 21:20:15 UTC, Adam D. Ruppe wrote:
 On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:
 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another 
 basic type :))
Also trivially easy with mixins. Normally I don't suggest doing things this way but for basic types mixin is guaranteed to work. ``` enum basic_types = ["bool", "ubyte", "char", "byte", "ushort", "wchar", "short", "uint", "dchar", "int", "ulong", "long"]; // to not import stdlib alias AliasSeq(T...) = T; string join(string[] s, string j) { string a; foreach(i; s) a ~= i ~ j; return a; } template basicTypeConvTargets(T) { // again, I copy/pasted your code with very slight modifications string[] helper() { string[] targets; targets.length = basic_types.length; size_t n = 0; static foreach(t;basic_types) { if (is(T : mixin(t))) { targets[n++] = t; } } return targets[0 .. n]; } mixin("alias basicTypeConvTargets = AliasSeq!(" ~ helper().join(",") ~ ");"); } pragma(msg, basicTypeConvTargets!ushort);
Thats still kinda hideous compared to the TF version. I mean you can grep the TF version at first glance, you literally barely have to think about it. Your mixin version takes some skipping back and forth between the code to work out what its doing. There's that maxim that your code is read many times more that it is written, even by yourself never mind by other people. So a piece of code that takes 10 seconds to understand vs 30 seconds is a huge win. Faster for people to debug modify, and less likely they'll introduce new bugs. I mean if somebody comes along a year or two down the line to modify the TF version and the Mixin version, its not hard to imagine which one are they more likely to screw up. Which makes me wonder what is going on with the people steering D that this doesnt seem to have any importance anymore?
Oct 05 2020
prev sibling next sibling parent reply Meta <jared771 gmail.com> writes:
On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:
 Hi,

 I've posted an incomplete version of a semantic translation of 
 ImplictConvTargets from a template into a type function.

 After a few rather trivial fixes let me show you what the code 
 looks like now.

 ---
 alias type = alias;

 // needed to avoid deeper changes ... in the future it may be 
 unnecessary.
 auto makeAliasArray(type[] types ...)
 {
     return types;
 }

 enum basic_types = makeAliasArray(bool, ubyte, char, byte, 
 ushort, wchar, short, uint, dchar, int, ulong, long);

 type[] convTargets(type T)
 {
     if (isBasicType(T))
         return basicTypeConvTargets(T);
     return null;
 }

 bool isBasicType(type T)
 {
     foreach(t;basic_types)
     {
         if (is(T == t))
             return true;
     }
     return false;
 }

 type[] basicTypeConvTargets(type T)
 {
     type[] targets;
     targets.length = basic_types.length;
     assert(isBasicType(T), "You may not call this function when 
 you don't have a basic type ... (given: " ~ T.stringof ~ ")");
     size_t n = 0;
     foreach(t;basic_types)
     {
         if (is(T : t))
         {
             targets[n++] = t;
         }
     }
     return targets[0 .. n];
 }
 // 42 lines including whitespace and comments

 pragma(msg, convTargets(long)); // outputs [(ulong), (long)]
It would be very impressive if the following works: type[] basicTypeConvTargets(type T) { assert(isBasicType(T), "You may not call this function when you don't have a basic type ... (given: " ~ T.stringof ~ ")"); return basic_types.filter!((alias U) => is(T: U)).array; } Yes, it instantiates a few templates, but it also demonstrates that working with type-values is as easy as working with regular values.
Oct 05 2020
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 October 2020 at 22:53:31 UTC, Meta wrote:
 type[] basicTypeConvTargets(type T)
 {
     assert(isBasicType(T), "You may not call this function when 
 you don't have a basic type ... (given: " ~ T.stringof ~ ")");
     return basic_types.filter!((alias U) => is(T: U)).array;
 }

 Yes, it instantiates a few templates, but it also demonstrates 
 that working with type-values is as easy as working with 
 regular values.
That should work actually. as long as you include "alias type = alias;" which gives you the type type. And then you can write basic_types.filter!((type U) => is(T: U)).array; If that doesn't work, I am going to fix it shortly.
Oct 05 2020
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:
 Hi,

 I've posted an incomplete version of a semantic translation of 
 ImplictConvTargets from a template into a type function.

 After a few rather trivial fixes let me show you what the code 
 looks like now.

 ---
 alias type = alias;

 // needed to avoid deeper changes ... in the future it may be 
 unnecessary.
 auto makeAliasArray(type[] types ...)
 {
     return types;
 }

 enum basic_types = makeAliasArray(bool, ubyte, char, byte, 
 ushort, wchar, short, uint, dchar, int, ulong, long);

 type[] convTargets(type T)
 {
     if (isBasicType(T))
         return basicTypeConvTargets(T);
     return null;
 }

 bool isBasicType(type T)
 {
     foreach(t;basic_types)
     {
         if (is(T == t))
             return true;
     }
     return false;
 }

 type[] basicTypeConvTargets(type T)
 {
     type[] targets;
     targets.length = basic_types.length;
     assert(isBasicType(T), "You may not call this function when 
 you don't have a basic type ... (given: " ~ T.stringof ~ ")");
     size_t n = 0;
     foreach(t;basic_types)
     {
         if (is(T : t))
         {
             targets[n++] = t;
         }
     }
     return targets[0 .. n];
 }
 // 42 lines including whitespace and comments

 pragma(msg, convTargets(long)); // outputs [(ulong), (long)]
 ---

 And again here is the part of the template that we just 
 re-implemented:
 ---
     static if (is(T == bool))
         alias ImplicitConversionTargets =
             AliasSeq!(byte, ubyte, short, ushort, int, uint, 
 long, ulong, CentTypeList,
                        float, double, real, char, wchar, dchar);
     else static if (is(T == byte))
         alias ImplicitConversionTargets =
             AliasSeq!(short, ushort, int, uint, long, ulong, 
 CentTypeList,
                        float, double, real, char, wchar, dchar);
     else static if (is(T == ubyte))
         alias ImplicitConversionTargets =
             AliasSeq!(short, ushort, int, uint, long, ulong, 
 CentTypeList,
                        float, double, real, char, wchar, dchar);
     else static if (is(T == short))
         alias ImplicitConversionTargets =
             AliasSeq!(int, uint, long, ulong, CentTypeList, 
 float, double, real);
     else static if (is(T == ushort))
         alias ImplicitConversionTargets =
             AliasSeq!(int, uint, long, ulong, CentTypeList, 
 float, double, real);
     else static if (is(T == int))
         alias ImplicitConversionTargets =
             AliasSeq!(long, ulong, CentTypeList, float, double, 
 real);
     else static if (is(T == uint))
         alias ImplicitConversionTargets =
             AliasSeq!(long, ulong, CentTypeList, float, double, 
 real);
     else static if (is(T == long))
         alias ImplicitConversionTargets = AliasSeq!(float, 
 double, real);
     else static if (is(T == ulong))
         alias ImplicitConversionTargets = AliasSeq!(float, 
 double, real);
     // part omitted because we don't have ucent and cent in our 
 list
     else static if (is(T == char))
         alias ImplicitConversionTargets =
             AliasSeq!(wchar, dchar, byte, ubyte, short, ushort,
                        int, uint, long, ulong, CentTypeList, 
 float, double, real);
     else static if (is(T == wchar))
         alias ImplicitConversionTargets =
             AliasSeq!(dchar, short, ushort, int, uint, long, 
 ulong, CentTypeList,
                        float, double, real);
     else static if (is(T == dchar))
         alias ImplicitConversionTargets =
             AliasSeq!(int, uint, long, ulong, CentTypeList, 
 float, double, real);
     // 41 lines including white-space and comments (only that 
 there is no white-space or comments)
 ---

 I leave it up to you to decide which version is more 
 understandable and extendable (should we ever get another basic 
 type :))

 As noted previously please do discuss!
 Maybe you like the template version more?
 Let me know.
And here is the performance comparison: uplimk uplimk-virtual-machine:~/d/dmd$ hyperfine -w 10 "generated/linux/release/64/dmd xx_tf.d -sktf -c" "generated/linux/release/64/dmd xx.d -c " "generated/linux/release/64/dmd xx_adam.d -c" Time (mean ± σ): 9.1 ms ± 0.4 ms [User: 5.8 ms, System: 3.2 ms] Range (min … max): 8.5 ms … 10.7 ms 288 runs Time (mean ± σ): 17.3 ms ± 0.6 ms [User: 11.3 ms, System: 5.9 ms] Range (min … max): 16.5 ms … 19.5 ms 166 runs Time (mean ± σ): 20.4 ms ± 0.7 ms [User: 15.2 ms, System: 5.0 ms] Range (min … max): 19.5 ms … 22.7 ms 144 runs Summary 'generated/linux/release/64/dmd xx_tf.d -sktf -c' ran 1.91 ± 0.11 times faster than 'generated/linux/release/64/dmd xx.d -c ' 2.24 ± 0.13 times faster than 'generated/linux/release/64/dmd xx_adam.d -c' The source code can be found here: https://gist.github.com/UplinkCoder/d29dd143b352d28390426a3ffedf9521 So can the benchmark results. Note: not importing std.meta gives the template version (xx.d) a speed boost of 3 milliseconds.
Oct 06 2020
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 6 October 2020 at 11:14:05 UTC, Stefan Koch wrote:
 [snip]
 The source code can be found here:

 https://gist.github.com/UplinkCoder/d29dd143b352d28390426a3ffedf9521

 So can the benchmark results.
 Note: not importing std.meta gives the template version (xx.d) 
 a speed boost of 3 milliseconds.
Thanks for providing these results. You had previously provided some evidence of a significant reduction in memory consumption. You might consider adding that information (using these examples) to this gist to bolster your argument.
Oct 06 2020
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 6 October 2020 at 11:25:41 UTC, jmh530 wrote:
 On Tuesday, 6 October 2020 at 11:14:05 UTC, Stefan Koch wrote:
 [snip]
 The source code can be found here:

 https://gist.github.com/UplinkCoder/d29dd143b352d28390426a3ffedf9521

 So can the benchmark results.
 Note: not importing std.meta gives the template version (xx.d) 
 a speed boost of 3 milliseconds.
Thanks for providing these results. You had previously provided some evidence of a significant reduction in memory consumption. You might consider adding that information (using these examples) to this gist to bolster your argument.
Memory consumption is reduced by 2x roughly. (adams and pauls version are about the same, with pauls sucking a bit more memory, perhaps because there is more to parse?) I'll add it to the gist.
Oct 06 2020
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 6 October 2020 at 11:25:41 UTC, jmh530 wrote:
 On Tuesday, 6 October 2020 at 11:14:05 UTC, Stefan Koch wrote:
 [snip]
 The source code can be found here:

 https://gist.github.com/UplinkCoder/d29dd143b352d28390426a3ffedf9521

 So can the benchmark results.
 Note: not importing std.meta gives the template version (xx.d) 
 a speed boost of 3 milliseconds.
Thanks for providing these results. You had previously provided some evidence of a significant reduction in memory consumption. You might consider adding that information (using these examples) to this gist to bolster your argument.
I've just added the gist: https://gist.github.com/UplinkCoder/d29dd143b352d28390426a3ffedf9521#file-benchmark-results-txt Perhaps there is a better way than /usr/bin/time ?
Oct 06 2020
prev sibling parent reply Daniel K <dkm4i1 gmail.com> writes:
On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:
 Maybe you like the template version more?
 Let me know.
I love the premise. Like watching people in TV ads compare the knife of the competition with their own. Completely mutilating a loaf of bread with the competition. What if we make an honest comparison? I spent 15 minutes writing this: template Tuple(T...) { alias Tuple = T; } alias basic_types = Tuple!(bool, ubyte, char, byte, ushort, wchar, short, uint, dchar, int, ulong, long); template ImplicitConvertible(T, Candidate, Candidates...) { static if (is (T : Candidate)) alias Include = Candidate; else alias Include = Tuple!(); static if (Candidates.length > 0) alias ImplicitConvertible = Tuple!(Include, ImplicitConvertible!(T, Candidates)); else alias ImplicitConvertible = Include; } template ImplicitConversionTargets(T) { alias ImplicitConversionTargets = ImplicitConvertible!(T, basic_types); } pragma (msg, ImplicitConversionTargets!(long)); That's 16 lines of code. Heck it even compiles in D1 if only the alias declarations are written in the old style. Personally I prefer using existing language features. /Daniel K
Oct 06 2020
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 6 October 2020 at 18:00:29 UTC, Daniel K wrote:
 On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:
 [...]
I love the premise. Like watching people in TV ads compare the knife of the competition with their own. Completely mutilating a loaf of bread with the competition. [...]
As is your right. I spent 5 minutes on the type-function, because there's nothing to think about. You just loop and that's it :)
Oct 06 2020
parent Daniel K <dkm4i1 gmail.com> writes:
On Tuesday, 6 October 2020 at 18:43:42 UTC, Stefan Koch wrote:
 On Tuesday, 6 October 2020 at 18:00:29 UTC, Daniel K wrote:
 On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:
 [...]
I love the premise. Like watching people in TV ads compare the knife of the competition with their own. Completely mutilating a loaf of bread with the competition. [...]
As is your right. I spent 5 minutes on the type-function, because there's nothing to think about. You just loop and that's it :)
I like how you concisely argued technical aspects. Only people with weak arguments would have picked irrelevant points. Wait... oh... /Daniel K
Oct 06 2020
prev sibling parent reply claptrap <clap trap.com> writes:
On Tuesday, 6 October 2020 at 18:00:29 UTC, Daniel K wrote:
 On Monday, 5 October 2020 at 11:44:34 UTC, Stefan Koch wrote:

 template ImplicitConversionTargets(T)
 {
 	alias ImplicitConversionTargets = ImplicitConvertible!(T, 
 basic_types);
 }

 pragma (msg, ImplicitConversionTargets!(long));

 That's 16 lines of code. Heck it even compiles in D1 if only 
 the alias declarations are written in the old style.

 Personally I prefer using existing language features.

 /Daniel K
If someone said to you "I have this list of integers and I want to get a new list containing the ones are divisible by a specific value", would you write this... ---- int[] vals = [4,7,28,23,585,73,12]; int[] getMultiplesX(int i, int candidate, int[] candidates) { int[] result; if ((candidate % i) == 0) result ~= candidate; if (candidates.length > 0) return result ~ getMultiplesX(i, candidates[0], candidates[1..$]); else return result; } int[] getMultiplesOf(int i) { return getMultiplesX(i, vals[0], vals[1..$]); } ---- Or would you write it like this... int[] vals = [4,7,28,23,585,73,12]; int[] getMultiplesOf(int i) { int[] result; foreach(v; vals) if ((v % i) == 0) result ~= v; return result; } ---- Its the same basic algorithm, search a list and make a new list from the entries that satisfy a certain condition. It beggars belief that anyone would argue that the second version is not better on every metric that is important in writing good code. It's clearer, more intuitive, more concise, it will almost definitely be faster and less wasteful of resources. Newbies will grep it far quicker than the other versions. That means faster to write and easier to maintain and debug. I'm honestly losing the will to live here.
Oct 06 2020
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via Digitalmars-d wrote:
[...]
 If someone said to you "I have this list of integers and I want to get
 a new list containing the ones are divisible by a specific value",
 would you write this...
 
 ----
 
 int[] vals = [4,7,28,23,585,73,12];
 
 int[] getMultiplesX(int i, int candidate, int[] candidates)
 {
     int[] result;
     if ((candidate % i) == 0)
         result ~= candidate;
     if (candidates.length > 0)
         return result ~ getMultiplesX(i, candidates[0], candidates[1..$]);
     else
        return result;
 }
 
 int[]  getMultiplesOf(int i)
 {
     return getMultiplesX(i, vals[0], vals[1..$]);
 }
 
 ----
 
 Or would you write it like this...
 
 int[] vals = [4,7,28,23,585,73,12];
 
 int[] getMultiplesOf(int i)
 {
     int[] result;
     foreach(v; vals)
         if ((v % i) == 0) result ~= v;
     return result;
 }
I would write it like this: int[] vals = [4,7,28,23,585,73,12]; int[] getMultiplesOf(int i) { return vals.filter!(v => (v % i) == 0).array; } One line vs. 4, even more concise. ;-) Thing is, what someone finds intuitive or not, is a pretty subjective matter, and depends on what programming style he's familiar with and/or prefers. What a C programmer finds readable and obvious may be needlessly arcane to a Java programmer, and what an APL programmer finds totally obvious may be completely impenetrable to anyone else. :-P If we're going to argue over merits, it would really help resolve matters if we stuck to things that are objectively measurable, since reasonable people are going to disagree on the subjective points. T -- Being able to learn is a great learning; being able to unlearn is a greater learning.
Oct 06 2020
parent reply claptrap <clap trap.com> writes:
On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:
 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via 
 Digitalmars-d wrote: [...]

 I would write it like this:

 	int[] vals = [4,7,28,23,585,73,12];

 	int[] getMultiplesOf(int i)
 	{
 	    return vals.filter!(v => (v % i) == 0).array;
 	}

 One line vs. 4, even more concise. ;-)
The point is to show language not library.
 Thing is, what someone finds intuitive or not, is a pretty 
 subjective matter, and depends on what programming style he's 
 familiar with and/or prefers.  What a C programmer finds 
 readable and obvious may be needlessly arcane to a Java 
 programmer, and what an APL programmer finds totally obvious 
 may be completely impenetrable to anyone else. :-P
We're not looking for "is this intuitive to Java programmers", we're asking is this intuitive to D programmers, so if they already know D then *you have context* in which to judge whether it's intuitive or not. And "It's just like regular D code but with types" pretty much hits the nail on the head as fair as intuitive goes.
Oct 06 2020
next sibling parent reply Daniel K <dkm4i1 gmail.com> writes:
On Wednesday, 7 October 2020 at 01:07:17 UTC, claptrap wrote:
 On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:
 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via 
 Digitalmars-d wrote: [...]

 I would write it like this:

 	int[] vals = [4,7,28,23,585,73,12];

 	int[] getMultiplesOf(int i)
 	{
 	    return vals.filter!(v => (v % i) == 0).array;
 	}

 One line vs. 4, even more concise. ;-)
The point is to show language not library.
 Thing is, what someone finds intuitive or not, is a pretty 
 subjective matter, and depends on what programming style he's 
 familiar with and/or prefers.  What a C programmer finds 
 readable and obvious may be needlessly arcane to a Java 
 programmer, and what an APL programmer finds totally obvious 
 may be completely impenetrable to anyone else. :-P
We're not looking for "is this intuitive to Java programmers", we're asking is this intuitive to D programmers, so if they already know D then *you have context* in which to judge whether it's intuitive or not. And "It's just like regular D code but with types" pretty much hits the nail on the head as fair as intuitive goes.
If recursive templates are not intuitive to you, perhaps you still have more D to learn, to become this mythical "D programmer". My 16 lines of template essentially compiled in D for at least the past 10 years. It literally is "regular D code". If recursion and declarative programming isn't intuitive to you in general, then perhaps that's not D's problem at all. /Daniel K
Oct 06 2020
parent claptrap <clap trap.com> writes:
On Wednesday, 7 October 2020 at 02:10:03 UTC, Daniel K wrote:
 On Wednesday, 7 October 2020 at 01:07:17 UTC, claptrap wrote:
 On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:
 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via 
 Digitalmars-d wrote: [...]

 I would write it like this:

 	int[] vals = [4,7,28,23,585,73,12];

 	int[] getMultiplesOf(int i)
 	{
 	    return vals.filter!(v => (v % i) == 0).array;
 	}

 One line vs. 4, even more concise. ;-)
The point is to show language not library.
 Thing is, what someone finds intuitive or not, is a pretty 
 subjective matter, and depends on what programming style he's 
 familiar with and/or prefers.  What a C programmer finds 
 readable and obvious may be needlessly arcane to a Java 
 programmer, and what an APL programmer finds totally obvious 
 may be completely impenetrable to anyone else. :-P
We're not looking for "is this intuitive to Java programmers", we're asking is this intuitive to D programmers, so if they already know D then *you have context* in which to judge whether it's intuitive or not. And "It's just like regular D code but with types" pretty much hits the nail on the head as fair as intuitive goes.
If recursive templates are not intuitive to you, perhaps you still have more D to learn, to become this mythical "D programmer".
There's 2 or 3 times more stuff going on in the template version. Two function calls, more things passed around, array or alias seq slicing, appending, more ifs and elses, etc... Is less intuitive because it's an algorithm that makes more sense as iteration, so you have to bend it all out of shape to make it recursive. I like code as simple as possible, that isnt it.
 My 16 lines of template essentially compiled in D for at least 
 the past 10 years.
 It literally is "regular D code".
By regular D code, I mean non-teplate / meta code. I thought that was obvious.
 If recursion and declarative programming isn't intuitive to you 
 in general, then perhaps that's not D's problem at all.
Recursion is fine, there's algorithms that make more sense that way, this isnt one of them. Thats the point. You're forcing everything in meta land to be recursive. If all you have is a hammer.... And if all you've had is a hammer for 10 years, you start to doubt the existing of screws.
Oct 07 2020
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/6/20 9:07 PM, claptrap wrote:
 On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:
 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via Digitalmars-d 
 wrote: [...]

 I would write it like this:

     int[] vals = [4,7,28,23,585,73,12];

     int[] getMultiplesOf(int i)
     {
         return vals.filter!(v => (v % i) == 0).array;
     }

 One line vs. 4, even more concise. ;-)
The point is to show language not library.
That's a made-up restriction, and it's odd that it is being discussed here as a virtue. Beginners are attracted to large languages that have everything built in. A good language is focused on general primitives that allow writing a great deal in libraries.
Oct 06 2020
next sibling parent claptrap <clap trap.com> writes:
On Wednesday, 7 October 2020 at 02:33:21 UTC, Andrei Alexandrescu 
wrote:
 On 10/6/20 9:07 PM, claptrap wrote:
 On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:
 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via 
 Digitalmars-d wrote: [...]

 I would write it like this:

     int[] vals = [4,7,28,23,585,73,12];

     int[] getMultiplesOf(int i)
     {
         return vals.filter!(v => (v % i) == 0).array;
     }

 One line vs. 4, even more concise. ;-)
The point is to show language not library.
That's a made-up restriction, and it's odd that it is being discussed here as a virtue.
If you're trying to compare how the two features work in practice it's not going to be very illuminating if you hide all the workings behind a function call. Ie what good is int[] getMultiplesOf(int i) { return vals.tf_filter(v => (v % i) == 0); } vs... int[] getMultiplesOf(int i) { return vals.filter!(v => (v % i) == 0).array; } Yeah that helps a lot :)
Oct 07 2020
prev sibling next sibling parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Wednesday, 7 October 2020 at 02:33:21 UTC, Andrei Alexandrescu 
wrote:
 On 10/6/20 9:07 PM, claptrap wrote:
 On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:
 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via 
 Digitalmars-d wrote: [...]

 I would write it like this:

     int[] vals = [4,7,28,23,585,73,12];

     int[] getMultiplesOf(int i)
     {
         return vals.filter!(v => (v % i) == 0).array;
     }

 One line vs. 4, even more concise. ;-)
The point is to show language not library.
That's a made-up restriction, and it's odd that it is being discussed here as a virtue.
No, it's not. It's central to the argument.
 Beginners are attracted to large languages that have everything 
 built in. A good language is focused on general primitives that 
 allow writing a great deal in libraries.
Then do lisp or forth but not D or C++.
Oct 07 2020
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 7 October 2020 at 08:49:21 UTC, Patrick Schluter 
wrote:
 On Wednesday, 7 October 2020 at 02:33:21 UTC, Andrei 
 Alexandrescu wrote:
 That's a made-up restriction, and it's odd that it is being 
 discussed here as a virtue.
No, it's not. It's central to the argument.
Exactly right. There comes a time when you try to do something that isn't covered by libraries, what do you do then?
 Beginners are attracted to large languages that have 
 everything built in. A good language is focused on general 
 primitives that allow writing a great deal in libraries.
Then do lisp or forth but not D or C++.
Right again. Funtional programming is only pleasant in a dedicated FP language, and even then you need to memorize a large set of library constructs to be productive. D could improve on FP, both language and performance, but adding type variables is a better cultural fit... (And Stefan is only putting forth a concept, not the end result)
Oct 07 2020
parent reply claptrap <clap trap.com> writes:
On Wednesday, 7 October 2020 at 09:04:59 UTC, Ola Fosheim Grøstad 
wrote:
 On Wednesday, 7 October 2020 at 08:49:21 UTC, Patrick Schluter 
 wrote:
 On Wednesday, 7 October 2020 at 02:33:21 UTC, Andrei 
 Alexandrescu wrote:
Right again. Funtional programming is only pleasant in a dedicated FP language, and even then you need to memorize a large set of library constructs to be productive.
How many pure funtional languages are in the tiobe top 10? None or maybe 1 im not sure. Theres a reason for that i reckon.
Oct 07 2020
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Wednesday, 7 October 2020 at 11:19:24 UTC, claptrap wrote:
 On Wednesday, 7 October 2020 at 09:04:59 UTC, Ola Fosheim 
 Grøstad wrote:
 On Wednesday, 7 October 2020 at 08:49:21 UTC, Patrick Schluter 
 wrote:
 On Wednesday, 7 October 2020 at 02:33:21 UTC, Andrei 
 Alexandrescu wrote:
Right again. Funtional programming is only pleasant in a dedicated FP language, and even then you need to memorize a large set of library constructs to be productive.
How many pure funtional languages are in the tiobe top 10? None or maybe 1 im not sure. Theres a reason for that i reckon.
We are using a pure functional language for web frontend development, and it's a joy ...
Oct 07 2020
parent reply claptrap <clap trap.com> writes:
On Wednesday, 7 October 2020 at 11:55:00 UTC, Paolo Invernizzi 
wrote:
 On Wednesday, 7 October 2020 at 11:19:24 UTC, claptrap wrote:
 On Wednesday, 7 October 2020 at 09:04:59 UTC, Ola Fosheim 
 Grøstad wrote:
 On Wednesday, 7 October 2020 at 08:49:21 UTC, Patrick 
 Schluter wrote:
 On Wednesday, 7 October 2020 at 02:33:21 UTC, Andrei 
 Alexandrescu wrote:
Right again. Funtional programming is only pleasant in a dedicated FP language, and even then you need to memorize a large set of library constructs to be productive.
How many pure funtional languages are in the tiobe top 10? None or maybe 1 im not sure. Theres a reason for that i reckon.
We are using a pure functional language for web frontend development, and it's a joy ...
Maybe they're great in specific circumstances? I dont know what the reason is but if FP was so intuitive then you'd expect it'd be the norm, or at least on a par, or even in the same ballpark as imperative?
Oct 07 2020
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 07.10.20 14:32, claptrap wrote:
 On Wednesday, 7 October 2020 at 11:55:00 UTC, Paolo Invernizzi wrote:
 On Wednesday, 7 October 2020 at 11:19:24 UTC, claptrap wrote:
 On Wednesday, 7 October 2020 at 09:04:59 UTC, Ola Fosheim Grøstad wrote:
 On Wednesday, 7 October 2020 at 08:49:21 UTC, Patrick Schluter wrote:
 On Wednesday, 7 October 2020 at 02:33:21 UTC, Andrei Alexandrescu 
 wrote:
Right again. Funtional programming is only pleasant in a dedicated FP language, and even then you need to memorize a large set of library constructs to be productive.
How many pure funtional languages are in the tiobe top 10? None or maybe 1 im not sure. Theres a reason for that i reckon.
We are using a pure functional language for web frontend development, and it's a joy ...
Maybe they're great in specific circumstances? I dont know what the reason is but if FP was so intuitive then you'd expect it'd be the norm, or at least on a par, or even in the same ballpark as imperative?
This particular discussion is not about functional vs imperative at all, it is about awkward vs decent.
Oct 07 2020
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/7/20 4:49 AM, Patrick Schluter wrote:
 On Wednesday, 7 October 2020 at 02:33:21 UTC, Andrei Alexandrescu wrote:
 On 10/6/20 9:07 PM, claptrap wrote:
 On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:
 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via Digitalmars-d 
 wrote: [...]

 I would write it like this:

     int[] vals = [4,7,28,23,585,73,12];

     int[] getMultiplesOf(int i)
     {
         return vals.filter!(v => (v % i) == 0).array;
     }

 One line vs. 4, even more concise. ;-)
The point is to show language not library.
That's a made-up restriction, and it's odd that it is being discussed here as a virtue.
No, it's not. It's central to the argument.
Then the argument is specious. I've been also tempted to do this on occasion to tilt a comparison one way or another - take C++ without STL or Boost, or Haskell without Prelude. The reality is these need to be considered together. (Make a hashtable in C++, no standard library allowed...)
 Beginners are attracted to large languages that have everything built 
 in. A good language is focused on general primitives that allow 
 writing a great deal in libraries.
Then do lisp or forth but not D or C++.
Much of the size of C++ and D caters to library writers, and their standard libraries grew way faster than the core language - as they should.
Oct 07 2020
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 7 October 2020 at 11:10:15 UTC, Andrei Alexandrescu 
wrote:
 I've been also tempted to do this on occasion to tilt a 
 comparison one way or another - take C++ without STL or Boost, 
 or Haskell without Prelude. The reality is these need to be 
 considered together. (Make a hashtable in C++, no standard
Your argument would work for Go, but not for C++. Lots of C++ codebases rely only on the most basic stuff like malloc and memcpy. boost is only useful for prototyping, you find better dedicated libraries. same for much of STL Cppcon has a 2 HOURS presentation explaning just type traits... That tells me that something went wrong somewhere... Not generic enough, too large vocubalary. Standard libs should be small and composable. Over 80% of C++ stdlib is useless or outdated. Of course they have like a 20+ years deprecation cycle...
Oct 07 2020
prev sibling parent reply claptrap <clap trap.com> writes:
On Wednesday, 7 October 2020 at 11:10:15 UTC, Andrei Alexandrescu 
wrote:
 On 10/7/20 4:49 AM, Patrick Schluter wrote:
 On Wednesday, 7 October 2020 at 02:33:21 UTC, Andrei 
 Alexandrescu wrote:
 On 10/6/20 9:07 PM, claptrap wrote:
 On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:
 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via 
 Digitalmars-d wrote: [...]

 I would write it like this:

     int[] vals = [4,7,28,23,585,73,12];

     int[] getMultiplesOf(int i)
     {
         return vals.filter!(v => (v % i) == 0).array;
     }

 One line vs. 4, even more concise. ;-)
The point is to show language not library.
That's a made-up restriction, and it's odd that it is being discussed here as a virtue.
No, it's not. It's central to the argument.
Then the argument is specious. I've been also tempted to do this on occasion to tilt a comparison one way or another - take C++ without STL or Boost, or Haskell without Prelude. The reality is these need to be considered together. (Make a hashtable in C++, no standard library allowed...)
If you're just asking is this easier in this language or that then it is unheplful to say no stdlib. But thats not whats going on here. You're comparing two language features within one language. The incubant feature has 20 years of library support behind it. The other does not. Lets reverse the roles, say TF were invented first, and somebody was arguing for templates to be added now. The TP version would be a couple of stdlib calls and the template version a whole page. Its a nonsense to make a comparison like that. Maybe your "temptation too tilt" is at play here but you havent realised it yet?
Oct 07 2020
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/7/20 7:58 AM, claptrap wrote:
 Lets reverse the roles, say TF were invented first, and somebody was 
 arguing for templates to be added now. The TP version would be a couple 
 of stdlib calls and the template version a whole page.
Incumbency is a huge matter in programming language design. Of course I would not propose another way of doing the same thing.
Oct 07 2020
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 7 October 2020 at 12:30:15 UTC, Andrei Alexandrescu 
wrote:
 On 10/7/20 7:58 AM, claptrap wrote:
 Lets reverse the roles, say TF were invented first, and 
 somebody was arguing for templates to be added now. The TP 
 version would be a couple of stdlib calls and the template 
 version a whole page.
Incumbency is a huge matter in programming language design. Of course I would not propose another way of doing the same thing.
I just looked up Incumbency, but the German translation was 'Time someone has been in office' ... I am not seeing how these things relate. Perhaps you can say this in different words?
Oct 07 2020
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 7 October 2020 at 12:37:48 UTC, Stefan Koch wrote:
 On Wednesday, 7 October 2020 at 12:30:15 UTC, Andrei 
 Alexandrescu wrote:
 Incumbency is a huge matter in programming language design. Of 
 course I would not propose another way of doing the same thing.
Perhaps you can say this in different words?
Basically we just prefer to work with what we have before adding new stuff to the language. So whatever features get added first have an automatic advantage in any comparison just because they are already there.
Oct 07 2020
parent reply Daniel K <dkm4i1 gmail.com> writes:
On Wednesday, 7 October 2020 at 12:56:42 UTC, Adam D. Ruppe wrote:
 On Wednesday, 7 October 2020 at 12:37:48 UTC, Stefan Koch wrote:
 On Wednesday, 7 October 2020 at 12:30:15 UTC, Andrei 
 Alexandrescu wrote:
 Incumbency is a huge matter in programming language design. 
 Of course I would not propose another way of doing the same 
 thing.
Perhaps you can say this in different words?
Basically we just prefer to work with what we have before adding new stuff to the language. So whatever features get added first have an automatic advantage in any comparison just because they are already there.
I agree. And existing features got added after already having their fundamental ideas, advantages, and deficiencies scrutinized. The fact that we have meta-programming in the form we have, is not by chance. It is a super strong and capable design. I am biased, but I would say "state of the art". If we boil down the Type Function proposal to its fundamental idea, advantages, and deficiencies. What Do we get? Fundamental idea: Language level reification of a set of types, making a set of types mutable to enable imperative programming style, for what is essentially only changing that set of types. But at that, only in the context of a special kind of CTFE-like function. Other than mutating a set of types, This function cant do anything that cant already be achieved rather simply with existing language features. As has been demonstrated a number of times in this thread. Advantage: Mutating a set of types in an imperative style. Something that is already quite simple, and requires virtually the same, or even less code in practice when written with existing language features. Disadvantage: A new feature people will have to learn and understand, document, maintain, bugfix. No actual substantial future gain. Is this wrong? Or where is that striking idea nobody can spot, that we just must enable with language level complexity? If the best answer is "Imperative type set manipulation"… Geeezzz… I would love to see an example, of the epitome of what goodness TypeFunctions could unlock. What I've seen so far is basically nothing, at the cost of far more than nothing. No Deal. /Daniel K
Oct 07 2020
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 7 October 2020 at 14:01:51 UTC, Daniel K wrote:
 The fact that we have meta-programming in the form we have, is 
 not by chance. It is a super strong and capable design.
 I am biased, but I would say "state of the art".
State of the art requires rewrite-capabilities like Lisp, Pure and many other languages have. C++ chose simple templating, but got burned as users abused it to do more. That aspect of C++ was an unintended design flaw! D chose the same course, but through a deliberate design. So they could have a more readable syntax (the semantics differ too, but both have advantages/disadvantages).
Oct 07 2020
prev sibling next sibling parent Paul Backus <snarwin gmail.com> writes:
On Wednesday, 7 October 2020 at 14:01:51 UTC, Daniel K wrote:
 If we boil down the Type Function proposal to its fundamental 
 idea, advantages, and deficiencies. What Do we get?

 [...]

 Advantage:
 Mutating a set of types in an imperative style. Something that 
 is already quite simple, and requires virtually the same, or 
 even less code in practice when written with existing language 
 features.

 Disadvantage: A new feature people will have to learn and 
 understand, document, maintain, bugfix.
 No actual substantial future gain.

 Is this wrong?
The big advantage you're missing is that, because type functions are so restricted in what they can do, they can be implemented much more efficiently in the compiler. For large programs that make heavy use of D's metaprogramming features, that can really add up.
Oct 07 2020
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 7 October 2020 at 14:01:51 UTC, Daniel K wrote:
 On Wednesday, 7 October 2020 at 12:56:42 UTC, Adam D. Ruppe 
 wrote:
 On Wednesday, 7 October 2020 at 12:37:48 UTC, Stefan Koch 
 wrote:
 On Wednesday, 7 October 2020 at 12:30:15 UTC, Andrei 
 Alexandrescu wrote:
 Incumbency is a huge matter in programming language design. 
 Of course I would not propose another way of doing the same 
 thing.
Perhaps you can say this in different words?
Basically we just prefer to work with what we have before adding new stuff to the language. So whatever features get added first have an automatic advantage in any comparison just because they are already there.
I agree. And existing features got added after already having their fundamental ideas, advantages, and deficiencies scrutinized. The fact that we have meta-programming in the form we have, is not by chance. It is a super strong and capable design. I am biased, but I would say "state of the art". If we boil down the Type Function proposal to its fundamental idea, advantages, and deficiencies. What Do we get? Fundamental idea: Language level reification of a set of types, making a set of types mutable to enable imperative programming style, for what is essentially only changing that set of types. But at that, only in the context of a special kind of CTFE-like function.
This is quite close, thanks for your description. The only thing I would change is "CTFE-like" to CTFE. The language changes are as follows: - Introduce a new basic type "alias" which is the type of types. This type does not have a runtime representation, and can only exist at compile time/CTFE. - Variables of this type emulate the interface that Type parameters in templates have. - Add an implicit conversion such that any type implicitly converts to alias. - Allow functions to be called with Types (TypeExps are a concept which already exists in the compiler but not in the language) - Functions which take alias parameters or types drives of alias parameters or (such as alias[], or structs which contain alias fields) or return such values cannot be called at runtime, and will never be generated in the binary (this is because alias does not have an ABI representation but helps out with compile speed as well). - Lastly (and this one is actually one I am not quite happy with) but it's needed is the alias[] -> TypeTuple via applying .tupleof to a statically determined (compile-time constant) alias[]. The CTFE functionality is 100% reused. (which also means when newCTFE hits complex type functions get a speed boost) The alias type is integrated into the type-system except for the special handling of alias[].tupleof it behaves as expected. For example you can do use types as keys for associative arrays if you wish to. Which then allows more freedom in algorithm design in the domain of introspection/ compile time code-generation in particular.
 Disadvantage: A new feature people will have to learn and 
 understand, document, maintain, bugfix.
 No actual substantial future gain.
As for the disadvantages. Yes it's a new feature but it blends so well with the current language that people on here post correct type function code, (presumably) without having a compiler that is capable of running it. (It's of course possible that claptrap cloned my branch and run the code, but I don't think so ...) As for the feature gain ... well it's akin to comparing c++ templates with D templates. They are equivalent in power, just that one easier to write than the other. Daniel K: I am thankful for your analysis, would you be opposed if I reused your characterization in the DIP?
Oct 07 2020
next sibling parent reply claptrap <clap trap.com> writes:
On Wednesday, 7 October 2020 at 14:40:41 UTC, Stefan Koch wrote:
 On Wednesday, 7 October 2020 at 14:01:51 UTC, Daniel K wrote:
 On Wednesday, 7 October 2020 at 12:56:42 UTC, Adam D. Ruppe 
 wrote:
 On Wednesday, 7 October 2020 at 12:37:48 UTC, Stefan Koch 
 wrote:
 On Wednesday, 7 October 2020 at
 As for the disadvantages. Yes it's a new feature but it blends 
 so well with the current language that people on here post 
 correct type function code, (presumably) without having a 
 compiler that is capable of running it. (It's of course 
 possible that claptrap cloned my branch and run the code, but I 
 don't think so ...)
I just based it off what youve posted in this thread. Maybe my perspective as a relative newbie to D makes me see that as a much bigger payoff than some of the resident greybeards.
Oct 07 2020
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 7 October 2020 at 14:59:04 UTC, claptrap wrote:
 On Wednesday, 7 October 2020 at 14:40:41 UTC, Stefan Koch wrote:
 On Wednesday, 7 October 2020 at 14:01:51 UTC, Daniel K wrote:
 On Wednesday, 7 October 2020 at 12:56:42 UTC, Adam D. Ruppe 
 wrote:
 On Wednesday, 7 October 2020 at 12:37:48 UTC, Stefan Koch 
 wrote:
 On Wednesday, 7 October 2020 at
 As for the disadvantages. Yes it's a new feature but it blends 
 so well with the current language that people on here post 
 correct type function code, (presumably) without having a 
 compiler that is capable of running it. (It's of course 
 possible that claptrap cloned my branch and run the code, but 
 I don't think so ...)
I just based it off what youve posted in this thread. Maybe my perspective as a relative newbie to D makes me see that as a much bigger payoff than some of the resident greybeards.
You might be happy to see that this exmaple just works with talias_master: --- alias type = alias; struct KV { string key; type value; } auto makeAA(KV[] kvs ...) { type[string] result; foreach(kv;kvs) { result[kv.key] = kv.value; } return result; } pragma(msg, makeAA(KV("u32", uint), KV("kv", KV))); // outputs: ["u32":(uint), "kv":(KV)] --
Oct 07 2020
prev sibling parent reply Daniel K <dkm4i1 gmail.com> writes:
On Wednesday, 7 October 2020 at 14:40:41 UTC, Stefan Koch wrote:
  Daniel K: I am thankful for your analysis, would you be 
 opposed if I reused your characterization in the DIP?
You are free to use any part of it, if it doesn't reference back to me or "the community". But if you use it to insinuate endorsement from me or "the community", you will have to include all of the opposing points. No cherry picking. /Daniel K
Oct 07 2020
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 7 October 2020 at 15:00:16 UTC, Daniel K wrote:
 On Wednesday, 7 October 2020 at 14:40:41 UTC, Stefan Koch wrote:
  Daniel K: I am thankful for your analysis, would you be 
 opposed if I reused your characterization in the DIP?
You are free to use any part of it, if it doesn't reference back to me or "the community". But if you use it to insinuate endorsement from me or "the community", you will have to include all of the opposing points. No cherry picking. /Daniel K
That is fair. Thanks!
Oct 07 2020
prev sibling parent claptrap <clap trap.com> writes:
On Wednesday, 7 October 2020 at 12:37:48 UTC, Stefan Koch wrote:
 On Wednesday, 7 October 2020 at 12:30:15 UTC, Andrei 
 Alexandrescu wrote:
 On 10/7/20 7:58 AM, claptrap wrote:
 Lets reverse the roles, say TF were invented first, and 
 somebody was arguing for templates to be added now. The TP 
 version would be a couple of stdlib calls and the template 
 version a whole page.
Incumbency is a huge matter in programming language design. Of course I would not propose another way of doing the same thing.
I just looked up Incumbency, but the German translation was 'Time someone has been in office' ... I am not seeing how these things relate. Perhaps you can say this in different words?
incumbent = the established order. It's usually used in the context of a politician in office who defending his position rather than challenging for it. IE. Trump is the incumbent, Biden is the challenger.
Oct 07 2020
prev sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 7 October 2020 at 12:30:15 UTC, Andrei Alexandrescu 
wrote:
 On 10/7/20 7:58 AM, claptrap wrote:
 Lets reverse the roles, say TF were invented first, and 
 somebody was arguing for templates to be added now. The TP 
 version would be a couple of stdlib calls and the template 
 version a whole page.
Incumbency is a huge matter in programming language design. Of course I would not propose another way of doing the same thing.
c++ concepts is another way of doing the same thing. So you are basically saying they should not have done it? c++ modules is another way of doing the same thing. Etc. C++ concepts is a prettier and more useful version with compile traits qualities. Probably the most important improvement since RAII... Nobody that values their time wants to juggle types in c++. D isnt on a different plane either. Improving usability is a survivalstrategy for any language... Haskell is failing for that reason, it is not going to improve. It is primarily a research test bed, just like ML and descendants. Academic languages evolve by death and rebirth, not by additions. So if you want D3, ok. But if you want D2, then you cannot pick the path that academics follow.
Oct 07 2020
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 7 October 2020 at 11:58:35 UTC, claptrap wrote:
 Lets reverse the roles, say TF were invented first, and 
 somebody was arguing for templates to be added now. The TP 
 version would be a couple of stdlib calls and the template 
 version a whole page.
Templates actually enable all kinds of new things. A type function cannot actually generate any new code nor create new types. Those are explicit limitations in the design right now, because by keeping them so strict it enables some optimizations that are... at best tricky with templates (many attempts to optimize templates end up causing compiler bug errors or, perhaps worse yet, random linker errors). So supposed we lived in a world with only TF. You can make type lists and return basic type constants based on limited reflection over type (note that TFs will probably *not* support advanced `is` expression pattern matching as it is right now!) Then someone comes along and says let's add templates. Templates can do everything a type function can do AND you can generate new code based on that information! No more awkwardly returning string literals and mixing them in at the usage point, the compiler will generate it all directly! Oh there'd probably be pushback, I'd probably be arguing "but we can already mixin!" myself :) (though mixin sucks for name generation, you have to use __LINE__ or cookie string hacks, whereas templates automatically get unique names!) But it'd also be a compelling new feature. That's why I'd be a bit more excited about the type functions if they prove capable of doing things we can't really do right now, like transform function argument lists through tuple manipulation. That'd be a cool application of the same general principle of reordering alias[] entities.
Oct 07 2020
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmaill.com> writes:
On Wednesday, 7 October 2020 at 12:54:09 UTC, Adam D. Ruppe wrote:
 That's why I'd be a bit more excited about the type functions 
 if they prove capable of doing things we can't really do right 
 now, like transform function argument lists through tuple 
 manipulation. That'd be a cool application of the same general 
 principle of reordering alias[] entities.
I think you should just add type variables and improve on other mechanisms until you have those capabilities, the you restrict the usage of type variables to what Stefan can implement so that this feature gets a release date. After that it can be extended one step at the time, maybe even to runtime. The whole has to be specced out before release, but the initial release can be constrained to a subset.
Oct 07 2020
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 07.10.20 04:33, Andrei Alexandrescu wrote:
 On 10/6/20 9:07 PM, claptrap wrote:
 On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:
 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via Digitalmars-d 
 wrote: [...]

 I would write it like this:

     int[] vals = [4,7,28,23,585,73,12];

     int[] getMultiplesOf(int i)
     {
         return vals.filter!(v => (v % i) == 0).array;
     }

 One line vs. 4, even more concise. ;-)
The point is to show language not library.
That's a made-up restriction, and it's odd that it is being discussed here as a virtue. ...
I agree. This is a bad way to market a new language feature. However, I don't think "library over language" is necessarily a good justification for an inefficient implementation of core language features.
 Beginners are attracted to large languages that have everything built in.
Isn't it rather that beginners can't tell the difference?
 A good language is focused on general primitives that allow writing 
 a great deal in libraries.
 
 
Certainly, but a good language does not require you to write the same functionality multiple times.
Oct 07 2020
prev sibling parent foobar <foo bar.com> writes:
On Wednesday, 7 October 2020 at 01:07:17 UTC, claptrap wrote:
 On Tuesday, 6 October 2020 at 23:39:24 UTC, H. S. Teoh wrote:
 On Tue, Oct 06, 2020 at 11:16:47PM +0000, claptrap via 
 Digitalmars-d wrote: [...]

 I would write it like this:

 	int[] vals = [4,7,28,23,585,73,12];

 	int[] getMultiplesOf(int i)
 	{
 	    return vals.filter!(v => (v % i) == 0).array;
 	}

 One line vs. 4, even more concise. ;-)
The point is to show language not library.
Why?
Oct 06 2020
prev sibling parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Tuesday, 6 October 2020 at 23:16:47 UTC, claptrap wrote:
[...]
 Its the same basic algorithm, search a list and make a new list 
 from the entries that satisfy a certain condition. It beggars 
 belief that anyone would argue that the second version is not 
 better on every metric that is important in writing good code. 
 It's clearer, more intuitive, more concise, it will almost 
 definitely be faster and less wasteful of resources. Newbies 
 will grep it far quicker than the other versions. That means 
 faster to write and easier to maintain and debug.

 I'm honestly losing the will to live here.
Thank you. I was myself wondering why this feature was encountering such an amount of resistance, especially from the top tier of the language specialists. Maybe they are so used to the complexity of the template language that they don't perceive how challenging for the mere mortal it is in practice.
Oct 07 2020
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 07.10.20 10:25, Patrick Schluter wrote:
 I'm honestly losing the will to live here.
Thank you. I was myself wondering why this feature was encountering such an amount of resistance, especially from the top tier of the language specialists.
Fear of losing the edge? :o)
Oct 07 2020