www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Perfect forwarding

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
This topic came about during beerconf (it was fun!): Write an idiomatic 
template `forward` that takes an alias `fun` and defines (generates) one 
overload for each overload of `fun`. Example:

template forward(alias fun)
{
     ...
}

Now for this function:

void myfun(int, ref double, out string);
int myfun(in string, inout double);

the instantiation would be (stylized):

template forward!myfun
{
     void myfun(int a, ref double b, out string c)
     {
         return myfun(a, b, c);
     }
     int myfun(in string a, inout double b)
     {
         return myfun(a, b);
     }
}

So the challenge is implementing forward() to do this.
Jul 25 2020
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 26 July 2020 at 01:29:21 UTC, Andrei Alexandrescu 
wrote:
 So the challenge is implementing forward() to do this.
I know the trick to do this, but I won't post the answer yet because I don't want to ruin it for others who want to try. However I'm going to point out that my trick doesn't work if you are trying to forward a template, since template params cannot be introspected. As far as I know, there is no solution to that. My impl also does a bonus: forwarding UDAs as well. That requires very new dmd though because until very recently that would hit a compiler bug! But it is possible. Note: part of the challenge is making sure it works with user-defined types too. Your first thought of an implementation might not, so test that with a struct from an import when there's a local struct with the same name....
Jul 25 2020
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/25/20 9:44 PM, Adam D. Ruppe wrote:
 However I'm going to point out that my trick doesn't work if you are 
 trying to forward a template, since template params cannot be 
 introspected. As far as I know, there is no solution to that.
That needs fixed. What primitives would you need for that?
Jul 25 2020
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 26 July 2020 at 02:10:19 UTC, Andrei Alexandrescu 
wrote:
 That needs fixed. What primitives would you need for that?
Hmm, hard to say, and I'm also hesitant because it makes template optimization even harder - the more of the innards we expose, the less the compiler is able to discard, even in theory. (We probably will need some kind of specifically restricted template at some point as a practical optimization matter. Stefan's not wrong about the complications here.) But anyway, I think we'd have to see at a minimum a list of template parameters. Constraint might be useful too, but that can be at least tested via __traits(compiles) once you have the template parameter list. The tricky thing is figuring out how to represent them in the language. Regular function parameters are not representable either... you probably know how awkward it is to get function default arguments. (Oh, btw, those of you doing the OP challenge, remember default arguments should be forwarded too!) You have to make a helper function slicing the params and return the value since D doesn't really represent this data, it is more like an opaque object you can just use in certain contexts. But at least with ordinary parameters, you can drill down with typeof(param). That would work for template value parameters, but what is T? Well, is(T) might help there. static if(is(Param)) { /* template type param */ } else static if(is(typeof(Param) Type)) { /* template value param */ } The big remaining question is an alias param. If we had some kind __traits(isAlias) that could be used here as well as in other places of reflection. Good idea regardless imo. Then you need to be able ot get the alias name itself, not just the identifier of what it is an alias of. I think __traits(identifier) and/or .stringof can do this now but unable to check as I type this email. But should make that formally possible - remember a template alias param will not actually be an alias of anything yet! Ditto on T, it is a speculative type, not an actual type. So regular reflection might not quite work, just it is as close as we can get. And then T... params. That probably needs a specific detection too, could be like __traits(isAliasSeq) or something. So to sum up before my laptop dies: * a way to get the list of template parameters * a way to identify the different kinds of template params. Some overlap with function parameters is possible but it will be tricky because template(T) is not actually a type. But if the compiler made it a placeholder type for this purpose then we can reuse some of the machinery. * alias and tuple params could use specific attention, which can be written in a general-purpose manner. Then I think we can make this happen. One of the guys at work has hit this trouble before too, perhaps on Monday we can loop him in and get a second opinion to see if I missed anything.
Jul 25 2020
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
Oops, I forgot about default arguments and specializations.

Default arguments are relatively straightforward, can do it at 
least as hacky as the runtime function variety.

But specializations are tricky to represent. The closest thing we 
have in the normal language is the `is` expression... but it 
isn't exactly the same... still if it could be like an is lambda 
it might work...

To be honest I suspect the best way to do this at this point 
would be to make it part of the opaque slice type. So if you 
define a new template with it, it inherits that but otherwise you 
can't really look in. I'm not satisfied with that but it might be 
the most practical thing to do right now and can be revisited in 
the future once the basic functionality actually works.
Jul 25 2020
parent reply Mathias LANG <geod24 gmail.com> writes:
On Sunday, 26 July 2020 at 03:59:00 UTC, Adam D. Ruppe wrote:
 Oops, I forgot about default arguments and specializations.

 Default arguments are relatively straightforward, can do it at 
 least as hacky as the runtime function variety.
How do you forward this: ``` class Foo { void set (time_t value = time()) {} } ```
Jul 26 2020
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 26 July 2020 at 10:51:44 UTC, Mathias LANG wrote:
 How do you forward this:
 ```
 class Foo
 {
     void set (time_t value = time()) {}
 }
 ```
The same as the others wrt the default argument. The fact it is in a class complicates things a little since `this` is not propagated through... but if I remove it from there it still works. Here's my solution as a link so people can look at it (and prolly destroy holes i missed lol) if you want but not click if you still wanna play with the challenge. http://arsdnet.net/dcode/forward.d The 8 lines at top are the forward, then the rest are the test conditions. BTW I'd note that just doing a speculative forward is a bit simpler than this - you can just do (Args...)(auto ref Args args) - but the linked technique does it ahead-of-time, as if you wrote it by hand, meaning reflection still works on the forwarded definitions too.
Jul 26 2020
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2020-07-26 04:10, Andrei Alexandrescu wrote:

 That needs fixed. What primitives would you need for that?
It has already been implemented but rejected [1]. Andrei, you were part of the discussions and approved the changes. [1] https://github.com/dlang/dmd/pull/5201 -- /Jacob Carlborg
Jul 26 2020
parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Sunday, 26 July 2020 at 11:56:16 UTC, Jacob Carlborg wrote:
 On 2020-07-26 04:10, Andrei Alexandrescu wrote:

 That needs fixed. What primitives would you need for that?
It has already been implemented but rejected [1]. Andrei, you were part of the discussions and approved the changes. [1] https://github.com/dlang/dmd/pull/5201
Ah, this was in the back of my head when someone mentioned template parameters. I haven't looked at that in a long time.
Jul 26 2020
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.com> writes:
On 7/26/20 8:19 AM, Andrej Mitrovic wrote:
 On Sunday, 26 July 2020 at 11:56:16 UTC, Jacob Carlborg wrote:
 On 2020-07-26 04:10, Andrei Alexandrescu wrote:

 That needs fixed. What primitives would you need for that?
It has already been implemented but rejected [1]. Andrei, you were part of the discussions and approved the changes. [1] https://github.com/dlang/dmd/pull/5201
Ah, this was in the back of my head when someone mentioned template parameters. I haven't looked at that in a long time.
Well... would you like to revive it?
Jul 27 2020
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Monday, 27 July 2020 at 14:08:30 UTC, Andrei Alexandrescu 
wrote:
 On 7/26/20 8:19 AM, Andrej Mitrovic wrote:
 On Sunday, 26 July 2020 at 11:56:16 UTC, Jacob Carlborg wrote:
 On 2020-07-26 04:10, Andrei Alexandrescu wrote:

 That needs fixed. What primitives would you need for that?
It has already been implemented but rejected [1]. Andrei, you were part of the discussions and approved the changes. [1] https://github.com/dlang/dmd/pull/5201
Ah, this was in the back of my head when someone mentioned template parameters. I haven't looked at that in a long time.
Well... would you like to revive it?
One question that was brought up recently by an unrelated DMD PR [1] was whether new traits require a DIP. None of the traits we added in the past few years have gone through the DIP process as far as I remember. Of course, going through the DIP process can only improve the quality, but on the other hand, one could argue that it would make things unnecessary hard for easy additions like isDeprecated [2] and isDisabled [3]. How high should the bar be? Should we decide on a case-by-case basis? [1]: https://github.com/dlang/dmd/pull/11442#issuecomment-661996482 [2]: https://github.com/dlang/dmd/pull/7178 [3]: https://github.com/dlang/dmd/pull/7569
Jul 27 2020
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.com> writes:
On 7/27/20 11:57 AM, Petar Kirov [ZombineDev] wrote:
 On Monday, 27 July 2020 at 14:08:30 UTC, Andrei Alexandrescu wrote:
 On 7/26/20 8:19 AM, Andrej Mitrovic wrote:
 On Sunday, 26 July 2020 at 11:56:16 UTC, Jacob Carlborg wrote:
 On 2020-07-26 04:10, Andrei Alexandrescu wrote:

 That needs fixed. What primitives would you need for that?
It has already been implemented but rejected [1]. Andrei, you were part of the discussions and approved the changes. [1] https://github.com/dlang/dmd/pull/5201
Ah, this was in the back of my head when someone mentioned template parameters. I haven't looked at that in a long time.
Well... would you like to revive it?
One question that was brought up recently by an unrelated DMD PR [1] was whether new traits require a DIP. None of the traits we added in the past few years have gone through the DIP process as far as I remember. Of course, going through the DIP process can only improve the quality, but on the other hand, one could argue that it would make things unnecessary hard for easy additions like isDeprecated [2] and isDisabled [3]. How high should the bar be? Should we decide on a case-by-case basis? [1]: https://github.com/dlang/dmd/pull/11442#issuecomment-661996482 [2]: https://github.com/dlang/dmd/pull/7178 [3]: https://github.com/dlang/dmd/pull/7569
Thanks for asking. I'm no longer in the decision loop but here's what I think. There'd be somewhat obvious traits to have ("is this symbol deprecated?") and traits that work together toward a larger goal, such as perfect forwarding or general introspection itself. In the former case acceptance could and should be somewhat quick, whereas in the latter case we risk having a hodge-podge of traits that don't combine properly to attain the initial goal. In a way we're there right now - we added traits on a need basis and we're unclear on what the limits of what we can and what we can't do, or how to do it.
Jul 27 2020
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Monday, 27 July 2020 at 21:08:52 UTC, Andrei Alexandrescu 
wrote:
 [..]

 Thanks for asking. I'm no longer in the decision loop but 
 here's what I think. There'd be somewhat obvious traits to have 
 ("is this symbol deprecated?") and traits that work together 
 toward a larger goal, such as perfect forwarding or general 
 introspection itself. In the former case acceptance could and 
 should be somewhat quick, whereas in the latter case we risk 
 having a hodge-podge of traits that don't combine properly to 
 attain the initial goal. In a way we're there right now - we 
 added traits on a need basis and we're unclear on what the 
 limits of what we can and what we can't do, or how to do it.
So the question is what do we do about the new traits proposed by [1]? How formal does the process needs to be about them? I see 3 options: 1. Traits or otherwise any language features part of a larger story (e.g. template parameter introspection, perfect forwarding) need to go through a 2 step DIP process: 1) A strategic/vision DIP that states the high-level problem that we want to solve. The goal of this DIP would be form community and leadership consensus on whether something is a strategic goal for D and whether we need to solve it. 2) A tactical DIP that proposes concrete language features and defines their semantics. In this case these would be the traits proposed by [1]. 2. The language maintainers (Atila and Walter) decide on each DMD PR on a case-by-case basis. 3. New traits that haven't gone through a DIP process are considered experimental and can be added behind a compiler switch (which either enables all experimental traits, or each one individually). Such traits continue to have experimental status for several releases until we gather real-world experience and we decide to either remove, change, or to stabilize them (make them generally part of the language and available without any switch). Unlike 2. any dmd contributor (or perhaps a group of minimum of 2/3 people?) with write access to the dmd repo can decide to merge such experimental traits, after some minimal requirements are met, such as a changelog entry and sufficiently thorough test(s) that demonstrate the usefulness of the feature and cover all knownedge edge cases (e.g. each added error message is tested). Andrei Atila Walter what do you guys think? [1]: https://github.com/dlang/dmd/pull/5201
Jul 27 2020
next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Tuesday, 28 July 2020 at 05:34:59 UTC, Petar Kirov 
[ZombineDev] wrote:
  Andrei  Atila  Walter what do you guys think?

 [1]: https://github.com/dlang/dmd/pull/5201
The community (and maintainers) need to settle on the desired set of traits, and then we can think about how to best implement it. I'm sure the PR is very outdated by now, there's probably better ways of implementing it.
Jul 28 2020
prev sibling parent Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 28 July 2020 at 05:34:59 UTC, Petar Kirov 
[ZombineDev] wrote:
 On Monday, 27 July 2020 at 21:08:52 UTC, Andrei Alexandrescu 
 wrote:
 [...]
So the question is what do we do about the new traits proposed by [1]? How formal does the process needs to be about them? I see 3 options: 1. Traits or otherwise any language features part of a larger story (e.g. template parameter introspection, perfect forwarding) need to go through a 2 step DIP process: 1) A strategic/vision DIP that states the high-level problem that we want to solve. The goal of this DIP would be form community and leadership consensus on whether something is a strategic goal for D and whether we need to solve it. 2) A tactical DIP that proposes concrete language features and defines their semantics. In this case these would be the traits proposed by [1]. [...]
I'd say new traits would be case-by-case. My gut feeling is that they're too small for a DIP, but I also understand why you'd ask if there would have to be one first.
Aug 24 2020
prev sibling parent Jean-Louis Leroy <jl leroy.nyc> writes:
On Sunday, 26 July 2020 at 02:10:19 UTC, Andrei Alexandrescu 
wrote:
 On 7/25/20 9:44 PM, Adam D. Ruppe wrote:
 However I'm going to point out that my trick doesn't work if 
 you are trying to forward a template, since template params 
 cannot be introspected. As far as I know, there is no solution 
 to that.
That needs fixed. What primitives would you need for that?
I need it to support open method templates.
Jul 26 2020
prev sibling next sibling parent Jean-Louis Leroy <jl leroy.nyc> writes:
On Sunday, 26 July 2020 at 01:29:21 UTC, Andrei Alexandrescu 
wrote:
 This topic came about during beerconf (it was fun!): Write an 
 idiomatic template `forward` that takes an alias `fun` and 
 defines (generates) one overload for each overload of `fun`. 
 Example:

 template forward(alias fun)
 {
     ...
 }

 Now for this function:

 void myfun(int, ref double, out string);
 int myfun(in string, inout double);

 the instantiation would be (stylized):

 template forward!myfun
 {
     void myfun(int a, ref double b, out string c)
     {
         return myfun(a, b, c);
     }
     int myfun(in string a, inout double b)
     {
         return myfun(a, b);
     }
 }

 So the challenge is implementing forward() to do this.
Using bolts' experimental refraction module: module challenge; import std.format; import bolts.experimental.refraction; template forward(alias fun) { enum name = __traits(identifier, fun); static foreach (ovl; __traits(getOverloads, __traits(parent, fun), name)) { mixin( { enum original = refract!(ovl, "ovl"); return original.withBody(q{{ return %s(%s); }}.format(original.name, original.argumentMixture)) .mixture; }()); } } void myfun(int, ref double, out string); int myfun(in string, inout double); pragma(msg, typeof(__traits(getOverloads, forward!myfun, "myfun")[0])); pragma(msg, typeof(__traits(getOverloads, forward!myfun, "myfun")[1])); Output: system void(int _0, ref double _1, out string _2) system int(const(string) _0, inout(double) _1) Okay the "in" is missing in front of the first argument of the second overload, but it doesn't seem to be there in the first place: pragma(msg, typeof(__traits(getOverloads, challenge, "myfun")[1])); Output: int(const(string), inout(double)) Also the body of the forwarders should probably use `std.functional.forward`.
Jul 26 2020
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On Sun, Jul 26, 2020 at 11:30 AM Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 This topic came about during beerconf (it was fun!): Write an idiomatic
 template `forward` that takes an alias `fun` and defines (generates) one
 overload for each overload of `fun`. Example:

 template forward(alias fun)
 {
      ...
 }

 Now for this function:

 void myfun(int, ref double, out string);
 int myfun(in string, inout double);

 the instantiation would be (stylized):

 template forward!myfun
 {
      void myfun(int a, ref double b, out string c)
      {
          return myfun(a, b, c);
      }
      int myfun(in string a, inout double b)
      {
          return myfun(a, b);
      }
 }

 So the challenge is implementing forward() to do this.
As someone who's been doing exactly this repeatedly basically-forever, I can say the exercise is absolutely no fun at all. It is my opinion that if the solution involves a text-mixin, the author gets an instant FAIL. The major hangup in this exercise is dealing with 'storage class', which is impossible because it's not part of the language, and instantly forces synthesising strings. Forwarding the default args is tricky; and I've never managed to produce a uniform solution that doesn't suffer some edge cases, but I've always made it work for the specific set of things I'm wrapping. Template arg forwarding is another level above, but I think that should be taken as a secondary matter. Solve for normal functions first, and then maybe there's a hope.
Jul 28 2020
parent reply Jean-Louis Leroy <jl leroy.nyc> writes:
On Tuesday, 28 July 2020 at 22:48:16 UTC, Manu wrote:
 It is my opinion that if the solution involves a text-mixin, 
 the author gets an instant FAIL.
But:
 The major hangup in this exercise is dealing with 'storage 
 class', which is
 impossible because it's not part of the language, and instantly 
 forces synthesising strings.
Guaranteed failure then? ;-)
Jul 28 2020
parent reply Manu <turkeyman gmail.com> writes:
On Wed, Jul 29, 2020 at 9:40 AM Jean-Louis Leroy via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Tuesday, 28 July 2020 at 22:48:16 UTC, Manu wrote:
 It is my opinion that if the solution involves a text-mixin,
 the author gets an instant FAIL.
But:
 The major hangup in this exercise is dealing with 'storage
 class', which is
 impossible because it's not part of the language, and instantly
 forces synthesising strings.
Guaranteed failure then? ;-)
Yes.
Jul 29 2020
next sibling parent reply Jean-Louis Leroy <jl leroy.nyc> writes:
On Wednesday, 29 July 2020 at 10:38:29 UTC, Manu wrote:
 On Wed, Jul 29, 2020 at 9:40 AM Jean-Louis Leroy via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 On Tuesday, 28 July 2020 at 22:48:16 UTC, Manu wrote:
 It is my opinion that if the solution involves a text-mixin, 
 the author gets an instant FAIL.
But:
 The major hangup in this exercise is dealing with 'storage
 class', which is
 impossible because it's not part of the language, and 
 instantly
 forces synthesising strings.
Guaranteed failure then? ;-)
Yes.
This is a problem I spent a lot of time on while trying to support all the variations of functions in openmethods (and the problem is even harder because I need to alter the storage classes of a subset of the parameters). Doesn't this cut it? module challenge; template forward(alias fun) { import std.format; import std.traits; enum name = __traits(identifier, fun); static foreach (ovl; __traits(getOverloads, __traits(parent, fun), name)) { mixin(q{ auto ref %s(Parameters!ovl args) { return __traits(parent, fun).%s(args); } }.format(name, name)); } } void myfun(int, ref double, out string); int myfun(in string, inout double); pragma(msg, typeof(__traits(getOverloads, forward!myfun, "myfun")[0])); pragma(msg, typeof(__traits(getOverloads, forward!myfun, "myfun")[1])); Output: void(int, ref double, out string) int(const(string), inout(double)) This deals with storage classes all right. As for the string mixin, it would be nice to be able to do with it, or at least to narrow it to the function name, but in this respect D is more C than Lisp.
Jul 29 2020
parent reply Manu <turkeyman gmail.com> writes:
On Wed, Jul 29, 2020 at 10:35 PM Jean-Louis Leroy via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Wednesday, 29 July 2020 at 10:38:29 UTC, Manu wrote:
 On Wed, Jul 29, 2020 at 9:40 AM Jean-Louis Leroy via
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 On Tuesday, 28 July 2020 at 22:48:16 UTC, Manu wrote:
 It is my opinion that if the solution involves a text-mixin,
 the author gets an instant FAIL.
But:
 The major hangup in this exercise is dealing with 'storage
 class', which is
 impossible because it's not part of the language, and
 instantly
 forces synthesising strings.
Guaranteed failure then? ;-)
Yes.
This is a problem I spent a lot of time on while trying to support all the variations of functions in openmethods (and the problem is even harder because I need to alter the storage classes of a subset of the parameters). Doesn't this cut it? module challenge; template forward(alias fun) { import std.format; import std.traits; enum name = __traits(identifier, fun); static foreach (ovl; __traits(getOverloads, __traits(parent, fun), name)) { mixin(q{ auto ref %s(Parameters!ovl args) { return __traits(parent, fun).%s(args); } }.format(name, name)); } } void myfun(int, ref double, out string); int myfun(in string, inout double); pragma(msg, typeof(__traits(getOverloads, forward!myfun, "myfun")[0])); pragma(msg, typeof(__traits(getOverloads, forward!myfun, "myfun")[1])); Output: void(int, ref double, out string) int(const(string), inout(double)) This deals with storage classes all right. As for the string mixin, it would be nice to be able to do with it, or at least to narrow it to the function name, but in this respect D is more C than Lisp.
I mean... std.format in a forward macro? I find that embarrassing. Compile times are important. Of course, the usual casualties occur; debugging steps through string mixins, etc. Also, your solution doesn't perform any interesting transformation on args... forwarding functions usually transform some argument(/s) in some way, or inject additional arguments. Hard and really lame to write any of the actual meat of the exercise inside a string. I'm not saying your solution is bad, just that the problem is bad, and it really sucks that this thread exists. I've been complaining about this since day-one. 'Storage class' is the problem as usual.
Jul 29 2020
parent Jean-Louis Leroy <jl leroy.nyc> writes:
On Thursday, 30 July 2020 at 01:31:59 UTC, Manu wrote:

 Also, your solution doesn't perform any interesting 
 transformation on
 args... forwarding functions usually transform some 
 argument(/s) in some
 way, or inject additional arguments.
I struggled with that in openmethods where transformations do occur, and that's why I created this thingy: https://github.com/aliak00/bolts/blob/master/source/bolts/experi ental/refraction.d, which allows you to create a function from a function, while manipulating its "aspects" in any way you want. But yeah it's just a saner way of building a string mixin, and annoyingly, they are almost inevitable...just creating a function with anything but a fixed name requires a string mixin afaik.. Andrei's challenge and Adam's variation are among the simplest cases because the storage classes and function attributes can be inferred from the wrapped function. std.typecons.wrap, on the other hand, has to mess with function attributes.
Jul 29 2020
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 29 July 2020 at 10:38:29 UTC, Manu wrote:
 On Wed, Jul 29, 2020 at 9:40 AM Jean-Louis Leroy via
 Guaranteed failure then? ;-)
Yes.
What's wrong with my solution earlier in the thread?
Jul 29 2020
next sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Wednesday, 29 July 2020 at 12:54:36 UTC, Adam D. Ruppe wrote:
 On Wednesday, 29 July 2020 at 10:38:29 UTC, Manu wrote:
 On Wed, Jul 29, 2020 at 9:40 AM Jean-Louis Leroy via
 Guaranteed failure then? ;-)
Yes.
What's wrong with my solution earlier in the thread?
That it uses a string mixin :P What Manu is arguing is that if parameter storage classes were instead proper type qualifiers, then one could trivially manipulate them with std.meta. And then supposedly there would be no need to string mixins at all. I haven't spent the time to actually verify if that's the case here (perhaps the only need for using a string mixin is to generate variable names like Param1, Param2, ... ParamN) as is(X == __parameters) solves the issue with forwarding the whole parameter list (including storage classes), but in general, this doesn't scale if you need to do type manipulation with std.meta, as `alias RefInt = ref int` drops the `ref` storage class and this breaks everything like staticMap, Filter, etc.
Jul 29 2020
next sibling parent reply Jean-Louis Leroy <jl leroy.nyc> writes:
On Wednesday, 29 July 2020 at 16:11:02 UTC, Petar Kirov 
[ZombineDev] wrote:
 On Wednesday, 29 July 2020 at 12:54:36 UTC, Adam D. Ruppe wrote:
 On Wednesday, 29 July 2020 at 10:38:29 UTC, Manu wrote:
 On Wed, Jul 29, 2020 at 9:40 AM Jean-Louis Leroy via
 Guaranteed failure then? ;-)
Yes.
What's wrong with my solution earlier in the thread?
That it uses a string mixin :P What Manu is arguing is that if parameter storage classes were instead proper type qualifiers, then one could trivially manipulate them with std.meta. And then supposedly there would be no need to string mixins at all.
If we go back to the original problem, you still need a string mixin to inject the function name in two places. As for Adam's solution, it solves a slightly different problem but I don't see why he uses a string mixin: template forward(alias fun) { import std.traits; (__traits(getAttributes, fun)) auto ref forward(Parameters!fun args) { return fun(args); } } (42) void myfun(int, ref double x, out string s); pragma(msg, typeof(forward!myfun)); // void function(int _param_0, ref double _param_1, out string _param_2) system pragma(msg, __traits(getAttributes, forward!myfun)); // tuple(42) Anyway, storage classes are not a difficulty, as long as you use the whole __parameters, or slice it (__parameters[0..1]), and refrain from indexing it (__parameters[0] loses storage classes). Strange beast...
Jul 29 2020
next sibling parent reply Jean-Louis Leroy <jl leroy.nyc> writes:
On Wednesday, 29 July 2020 at 16:31:08 UTC, Jean-Louis Leroy 
wrote:
 As for Adam's solution, it solves a slightly different problem 
 but I don't see why he uses a string mixin:
OK I see why Adam uses a string mixin: to generate identifiers to alias __parameters to. But that is not necessary. His complete example sans string mixins here: https://gist.github.com/jll63/d6575d2ec5318c355f164e529453db73
Jul 29 2020
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 29 July 2020 at 16:52:32 UTC, Jean-Louis Leroy 
wrote:
 OK I see why Adam uses a string mixin: to generate identifiers 
 to alias __parameters to. But that is not necessary.
Yeah, that works too, but it introduces a template which I also try to avoid due to compiler memory management issues (the trivial mixin eats less dmd ram).
Jul 29 2020
prev sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Wednesday, 29 July 2020 at 16:31:08 UTC, Jean-Louis Leroy 
wrote:
 On Wednesday, 29 July 2020 at 16:11:02 UTC, Petar Kirov 
 [ZombineDev] wrote:
 On Wednesday, 29 July 2020 at 12:54:36 UTC, Adam D. Ruppe 
 wrote:
 On Wednesday, 29 July 2020 at 10:38:29 UTC, Manu wrote:
 On Wed, Jul 29, 2020 at 9:40 AM Jean-Louis Leroy via
 Guaranteed failure then? ;-)
Yes.
What's wrong with my solution earlier in the thread?
That it uses a string mixin :P What Manu is arguing is that if parameter storage classes were instead proper type qualifiers, then one could trivially manipulate them with std.meta. And then supposedly there would be no need to string mixins at all.
If we go back to the original problem, you still need a string mixin to inject the function name in two places. As for Adam's solution, it solves a slightly different problem but I don't see why he uses a string mixin: template forward(alias fun) { import std.traits; (__traits(getAttributes, fun)) auto ref forward(Parameters!fun args) { return fun(args); } } (42) void myfun(int, ref double x, out string s); pragma(msg, typeof(forward!myfun)); // void function(int _param_0, ref double _param_1, out string _param_2) system pragma(msg, __traits(getAttributes, forward!myfun)); // tuple(42) Anyway, storage classes are not a difficulty, as long as you use the whole __parameters, or slice it (__parameters[0..1]), and refrain from indexing it (__parameters[0] loses storage classes). Strange beast...
My point, that you can't do much useful processing with `__parameters` (except pass all of subset of it), still stands. Say you have a function with N parameters. Some of those parameters are integers and may have a UDA attached to them that specifies the minimum and maximum value they may receive. For example: R fun( scope P1 arg1, return int* arg2, lazy intetval(-16, 320) long arg3, interval(0, 127) uint arg4, ref interval(0, 8192) size_t arg5 ); The task is create a template, which given a function type like typeof(&fun) returns a new function type with all integer parameters replaced by the smallest type that is big enough to hold the interval specified by the UDA and preserves all storage classes. R hun( scope P1 arg1, return int* arg2, lazy intetval(-16, 320) short arg3, interval(0, 127) ubyte arg4, ref interval(0, 8192) ushort arg5 ); The way I'd like to go about solving this is like this: template TightenIntegerParams(Fun) { alias TightenIntegerParams = ReturnType!Fun function( staticMap!( TightenParam, Parameters!Fun ) ); } However the moment staticMap internally does `Args[i]` in order to pass the i-th element to the mapping function `F` we lose all information about the storage classes and UDAs.
Jul 29 2020
next sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Thursday, 30 July 2020 at 04:56:55 UTC, Petar Kirov 
[ZombineDev] wrote:
 On Wednesday, 29 July 2020 at 16:31:08 UTC, Jean-Louis Leroy 
 wrote:
 [...]
My point, that you can't do much useful processing with `__parameters` (except pass all of subset of it), still stands. Say you have a function with N parameters. Some of those parameters are integers and may have a UDA attached to them that specifies the minimum and maximum value they may receive. For example: R fun( scope P1 arg1, return int* arg2, lazy intetval(-16, 320) long arg3, interval(0, 127) uint arg4, ref interval(0, 8192) size_t arg5 ); The task is create a template, which given a function type like typeof(&fun) returns a new function type with all integer parameters replaced by the smallest type that is big enough to hold the interval specified by the UDA and preserves all storage classes. R hun( scope P1 arg1, return int* arg2, lazy intetval(-16, 320) short arg3, interval(0, 127) ubyte arg4, ref interval(0, 8192) ushort arg5 ); The way I'd like to go about solving this is like this: template TightenIntegerParams(Fun) { alias TightenIntegerParams = ReturnType!Fun function( staticMap!( TightenParam, Parameters!Fun ) ); } However the moment staticMap internally does `Args[i]` in order to pass the i-th element to the mapping function `F` we lose all information about the storage classes and UDAs.
I now saw that Manu clarified what he meant. Also I do realize that what we actually need is for `Parameters!fun[0]` to return a parameter object that includes not just the type but also the parameter name, UDAs attached and storage classes. If for example `ref` was a type qualifier, or just otherwise preserved by `alias`-ing it would help, but not solve the whole issue.
Jul 29 2020
parent reply Manu <turkeyman gmail.com> writes:
On Thu, Jul 30, 2020 at 3:45 PM Petar via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Thursday, 30 July 2020 at 04:56:55 UTC, Petar Kirov
 [ZombineDev] wrote:
 On Wednesday, 29 July 2020 at 16:31:08 UTC, Jean-Louis Leroy
 wrote:
 [...]
My point, that you can't do much useful processing with `__parameters` (except pass all of subset of it), still stands. Say you have a function with N parameters. Some of those parameters are integers and may have a UDA attached to them that specifies the minimum and maximum value they may receive. For example: R fun( scope P1 arg1, return int* arg2, lazy intetval(-16, 320) long arg3, interval(0, 127) uint arg4, ref interval(0, 8192) size_t arg5 ); The task is create a template, which given a function type like typeof(&fun) returns a new function type with all integer parameters replaced by the smallest type that is big enough to hold the interval specified by the UDA and preserves all storage classes. R hun( scope P1 arg1, return int* arg2, lazy intetval(-16, 320) short arg3, interval(0, 127) ubyte arg4, ref interval(0, 8192) ushort arg5 ); The way I'd like to go about solving this is like this: template TightenIntegerParams(Fun) { alias TightenIntegerParams = ReturnType!Fun function( staticMap!( TightenParam, Parameters!Fun ) ); } However the moment staticMap internally does `Args[i]` in order to pass the i-th element to the mapping function `F` we lose all information about the storage classes and UDAs.
I now saw that Manu clarified what he meant. Also I do realize that what we actually need is for `Parameters!fun[0]` to return a parameter object that includes not just the type but also the parameter name, UDAs attached and storage classes. If for example `ref` was a type qualifier, or just otherwise preserved by `alias`-ing it would help, but not solve the whole issue.
UDA's and default args are in-language concepts, alias/tuples can handle them, and don't require any particularly special meta to handle.
Jul 30 2020
parent Boris Carvajal <boris2.9 gmail.com> writes:
On Thursday, 30 July 2020 at 08:36:26 UTC, Manu wrote:
 I now saw that Manu clarified what he meant.

 Also I do realize that what we actually need is for 
 `Parameters!fun[0]` to return a parameter object that includes 
 not just the type but also the parameter name, UDAs attached 
 and storage classes. If for example `ref` was a type 
 qualifier, or just otherwise preserved by `alias`-ing it would 
 help, but not solve the whole issue.
UDA's and default args are in-language concepts, alias/tuples can handle them, and don't require any particularly special meta to handle.
You can't alias default args like: fun(time_t a = time(null))
Jul 31 2020
prev sibling next sibling parent Jean-Louis Leroy <jl leroy.nyc> writes:
On Thursday, 30 July 2020 at 04:56:55 UTC, Petar Kirov 
[ZombineDev] wrote:

 For example:

 R fun(
   scope P1 arg1,
   return int* arg2,
   lazy  intetval(-16, 320) long arg3,
    interval(0, 127) uint arg4,
   ref  interval(0, 8192) size_t arg5
 );

 The task is create a template, which given a function type like 
 typeof(&fun) returns a new function type with all integer 
 parameters replaced by the smallest type that is big enough to 
 hold the interval specified by the UDA and preserves all 
 storage classes.


 R hun(
   scope P1 arg1,
   return int* arg2,
   lazy  intetval(-16, 320) short arg3,
    interval(0, 127) ubyte arg4,
   ref  interval(0, 8192) ushort arg5
 );

 The way I'd like to go about solving this is like this:

 template TightenIntegerParams(Fun)
 {
   alias TightenIntegerParams =
     ReturnType!Fun function(
       staticMap!(
         TightenParam,
         Parameters!Fun
       )
     );
 }

 However the moment staticMap internally does `Args[i]` in order 
 to pass the i-th element to the mapping function `F` we lose 
 all information about the storage classes and UDAs.
Yes I know. My original problem was: // from attrs ref int foo( otherattrs virtual!Foo obj1, lazy int var, ref virtual!Bar obj2); // make: attrs ref int foo( otherattrs Foo obj1, lazy int var, ref Bar obj2) { return dispatch(obj1, obj2)(obj1, var, obj2); } Back to what you described, is it a real use case? I am thinking of writing an article on function generation for the D blog. I was wondering how often this sort of transformation is needed.
Jul 30 2020
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 30 July 2020 at 04:56:55 UTC, Petar Kirov 
[ZombineDev] wrote:
 My point, that you can't do much useful processing with 
 `__parameters` (except pass all of subset of it), still stands.
Yeah, I agree it is hard to do a substitution there. The foo... static map proposal a little while ago Manu wrote about is potentially exciting because it enables a lot of new cases, and I think this is another example.
Jul 30 2020
parent Manu <turkeyman gmail.com> writes:
On Thu, Jul 30, 2020 at 10:25 PM Adam D. Ruppe via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Thursday, 30 July 2020 at 04:56:55 UTC, Petar Kirov
 [ZombineDev] wrote:
 My point, that you can't do much useful processing with
 `__parameters` (except pass all of subset of it), still stands.
Yeah, I agree it is hard to do a substitution there. The foo... static map proposal a little while ago Manu wrote about is potentially exciting because it enables a lot of new cases, and I think this is another example.
This is actually one of many such known motivators for the DIP.
Jul 30 2020
prev sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 29 July 2020 at 16:11:02 UTC, Petar Kirov 
[ZombineDev] wrote:
 That it uses a string mixin :P
Not really - that string mixin is only there to work around a compiler bug leaking a name out of the scope https://issues.dlang.org/show_bug.cgi?id=21078 Notice that the only thing actually mixed in is a random number to give a temporary variable a unique name. It has nothing to do with storage classes or types or anything else.
Jul 29 2020
prev sibling parent Manu <turkeyman gmail.com> writes:
On Wed, Jul 29, 2020 at 10:55 PM Adam D. Ruppe via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Wednesday, 29 July 2020 at 10:38:29 UTC, Manu wrote:
 On Wed, Jul 29, 2020 at 9:40 AM Jean-Louis Leroy via
 Guaranteed failure then? ;-)
Yes.
What's wrong with my solution earlier in the thread?
It's pretty good, except my experience is that __parameters works only in the narrow case of a verbatim forward. I rarely find I need to implement a verbatim forward; I almost always seem to encounter a forwarding pattern when it is necessary to perform a parameter transformation of some kind, or a parameter injection. You need to be able to handle the arguments and iterate/manipulate them. I find I'm typically synthesising wrappers or shim's, not verbatim forwarding... which isn't strictly the topic of this thread, but it's a 100% related problem, and it seems to be what I actually need to do 95% of the time rather than the challenge in the OP.
Jul 29 2020
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 26 July 2020 at 01:29:21 UTC, Andrei Alexandrescu 
wrote:
 This topic came about during beerconf (it was fun!): Write an 
 idiomatic template `forward` that takes an alias `fun` and 
 defines (generates) one overload for each overload of `fun`. 
 Example:

 template forward(alias fun)
 {
     ...
 }

 Now for this function:

 void myfun(int, ref double, out string);
 int myfun(in string, inout double);

 the instantiation would be (stylized):

 template forward!myfun
 {
     void myfun(int a, ref double b, out string c)
     {
         return myfun(a, b, c);
     }
     int myfun(in string a, inout double b)
     {
         return myfun(a, b);
     }
 }

 So the challenge is implementing forward() to do this.
I would be interested an actual use-cases for this? Where would you use, forward? Where would it be the most pleasant solution, and others would be vastly inferior?
Jul 29 2020
parent reply Chad Joan <chadjoan gmail.com> writes:
On Wednesday, 29 July 2020 at 20:07:35 UTC, Stefan Koch wrote:
 I would be interested an actual use-cases for this?

 Where would you use, forward? Where would it be the most 
 pleasant solution, and others would be vastly inferior?
I just ran into a use-case tonight! Maybe. The caveat is that my use case might be pretty easy compared to what Andrei is asking for. My use case requires modifying the function signature a lot, but that can actually be exploited to make my solution easier. I probably only need to forward storage class. It's still kinda leading to code that feels wrong or isn't obvious, so maybe it will help your discussion. Now for the use-case itself: I am considering putting a "stringize" method in my types, with a signature (by convention) like so: nogc nothrow void stringize(return scope StringVacuum writer) { ... } (I also plan to support a UFCS-style variant, but let's not get distracted.) This method allows a type to be converted to a string, much like with "toString", but without a direct implication of heap-based (e.g. GC) memory allocation. If the type's string representation is being consumed by a non-memory source, such as stdout or a file stream, then it should be *possible* to avoid memory allocations entirely, at least in most cases, and with the very technical exception of small stack-allocated buffers for handling things like integer/float stringization (not shown here). Gee, it would really suck if this were some kind of departure from the norm that caused incompatibilities with existing code and infrastructure. Whoops. But it's OK, because "stringize" is toString-but-better, so it should be possible to just mechanically derive a "toString" function from every "stringize" function. Whenever this is used, it does abandon the original zero-allocation-conversion-to-string feature, but "toString" never had that to begin with (at least not in the general case; just for static strings and such). Code that acknowledges "stringize" can level-up. Code that depends on "toString" still works. Cool. Now it would be really nice if the "toString" method generated from a "stringize" method would inherit its compile-time guarantees wherever possible. And sometimes it isn't possible. If "stringize" is 'nothrow', then "toString" based on "stringize" should also be 'nothrow'. However, if "stringize" is 'nogc', this does not automatically provide an nogc "toString", because "toString" is required to concatenate a bunch of stringize's text fragments, and that concatenation requires GC allocations regardless of how nogc "stringize" may be. So a minor step in this process is to forward a permutation of "stringize" to "toString". Here is more detailed code: https://pastebin.com/JGfkkTxz I am going to be in drafting-mode for a while and I haven't tried to compile any of that code. Please expect mistakes; I'm providing this to illustrate intent. I hope it helps.
Jul 31 2020
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 31 July 2020 at 09:10:44 UTC, Chad Joan wrote:
 Whenever this is used, it does abandon the original 
 zero-allocation-conversion-to-string feature, but "toString" 
 never had that to begin with (at least not in the general case; 
 just for static strings and such).
kinda a site node but be aware there are toString overloads that do offer this. see this for example: https://github.com/dlang/druntime/blob/master/src/object.d#L1995
Jul 31 2020
parent Chad Joan <chadjoan gmail.com> writes:
On Friday, 31 July 2020 at 13:01:37 UTC, Adam D. Ruppe wrote:
 On Friday, 31 July 2020 at 09:10:44 UTC, Chad Joan wrote:
 Whenever this is used, it does abandon the original 
 zero-allocation-conversion-to-string feature, but "toString" 
 never had that to begin with (at least not in the general 
 case; just for static strings and such).
kinda a site node but be aware there are toString overloads that do offer this. see this for example: https://github.com/dlang/druntime/blob/master/src/object.d#L1995
That's good to know. Thanks for mentioning that!
Jul 31 2020
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Friday, 31 July 2020 at 09:10:44 UTC, Chad Joan wrote:
 I am considering putting a "stringize" method in my types, with 
 a signature (by convention) like so:
  nogc nothrow void stringize(return scope StringVacuum writer) 
 { ... }
Are you aware that toString already supports this kind of overload? The documentation for it is under `std.format.formatValue`, so it is a bit easy to miss: https://dlang.org/phobos/std_format.html#.formatValue
Jul 31 2020
parent Chad Joan <chadjoan gmail.com> writes:
On Friday, 31 July 2020 at 13:48:37 UTC, Paul Backus wrote:
 On Friday, 31 July 2020 at 09:10:44 UTC, Chad Joan wrote:
 I am considering putting a "stringize" method in my types, 
 with a signature (by convention) like so:
  nogc nothrow void stringize(return scope StringVacuum writer) 
 { ... }
Are you aware that toString already supports this kind of overload? The documentation for it is under `std.format.formatValue`, so it is a bit easy to miss: https://dlang.org/phobos/std_format.html#.formatValue
I was not. That's good to know. This one even accepts an OutputRange. Cool. Thanks!
Jul 31 2020