digitalmars.D - Feature request: __traits(canInstantiate), like __traits(compiles) but
- FeepingCreature (9/9) Jan 15 2020 Right now it's impossible to check "is there a valid (template)
- H. S. Teoh (46/56) Jan 15 2020 Yeah, the error-gagging method of speculative template instantiation has
- Atila Neves (3/14) Jan 22 2020 I've somehow only just discovered that `-verrors=spec` is a thing
- H. S. Teoh (9/24) Jan 22 2020 Only barely, because it generally produces pages upon pages of
- MWumpusZ (14/18) Jan 16 2020 Another example:
- Steven Schveighoffer (9/19) Jan 16 2020 I know what you mean, but they are not syntax errors (which ARE errors,
- H. S. Teoh (31/45) Jan 16 2020 Yeah, they are gagged errors that are nevertheless relevant to the
- FeepingCreature (8/27) Jan 16 2020 I think he means they're semantic errors rather than parser
- Steven Schveighoffer (7/13) Jan 17 2020 Just to point you at my previous thread on this:
- Petar Kirov [ZombineDev] (29/43) Jan 17 2020 In other languages, such as C#, TypeScript and Rust, generic
- Steven Schveighoffer (27/51) Jan 17 2020 Yes, concepts would be useful, but also would be limited for how D does
- H. S. Teoh (48/75) Jan 17 2020 IMO, not declaring everything up front for use is an anti-pattern.
- FeepingCreature (11/29) Jan 18 2020 Yes it can!
- Steven Schveighoffer (9/41) Jan 21 2020 I don't see how that helps. isNumeric!T is ignoring the problem that X
- Steven Schveighoffer (29/38) Jan 21 2020 Wait, I think I get it.
- user1234 (4/21) Jan 22 2020 Yes and there's an API for that
- John Colvin (10/21) Jan 17 2020 Yes please (although I would say specialization and constraints,
Right now it's impossible to check "is there a valid (template) specialization for this function call" without also saying "and by the way, if there's a syntax error in the function, suppress the error and return false." This makes many Phobos and library functions very annoying to use. A simple (?) fix for this would be adding a new trait that checks if the expression resolves to a valid template specialization, without attempting to also check if the function body compiles, or else without suppressing internal syntax errors.
Jan 15 2020
On Wed, Jan 15, 2020 at 03:13:08PM +0000, FeepingCreature via Digitalmars-d wrote:Right now it's impossible to check "is there a valid (template) specialization for this function call" without also saying "and by the way, if there's a syntax error in the function, suppress the error and return false." This makes many Phobos and library functions very annoying to use.Yeah, the error-gagging method of speculative template instantiation has been costing me lots of wasted time, too, esp. in Phobos range functions like .find, .filter, etc., because a single typo in the lambda argument causes the template to fail to instantiate, but the only information you get out of it is "cannot instantiate because it must satisfy isInputRange", but the reason it can't satisfy isInputRange is because a syntax error makes the type void (or _error_), but it doesn't tell you WHY the type is void or _error_. Is it a syntax error? A mistyped symbol? A mismatching type? Who knows, you can't tease that information out of the compiler because errors are gagged. And no, turning on verbose output does not help because it floods you with truckloads of "errors" for *other* template instantiations that are completely unrelated to this one (and that actually *worked*, so that's just meaningless noise), and you're left to dig through the mountains of messages to find out which one might actually be relevant to your problem. Furthermore, the error often crops up deep inside a long UFCS chain with multiple levels of nesting, so you can't exactly insert diagnostic messages in the middle to figure out where the problem is. The whole chain is one expression, after all, and all you'll get out of the compiler is that the entire expression failed to compile. But it could be as simple as a single typo deep inside one of the lambdas, or a missing import, and you've no idea where. The only way I could diagnose problems of this sort was to copy-n-paste the entire chain, comment out most of the except the first N steps, and then iteratively uncomment them until it stops compiling. Then once it stops compiling and I identified the offending lambda, copy-n-paste the lambda body into an actual fake function with actual parameter types that the compiler is forced to compile on its own -- because otherwise all errors are gagged and it's impossible to know what the compiler didn't like about it -- just to get the actual error message out of the compiler. And very often, it's a really trivial problem like a typo, a missing import, or forgetting to convert a type to the expected return type, etc.. The sort of error you'd expect the compiler to pinpoint with the exact line number and reason, and that you can fix in 30 seconds. But no thanks to gagged errors, it often takes more like 30 *minutes* just to narrow down the cause of the problem. All in all, it makes for a very poor user experience. No wonder the naysayers have no good things to say about Phobos templates!A simple (?) fix for this would be adding a new trait that checks if the expression resolves to a valid template specialization, without attempting to also check if the function body compiles, or else without suppressing internal syntax errors.+1000. This would've saved me HOURS of debugging trivial problems like missing imports and mistyped identifiers. T -- Chance favours the prepared mind. -- Louis Pasteur
Jan 15 2020
On Wednesday, 15 January 2020 at 18:13:08 UTC, H. S. Teoh wrote:On Wed, Jan 15, 2020 at 03:13:08PM +0000, FeepingCreature via Digitalmars-d wrote:I've somehow only just discovered that `-verrors=spec` is a thing that one can pass to dmd. Does that help?Right now it's impossible to check "is there a valid (template) specialization for this function call" without also saying "and by the way, if there's a syntax error in the function, suppress the error and return false." This makes many Phobos and library functions very annoying to use.Yeah, the error-gagging method of speculative template instantiation has been costing me lots of wasted time, too,
Jan 22 2020
On Wed, Jan 22, 2020 at 03:53:15PM +0000, Atila Neves via Digitalmars-d wrote:On Wednesday, 15 January 2020 at 18:13:08 UTC, H. S. Teoh wrote:Only barely, because it generally produces pages upon pages of irrelevant errors, and I have to save the output to a file and search for keywords just to find approximately where the error is. And *then* I have to decipher which of the errors are actually relevant. IOW it's a pretty awful user experience. T -- I don't trust computers, I've spent too long programming to think that they can get anything right. -- James MillerOn Wed, Jan 15, 2020 at 03:13:08PM +0000, FeepingCreature via Digitalmars-d wrote:I've somehow only just discovered that `-verrors=spec` is a thing that one can pass to dmd. Does that help?Right now it's impossible to check "is there a valid (template) specialization for this function call" without also saying "and by the way, if there's a syntax error in the function, suppress the error and return false." This makes many Phobos and library functions very annoying to use.Yeah, the error-gagging method of speculative template instantiation has been costing me lots of wasted time, too,
Jan 22 2020
On Wednesday, 15 January 2020 at 15:13:08 UTC, FeepingCreature wrote:Right now it's impossible to check "is there a valid (template) specialization for this function call" without also saying "and by the way, if there's a syntax error in the function, suppress the error and return false."Another example: The case which bothered me, and which I nagged FeepingCreature about leading to the post above, is that he has written us code to automatically JSON-decode to simple D structs, and we can override that automatic decoding where necessary, by writing a T decode(T: SomeSpecialStruct)(const JSONValue value) This works very well.... BUT: If I make an error such that my decode does not even compile, then I get no indication of this - a decoder is automatically generated instead, and at runtime I get strange behavior because the "wrong" decoder is being used. Ideally I get decode errors, but it is easy to imagine cases where that would not even occur.
Jan 16 2020
On 1/15/20 10:13 AM, FeepingCreature wrote:Right now it's impossible to check "is there a valid (template) specialization for this function call" without also saying "and by the way, if there's a syntax error in the function, suppress the error and return false."I know what you mean, but they are not syntax errors (which ARE errors, even in templates). Which makes them really difficult to categorize.This makes many Phobos and library functions very annoying to use. A simple (?) fix for this would be adding a new trait that checks if the expression resolves to a valid template specialization, without attempting to also check if the function body compiles, or else without suppressing internal syntax errors.Generally, I use hasMember instead of __triats(compiles) and I'm better off for it. If something defines a member by a specific name, then it better be what I'm looking for, so try using it. If not, then you'll just have to name it something else. I'm not sure Phobos can take this road though. -Steve
Jan 16 2020
On Thu, Jan 16, 2020 at 11:00:03AM -0500, Steven Schveighoffer via Digitalmars-d wrote:On 1/15/20 10:13 AM, FeepingCreature wrote:Yeah, they are gagged errors that are nevertheless relevant to the problem at hand. But it's very hard, from the compiler's standpoint, to know which gagged errors are relevant and which are not (most are not, in speculative template instantiations). I'm tempted to propose that anything that passes sig constraints *should* force the compiler to treat any errors in the function body as hard (non-gagged) errors. But then you also have sig constraints that fail due to gagged errors, and it's not clear at all which of those should be treated as hard errors. [...]Right now it's impossible to check "is there a valid (template) specialization for this function call" without also saying "and by the way, if there's a syntax error in the function, suppress the error and return false."I know what you mean, but they are not syntax errors (which ARE errors, even in templates). Which makes them really difficult to categorize.Generally, I use hasMember instead of __triats(compiles) and I'm better off for it. If something defines a member by a specific name, then it better be what I'm looking for, so try using it. If not, then you'll just have to name it something else. I'm not sure Phobos can take this road though.[...] Yeah, over time I've learned to avoid __traits(compiles) as much as possible. It's just far too general, and unhelpful when problems occur. (Why doesn't it compile? It could be any number of reasons, most of which probably the person who wrote the __traits(compiles) didn't even think of.) But anyway, recent versions of dmd are now more helpful, by pointing out which sig constraints are failing, rather than just a blanket "cannot match template" error. The next step IMO is to somehow display a brief summary of why that (or those) particular constraint(s) are failing. It's already quite helpful to know, e.g., that a template function failed to match because isInputRange!R failed. But isInputRange contains quite a number of further constraints; it would be nice to at least have an option to show the dirty details of where that particular constraint failed. I've no idea how to implement such a thing, though. T -- Computerese Irregular Verb Conjugation: I have preferences. You have biases. He/She has prejudices. -- Gene Wirchenko
Jan 16 2020
On Thursday, 16 January 2020 at 17:30:57 UTC, H. S. Teoh wrote:On Thu, Jan 16, 2020 at 11:00:03AM -0500, Steven Schveighoffer via Digitalmars-d wrote:I think he means they're semantic errors rather than parser (syntax) errors.On 1/15/20 10:13 AM, FeepingCreature wrote:Yeah, they are gagged errors that are nevertheless relevant to the problem at hand.Right now it's impossible to check "is there a valid (template) specialization for this function call" without also saying "and by the way, if there's a syntax error in the function, suppress the error and return false."I know what you mean, but they are not syntax errors (which ARE errors, even in templates). Which makes them really difficult to categorize.I'm tempted to propose that anything that passes sig constraints *should* force the compiler to treat any errors in the function body as hard (non-gagged) errors. But then you also have sig constraints that fail due to gagged errors, and it's not clear at all which of those should be treated as hard errors.Would be nice, but would be a hard spec change. This is the sort of thing that D should have been doing from the beginning, but we may be too far in now for __traits(compiles) to be changed to do this. However, this is the behavior I would ask for for __traits(canInstantiateWith).
Jan 16 2020
On 1/17/20 12:54 AM, FeepingCreature wrote:Would be nice, but would be a hard spec change. This is the sort of thing that D should have been doing from the beginning, but we may be too far in now for __traits(compiles) to be changed to do this. However, this is the behavior I would ask for for __traits(canInstantiateWith).Just to point you at my previous thread on this: https://forum.dlang.org/post/q1jequ$r63$1 digitalmars.com I think more tools for diagnosing why something doesn't happen would be good, but the request of "tell me if I was stupid" is I think impossible for the compiler to figure out. -Steve
Jan 17 2020
On Friday, 17 January 2020 at 14:57:27 UTC, Steven Schveighoffer wrote:On 1/17/20 12:54 AM, FeepingCreature wrote:functions can use only properties of generic types iff such properties were required in the function generic type constraints. AFAIK Nim's Concepts [1] are also moving in that direction, even though their current way is more similar to D (I chatted with one of their core developers, a couple of months ago at a conference, but I may be misremembering some details). In other words: void use(T)(T obj) where (hasMethod!(T, "callMe", int function(string, bool))) { int x = obj.callMe("hello", true); // this works // the following would cause the template definition to // fail to type check (before even being instantiated): // obj.someThingElse = 3; } Of course, requiring complete constraints for D templates would be quite controversial, (although many languages are able to get away with that), but I think it could be a good if we could opt-into that on a per function basis (e.g. using `where`, instead of `if`). If D has such a feature, I believe that libraries like phobos would use it extensively, while user (app) code would use current template duck typing model and only opt-into a stronger model when a developer wants to debug why his template code doesn't work as expected. [1]: https://nim-lang.org/docs/manual_experimental.html#conceptsWould be nice, but would be a hard spec change. This is the sort of thing that D should have been doing from the beginning, but we may be too far in now for __traits(compiles) to be changed to do this. However, this is the behavior I would ask for for __traits(canInstantiateWith).Just to point you at my previous thread on this: https://forum.dlang.org/post/q1jequ$r63$1 digitalmars.com I think more tools for diagnosing why something doesn't happen would be good, but the request of "tell me if I was stupid" is I think impossible for the compiler to figure out. -Steve
Jan 17 2020
On 1/17/20 10:34 AM, Petar Kirov [ZombineDev] wrote:On Friday, 17 January 2020 at 14:57:27 UTC, Steven Schveighoffer wrote:Yes, concepts would be useful, but also would be limited for how D does duck typing. D's constraints are very much like concepts, but without having to declare everything up front for use in the function. But it wouldn't work for UFCS either I don't think. However, the real problem is things like typos that cause the function not to compile for reasons other than the intended failures. For example: T foo(T)(T x) { return X + 5; // typo on parameter name } This will NEVER compile, because X will never exist. The intention was for it to compile with types that support addition to integers (i.e. it's EXPECTED to fail for strings), but instead it fails for all types. The result is that your code selects some other path when you don't want it to. Sometimes this just means it works but is slower, which is even harder to figure out. But it's hard for the compiler to deduce intention from such typos. It would be great if the compiler could figure out this type of error, but I don't think it can. The only "fix" really is to judiciously unittest with the calls you expect to work. And that generally doesn't happen (we are a lazy bunch!). Perhaps, an "expected" clause or something to ensure it compiles for the expected types like: expected(T == int), which wouldn't unittest anything, just make sure it can compile for something that is expected. -SteveOn 1/17/20 12:54 AM, FeepingCreature wrote:can use only properties of generic types iff such properties were required in the function generic type constraints. AFAIK Nim's Concepts [1] are also moving in that direction, even though their current way is more similar to D (I chatted with one of their core developers, a couple of months ago at a conference, but I may be misremembering some details).Would be nice, but would be a hard spec change. This is the sort of thing that D should have been doing from the beginning, but we may be too far in now for __traits(compiles) to be changed to do this. However, this is the behavior I would ask for for __traits(canInstantiateWith).Just to point you at my previous thread on this: https://forum.dlang.org/post/q1jequ$r63$1 digitalmars.com I think more tools for diagnosing why something doesn't happen would be good, but the request of "tell me if I was stupid" is I think impossible for the compiler to figure out.
Jan 17 2020
On Fri, Jan 17, 2020 at 11:01:29AM -0500, Steven Schveighoffer via Digitalmars-d wrote: [...]Yes, concepts would be useful, but also would be limited for how D does duck typing. D's constraints are very much like concepts, but without having to declare everything up front for use in the function.IMO, not declaring everything up front for use is an anti-pattern. Basically, by accepting a template parameter T and performing operations on it, you're assuming the T supports said operations. If T doesn't support such operations, you shouldn't be receiving it in the first place. By not specifying what operations you expect T to support in your sig constraints, you're basically declaring that you accept *any* T, including one that supports no operations, yet you proceed to operate on it, which ought to be an error (not just a silent failure to instantiate). IOW, you're actually making assumptions on what T supports, yet you never declared such assumptions. Conversely, if you declare no assumptions on T, then neither should you be allowed to operate on it. A parameter T received without any assumptions should be treated as an opaque object that allows *no* operations. [...]However, the real problem is things like typos that cause the function not to compile for reasons other than the intended failures. For example: T foo(T)(T x) { return X + 5; // typo on parameter name } This will NEVER compile, because X will never exist. The intention was for it to compile with types that support addition to integers (i.e. it's EXPECTED to fail for strings), but instead it fails for all types.See, this is why this kind of code is bad. You basically wish to accept all T that support +, but by not declaring it as such, the compiler has no way to know what you intended. So it assumes that you somehow expect X to spring into existence given some magic value of T, and when that never happens, the compiler always skips over foo. Had you declared that you expect T to support +, then the compiler would know to instantiate foo when T==int, then it would have caught the typo as a compile error. As a bonus, your code would even accept custom types that support +, without any further effort on your part. Another side effect of not declaring assumptions up-front is that the compiler cannot produce better error messages. You're stuck with error gagging that hides typos, because the compiler simply doesn't have enough information to know what was intended. What if you intended for foo never to compile? Without declaring any assumptions the compiler couldn't know better.But it's hard for the compiler to deduce intention from such typos. It would be great if the compiler could figure out this type of error, but I don't think it can.It can if there was a requirement that template arguments are not allowed to be operated on unless their ability to support the operation is either declared in a sig constraint, or else tested with an appropriate static if condition.The only "fix" really is to judiciously unittest with the calls you expect to work. And that generally doesn't happen (we are a lazy bunch!).Exactly, that's why you have to force users to declare what assumptions they're making on the template parameters. It's more "convenient" to just take the easy way and allow everything without checking, but it only leads to pain and more pain down the road.Perhaps, an "expected" clause or something to ensure it compiles for the expected types like: expected(T == int), which wouldn't unittest anything, just make sure it can compile for something that is expected.Just do this instead: unittest { // will generate compile error if instantiation fails: alias A = myTemplate!int; } T -- Кто везде - тот нигде.
Jan 17 2020
On Friday, 17 January 2020 at 16:01:29 UTC, Steven Schveighoffer wrote:However, the real problem is things like typos that cause the function not to compile for reasons other than the intended failures. For example: T foo(T)(T x) { return X + 5; // typo on parameter name } This will NEVER compile, because X will never exist. The intention was for it to compile with types that support addition to integers (i.e. it's EXPECTED to fail for strings), but instead it fails for all types. The result is that your code selects some other path when you don't want it to. Sometimes this just means it works but is slower, which is even harder to figure out. But it's hard for the compiler to deduce intention from such typos. It would be great if the compiler could figure out this type of error, but I don't think it can.Yes it can! D already has support for exactly this kind of thing, with std.traits and `if()` constraints. If you have a metafunction that tries to see if "an alias is usable with string", and it checks that with __traits(canInstantiateCall, foo, string.init) or whatever, then you *should* get a syntax error even in the "return x + 5" case. If you want to avoid that, we don't need language changes, you just need to use the *already existing* mechanism of "if (isNumeric!T)".
Jan 18 2020
On 1/18/20 10:21 AM, FeepingCreature wrote:On Friday, 17 January 2020 at 16:01:29 UTC, Steven Schveighoffer wrote:I don't see how that helps. isNumeric!T is ignoring the problem that X is the wrong symbol name. In the usage of foo, if I say if __traits(compiles, foo(5)) or __traits(canInstantiate, foo, int), how does it know the error during instantiation was not intended? In other words, I need to know, was foo *intended* to compile with T, not *does* it compile with T. -SteveHowever, the real problem is things like typos that cause the function not to compile for reasons other than the intended failures. For example: T foo(T)(T x) { return X + 5; // typo on parameter name } This will NEVER compile, because X will never exist. The intention was for it to compile with types that support addition to integers (i.e. it's EXPECTED to fail for strings), but instead it fails for all types. The result is that your code selects some other path when you don't want it to. Sometimes this just means it works but is slower, which is even harder to figure out. But it's hard for the compiler to deduce intention from such typos. It would be great if the compiler could figure out this type of error, but I don't think it can.Yes it can! D already has support for exactly this kind of thing, with std.traits and `if()` constraints. If you have a metafunction that tries to see if "an alias is usable with string", and it checks that with __traits(canInstantiateCall, foo, string.init) or whatever, then you *should* get a syntax error even in the "return x + 5" case. If you want to avoid that, we don't need language changes, you just need to use the *already existing* mechanism of "if (isNumeric!T)".
Jan 21 2020
On 1/18/20 10:21 AM, FeepingCreature wrote:Yes it can! D already has support for exactly this kind of thing, with std.traits and `if()` constraints. If you have a metafunction that tries to see if "an alias is usable with string", and it checks that with __traits(canInstantiateCall, foo, string.init) or whatever, then you *should* get a syntax error even in the "return x + 5" case. If you want to avoid that, we don't need language changes, you just need to use the *already existing* mechanism of "if (isNumeric!T)".Wait, I think I get it. You want something that says if the constraints allow the instantiation to proceed, then instantiate, and display the errors as generated. If not, then return false. So to further explore the stupid example I have: T foo(T)(T x) if(isNumeric!T) { return X + 5; } Then __traits(canInstantiate, foo, "") returns false because the constraints are false, and __traits(canInstantiate, foo, 5) produces an error because the implementation is bad. This is definitely possible. But, it could get really annoying. For instance, you may have to repeat the actual implementation in the constraints to get this to match up correctly. e.g.: T foo(T)(T x) if(__traits(compiles, T.init + 5)) { return X + 5; } I mean, isNumeric might not be what you want exactly, for example, you may want foo!BigInt to compile. But, a constraint-free template might be exactly what you wanted to do. Many range types don't care what their element type does, they are just focused on being a container for them. So there's no constraints based on the type. The idea has potential. -Steve
Jan 21 2020
On Tuesday, 21 January 2020 at 22:32:45 UTC, Steven Schveighoffer wrote:On 1/18/20 10:21 AM, FeepingCreature wrote:Yes and there's an API for that https://github.com/dlang/dmd/blob/ab2ea3880a144957811d3a8f7c63dfab0e351202/src/dmd/dtemplate.d#L762Yes it can! D already has support for exactly this kind of thing, with std.traits and `if()` constraints. If you have a metafunction that tries to see if "an alias is usable with string", and it checks that with __traits(canInstantiateCall, foo, string.init) or whatever, then you *should* get a syntax error even in the "return x + 5" case. If you want to avoid that, we don't need language changes, you just need to use the *already existing* mechanism of "if (isNumeric!T)".Wait, I think I get it. You want something that says if the constraints allow the instantiation to proceed, then instantiate, and display the errors as generated. If not, then return false. [...] -Steve
Jan 22 2020
On Wednesday, 15 January 2020 at 15:13:08 UTC, FeepingCreature wrote:Right now it's impossible to check "is there a valid (template) specialization for this function call" without also saying "and by the way, if there's a syntax error in the function, suppress the error and return false." This makes many Phobos and library functions very annoying to use. A simple (?) fix for this would be adding a new trait that checks if the expression resolves to a valid template specialization, without attempting to also check if the function body compiles, or else without suppressing internal syntax errors.Yes please (although I would say specialization and constraints, for all templates). I worked around this in a commercial project a few years ago by creating an overload set of the passed symbol and a catch-all template, then instantiate and see if I got the catch-all (meaning no match), but obviously this is pretty nasty and doesn't work if one of the overloads of the original template is a catch-all as well.
Jan 17 2020