www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - __traits(compiles, ...) with syntax errors

reply Steven Schveighoffer <schveiguy gmail.com> writes:
How many times have you written something like:

void foo(T)(T t) if (__traits(compiles, t.bar())) // or is(typeof(t.bar()))
{
    t.bar();
}

And somehow, somewhere, this isn't called? Then you remove the 
constraint, and find that there's a syntax error in the bar template 
function (like a missing semicolon).

I often times, to avoid this, simply just use __traits(hasMember, T, 
"bar"), and live with the consequences.

I'd love to have something like __traits(compiles), but have it be an 
error if the syntax is incorrect.

In other words, the constraint above only fails if there is a missing 
symbol, or incorrect type conversion, not that there is a stupid syntax 
error that would NEVER compile.

Ironically, it would be something like compilesOrSyntaxError.

Any thoughts?

-Steve
Jan 14
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/14/19 4:47 PM, Steven Schveighoffer wrote:
 How many times have you written something like:
 
 void foo(T)(T t) if (__traits(compiles, t.bar())) // or is(typeof(t.bar()))
 {
     t.bar();
 }
 
 And somehow, somewhere, this isn't called? Then you remove the 
 constraint, and find that there's a syntax error in the bar template 
 function (like a missing semicolon).
 
 I often times, to avoid this, simply just use __traits(hasMember, T, 
 "bar"), and live with the consequences.
 
 I'd love to have something like __traits(compiles), but have it be an 
 error if the syntax is incorrect.
 
 In other words, the constraint above only fails if there is a missing 
 symbol, or incorrect type conversion, not that there is a stupid syntax 
 error that would NEVER compile.
 
 Ironically, it would be something like compilesOrSyntaxError.
Another option is to have __traits(compiles) spit out an error type that has truthiness if it's not an error (for backwards compatibility), then you could something like: if(__traits(compiles, ...) != COMPILER_ERROR.SYNTAX) -Steve
Jan 14
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/14/19 4:52 PM, Steven Schveighoffer wrote:
 Another option is to have __traits(compiles) spit out an error type that 
 has truthiness if it's not an error (for backwards compatibility), then 
 you could something like:
 
 if(__traits(compiles, ...) != COMPILER_ERROR.SYNTAX)
Oops, it would have to be success or syntax error, so something like: if(successOrSyntaxError!(__traits(compiles, ...))) successOrSyntaxError would be a simple template -Steve
Jan 14
prev sibling next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer 
wrote:
 How many times have you written something like:

 void foo(T)(T t) if (__traits(compiles, t.bar())) // or 
 is(typeof(t.bar()))
 {
    t.bar();
 }

 And somehow, somewhere, this isn't called? Then you remove the 
 constraint, and find that there's a syntax error in the bar 
 template function (like a missing semicolon).
Ideally, you would catch this in the unit tests for `bar`. You are writing unit tests, aren't you? ;)
Jan 14
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/14/19 4:57 PM, Paul Backus wrote:
 On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer wrote:
 How many times have you written something like:

 void foo(T)(T t) if (__traits(compiles, t.bar())) // or 
 is(typeof(t.bar()))
 {
    t.bar();
 }

 And somehow, somewhere, this isn't called? Then you remove the 
 constraint, and find that there's a syntax error in the bar template 
 function (like a missing semicolon).
Ideally, you would catch this in the unit tests for `bar`. You are writing unit tests, aren't you? ;)
It's not always that simple. Sometimes bar is part of a hook or something to a complex system, and you expect it to be called one way, and it's called another way (or the lack of ability to call it changes how the system behaves). Do you unit test all your calls to front(), popFront or empty individually? I generally don't, I just put it in as a range, and see if that works. Then I get an error like "this isn't a range", and it's not super-helpful. Or worse, I get a "I can't call any of these 10 template overloads, each with a 5-term constraint" -Steve
Jan 14
prev sibling next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Monday, January 14, 2019 2:57:28 PM MST Paul Backus via Digitalmars-d 
wrote:
 On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer

 wrote:
 How many times have you written something like:

 void foo(T)(T t) if (__traits(compiles, t.bar())) // or
 is(typeof(t.bar()))
 {

    t.bar();

 }

 And somehow, somewhere, this isn't called? Then you remove the
 constraint, and find that there's a syntax error in the bar
 template function (like a missing semicolon).
Ideally, you would catch this in the unit tests for `bar`. You are writing unit tests, aren't you? ;)
Good unit tests catch a lot, but it's hard for them to catch everything, and having to test for stuff that the compiler can trivially tell you can get really annoying. That's not to say that the compiler should find everything for you, but sometimes, having it be able to tell you a bit more than currently it does could save a lot of trouble. In this particular case, it's not uncommon to have alternate implementations depending on compile time conditionals. e.g. stuff like static if(isRandomAccessRange!R) { ... } else { ... } isn't all that uncommon in optimized range code. The function may work just fine whether the range is random-access or not, but the actual code that gets compiled in (and therefore its performance) could vary quite a bit depending on the range's capabilities. Syntax errors are less likely in code like that than code that uses is expressions or __traits(compiles, ...) directly, but the problem that Steven brings up is very real - and in a way that it can actually be rather difficult to test for. In many cases, you're basically forced to put in pragmas or invalid code to see which branches are being compiled. Having a trait similar to __traits(compiles, ...) but which actually gives you an error for syntax errors would catch _some_ of the problems surrounding branching on compile-time conditionals. However, given that it's still not verifying that the correct branch was taken (just that the test is syntactically valid), it still wouldn't solve the entire problem. It would catch a certain class of bugs though. And they're bugs that wouldn't exist anywhere outside of an is expression or __traits(compiles), ...), because everywhere else, such syntax errors would be caught. It's just that with those constructs, those errors basically get shut up and treated simply as false (which is probably the wrong behavior in all cases except perhaps for those that involve checking code that's been mixed in). - Jonathan M Davis
Jan 14
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Jan 14, 2019 at 04:31:06PM -0700, Jonathan M Davis via Digitalmars-d
wrote:
 On Monday, January 14, 2019 2:57:28 PM MST Paul Backus via Digitalmars-d 
 wrote:
 On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer

 wrote:
 How many times have you written something like:

 void foo(T)(T t) if (__traits(compiles, t.bar())) // or
 is(typeof(t.bar()))
 {

    t.bar();

 }

 And somehow, somewhere, this isn't called? Then you remove the
 constraint, and find that there's a syntax error in the bar
 template function (like a missing semicolon).
Yeah, I've run into this problem far too many times. It's extremely annoying because the error is silent, and makes the compiler just move on to a different overload, leaving you scratching your head as to why it's not picking the "correct" overload.
 Ideally, you would catch this in the unit tests for `bar`. You are
 writing unit tests, aren't you? ;)
[...] The problem is not just trivial syntax errors like Steven's example. The problem is that sufficiently complex sig constraints will sometimes have unexpected corner cases that works in general, but fails for specific cases. One nasty example that comes to mind is when the sig constraint only works with mutable types (or some other such qualified type) because you used an overly-restrictive test. Initial unittests work because you're not expecting that changing the input to const would cause any trouble. Then some time later, you happen to need to pass a const object, and suddenly the compiler seemingly refuses to acknowledge that this overload of the template function exists, and insists on trying a different overload, or failing with a totally unhelpful error saying no overloads match. If you're dealing with something with as many overloads as std.conv.toImpl, for example, good luck finding what went wrong. You first have to parse every single darned sig constraint on every single overload, just to find the one overload that *should* match, but doesn't. Then you have to parse the sig constraint clause by clause just to determine which clause failed. And *then*, if you're really unlucky and the clause is a complex one (that calls another template, say, like isXyzRange or something equally complex), *then* you have to find the definition of that helper template, then parse through every clause in *that* to find out what went wrong. And if you're *truly* unlucky, the helper template is of the form: isBlahBlahBlah(T) = __traits(compiles, (T t) { ... // arbitrarily complex code here }); then you have to copy-n-paste the stupid lambda and instantiate it with your exact version of T (and that's if the original template function's sig constraint hadn't munged it with things like Unqual or any of the other type-changing templates -- then good luck figuring out what T is for your particular function call), just to get the stupid compiler to tell you the stupid error that made it fail, because by default all errors are gagged inside template instantiations during IFTI, so there's nothing at all that even hints at what went wrong. I've had to do this so many times when working with template-heavy code, that recently I've started to develop a distaste for sig constraints. Unless I'm working with complex overload sets where sig constraints are unavoidable, I'd rather just leave the function without any sig constraints -- then at least when something fails, you've a fighting chance of knowing what went wrong without going through the above dance *every* *single* *time*, because the compiler will now actually tell you what the lousy compile error was. And if you're really cursed, you'd run into a case like Jonathan says, where there's a static if inside the template function that goes to the else clause because it doesn't like your template argument for whatever reason, so you just got switched to a different implementation with not a clue as to how it ended up there. Given enough nested static if's, good luck figuring out just which one of them had the failing condition that sent you down the wrong rabbit hole -- the failure is completely silent; and remember, if the else-branch of the static if fails to compile, the compiler will just gag the error and move on to the next overload at the top level. Good luck finding that one static if that failed. (Oh, and did I mention, almost all of the above requires you to edit the *callee* code in order to find out where the template instantiation went? Good luck if it goes into some common templates used everywhere else, that if you change one line, something else unrelated in your program will stop compiling so you won't even get to the point of failure in the first place. Sometimes the only solution here is to copy-n-paste the entire lousy template function into a different module (possibly along with most/all of its dependencies) and call *that* instead, so that the compiler won't prematurely die in unrelated code.) This is a very real problem, and an extremely annoying one. In a hair-pullingly frustrating manner. And of course, it makes you feel *really* good 2 hours later after all of this jazz to discover that the error came from a simple typo. T -- Life is too short to run proprietary software. -- Bdale Garbee
Jan 14
next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 01/14/2019 03:58 PM, H. S. Teoh wrote:

 the error is silent, and makes the compiler just move
 on to a different overload, leaving you scratching your head as to why
 it's not picking the "correct" overload.
I've recently opened the following bug which I believe is due to the same underlying issue: A compilation error is silently skipped. https://issues.dlang.org/show_bug.cgi?id=19541
 (Oh, and did I mention, almost all of the above requires you to edit the
 *callee* code in order to find out where the template instantiation
 went?
Yep. Similar to my choice of words in that bug report: "the user [...] is now looking for unnecessary workarounds". Ali
Jan 14
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/14/19 6:58 PM, H. S. Teoh wrote:
 On Mon, Jan 14, 2019 at 04:31:06PM -0700, Jonathan M Davis via Digitalmars-d
wrote:
 On Monday, January 14, 2019 2:57:28 PM MST Paul Backus via Digitalmars-d
 wrote:
 On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer

 wrote:
 How many times have you written something like:

 void foo(T)(T t) if (__traits(compiles, t.bar())) // or
 is(typeof(t.bar()))
 {

     t.bar();

 }

 And somehow, somewhere, this isn't called? Then you remove the
 constraint, and find that there's a syntax error in the bar
 template function (like a missing semicolon).
Yeah, I've run into this problem far too many times. It's extremely annoying because the error is silent, and makes the compiler just move on to a different overload, leaving you scratching your head as to why it's not picking the "correct" overload.
 Ideally, you would catch this in the unit tests for `bar`. You are
 writing unit tests, aren't you? ;)
[...] The problem is not just trivial syntax errors like Steven's example. The problem is that sufficiently complex sig constraints will sometimes have unexpected corner cases that works in general, but fails for specific cases. One nasty example that comes to mind is when the sig constraint only works with mutable types (or some other such qualified type) because you used an overly-restrictive test. Initial unittests work because you're not expecting that changing the input to const would cause any trouble. Then some time later, you happen to need to pass a const object, and suddenly the compiler seemingly refuses to acknowledge that this overload of the template function exists, and insists on trying a different overload, or failing with a totally unhelpful error saying no overloads match.
So to clarify a bit, I'm not expecting miracles here. If your code is valid, but just isn't callable in the way you expect it to be, I don't see how the compiler can figure out that you meant it another way. But my overall point really is that we have a mechanism to say "Does this compile?", when we really mean "Can I use this thing this way?". I wondered if it wouldn't be worth having a way to assume that all code is syntactically valid, and just throw errors when it's not. And now that I'm thinking about it some more, I'm completely wrong. It is little typos that are syntactically valid that fail to compile -- syntax errors are still flagged (unless you are using mixins). So this doesn't help at all :( But I still face the same problems you bring up.
 (Oh, and did I mention, almost all of the above requires you to edit the
 *callee* code in order to find out where the template instantiation
 went? Good luck if it goes into some common templates used everywhere
 else, that if you change one line, something else unrelated in your
 program will stop compiling so you won't even get to the point of
 failure in the first place. Sometimes the only solution here is to
 copy-n-paste the entire lousy template function into a different module
 (possibly along with most/all of its dependencies) and call *that*
 instead, so that the compiler won't prematurely die in unrelated code.)
Oh yeah, this is a really hard problem. This captures a lot of the frustration that I've had too. I'm wondering instead of having a way to help with this, if we can't specify (for debugging purposes), "this overload should be used". In other words, get rid of the constraints and force the compiler to use a certain call. That would help IMMENSELY. I'm actually thinking of a library mechanism to do this, which is horrible, and awful, but might actually work: struct PickMe { string file; size_t line; } enum isMyOverload(...) more cruft enum ItPickedMe(T, string file == __FILE__, size_t line == __LINE__) = hasUDA!(PickMe) && anySatisfy!(isMyOverload, getUDAs!(T, PickMe)) void foo(T)(T t) if(ItPickedMe!T || __traits(compiles, t.bar())) { t.bar(); } I don't want to do this, but a compiler feature that allowed it would be cool. Having to make copies of templates to be able to debug things is so horrid. -Steve
Jan 14
parent Olivier FAURE <couteaubleu gmail.com> writes:
On Tuesday, 15 January 2019 at 01:59:58 UTC, Steven Schveighoffer 
wrote:
 I don't want to do this, but a compiler feature that allowed it 
 would be cool. Having to make copies of templates to be able to 
 debug things is so horrid.

 -Steve
Maybe a sub-name system? Like: T* myFunc.withFoo(T)(T t) if(__traits(compiles, t.foo())); T* myFunc.withBar(T)(T t) if(__traits(compiles, t.bar())); Then if you call `myFunc.withFoo(t)`, you get something like: Error: Cannot call withFoo overload of myFunc Error: because constraint if(__traits(compiles, t.foo())) is not satisfied (it would be especially helpful with the "multiple constraints" DIP) It would probably be pretty hard to implement, and make name resolution even more complicated, though.
Jan 15
prev sibling next sibling parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Mon, 14 Jan 2019 at 22:50, Steven Schveighoffer via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Any thoughts?
Compile with -verrors=spec (or -Wspeculative if using gdc). -- Iain
Jan 14
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/14/19 5:25 PM, Iain Buclaw wrote:
 On Mon, 14 Jan 2019 at 22:50, Steven Schveighoffer via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 Any thoughts?
Compile with -verrors=spec (or -Wspeculative if using gdc).
But wouldn't that show all errors, not just syntax errors? That might be a bit difficult to wade through. -Steve
Jan 14
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/14/19 5:32 PM, Steven Schveighoffer wrote:
 On 1/14/19 5:25 PM, Iain Buclaw wrote:
 On Mon, 14 Jan 2019 at 22:50, Steven Schveighoffer via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 Any thoughts?
Compile with -verrors=spec (or -Wspeculative if using gdc).
But wouldn't that show all errors, not just syntax errors? That might be a bit difficult to wade through.
BTW, I didn't know about this switch, thanks for pointing it out. -Steve
Jan 14
prev sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Monday, 14 January 2019 at 22:32:59 UTC, Steven Schveighoffer 
wrote:
 On 1/14/19 5:25 PM, Iain Buclaw wrote:
 On Mon, 14 Jan 2019 at 22:50, Steven Schveighoffer via 
 Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 Any thoughts?
Compile with -verrors=spec (or -Wspeculative if using gdc).
But wouldn't that show all errors, not just syntax errors? That might be a bit difficult to wade through. -Steve
Yep, the SNR on that switch is abominable.
Jan 14
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/14/19 5:50 PM, Nicholas Wilson wrote:
 On Monday, 14 January 2019 at 22:32:59 UTC, Steven Schveighoffer wrote:
 On 1/14/19 5:25 PM, Iain Buclaw wrote:
 On Mon, 14 Jan 2019 at 22:50, Steven Schveighoffer via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 Any thoughts?
Compile with -verrors=spec (or -Wspeculative if using gdc).
But wouldn't that show all errors, not just syntax errors? That might be a bit difficult to wade through.
Yep, the SNR on that switch is abominable.
Could there be a way to focus that switch on certain files/lines? That would be much more useful. -Steve
Jan 16
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer 
wrote:
 How many times have you written something like:

 void foo(T)(T t) if (__traits(compiles, t.bar())) // or 
 is(typeof(t.bar()))
 {
    t.bar();
 }

 And somehow, somewhere, this isn't called? Then you remove the 
 constraint, and find that there's a syntax error in the bar 
 template function (like a missing semicolon).
struct A { int f()() { return 0 } } Like this? There's no way for it to ever compile.
Jan 15
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/15/19 3:26 AM, Kagamin wrote:
 On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer wrote:
 How many times have you written something like:

 void foo(T)(T t) if (__traits(compiles, t.bar())) // or 
 is(typeof(t.bar()))
 {
    t.bar();
 }

 And somehow, somewhere, this isn't called? Then you remove the 
 constraint, and find that there's a syntax error in the bar template 
 function (like a missing semicolon).
struct A {     int f()()     {         return 0     } } Like this? There's no way for it to ever compile.
Indeed, I realized after posting [1] that it's really stupid typos that I make that are still valid syntax, but obviously unintentional. By that I mean, it will never compile. Like: void foo(T)(T t) { int x = 1; t.fun(X); } What I was trying to say is that I wanted a way to separate "obvious" errors from errors that happen because of the template parameters. But I think there simply isn't a way to do it. The best we can do might be to try and force specific instantiations to see why they don't compile. -Steve [1] https://forum.dlang.org/post/q1jequ$r63$1 digitalmars.com
Jan 16
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer 
wrote:
 How many times have you written something like:
More times than I can count...
 void foo(T)(T t) if (__traits(compiles, t.bar())) // or 
 is(typeof(t.bar()))
 {
    t.bar();
 }

 [...]
I'm not sure how best to go about this.
Jan 15
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Tuesday, 15 January 2019 at 11:15:21 UTC, Atila Neves wrote:
 On Monday, 14 January 2019 at 21:47:40 UTC, Steven 
 Schveighoffer wrote:
 How many times have you written something like:
More times than I can count...
 void foo(T)(T t) if (__traits(compiles, t.bar())) // or 
 is(typeof(t.bar()))
 {
    t.bar();
 }

 [...]
I'm not sure how best to go about this.
Sounds like a job for `static try` too keep things DRY: void fun(T)(T t) if (__traits(compiles, t.foo())) { t.foo(); } void fun(T)(T t) if (__traits(compiles, t.bar())) { t.bar(); } // ----v void fun(T)(T t) { static try { t.bar(); } else static try { t.foo(); } else static assert(0, T.stringof ~ "does not support neither `foo`, nor `bar`"); }
Jan 15
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/15/19 6:59 AM, Petar Kirov [ZombineDev] wrote:
 On Tuesday, 15 January 2019 at 11:15:21 UTC, Atila Neves wrote:
 On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer wrote:
 How many times have you written something like:
More times than I can count...
 void foo(T)(T t) if (__traits(compiles, t.bar())) // or 
 is(typeof(t.bar()))
 {
    t.bar();
 }

 [...]
I'm not sure how best to go about this.
Sounds like a job for `static try` too keep things DRY: void fun(T)(T t) if (__traits(compiles, t.foo())) {     t.foo(); } void fun(T)(T t) if (__traits(compiles, t.bar())) {     t.bar(); } // ----v void fun(T)(T t) {     static try { t.bar(); }     else static try { t.foo(); }     else static assert(0, T.stringof ~ "does not support neither `foo`, nor `bar`"); }
It might be nice to have something like this. Note that the two code samples aren't equivalent -- something that defines both bar and foo would not work with the first iteration, but would work with the second. The DRYness of the constraint is annoying too, but this misses the point that t.bar and t.foo may be intended to BOTH compile, but only one does because of a "stupid" error. And so the wrong path is taken, and you either get weird other errors, or you get no errors, but the code doesn't work right or is unintentionally lower performance. Understanding WHY a template constraint fails is important, but when you have __traits(compiles), it's a very blunt instrument. There are so many reasons why it might not compile, and it's really hard to figure out the answer, especially if you don't control the templates. Overriding the constraints might be the easiest way to probe the answers out. -Steve
Jan 16