www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - __traits(compiles, ...) with syntax errors

reply Steven Schveighoffer <schveiguy gmail.com> writes:
How many times have you written something like:

void foo(T)(T t) if (__traits(compiles, t.bar())) // or is(typeof(t.bar()))
{
    t.bar();
}

And somehow, somewhere, this isn't called? Then you remove the 
constraint, and find that there's a syntax error in the bar template 
function (like a missing semicolon).

I often times, to avoid this, simply just use __traits(hasMember, T, 
"bar"), and live with the consequences.

I'd love to have something like __traits(compiles), but have it be an 
error if the syntax is incorrect.

In other words, the constraint above only fails if there is a missing 
symbol, or incorrect type conversion, not that there is a stupid syntax 
error that would NEVER compile.

Ironically, it would be something like compilesOrSyntaxError.

Any thoughts?

-Steve
Jan 14 2019
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/14/19 4:47 PM, Steven Schveighoffer wrote:

 How many times have you written something like:
 
 void foo(T)(T t) if (__traits(compiles, t.bar())) // or is(typeof(t.bar()))
 {
     t.bar();
 }
 
 And somehow, somewhere, this isn't called? Then you remove the 
 constraint, and find that there's a syntax error in the bar template 
 function (like a missing semicolon).
 
 I often times, to avoid this, simply just use __traits(hasMember, T, 
 "bar"), and live with the consequences.
 
 I'd love to have something like __traits(compiles), but have it be an 
 error if the syntax is incorrect.
 
 In other words, the constraint above only fails if there is a missing 
 symbol, or incorrect type conversion, not that there is a stupid syntax 
 error that would NEVER compile.
 
 Ironically, it would be something like compilesOrSyntaxError.


Another option is to have __traits(compiles) spit out an error type that 
has truthiness if it's not an error (for backwards compatibility), then 
you could something like:

if(__traits(compiles, ...) != COMPILER_ERROR.SYNTAX)

-Steve
Jan 14 2019
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/14/19 4:52 PM, Steven Schveighoffer wrote:

 Another option is to have __traits(compiles) spit out an error type that 
 has truthiness if it's not an error (for backwards compatibility), then 
 you could something like:
 
 if(__traits(compiles, ...) != COMPILER_ERROR.SYNTAX)


Oops, it would have to be success or syntax error, so something like:

if(successOrSyntaxError!(__traits(compiles, ...)))

successOrSyntaxError would be a simple template

-Steve
Jan 14 2019
prev sibling next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer 
wrote:

 How many times have you written something like:

 void foo(T)(T t) if (__traits(compiles, t.bar())) // or 
 is(typeof(t.bar()))
 {
    t.bar();
 }

 And somehow, somewhere, this isn't called? Then you remove the 
 constraint, and find that there's a syntax error in the bar 
 template function (like a missing semicolon).


Ideally, you would catch this in the unit tests for `bar`. You 
are writing unit tests, aren't you? ;)
Jan 14 2019
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/14/19 4:57 PM, Paul Backus wrote:

 On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer wrote:

 How many times have you written something like:

 void foo(T)(T t) if (__traits(compiles, t.bar())) // or 
 is(typeof(t.bar()))
 {
    t.bar();
 }

 And somehow, somewhere, this isn't called? Then you remove the 
 constraint, and find that there's a syntax error in the bar template 
 function (like a missing semicolon).


 
 Ideally, you would catch this in the unit tests for `bar`. You are 
 writing unit tests, aren't you? ;)


It's not always that simple. Sometimes bar is part of a hook or 
something to a complex system, and you expect it to be called one way, 
and it's called another way (or the lack of ability to call it changes 
how the system behaves).

Do you unit test all your calls to front(), popFront or empty 
individually? I generally don't, I just put it in as a range, and see if 
that works.

Then I get an error like "this isn't a range", and it's not super-helpful.

Or worse, I get a "I can't call any of these 10 template overloads, each 
with a 5-term constraint"

-Steve
Jan 14 2019
prev sibling next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Monday, January 14, 2019 2:57:28 PM MST Paul Backus via Digitalmars-d 
wrote:

 On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer

 wrote:

 How many times have you written something like:

 void foo(T)(T t) if (__traits(compiles, t.bar())) // or
 is(typeof(t.bar()))
 {

    t.bar();

 }

 And somehow, somewhere, this isn't called? Then you remove the
 constraint, and find that there's a syntax error in the bar
 template function (like a missing semicolon).


 Ideally, you would catch this in the unit tests for `bar`. You
 are writing unit tests, aren't you? ;)


Good unit tests catch a lot, but it's hard for them to catch everything, and
having to test for stuff that the compiler can trivially tell you can get
really annoying. That's not to say that the compiler should find everything
for you, but sometimes, having it be able to tell you a bit more than
currently it does could save a lot of trouble. In this particular case, it's
not uncommon to have alternate implementations depending on compile time
conditionals. e.g. stuff like

static if(isRandomAccessRange!R)
{
    ...
}
else
{
    ...
}

isn't all that uncommon in optimized range code. The function may work just
fine whether the range is random-access or not, but the actual code that
gets compiled in (and therefore its performance) could vary quite a bit
depending on the range's capabilities.

Syntax errors are less likely in code like that than code that uses is
expressions or __traits(compiles, ...) directly, but the problem that Steven
brings up is very real - and in a way that it can actually be rather
difficult to test for. In many cases, you're basically forced to put in
pragmas or invalid code to see which branches are being compiled. Having a
trait similar to __traits(compiles, ...) but which actually gives you an
error for syntax errors would catch _some_ of the problems surrounding
branching on compile-time conditionals. However, given that it's still not
verifying that the correct branch was taken (just that the test is
syntactically valid), it still wouldn't solve the entire problem. It would
catch a certain class of bugs though. And they're bugs that wouldn't exist
anywhere outside of an is expression or __traits(compiles), ...), because
everywhere else, such syntax errors would be caught. It's just that with
those constructs, those errors basically get shut up and treated simply as
false (which is probably the wrong behavior in all cases except perhaps for
those that involve checking code that's been mixed in).

- Jonathan M Davis
Jan 14 2019
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Jan 14, 2019 at 04:31:06PM -0700, Jonathan M Davis via Digitalmars-d
wrote:

 On Monday, January 14, 2019 2:57:28 PM MST Paul Backus via Digitalmars-d 
 wrote:

 On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer

 wrote:

 How many times have you written something like:

 void foo(T)(T t) if (__traits(compiles, t.bar())) // or
 is(typeof(t.bar()))
 {

    t.bar();

 }

 And somehow, somewhere, this isn't called? Then you remove the
 constraint, and find that there's a syntax error in the bar
 template function (like a missing semicolon).






Yeah, I've run into this problem far too many times.  It's extremely
annoying because the error is silent, and makes the compiler just move
on to a different overload, leaving you scratching your head as to why
it's not picking the "correct" overload.



 Ideally, you would catch this in the unit tests for `bar`. You are
 writing unit tests, aren't you? ;)




[...]

The problem is not just trivial syntax errors like Steven's example.
The problem is that sufficiently complex sig constraints will sometimes
have unexpected corner cases that works in general, but fails for
specific cases.

One nasty example that comes to mind is when the sig constraint only
works with mutable types (or some other such qualified type) because you
used an overly-restrictive test.  Initial unittests work because you're
not expecting that changing the input to const would cause any trouble.
Then some time later, you happen to need to pass a const object, and
suddenly the compiler seemingly refuses to acknowledge that this
overload of the template function exists, and insists on trying a
different overload, or failing with a totally unhelpful error saying no
overloads match.

If you're dealing with something with as many overloads as
std.conv.toImpl, for example, good luck finding what went wrong. You
first have to parse every single darned sig constraint on every single
overload, just to find the one overload that *should* match, but
doesn't.  Then you have to parse the sig constraint clause by clause
just to determine which clause failed.  And *then*, if you're really
unlucky and the clause is a complex one (that calls another template,
say, like isXyzRange or something equally complex), *then* you have to
find the definition of that helper template, then parse through every
clause in *that* to find out what went wrong.  And if you're *truly*
unlucky, the helper template is of the form:

	isBlahBlahBlah(T) = __traits(compiles, (T t) {
		... // arbitrarily complex code here
	});

then you have to copy-n-paste the stupid lambda and instantiate it with
your exact version of T (and that's if the original template function's
sig constraint hadn't munged it with things like Unqual or any of the
other type-changing templates -- then good luck figuring out what T is
for your particular function call), just to get the stupid compiler to
tell you the stupid error that made it fail, because by default all
errors are gagged inside template instantiations during IFTI, so there's
nothing at all that even hints at what went wrong.

I've had to do this so many times when working with template-heavy code,
that recently I've started to develop a distaste for sig constraints.
Unless I'm working with complex overload sets where sig constraints are
unavoidable, I'd rather just leave the function without any sig
constraints -- then at least when something fails, you've a fighting
chance of knowing what went wrong without going through the above dance
*every* *single* *time*, because the compiler will now actually tell you
what the lousy compile error was.

And if you're really cursed, you'd run into a case like Jonathan says,
where there's a static if inside the template function that goes to the
else clause because it doesn't like your template argument for whatever
reason, so you just got switched to a different implementation with not
a clue as to how it ended up there.  Given enough nested static if's,
good luck figuring out just which one of them had the failing condition
that sent you down the wrong rabbit hole -- the failure is completely
silent; and remember, if the else-branch of the static if fails to
compile, the compiler will just gag the error and move on to the next
overload at the top level.  Good luck finding that one static if that
failed.

(Oh, and did I mention, almost all of the above requires you to edit the
*callee* code in order to find out where the template instantiation
went? Good luck if it goes into some common templates used everywhere
else, that if you change one line, something else unrelated in your
program will stop compiling so you won't even get to the point of
failure in the first place. Sometimes the only solution here is to
copy-n-paste the entire lousy template function into a different module
(possibly along with most/all of its dependencies) and call *that*
instead, so that the compiler won't prematurely die in unrelated code.)

This is a very real problem, and an extremely annoying one.  In a
hair-pullingly frustrating manner.  And of course, it makes you feel
*really* good 2 hours later after all of this jazz to discover that the
error came from a simple typo.


T

-- 
Life is too short to run proprietary software. -- Bdale Garbee
Jan 14 2019
next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 01/14/2019 03:58 PM, H. S. Teoh wrote:


 the error is silent, and makes the compiler just move
 on to a different overload, leaving you scratching your head as to why
 it's not picking the "correct" overload.


I've recently opened the following bug which I believe is due to the 
same underlying issue: A compilation error is silently skipped.

   https://issues.dlang.org/show_bug.cgi?id=19541


 (Oh, and did I mention, almost all of the above requires you to edit the
 *callee* code in order to find out where the template instantiation
 went?


Yep. Similar to my choice of words in that bug report: "the user [...] 
is now looking for unnecessary workarounds".

Ali
Jan 14 2019
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/14/19 6:58 PM, H. S. Teoh wrote:

 On Mon, Jan 14, 2019 at 04:31:06PM -0700, Jonathan M Davis via Digitalmars-d
wrote:

 On Monday, January 14, 2019 2:57:28 PM MST Paul Backus via Digitalmars-d
 wrote:

 On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer

 wrote:

 How many times have you written something like:

 void foo(T)(T t) if (__traits(compiles, t.bar())) // or
 is(typeof(t.bar()))
 {

     t.bar();

 }

 And somehow, somewhere, this isn't called? Then you remove the
 constraint, and find that there's a syntax error in the bar
 template function (like a missing semicolon).






 
 Yeah, I've run into this problem far too many times.  It's extremely
 annoying because the error is silent, and makes the compiler just move
 on to a different overload, leaving you scratching your head as to why
 it's not picking the "correct" overload.
 
 

 Ideally, you would catch this in the unit tests for `bar`. You are
 writing unit tests, aren't you? ;)




 [...]
 
 The problem is not just trivial syntax errors like Steven's example.
 The problem is that sufficiently complex sig constraints will sometimes
 have unexpected corner cases that works in general, but fails for
 specific cases.
 
 One nasty example that comes to mind is when the sig constraint only
 works with mutable types (or some other such qualified type) because you
 used an overly-restrictive test.  Initial unittests work because you're
 not expecting that changing the input to const would cause any trouble.
 Then some time later, you happen to need to pass a const object, and
 suddenly the compiler seemingly refuses to acknowledge that this
 overload of the template function exists, and insists on trying a
 different overload, or failing with a totally unhelpful error saying no
 overloads match.


So to clarify a bit, I'm not expecting miracles here. If your code is 
valid, but just isn't callable in the way you expect it to be, I don't 
see how the compiler can figure out that you meant it another way.

But my overall point really is that we have a mechanism to say "Does 
this compile?", when we really mean "Can I use this thing this way?". I 
wondered if it wouldn't be worth having a way to assume that all code is 
syntactically valid, and just throw errors when it's not.

And now that I'm thinking about it some more, I'm completely wrong. It 
is little typos that are syntactically valid that fail to compile -- 
syntax errors are still flagged (unless you are using mixins).

So this doesn't help at all :(

But I still face the same problems you bring up.


 (Oh, and did I mention, almost all of the above requires you to edit the
 *callee* code in order to find out where the template instantiation
 went? Good luck if it goes into some common templates used everywhere
 else, that if you change one line, something else unrelated in your
 program will stop compiling so you won't even get to the point of
 failure in the first place. Sometimes the only solution here is to
 copy-n-paste the entire lousy template function into a different module
 (possibly along with most/all of its dependencies) and call *that*
 instead, so that the compiler won't prematurely die in unrelated code.)


Oh yeah, this is a really hard problem. This captures a lot of the 
frustration that I've had too.

I'm wondering instead of having a way to help with this, if we can't 
specify (for debugging purposes), "this overload should be used". In 
other words, get rid of the constraints and force the compiler to use a 
certain call. That would help IMMENSELY.

I'm actually thinking of a library mechanism to do this, which is 
horrible, and awful, but might actually work:

struct PickMe
{
    string file;
    size_t line;
}

enum isMyOverload(...) more cruft

enum ItPickedMe(T, string file == __FILE__, size_t line == __LINE__) = 
hasUDA!(PickMe) && anySatisfy!(isMyOverload, getUDAs!(T, PickMe))

void foo(T)(T t) if(ItPickedMe!T || __traits(compiles, t.bar()))
{
    t.bar();
}

I don't want to do this, but a compiler feature that allowed it would be 
cool. Having to make copies of templates to be able to debug things is 
so horrid.

-Steve
Jan 14 2019
parent Olivier FAURE <couteaubleu gmail.com> writes:
On Tuesday, 15 January 2019 at 01:59:58 UTC, Steven Schveighoffer 
wrote:

 I don't want to do this, but a compiler feature that allowed it 
 would be cool. Having to make copies of templates to be able to 
 debug things is so horrid.

 -Steve


Maybe a sub-name system? Like:

     T* myFunc.withFoo(T)(T t) if(__traits(compiles, t.foo()));
     T* myFunc.withBar(T)(T t) if(__traits(compiles, t.bar()));

Then if you call `myFunc.withFoo(t)`, you get something like:

    Error: Cannot call withFoo overload of myFunc
    Error: because constraint if(__traits(compiles, t.foo())) is 
not satisfied

(it would be especially helpful with the "multiple constraints" 
DIP)

It would probably be pretty hard to implement, and make name 
resolution even more complicated, though.
Jan 15 2019
prev sibling next sibling parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Mon, 14 Jan 2019 at 22:50, Steven Schveighoffer via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 Any thoughts?


Compile with -verrors=spec (or -Wspeculative if using gdc).

-- 
Iain
Jan 14 2019
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/14/19 5:25 PM, Iain Buclaw wrote:

 On Mon, 14 Jan 2019 at 22:50, Steven Schveighoffer via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 Any thoughts?


 
 Compile with -verrors=spec (or -Wspeculative if using gdc).
 


But wouldn't that show all errors, not just syntax errors? That might be 
a bit difficult to wade through.

-Steve
Jan 14 2019
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/14/19 5:32 PM, Steven Schveighoffer wrote:

 On 1/14/19 5:25 PM, Iain Buclaw wrote:

 On Mon, 14 Jan 2019 at 22:50, Steven Schveighoffer via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 Any thoughts?


 Compile with -verrors=spec (or -Wspeculative if using gdc).


 
 But wouldn't that show all errors, not just syntax errors? That might be 
 a bit difficult to wade through.


BTW, I didn't know about this switch, thanks for pointing it out.

-Steve
Jan 14 2019
prev sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Monday, 14 January 2019 at 22:32:59 UTC, Steven Schveighoffer 
wrote:

 On 1/14/19 5:25 PM, Iain Buclaw wrote:

 On Mon, 14 Jan 2019 at 22:50, Steven Schveighoffer via 
 Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 Any thoughts?


 
 Compile with -verrors=spec (or -Wspeculative if using gdc).
 


 But wouldn't that show all errors, not just syntax errors? That 
 might be a bit difficult to wade through.

 -Steve


Yep, the SNR on that switch is abominable.
Jan 14 2019
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/14/19 5:50 PM, Nicholas Wilson wrote:

 On Monday, 14 January 2019 at 22:32:59 UTC, Steven Schveighoffer wrote:

 On 1/14/19 5:25 PM, Iain Buclaw wrote:

 On Mon, 14 Jan 2019 at 22:50, Steven Schveighoffer via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 Any thoughts?


 Compile with -verrors=spec (or -Wspeculative if using gdc).


 But wouldn't that show all errors, not just syntax errors? That might 
 be a bit difficult to wade through.


 
 Yep, the SNR on that switch is abominable.


Could there be a way to focus that switch on certain files/lines? That 
would be much more useful.

-Steve
Jan 16 2019
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer 
wrote:

 How many times have you written something like:

 void foo(T)(T t) if (__traits(compiles, t.bar())) // or 
 is(typeof(t.bar()))
 {
    t.bar();
 }

 And somehow, somewhere, this isn't called? Then you remove the 
 constraint, and find that there's a syntax error in the bar 
 template function (like a missing semicolon).


struct A
{
     int f()()
     {
         return 0
     }
}

Like this? There's no way for it to ever compile.
Jan 15 2019
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/15/19 3:26 AM, Kagamin wrote:

 On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer wrote:

 How many times have you written something like:

 void foo(T)(T t) if (__traits(compiles, t.bar())) // or 
 is(typeof(t.bar()))
 {
    t.bar();
 }

 And somehow, somewhere, this isn't called? Then you remove the 
 constraint, and find that there's a syntax error in the bar template 
 function (like a missing semicolon).


 
 struct A
 {
      int f()()
      {
          return 0
      }
 }
 
 Like this? There's no way for it to ever compile.


Indeed, I realized after posting [1] that it's really stupid typos that 
I make that are still valid syntax, but obviously unintentional. By that 
I mean, it will never compile.

Like:

void foo(T)(T t)
{
    int x = 1;
    t.fun(X);
}

What I was trying to say is that I wanted a way to separate "obvious" 
errors from errors that happen because of the template parameters. But I 
think there simply isn't a way to do it. The best we can do might be to 
try and force specific instantiations to see why they don't compile.

-Steve

[1] https://forum.dlang.org/post/q1jequ$r63$1 digitalmars.com
Jan 16 2019
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer 
wrote:

 How many times have you written something like:


More times than I can count...


 void foo(T)(T t) if (__traits(compiles, t.bar())) // or 
 is(typeof(t.bar()))
 {
    t.bar();
 }

 [...]


I'm not sure how best to go about this.
Jan 15 2019
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Tuesday, 15 January 2019 at 11:15:21 UTC, Atila Neves wrote:

 On Monday, 14 January 2019 at 21:47:40 UTC, Steven 
 Schveighoffer wrote:

 How many times have you written something like:


 More times than I can count...


 void foo(T)(T t) if (__traits(compiles, t.bar())) // or 
 is(typeof(t.bar()))
 {
    t.bar();
 }

 [...]


 I'm not sure how best to go about this.


Sounds like a job for `static try` too keep things DRY:

void fun(T)(T t) if (__traits(compiles, t.foo()))
{
     t.foo();
}

void fun(T)(T t) if (__traits(compiles, t.bar()))
{
     t.bar();
}

// ----v

void fun(T)(T t)
{
     static try { t.bar(); }
     else static try { t.foo(); }
     else static assert(0, T.stringof ~ "does not support neither 
`foo`, nor `bar`");
}
Jan 15 2019
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/15/19 6:59 AM, Petar Kirov [ZombineDev] wrote:

 On Tuesday, 15 January 2019 at 11:15:21 UTC, Atila Neves wrote:

 On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer wrote:

 How many times have you written something like:


 More times than I can count...


 void foo(T)(T t) if (__traits(compiles, t.bar())) // or 
 is(typeof(t.bar()))
 {
    t.bar();
 }

 [...]


 I'm not sure how best to go about this.


 
 Sounds like a job for `static try` too keep things DRY:
 
 void fun(T)(T t) if (__traits(compiles, t.foo()))
 {
      t.foo();
 }
 
 void fun(T)(T t) if (__traits(compiles, t.bar()))
 {
      t.bar();
 }
 
 // ----v
 
 void fun(T)(T t)
 {
      static try { t.bar(); }
      else static try { t.foo(); }
      else static assert(0, T.stringof ~ "does not support neither `foo`, 
 nor `bar`");
 }


It might be nice to have something like this. Note that the two code 
samples aren't equivalent -- something that defines both bar and foo 
would not work with the first iteration, but would work with the second.

The DRYness of the constraint is annoying too, but this misses the point 
that t.bar and t.foo may be intended to BOTH compile, but only one does 
because of a "stupid" error. And so the wrong path is taken, and you 
either get weird other errors, or you get no errors, but the code 
doesn't work right or is unintentionally lower performance.

Understanding WHY a template constraint fails is important, but when you 
have __traits(compiles), it's a very blunt instrument. There are so many 
reasons why it might not compile, and it's really hard to figure out the 
answer, especially if you don't control the templates. Overriding the 
constraints might be the easiest way to probe the answers out.

-Steve
Jan 16 2019