www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Improving assert-printing in DMD

reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
As a follow-up to

https://github.com/D-Programming-Language/phobos/pull/3207#issuecomment-144073495

I starting digging in DMD for logic controlling behaviour of 
assert(), especially whether it's possible to add automatic 
printing of `lhs` and `rhs` upon assertion failure if `AssertExp` 
is a binary expression say `lhs == rhs`.

After grepping for `AssertExp` the only possible place I could 
think of was

ToElemVisitor::visit(AssertExp *ae)

inside

elem *toElem(Expression *e, IRState *irs)

in file e2ir.c.

Questions:

1. Is this the right place where this lhs-rhs-printing logic 
should be added? If so could somebody tell me how to make this 
happen?

2. Is it possible to from within DMD generate expressions that do

`import std.stdio : write`

and then calls write on the `lhs` and `rsh`...or this a 
completely wrong approach to solving this problem?
Sep 29 2015
next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Tuesday, 29 September 2015 at 21:02:42 UTC, Nordlöw wrote:

 As a follow-up to

 https://github.com/D-Programming-Language/phobos/pull/3207#issuecomment-144073495

 I starting digging in DMD for logic controlling behaviour of 
 assert(), especially whether it's possible to add automatic 
 printing of `lhs` and `rhs` upon assertion failure if 
 `AssertExp` is a binary expression say `lhs == rhs`.

 After grepping for `AssertExp` the only possible place I could 
 think of was

 ToElemVisitor::visit(AssertExp *ae)

 inside

 elem *toElem(Expression *e, IRState *irs)

 in file e2ir.c.

 Questions:

 1. Is this the right place where this lhs-rhs-printing logic 
 should be added? If so could somebody tell me how to make this 
 happen?

 2. Is it possible to from within DMD generate expressions that 
 do

 `import std.stdio : write`

 and then calls write on the `lhs` and `rsh`...or this a 
 completely wrong approach to solving this problem?


Someone will write something like this:

assert(plainPassword == plainPassword.toLower());

and plaintext passwords will end up in stderr.
Sep 29 2015
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Sep 29, 2015 at 09:13:55PM +0000, John Colvin via Digitalmars-d wrote:

 On Tuesday, 29 September 2015 at 21:02:42 UTC, Nordlw wrote:


[...]

2. Is it possible to from within DMD generate expressions that do

`import std.stdio : write`

and then calls write on the `lhs` and `rsh`...or this a completely
wrong approach to solving this problem?


 
 Someone will write something like this:
 
 assert(plainPassword == plainPassword.toLower());
 
 and plaintext passwords will end up in stderr.


That's an instance of assert abuse, and we probably don't have to be
responsible for it.  Such checking belongs in enforce, or an explicit
check with an exception throw, NOT an assert.


T

-- 
If blunt statements had a point, they wouldn't be blunt...
Sep 29 2015
parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Tuesday, 29 September 2015 at 21:22:43 UTC, H. S. Teoh wrote:

 On Tue, Sep 29, 2015 at 09:13:55PM +0000, John Colvin via 
 Digitalmars-d wrote:

 On Tuesday, 29 September 2015 at 21:02:42 UTC, Nordlöw wrote:


 [...]

2. Is it possible to from within DMD generate expressions 
that do

`import std.stdio : write`

and then calls write on the `lhs` and `rsh`...or this a 
completely wrong approach to solving this problem?


 
 Someone will write something like this:
 
 assert(plainPassword == plainPassword.toLower());
 
 and plaintext passwords will end up in stderr.


 That's an instance of assert abuse, and we probably don't have 
 to be responsible for it.  Such checking belongs in enforce, or 
 an explicit check with an exception throw, NOT an assert.


 T


Not necessarily. It could just be a defensive assert for 
something that should already have been verified/cleaned/caught 
earlier.

auto pass = getPassword();
pass.clean();
assert(pass == pass.toLower());
//and on we go ...
Sep 29 2015
next sibling parent John Colvin <john.loughran.colvin gmail.com> writes:
On Tuesday, 29 September 2015 at 21:26:00 UTC, John Colvin wrote:

 On Tuesday, 29 September 2015 at 21:22:43 UTC, H. S. Teoh wrote:

 On Tue, Sep 29, 2015 at 09:13:55PM +0000, John Colvin via 
 Digitalmars-d wrote:

 On Tuesday, 29 September 2015 at 21:02:42 UTC, Nordlöw wrote:


 [...]

2. Is it possible to from within DMD generate expressions 
that do

`import std.stdio : write`

and then calls write on the `lhs` and `rsh`...or this a 
completely wrong approach to solving this problem?


 
 Someone will write something like this:
 
 assert(plainPassword == plainPassword.toLower());
 
 and plaintext passwords will end up in stderr.


 That's an instance of assert abuse, and we probably don't have 
 to be responsible for it.  Such checking belongs in enforce, 
 or an explicit check with an exception throw, NOT an assert.


 T


 Not necessarily. It could just be a defensive assert for 
 something that should already have been verified/cleaned/caught 
 earlier.

 auto pass = getPassword();
 pass.clean();
 assert(pass == pass.toLower());
 //and on we go ...


It could even be in an `in { }` block
Sep 29 2015
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 09/29/2015 05:25 PM, John Colvin wrote:

 auto pass = getPassword();
 pass.clean();
 assert(pass == pass.toLower());
 //and on we go ...


Interesting angle, but it's not something we should worry about. -- Andrei
Sep 29 2015
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/30/2015 05:08 AM, Andrei Alexandrescu wrote:

 On 09/29/2015 05:25 PM, John Colvin wrote:

 auto pass = getPassword();
 pass.clean();
 assert(pass == pass.toLower());
 //and on we go ...


 Interesting angle, but it's not something we should worry about. -- Andrei


There's also the possibility of failing assertions within toString 
functions causing stack overflows.
Sep 30 2015
prev sibling parent Kapps <opantm2+spam gmail.com> writes:
On Tuesday, 29 September 2015 at 21:26:00 UTC, John Colvin wrote:

 Not necessarily. It could just be a defensive assert for 
 something that should already have been verified/cleaned/caught 
 earlier.

 auto pass = getPassword();
 pass.clean();
 assert(pass == pass.toLower());
 //and on we go ...


There are a few flaws with this:
First, your assert applies only in debug mode. You're likely not 
deploying your service in debug mode, so your attempt at 
defensive programming does nothing to protect you when you 
actually need it.

Second, and more critical, the way assert is apparently intended 
to be, is a guarantee that can be used towards optimization. The 
existence of an assert means that that situation can *never* 
happen. You take your existing code, and then later on you decide 
that an assert isn't enough so you add another check for release 
mode. In which case (as far as I understand it), you run into the 
following situation:

auto pass = getPassword();
pass.clean();
assert(pass == pass.toLower());
// Later on...
enforce(pass == pass.toLower());

Now not only is your assert not triggered because you're in 
release mode, but that assert provides a guarantee to the 
compiler that pass is *always* equal to pass.toLower, causing the 
compiler to optimize out that enforce call as it's redundant. 
Therefore, your assert does nothing, and your enforce now does 
nothing as well, masking a potential vulnerability.
Oct 01 2015
prev sibling next sibling parent reply Andrej Mitrovic via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 9/29/15, John Colvin via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 Someone will write something like this:

 assert(plainPassword == plainPassword.toLower());

 and plaintext passwords will end up in stderr.


If you have plaintext passwords stored anywhere you are already screwed. ;)
Sep 29 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-09-29 23:32, Andrej Mitrovic via Digitalmars-d wrote:


 If you have plaintext passwords stored anywhere you are already screwed. ;)


The password always starts out in plaintext, or do you hash it in the 
front end, as the users types? Since the back end shouldn't trust the 
front end, it needs to hash it again.

-- 
/Jacob Carlborg
Sep 29 2015
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Sep 30, 2015 at 08:30:47AM +0200, Jacob Carlborg via Digitalmars-d
wrote:

 On 2015-09-29 23:32, Andrej Mitrovic via Digitalmars-d wrote:
 

If you have plaintext passwords stored anywhere you are already
screwed. ;)


 
 The password always starts out in plaintext, or do you hash it in the
 front end, as the users types? Since the back end shouldn't trust the
 front end, it needs to hash it again.


[...]

The right way to do it is for the server to send a random challenge
which the front end (presumably running on the user's machine) encrypts
with the password, sending the ciphertext back to the server.  The
plaintext password is never sent over wire, yet the only way the client
can provide the correct response is if it knows the password to begin
with.


T

-- 
LINUX = Lousy Interface for Nefarious Unix Xenophobes.
Sep 30 2015
parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Wednesday, 30 September 2015 at 14:53:31 UTC, H. S. Teoh wrote:

 On Wed, Sep 30, 2015 at 08:30:47AM +0200, Jacob Carlborg via 
 Digitalmars-d wrote:

 On 2015-09-29 23:32, Andrej Mitrovic via Digitalmars-d wrote:
 

If you have plaintext passwords stored anywhere you are 
already screwed. ;)


 
 The password always starts out in plaintext, or do you hash it 
 in the front end, as the users types? Since the back end 
 shouldn't trust the front end, it needs to hash it again.


 [...]

 The right way to do it is for the server to send a random 
 challenge which the front end (presumably running on the user's 
 machine) encrypts with the password, sending the ciphertext 
 back to the server.  The plaintext password is never sent over 
 wire, yet the only way the client can provide the correct 
 response is if it knows the password to begin with.


 T


right. Nonetheless, sometimes code does have to work with 
sensitive data and you don't want it to leak outside the program 
in unexpected ways.
Sep 30 2015
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Sep 30, 2015 at 04:14:59PM +0000, John Colvin via Digitalmars-d wrote:

 On Wednesday, 30 September 2015 at 14:53:31 UTC, H. S. Teoh wrote:

On Wed, Sep 30, 2015 at 08:30:47AM +0200, Jacob Carlborg via Digitalmars-d
wrote:

On 2015-09-29 23:32, Andrej Mitrovic via Digitalmars-d wrote:


If you have plaintext passwords stored anywhere you are >already


screwed. ;)

The password always starts out in plaintext, or do you hash it in
the front end, as the users types? Since the back end shouldn't
trust the front end, it needs to hash it again.


[...]

The right way to do it is for the server to send a random challenge
which the front end (presumably running on the user's machine)
encrypts with the password, sending the ciphertext back to the
server.  The plaintext password is never sent over wire, yet the only
way the client can provide the correct response is if it knows the
password to begin with.


T


 
 right. Nonetheless, sometimes code does have to work with sensitive
 data and you don't want it to leak outside the program in unexpected
 ways.


Certainly.  But I have a hard time imagining a scenario where I'd use
assert() on sensitive data.  After all, assert() should be used to
verify program *logic*, not the data that the program is processing.
That's clearly in the realm of enforce() or just plain ole if(), IMO.


T

-- 
Doubt is a self-fulfilling prophecy.
Sep 30 2015
parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Wednesday, 30 September 2015 at 17:59:41 UTC, H. S. Teoh wrote:

 On Wed, Sep 30, 2015 at 04:14:59PM +0000, John Colvin via 
 Digitalmars-d wrote:

 On Wednesday, 30 September 2015 at 14:53:31 UTC, H. S. Teoh 
 wrote:

On Wed, Sep 30, 2015 at 08:30:47AM +0200, Jacob Carlborg via 
Digitalmars-d wrote:

On 2015-09-29 23:32, Andrej Mitrovic via Digitalmars-d wrote:


If you have plaintext passwords stored anywhere you are

already




screwed. ;)

The password always starts out in plaintext, or do you hash 
it in the front end, as the users types? Since the back end 
shouldn't trust the front end, it needs to hash it again.


[...]

The right way to do it is for the server to send a random 
challenge which the front end (presumably running on the 
user's machine) encrypts with the password, sending the 
ciphertext back to the server.  The plaintext password is 
never sent over wire, yet the only way the client can provide 
the correct response is if it knows the password to begin 
with.


T


 
 right. Nonetheless, sometimes code does have to work with 
 sensitive data and you don't want it to leak outside the 
 program in unexpected ways.


 Certainly.  But I have a hard time imagining a scenario where 
 I'd use
 assert() on sensitive data.  After all, assert() should be used 
 to
 verify program *logic*, not the data that the program is 
 processing.
 That's clearly in the realm of enforce() or just plain ole 
 if(), IMO.


 T


Checks involving sensitive data after processing can definitely 
be a check of program logic.

Sensitive data enters program
Sensitive data is checked using enforce
Sensitive data is passed to another function, but something goes 
wrong (not enough checking before, wrong function called, HDD 
dies, someone trips over a network cable), an assert is 
triggered, the sensitive data spills to stderr.

A perfectly correct program in perfect circumstances will never 
assert, but real programs in real situations will.


At the very least there should be a compiler switch to turn 
assert-printing on/off
Oct 01 2015
next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 01-Oct-2015 11:46, John Colvin wrote:

 On Wednesday, 30 September 2015 at 17:59:41 UTC, H. S. Teoh wrote:

 On Wed, Sep 30, 2015 at 04:14:59PM +0000, John Colvin via
 Digitalmars-d wrote:

 On Wednesday, 30 September 2015 at 14:53:31 UTC, H. S. Teoh wrote:

On Wed, Sep 30, 2015 at 08:30:47AM +0200, Jacob Carlborg via
Digitalmars-d wrote:

On 2015-09-29 23:32, Andrej Mitrovic via Digitalmars-d wrote:


If you have plaintext passwords stored anywhere you are

already




screwed. ;)

The password always starts out in plaintext, or do you hash >>it in




 the front end, as the users types? Since the back end >>shouldn't
 trust the front end, it needs to hash it again.

[...]

The right way to do it is for the server to send a random >challenge


 which the front end (presumably running on the >user's machine)
 encrypts with the password, sending the >ciphertext back to the
 server.  The plaintext password is >never sent over wire, yet the
 only way the client can provide >the correct response is if it knows
 the password to begin >with.

T


 right. Nonetheless, sometimes code does have to work with sensitive
 data and you don't want it to leak outside the program in unexpected
 ways.


 Certainly.  But I have a hard time imagining a scenario where I'd use
 assert() on sensitive data.  After all, assert() should be used to
 verify program *logic*, not the data that the program is processing.
 That's clearly in the realm of enforce() or just plain ole if(), IMO.


 T


 Checks involving sensitive data after processing can definitely be a
 check of program logic.

 Sensitive data enters program
 Sensitive data is checked using enforce
 Sensitive data is passed to another function, but something goes wrong
 (not enough checking before, wrong function called, HDD dies, someone
 trips over a network cable), an assert is triggered, the sensitive data
 spills to stderr.

 A perfectly correct program in perfect circumstances will never assert,
 but real programs in real situations will.


To be honest to save us from this we should simply ban printing to 
stdout/stderr because some debug statement may print sensitive data.

Come on, there are better way to protect sensitive data then crippling 
language primitives/compiler switches.

Lastly - ALL data in a PC's RAM is wide open to sysadmins so anything in 
sitting in RAM is vulnerable anyhow.


 At the very least there should be a compiler switch to turn
 assert-printing on/off



-- 
Dmitry Olshansky
Oct 01 2015
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, 1 October 2015 at 08:46:51 UTC, John Colvin wrote:

 Checks involving sensitive data after processing can definitely 
 be a check of program logic.

 Sensitive data enters program
 Sensitive data is checked using enforce
 Sensitive data is passed to another function, but something 
 goes wrong (not enough checking before, wrong function called, 
 HDD dies, someone trips over a network cable), an assert is 
 triggered, the sensitive data spills to stderr.

 A perfectly correct program in perfect circumstances will never 
 assert, but real programs in real situations will.


In almost all cases, printing out an assertion is by far the best 
thing to do, and in general, the more information that's printed, 
the better. But if anyone is dealing with sensitive data, they 
_have_ to be smart about what they do with it, and if that means 
that they can't put it an assertion directly, because assertions 
print too much information, then so be it. We're talking about a 
minority case where almost every other program would benefit. 
Most programs don't even have assertions turned on in release, 
which is where this would matter.

If a failed assertion could print out sensitive information, then 
just don't put the information directly in the assertion. Heck, 
anyone paranoid enough about it could create a special 
secureAssert function or something that did something like

void secureAssert(E)(lazy E expression,
                      lazy string msg,
                      string file = __FILE__,
                      size_T line = __LINE__)
{
     version(assert)
     {
         immutable result = cast(bool)expression();
         if(!result)
             throw new AssertError(msg, file, line);
     }
}

I see no reason to make the debugging tools that we have worse 
because of fear of someone who's writing code that processes 
sensitive information. That code just needs to be written in a 
way that's not going to leak the information. As Dmitry points 
out, there are plenty of other ways for the information to be 
leaked. So, not improving assertions due to security concerns 
really isn't a valid approach IMHO.

Now, there are other reasons why we might not want to (e.g. if 
the performance were worse, you might not want to make assertions 
print additional information), but I think that it's far better 
for code that needs to be that secure to do its own thing than to 
hamper everyone over the concerns of the minority.


 At the very least there should be a compiler switch to turn 
 assert-printing on/off


I expect that most folks who would want to avoid have assertions 
printing out sensitive information would still want assertions 
that didn't relate to sensitive information to print. A compiler 
switch is a pretty blunt instrument, especially when someone 
could write their own code which asserts without showing any 
sensitive information while still testing that information.

- Jonathan M Davis
Oct 01 2015
parent John Colvin <john.loughran.colvin gmail.com> writes:
On Thursday, 1 October 2015 at 09:59:41 UTC, Jonathan M Davis 
wrote:

 In almost all cases, printing out an assertion is by far the 
 best thing to do, and in general, the more information that's 
 printed, the better. But if anyone is dealing with sensitive 
 data, they _have_ to be smart about what they do with it, and 
 if that means that they can't put it an assertion directly, 
 because assertions print too much information, then so be it. 
 We're talking about a minority case where almost every other 
 program would benefit. Most programs don't even have assertions 
 turned on in release, which is where this would matter.

 If a failed assertion could print out sensitive information, 
 then just don't put the information directly in the assertion. 
 Heck, anyone paranoid enough about it could create a special 
 secureAssert function or something that did something like

 void secureAssert(E)(lazy E expression,
                      lazy string msg,
                      string file = __FILE__,
                      size_T line = __LINE__)
 {
     version(assert)
     {
         immutable result = cast(bool)expression();
         if(!result)
             throw new AssertError(msg, file, line);
     }
 }

 I see no reason to make the debugging tools that we have worse 
 because of fear of someone who's writing code that processes 
 sensitive information. That code just needs to be written in a 
 way that's not going to leak the information. As Dmitry points 
 out, there are plenty of other ways for the information to be 
 leaked. So, not improving assertions due to security concerns 
 really isn't a valid approach IMHO.

 Now, there are other reasons why we might not want to (e.g. if 
 the performance were worse, you might not want to make 
 assertions print additional information), but I think that it's 
 far better for code that needs to be that secure to do its own 
 thing than to hamper everyone over the concerns of the minority.


 At the very least there should be a compiler switch to turn 
 assert-printing on/off


 I expect that most folks who would want to avoid have 
 assertions printing out sensitive information would still want 
 assertions that didn't relate to sensitive information to 
 print. A compiler switch is a pretty blunt instrument, 
 especially when someone could write their own code which 
 asserts without showing any sensitive information while still 
 testing that information.

 - Jonathan M Davis


All reasonable points, so I guess it's not such a bad thing after 
all. I just wanted to be sure that security was considered here.
Oct 01 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/01/2015 04:46 AM, John Colvin wrote:

 Checks involving sensitive data after processing can definitely be a
 check of program logic.

 Sensitive data enters program
 Sensitive data is checked using enforce
 Sensitive data is passed to another function, but something goes wrong
 (not enough checking before, wrong function called, HDD dies, someone
 trips over a network cable), an assert is triggered, the sensitive data
 spills to stderr.


I think worrying about this would be an exaggeration.


 At the very least there should be a compiler switch to turn
 assert-printing on/off


Please, no. Keep it simple and automatic. Zero-work improvement for the 
client.

Printing good messages in assertions is a good idea. For most ideas of 
sufficient complexity, situations can be imagined in which there are 
disadvantages. It has happened before in our community that good ideas 
didn't get worked on because a few folks dwelt on the disadvantages with 
too much vividness.

Whoever wants to work on better assert expression printing: make sure 
you specify which grammar constructs are supported, and how the parts 
involved are printed. Expressing semantics via lowering would be great. 
Write a DIP, discuss, implement. I'll have your six.


Andrei
Oct 01 2015
next sibling parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 1 October 2015 at 14:37:55 UTC, Andrei Alexandrescu 
wrote:

 Whoever wants to work on better assert expression printing: 
 make sure you specify which grammar constructs are supported, 
 and how the parts involved are printed. Expressing semantics 
 via lowering would be great. Write a DIP, discuss, implement. 
 I'll have your six.


I registered a user named `nordlow` at the D Wiki but I can't 
find a way to write a DIP.

Help please.
Oct 01 2015
parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 1 October 2015 at 16:35:51 UTC, Per Nordlöw wrote:

 Help please.


I figured it out.
Oct 01 2015
prev sibling parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 1 October 2015 at 14:37:55 UTC, Andrei Alexandrescu 
wrote:

 Whoever wants to work on better assert expression printing: 
 make sure you specify which grammar constructs are supported, 
 and how the parts involved are printed. Expressing semantics 
 via lowering would be great. Write a DIP, discuss, implement. 
 I'll have your six.


 Andrei


A first version:

http://wiki.dlang.org/DIP83
Oct 01 2015
next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Thursday, 1 October 2015 at 17:19:39 UTC, Per Nordlöw wrote:

 A first version:

 http://wiki.dlang.org/DIP83


Looks good! But I lack the knowledge to comment on the 
implementation details for the compiler mentioned in the DIP.

Bikesheading: could you change "being" in "([1,2,3][2] being 3) 
!= ([1,2,4][2] being 4)" and the other examples to "is"? Using 
"is" is much more natural sounding English.
Oct 01 2015
parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 1 October 2015 at 17:33:51 UTC, Jack Stouffer wrote:

 Bikesheading: could you change "being" in "([1,2,3][2] being 3) 
 != ([1,2,4][2] being 4)" and the other examples to "is"?


Done.
Oct 01 2015
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/01/2015 01:19 PM, Per Nordlöw wrote:

 On Thursday, 1 October 2015 at 14:37:55 UTC, Andrei Alexandrescu wrote:

 Whoever wants to work on better assert expression printing: make sure
 you specify which grammar constructs are supported, and how the parts
 involved are printed. Expressing semantics via lowering would be
 great. Write a DIP, discuss, implement. I'll have your six.


 Andrei


 A first version:

 http://wiki.dlang.org/DIP83


A good start. Few thoughts:

* I don't think we need a new flag, just make the new behavior work.

* Should the lowering happen only on the function called if the 
assertion actually fails? Then no more need for laziness and other 
complications.

* Extend to other expressions (!=, ordering etc).


Andrei
Oct 01 2015
next sibling parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 1 October 2015 at 19:04:51 UTC, Andrei Alexandrescu 
wrote:

 * I don't think we need a new flag, just make the new behavior 
 work.


So you mean that extra diagnostics should kick in when extra 
overloads are made visible via import of, for instance, 
`core.assert`?


 * Should the lowering happen only on the function called if the 
 assertion actually fails? Then no more need for laziness and 
 other complications.


Could explain what you mean by *lowering*, please?

I'm currently unsure whether
     in L lhs
or
     lazy L lhs

should be used and whether or not we should use
     version(assert)

See the added example at
http://wiki.dlang.org/DIP83


 * Extend to other expressions (!=, ordering etc).


How should we categorize expressions? Like this
- Unary: assert(x UNOP y)
- Binary: assert(x BINOP y)
- Function Calls: assert(f(x,y))
- Other: assert(x)

Does it suffice to just mention these or should I be explicit 
exactly about which operators for each category that should be 
included?
Oct 01 2015
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-10-02 08:31, Per Nordlöw wrote:


 Could explain what you mean by *lowering*, please?


"lowering" means that a feature is implemented using another feature. 
For example, "foreach" is lowered to a for-loop:

foreach(i ; 0 .. 10){}

Is lowered to:

for (int i = 0; i < 10; i++) {}

The compiler rewrites the AST of the foreach-loop to the same AST that 
the corresponding for-loop would have. After that lowering step, the 
compiler doesn't need to know anything about foreach-loops.

-- 
/Jacob Carlborg
Oct 02 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/02/2015 02:31 AM, Per Nordlöw wrote:

 On Thursday, 1 October 2015 at 19:04:51 UTC, Andrei Alexandrescu wrote:

 * I don't think we need a new flag, just make the new behavior work.


 So you mean that extra diagnostics should kick in when extra overloads
 are made visible via import of, for instance, `core.assert`?


 * Should the lowering happen only on the function called if the
 assertion actually fails? Then no more need for laziness and other
 complications.


 Could explain what you mean by *lowering*, please?


Rewrite proposed constructs into simpler constructs that already exist. 
That way we don't need to invent semantics, just "see lowering".


 I'm currently unsure whether
      in L lhs
 or
      lazy L lhs


That's why I think it's better to go with calling the function only 
after the assertion has failed. E.g.,

assert(e1 == e2)

could be lowered into:

{
   auto a = e1, b = e2;
   if (a == b) return;
   onAssertFailed!"=="(a, b, __FILE__, __LINE__, __FUNCTION__, __MODULE__);
}()

or something similar (I'm glossing over the details). Point is the 
expressions are only evaluated once and then passed into onAssertFailed.

There'd be a default definition of onAssertFailed in object.d.


 should be used and whether or not we should use
      version(assert)


Yah, all expansions related to assert() are only in effect if assertions 
are used. Otherwise, assert disappears and that's not configurable.


 See the added example at
 http://wiki.dlang.org/DIP83


 * Extend to other expressions (!=, ordering etc).


 How should we categorize expressions? Like this
 - Unary: assert(x UNOP y)
 - Binary: assert(x BINOP y)
 - Function Calls: assert(f(x,y))
 - Other: assert(x)

 Does it suffice to just mention these or should I be explicit exactly
 about which operators for each category that should be included?


I think you should use the names of the grammatical constructs, e.g. 
http://dlang.org/expression.html#EqualExpression.


Andrei
Oct 02 2015
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 2 October 2015 at 11:19:51 UTC, Andrei Alexandrescu 
wrote:

 assert(e1 == e2)

 could be lowered into:

 {
   auto a = e1, b = e2;
   if (a == b) return;
   onAssertFailed!"=="(a, b, __FILE__, __LINE__, __FUNCTION__, 
 __MODULE__);
 }()


So lowering is kind of like macro expansion for AST-nodes, then?

Is DMD clever enough to avoid trigger postblits for


   auto a = e1, b = e2;
   if (a == b) return;


? Or is that part of the question whether this will work?

I guess we only need on symbol name for `onAssertFailed` then 
instead of `assertBinOp` and `assertUnOp`, right?
Oct 02 2015
next sibling parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 2 October 2015 at 12:15:13 UTC, Per Nordlöw wrote:

 I guess we only need on symbol name for `onAssertFailed` then 
 instead of `assertBinOp` and `assertUnOp`, right?


And two overloads

Binary case:

     onAssertFailed(string op)(e1, e2, __FILE__, ...)

Unary case:

     onAssertFailed(string op)(e, __FILE__, ...)

I presume?

Because number of arguments to each overload will be fixed, right?

What about the case

assert(f(expr))
assert(symbol)

Should `op` be empty in that casesor should we use yet another 
overload

     onAssertFailed(e, __FILE__, ...)

for that case?
Oct 02 2015
next sibling parent reply Dicebot <public dicebot.lv> writes:
On Friday, 2 October 2015 at 12:27:11 UTC, Per Nordlöw wrote:

 What about the case

 assert(f(expr))
 assert(symbol)

 Should `op` be empty in that casesor should we use yet another 
 overload

     onAssertFailed(e, __FILE__, ...)

 for that case?


empty op for unary overload
Oct 02 2015
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/02/2015 08:31 AM, Dicebot wrote:

 On Friday, 2 October 2015 at 12:27:11 UTC, Per Nordlöw wrote:

 What about the case

 assert(f(expr))
 assert(symbol)

 Should `op` be empty in that casesor should we use yet another overload

     onAssertFailed(e, __FILE__, ...)

 for that case?


 empty op for unary overload


Even better. -- Andrei
Oct 02 2015
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/02/2015 08:27 AM, Per Nordlöw wrote:

 On Friday, 2 October 2015 at 12:15:13 UTC, Per Nordlöw wrote:

 I guess we only need on symbol name for `onAssertFailed` then instead
 of `assertBinOp` and `assertUnOp`, right?


 And two overloads

 Binary case:

      onAssertFailed(string op)(e1, e2, __FILE__, ...)

 Unary case:

      onAssertFailed(string op)(e, __FILE__, ...)

 I presume?


Sounds good.


 Because number of arguments to each overload will be fixed, right?

 What about the case

 assert(f(expr))
 assert(symbol)

 Should `op` be empty in that casesor should we use yet another overload

      onAssertFailed(e, __FILE__, ...)

 for that case?


I'd say lower the same as (e !is 0) for numerics and (e !is null) for 
the others.


Andrei
Oct 02 2015
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/02/2015 08:15 AM, Per Nordlöw wrote:

 On Friday, 2 October 2015 at 11:19:51 UTC, Andrei Alexandrescu wrote:

 assert(e1 == e2)

 could be lowered into:

 {
   auto a = e1, b = e2;
   if (a == b) return;
   onAssertFailed!"=="(a, b, __FILE__, __LINE__, __FUNCTION__,
 __MODULE__);
 }()


 So lowering is kind of like macro expansion for AST-nodes, then?


Not sure what you mean. The code up there will be replaced with the code 
down there :o).


 Is DMD clever enough to avoid trigger postblits for


   auto a = e1, b = e2;
   if (a == b) return;


 ? Or is that part of the question whether this will work?


Ah, interesting. There is a means in the compiler to generate ref 
variables, which is not accessible for user code. But perhaps that's not 
necessary if we do the lowering as such:

(auto ref a, auto ref b) {
   if (a == b) return;
   onAssertFailed!"=="(a, b, __FILE__, __LINE__, __FUNCTION__, __MODULE__);
}(e1, e2)

So that evaluates the two expressions and avoids creating copies for 
lvalues.


 I guess we only need on symbol name for `onAssertFailed` then instead of
 `assertBinOp` and `assertUnOp`, right?


Probably a judgment call. I'd start with one and see what happens.


Andrei
Oct 02 2015
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-10-02 14:15, Per Nordlöw wrote:


 So lowering is kind of like macro expansion for AST-nodes, then?

 Is DMD clever enough to avoid trigger postblits for


The compiler just rewrites the AST.

-- 
/Jacob Carlborg
Oct 02 2015
parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 2 October 2015 at 14:08:06 UTC, Jacob Carlborg wrote:

 On 2015-10-02 14:15, Per Nordlöw wrote:


 So lowering is kind of like macro expansion for AST-nodes, 
 then?

 Is DMD clever enough to avoid trigger postblits for


 The compiler just rewrites the AST.


So you mean that no postblits are called and we should go with 
Andreis first lowering proposal?
Oct 02 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/02/2015 10:30 AM, Nordlöw wrote:

 On Friday, 2 October 2015 at 14:08:06 UTC, Jacob Carlborg wrote:

 On 2015-10-02 14:15, Per Nordlöw wrote:


 So lowering is kind of like macro expansion for AST-nodes, then?

 Is DMD clever enough to avoid trigger postblits for


 The compiler just rewrites the AST.


 So you mean that no postblits are called and we should go with Andreis
 first lowering proposal?


My first proposed lowering creates more copies than needed. My second 
one is therefore probably better, without resorting to compiler magic. 
-- Andrei
Oct 02 2015
parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 2 October 2015 at 14:54:08 UTC, Andrei Alexandrescu 
wrote:

 My first proposed lowering creates more copies than needed. My 
 second one is therefore probably better, without resorting to 
 compiler magic. -- Andrei


I update the spec. Could you take a look?

I'll update the operators, next.
Oct 05 2015
prev sibling parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 1 October 2015 at 19:04:51 UTC, Andrei Alexandrescu 
wrote:

 * I don't think we need a new flag, just make the new behavior 
 work.


Should I remove all mentioning of extra compiler flags?
Oct 05 2015
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Thursday, 1 October 2015 at 17:19:39 UTC, Per Nordlöw wrote:

 On Thursday, 1 October 2015 at 14:37:55 UTC, Andrei 
 Alexandrescu wrote:

 Whoever wants to work on better assert expression printing: 
 make sure you specify which grammar constructs are supported, 
 and how the parts involved are printed. Expressing semantics 
 via lowering would be great. Write a DIP, discuss, implement. 
 I'll have your six.


 Andrei


 A first version:

 http://wiki.dlang.org/DIP83


I like the idea, I just have a practical question on 
std.experimental.testing: should it go ahead as-is? Should the 
custom assertions be removed and the rest kept? Should they go in 
as-is and then deleted when/if this DIP is implemented?

Atila
Oct 02 2015
next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, 2 October 2015 at 09:32:23 UTC, Atila Neves wrote:

 On Thursday, 1 October 2015 at 17:19:39 UTC, Per Nordlöw wrote:

 On Thursday, 1 October 2015 at 14:37:55 UTC, Andrei 
 Alexandrescu wrote:

 Whoever wants to work on better assert expression printing: 
 make sure you specify which grammar constructs are supported, 
 and how the parts involved are printed. Expressing semantics 
 via lowering would be great. Write a DIP, discuss, implement. 
 I'll have your six.


 Andrei


 A first version:

 http://wiki.dlang.org/DIP83


 I like the idea, I just have a practical question on 
 std.experimental.testing: should it go ahead as-is? Should the 
 custom assertions be removed and the rest kept? Should they go 
 in as-is and then deleted when/if this DIP is implemented?


If we're going to be improving the built-in assertions, then I 
don't think that it makes sense to add custom assertions to 
Phobos, and I definitely don't like the idea of adding them with 
the idea that we'll just be deprecating them later. If it turns 
out that that isn't going work (Walter rejected it the last time 
it was done, but maybe that will go differently this time - 
particularly since Andrei is very much in favor of it), then we 
can add the custom assertions later.

Personally, I think that the parts of this module which really 
matter are the unittest names (though I still wish that we'd just 
put that in the language itself - particularly since each 
unittest block already has a corresponding function that's named 
after where it is and it really shouldn't be a big deal to just 
allow the user to provide a name to use instead) and the parts 
about parallelizing unittest blocks. Even if we don't end up with 
improvements to the built-in assertions, the custom assertions 
are just gravy that makes it so that you're less like to have to 
go change your unit tests to print more info when they fail. So, 
I don't think that it's a big deal if they get delayed while we 
wait and see if we can get the built-in assertions improved, and 
if the built-in assertions do get improved, then we'll never need 
the custom ones anyway.

- Jonathan M Davis
Oct 02 2015
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-10-02 11:32, Atila Neves wrote:


 I like the idea, I just have a practical question on
 std.experimental.testing: should it go ahead as-is?


Yes, as-is. That's the whole point of std.experimental.

-- 
/Jacob Carlborg
Oct 02 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/02/2015 05:32 AM, Atila Neves wrote:

 I like the idea, I just have a practical question on
 std.experimental.testing: should it go ahead as-is? Should the custom
 assertions be removed and the rest kept? Should they go in as-is and
 then deleted when/if this DIP is implemented?


As long as it's in experimental, proceed as you find fit. We can decide 
later whether changes are in order. -- Andrei
Oct 02 2015
parent reply Atila Neves <atila.neves gmail.com> writes:
On Friday, 2 October 2015 at 11:22:00 UTC, Andrei Alexandrescu 
wrote:

 On 10/02/2015 05:32 AM, Atila Neves wrote:

 I like the idea, I just have a practical question on
 std.experimental.testing: should it go ahead as-is? Should the 
 custom
 assertions be removed and the rest kept? Should they go in 
 as-is and
 then deleted when/if this DIP is implemented?


 As long as it's in experimental, proceed as you find fit. We 
 can decide later whether changes are in order. -- Andrei


That's what I was hoping for. Good, unless anybody has comments 
to make on the current state of the PR, I'll leave everything as 
it is now.

Atila
Oct 02 2015
parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 2 October 2015 at 11:54:31 UTC, Atila Neves wrote:

 That's what I was hoping for. Good, unless anybody has comments 
 to make on the current state of the PR, I'll leave everything 
 as it is now.


Fine with me :)
Oct 02 2015
prev sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Sep 29, 2015 at 11:32:14PM +0200, Andrej Mitrovic via Digitalmars-d
wrote:

 On 9/29/15, John Colvin via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 Someone will write something like this:

 assert(plainPassword == plainPassword.toLower());

 and plaintext passwords will end up in stderr.


 
 If you have plaintext passwords stored anywhere you are already
 screwed. ;)


Point. :-)


T

-- 
There is no gravity. The earth sucks.
Sep 29 2015
prev sibling next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Sep 29, 2015 at 09:02:40PM +0000, Nordlöw via Digitalmars-d wrote:
[...]

 2. Is it possible to from within DMD generate expressions that do
 
 `import std.stdio : write`
 
 and then calls write on the `lhs` and `rsh`...or this a completely
 wrong approach to solving this problem?


I dunno about right or wrong approaches, but something like this would
make the compiler dependent on Phobos, which is a big no-no for those
embedded OS people who need to be able to plug in their own runtime
without being forced to implement what Phobos implements, which may lead
to prohibitive bloat for embedded software.

The most that the compiler can depend on is druntime, which,
fortunately, does depend on the C library (so far), so perhaps you
could get away with `import std.c.stdio : printf;` and using C-style
printf() calls to print the output. The existing stacktrace dump
functions may already be using printf from the C library, and the
standard C library seems common enough even on embedded platforms that
you can probably get away with it.

Alternatively, maybe what you really want is for the compiler to emit a
call to a druntime function that does the printing, then the people who
need to replace druntime with their own lightweight version can simply
implement this function, and things would Just Work.


T

-- 
Любишь кататься - люби и саночки возить.
Sep 29 2015
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, 29 September 2015 at 21:02:42 UTC, Nordlöw wrote:

 As a follow-up to

 https://github.com/D-Programming-Language/phobos/pull/3207#issuecomment-144073495

 I starting digging in DMD for logic controlling behaviour of 
 assert(), especially whether it's possible to add automatic 
 printing of `lhs` and `rhs` upon assertion failure if 
 `AssertExp` is a binary expression say `lhs == rhs`.

 After grepping for `AssertExp` the only possible place I could 
 think of was

 ToElemVisitor::visit(AssertExp *ae)

 inside

 elem *toElem(Expression *e, IRState *irs)

 in file e2ir.c.

 Questions:

 1. Is this the right place where this lhs-rhs-printing logic 
 should be added? If so could somebody tell me how to make this 
 happen?

 2. Is it possible to from within DMD generate expressions that 
 do

 `import std.stdio : write`

 and then calls write on the `lhs` and `rsh`...or this a 
 completely wrong approach to solving this problem?


Several years ago, I proposed a really nice function called 
assertPred which did all this kind of stuff for you, and it 
supported a bunch of different operations - e.g. 
assertPred!"=="(a, b), assertPred!"+=(a, b, result), 
assertPred!"opCmp"(a, b). But it was ultimately rejected with the 
decision being that we'd improve assert to do this instead (which 
lost out on some of the fancier overloads of assertPred but would 
be great for the common case). However, after some work was 
actually done to implement that, it was finally decided that it 
was too expensive to put it into assert:

https://issues.dlang.org/show_bug.cgi?id=5547

So, the proposed alternative to assertPred which got it rejected 
ultimately got rejected itself.

But as great as assertPred was, it turns out that turning all of 
your checks in unit tests into templated functions rather than 
just using assert, results in a lot of template bloat, and unit 
tests require more memory and take longer to compile. So, I've 
pretty much gotten to the point where I think that it's just 
simplest to use assert as-is and then go back and tweak your unit 
test to print out the information you need if there's a failure. 
Any information in a unit test that isn't reproducible should 
already have been being printed on failure anyway (e.g. the seed 
value for the random number generator), so it really shouldn't be 
a big deal to tweak a test to print out what's going on. As nice 
as it might be to have the assertion print out better 
information, I don't think that it's worth adding a function just 
for that. It's just too expensive in terms of compilation time 
and memory.

- Jonathan M Davis
Sep 29 2015
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 09/29/2015 05:02 PM, Nordlöw wrote:

 As a follow-up to

 https://github.com/D-Programming-Language/phobos/pull/3207#issuecomment-144073495


 I starting digging in DMD for logic controlling behaviour of assert(),


Great idea. While at it could you please make sure the format is 
compatible with that of compile-time errors? Errors during compilation 
and unit testing are close in the edit-build-test continuum. Thanks! -- 
Andrei
Sep 29 2015
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 29 September 2015 at 21:02:42 UTC, Nordlöw wrote:

 As a follow-up to

 https://github.com/D-Programming-Language/phobos/pull/3207#issuecomment-144073495

 I starting digging in DMD for logic controlling behaviour of 
 assert(), especially whether it's possible to add automatic 
 printing of `lhs` and `rhs` upon assertion failure if 
 `AssertExp` is a binary expression say `lhs == rhs`.

 After grepping for `AssertExp` the only possible place I could 
 think of was

 ToElemVisitor::visit(AssertExp *ae)

 inside

 elem *toElem(Expression *e, IRState *irs)

 in file e2ir.c.

 Questions:

 1. Is this the right place where this lhs-rhs-printing logic 
 should be added? If so could somebody tell me how to make this 
 happen?

 2. Is it possible to from within DMD generate expressions that 
 do

 `import std.stdio : write`

 and then calls write on the `lhs` and `rsh`...or this a 
 completely wrong approach to solving this problem?


I do think wiring this in the compiler is probably not the right 
way forward. Most language have some library functions like :

expect(exprssion).toBeTrue();
expect(function/delegate).toThrow!ExceptionType

And so on.

This allow to get nice error and do not need wiring int he 
language. This is extensible and all. I'd rather have something 
like this.
Sep 29 2015
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-09-30 05:45, deadalnix wrote:

 On Tuesday, 29 September 2015 at 21:02:42 UTC, Nordlöw wrote:

 As a follow-up to

 https://github.com/D-Programming-Language/phobos/pull/3207#issuecomment-144073495





 I do think wiring this in the compiler is probably not the right way
 forward. Most language have some library functions like :

 expect(exprssion).toBeTrue();
 expect(function/delegate).toThrow!ExceptionType

 And so on.

 This allow to get nice error and do not need wiring int he language.
 This is extensible and all. I'd rather have something like this.


That's already what the above linked pull request does.

-- 
/Jacob Carlborg
Sep 29 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 09/29/2015 11:45 PM, deadalnix wrote:

 On Tuesday, 29 September 2015 at 21:02:42 UTC, Nordlöw wrote:

 As a follow-up to

 https://github.com/D-Programming-Language/phobos/pull/3207#issuecomment-144073495


 I starting digging in DMD for logic controlling behaviour of assert(),
 especially whether it's possible to add automatic printing of `lhs`
 and `rhs` upon assertion failure if `AssertExp` is a binary expression
 say `lhs == rhs`.

 After grepping for `AssertExp` the only possible place I could think
 of was

 ToElemVisitor::visit(AssertExp *ae)

 inside

 elem *toElem(Expression *e, IRState *irs)

 in file e2ir.c.

 Questions:

 1. Is this the right place where this lhs-rhs-printing logic should be
 added? If so could somebody tell me how to make this happen?

 2. Is it possible to from within DMD generate expressions that do

 `import std.stdio : write`

 and then calls write on the `lhs` and `rsh`...or this a completely
 wrong approach to solving this problem?


 I do think wiring this in the compiler is probably not the right way
 forward. Most language have some library functions like :

 expect(exprssion).toBeTrue();
 expect(function/delegate).toThrow!ExceptionType

 And so on.

 This allow to get nice error and do not need wiring int he language.
 This is extensible and all. I'd rather have something like this.


I encourage making assert smarter seeing (a) it's already used 
everywhere so the benefits will come for free and (b) it's a built-in. 
-- Andrei
Sep 30 2015
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09/30/2015 03:10 PM, Andrei Alexandrescu wrote:

 I encourage making assert smarter seeing (a) it's already used
 everywhere so the benefits will come for free  and (b) it's a built-in.
 -- Andrei


About (b): I'm surprised to see that you seem to have so fundamentally 
changed your attitude towards magical semantics for built-ins.
Oct 01 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/01/2015 06:41 PM, Timon Gehr wrote:

 On 09/30/2015 03:10 PM, Andrei Alexandrescu wrote:

 I encourage making assert smarter seeing (a) it's already used
 everywhere so the benefits will come for free  and (b) it's a built-in.
 -- Andrei


 About (b): I'm surprised to see that you seem to have so fundamentally
 changed your attitude towards magical semantics for built-ins.


I haven't - I still think making "assert" a built-in and ascribing a 
keyword to it was a minor mistake. But then that sail has shipped, so 
let's make the best use of the situation. -- Andrei
Oct 01 2015
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/02/2015 01:33 AM, Andrei Alexandrescu wrote:

 On 10/01/2015 06:41 PM, Timon Gehr wrote:

 On 09/30/2015 03:10 PM, Andrei Alexandrescu wrote:

 I encourage making assert smarter seeing (a) it's already used
 everywhere so the benefits will come for free  and (b) it's a built-in.
 -- Andrei


 About (b): I'm surprised to see that you seem to have so fundamentally
 changed your attitude towards magical semantics for built-ins.


 I haven't - I still think making "assert" a built-in and ascribing a
 keyword to it was a minor mistake. But then that sail has shipped, so
 let's make the best use of the situation. -- Andrei


Ok, but if assert gets special error printing capabilities that are not 
available at the same level of convenience to e.g. enforce, then this is 
a roughly analogous situation to having e.g. B[] : const(A)[] for B : A, 
which cannot currently be simulated in the library in a satisfactory 
way. Those situations usually add friction. They tend to result in 
frustration and often culminate in proposals for new, often ad-hoc 
language features. IMHO, there ought to be a better way, but I don't 
have a very strong opinion about this particular case.
Oct 01 2015
parent reply deadalnix <deadalnix gmail.com> writes:
On Friday, 2 October 2015 at 00:05:33 UTC, Timon Gehr wrote:

 On 10/02/2015 01:33 AM, Andrei Alexandrescu wrote:

 On 10/01/2015 06:41 PM, Timon Gehr wrote:

 On 09/30/2015 03:10 PM, Andrei Alexandrescu wrote:

 I encourage making assert smarter seeing (a) it's already 
 used
 everywhere so the benefits will come for free  and (b) it's 
 a built-in.
 -- Andrei


 About (b): I'm surprised to see that you seem to have so 
 fundamentally
 changed your attitude towards magical semantics for built-ins.


 I haven't - I still think making "assert" a built-in and 
 ascribing a
 keyword to it was a minor mistake. But then that sail has 
 shipped, so
 let's make the best use of the situation. -- Andrei


 Ok, but if assert gets special error printing capabilities that 
 are not available at the same level of convenience to e.g. 
 enforce, then this is a roughly analogous situation to having 
 e.g. B[] : const(A)[] for B : A, which cannot currently be 
 simulated in the library in a satisfactory way. Those 
 situations usually add friction. They tend to result in 
 frustration and often culminate in proposals for new, often 
 ad-hoc language features. IMHO, there ought to be a better way, 
 but I don't have a very strong opinion about this particular 
 case.


On the other hand, changing the language to provide access to 
this as library would require something macro like, and both 
Andrei and Walter seems to be really against it.

As we are on assert, 2 things :
  - Before considering adding more magic, can we get line numbers 
in stack traces ? It really seems to me like improving the 
message no stack trace is available is not focusing on impact.
  - assert already have a fair amount of magic, notably assert(0) 
change the way control flow works. Having it as an expression is 
making everything very convoluted for no reason.
Oct 01 2015
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 10/02/2015 02:47 AM, deadalnix wrote:

 ...
   - assert already have a fair amount of magic, notably assert(0) change
 the way control flow works.


Yes, and it is not stellar design. Generally, if it is not possible to 
wrap a statement or expression in a function without changing its 
semantics, the language provides too much magic and insufficient 
abstraction capabilities. (Of course, in practice, there might be 
legitimate reasons to deviate from this principle, but less often than 
one might think.)


 Having it as an expression is making
 everything very convoluted for no reason.


What is the problem with assert expressions?
Oct 02 2015
prev sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Thursday, 1 October 2015 at 23:33:39 UTC, Andrei Alexandrescu 
wrote:

 I haven't - I still think making "assert" a built-in and 
 ascribing a keyword to it was a minor mistake. But then that 
 sail has shipped, so let's make the best use of the situation. 
 -- Andrei


Wouldn't that mean that every file that uses unit tests would 
have to import std.assert or something? So pretty much every file 
would be importing it anyway.
Oct 01 2015
parent reply David Nadlinger <code klickverbot.at> writes:
On Friday, 2 October 2015 at 00:25:18 UTC, jmh530 wrote:

 Wouldn't that mean that every file that uses unit tests would 
 have to import std.assert or something? So pretty much every 
 file would be importing it anyway.


It could, for example, still be in object.

  — David
Oct 01 2015
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, 2 October 2015 at 00:33:13 UTC, David Nadlinger wrote:

 On Friday, 2 October 2015 at 00:25:18 UTC, jmh530 wrote:

 Wouldn't that mean that every file that uses unit tests would 
 have to import std.assert or something? So pretty much every 
 file would be importing it anyway.


 It could, for example, still be in object.


Yeah. We already have that with several symbols (e.g. size_t and 
string), and arguably we should do it with a few others (like the 
range primitives for arrays). So, if we'd made new and assert 
into library functions instead of built-ins like we arguably 
should have been, then they'd probably be in object.d, but that 
shipped has long since sailed.

- Jonathan M Davis
Oct 01 2015
prev sibling next sibling parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 29 September 2015 at 21:02:42 UTC, Nordlöw wrote:

 As a follow-up to

 https://github.com/D-Programming-Language/phobos/pull/3207#issuecomment-144073495


I added a long comment about a new more flexible solution to this 
problem:

https://github.com/D-Programming-Language/phobos/pull/3207#issuecomment-144701371
Oct 01 2015
parent reply Atila Neves <atila.neves gmail.com> writes:
On Thursday, 1 October 2015 at 11:51:29 UTC, Per Nordlöw wrote:

 On Tuesday, 29 September 2015 at 21:02:42 UTC, Nordlöw wrote:

 As a follow-up to

 https://github.com/D-Programming-Language/phobos/pull/3207#issuecomment-144073495


 I added a long comment about a new more flexible solution to 
 this problem:

 https://github.com/D-Programming-Language/phobos/pull/3207#issuecomment-144701371


Would `onAssertFailed` have an implementation in druntime? If one 
is included with D after this DIP is implemented, how would one 
override the default behaviour? Do I understand it correctly that 
`assert(a == b)` and `assert(a != b)` would call different 
overloads? The reason I ask about that last one is because 
there's no overloading of `!=` in D.

Atila
Oct 26 2015
parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 26 October 2015 at 15:17:37 UTC, Atila Neves wrote:

 Would `onAssertFailed` have an implementation in druntime?


Yes, that's my plan.


 If one is included with D after this DIP is implemented, how 
 would one override the default behaviour?



 Do I understand it correctly that `assert(a == b)` and 
 `assert(a != b)` would call different overloads?


That depends on how we want this to work.


 The reason I ask about that last one is because there's no 
 overloading of `!=` in D.

 Atila


Ahh, good catch.

Eventhough compiler rewrites

     X != Y

into

     !(X == Y)

and we cannot explicit overload opBinary!"!=" there is nothing 
stopping us allowing

     onBinaryAssert!"!="

in the rewriter.

I'll update

http://wiki.dlang.org/DIP83

with a proposed solution for `!=`-lowering.
Oct 26 2015
next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 26 October 2015 at 17:36:37 UTC, Nordlöw wrote:

 On Monday, 26 October 2015 at 15:17:37 UTC, Atila Neves wrote:

 Would `onAssertFailed` have an implementation in druntime?


 Yes, that's my plan.


 If one is included with D after this DIP is implemented, how 
 would one override the default behaviour?




I haven't thought about that in detail. One way would be to have 
a default implementation in druntime, say in `core.assert` that 
triggers pretty pretting when imported. For this work in a 
convenient way, it could be convenient to move logic from 
std.conv to, say, `core.conv`. `core.assert` could then import 
`core.conv` in order to reuse toString-conversion logic for, say, 
builtin types and types that have the member `string toString`.

Maybe we need to propose a set of alternatives and do a vote 
about how to solve this.
Oct 26 2015
parent Atila Neves <atila.neves gmail.com> writes:
On Monday, 26 October 2015 at 17:50:13 UTC, Nordlöw wrote:

 On Monday, 26 October 2015 at 17:36:37 UTC, Nordlöw wrote:

 On Monday, 26 October 2015 at 15:17:37 UTC, Atila Neves wrote:

 [...]


 Yes, that's my plan.


 [...]




 I haven't thought about that in detail. One way would be to 
 have a default implementation in druntime, say in `core.assert` 
 that triggers pretty pretting when imported. For this work in a 
 convenient way, it could be convenient to move logic from 
 std.conv to, say, `core.conv`. `core.assert` could then import 
 `core.conv` in order to reuse toString-conversion logic for, 
 say, builtin types and types that have the member `string 
 toString`.

 Maybe we need to propose a set of alternatives and do a vote 
 about how to solve this.


I think druntime should at least be mentioned in the DIP.

Atila
Oct 26 2015
prev sibling next sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 26 October 2015 at 17:36:37 UTC, Nordlöw wrote:

 I'll update

 http://wiki.dlang.org/DIP83

 with a proposed solution for `!=`-lowering.


Update along with other changes...done.
Oct 26 2015
prev sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 26 October 2015 at 17:36:37 UTC, Nordlöw wrote:

 Eventhough compiler rewrites

     X != Y

 into

     !(X == Y)

 and we cannot explicit overload opBinary!"!=" there is nothing 
 stopping us allowing

     onBinaryAssert!"!="

 in the rewriter.


Correction:

there is nothing stopping us from allowing

     onAssertFailed!"!="

to be defined as long as the compiler lowering logic can do the 
correct rewrite.
Oct 26 2015
parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 26 October 2015 at 19:17:13 UTC, Nordlöw wrote:

 there is nothing stopping us from allowing

     onAssertFailed!"!="

 to be defined as long as the compiler lowering logic can do the 
 correct rewrite.


For further details, see also:

http://wiki.dlang.org/DIP83#Non-Equality_Operator_Lowering
Oct 26 2015
prev sibling parent Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 29 September 2015 at 21:02:42 UTC, Nordlöw wrote:

 As a follow-up to

 https://github.com/D-Programming-Language/phobos/pull/3207#issuecomment-144073495

 I starting digging in DMD for logic controlling behaviour of 
 assert(), especially whether it's possible to add automatic 
 printing of `lhs` and `rhs` upon assertion failure if 
 `AssertExp` is a binary expression say `lhs == rhs`.

 After grepping for `AssertExp` the only possible place I could 
 think of was

 ToElemVisitor::visit(AssertExp *ae)

 inside

 elem *toElem(Expression *e, IRState *irs)

 in file e2ir.c.

 Questions:

 1. Is this the right place where this lhs-rhs-printing logic 
 should be added? If so could somebody tell me how to make this 
 happen?

 2. Is it possible to from within DMD generate expressions that 
 do

 `import std.stdio : write`

 and then calls write on the `lhs` and `rsh`...or this a 
 completely wrong approach to solving this problem?


More thoughts on this: what about ranges and testing for 
equality? It would be helpful if either this DIP made the 
compiler recognise and deal with `std.algorithm.equal` 
differently or recognise that on one or both sides of `==` 
there's a range.

I'm thinking of all the different `shouldXXX` functions I wrote 
for unit-threaded. Equality is by far the most common thing to 
assert on in a test, but there are other useful ones that need to 
be considered. Two that come to mind right now are asserting that 
a range contains an element and that two ranges are the same 
ignoring ordering.

For the former, `assert(range.canFind(elt))` would be hard to 
deal with in this DIP. I don't see a better way, however, since 
`in` only works for AAs.

For the latter... I don't know either.

Atila
Oct 31 2015