www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Another use case for __traits(docComment)

reply Adam D. Ruppe <destructionator gmail.com> writes:
Writing code to wrap some Phobos functions for a script language 
and need documentation to be displayed at runtime...

Would be really nice to just say if(other_docs.length == 0) 
return __traits(docComment, item).

A UDA can populate other_docs in some functions, but when 
wrapping libraries you can't add that. Having the doc comment 
available through reflection means we can reuse these existing 
items.
Nov 08 2019
next sibling parent FeepingCreature <feepingcreature gmail.com> writes:
On Friday, 8 November 2019 at 14:00:03 UTC, Adam D. Ruppe wrote:

 Writing code to wrap some Phobos functions for a script 
 language and need documentation to be displayed at runtime...

 Would be really nice to just say if(other_docs.length == 0) 
 return __traits(docComment, item).

 A UDA can populate other_docs in some functions, but when 
 wrapping libraries you can't add that. Having the doc comment 
 available through reflection means we can reuse these existing 
 items.


Yes, please!!
Nov 11 2019
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/8/19 9:00 AM, Adam D. Ruppe wrote:

 Writing code to wrap some Phobos functions for a script language and 
 need documentation to be displayed at runtime...
 
 Would be really nice to just say if(other_docs.length == 0) return 
 __traits(docComment, item).
 
 A UDA can populate other_docs in some functions, but when wrapping 
 libraries you can't add that. Having the doc comment available through 
 reflection means we can reuse these existing items.


I still am very VERY leery of having anything in comments affect the 
code in any way.

What about a compromise? Provide access to the docs but ONLY at runtime. 
That is, you will not know anything about the string at compiletime, 
just at runtime, and you can make your decisions there.

Should work for your use case.

-Steve
Nov 11 2019
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 11 November 2019 at 17:16:20 UTC, Steven Schveighoffer 
wrote:

 I still am very VERY leery of having anything in comments 
 affect the code in any way.


then just.... don't do that.

And btw for the record you could have comments affect code right 
now:

mixin((__LINE__ % 3) ? "one thing" : "other thing"); // adding a 
comment above would change the line number and possibly trigger 
the other thing!


but such a thing is obviously absurd and nobody would write that, 
unless maybe doing an obfuscated contest.

The same thing is true of documentation comments. You *could* 
abuse... but why would you? UDAs are easier to use for other 
purposes anyway. (just right now we are using UDAs for docs just 
because it is needed elsewhere!)


 What about a compromise? Provide access to the docs but ONLY at 
 runtime. That is, you will not know anything about the string 
 at compiletime, just at runtime, and you can make your 
 decisions there.


Yeah, that'd work for this case but it is a pointless restriction.
Nov 11 2019
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/11/19 12:54 PM, Adam D. Ruppe wrote:

 On Monday, 11 November 2019 at 17:16:20 UTC, Steven Schveighoffer wrote:

 I still am very VERY leery of having anything in comments affect the 
 code in any way.


 
 then just.... don't do that.
 
 And btw for the record you could have comments affect code right now:
 
 mixin((__LINE__ % 3) ? "one thing" : "other thing"); // adding a comment 
 above would change the line number and possibly trigger the other thing!


Haha, yeah, that is what I don't want.

But I think we can safely say that using comments to affect code can 
result in reasonable looking things (the above is NOT reasonable) that 
can be exploited by adjusting the comments.

Certainly, it could be allowed at compile-time and just be a guideline 
not to write code that cares about the data. But we all know that it 
will happen.

For instance, I like how documented unit tests simply turns the unittest 
into a ddoc, instead of making ddoc turn into a unit test (which likely 
could have been the solution if __traits(docComment) were available back 
then). It's the correct translation, because code can always be turned 
into comments, but the other way around is questionable.


 The same thing is true of documentation comments. You *could* abuse... 
 but why would you? UDAs are easier to use for other purposes anyway. 
 (just right now we are using UDAs for docs just because it is needed 
 elsewhere!)


One thing that's different from __LINE__ is that __LINE__ is only 
provided to that line of code. It's not accessible outside the line. 
__traits(docComment, x) would provide access to the comments from 
elsewhere. So it may be external abuse that happens.


 

 What about a compromise? Provide access to the docs but ONLY at 
 runtime. That is, you will not know anything about the string at 
 compiletime, just at runtime, and you can make your decisions there.


 
 Yeah, that'd work for this case but it is a pointless restriction.


It would be something that could potentially be expanded later if 
needed. It's easier to let the toothpaste out of the tube slowly than 
try to put some of it back in.

-Steve
Nov 11 2019
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 11 November 2019 at 20:15:56 UTC, Steven Schveighoffer 
wrote:

 Certainly, it could be allowed at compile-time and just be a 
 guideline not to write code that cares about the data. But we 
 all know that it will happen.


Indeed, I have seen a lot of trash D code out there.

But I don't want to remove useful features because of it.


 For instance, I like how documented unit tests simply turns the 
 unittest into a ddoc, instead of making ddoc turn into a unit 
 test (which likely could have been the solution if 
 __traits(docComment) were available back then).


I actually doubt that would have been used because that needs 
significant progressing anyway. What we did use before the 
unittest thing was just extract the code from the generated HTML 
documentation which is pretty easy too.

(that's probably what I'll do with my program here, just dump the 
comments to a separate file and then `import` it to populate the 
map. but so much more complicated and fragile than just pulling 
the comment off while reflecting anyway)


 It would be something that could potentially be expanded later 
 if needed. It's easier to let the toothpaste out of the tube 
 slowly than try to put some of it back in.


true. like I'd rather have it just for runtime processing than 
not at all, but it is still just bleh.
Nov 11 2019
prev sibling parent reply Exil <Exil gmall.com> writes:
On Monday, 11 November 2019 at 17:54:07 UTC, Adam D. Ruppe wrote:

 On Monday, 11 November 2019 at 17:16:20 UTC, Steven 
 Schveighoffer wrote:

 I still am very VERY leery of having anything in comments 
 affect the code in any way.


 then just.... don't do that.


That won't stop other people.


 And btw for the record you could have comments affect code 
 right now:

 mixin((__LINE__ % 3) ? "one thing" : "other thing"); // adding 
 a comment above would change the line number and possibly 
 trigger the other thing!


 but such a thing is obviously absurd and nobody would write 
 that, unless maybe doing an obfuscated contest.


That isn't really all that useful. That line of code's only 
purpose is to be misused and cause misunderstanding.

Someone might come around and say, why modify documentation and 
code? Why not just modify documentation and then have the code 
read from that documentation and change behavior instead. That 
way you keep documentation and implementation in sync!

Kind of odd you have a 1:1 mapping for a scripting language 
anyways. That doesn't seem like a very rare occurrence. You don't 
pass any extra meta data for your implementations? They all have 
1:1 mapping for types? You implement things like pointers in your 
scripting language?
Nov 11 2019
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 12 November 2019 at 01:22:09 UTC, Exil wrote:

 Someone might come around and say, why modify documentation and 
 code? Why not just modify documentation and then have the code 
 read from that documentation and change behavior instead. That 
 way you keep documentation and implementation in sync!


Heh, that might actually be kinda interesting.

But you can already do that kind of thing, just with slightly 
different syntax. You can put that in a UDA and process it with 
reflection at CT and embed it into documentation through the UDA 
too. (my doc generator supports pulling in UDAs to work around 
the lack of ddoc in reflection; I had to move docs to a UDA and 
now they're combined anyway, it is just hideous and not as 
compatible with third party things)


 You don't pass any extra meta data for your implementations?


It depends, but a lot of libraries actually do just work with the 
script languages and it is nice to just do, for example 
`script.parseHTML = (string html) => new Document(html);` and 
then much of the Document object is now available to the script!

But since the Document library is written with ddoc, it is 
impossible for me to bring in those docs (without an external map 
generator and loader) for the interactive script.

$ var document = parseHTML(....);
$ \help document.querySelector

no help tho ddoc is present and the method is available to the 
script :(


I also do a web interface generator. You give it a class, it 
wraps it up in HTML forms, etc. It would be nice if it gave you 
the documentation... but it can't even tell if it is there. Best 
it can do is blindly link to my documentation website and *hope* 
it is there. (which actually isn't bad, but again it would be 
cool if it were more fully integrated)

Or use a UDA, since these wrappers don't go as deep as 
automatically as the script language, so it isn't too bad either, 
it would just be nice if the ddoc worked.
Nov 11 2019
parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Tuesday, 12 November 2019 at 01:59:55 UTC, Adam D. Ruppe wrote:

 On Tuesday, 12 November 2019 at 01:22:09 UTC, Exil wrote:

 Someone might come around and say, why modify documentation 
 and code? Why not just modify documentation and then have the 
 code read from that documentation and change behavior instead. 
 That way you keep documentation and implementation in sync!


 Heh, that might actually be kinda interesting.

 But you can already do that kind of thing, just with slightly 
 different syntax. You can put that in a UDA and process it with 
 reflection at CT and embed it into documentation through the 
 UDA too. (my doc generator supports pulling in UDAs to work 
 around the lack of ddoc in reflection; I had to move docs to a 
 UDA and now they're combined anyway, it is just hideous and not 
 as compatible with third party things)


This is also my view. D already supports things that are this 
messy or more. All this restriction does is lock the barn after 
the horse has bolted.

You can already do the same degree of mess. You just have to make 
it ugly. All this restriction does is make good, useful code 
worse for little benefit.
Nov 11 2019
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/12/19 1:32 AM, FeepingCreature wrote:

 On Tuesday, 12 November 2019 at 01:59:55 UTC, Adam D. Ruppe wrote:

 On Tuesday, 12 November 2019 at 01:22:09 UTC, Exil wrote:

 Someone might come around and say, why modify documentation and code? 
 Why not just modify documentation and then have the code read from 
 that documentation and change behavior instead. That way you keep 
 documentation and implementation in sync!


 Heh, that might actually be kinda interesting.

 But you can already do that kind of thing, just with slightly 
 different syntax. You can put that in a UDA and process it with 
 reflection at CT and embed it into documentation through the UDA too. 
 (my doc generator supports pulling in UDAs to work around the lack of 
 ddoc in reflection; I had to move docs to a UDA and now they're 
 combined anyway, it is just hideous and not as compatible with third 
 party things)


 
 This is also my view. D already supports things that are this messy or 
 more. All this restriction does is lock the barn after the horse has 
 bolted.
 
 You can already do the same degree of mess. You just have to make it 
 ugly. All this restriction does is make good, useful code worse for 
 little benefit.


Breaking the fundamental rule of "this is a comment, it doesn't affect 
code" that we all learn on day 1 of programming needs a very very 
compelling use case. I haven't seen it yet.

Alternatively, we could use something other than comments for 
documentation. e.g.:

 ddoc {
.... // documentation
}

-Steve
Nov 12 2019
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 12 November 2019 at 15:14:17 UTC, Steven 
Schveighoffer wrote:

 Alternatively, we could use something other than comments for 
 documentation. e.g.:


lol

 ddoc!q{
/++

+/
}


that would actually work and the doc generator can recognize the 
pattern too.

But there's a big difference:

struct A {
    int a; /// does something
}

that is cute syntax and not repeatable with UDAs.

And of course, my big use case is to fetch docs for third party 
libs I'd rather not edit the source of anyway.


Now see, this is an argument against the fear that it will get 
abused though simply because the language has easier ways to do 
other things. Using UDAs for docs is doable but awkward, so we 
prefer to use /++ +/.

Similarly, abusing doc comments as UDAs is going to be more 
awkward than using the real thing - you can use the language to 
structure it and all. So why do things the hard way when there's 
an easy way?
Nov 12 2019
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/12/19 11:23 AM, Adam D. Ruppe wrote:

 On Tuesday, 12 November 2019 at 15:14:17 UTC, Steven Schveighoffer wrote:

 Alternatively, we could use something other than comments for 
 documentation. e.g.:


 
 lol
 
  ddoc!q{
 /++
 
 +/
 }
 
 
 that would actually work and the doc generator can recognize the pattern 
 too.
 
 But there's a big difference:
 
 struct A {
     int a; /// does something
 }


This is really not much different:

struct A {
      ddoc(does something)
     int a;
}

But I mean, we can do whatever we want. It doesn't have to fit into the 
world of UDA. It can be its own syntax.

struct A {
     int a; !ddoc does something
}

It just shouldn't be a comment, which the compiler is supposed to ignore.


 And of course, my big use case is to fetch docs for third party libs I'd 
 rather not edit the source of anyway.


Well, assuming this isn't just a mental exercise, the answer would be, 
if you want it to be accessible to __traits(getDocs), you need to write 
it the other way. If that means you can't access the docs, then you 
can't access the docs. I would expect a dfix solution is braindead 
simple here.


 Now see, this is an argument against the fear that it will get abused 
 though simply because the language has easier ways to do other things. 
 Using UDAs for docs is doable but awkward, so we prefer to use /++ +/.
 
 Similarly, abusing doc comments as UDAs is going to be more awkward than 
 using the real thing - you can use the language to structure it and all. 
 So why do things the hard way when there's an easy way?


The thought of comments *ever* affecting the code is just plain wrong to 
me. Comments are not code, we should keep that line bright.

-Steve
Nov 13 2019
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 13 November 2019 at 16:03:55 UTC, Steven 
Schveighoffer wrote:

 It just shouldn't be a comment, which the compiler is supposed 
 to ignore.


The D specification makes a distinction between ordinary comments 
and ddoc comments. ddoc comments are well-defined by the language 
and attached to symbols by parsing rules.

It currently doesn't expose it directly - it only processes it 
into separate files (not ignoring them!) - but I want to change 
that, and appealing to the authority of the status quo isn't much 
of a counterargument.


 The thought of comments *ever* affecting the code is just plain 
 wrong to me. Comments are not code, we should keep that line 
 bright.


I agree for regular comments, but ddoc is already a part of the 
language and processed by the compiler! All I want to do is let 
other parts of the language see it too.

And again, UDAs do a better job at embedding non-doc stuff than 
doc comments. So I really think the risk of people going crazy 
with them is quite small since the alternative is superior for 
this task.
Nov 13 2019
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/13/19 11:12 AM, Adam D. Ruppe wrote:

 On Wednesday, 13 November 2019 at 16:03:55 UTC, Steven Schveighoffer wrote:

 It just shouldn't be a comment, which the compiler is supposed to ignore.


 
 The D specification makes a distinction between ordinary comments and 
 ddoc comments. ddoc comments are well-defined by the language and 
 attached to symbols by parsing rules.


Well defined, but it still compiles even if you don't follow the spec. 
These are comments still, and a secondary product from the binary output.


 It currently doesn't expose it directly - it only processes it into 
 separate files (not ignoring them!) - but I want to change that, and 
 appealing to the authority of the status quo isn't much of a 
 counterargument.


In reality you could separate the ddoc parsing and generation from the 
compile step anyway. It's just that it's much more convenient to use the 
already parsed AST for the code to generate the documentation.

In this way, the *code* affects the documentation, not the other way 
around. The status quo should remain this way because code is 
complicated enough without requiring comments to "compile" correctly too.

However, I'm OK with something other than comments being used to 
generate the doc reading code, OR deal with it at runtime when the 
compiler is not involved.

Note that ddoc comments are still not ddoc data. You are missing a lot 
of stuff with just the string. Wouldn't it be nice to just import the 
JSON data generated by the compiler? That's how I would do it.


 

 The thought of comments *ever* affecting the code is just plain wrong 
 to me. Comments are not code, we should keep that line bright.


 
 I agree for regular comments, but ddoc is already a part of the language 
 and processed by the compiler! All I want to do is let other parts of 
 the language see it too.
 
 And again, UDAs do a better job at embedding non-doc stuff than doc 
 comments. So I really think the risk of people going crazy with them is 
 quite small since the alternative is superior for this task.


I've seen lots of schemes where comments are significant to some process 
or another. IDE directives for instance.

But these schemes can play around in this realm *because* the compiler 
ignores them. I never want to see a case where comments are significant 
and break compilation or introduce bugs. We have enough to review in the 
code itself...

-Steve
Nov 13 2019
parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Wednesday, 13 November 2019 at 16:33:58 UTC, Steven 
Schveighoffer wrote:

 On 11/13/19 11:12 AM, Adam D. Ruppe wrote:

 On Wednesday, 13 November 2019 at 16:03:55 UTC, Steven 
 Schveighoffer wrote:

 It just shouldn't be a comment, which the compiler is 
 supposed to ignore.


 
 The D specification makes a distinction between ordinary 
 comments and ddoc comments. ddoc comments are well-defined by 
 the language and attached to symbols by parsing rules.


 Well defined, but it still compiles even if you don't follow 
 the spec. These are comments still, and a secondary product 
 from the binary output.


Easily fixed: make the trait fail if the comment violates the 
Ddoc guidelines.


 It currently doesn't expose it directly - it only processes it 
 into separate files (not ignoring them!) - but I want to 
 change that, and appealing to the authority of the status quo 
 isn't much of a counterargument.


 In reality you could separate the ddoc parsing and generation 
 from the compile step anyway. It's just that it's much more 
 convenient to use the already parsed AST for the code to 
 generate the documentation.

 In this way, the *code* affects the documentation, not the 
 other way around. The status quo should remain this way because 
 code is complicated enough without requiring comments to 
 "compile" correctly too.


We already require comments to compile correctly. That's what 
Ddoc *is*. Documentation should not be considered an optional 
part of functionality, especially if the user is trying to use 
Ddoc comments.


 However, I'm OK with something other than comments being used 
 to generate the doc reading code, OR deal with it at runtime 
 when the compiler is not involved.

 Note that ddoc comments are still not ddoc data. You are 
 missing a lot of stuff with just the string. Wouldn't it be 
 nice to just import the JSON data generated by the compiler? 
 That's how I would do it.


That would be nice. However, imagine this usecase: Swagger is a 
documentation format for REST requests. It would be handy if this 
documentation could be served right alongside the requests 
themselves. However, this would require a two-step process where 
the code is compiled, the Ddoc comments are discarded; then the 
code is, for no good reason, compiled again with a separate tool 
that wraps the D compiler, having to ensure it gets the same 
compiler binary and same compiler flags, creating an additional 
point of error (for no reason!), then dynamically loaded back 
into a process and processed at runtime, when they could have 
been equally well evaluated via a trait at compiletime and 
processed in ctfe with std.json to generate static output with 
nicely integrated error checking with line numbers.


 I've seen lots of schemes where comments are significant to 
 some process or another. IDE directives for instance.

 But these schemes can play around in this realm *because* the 
 compiler ignores them. I never want to see a case where 
 comments are significant and break compilation or introduce 
 bugs. We have enough to review in the code itself...

 -Steve


Again: Ddoc comments are *already* a mandatory part of 
compilation. If code compiles but the comments violate Ddoc, it 
should under no circumstances be considered "working."
D is a static language. To me, that means: any runtime failure 
that can be a compiletime failure should be. Compilation failing 
is annoying; but, compilation failing *rather than* runtime 
failing is the entire point of static types.
Nov 13 2019
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/13/19 12:19 PM, FeepingCreature wrote:

 On Wednesday, 13 November 2019 at 16:33:58 UTC, Steven Schveighoffer wrote:



 I've seen lots of schemes where comments are significant to some 
 process or another. IDE directives for instance.

 But these schemes can play around in this realm *because* the compiler 
 ignores them. I never want to see a case where comments are 
 significant and break compilation or introduce bugs. We have enough to 
 review in the code itself...


 
 Again: Ddoc comments are *already* a mandatory part of compilation. If 
 code compiles but the comments violate Ddoc, it should under no 
 circumstances be considered "working."
 D is a static language. To me, that means: any runtime failure that can 
 be a compiletime failure should be. Compilation failing is annoying; 
 but, compilation failing *rather than* runtime failing is the entire 
 point of static types.


I'm not convinced:

import std.stdio;

/// $(BADMACRO boo)
void main()
{
     writeln("compiles and runs!");
}

Even generates ddoc! Just that,  (BADMACRO boo) does nothing. It doesn't 
even produce an error.

Let's mess with the parsing:

/// $(BADMACRO boo

note the missing parentheses. No errors, even when compiled with -D. I 
get a correct html doc, but now I see "(BADMACRO boo" as the docs for 
main as if it were just a string (curiously missing the $ but whatevs, 
it's just comments and docs!)

I wouldn't consider this to be "mandatory" or even not "working". I get 
a working binary, and I get a working ddoc (I see main in the docs, and 
the junk that I stuck in there). You want to introduce into this a way 
to create *errors* when existing ddocs don't have to even parse correctly?

The D spec says what DDoc is, but doesn't say that anything has to 
conform to ddoc (indeed, someone might even use another doc system and 
have their comments written for it), or even that DDoc will fail to 
produce documentation if it's not correct!

-Steve
Nov 13 2019
prev sibling parent reply Dennis <dkorpel gmail.com> writes:
On Tuesday, 12 November 2019 at 06:32:35 UTC, FeepingCreature 
wrote:

 You can already do the same degree of mess. You just have to 
 make it ugly. All this restriction does is make good, useful 
 code worse for little benefit.


And that's exactly the point: you have to make it ugly, so it 
isn't an attractive option.
Some things are ugly by design, such as __gshared, discouraging 
users from using it while still allowing it if needed.

Similarly, version() statements don't allow boolean logic, 
discouraging those annoying double negations and version formulas 
(see e.g. [1], [2]) and encouraging 'positive' identifier names 
that don't have 'no' in them. You can still make version 
spaghetti with `static if` if you want, but it won't be as 
appealing.

Technically you can still do C macro shenanigans like `#define 
BEGIN {` in D by putting your module in a q{} string, doing a 
string-replace in CTFE and mixing it in.
"You just have to make it ugly".
And that's exactly why D programmers don't tend to do that.

While I do see the value in __traits(docComment) (I would use it 
myself in the same way Adam wants to use it), I do have to agree 
with Steven that it just feels to powerful.
As soon as some hacker discovers `__traits(docComment)` he will 
have the 'brilliant' idea to inspect documentation and use parts 
of it for his code generation, and it will look elegant so it 
doesn't seem wrong (unlike some convoluted 
parse(import(__FILE__)) scheme).
And then a clueless contributor will be like 'how the heck did 
running dfmt on this codebase make the unittests fail'?

UDA's use string literals, so the rules of what string you get 
are exactly clear and a formatter won't touch it. How does 
__traits(docComment) behave on these two comments:

```
/** Word
  *  Word
  */

/******************
Word
Word
*/
```

What strings do they produce?

[1] 
https://github.com/PetteriAimonen/libfixmath/blob/c3017d7d74867398dcdcaa21d93c6a0b039b9bca/libfixmath/fix16.c#L8
[2] 
https://github.com/PetteriAimonen/libfixmath/blob/c3017d7d74867398dcdcaa21d93c6a0b039b9bca/libfixmath/fix16.c#L67
Nov 13 2019
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 13 November 2019 at 15:22:23 UTC, Dennis wrote:

 UDA's use string literals, so the rules of what string you get 
 are exactly clear and a formatter won't touch it. How does 
 __traits(docComment) behave on these two comments:


This is actually defined in the D language specification

https://dlang.org/spec/ddoc.html

"The extra *'s and +'s on the comment opening, closing and left 
margin are ignored and are not part of the embedded 
documentation. Comments not following one of those forms are not 
documentation comments. "

It is a little bit ambiguous to get

/**
   Word
   * word
*/

Is that "* word" supposed to ignore the star? dmd ignores it 
which is probably most consistent with the spec. My adrdox does 
NOT ignore that and can use it for markdown-style lists...

But regardless, the __trait thing should do what the compiler 
already does for ddoc.

Though I would NOT parse the ddoc, just give the string as the 
compiler read it.


 ```
 /** Word
  *  Word
  */

 /******************
 Word
 Word
 */
 ```

 What strings do they produce?


" Word\n  Word\n "

and

"Word\nWord\n"

So not identical, but this is well defined by the D language spec!
Nov 13 2019
parent reply Dennis <dkorpel gmail.com> writes:
On Wednesday, 13 November 2019 at 15:40:34 UTC, Adam D. Ruppe 
wrote:

 So not identical, but this is well defined by the D language 
 spec!


Well defined, but it requires the programmer to know those 
defined rules.
I wasn't sure whether in my first example there would be a double 
space between the two 'Word's, but it turns out there is, and now 
I need to account for that when printing the text after getting 
it with __traits(docComment).

I don't have this problem when doing documentation generation, I 
just do `dub build --build=docs` and I get html pages that look 
fine (no double spaces), without needing to know what happens 
under the hood.


 Is that "* word" supposed to ignore the star? dmd ignores it 
 which is probably most consistent with the spec. My adrdox does 
 NOT ignore that and can use it for markdown-style lists...


That's probably going to give new bug reports too. ("Why did my 
list bullets disappear with __traits(docComment)? I see it just 
fine when rendering with adrdox!")
Nov 13 2019
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 13 November 2019 at 18:02:35 UTC, Dennis wrote:

 I don't have this problem when doing documentation generation, 
 I just do `dub build --build=docs` and I get html pages that 
 look fine (no double spaces), without needing to know what 
 happens under the hood.


That's because HTML ignores extra spaces. If you were to do 
__traits(docComment) and output it to an HTML file, you'd see 
that same result. The space is still there, it is just rendered 
differently by the target environment.

All __traits(docComment) should do is return the source code 
after the compiler read it. (actually, I wouldn't even mind if it 
was the whole comment, in full, including the delimiters, but if 
the compiler already removes stars I'm ok with that too)


 That's probably going to give new bug reports too. ("Why did my 
 list bullets disappear with __traits(docComment)? I see it just 
 fine when rendering with adrdox!")


Well, adrdox is not compatible with ddoc exactly, so I'd just 
accept that.
Nov 13 2019
parent reply Dennis <dkorpel gmail.com> writes:
On Wednesday, 13 November 2019 at 18:46:43 UTC, Adam D. Ruppe 
wrote:

 That's because HTML ignores extra spaces.


Of course, but the overall point I'm trying to make is that the 
trait is non-trivial and adds language complexity.

It gets implemented one way, and then someone comes around 
wanting it to be different. "please give me the html instead of 
raw comment, parsing it with CTFE is slow and cumbersome"
"please include the /** or /++, I'm using __traits(docComment) to 
statically assert my preferred documentation code style"
"is it a bug that leading/trailing whitespace is/isn't trimmed?"
"the previous changes broke my existing CTFE documentation parser"
"__traits(docComment) does not work when the comment is in a 
mixin that is seen later in semantic analysis"
"I want only my doc comment of my function to depend on the build 
type, but putting the comment inside version blocks does not 
work! How to fix?"

This can of worms can stay closed by just sticking with good old 
predictable and flexible UDA strings.
Nov 13 2019
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 13 November 2019 at 19:15:56 UTC, Dennis wrote:

 Of course, but the overall point I'm trying to make is that the 
 trait is non-trivial and adds language complexity.


It doesn't have to be. All I want is the existing string.

The ONLY addition I am calling for is that a trait docComment (or 
whatever) returns the ddoc string attached to the declaration 
(and if the existing implementation strips delimiters, so be it, 
and if not, so be it, the spec does define stripping them so i 
assume it does but I don't really care).

Everything else is already defined or out of scope, so we really 
shouldn't be worrying about them.


 It gets implemented one way, and then someone comes around 
 wanting it to be different. "please give me the html instead of 
 raw comment


Impossible, since that's undefined.


 "is it a bug that leading/trailing whitespace is/isn't trimmed?"


Clear answer to this: no, it is not a bug. That's defined to be 
there.


 "the previous changes broke my existing CTFE documentation 
 parser"


ddoc is part of the spec and it has had breaking changes, we just 
deal with it.


 "__traits(docComment) does not work when the comment is in a 
 mixin that is seen later in semantic analysis"


This is similarly already an open question with the language. 
Should __traits(allMembers) return stuff from mixins?


 "I want only my doc comment of my function to depend on the 
 build type, but putting the comment inside version blocks does 
 not work! How to fix?"


This is also already well-defined, version applies to 
declarations, doc comments are part of a declaration, not a 
separate entity.
Nov 13 2019
parent reply Dennis <dkorpel gmail.com> writes:
On Wednesday, 13 November 2019 at 19:44:38 UTC, Adam D. Ruppe 
wrote:

 The ONLY addition I am calling for is that a trait docComment 
 (or whatever) returns the ddoc string attached to the 
 declaration


That may be the only thing _you_ want, but then someone else 
comes along who wants something different.
All Manu wanted was a mangle-only `extern(C++, "ns")` as an 
additional option, so simple. Why was Walter initially so 
resistant for such a harmless addition? Then it got merged.

https://github.com/dlang/dmd/pull/8667

Then the issues come in.

https://issues.dlang.org/show_bug.cgi?id=19278
https://issues.dlang.org/show_bug.cgi?id=19339
https://issues.dlang.org/show_bug.cgi?id=19557
https://issues.dlang.org/show_bug.cgi?id=19574
https://issues.dlang.org/show_bug.cgi?id=19893
https://issues.dlang.org/show_bug.cgi?id=19920

And someone wants to deprecate the old way.

https://github.com/dlang/dmd/pull/10031

I'm not saying it wasn't worth it in that case, but those 
fictional remarks about __traits(docComment) were supposed to 
show that the trait will be ANOTHER source of enhancement 
requests, introduces ANOTHER thing that can get queried during 
any point of semantic analysis, probably making it ANOTHER source 
of DMD bugs, and it will be ANOTHER source of confusion / 
material for D learners to swallow.

That's just how it goes despite how simple it may look, and the 
question is whether the new trait justifies this cost. Not every 
code annoyance warrants a new feature, sometimes it's better to 
e.g. duplicate a few lines of code instead of introducing another 
C-macro.

Likewise, in this case having some duplication in UDA's or having 
another build step may be the easier solution in the end.
Nov 13 2019
parent reply Jab <jab_293 gmall.com> writes:
On Wednesday, 13 November 2019 at 20:38:07 UTC, Dennis wrote:

 On Wednesday, 13 November 2019 at 19:44:38 UTC, Adam D. Ruppe 
 wrote:

 The ONLY addition I am calling for is that a trait docComment 
 (or whatever) returns the ddoc string attached to the 
 declaration


 That may be the only thing _you_ want, but then someone else 
 comes along who wants something different.
 All Manu wanted was a mangle-only `extern(C++, "ns")` as an 
 additional option, so simple. Why was Walter initially so 
 resistant for such a harmless addition? Then it got merged.

 https://github.com/dlang/dmd/pull/8667

 Then the issues come in.

 https://issues.dlang.org/show_bug.cgi?id=19278
 https://issues.dlang.org/show_bug.cgi?id=19339
 https://issues.dlang.org/show_bug.cgi?id=19557
 https://issues.dlang.org/show_bug.cgi?id=19574
 https://issues.dlang.org/show_bug.cgi?id=19893
 https://issues.dlang.org/show_bug.cgi?id=19920


IIRC there was a WIP pull request accompanied by a DIP, the pull 
request was merged while it was still being ironed out along with 
the DIP. Those were all easy fixes foir the most part. Hell 
there's still bugs being fixed regarding extern(C). With that 
kind of justification no new features should ever be added. Not 
to mention bug fixes that just cause no bugs. We should just stop 
making changes all together. Point to one of D's biggest 
mistakes, even then I would still say that's not a good enough 
reason to not make changes.


 And someone wants to deprecate the old way.

 https://github.com/dlang/dmd/pull/10031



They probably want to, because the old way is impeding the new 
method, you can't use CTFE to construct a string.



 I'm not saying it wasn't worth it in that case, but those 
 fictional remarks about __traits(docComment) were supposed to 
 show that the trait will be ANOTHER source of enhancement 
 requests, introduces ANOTHER thing that can get queried during 
 any point of semantic analysis, probably making it ANOTHER 
 source of DMD bugs, and it will be ANOTHER source of confusion 
 / material for D learners to swallow.

 That's just how it goes despite how simple it may look, and the 
 question is whether the new trait justifies this cost. Not 
 every code annoyance warrants a new feature, sometimes it's 
 better to e.g. duplicate a few lines of code instead of 
 introducing another C-macro.


Here's a counter example to your example: 
https://github.com/dlang/dmd/pull/10013

I wouldn't really say it's a new "feature". It'd just be exposing 
the information that is already available



 Likewise, in this case having some duplication in UDA's or


It's not just duplication, you'd have to modifying other people's 
source code. How would you duplicate this doc:

/// ditto

The implementation then becomes much more convoluted.


 having another build step may be the easier solution in the end.


Using dub? That's going to be a pain in the ass honestly. This 
sort of implementation you'd have to do isn't that simple. You'd 
have to effectively run a dub command twice, once to generate the 
doc output and then run it again to compile the code with the now 
available documentation. Dub wasn't designed for that, and build 
systems for C/C++ are often overly complicated because the 
languages don't expose information that the compiler already has 
available to it. Basically of what D does right with it's compile 
time features. __traits already has an atrocious eye catching 
name. If you need to find code somewhere that might be doing 
something with comments that it shouldn't be doing, it's not 
going to be that difficult to find.
Nov 13 2019
parent reply Dennis <dkorpel gmail.com> writes:
On Thursday, 14 November 2019 at 05:03:49 UTC, Jab wrote:

 With that kind of justification no new features should ever be 
 added. Not to mention bug fixes that just cause no bugs. We 
 should just stop making changes all together.


I just said that every feature has (hidden) cost, not that that 
cost is never justified.


 They probably want to, because the old way is impeding the new 
 method, you can't use CTFE to construct a string.


That's not the point, the point is that it always starts with 
"but I just want X, nothing more!", and later inevitably someone 
wants Y too. Concretely in that case: "It can live peacefully 
beside your preferred design with the scopes." followed by the PR 
to deprecate extern(C++, identifier).


 Here's a counter example to your example: 
 https://github.com/dlang/dmd/pull/10013


I actually think that got merged too quickly without thinking all 
special cases and consequences through.


 I wouldn't really say it's a new "feature". It'd just be 
 exposing the information that is already available


Whatever you call it, it's another source of complexity as I 
mentioned before.


 How would you duplicate this doc:

 /// ditto


Put the UDA string in an enum?
```
enum string fooComment = "does foo"
/// does foo
 (fooComment) void foo(int a);
/// ditto
 (fooComment) void foo(string a);
```

It's hard to argue what the best current solution is without a 
concrete usecase.
If you write some functions / fields yourself, you can write some 
UDA strings that may be redundant with your doc comments.
If you want to expose documentation from an existing library 
(e.g. Phobos), it can be extracted beforehand instead.
Nov 14 2019
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 14 November 2019 at 09:54:35 UTC, Dennis wrote:

 That's not the point, the point is that it always starts with 
 "but I just want X, nothing more!", and later inevitably 
 someone wants Y too.


Once I have what I want, I'm not above just saying no to anyone 
else :P
Nov 14 2019
prev sibling parent reply Jab <jab_293 gmall.com> writes:
On Thursday, 14 November 2019 at 09:54:35 UTC, Dennis wrote:

 On Thursday, 14 November 2019 at 05:03:49 UTC, Jab wrote:

 With that kind of justification no new features should ever be 
 added. Not to mention bug fixes that just cause no bugs. We 
 should just stop making changes all together.


 I just said that every feature has (hidden) cost, not that that 
 cost is never justified.


Your making it sound like it isn't justified because of those 
hidden costs.


 They probably want to, because the old way is impeding the new 
 method, you can't use CTFE to construct a string.


 That's not the point, the point is that it always starts with 
 "but I just want X, nothing more!", and later inevitably 
 someone wants Y too. Concretely in that case: "It can live 
 peacefully beside your preferred design with the scopes." 
 followed by the PR to deprecate extern(C++, identifier).


The whole purpose of extern(C++, "") was because extern(C++, 
identifier) was inadequate.

If you want to look at it one way. The extern(C++, "") syntax was 
only wanted because the extern(C++, identifier) was there. Should 
we have not added extern(C++, identifier) to D to begin with? 
People are going to want to make changes to things they don't 
like. Before extern(C++, identifer) was added people were 
probably asking for a way to link with C++ mangling. Should they 
have not been given that feature because down the line there's 
going to be numerous bugs with it that will need to be fixed and 
people will then be wanting more and more changes to it?


 Here's a counter example to your example: 
 https://github.com/dlang/dmd/pull/10013


 I actually think that got merged too quickly without thinking 
 all special cases and consequences through.


Has there been any consequences?


 I wouldn't really say it's a new "feature". It'd just be 
 exposing the information that is already available


 Whatever you call it, it's another source of complexity as I 
 mentioned before.


 How would you duplicate this doc:

 /// ditto


 Put the UDA string in an enum?
 ```
 enum string fooComment = "does foo"
 /// does foo
  (fooComment) void foo(int a);
 /// ditto
  (fooComment) void foo(string a);
 ```

 It's hard to argue what the best current solution is without a 
 concrete usecase.
 If you write some functions / fields yourself, you can write 
 some UDA strings that may be redundant with your doc comments.
 If you want to expose documentation from an existing library 
 (e.g. Phobos), it can be extracted beforehand instead.


Was kind of curious what that looks like with a comment from 
phobos, I think it's worse than I expected.


enum cmpComment = `
Performs a lexicographical comparison on two
$(REF_ALTTEXT input ranges, isInputRange, std,range,primitives).
Iterating `r1` and `r2` in lockstep, `cmp` compares each element
`e1` of `r1` with the corresponding element `e2` in `r2`. If one
of the ranges has been finished, `cmp` returns a negative value
if `r1` has fewer elements than `r2`, a positive value if `r1`
has more elements than `r2`, and `0` if the ranges have the same
number of elements.
If the ranges are strings, `cmp` performs UTF decoding
appropriately and compares the ranges one code point at a time.
A custom predicate may be specified, in which case `cmp` performs
a three-way lexicographical comparison using `pred`. Otherwise
the elements are compared using `opCmp`.
Params:
     pred = Predicate used for comparison. Without a predicate
         specified the ordering implied by `opCmp` is used.
     r1 = The first range.
     r2 = The second range.
Returns:
     `0` if the ranges compare equal. A negative value if `r1` is 
a prefix of `r2` or
     the first differing element of `r1` is less than the 
corresponding element of `r2`
     according to `pred`. A positive value if `r2` is a prefix of 
`r1` or the first
     differing element of `r2` is less than the corresponding 
element of `r1`
     according to `pred`.
Note:
     An earlier version of the documentation incorrectly stated 
that `-1` is the
     only negative value returned and `1` is the only positive 
value returned.
     Whether that is true depends on the types being compared.`;

/**********************************
Performs a lexicographical comparison on two
$(REF_ALTTEXT input ranges, isInputRange, std,range,primitives).
Iterating `r1` and `r2` in lockstep, `cmp` compares each element
`e1` of `r1` with the corresponding element `e2` in `r2`. If one
of the ranges has been finished, `cmp` returns a negative value
if `r1` has fewer elements than `r2`, a positive value if `r1`
has more elements than `r2`, and `0` if the ranges have the same
number of elements.
If the ranges are strings, `cmp` performs UTF decoding
appropriately and compares the ranges one code point at a time.
A custom predicate may be specified, in which case `cmp` performs
a three-way lexicographical comparison using `pred`. Otherwise
the elements are compared using `opCmp`.
Params:
     pred = Predicate used for comparison. Without a predicate
         specified the ordering implied by `opCmp` is used.
     r1 = The first range.
     r2 = The second range.
Returns:
     `0` if the ranges compare equal. A negative value if `r1` is 
a prefix of `r2` or
     the first differing element of `r1` is less than the 
corresponding element of `r2`
     according to `pred`. A positive value if `r2` is a prefix of 
`r1` or the first
     differing element of `r2` is less than the corresponding 
element of `r1`
     according to `pred`.
Note:
     An earlier version of the documentation incorrectly stated 
that `-1` is the
     only negative value returned and `1` is the only positive 
value returned.
     Whether that is true depends on the types being compared.
*/
 docComment(cmpComment) auto cmp(R1, R2)(R1 r1, R2 r2);
Nov 14 2019
next sibling parent reply Dennis <dkorpel gmail.com> writes:
On Thursday, 14 November 2019 at 17:18:40 UTC, Jab wrote:

 Should we have not added extern(C++, identifier) to D to begin 
 with?



 Should they have not been given that feature because down the 
 line there's going to be numerous bugs with it that will need 
 to be fixed and people will then be wanting more and more 
 changes to it?


I haven't interfaced with C++ so I can't judge that, but from a 
spectator of the discussion point of view `extern(C++, "ns")` 
seemed like a worthy addition.
I only brought this up as an example to show how these things 
don't end with "just this one small addition" like Adam makes it 
out to be in the case of __traits(docComment).


 Was kind of curious what that looks like with a comment from 
 phobos, I think it's worse than I expected.


So I wouldn't use that for Phobos, but export Phobos 
documentation to a preferred file format and use that. What would 
you do with the raw ddoc comment anyway, print it in the terminal 
as-is with $() and all? Re-implement macro parsing in CTFE?

The UDA duplication is acceptable when you are writing some small 
comments yourself:

```
struct WindowSettings
{
      ("The default width of the window in pixels.")
     int width = 1280; /// default window width in pixels

      ("The default height of the window in pixels.")
     int height = 720; /// default window height in pixels
}
```

If you want to make a GUI for setting the width and height based 
on the struct then you want to poll the UDA string to get a nice 
description for the user.
And I know it is tempting to want to combine the two, based on 
the availability of __traits(docComment):

```
struct WindowSettings
{
     int width = 1280; /// The default width of the window in 
pixels.
     int height = 720; /// The default height of the window in 
pixels.
}
```

But before you know it you deal with all kinds of issues, like 
the user-facing comment and the programmer-facing comment 
diverging, needing to strip the leading space, the formatter 
changing it to a multi-line comment introducing other whitespace, 
wanting to the comment to be generated at CTFE so you have to 
turn back to UDA's anyway etc.

And it may just turn out that simple > clever all along.
Nov 14 2019
parent reply Jab <jab_293 gmall.com> writes:
On Thursday, 14 November 2019 at 20:24:46 UTC, Dennis wrote:

 On Thursday, 14 November 2019 at 17:18:40 UTC, Jab wrote:

 Should we have not added extern(C++, identifier) to D to begin 
 with?



 Should they have not been given that feature because down the 
 line there's going to be numerous bugs with it that will need 
 to be fixed and people will then be wanting more and more 
 changes to it?


 I haven't interfaced with C++ so I can't judge that, but from a 
 spectator of the discussion point of view `extern(C++, "ns")` 
 seemed like a worthy addition.
 I only brought this up as an example to show how these things 
 don't end with "just this one small addition" like Adam makes 
 it out to be in the case of __traits(docComment).


For him it probably would be, but there's numerous people out 
there that want something else. There's always going to be 
something people will want to improve on. I don't think that's a 
good enough reason to not do something. The merits of any 
additional changes should be evaluated on their merits if they do 
come up afterwards.


 Was kind of curious what that looks like with a comment from 
 phobos, I think it's worse than I expected.


 So I wouldn't use that for Phobos, but export Phobos 
 documentation to a preferred file format and use that. What 
 would you do with the raw ddoc comment anyway, print it in the 
 terminal as-is with $() and all? Re-implement macro parsing in 
 CTFE?

 The UDA duplication is acceptable when you are writing some 
 small comments yourself:

 ```
 struct WindowSettings
 {
      ("The default width of the window in pixels.")
     int width = 1280; /// default window width in pixels

      ("The default height of the window in pixels.")
     int height = 720; /// default window height in pixels
 }
 ```

 If you want to make a GUI for setting the width and height 
 based on the struct then you want to poll the UDA string to get 
 a nice description for the user.
 And I know it is tempting to want to combine the two, based on 
 the availability of __traits(docComment):

 ```
 struct WindowSettings
 {
     int width = 1280; /// The default width of the window in 
 pixels.
     int height = 720; /// The default height of the window in 
 pixels.
 }
 ```

 But before you know it you deal with all kinds of issues, like 
 the user-facing comment and the programmer-facing comment 
 diverging, needing to strip the leading space, the formatter 
 changing it to a multi-line comment introducing other 
 whitespace, wanting to the comment to be generated at CTFE so 
 you have to turn back to UDA's anyway etc.

 And it may just turn out that simple > clever all along.


See small examples like that aren't indicative of actual code. 
Struct settings I've used look more like this, that you usually 
can't see in one full screen window when everything is on one 
line:

struct WindowSettings
{
      ("The default width of the window in pixels.")
     int width = 1280; /// default window width in pixels
      ("The default height of the window in pixels.")
     int height = 720; /// default window height in pixels
      ("The default width of the window in pixels.")
     int width = 1280; /// default window width in pixels
      ("The default height of the window in pixels.")
     int height = 720; /// default window height in pixels
      ("The default width of the window in pixels.")
     int width = 1280; /// default window width in pixels
      ("The default height of the window in pixels.")
     int height = 720; /// default window height in pixels
      ("The default width of the window in pixels.")
     int width = 1280; /// default window width in pixels
      ("The default height of the window in pixels.")
     int height = 720; /// default window height in pixels
      ("The default width of the window in pixels.")
     int width = 1280; /// default window width in pixels
      ("The default height of the window in pixels.")
     int height = 720; /// default window height in pixels
      ("The default width of the window in pixels.")
     int width = 1280; /// default window width in pixels
      ("The default height of the window in pixels.")
     int height = 720; /// default window height in pixels
      ("The default width of the window in pixels.")
     int width = 1280; /// default window width in pixels
      ("The default height of the window in pixels.")
     int height = 720; /// default window height in pixels
      ("The default width of the window in pixels.")
     int width = 1280; /// default window width in pixels
      ("The default height of the window in pixels.")
     int height = 720; /// default window height in pixels
      ("The default width of the window in pixels.")
     int width = 1280; /// default window width in pixels
      ("The default height of the window in pixels.")
     int height = 720; /// default window height in pixels
      ("The default width of the window in pixels.")
     int width = 1280; /// default window width in pixels
      ("The default height of the window in pixels.")
     int height = 720; /// default window height in pixels
}

struct WindowSettings
{
     int width = 1280; /// default window width in pixels
     int height = 720; /// default window height in pixels
     int width = 1280; /// default window width in pixels
     int height = 720; /// default window height in pixels
     int width = 1280; /// default window width in pixels
     int height = 720; /// default window height in pixels
     int width = 1280; /// default window width in pixels
     int height = 720; /// default window height in pixels
     int width = 1280; /// default window width in pixels
     int height = 720; /// default window height in pixels
     int width = 1280; /// default window width in pixels
     int height = 720; /// default window height in pixels
     int width = 1280; /// default window width in pixels
     int height = 720; /// default window height in pixels
     int width = 1280; /// default window width in pixels
     int height = 720; /// default window height in pixels
     int width = 1280; /// default window width in pixels
     int height = 720; /// default window height in pixels
     int width = 1280; /// default window width in pixels
     int height = 720; /// default window height in pixels
}

Even that is still small in comparison. You could put everything 
on one line but then you are going to be going past any 
reasonable width of code. It'd make it even more unreadable if 
the doc is longer than usual as well. It'll just be doubling the 
width instead of the height. Neither is ideal.
Nov 15 2019
parent reply Dennis <dkorpel gmail.com> writes:
On Friday, 15 November 2019 at 17:15:12 UTC, Jab wrote:

 Struct settings I've used look more like this, that you usually 
 can't see in one full screen window when everything is on one 
 line:


As you get more and more struct members the chance that the 
documentation comment can actually double as user-facing comments 
becomes smaller and smaller.
Trying to keep everything simple and compact like that is not 
going to scale, as soon as e.g. your application gets localized 
and supports foreign languages your one-liner multi-purpose 
documentation comments have to go anyway.
Nov 15 2019
parent reply Jab <jab_293 gmall.com> writes:
On Friday, 15 November 2019 at 17:57:59 UTC, Dennis wrote:

 On Friday, 15 November 2019 at 17:15:12 UTC, Jab wrote:

 Struct settings I've used look more like this, that you 
 usually can't see in one full screen window when everything is 
 on one line:


 As you get more and more struct members the chance that the 
 documentation comment can actually double as user-facing 
 comments becomes smaller and smaller.
 Trying to keep everything simple and compact like that is not 
 going to scale, as soon as e.g. your application gets localized 
 and supports foreign languages your one-liner multi-purpose 
 documentation comments have to go anyway.


Docs don't really ever get localized. Otherwise phobos, D, and 
basically all programming languages all have the same problem.
Nov 18 2019
parent Dennis <dkorpel gmail.com> writes:
On Tuesday, 19 November 2019 at 05:30:46 UTC, Jab wrote:

 Docs don't really ever get localized.


Yeah but you want to make documentation comment also be the 
explanation text that appears in the GUI of an application (in 
this example), which might get localized.
Nov 19 2019
prev sibling parent Dennis <dkorpel gmail.com> writes:
On Thursday, 14 November 2019 at 17:18:40 UTC, Jab wrote:

 Here's a counter example to your example: 
 https://github.com/dlang/dmd/pull/10013


 I actually think that got merged too quickly without thinking 
 all special cases and consequences through.


 Has there been any consequences?


I luckily haven't seen anything bad yet, though it's really new.
I would say a consequence is that it makes re-ordering 
module-level declarations a potentially code-breaking change, 
though that information was already leaked with __LINE__, 
__traits(allMembers) and some DMD bugs [1] so 'luckily' the 
programmer couldn't count on that anyway.

[1] https://issues.dlang.org/show_bug.cgi?id=9125
Nov 14 2019
prev sibling parent reply Laeeth Isharc <laeeth kaleidic.io> writes:
On Monday, 11 November 2019 at 17:16:20 UTC, Steven Schveighoffer 
wrote:

 On 11/8/19 9:00 AM, Adam D. Ruppe wrote:

 Writing code to wrap some Phobos functions for a script 
 language and need documentation to be displayed at runtime...
 
 Would be really nice to just say if(other_docs.length == 0) 
 return __traits(docComment, item).
 
 A UDA can populate other_docs in some functions, but when 
 wrapping libraries you can't add that. Having the doc comment 
 available through reflection means we can reuse these existing 
 items.


 I still am very VERY leery of having anything in comments 
 affect the code in any way.

 What about a compromise? Provide access to the docs but ONLY at 
 runtime. That is, you will not know anything about the string 
 at compiletime, just at runtime, and you can make your 
 decisions there.

 Should work for your use case.

 -Steve


Compiler flag as we do with string imports?  Most projects won't 
use this feature.  If it does end up being used then at least it 
will be clear.  It's difficult to save people from themselves 
completely - and I think there are many more likely ways to end 
up abusing the powerful features of D.

Suppose in scenario 1 you use doc comment traits and in scenario 
2 you have an extra build stage to achieve the same effect; and 
suppose through unfortunate choices the documentation ends up 
affecting the runtime behaviour of the code in non-documentation 
related ways.  Are you better off in scenario 1 or 2?  It's not 
clear, but 1 might be better.

D has goto - people don't habitually abuse it.
Nov 13 2019
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/13/19 2:30 PM, Laeeth Isharc wrote:

 On Monday, 11 November 2019 at 17:16:20 UTC, Steven Schveighoffer wrote:

 I still am very VERY leery of having anything in comments affect the 
 code in any way.

 What about a compromise? Provide access to the docs but ONLY at 
 runtime. That is, you will not know anything about the string at 
 compiletime, just at runtime, and you can make your decisions there.

 Should work for your use case.


 
 Compiler flag as we do with string imports?  Most projects won't use 
 this feature.  If it does end up being used then at least it will be 
 clear.  It's difficult to save people from themselves completely - and I 
 think there are many more likely ways to end up abusing the powerful 
 features of D.


I'm trying to save the reviewers and the maintainers. Code review 
shouldn't include reviewing stuff that isn't compiled. If you are doing 
weird imports, that's an unusual situation and requires unusual knowledge.

If you put this into traits, then it's a usual and easy tool, and now 
you have to verify comments are correctly written.

If all you want is to fetch the documentation for a function, it can be 
done in better ways.


 
 Suppose in scenario 1 you use doc comment traits and in scenario 2 you 
 have an extra build stage to achieve the same effect; and suppose 
 through unfortunate choices the documentation ends up affecting the 
 runtime behaviour of the code in non-documentation related ways.  Are 
 you better off in scenario 1 or 2?  It's not clear, but 1 might be better.


For sure, it's possible already to run extra steps to have *anything* 
affect the code. You could use a preprocessor that replaces ddocs with 
UDAs for the same effect. But in that case, you are altering the input 
to the compiler, not using the compiler to build something that isn't 
normal compiler input.

Of course, you can already do such things with string imports.


 D has goto - people don't habitually abuse it.


This is more akin to mixins, not goto. Goto is actually easy to review 
(automatic rejection! /jk), you know exactly what it does.

And yes, mixins are abused. I just did some work to the diet-ng project 
to make it possible to update non-interpolated HTML and have it figured 
out at runtime[1]. The way diet imports non-D code and changes it into D 
code to mixin, it's cool, but I'd consider it an abuse, and really 
difficult to comprehend. I'm getting to the point where I just think an 
external tool is better[2].

-Steve

[1] https://github.com/rejectedsoftware/diet-ng/pull/70
[2] https://github.com/schveiguy/dietpc
Nov 13 2019
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 13 November 2019 at 22:53:49 UTC, Steven 
Schveighoffer wrote:

 I'm trying to save the reviewers and the maintainers. Code 
 review shouldn't include reviewing stuff that isn't compiled.


Documentation is a key component of code review. And any code 
reviewer would surely be skeptical of a __trait(getDoc) that does 
anything other than use it for documentation.

(and you could actually even static assert it is written 
reasonably with the trait but i could see that being hated too 
lol.)
Nov 13 2019
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/13/19 6:03 PM, Adam D. Ruppe wrote:

 On Wednesday, 13 November 2019 at 22:53:49 UTC, Steven Schveighoffer wrote:

 I'm trying to save the reviewers and the maintainers. Code review 
 shouldn't include reviewing stuff that isn't compiled.


 
 Documentation is a key component of code review. And any code reviewer 
 would surely be skeptical of a __trait(getDoc) that does anything other 
 than use it for documentation.


Where it starts becoming a problem is when users of a library say "you 
changed your docs, and it broke my code". This is always the argument 
against named parameters (you changed parameter X's name, and it broke 
my code).

Anything that becomes observable at compile-time now becomes part of the 
API.

Theoretically, even a runtime string could cause this. But there is much 
less possibility for abuse, and at least the compiler doesn't write 
*different* binary instructions based on comments.


 (and you could actually even static assert it is written reasonably with 
 the trait but i could see that being hated too lol.)


Isn't that what commit hooks are for? ;)

-Steve
Nov 13 2019
next sibling parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Thursday, 14 November 2019 at 03:17:38 UTC, Steven 
Schveighoffer wrote:

 On 11/13/19 6:03 PM, Adam D. Ruppe wrote:

 On Wednesday, 13 November 2019 at 22:53:49 UTC, Steven 
 Schveighoffer wrote:

 I'm trying to save the reviewers and the maintainers. Code 
 review shouldn't include reviewing stuff that isn't compiled.


 
 Documentation is a key component of code review. And any code 
 reviewer would surely be skeptical of a __trait(getDoc) that 
 does anything other than use it for documentation.


 Where it starts becoming a problem is when users of a library 
 say "you changed your docs, and it broke my code". This is 
 always the argument against named parameters (you changed 
 parameter X's name, and it broke my code).


I'm in the camp of having parameters names "private by default" 
and requiring libraries to opt-in to exposing them.  By my 
estimate, named parameters are only useful about 15% of the time. 
  Up till now, D code is written assuming that parameter names are 
private.  They're inconsistent and not well thought out because 
they were never part of the function interface. We could come up 
with naming standards and go through all the code to fix this, 
which would be a big effort, but it's hard to justify if only a 
small percentage of functions would actually benefit from named 
parameters.  By making them "opt-in" you can fix these things 
before you expose them.

That being said, Ddoc comments differ from named parameters. DDoc 
comments are already exposed to the compiler and external tools.  
Unlike parameter names, they are already written to be publicly 
consumed by external components.  Adding another access point via 
__traits is less significant since this information is already 
exposed to other external components. DDoc comments are also 
"opt-in".  You can use regular comments if you don't want to 
expose/maintain well-formatted metadata about your code.

I'd also be careful about thinking about DDoc comments like they 
are "comments". DDoc comments may be called comments but they 
don't fit the criteria of normal comments.  They have their own 
format (a mini language within D) and have lexical significance 
to the compiler. They actually behave more like user-defined 
attributes rather than normal comments.  I have a feeling that if 
they were called something else like "DDoc Metadata" rather than 
comments then there wouldn't be much resistance to exposing them 
via __traits.
Nov 13 2019
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 14 November 2019 at 07:50:31 UTC, Jonathan Marler 
wrote:

  Up till now, D code is written assuming that parameter names 
 are private.


Well, like I said in my last email, that assumption is wrong. 
Parameter names have been exposed via CT reflection since at 
least 2011 when I wrote my first web interface generator library 
with it.

They're even fairly conveniently available with Phobos 
<http://dpldocs.info/ParameterIdentifierTuple> and you can use 
this to write named param library hacks today.

call!foo(name => "bar", other_name => 23); // possible now

and of course they are always in documentation which is even more 
important imo.


 I'd also be careful about thinking about DDoc comments like 
 they are "comments". DDoc comments may be called comments but 
 they don't fit the criteria of normal comments.


Indeed.
Nov 14 2019
parent Ethan <gooberman gmail.com> writes:
On Thursday, 14 November 2019 at 12:38:33 UTC, Adam D. Ruppe 
wrote:

 On Thursday, 14 November 2019 at 07:50:31 UTC, Jonathan Marler 
 wrote:

  Up till now, D code is written assuming that parameter names 
 are private.


 Well, like I said in my last email, that assumption is wrong. 
 Parameter names have been exposed via CT reflection since at 
 least 2011 when I wrote my first web interface generator 
 library with it.


Replying to state that Binderoo wouldn't be able to do fancy 

functionality. If you've written D code assuming they're private, 
then that code is wrong.

Design by introspection. I can even tell whether a symbol in your 
class is an alias or not.
Nov 14 2019
prev sibling next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/14/19 2:50 AM, Jonathan Marler wrote:


 That being said, Ddoc comments differ from named parameters. DDoc 
 comments are already exposed to the compiler and external tools. Unlike 
 parameter names, they are already written to be publicly consumed by 
 external components.  Adding another access point via __traits is less 
 significant since this information is already exposed to other external 
 components. DDoc comments are also "opt-in".  You can use regular 
 comments if you don't want to expose/maintain well-formatted metadata 
 about your code.


You aren't exposing anything, as long as you don't include the source. 
You can't access the compiler's comment representation. Not only that, 
but the compiler doesn't do crap unless you are outputting documents. 
The lexer will treat ddoc comments as comments and COMPLETELY IGNORE 
THEM unless it is told to deal with them, and even then, all it does is 
scrub the comment into a nicer form (removing strings of ***** before 
the /, etc.). Then it's up to the document generator to figure it out. 
But that should be considered a separate consumer of the comments, not 
part of compilation.


 I'd also be careful about thinking about DDoc comments like they are 
 "comments". DDoc comments may be called comments but they don't fit the 
 criteria of normal comments.  They have their own format (a mini 
 language within D) and have lexical significance to the compiler. They 
 actually behave more like user-defined attributes rather than normal 
 comments.  I have a feeling that if they were called something else like 
 "DDoc Metadata" rather than comments then there wouldn't be much 
 resistance to exposing them via __traits.


DDoc comments are not *required* to conform to anything. I can write 
whatever I want in a ddoc comment, and the effect is, the doc generator 
(when asked to) will generate documentation that includes the symbol 
which is associated with the comment. If you write malformed DDoc 
comments, it's still valid D code to the compiler.

What goes in that documentation is only going to look nice if you follow 
the rules, but they are not really requirements. DDoc still generates as 
well as it can if you don't give it well-formed syntax.

Or, you can just use your own doc generator, and leave the compiler out 
of it. They are just comments after all, ignored by the compiler for 
compiling code (so far).

The thing is, a documentation generator needs a parser (and in D's case, 
some semantic analysis for e.g. attribute/type inference) to generate 
nice docs. The fact that the compiler already does this helps when 
generating documentation, but generating docs is not part of compilation.

Yes, they truly are comments, and in the current state of affairs, a 
documentation generator that is included with the compiler can choose to 
do something with those comments. The code generating part is, and 
should remain, completely oblivious to the comment data.

I'm not going to continue to go around in circles here, comments are 
comments and should not affect code. I'd say the options I would agree 
with (and take this with the grain of salt that I have literally 0 power 
over what actually gets implemented) is:

1. We support fetching ddoc comment data for runtime consumption only 
(maybe even the JSON form)
2. We support a ddoc style that is not comments, and that can be both 
verified for correctness and accessed via __traits
3. Some really smart person figures out how to use the generated JSON at 
runtime and requires no updates from the compiler.

-Steve
Nov 14 2019
prev sibling parent Jab <jab_293 gmall.com> writes:
On Thursday, 14 November 2019 at 07:50:31 UTC, Jonathan Marler 
wrote:

 On Thursday, 14 November 2019 at 03:17:38 UTC, Steven 
 Schveighoffer wrote:

 On 11/13/19 6:03 PM, Adam D. Ruppe wrote:

 On Wednesday, 13 November 2019 at 22:53:49 UTC, Steven 
 Schveighoffer wrote:

 I'm trying to save the reviewers and the maintainers. Code 
 review shouldn't include reviewing stuff that isn't compiled.


 
 Documentation is a key component of code review. And any code 
 reviewer would surely be skeptical of a __trait(getDoc) that 
 does anything other than use it for documentation.


 Where it starts becoming a problem is when users of a library 
 say "you changed your docs, and it broke my code". This is 
 always the argument against named parameters (you changed 
 parameter X's name, and it broke my code).


 I'm in the camp of having parameters names "private by default" 
 and requiring libraries to opt-in to exposing them.  By my 
 estimate, named parameters are only useful about 15% of the 
 time.
  Up till now, D code is written assuming that parameter names 
 are private.  They're inconsistent and not well thought out 
 because they were never part of the function interface. We 
 could come up with naming standards and go through all the code 
 to fix this, which would be a big effort, but it's hard to 
 justify if only a small percentage of functions would actually 
 benefit from named parameters.  By making them "opt-in" you can 
 fix these things before you expose them.


That shouldn't be up to the developer of the library. If I want 
to use named parameters and I go look at the function and I see 
the author gave them horrible names. I'm probably not going to 
continue to use it and I'd probably find a different library to 
use. It's already considered bad practice to give parameters bad 
names. It is unfortunate that Phobos has some horrible names but 
it doesn't really follow a lot of best practices all together.

Making them opt-in makes the feature pointless and more 
convoluted. It's the user's choice whether they want to use them 
or not. It shouldn't be the author of a library dictating where 
the user can use named parameters.

I've seen library maintainers for C++ include function parameter 
name changes as part of the change log. Even though C++ doesn't 
have named parameters. Cause in reality the parameter name is 
already part of the interface and documentation of a function. 
That's some of the only information a user of the library sees 
when they go look at it. Developers should be giving them 
meaningful names, with or without named parameters. I don't think 
we should make a feature more convoluted because some individuals 
have bad naming practices in general.
Nov 14 2019
prev sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 14 November 2019 at 03:17:38 UTC, Steven 
Schveighoffer wrote:

 This is always the argument against named parameters (you 
 changed parameter X's name, and it broke my code).


And it is wrong there too! Doubly so, especially because 
parameter names are already exposed by CT reflection. (I use it 
on my web generator to make nice function forms, for example. But 
this isn't black magic only I know - see Phobos' 
ParameterIdentifierTuple).

You can leave names out when just interfacing with precompiled 
code, in which case they would not be accessible, same as a 
interface file without comments.

Of course, I'm also of the opinion that documentation is more 
important than code. If something appears in the documentation 
without a special warning, it is a solid part of the interface 
and cannot change. If not, it is undefined behavior and you rely 
on it at you own risk. Parameter names are documented.


 Isn't that what commit hooks are for? ;)


Most of D's ctfe stuff could be done by external build system 
things.... often more efficiently too... but it is still really 
nice to have in there.
Nov 14 2019
prev sibling parent FeepingCreature <feepingcreature gmail.com> writes:
On Wednesday, 13 November 2019 at 22:53:49 UTC, Steven 
Schveighoffer wrote:

 On 11/13/19 2:30 PM, Laeeth Isharc wrote:

 
 Compiler flag as we do with string imports?  Most projects 
 won't use this feature.  If it does end up being used then at 
 least it will be clear.  It's difficult to save people from 
 themselves completely - and I think there are many more likely 
 ways to end up abusing the powerful features of D.


 I'm trying to save the reviewers and the maintainers. Code 
 review shouldn't include reviewing stuff that isn't compiled. 
 If you are doing weird imports, that's an unusual situation and 
 requires unusual knowledge.


Our code review already includes Ddoc.

That was my point, really.

The compiler might not complain right now about invalid Ddoc, but 
I'd be happy if it did. It would not create extra effort, but 
save us effort.
Nov 13 2019