www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Another use case for __traits(docComment)

reply Adam D. Ruppe <destructionator gmail.com> writes:
Writing code to wrap some Phobos functions for a script language 
and need documentation to be displayed at runtime...

Would be really nice to just say if(other_docs.length == 0) 
return __traits(docComment, item).

A UDA can populate other_docs in some functions, but when 
wrapping libraries you can't add that. Having the doc comment 
available through reflection means we can reuse these existing 
items.
Nov 08
next sibling parent FeepingCreature <feepingcreature gmail.com> writes:
On Friday, 8 November 2019 at 14:00:03 UTC, Adam D. Ruppe wrote:
 Writing code to wrap some Phobos functions for a script 
 language and need documentation to be displayed at runtime...

 Would be really nice to just say if(other_docs.length == 0) 
 return __traits(docComment, item).

 A UDA can populate other_docs in some functions, but when 
 wrapping libraries you can't add that. Having the doc comment 
 available through reflection means we can reuse these existing 
 items.
Yes, please!!
Nov 11
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/8/19 9:00 AM, Adam D. Ruppe wrote:
 Writing code to wrap some Phobos functions for a script language and 
 need documentation to be displayed at runtime...
 
 Would be really nice to just say if(other_docs.length == 0) return 
 __traits(docComment, item).
 
 A UDA can populate other_docs in some functions, but when wrapping 
 libraries you can't add that. Having the doc comment available through 
 reflection means we can reuse these existing items.
I still am very VERY leery of having anything in comments affect the code in any way. What about a compromise? Provide access to the docs but ONLY at runtime. That is, you will not know anything about the string at compiletime, just at runtime, and you can make your decisions there. Should work for your use case. -Steve
Nov 11
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 11 November 2019 at 17:16:20 UTC, Steven Schveighoffer 
wrote:
 I still am very VERY leery of having anything in comments 
 affect the code in any way.
then just.... don't do that. And btw for the record you could have comments affect code right now: mixin((__LINE__ % 3) ? "one thing" : "other thing"); // adding a comment above would change the line number and possibly trigger the other thing! but such a thing is obviously absurd and nobody would write that, unless maybe doing an obfuscated contest. The same thing is true of documentation comments. You *could* abuse... but why would you? UDAs are easier to use for other purposes anyway. (just right now we are using UDAs for docs just because it is needed elsewhere!)
 What about a compromise? Provide access to the docs but ONLY at 
 runtime. That is, you will not know anything about the string 
 at compiletime, just at runtime, and you can make your 
 decisions there.
Yeah, that'd work for this case but it is a pointless restriction.
Nov 11
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/11/19 12:54 PM, Adam D. Ruppe wrote:
 On Monday, 11 November 2019 at 17:16:20 UTC, Steven Schveighoffer wrote:
 I still am very VERY leery of having anything in comments affect the 
 code in any way.
then just.... don't do that. And btw for the record you could have comments affect code right now: mixin((__LINE__ % 3) ? "one thing" : "other thing"); // adding a comment above would change the line number and possibly trigger the other thing!
Haha, yeah, that is what I don't want. But I think we can safely say that using comments to affect code can result in reasonable looking things (the above is NOT reasonable) that can be exploited by adjusting the comments. Certainly, it could be allowed at compile-time and just be a guideline not to write code that cares about the data. But we all know that it will happen. For instance, I like how documented unit tests simply turns the unittest into a ddoc, instead of making ddoc turn into a unit test (which likely could have been the solution if __traits(docComment) were available back then). It's the correct translation, because code can always be turned into comments, but the other way around is questionable.
 The same thing is true of documentation comments. You *could* abuse... 
 but why would you? UDAs are easier to use for other purposes anyway. 
 (just right now we are using UDAs for docs just because it is needed 
 elsewhere!)
One thing that's different from __LINE__ is that __LINE__ is only provided to that line of code. It's not accessible outside the line. __traits(docComment, x) would provide access to the comments from elsewhere. So it may be external abuse that happens.
 
 What about a compromise? Provide access to the docs but ONLY at 
 runtime. That is, you will not know anything about the string at 
 compiletime, just at runtime, and you can make your decisions there.
Yeah, that'd work for this case but it is a pointless restriction.
It would be something that could potentially be expanded later if needed. It's easier to let the toothpaste out of the tube slowly than try to put some of it back in. -Steve
Nov 11
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 11 November 2019 at 20:15:56 UTC, Steven Schveighoffer 
wrote:
 Certainly, it could be allowed at compile-time and just be a 
 guideline not to write code that cares about the data. But we 
 all know that it will happen.
Indeed, I have seen a lot of trash D code out there. But I don't want to remove useful features because of it.
 For instance, I like how documented unit tests simply turns the 
 unittest into a ddoc, instead of making ddoc turn into a unit 
 test (which likely could have been the solution if 
 __traits(docComment) were available back then).
I actually doubt that would have been used because that needs significant progressing anyway. What we did use before the unittest thing was just extract the code from the generated HTML documentation which is pretty easy too. (that's probably what I'll do with my program here, just dump the comments to a separate file and then `import` it to populate the map. but so much more complicated and fragile than just pulling the comment off while reflecting anyway)
 It would be something that could potentially be expanded later 
 if needed. It's easier to let the toothpaste out of the tube 
 slowly than try to put some of it back in.
true. like I'd rather have it just for runtime processing than not at all, but it is still just bleh.
Nov 11
prev sibling parent reply Exil <Exil gmall.com> writes:
On Monday, 11 November 2019 at 17:54:07 UTC, Adam D. Ruppe wrote:
 On Monday, 11 November 2019 at 17:16:20 UTC, Steven 
 Schveighoffer wrote:
 I still am very VERY leery of having anything in comments 
 affect the code in any way.
then just.... don't do that.
That won't stop other people.
 And btw for the record you could have comments affect code 
 right now:

 mixin((__LINE__ % 3) ? "one thing" : "other thing"); // adding 
 a comment above would change the line number and possibly 
 trigger the other thing!


 but such a thing is obviously absurd and nobody would write 
 that, unless maybe doing an obfuscated contest.
That isn't really all that useful. That line of code's only purpose is to be misused and cause misunderstanding. Someone might come around and say, why modify documentation and code? Why not just modify documentation and then have the code read from that documentation and change behavior instead. That way you keep documentation and implementation in sync! Kind of odd you have a 1:1 mapping for a scripting language anyways. That doesn't seem like a very rare occurrence. You don't pass any extra meta data for your implementations? They all have 1:1 mapping for types? You implement things like pointers in your scripting language?
Nov 11
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 12 November 2019 at 01:22:09 UTC, Exil wrote:
 Someone might come around and say, why modify documentation and 
 code? Why not just modify documentation and then have the code 
 read from that documentation and change behavior instead. That 
 way you keep documentation and implementation in sync!
Heh, that might actually be kinda interesting. But you can already do that kind of thing, just with slightly different syntax. You can put that in a UDA and process it with reflection at CT and embed it into documentation through the UDA too. (my doc generator supports pulling in UDAs to work around the lack of ddoc in reflection; I had to move docs to a UDA and now they're combined anyway, it is just hideous and not as compatible with third party things)
 You don't pass any extra meta data for your implementations?
It depends, but a lot of libraries actually do just work with the script languages and it is nice to just do, for example `script.parseHTML = (string html) => new Document(html);` and then much of the Document object is now available to the script! But since the Document library is written with ddoc, it is impossible for me to bring in those docs (without an external map generator and loader) for the interactive script. $ var document = parseHTML(....); $ \help document.querySelector no help tho ddoc is present and the method is available to the script :( I also do a web interface generator. You give it a class, it wraps it up in HTML forms, etc. It would be nice if it gave you the documentation... but it can't even tell if it is there. Best it can do is blindly link to my documentation website and *hope* it is there. (which actually isn't bad, but again it would be cool if it were more fully integrated) Or use a UDA, since these wrappers don't go as deep as automatically as the script language, so it isn't too bad either, it would just be nice if the ddoc worked.
Nov 11
parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Tuesday, 12 November 2019 at 01:59:55 UTC, Adam D. Ruppe wrote:
 On Tuesday, 12 November 2019 at 01:22:09 UTC, Exil wrote:
 Someone might come around and say, why modify documentation 
 and code? Why not just modify documentation and then have the 
 code read from that documentation and change behavior instead. 
 That way you keep documentation and implementation in sync!
Heh, that might actually be kinda interesting. But you can already do that kind of thing, just with slightly different syntax. You can put that in a UDA and process it with reflection at CT and embed it into documentation through the UDA too. (my doc generator supports pulling in UDAs to work around the lack of ddoc in reflection; I had to move docs to a UDA and now they're combined anyway, it is just hideous and not as compatible with third party things)
This is also my view. D already supports things that are this messy or more. All this restriction does is lock the barn after the horse has bolted. You can already do the same degree of mess. You just have to make it ugly. All this restriction does is make good, useful code worse for little benefit.
Nov 11
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/12/19 1:32 AM, FeepingCreature wrote:
 On Tuesday, 12 November 2019 at 01:59:55 UTC, Adam D. Ruppe wrote:
 On Tuesday, 12 November 2019 at 01:22:09 UTC, Exil wrote:
 Someone might come around and say, why modify documentation and code? 
 Why not just modify documentation and then have the code read from 
 that documentation and change behavior instead. That way you keep 
 documentation and implementation in sync!
Heh, that might actually be kinda interesting. But you can already do that kind of thing, just with slightly different syntax. You can put that in a UDA and process it with reflection at CT and embed it into documentation through the UDA too. (my doc generator supports pulling in UDAs to work around the lack of ddoc in reflection; I had to move docs to a UDA and now they're combined anyway, it is just hideous and not as compatible with third party things)
This is also my view. D already supports things that are this messy or more. All this restriction does is lock the barn after the horse has bolted. You can already do the same degree of mess. You just have to make it ugly. All this restriction does is make good, useful code worse for little benefit.
Breaking the fundamental rule of "this is a comment, it doesn't affect code" that we all learn on day 1 of programming needs a very very compelling use case. I haven't seen it yet. Alternatively, we could use something other than comments for documentation. e.g.: ddoc { .... // documentation } -Steve
Nov 12
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 12 November 2019 at 15:14:17 UTC, Steven 
Schveighoffer wrote:
 Alternatively, we could use something other than comments for 
 documentation. e.g.:
lol ddoc!q{ /++ +/ } that would actually work and the doc generator can recognize the pattern too. But there's a big difference: struct A { int a; /// does something } that is cute syntax and not repeatable with UDAs. And of course, my big use case is to fetch docs for third party libs I'd rather not edit the source of anyway. Now see, this is an argument against the fear that it will get abused though simply because the language has easier ways to do other things. Using UDAs for docs is doable but awkward, so we prefer to use /++ +/. Similarly, abusing doc comments as UDAs is going to be more awkward than using the real thing - you can use the language to structure it and all. So why do things the hard way when there's an easy way?
Nov 12
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/12/19 11:23 AM, Adam D. Ruppe wrote:
 On Tuesday, 12 November 2019 at 15:14:17 UTC, Steven Schveighoffer wrote:
 Alternatively, we could use something other than comments for 
 documentation. e.g.:
lol ddoc!q{ /++ +/ } that would actually work and the doc generator can recognize the pattern too. But there's a big difference: struct A {    int a; /// does something }
This is really not much different: struct A { ddoc(does something) int a; } But I mean, we can do whatever we want. It doesn't have to fit into the world of UDA. It can be its own syntax. struct A { int a; !ddoc does something } It just shouldn't be a comment, which the compiler is supposed to ignore.
 And of course, my big use case is to fetch docs for third party libs I'd 
 rather not edit the source of anyway.
Well, assuming this isn't just a mental exercise, the answer would be, if you want it to be accessible to __traits(getDocs), you need to write it the other way. If that means you can't access the docs, then you can't access the docs. I would expect a dfix solution is braindead simple here.
 Now see, this is an argument against the fear that it will get abused 
 though simply because the language has easier ways to do other things. 
 Using UDAs for docs is doable but awkward, so we prefer to use /++ +/.
 
 Similarly, abusing doc comments as UDAs is going to be more awkward than 
 using the real thing - you can use the language to structure it and all. 
 So why do things the hard way when there's an easy way?
The thought of comments *ever* affecting the code is just plain wrong to me. Comments are not code, we should keep that line bright. -Steve
Nov 13
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 13 November 2019 at 16:03:55 UTC, Steven 
Schveighoffer wrote:
 It just shouldn't be a comment, which the compiler is supposed 
 to ignore.
The D specification makes a distinction between ordinary comments and ddoc comments. ddoc comments are well-defined by the language and attached to symbols by parsing rules. It currently doesn't expose it directly - it only processes it into separate files (not ignoring them!) - but I want to change that, and appealing to the authority of the status quo isn't much of a counterargument.
 The thought of comments *ever* affecting the code is just plain 
 wrong to me. Comments are not code, we should keep that line 
 bright.
I agree for regular comments, but ddoc is already a part of the language and processed by the compiler! All I want to do is let other parts of the language see it too. And again, UDAs do a better job at embedding non-doc stuff than doc comments. So I really think the risk of people going crazy with them is quite small since the alternative is superior for this task.
Nov 13
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/13/19 11:12 AM, Adam D. Ruppe wrote:
 On Wednesday, 13 November 2019 at 16:03:55 UTC, Steven Schveighoffer wrote:
 It just shouldn't be a comment, which the compiler is supposed to ignore.
The D specification makes a distinction between ordinary comments and ddoc comments. ddoc comments are well-defined by the language and attached to symbols by parsing rules.
Well defined, but it still compiles even if you don't follow the spec. These are comments still, and a secondary product from the binary output.
 It currently doesn't expose it directly - it only processes it into 
 separate files (not ignoring them!) - but I want to change that, and 
 appealing to the authority of the status quo isn't much of a 
 counterargument.
In reality you could separate the ddoc parsing and generation from the compile step anyway. It's just that it's much more convenient to use the already parsed AST for the code to generate the documentation. In this way, the *code* affects the documentation, not the other way around. The status quo should remain this way because code is complicated enough without requiring comments to "compile" correctly too. However, I'm OK with something other than comments being used to generate the doc reading code, OR deal with it at runtime when the compiler is not involved. Note that ddoc comments are still not ddoc data. You are missing a lot of stuff with just the string. Wouldn't it be nice to just import the JSON data generated by the compiler? That's how I would do it.
 
 The thought of comments *ever* affecting the code is just plain wrong 
 to me. Comments are not code, we should keep that line bright.
I agree for regular comments, but ddoc is already a part of the language and processed by the compiler! All I want to do is let other parts of the language see it too. And again, UDAs do a better job at embedding non-doc stuff than doc comments. So I really think the risk of people going crazy with them is quite small since the alternative is superior for this task.
I've seen lots of schemes where comments are significant to some process or another. IDE directives for instance. But these schemes can play around in this realm *because* the compiler ignores them. I never want to see a case where comments are significant and break compilation or introduce bugs. We have enough to review in the code itself... -Steve
Nov 13
parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Wednesday, 13 November 2019 at 16:33:58 UTC, Steven 
Schveighoffer wrote:
 On 11/13/19 11:12 AM, Adam D. Ruppe wrote:
 On Wednesday, 13 November 2019 at 16:03:55 UTC, Steven 
 Schveighoffer wrote:
 It just shouldn't be a comment, which the compiler is 
 supposed to ignore.
The D specification makes a distinction between ordinary comments and ddoc comments. ddoc comments are well-defined by the language and attached to symbols by parsing rules.
Well defined, but it still compiles even if you don't follow the spec. These are comments still, and a secondary product from the binary output.
Easily fixed: make the trait fail if the comment violates the Ddoc guidelines.
 It currently doesn't expose it directly - it only processes it 
 into separate files (not ignoring them!) - but I want to 
 change that, and appealing to the authority of the status quo 
 isn't much of a counterargument.
In reality you could separate the ddoc parsing and generation from the compile step anyway. It's just that it's much more convenient to use the already parsed AST for the code to generate the documentation. In this way, the *code* affects the documentation, not the other way around. The status quo should remain this way because code is complicated enough without requiring comments to "compile" correctly too.
We already require comments to compile correctly. That's what Ddoc *is*. Documentation should not be considered an optional part of functionality, especially if the user is trying to use Ddoc comments.
 However, I'm OK with something other than comments being used 
 to generate the doc reading code, OR deal with it at runtime 
 when the compiler is not involved.

 Note that ddoc comments are still not ddoc data. You are 
 missing a lot of stuff with just the string. Wouldn't it be 
 nice to just import the JSON data generated by the compiler? 
 That's how I would do it.
That would be nice. However, imagine this usecase: Swagger is a documentation format for REST requests. It would be handy if this documentation could be served right alongside the requests themselves. However, this would require a two-step process where the code is compiled, the Ddoc comments are discarded; then the code is, for no good reason, compiled again with a separate tool that wraps the D compiler, having to ensure it gets the same compiler binary and same compiler flags, creating an additional point of error (for no reason!), then dynamically loaded back into a process and processed at runtime, when they could have been equally well evaluated via a trait at compiletime and processed in ctfe with std.json to generate static output with nicely integrated error checking with line numbers.
 I've seen lots of schemes where comments are significant to 
 some process or another. IDE directives for instance.

 But these schemes can play around in this realm *because* the 
 compiler ignores them. I never want to see a case where 
 comments are significant and break compilation or introduce 
 bugs. We have enough to review in the code itself...

 -Steve
Again: Ddoc comments are *already* a mandatory part of compilation. If code compiles but the comments violate Ddoc, it should under no circumstances be considered "working." D is a static language. To me, that means: any runtime failure that can be a compiletime failure should be. Compilation failing is annoying; but, compilation failing *rather than* runtime failing is the entire point of static types.
Nov 13
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/13/19 12:19 PM, FeepingCreature wrote:
 On Wednesday, 13 November 2019 at 16:33:58 UTC, Steven Schveighoffer wrote:
 I've seen lots of schemes where comments are significant to some 
 process or another. IDE directives for instance.

 But these schemes can play around in this realm *because* the compiler 
 ignores them. I never want to see a case where comments are 
 significant and break compilation or introduce bugs. We have enough to 
 review in the code itself...
Again: Ddoc comments are *already* a mandatory part of compilation. If code compiles but the comments violate Ddoc, it should under no circumstances be considered "working." D is a static language. To me, that means: any runtime failure that can be a compiletime failure should be. Compilation failing is annoying; but, compilation failing *rather than* runtime failing is the entire point of static types.
I'm not convinced: import std.stdio; /// $(BADMACRO boo) void main() { writeln("compiles and runs!"); } Even generates ddoc! Just that, (BADMACRO boo) does nothing. It doesn't even produce an error. Let's mess with the parsing: /// $(BADMACRO boo note the missing parentheses. No errors, even when compiled with -D. I get a correct html doc, but now I see "(BADMACRO boo" as the docs for main as if it were just a string (curiously missing the $ but whatevs, it's just comments and docs!) I wouldn't consider this to be "mandatory" or even not "working". I get a working binary, and I get a working ddoc (I see main in the docs, and the junk that I stuck in there). You want to introduce into this a way to create *errors* when existing ddocs don't have to even parse correctly? The D spec says what DDoc is, but doesn't say that anything has to conform to ddoc (indeed, someone might even use another doc system and have their comments written for it), or even that DDoc will fail to produce documentation if it's not correct! -Steve
Nov 13
prev sibling parent reply Dennis <dkorpel gmail.com> writes:
On Tuesday, 12 November 2019 at 06:32:35 UTC, FeepingCreature 
wrote:
 You can already do the same degree of mess. You just have to 
 make it ugly. All this restriction does is make good, useful 
 code worse for little benefit.
And that's exactly the point: you have to make it ugly, so it isn't an attractive option. Some things are ugly by design, such as __gshared, discouraging users from using it while still allowing it if needed. Similarly, version() statements don't allow boolean logic, discouraging those annoying double negations and version formulas (see e.g. [1], [2]) and encouraging 'positive' identifier names that don't have 'no' in them. You can still make version spaghetti with `static if` if you want, but it won't be as appealing. Technically you can still do C macro shenanigans like `#define BEGIN {` in D by putting your module in a q{} string, doing a string-replace in CTFE and mixing it in. "You just have to make it ugly". And that's exactly why D programmers don't tend to do that. While I do see the value in __traits(docComment) (I would use it myself in the same way Adam wants to use it), I do have to agree with Steven that it just feels to powerful. As soon as some hacker discovers `__traits(docComment)` he will have the 'brilliant' idea to inspect documentation and use parts of it for his code generation, and it will look elegant so it doesn't seem wrong (unlike some convoluted parse(import(__FILE__)) scheme). And then a clueless contributor will be like 'how the heck did running dfmt on this codebase make the unittests fail'? UDA's use string literals, so the rules of what string you get are exactly clear and a formatter won't touch it. How does __traits(docComment) behave on these two comments: ``` /** Word * Word */ /****************** Word Word */ ``` What strings do they produce? [1] https://github.com/PetteriAimonen/libfixmath/blob/c3017d7d74867398dcdcaa21d93c6a0b039b9bca/libfixmath/fix16.c#L8 [2] https://github.com/PetteriAimonen/libfixmath/blob/c3017d7d74867398dcdcaa21d93c6a0b039b9bca/libfixmath/fix16.c#L67
Nov 13
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 13 November 2019 at 15:22:23 UTC, Dennis wrote:
 UDA's use string literals, so the rules of what string you get 
 are exactly clear and a formatter won't touch it. How does 
 __traits(docComment) behave on these two comments:
This is actually defined in the D language specification https://dlang.org/spec/ddoc.html "The extra *'s and +'s on the comment opening, closing and left margin are ignored and are not part of the embedded documentation. Comments not following one of those forms are not documentation comments. " It is a little bit ambiguous to get /** Word * word */ Is that "* word" supposed to ignore the star? dmd ignores it which is probably most consistent with the spec. My adrdox does NOT ignore that and can use it for markdown-style lists... But regardless, the __trait thing should do what the compiler already does for ddoc. Though I would NOT parse the ddoc, just give the string as the compiler read it.
 ```
 /** Word
  *  Word
  */

 /******************
 Word
 Word
 */
 ```

 What strings do they produce?
" Word\n Word\n " and "Word\nWord\n" So not identical, but this is well defined by the D language spec!
Nov 13
parent reply Dennis <dkorpel gmail.com> writes:
On Wednesday, 13 November 2019 at 15:40:34 UTC, Adam D. Ruppe 
wrote:
 So not identical, but this is well defined by the D language 
 spec!
Well defined, but it requires the programmer to know those defined rules. I wasn't sure whether in my first example there would be a double space between the two 'Word's, but it turns out there is, and now I need to account for that when printing the text after getting it with __traits(docComment). I don't have this problem when doing documentation generation, I just do `dub build --build=docs` and I get html pages that look fine (no double spaces), without needing to know what happens under the hood.
 Is that "* word" supposed to ignore the star? dmd ignores it 
 which is probably most consistent with the spec. My adrdox does 
 NOT ignore that and can use it for markdown-style lists...
That's probably going to give new bug reports too. ("Why did my list bullets disappear with __traits(docComment)? I see it just fine when rendering with adrdox!")
Nov 13
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 13 November 2019 at 18:02:35 UTC, Dennis wrote:
 I don't have this problem when doing documentation generation, 
 I just do `dub build --build=docs` and I get html pages that 
 look fine (no double spaces), without needing to know what 
 happens under the hood.
That's because HTML ignores extra spaces. If you were to do __traits(docComment) and output it to an HTML file, you'd see that same result. The space is still there, it is just rendered differently by the target environment. All __traits(docComment) should do is return the source code after the compiler read it. (actually, I wouldn't even mind if it was the whole comment, in full, including the delimiters, but if the compiler already removes stars I'm ok with that too)
 That's probably going to give new bug reports too. ("Why did my 
 list bullets disappear with __traits(docComment)? I see it just 
 fine when rendering with adrdox!")
Well, adrdox is not compatible with ddoc exactly, so I'd just accept that.
Nov 13
parent reply Dennis <dkorpel gmail.com> writes:
On Wednesday, 13 November 2019 at 18:46:43 UTC, Adam D. Ruppe 
wrote:
 That's because HTML ignores extra spaces.
Of course, but the overall point I'm trying to make is that the trait is non-trivial and adds language complexity. It gets implemented one way, and then someone comes around wanting it to be different. "please give me the html instead of raw comment, parsing it with CTFE is slow and cumbersome" "please include the /** or /++, I'm using __traits(docComment) to statically assert my preferred documentation code style" "is it a bug that leading/trailing whitespace is/isn't trimmed?" "the previous changes broke my existing CTFE documentation parser" "__traits(docComment) does not work when the comment is in a mixin that is seen later in semantic analysis" "I want only my doc comment of my function to depend on the build type, but putting the comment inside version blocks does not work! How to fix?" This can of worms can stay closed by just sticking with good old predictable and flexible UDA strings.
Nov 13
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 13 November 2019 at 19:15:56 UTC, Dennis wrote:
 Of course, but the overall point I'm trying to make is that the 
 trait is non-trivial and adds language complexity.
It doesn't have to be. All I want is the existing string. The ONLY addition I am calling for is that a trait docComment (or whatever) returns the ddoc string attached to the declaration (and if the existing implementation strips delimiters, so be it, and if not, so be it, the spec does define stripping them so i assume it does but I don't really care). Everything else is already defined or out of scope, so we really shouldn't be worrying about them.
 It gets implemented one way, and then someone comes around 
 wanting it to be different. "please give me the html instead of 
 raw comment
Impossible, since that's undefined.
 "is it a bug that leading/trailing whitespace is/isn't trimmed?"
Clear answer to this: no, it is not a bug. That's defined to be there.
 "the previous changes broke my existing CTFE documentation 
 parser"
ddoc is part of the spec and it has had breaking changes, we just deal with it.
 "__traits(docComment) does not work when the comment is in a 
 mixin that is seen later in semantic analysis"
This is similarly already an open question with the language. Should __traits(allMembers) return stuff from mixins?
 "I want only my doc comment of my function to depend on the 
 build type, but putting the comment inside version blocks does 
 not work! How to fix?"
This is also already well-defined, version applies to declarations, doc comments are part of a declaration, not a separate entity.
Nov 13
parent reply Dennis <dkorpel gmail.com> writes:
On Wednesday, 13 November 2019 at 19:44:38 UTC, Adam D. Ruppe 
wrote:
 The ONLY addition I am calling for is that a trait docComment 
 (or whatever) returns the ddoc string attached to the 
 declaration
That may be the only thing _you_ want, but then someone else comes along who wants something different. All Manu wanted was a mangle-only `extern(C++, "ns")` as an additional option, so simple. Why was Walter initially so resistant for such a harmless addition? Then it got merged. https://github.com/dlang/dmd/pull/8667 Then the issues come in. https://issues.dlang.org/show_bug.cgi?id=19278 https://issues.dlang.org/show_bug.cgi?id=19339 https://issues.dlang.org/show_bug.cgi?id=19557 https://issues.dlang.org/show_bug.cgi?id=19574 https://issues.dlang.org/show_bug.cgi?id=19893 https://issues.dlang.org/show_bug.cgi?id=19920 And someone wants to deprecate the old way. https://github.com/dlang/dmd/pull/10031 I'm not saying it wasn't worth it in that case, but those fictional remarks about __traits(docComment) were supposed to show that the trait will be ANOTHER source of enhancement requests, introduces ANOTHER thing that can get queried during any point of semantic analysis, probably making it ANOTHER source of DMD bugs, and it will be ANOTHER source of confusion / material for D learners to swallow. That's just how it goes despite how simple it may look, and the question is whether the new trait justifies this cost. Not every code annoyance warrants a new feature, sometimes it's better to e.g. duplicate a few lines of code instead of introducing another C-macro. Likewise, in this case having some duplication in UDA's or having another build step may be the easier solution in the end.
Nov 13
parent reply Jab <jab_293 gmall.com> writes:
On Wednesday, 13 November 2019 at 20:38:07 UTC, Dennis wrote:
 On Wednesday, 13 November 2019 at 19:44:38 UTC, Adam D. Ruppe 
 wrote:
 The ONLY addition I am calling for is that a trait docComment 
 (or whatever) returns the ddoc string attached to the 
 declaration
That may be the only thing _you_ want, but then someone else comes along who wants something different. All Manu wanted was a mangle-only `extern(C++, "ns")` as an additional option, so simple. Why was Walter initially so resistant for such a harmless addition? Then it got merged. https://github.com/dlang/dmd/pull/8667 Then the issues come in. https://issues.dlang.org/show_bug.cgi?id=19278 https://issues.dlang.org/show_bug.cgi?id=19339 https://issues.dlang.org/show_bug.cgi?id=19557 https://issues.dlang.org/show_bug.cgi?id=19574 https://issues.dlang.org/show_bug.cgi?id=19893 https://issues.dlang.org/show_bug.cgi?id=19920
IIRC there was a WIP pull request accompanied by a DIP, the pull request was merged while it was still being ironed out along with the DIP. Those were all easy fixes foir the most part. Hell there's still bugs being fixed regarding extern(C). With that kind of justification no new features should ever be added. Not to mention bug fixes that just cause no bugs. We should just stop making changes all together. Point to one of D's biggest mistakes, even then I would still say that's not a good enough reason to not make changes.
 And someone wants to deprecate the old way.

 https://github.com/dlang/dmd/pull/10031
They probably want to, because the old way is impeding the new method, you can't use CTFE to construct a string.
 I'm not saying it wasn't worth it in that case, but those 
 fictional remarks about __traits(docComment) were supposed to 
 show that the trait will be ANOTHER source of enhancement 
 requests, introduces ANOTHER thing that can get queried during 
 any point of semantic analysis, probably making it ANOTHER 
 source of DMD bugs, and it will be ANOTHER source of confusion 
 / material for D learners to swallow.

 That's just how it goes despite how simple it may look, and the 
 question is whether the new trait justifies this cost. Not 
 every code annoyance warrants a new feature, sometimes it's 
 better to e.g. duplicate a few lines of code instead of 
 introducing another C-macro.
Here's a counter example to your example: https://github.com/dlang/dmd/pull/10013 I wouldn't really say it's a new "feature". It'd just be exposing the information that is already available
 Likewise, in this case having some duplication in UDA's or
It's not just duplication, you'd have to modifying other people's source code. How would you duplicate this doc: /// ditto The implementation then becomes much more convoluted.
 having another build step may be the easier solution in the end.
Using dub? That's going to be a pain in the ass honestly. This sort of implementation you'd have to do isn't that simple. You'd have to effectively run a dub command twice, once to generate the doc output and then run it again to compile the code with the now available documentation. Dub wasn't designed for that, and build systems for C/C++ are often overly complicated because the languages don't expose information that the compiler already has available to it. Basically of what D does right with it's compile time features. __traits already has an atrocious eye catching name. If you need to find code somewhere that might be doing something with comments that it shouldn't be doing, it's not going to be that difficult to find.
Nov 13
parent reply Dennis <dkorpel gmail.com> writes:
On Thursday, 14 November 2019 at 05:03:49 UTC, Jab wrote:
 With that kind of justification no new features should ever be 
 added. Not to mention bug fixes that just cause no bugs. We 
 should just stop making changes all together.
I just said that every feature has (hidden) cost, not that that cost is never justified.
 They probably want to, because the old way is impeding the new 
 method, you can't use CTFE to construct a string.
That's not the point, the point is that it always starts with "but I just want X, nothing more!", and later inevitably someone wants Y too. Concretely in that case: "It can live peacefully beside your preferred design with the scopes." followed by the PR to deprecate extern(C++, identifier).
 Here's a counter example to your example: 
 https://github.com/dlang/dmd/pull/10013
I actually think that got merged too quickly without thinking all special cases and consequences through.
 I wouldn't really say it's a new "feature". It'd just be 
 exposing the information that is already available
Whatever you call it, it's another source of complexity as I mentioned before.
 How would you duplicate this doc:

 /// ditto
Put the UDA string in an enum? ``` enum string fooComment = "does foo" /// does foo (fooComment) void foo(int a); /// ditto (fooComment) void foo(string a); ``` It's hard to argue what the best current solution is without a concrete usecase. If you write some functions / fields yourself, you can write some UDA strings that may be redundant with your doc comments. If you want to expose documentation from an existing library (e.g. Phobos), it can be extracted beforehand instead.
Nov 14
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 14 November 2019 at 09:54:35 UTC, Dennis wrote:
 That's not the point, the point is that it always starts with 
 "but I just want X, nothing more!", and later inevitably 
 someone wants Y too.
Once I have what I want, I'm not above just saying no to anyone else :P
Nov 14
prev sibling parent reply Jab <jab_293 gmall.com> writes:
On Thursday, 14 November 2019 at 09:54:35 UTC, Dennis wrote:
 On Thursday, 14 November 2019 at 05:03:49 UTC, Jab wrote:
 With that kind of justification no new features should ever be 
 added. Not to mention bug fixes that just cause no bugs. We 
 should just stop making changes all together.
I just said that every feature has (hidden) cost, not that that cost is never justified.
Your making it sound like it isn't justified because of those hidden costs.
 They probably want to, because the old way is impeding the new 
 method, you can't use CTFE to construct a string.
That's not the point, the point is that it always starts with "but I just want X, nothing more!", and later inevitably someone wants Y too. Concretely in that case: "It can live peacefully beside your preferred design with the scopes." followed by the PR to deprecate extern(C++, identifier).
The whole purpose of extern(C++, "") was because extern(C++, identifier) was inadequate. If you want to look at it one way. The extern(C++, "") syntax was only wanted because the extern(C++, identifier) was there. Should we have not added extern(C++, identifier) to D to begin with? People are going to want to make changes to things they don't like. Before extern(C++, identifer) was added people were probably asking for a way to link with C++ mangling. Should they have not been given that feature because down the line there's going to be numerous bugs with it that will need to be fixed and people will then be wanting more and more changes to it?
 Here's a counter example to your example: 
 https://github.com/dlang/dmd/pull/10013
I actually think that got merged too quickly without thinking all special cases and consequences through.
Has there been any consequences?
 I wouldn't really say it's a new "feature". It'd just be 
 exposing the information that is already available
Whatever you call it, it's another source of complexity as I mentioned before.
 How would you duplicate this doc:

 /// ditto
Put the UDA string in an enum? ``` enum string fooComment = "does foo" /// does foo (fooComment) void foo(int a); /// ditto (fooComment) void foo(string a); ``` It's hard to argue what the best current solution is without a concrete usecase. If you write some functions / fields yourself, you can write some UDA strings that may be redundant with your doc comments. If you want to expose documentation from an existing library (e.g. Phobos), it can be extracted beforehand instead.
Was kind of curious what that looks like with a comment from phobos, I think it's worse than I expected. enum cmpComment = ` Performs a lexicographical comparison on two $(REF_ALTTEXT input ranges, isInputRange, std,range,primitives). Iterating `r1` and `r2` in lockstep, `cmp` compares each element `e1` of `r1` with the corresponding element `e2` in `r2`. If one of the ranges has been finished, `cmp` returns a negative value if `r1` has fewer elements than `r2`, a positive value if `r1` has more elements than `r2`, and `0` if the ranges have the same number of elements. If the ranges are strings, `cmp` performs UTF decoding appropriately and compares the ranges one code point at a time. A custom predicate may be specified, in which case `cmp` performs a three-way lexicographical comparison using `pred`. Otherwise the elements are compared using `opCmp`. Params: pred = Predicate used for comparison. Without a predicate specified the ordering implied by `opCmp` is used. r1 = The first range. r2 = The second range. Returns: `0` if the ranges compare equal. A negative value if `r1` is a prefix of `r2` or the first differing element of `r1` is less than the corresponding element of `r2` according to `pred`. A positive value if `r2` is a prefix of `r1` or the first differing element of `r2` is less than the corresponding element of `r1` according to `pred`. Note: An earlier version of the documentation incorrectly stated that `-1` is the only negative value returned and `1` is the only positive value returned. Whether that is true depends on the types being compared.`; /********************************** Performs a lexicographical comparison on two $(REF_ALTTEXT input ranges, isInputRange, std,range,primitives). Iterating `r1` and `r2` in lockstep, `cmp` compares each element `e1` of `r1` with the corresponding element `e2` in `r2`. If one of the ranges has been finished, `cmp` returns a negative value if `r1` has fewer elements than `r2`, a positive value if `r1` has more elements than `r2`, and `0` if the ranges have the same number of elements. If the ranges are strings, `cmp` performs UTF decoding appropriately and compares the ranges one code point at a time. A custom predicate may be specified, in which case `cmp` performs a three-way lexicographical comparison using `pred`. Otherwise the elements are compared using `opCmp`. Params: pred = Predicate used for comparison. Without a predicate specified the ordering implied by `opCmp` is used. r1 = The first range. r2 = The second range. Returns: `0` if the ranges compare equal. A negative value if `r1` is a prefix of `r2` or the first differing element of `r1` is less than the corresponding element of `r2` according to `pred`. A positive value if `r2` is a prefix of `r1` or the first differing element of `r2` is less than the corresponding element of `r1` according to `pred`. Note: An earlier version of the documentation incorrectly stated that `-1` is the only negative value returned and `1` is the only positive value returned. Whether that is true depends on the types being compared. */ docComment(cmpComment) auto cmp(R1, R2)(R1 r1, R2 r2);
Nov 14
next sibling parent reply Dennis <dkorpel gmail.com> writes:
On Thursday, 14 November 2019 at 17:18:40 UTC, Jab wrote:
 Should we have not added extern(C++, identifier) to D to begin 
 with?
 Should they have not been given that feature because down the 
 line there's going to be numerous bugs with it that will need 
 to be fixed and people will then be wanting more and more 
 changes to it?
I haven't interfaced with C++ so I can't judge that, but from a spectator of the discussion point of view `extern(C++, "ns")` seemed like a worthy addition. I only brought this up as an example to show how these things don't end with "just this one small addition" like Adam makes it out to be in the case of __traits(docComment).
 Was kind of curious what that looks like with a comment from 
 phobos, I think it's worse than I expected.
So I wouldn't use that for Phobos, but export Phobos documentation to a preferred file format and use that. What would you do with the raw ddoc comment anyway, print it in the terminal as-is with $() and all? Re-implement macro parsing in CTFE? The UDA duplication is acceptable when you are writing some small comments yourself: ``` struct WindowSettings { ("The default width of the window in pixels.") int width = 1280; /// default window width in pixels ("The default height of the window in pixels.") int height = 720; /// default window height in pixels } ``` If you want to make a GUI for setting the width and height based on the struct then you want to poll the UDA string to get a nice description for the user. And I know it is tempting to want to combine the two, based on the availability of __traits(docComment): ``` struct WindowSettings { int width = 1280; /// The default width of the window in pixels. int height = 720; /// The default height of the window in pixels. } ``` But before you know it you deal with all kinds of issues, like the user-facing comment and the programmer-facing comment diverging, needing to strip the leading space, the formatter changing it to a multi-line comment introducing other whitespace, wanting to the comment to be generated at CTFE so you have to turn back to UDA's anyway etc. And it may just turn out that simple > clever all along.
Nov 14
parent reply Jab <jab_293 gmall.com> writes:
On Thursday, 14 November 2019 at 20:24:46 UTC, Dennis wrote:
 On Thursday, 14 November 2019 at 17:18:40 UTC, Jab wrote:
 Should we have not added extern(C++, identifier) to D to begin 
 with?
 Should they have not been given that feature because down the 
 line there's going to be numerous bugs with it that will need 
 to be fixed and people will then be wanting more and more 
 changes to it?
I haven't interfaced with C++ so I can't judge that, but from a spectator of the discussion point of view `extern(C++, "ns")` seemed like a worthy addition. I only brought this up as an example to show how these things don't end with "just this one small addition" like Adam makes it out to be in the case of __traits(docComment).
For him it probably would be, but there's numerous people out there that want something else. There's always going to be something people will want to improve on. I don't think that's a good enough reason to not do something. The merits of any additional changes should be evaluated on their merits if they do come up afterwards.
 Was kind of curious what that looks like with a comment from 
 phobos, I think it's worse than I expected.
So I wouldn't use that for Phobos, but export Phobos documentation to a preferred file format and use that. What would you do with the raw ddoc comment anyway, print it in the terminal as-is with $() and all? Re-implement macro parsing in CTFE? The UDA duplication is acceptable when you are writing some small comments yourself: ``` struct WindowSettings { ("The default width of the window in pixels.") int width = 1280; /// default window width in pixels ("The default height of the window in pixels.") int height = 720; /// default window height in pixels } ``` If you want to make a GUI for setting the width and height based on the struct then you want to poll the UDA string to get a nice description for the user. And I know it is tempting to want to combine the two, based on the availability of __traits(docComment): ``` struct WindowSettings { int width = 1280; /// The default width of the window in pixels. int height = 720; /// The default height of the window in pixels. } ``` But before you know it you deal with all kinds of issues, like the user-facing comment and the programmer-facing comment diverging, needing to strip the leading space, the formatter changing it to a multi-line comment introducing other whitespace, wanting to the comment to be generated at CTFE so you have to turn back to UDA's anyway etc. And it may just turn out that simple > clever all along.
See small examples like that aren't indicative of actual code. Struct settings I've used look more like this, that you usually can't see in one full screen window when everything is on one line: struct WindowSettings { ("The default width of the window in pixels.") int width = 1280; /// default window width in pixels ("The default height of the window in pixels.") int height = 720; /// default window height in pixels ("The default width of the window in pixels.") int width = 1280; /// default window width in pixels ("The default height of the window in pixels.") int height = 720; /// default window height in pixels ("The default width of the window in pixels.") int width = 1280; /// default window width in pixels ("The default height of the window in pixels.") int height = 720; /// default window height in pixels ("The default width of the window in pixels.") int width = 1280; /// default window width in pixels ("The default height of the window in pixels.") int height = 720; /// default window height in pixels ("The default width of the window in pixels.") int width = 1280; /// default window width in pixels ("The default height of the window in pixels.") int height = 720; /// default window height in pixels ("The default width of the window in pixels.") int width = 1280; /// default window width in pixels ("The default height of the window in pixels.") int height = 720; /// default window height in pixels ("The default width of the window in pixels.") int width = 1280; /// default window width in pixels ("The default height of the window in pixels.") int height = 720; /// default window height in pixels ("The default width of the window in pixels.") int width = 1280; /// default window width in pixels ("The default height of the window in pixels.") int height = 720; /// default window height in pixels ("The default width of the window in pixels.") int width = 1280; /// default window width in pixels ("The default height of the window in pixels.") int height = 720; /// default window height in pixels ("The default width of the window in pixels.") int width = 1280; /// default window width in pixels ("The default height of the window in pixels.") int height = 720; /// default window height in pixels } struct WindowSettings { int width = 1280; /// default window width in pixels int height = 720; /// default window height in pixels int width = 1280; /// default window width in pixels int height = 720; /// default window height in pixels int width = 1280; /// default window width in pixels int height = 720; /// default window height in pixels int width = 1280; /// default window width in pixels int height = 720; /// default window height in pixels int width = 1280; /// default window width in pixels int height = 720; /// default window height in pixels int width = 1280; /// default window width in pixels int height = 720; /// default window height in pixels int width = 1280; /// default window width in pixels int height = 720; /// default window height in pixels int width = 1280; /// default window width in pixels int height = 720; /// default window height in pixels int width = 1280; /// default window width in pixels int height = 720; /// default window height in pixels int width = 1280; /// default window width in pixels int height = 720; /// default window height in pixels } Even that is still small in comparison. You could put everything on one line but then you are going to be going past any reasonable width of code. It'd make it even more unreadable if the doc is longer than usual as well. It'll just be doubling the width instead of the height. Neither is ideal.
Nov 15
parent reply Dennis <dkorpel gmail.com> writes:
On Friday, 15 November 2019 at 17:15:12 UTC, Jab wrote:
 Struct settings I've used look more like this, that you usually 
 can't see in one full screen window when everything is on one 
 line:
As you get more and more struct members the chance that the documentation comment can actually double as user-facing comments becomes smaller and smaller. Trying to keep everything simple and compact like that is not going to scale, as soon as e.g. your application gets localized and supports foreign languages your one-liner multi-purpose documentation comments have to go anyway.
Nov 15
parent reply Jab <jab_293 gmall.com> writes:
On Friday, 15 November 2019 at 17:57:59 UTC, Dennis wrote:
 On Friday, 15 November 2019 at 17:15:12 UTC, Jab wrote:
 Struct settings I've used look more like this, that you 
 usually can't see in one full screen window when everything is 
 on one line:
As you get more and more struct members the chance that the documentation comment can actually double as user-facing comments becomes smaller and smaller. Trying to keep everything simple and compact like that is not going to scale, as soon as e.g. your application gets localized and supports foreign languages your one-liner multi-purpose documentation comments have to go anyway.
Docs don't really ever get localized. Otherwise phobos, D, and basically all programming languages all have the same problem.
Nov 18
parent Dennis <dkorpel gmail.com> writes:
On Tuesday, 19 November 2019 at 05:30:46 UTC, Jab wrote:
 Docs don't really ever get localized.
Yeah but you want to make documentation comment also be the explanation text that appears in the GUI of an application (in this example), which might get localized.
Nov 19
prev sibling parent Dennis <dkorpel gmail.com> writes:
On Thursday, 14 November 2019 at 17:18:40 UTC, Jab wrote:
 Here's a counter example to your example: 
 https://github.com/dlang/dmd/pull/10013
I actually think that got merged too quickly without thinking all special cases and consequences through.
Has there been any consequences?
I luckily haven't seen anything bad yet, though it's really new. I would say a consequence is that it makes re-ordering module-level declarations a potentially code-breaking change, though that information was already leaked with __LINE__, __traits(allMembers) and some DMD bugs [1] so 'luckily' the programmer couldn't count on that anyway. [1] https://issues.dlang.org/show_bug.cgi?id=9125
Nov 14
prev sibling parent reply Laeeth Isharc <laeeth kaleidic.io> writes:
On Monday, 11 November 2019 at 17:16:20 UTC, Steven Schveighoffer 
wrote:
 On 11/8/19 9:00 AM, Adam D. Ruppe wrote:
 Writing code to wrap some Phobos functions for a script 
 language and need documentation to be displayed at runtime...
 
 Would be really nice to just say if(other_docs.length == 0) 
 return __traits(docComment, item).
 
 A UDA can populate other_docs in some functions, but when 
 wrapping libraries you can't add that. Having the doc comment 
 available through reflection means we can reuse these existing 
 items.
I still am very VERY leery of having anything in comments affect the code in any way. What about a compromise? Provide access to the docs but ONLY at runtime. That is, you will not know anything about the string at compiletime, just at runtime, and you can make your decisions there. Should work for your use case. -Steve
Compiler flag as we do with string imports? Most projects won't use this feature. If it does end up being used then at least it will be clear. It's difficult to save people from themselves completely - and I think there are many more likely ways to end up abusing the powerful features of D. Suppose in scenario 1 you use doc comment traits and in scenario 2 you have an extra build stage to achieve the same effect; and suppose through unfortunate choices the documentation ends up affecting the runtime behaviour of the code in non-documentation related ways. Are you better off in scenario 1 or 2? It's not clear, but 1 might be better. D has goto - people don't habitually abuse it.
Nov 13
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/13/19 2:30 PM, Laeeth Isharc wrote:
 On Monday, 11 November 2019 at 17:16:20 UTC, Steven Schveighoffer wrote:
 I still am very VERY leery of having anything in comments affect the 
 code in any way.

 What about a compromise? Provide access to the docs but ONLY at 
 runtime. That is, you will not know anything about the string at 
 compiletime, just at runtime, and you can make your decisions there.

 Should work for your use case.
Compiler flag as we do with string imports?  Most projects won't use this feature.  If it does end up being used then at least it will be clear.  It's difficult to save people from themselves completely - and I think there are many more likely ways to end up abusing the powerful features of D.
I'm trying to save the reviewers and the maintainers. Code review shouldn't include reviewing stuff that isn't compiled. If you are doing weird imports, that's an unusual situation and requires unusual knowledge. If you put this into traits, then it's a usual and easy tool, and now you have to verify comments are correctly written. If all you want is to fetch the documentation for a function, it can be done in better ways.
 
 Suppose in scenario 1 you use doc comment traits and in scenario 2 you 
 have an extra build stage to achieve the same effect; and suppose 
 through unfortunate choices the documentation ends up affecting the 
 runtime behaviour of the code in non-documentation related ways.  Are 
 you better off in scenario 1 or 2?  It's not clear, but 1 might be better.
For sure, it's possible already to run extra steps to have *anything* affect the code. You could use a preprocessor that replaces ddocs with UDAs for the same effect. But in that case, you are altering the input to the compiler, not using the compiler to build something that isn't normal compiler input. Of course, you can already do such things with string imports.
 D has goto - people don't habitually abuse it.
This is more akin to mixins, not goto. Goto is actually easy to review (automatic rejection! /jk), you know exactly what it does. And yes, mixins are abused. I just did some work to the diet-ng project to make it possible to update non-interpolated HTML and have it figured out at runtime[1]. The way diet imports non-D code and changes it into D code to mixin, it's cool, but I'd consider it an abuse, and really difficult to comprehend. I'm getting to the point where I just think an external tool is better[2]. -Steve [1] https://github.com/rejectedsoftware/diet-ng/pull/70 [2] https://github.com/schveiguy/dietpc
Nov 13
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 13 November 2019 at 22:53:49 UTC, Steven 
Schveighoffer wrote:
 I'm trying to save the reviewers and the maintainers. Code 
 review shouldn't include reviewing stuff that isn't compiled.
Documentation is a key component of code review. And any code reviewer would surely be skeptical of a __trait(getDoc) that does anything other than use it for documentation. (and you could actually even static assert it is written reasonably with the trait but i could see that being hated too lol.)
Nov 13
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/13/19 6:03 PM, Adam D. Ruppe wrote:
 On Wednesday, 13 November 2019 at 22:53:49 UTC, Steven Schveighoffer wrote:
 I'm trying to save the reviewers and the maintainers. Code review 
 shouldn't include reviewing stuff that isn't compiled.
Documentation is a key component of code review. And any code reviewer would surely be skeptical of a __trait(getDoc) that does anything other than use it for documentation.
Where it starts becoming a problem is when users of a library say "you changed your docs, and it broke my code". This is always the argument against named parameters (you changed parameter X's name, and it broke my code). Anything that becomes observable at compile-time now becomes part of the API. Theoretically, even a runtime string could cause this. But there is much less possibility for abuse, and at least the compiler doesn't write *different* binary instructions based on comments.
 (and you could actually even static assert it is written reasonably with 
 the trait but i could see that being hated too lol.)
Isn't that what commit hooks are for? ;) -Steve
Nov 13
next sibling parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Thursday, 14 November 2019 at 03:17:38 UTC, Steven 
Schveighoffer wrote:
 On 11/13/19 6:03 PM, Adam D. Ruppe wrote:
 On Wednesday, 13 November 2019 at 22:53:49 UTC, Steven 
 Schveighoffer wrote:
 I'm trying to save the reviewers and the maintainers. Code 
 review shouldn't include reviewing stuff that isn't compiled.
Documentation is a key component of code review. And any code reviewer would surely be skeptical of a __trait(getDoc) that does anything other than use it for documentation.
Where it starts becoming a problem is when users of a library say "you changed your docs, and it broke my code". This is always the argument against named parameters (you changed parameter X's name, and it broke my code).
I'm in the camp of having parameters names "private by default" and requiring libraries to opt-in to exposing them. By my estimate, named parameters are only useful about 15% of the time. Up till now, D code is written assuming that parameter names are private. They're inconsistent and not well thought out because they were never part of the function interface. We could come up with naming standards and go through all the code to fix this, which would be a big effort, but it's hard to justify if only a small percentage of functions would actually benefit from named parameters. By making them "opt-in" you can fix these things before you expose them. That being said, Ddoc comments differ from named parameters. DDoc comments are already exposed to the compiler and external tools. Unlike parameter names, they are already written to be publicly consumed by external components. Adding another access point via __traits is less significant since this information is already exposed to other external components. DDoc comments are also "opt-in". You can use regular comments if you don't want to expose/maintain well-formatted metadata about your code. I'd also be careful about thinking about DDoc comments like they are "comments". DDoc comments may be called comments but they don't fit the criteria of normal comments. They have their own format (a mini language within D) and have lexical significance to the compiler. They actually behave more like user-defined attributes rather than normal comments. I have a feeling that if they were called something else like "DDoc Metadata" rather than comments then there wouldn't be much resistance to exposing them via __traits.
Nov 13
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 14 November 2019 at 07:50:31 UTC, Jonathan Marler 
wrote:
  Up till now, D code is written assuming that parameter names 
 are private.
Well, like I said in my last email, that assumption is wrong. Parameter names have been exposed via CT reflection since at least 2011 when I wrote my first web interface generator library with it. They're even fairly conveniently available with Phobos <http://dpldocs.info/ParameterIdentifierTuple> and you can use this to write named param library hacks today. call!foo(name => "bar", other_name => 23); // possible now and of course they are always in documentation which is even more important imo.
 I'd also be careful about thinking about DDoc comments like 
 they are "comments". DDoc comments may be called comments but 
 they don't fit the criteria of normal comments.
Indeed.
Nov 14
parent Ethan <gooberman gmail.com> writes:
On Thursday, 14 November 2019 at 12:38:33 UTC, Adam D. Ruppe 
wrote:
 On Thursday, 14 November 2019 at 07:50:31 UTC, Jonathan Marler 
 wrote:
  Up till now, D code is written assuming that parameter names 
 are private.
Well, like I said in my last email, that assumption is wrong. Parameter names have been exposed via CT reflection since at least 2011 when I wrote my first web interface generator library with it.
Replying to state that Binderoo wouldn't be able to do fancy things like generate C++/C# interfaces without this functionality. If you've written D code assuming they're private, then that code is wrong. Design by introspection. I can even tell whether a symbol in your class is an alias or not.
Nov 14
prev sibling next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/14/19 2:50 AM, Jonathan Marler wrote:

 That being said, Ddoc comments differ from named parameters. DDoc 
 comments are already exposed to the compiler and external tools. Unlike 
 parameter names, they are already written to be publicly consumed by 
 external components.  Adding another access point via __traits is less 
 significant since this information is already exposed to other external 
 components. DDoc comments are also "opt-in".  You can use regular 
 comments if you don't want to expose/maintain well-formatted metadata 
 about your code.
You aren't exposing anything, as long as you don't include the source. You can't access the compiler's comment representation. Not only that, but the compiler doesn't do crap unless you are outputting documents. The lexer will treat ddoc comments as comments and COMPLETELY IGNORE THEM unless it is told to deal with them, and even then, all it does is scrub the comment into a nicer form (removing strings of ***** before the /, etc.). Then it's up to the document generator to figure it out. But that should be considered a separate consumer of the comments, not part of compilation.
 I'd also be careful about thinking about DDoc comments like they are 
 "comments". DDoc comments may be called comments but they don't fit the 
 criteria of normal comments.  They have their own format (a mini 
 language within D) and have lexical significance to the compiler. They 
 actually behave more like user-defined attributes rather than normal 
 comments.  I have a feeling that if they were called something else like 
 "DDoc Metadata" rather than comments then there wouldn't be much 
 resistance to exposing them via __traits.
DDoc comments are not *required* to conform to anything. I can write whatever I want in a ddoc comment, and the effect is, the doc generator (when asked to) will generate documentation that includes the symbol which is associated with the comment. If you write malformed DDoc comments, it's still valid D code to the compiler. What goes in that documentation is only going to look nice if you follow the rules, but they are not really requirements. DDoc still generates as well as it can if you don't give it well-formed syntax. Or, you can just use your own doc generator, and leave the compiler out of it. They are just comments after all, ignored by the compiler for compiling code (so far). The thing is, a documentation generator needs a parser (and in D's case, some semantic analysis for e.g. attribute/type inference) to generate nice docs. The fact that the compiler already does this helps when generating documentation, but generating docs is not part of compilation. Yes, they truly are comments, and in the current state of affairs, a documentation generator that is included with the compiler can choose to do something with those comments. The code generating part is, and should remain, completely oblivious to the comment data. I'm not going to continue to go around in circles here, comments are comments and should not affect code. I'd say the options I would agree with (and take this with the grain of salt that I have literally 0 power over what actually gets implemented) is: 1. We support fetching ddoc comment data for runtime consumption only (maybe even the JSON form) 2. We support a ddoc style that is not comments, and that can be both verified for correctness and accessed via __traits 3. Some really smart person figures out how to use the generated JSON at runtime and requires no updates from the compiler. -Steve
Nov 14
prev sibling parent Jab <jab_293 gmall.com> writes:
On Thursday, 14 November 2019 at 07:50:31 UTC, Jonathan Marler 
wrote:
 On Thursday, 14 November 2019 at 03:17:38 UTC, Steven 
 Schveighoffer wrote:
 On 11/13/19 6:03 PM, Adam D. Ruppe wrote:
 On Wednesday, 13 November 2019 at 22:53:49 UTC, Steven 
 Schveighoffer wrote:
 I'm trying to save the reviewers and the maintainers. Code 
 review shouldn't include reviewing stuff that isn't compiled.
Documentation is a key component of code review. And any code reviewer would surely be skeptical of a __trait(getDoc) that does anything other than use it for documentation.
Where it starts becoming a problem is when users of a library say "you changed your docs, and it broke my code". This is always the argument against named parameters (you changed parameter X's name, and it broke my code).
I'm in the camp of having parameters names "private by default" and requiring libraries to opt-in to exposing them. By my estimate, named parameters are only useful about 15% of the time. Up till now, D code is written assuming that parameter names are private. They're inconsistent and not well thought out because they were never part of the function interface. We could come up with naming standards and go through all the code to fix this, which would be a big effort, but it's hard to justify if only a small percentage of functions would actually benefit from named parameters. By making them "opt-in" you can fix these things before you expose them.
That shouldn't be up to the developer of the library. If I want to use named parameters and I go look at the function and I see the author gave them horrible names. I'm probably not going to continue to use it and I'd probably find a different library to use. It's already considered bad practice to give parameters bad names. It is unfortunate that Phobos has some horrible names but it doesn't really follow a lot of best practices all together. Making them opt-in makes the feature pointless and more convoluted. It's the user's choice whether they want to use them or not. It shouldn't be the author of a library dictating where the user can use named parameters. I've seen library maintainers for C++ include function parameter name changes as part of the change log. Even though C++ doesn't have named parameters. Cause in reality the parameter name is already part of the interface and documentation of a function. That's some of the only information a user of the library sees when they go look at it. Developers should be giving them meaningful names, with or without named parameters. I don't think we should make a feature more convoluted because some individuals have bad naming practices in general.
Nov 14
prev sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 14 November 2019 at 03:17:38 UTC, Steven 
Schveighoffer wrote:
 This is always the argument against named parameters (you 
 changed parameter X's name, and it broke my code).
And it is wrong there too! Doubly so, especially because parameter names are already exposed by CT reflection. (I use it on my web generator to make nice function forms, for example. But this isn't black magic only I know - see Phobos' ParameterIdentifierTuple). You can leave names out when just interfacing with precompiled code, in which case they would not be accessible, same as a interface file without comments. Of course, I'm also of the opinion that documentation is more important than code. If something appears in the documentation without a special warning, it is a solid part of the interface and cannot change. If not, it is undefined behavior and you rely on it at you own risk. Parameter names are documented.
 Isn't that what commit hooks are for? ;)
Most of D's ctfe stuff could be done by external build system things.... often more efficiently too... but it is still really nice to have in there.
Nov 14
prev sibling parent FeepingCreature <feepingcreature gmail.com> writes:
On Wednesday, 13 November 2019 at 22:53:49 UTC, Steven 
Schveighoffer wrote:
 On 11/13/19 2:30 PM, Laeeth Isharc wrote:
 
 Compiler flag as we do with string imports?  Most projects 
 won't use this feature.  If it does end up being used then at 
 least it will be clear.  It's difficult to save people from 
 themselves completely - and I think there are many more likely 
 ways to end up abusing the powerful features of D.
I'm trying to save the reviewers and the maintainers. Code review shouldn't include reviewing stuff that isn't compiled. If you are doing weird imports, that's an unusual situation and requires unusual knowledge.
Our code review already includes Ddoc. That was my point, really. The compiler might not complain right now about invalid Ddoc, but I'd be happy if it did. It would not create extra effort, but save us effort.
Nov 13