digitalmars.D - __traits(documentation, X)
- Seb (18/20) Jan 16 2018 What
- rikki cattermole (20/20) Jan 16 2018 Just a fun bit of history:
- Laeeth Isharc (6/26) Jan 16 2018 In my case might be useful for generating foreign doc strings for
- Steven Schveighoffer (4/33) Jan 16 2018 I shudder at the thought of compiled code being affected by documentatio...
- ketmar (7/9) Jan 16 2018 it's not really different from `version(DDoc)`, or string mixins, or UDA...
- Steven Schveighoffer (13/27) Jan 17 2018 version(DDoc) is very different. It's a command line option passed to
- rjframe (3/13) Jan 17 2018 I have to agree with this. If translated documentation is the primary us...
- H. S. Teoh (12/22) Jan 17 2018 +1. The thought of comments affecting code scares me, and not just a
- Simen =?UTF-8?B?S2rDpnLDpXM=?= (9/14) Jan 17 2018 That's already perfectly possible today:
- H. S. Teoh (8/21) Jan 17 2018 [...]
- H. S. Teoh (42/46) Jan 17 2018 [...]
- Simen =?UTF-8?B?S2rDpnLDpXM=?= (15/24) Jan 18 2018 I found an interesting Rust project a few years back, called
- H. S. Teoh (32/35) Jan 17 2018 [...]
- Steven Schveighoffer (9/22) Jan 18 2018 Ironically, this doesn't need any comments to trigger :)
- Adam D. Ruppe (10/12) Jan 18 2018 It is an enormous pain to do it now.... well, sort of, actually,
- Patrick Schluter (2/15) Jan 18 2018 Irony?
- Steven Schveighoffer (9/21) Jan 18 2018 This is how I'd imagine doing something like this. I don't see it being
- Adam D. Ruppe (12/20) Jan 18 2018 Yeah, it isn't terrible, but it just seems heavy for something
- Timothee Cour (6/10) Jan 18 2018 That's already the case today that documentation can affect the code
- Steven Schveighoffer (16/29) Jan 18 2018 Of course, you can cobble together something that uses the docs in your
- Seb (10/14) Jan 18 2018 You can already do this fairly easy with libdparse. No need for a
- Adam D. Ruppe (12/14) Jan 18 2018 Note that we aren't talking about comments. We're talking about
- Steven Schveighoffer (14/23) Jan 18 2018 Hm... I don't see where Ddoc comments are defined separately from
- cosinus (5/11) Jan 18 2018 I think `__traits(documentation, X)` makes sens if the
- Patrick Schluter (5/16) Jan 18 2018 While I'm on your side of the issue, it is a fact that ddoc (or
- Jonathan M Davis (4/29) Jan 17 2018 +1
- Dukc (12/15) Jan 18 2018 If a GUI button simply executes a function. Tooltip for it could
What ---- ``` /// my fancy string enum documentedEnum = 1; enum funcDoc = __traits(documentation, documentedFunc); assert(funcDoc == "my fancy string") ``` See https://github.com/dlang/dmd/pull/6872 for better examples Status ------- The naive implementation leads to a small, but noticeable increase of DMD's compilation time. Andrei's words:I'd say keep it on the back burner until we find a couple of good ideas for using it. Discussing it in the forum may help.So do you have a good use cases for this? If this is a useful feature, the implementation can be improved to be zero-cost for normal runs. Post here or at https://github.com/dlang/dmd/pull/6872
Jan 16 2018
Just a fun bit of history: A few years ago I looked into implementing this myself. Discovering that without -D documentation is lost. Why did I want to do this? From memory it was to allow me to give greater control over exportation of models+routes in Cmsed as part of documentation (or something along those lines). I wouldn't want to do it this way now, but it does still feel like a hole in traits to me. But imagine a use case of translations: /** * Translation_English: * Foo bar * Translation_X: * SDFJL */ string welcome(Language language) { return getSectionFromDDOC(__traits(comments, welcome), "Translation_" ~ language.text); }
Jan 16 2018
On Wednesday, 17 January 2018 at 02:19:11 UTC, Seb wrote:What ---- ``` /// my fancy string enum documentedEnum = 1; enum funcDoc = __traits(documentation, documentedFunc); assert(funcDoc == "my fancy string") ``` See https://github.com/dlang/dmd/pull/6872 for better examples Status ------- The naive implementation leads to a small, but noticeable increase of DMD's compilation time. Andrei's words:In my case might be useful for generating foreign doc strings for automatically generated wrappers in other languages and for D functions intended for use directly but also mounted into an interpreted DSL.. You could duplicate docstring with attributes but hardly pretty.I'd say keep it on the back burner until we find a couple of good ideas for using it. Discussing it in the forum may help.So do you have a good use cases for this? If this is a useful feature, the implementation can be improved to be zero-cost for normal runs. Post here or at https://github.com/dlang/dmd/pull/6872
Jan 16 2018
On 1/16/18 9:19 PM, Seb wrote:What ---- ``` /// my fancy string enum documentedEnum = 1; enum funcDoc = __traits(documentation, documentedFunc); assert(funcDoc == "my fancy string") ``` See https://github.com/dlang/dmd/pull/6872 for better examples Status ------- The naive implementation leads to a small, but noticeable increase of DMD's compilation time. Andrei's words:I shudder at the thought of compiled code being affected by documentation. I don't like it, sorry. -SteveI'd say keep it on the back burner until we find a couple of good ideas for using it. Discussing it in the forum may help.So do you have a good use cases for this? If this is a useful feature, the implementation can be improved to be zero-cost for normal runs. Post here or at https://github.com/dlang/dmd/pull/6872
Jan 16 2018
Steven Schveighoffer wrote:I shudder at the thought of compiled code being affected by documentation. I don't like it, sorry.it's not really different from `version(DDoc)`, or string mixins, or UDAs. i can put documentation in UDA, and process code differently when UDA changes. yes, UDA is not a comment -- but it doesn't affect code generation directly too. there is just no way to stop people from writing abominations once we got metaprogramming. and adding artificial limitations on that just makes people more inventive, but won't stop infestation. ;-)
Jan 16 2018
On 1/16/18 10:59 PM, ketmar wrote:Steven Schveighoffer wrote:version(DDoc) is very different. It's a command line option passed to the build, and affecting the build with command line options is expected. This proposal has comment *contents* affecting the build. The charter of comments is to NOT affect code, ever. They can be used to affect other systems, such as the ddoc generator, ide hints, etc., but you can be sure (?) that comments won't change code. I think this is an important line not to cross. One of the features promoted in this is to have the documentation be used for command-line help. I think it would be better to allow ddoc generation be affected by code rather than the other way around in this regard. -SteveI shudder at the thought of compiled code being affected by documentation. I don't like it, sorry.it's not really different from `version(DDoc)`, or string mixins, or UDAs. i can put documentation in UDA, and process code differently when UDA changes. yes, UDA is not a comment -- but it doesn't affect code generation directly too. there is just no way to stop people from writing abominations once we got metaprogramming. and adding artificial limitations on that just makes people more inventive, but won't stop infestation. ;-)
Jan 17 2018
On Wed, 17 Jan 2018 11:52:40 -0500, Steven Schveighoffer wrote:version(DDoc) is very different. It's a command line option passed to the build, and affecting the build with command line options is expected. This proposal has comment *contents* affecting the build. The charter of comments is to NOT affect code, ever. They can be used to affect other systems, such as the ddoc generator, ide hints, etc., but you can be sure (?) that comments won't change code. I think this is an important line not to cross.I have to agree with this. If translated documentation is the primary use case, I'd imagine we could find better ways to do it.
Jan 17 2018
On Wed, Jan 17, 2018 at 11:52:40AM -0500, Steven Schveighoffer via Digitalmars-d wrote: [...]This proposal has comment *contents* affecting the build. The charter of comments is to NOT affect code, ever. They can be used to affect other systems, such as the ddoc generator, ide hints, etc., but you can be sure (?) that comments won't change code. I think this is an important line not to cross.+1. The thought of comments affecting code scares me, and not just a little.One of the features promoted in this is to have the documentation be used for command-line help. I think it would be better to allow ddoc generation be affected by code rather than the other way around in this regard.[...] That's actually a much better idea. Have some way of injecting, say, a string UDA into ddoc so that you can have one source that generates both documentation and command-line help. The latter is already possible, and easy to implement; now we just need the former. T -- MAS = Mana Ada Sistem?
Jan 17 2018
On Wednesday, 17 January 2018 at 16:52:40 UTC, Steven Schveighoffer wrote:The charter of comments is to NOT affect code, ever. They can be used to affect other systems, such as the ddoc generator, ide hints, etc., but you can be sure (?) that comments won't change code. I think this is an important line not to cross.That's already perfectly possible today: enum a = import(__FILE__); static if (a.indexOf("//") > -1) { fireZeMissiles(); } -- Simen
Jan 17 2018
On Wed, Jan 17, 2018 at 06:24:50PM +0000, Simen Kjærås via Digitalmars-d wrote:On Wednesday, 17 January 2018 at 16:52:40 UTC, Steven Schveighoffer wrote:[...] I can see it now. In the year 2030, a major exploit is found in a widely-deployed online D application, causing massive data loss and damages. The cause? A misplaced comment... :-P T -- Never trust an operating system you don't have source for! -- Martin SchulzeThe charter of comments is to NOT affect code, ever. They can be used to affect other systems, such as the ddoc generator, ide hints, etc., but you can be sure (?) that comments won't change code. I think this is an important line not to cross.That's already perfectly possible today: enum a = import(__FILE__); static if (a.indexOf("//") > -1) { fireZeMissiles(); }
Jan 17 2018
On Wed, Jan 17, 2018 at 06:24:50PM +0000, Simen Kjærås via Digitalmars-d wrote: [...]enum a = import(__FILE__); static if (a.indexOf("//") > -1) { fireZeMissiles(); }[...] OTOH, this combination of string import and self-referencing __FILE__ opens up a host of curious possibilities (all of which already work today, btw): 1) Trivial self-reproducing program in D: import std.stdio; void main() { write(import(__FILE__)); } 2) Self-checking programs: import std.algorithm; static assert(import(__FILE__).canFind("\n/**"), "Dude, why did you remove the ddoc comment?!"); /** Remove this comment to get a compile error */ void main() { } 3) Underhanded / deceptive "self-modifying" code: version(none) { // This code will never get compiled, right? Right??? import std.stdio; void main() { writeln("Launch nuclear missiles"); } } else { mixin(() { import std.array : replace; return import(__FILE__).replace("none", "all"); }()); } These are just trivial examples, of course. One can easily imagine more sophisticated variations that can do all sorts of weird / dangerous things. Imagine an advanced version of (2), for example, code that will force a compilation error if it wasn't formatted correctly. Or an advanced form of (3) where the real program code is inside an encrypted comment written in a different language that gets compiled into D by a CTFE compiler. Or a CTFE implementation of AST macros that uses libdparse to parse itself, apply modifications, and mixin. :-D T -- "Maybe" is a strange word. When mom or dad says it it means "yes", but when my big brothers say it it means "no"! -- PJ jr.
Jan 17 2018
On Wednesday, 17 January 2018 at 22:15:08 UTC, H. S. Teoh wrote:2) Self-checking programs: import std.algorithm; static assert(import(__FILE__).canFind("\n/**"), "Dude, why did you remove the ddoc comment?!"); /** Remove this comment to get a compile error */ void main() { }[snip]Imagine an advanced version of (2), for example, code that will force a compilation error if it wasn't formatted correctly.I found an interesting Rust project a few years back, called Launch-Code: https://github.com/kmcallister/launch-code The idea is that unsafe functions be cryptographically signed by some authority, and the code refuses to compile if a junior dev tries and fuck with it. On a per-function basis, so he could still mess up the code that doesn't launch missiles. We could do the same in D - parse the code with CTFE, hash the function body, then assert that the comment matches the hash. Should be fairly easy at that point to generate the signatures as well. -- Simen
Jan 18 2018
On Wed, Jan 17, 2018 at 02:15:08PM -0800, H. S. Teoh via Digitalmars-d wrote: [...]OTOH, this combination of string import and self-referencing __FILE__ opens up a host of curious possibilities (all of which already work today, btw):[...] Another interesting one: breaking through the barriers of modularization: // mod.d string getCode(string file = __FILE__)() { return import(file); } // main.d import mod; void main() { pragma(msg, getCode); } Here, module mod gains access to the source code of a file outside of itself, and it doesn't even need prior knowledge of the filenames. In this case nothing interesting happens, of course. But imagine if getCode did something else, like extract private symbol definitions, or ran a syntax checker, or AST transformations, etc.. All sorts of interesting effects ensue. Coming back on topic, one can easily imagine a CTFE D parser that basically implements __traits(documentation) as *library code*. You won't even need compiler support for that. Wow. Skeleton code: string traitsDocumentation(alias symbol, string file = __FILE__)() { enum code = import(file); auto s = searchForDocComment(symbol.stringof); return s; } T -- Let's call it an accidental feature. -- Larry Wall
Jan 17 2018
On 1/17/18 1:24 PM, Simen Kjærås wrote:On Wednesday, 17 January 2018 at 16:52:40 UTC, Steven Schveighoffer wrote:Ironically, this doesn't need any comments to trigger :) Furthermore, I'd ask, if it's possible today, why do we need a __traits to do it? Having a __traits option for looking at documentation sounds like a very rare need in the first place to warrant promoting such a hole in the expectations for comments. I find this to also be undesirable, and I would flag any code that used such a technique as unacceptable in code review. -SteveThe charter of comments is to NOT affect code, ever. They can be used to affect other systems, such as the ddoc generator, ide hints, etc., but you can be sure (?) that comments won't change code. I think this is an important line not to cross.That's already perfectly possible today: enum a = import(__FILE__); static if (a.indexOf("//") > -1) {    fireZeMissiles(); }
Jan 18 2018
On Thursday, 18 January 2018 at 16:41:04 UTC, Steven Schveighoffer wrote:Furthermore, I'd ask, if it's possible today, why do we need a __traits to do it?It is an enormous pain to do it now.... well, sort of, actually, the way I'd do it now is do a two-step makefile, where step 1 runs a ddoc generator and step 2 import("generated.html") it. But, for stuff like little command line arg stuff, or web interface creation, it is nice to be able to print that data more conveniently. Any code that uses it in an evil way should be stupid, but writeln(__traits(documentation, foo)) is useful and not really harmless.
Jan 18 2018
On Thursday, 18 January 2018 at 16:48:45 UTC, Adam D. Ruppe wrote:On Thursday, 18 January 2018 at 16:41:04 UTC, Steven Schveighoffer wrote:Irony?Furthermore, I'd ask, if it's possible today, why do we need a __traits to do it?It is an enormous pain to do it now.... well, sort of, actually, the way I'd do it now is do a two-step makefile, where step 1 runs a ddoc generator and step 2 import("generated.html") it. But, for stuff like little command line arg stuff, or web interface creation, it is nice to be able to print that data more conveniently. Any code that uses it in an evil way should be stupid, but writeln(__traits(documentation, foo)) is useful and not really harmless.
Jan 18 2018
On 1/18/18 11:48 AM, Adam D. Ruppe wrote:On Thursday, 18 January 2018 at 16:41:04 UTC, Steven Schveighoffer wrote:This is how I'd imagine doing something like this. I don't see it being a huge pain, just an extra build step.Furthermore, I'd ask, if it's possible today, why do we need a __traits to do it?It is an enormous pain to do it now.... well, sort of, actually, the way I'd do it now is do a two-step makefile, where step 1 runs a ddoc generator and step 2 import("generated.html") it.But, for stuff like little command line arg stuff, or web interface creation, it is nice to be able to print that data more conveniently. Any code that uses it in an evil way should be stupid, but writeln(__traits(documentation, foo)) is useful and not really harmless.Did you mean not really harmful? But in any case, the idea that comments affect the file you are compiling *right now*, and not some other tool-generated file makes me very nervous. Comments are supposed to not affect the code. Consider that with this feature, the documentation now becomes part of the API. -Steve
Jan 18 2018
On Thursday, 18 January 2018 at 18:31:28 UTC, Steven Schveighoffer wrote:This is how I'd imagine doing something like this. I don't see it being a huge pain, just an extra build step.Yeah, it isn't terrible, but it just seems heavy for something small like a command line app printing out its options.Did you mean not really harmful?lol yeah.But in any case, the idea that comments affect the file you are compiling *right now*, and not some other tool-generated file makes me very nervous. Comments are supposed to not affect the code. Consider that with this feature, the documentation now becomes part of the API.Meh, I see your point, I just don't agree it is that important. I'm iffy on a lot of the things people do with UFCS and optional parenthesis/properties and think it is ugly code*, but on the balance, I still like the feature. * did you see that SO thing a bit ago about " can if(a == 1 && a == 2 && a == 3) ever be true"? Ugly code there but impure properties are useful in other cases.
Jan 18 2018
That's already the case today that documentation can affect the code because `import(__FILE__)` is legal. Both string import and `__traits(documentation)` are tools and can be used for solving real problems or abused. Let's not invoke (impossible) ideals at the expense of pragmatism; D is supposed to be pragmatic, not dogmatic.But in any case, the idea that comments affect the file you are compiling *right now*, and not some other tool-generated file makes me very nervous. Comments are supposed to not affect the code. Consider that with this feature, the documentation now becomes part of the API.
Jan 18 2018
On 1/18/18 3:21 PM, Timothee Cour wrote:Of course, you can cobble together something that uses the docs in your build process, you don't need the compiler to help create such a monster. After all, they are just text files. The point remains -- if you normalize this feature, it will be abused far easier and more frequently than it would be if it's less trivial. In any case, I stand by my assertion -- comments should be comments, not code. A related improvement: Documented unit tests. Instead of adding a feature to D where examples in the documentation were tested directly (and I'm pretty sure this was proposed), we added the ability to document unit tests. I think this was a far saner direction. A further note: Does this need go away when we make the compiler a library? Then you can extract the docs all you want and manipulate them into whatever you want using a build tool. -SteveThat's already the case today that documentation can affect the code because `import(__FILE__)` is legal. Both string import and `__traits(documentation)` are tools and can be used for solving real problems or abused. Let's not invoke (impossible) ideals at the expense of pragmatism; D is supposed to be pragmatic, not dogmatic.But in any case, the idea that comments affect the file you are compiling *right now*, and not some other tool-generated file makes me very nervous. Comments are supposed to not affect the code. Consider that with this feature, the documentation now becomes part of the API.
Jan 18 2018
On Thursday, 18 January 2018 at 20:35:54 UTC, Steven Schveighoffer wrote:A further note: Does this need go away when we make the compiler a library? Then you can extract the docs all you want and manipulate them into whatever you want using a build tool. -SteveYou can already do this fairly easy with libdparse. No need for a compiler with semantics here. Here's a D-Scanner test I wrote a while ago that checks whether all comments of public symbols have a Params section of all their parameters and a returns section if they don't return void: https://github.com/dlang-community/D-Scanner/blob/master/src/analysis/properly_documented_public_functions.d It's a shame it's only partially enabled on Phobos as there are many old modules which I had to blacklist initially.
Jan 18 2018
On Thursday, 18 January 2018 at 20:35:54 UTC, Steven Schveighoffer wrote:In any case, I stand by my assertion -- comments should be comments, not code.Note that we aren't talking about comments. We're talking about ddoc strings. They just have very similar appearance to comments, but they are distinct entities defined by D to be part of the AST (and the compiler will parse their contents too if you ask it to!), unlike regular comments which are discarded before parsing. That's really the big difference that puts me over the top: documentation strings in D are defined to be attached to the symbol by the compiler. They're ALREADY different than plain comment-comments, just the compiler withholds this information from the code.
Jan 18 2018
On 1/18/18 4:01 PM, Adam D. Ruppe wrote:On Thursday, 18 January 2018 at 20:35:54 UTC, Steven Schveighoffer wrote:Hm... I don't see where Ddoc comments are defined separately from comments in the grammar. How they are stored internally is up to the compiler, and as far as I know, the D *language* makes no guarantees on what comes out: "The specification for the form of embedded documentation comments only specifies how information is to be presented to the compiler. It is implementation-defined how that information is used and the form of the final presentation." This brings up another thing: What if the compiler you are using decides to do something completely different with the documentation (or maybe even nothing)? Then __traits(documentation, X) may be completely different and cause different code to be generated. -SteveIn any case, I stand by my assertion -- comments should be comments, not code.Note that we aren't talking about comments. We're talking about ddoc strings. They just have very similar appearance to comments, but they are distinct entities defined by D to be part of the AST (and the compiler will parse their contents too if you ask it to!), unlike regular comments which are discarded before parsing.
Jan 18 2018
On Thursday, 18 January 2018 at 21:01:01 UTC, Adam D. Ruppe wrote:Note that we aren't talking about comments. We're talking about ddoc strings. They just have very similar appearance to comments, but they are distinct entities defined by D to be part of the AST (and the compiler will parse their contents too if you ask it to!), unlike regular comments which are discarded before parsing.I think `__traits(documentation, X)` makes sens if the documentation was generated by library (AST -> HTML) But as far as I know the documentation can't be generated by CTFE directly.
Jan 18 2018
On Thursday, 18 January 2018 at 18:31:28 UTC, Steven Schveighoffer wrote:On 1/18/18 11:48 AM, Adam D. Ruppe wrote:While I'm on your side of the issue, it is a fact that ddoc (or doxygen, javadoc) comments are more a language within a language (i.e. there is syntax in them) than simple comments.[...]This is how I'd imagine doing something like this. I don't see it being a huge pain, just an extra build step.[...]Did you mean not really harmful? But in any case, the idea that comments affect the file you are compiling *right now*, and not some other tool-generated file makes me very nervous. Comments are supposed to not affect the code. Consider that with this feature, the documentation now becomes part of the API.
Jan 18 2018
On Wednesday, January 17, 2018 11:52:40 Steven Schveighoffer via Digitalmars-d wrote:On 1/16/18 10:59 PM, ketmar wrote:+1 - Jonathan M DavisSteven Schveighoffer wrote:version(DDoc) is very different. It's a command line option passed to the build, and affecting the build with command line options is expected. This proposal has comment *contents* affecting the build. The charter of comments is to NOT affect code, ever. They can be used to affect other systems, such as the ddoc generator, ide hints, etc., but you can be sure (?) that comments won't change code. I think this is an important line not to cross. One of the features promoted in this is to have the documentation be used for command-line help. I think it would be better to allow ddoc generation be affected by code rather than the other way around in this regard.I shudder at the thought of compiled code being affected by documentation. I don't like it, sorry.it's not really different from `version(DDoc)`, or string mixins, or UDAs. i can put documentation in UDA, and process code differently when UDA changes. yes, UDA is not a comment -- but it doesn't affect code generation directly too. there is just no way to stop people from writing abominations once we got metaprogramming. and adding artificial limitations on that just makes people more inventive, but won't stop infestation. ;-)
Jan 17 2018
On Wednesday, 17 January 2018 at 02:19:11 UTC, Seb wrote:So do you have a good use cases for this? If this is a useful feature, the implementation can be improved to be zero-cost for normal runs.If a GUI button simply executes a function. Tooltip for it could be generated from the function documentation. About being able to use the documentation comment in mixins, of course it would be an ill recommended thing to do. But I don't think it would become much of an issue since you can hardly do that accidently. IMO, D can already be used more dangerously than even C++ if you want to: binary imports, string mixins and static ifs can be used for far more sophiscated hacks than the C preprocessor. But it does not matter since they do not encourage or trick anybody to do the said hacks.
Jan 18 2018