www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - json generation from ddoc: painfully close

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
I just experimented with a battery of macros (json.ddoc) for generating 
json via ddoc. Results for std.algorithm are in 
http://paste.ofcode.org/DFnxChvmRGJiXYpYYk2XWr.

There are a couple of things that make the generated json invalid:

1. I couldn't get escaping to work. My ESCAPES is:

ESCAPES=/\/\\/ /"/\"/ /&/&/ /</</ />/>/

So single backslashes will be doubled and quotes will be backslashed. 
Nice. The trouble is, escaping only does its job sometimes but not 
always. I haven't yet figured the circumstances.

2. This was mentioned before - paragraph, whitespace and especially 
newline handling are handled rather poorly in ddoc. The generated json 
is full of newlines in the middle of strings, which are invalid in json 
(\n must be used). I think every paragraph should be enclosed in a 
$(DDOC_PARAGRAPH) macros that has single newlines replaced with spaces. 
Alternatively, a built-in macro could escape strings appropriately.

3. Some descriptions feature multiple examples, leading to duplicate 
"DDOC_EXAMPLES" keys. Json strongly discourages (at least) duplicate 
keys in objects. There is no way to have some autoincrement thing that 
generates "DDOC_EXAMPLES_1", "DDOC_EXAMPLES_2" etc. It could be argued 
that that's an issue with the source, not the generator.

That said, this is pretty much it. No other major impediments seem to 
stop the show. Thoughts and ideas on how to get this to work?


Andrei
Jan 08 2015
next sibling parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 8/01/2015 9:32 p.m., Andrei Alexandrescu wrote:

 I just experimented with a battery of macros (json.ddoc) for generating
 json via ddoc. Results for std.algorithm are in
 http://paste.ofcode.org/DFnxChvmRGJiXYpYYk2XWr.

 There are a couple of things that make the generated json invalid:

 1. I couldn't get escaping to work. My ESCAPES is:

 ESCAPES=/\/\\/ /"/\"/ /&/&/ /</</ />/>/

 So single backslashes will be doubled and quotes will be backslashed.
 Nice. The trouble is, escaping only does its job sometimes but not
 always. I haven't yet figured the circumstances.

 2. This was mentioned before - paragraph, whitespace and especially
 newline handling are handled rather poorly in ddoc. The generated json
 is full of newlines in the middle of strings, which are invalid in json
 (\n must be used). I think every paragraph should be enclosed in a
 $(DDOC_PARAGRAPH) macros that has single newlines replaced with spaces.
 Alternatively, a built-in macro could escape strings appropriately.

 3. Some descriptions feature multiple examples, leading to duplicate
 "DDOC_EXAMPLES" keys. Json strongly discourages (at least) duplicate
 keys in objects. There is no way to have some autoincrement thing that
 generates "DDOC_EXAMPLES_1", "DDOC_EXAMPLES_2" etc. It could be argued
 that that's an issue with the source, not the generator.

 That said, this is pretty much it. No other major impediments seem to
 stop the show. Thoughts and ideas on how to get this to work?


 Andrei


Well we could do the evil method of enabling calling of code from a 
macro. There should be enough information to do that?
We have CTFE, might as well use it.
Jan 08 2015
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/8/2015 12:32 AM, Andrei Alexandrescu wrote:

 I haven't yet figured the circumstances.


We can't just let anybody know those things.
Jan 08 2015
prev sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Jan 08, 2015 at 12:32:10AM -0800, Andrei Alexandrescu via Digitalmars-d
wrote:

 I just experimented with a battery of macros (json.ddoc) for generating json
 via ddoc. Results for std.algorithm are in
 http://paste.ofcode.org/DFnxChvmRGJiXYpYYk2XWr.
 
 There are a couple of things that make the generated json invalid:
 
 1. I couldn't get escaping to work. My ESCAPES is:
 
 ESCAPES=/\/\\/ /"/\"/ /&/&/ /</</ />/>/
 
 So single backslashes will be doubled and quotes will be backslashed.
 Nice.  The trouble is, escaping only does its job sometimes but not
 always. I haven't yet figured the circumstances.


About time somebody acknowledged that Ddoc's escaping mechanism leaves
much to be desired!



 2. This was mentioned before - paragraph, whitespace and especially
 newline handling are handled rather poorly in ddoc. The generated json
 is full of newlines in the middle of strings, which are invalid in
 json (\n must be used). I think every paragraph should be enclosed in
 a $(DDOC_PARAGRAPH) macros that has single newlines replaced with
 spaces. Alternatively, a built-in macro could escape strings
 appropriately.


Yep, also known for a long time and only now ackowledged:

	https://issues.dlang.org/show_bug.cgi?id=9731



 3. Some descriptions feature multiple examples, leading to duplicate
 "DDOC_EXAMPLES" keys. Json strongly discourages (at least) duplicate
 keys in objects. There is no way to have some autoincrement thing that
 generates "DDOC_EXAMPLES_1", "DDOC_EXAMPLES_2" etc. It could be argued
 that that's an issue with the source, not the generator.


[...]

I disagree there's any issue with the source. The current system allows
the very nice idiom of:

	/// ... some docs here
	auto myFunc(...) { ... }

	/// This example shows feature X of myFunc.
	unittest { ... }

	/// This example shows feature Y of myFunc.
	unittest { ... }

The generated docs read like this:

	auto myFunc(...);

	... some docs here

	This example shows feature X of myFunc.

	... [first example here]

	This example shows feature Y of myFunc.

	... [second example here]

This is much more meaningful than a single gigantic example block that
the reader has to parse in his head in order to get through everything
being showcased.

Of course, the mechanics of it can be improved -- ddoc should know
better than to use the same heading "Examples:" over and over. Perhaps,
in terms of the underlying macros, it should have some kind of
autoincrementing label for each ddoc'd unittest block.


T

-- 
Stop staring at me like that! It's offens... no, you'll hurt your eyes!
Jan 08 2015