www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.internals - Potential changes to DDoc

reply David Gileadi <gileadisNOSPM gmail.com> writes:
(I don't know if this is the right forum for this post, so if not please 
let me know.)

I'm working on adding Markdown support to DDoc. I'm doing it by 
modifying the highlightText method in doc.d [1].

I've been trying to make my changes non-breaking, in that existing DDoc 
files should be rendered the same as they currently are. However I've 
realized that there may need to be some breakage, and I'd like to 
discuss it.

First of all, the CSS styles in the default DDoc theme remove all 
styling from headers, blockquotes, list items, pre and code sections, 
etc. [2] The effect of this is that Markdown features like headers, 
quotes, etc. don't display correctly under the default theme. I'd like 
to modify the default DDoc theme to avoid de-styling these elements.

Secondly, HTML is treated differently under Markdown and DDoc. In DDoc 
anything directly within an HTML tag (i.e. between < and >) is passed 
through unchanged. However if it's not part of the tag it is treated as 
regular DDoc text and subject to regular processing. In Markdown all 
text between a given set of HTML tags is passed through unchanged [3]. 
I'd prefer to switch to the CommonMark handling, allowing whole blocks 
of HTML to be passed through directly.

Thirdly and most controversially is how DDoc handles blank lines between 
paragraphs. Markdown wraps paragraphs in <p></p> tags and collapses 
multiple blank lines between paragraphs (i.e. ten blank lines is treated 
the same as one). DDoc inserts $(DDOC_BLANKLINE) macros on each blank 
line it encounters, which in the default theme inserts <br><br>, in the 
Phobos theme inserts <div class="blankline"/>, and for LaTeX inserts 
/par. Phobos docs also have the common practice of hand-wrapping 
paragraphs in $(P) macros which inserts <p></p> tags.

The result of DDoc's handling is that under the default theme if you 
have, say, three blank lines between paragraphs then it results in five 
blank lines being rendered between them.

I'm not yet sure it's feasible, but I'd really like to change DDoc to 
treat paragraphs the way Markdown does--wrap them in, say, a $(P) macro 
and collapse multiple blank lines. Any desired extra blank lines could 
be inserted by hand, via the $(BR) macro or similar. However this could 
change the rendering of existing documentation which depends on the 
$(DDOC_BLANKLINE) macro.

What do you think about the proposed changes? Should I try to implement 
any or all of them?

-Dave


Appendix A: Planned Markdown support
------------------------------------

I plan to support the following Markdown features, as specified by 
CommonMark [4]:

- Headings, both the ATX and setext kind
- Styling text with * (but not with _)
- Quoting text blocks with >
- Code blocks fenced with ```, ~~~ and DDoc's existing ---
- Backtick-quoted code as it currently works in DDoc
- Nesting lists
- Thematic breaks (<hr>)
- Inline links, although I'm not sure if I will support reference links [5]
- Markdown escapes with \
- HTML, although that's a discussion item (see above)

I don't plan to support the following Markdown features:

- Indented code blocks because existing DDoc comments are often already 
indented
- Using --- for HRs and second-level headings because DDoc uses it as a 
code block fence
- Tables
- Backtick-quoting code across multiple lines
- Github-specific features like task lists, mentioning people and teams, 
referencing issues and pull requests, and emoji


[1] https://github.com/dgileadi/dmd/tree/ddoc-markdown
[2] https://github.com/dlang/dmd/blob/master/res/default_ddoc_theme.ddoc#L70
[3] http://spec.commonmark.org/0.28/#html-blocks
[4] http://spec.commonmark.org/0.28/
[5] http://spec.commonmark.org/0.28/#reference-link
Jan 20 2018
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
I think it's great that you're working on implementing it!

I don't have time at the moment to go over your proposal, but here's the 
previous long thread on it:

http://www.digitalmars.com/d/archives/digitalmars/D/Adding_Markdown_to_Ddoc_308861.html

Regarding breaking changes:

There's far too much existing Ddoc code. I don't think we can afford
significant 
breaking changes.
Jan 20 2018
parent David Gileadi <gileadisNOSPM gmail.com> writes:
On 1/20/18 1:45 PM, Walter Bright wrote:

 I think it's great that you're working on implementing it!
 
 I don't have time at the moment to go over your proposal, but here's the 
 previous long thread on it:
 
 http://www.digitalmars.com/d/archives/digitalmars/D/Adding_Markdown
to_Ddoc_308861.html 
 
 
 Regarding breaking changes:
 
 There's far too much existing Ddoc code. I don't think we can afford 
 significant breaking changes.


I understand. When you get a few moments, I hope you can take a look at 
what I'm proposing.
Jan 20 2018
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2018-01-20 19:33, David Gileadi wrote:


 First of all, the CSS styles in the default DDoc theme remove all 
 styling from headers, blockquotes, list items, pre and code sections, 
 etc. [2] The effect of this is that Markdown features like headers, 
 quotes, etc. don't display correctly under the default theme. I'd like 
 to modify the default DDoc theme to avoid de-styling these elements.


I'm the author of the current Ddoc default theme. As far as I know it's 
common practice to reset the style for standard tags because the 
browsers might render it differently otherwise. If any tags are not 
render the way you like, just add the appropriate styling. But please, 
do not remove the resetting of the style for the standard tags.

-- 
/Jacob Carlborg
Jan 21 2018
next sibling parent David Gileadi <gileadisNOSPM gmail.com> writes:
On 1/21/18 10:55 AM, Jacob Carlborg wrote:

 On 2018-01-20 19:33, David Gileadi wrote:
 

 First of all, the CSS styles in the default DDoc theme remove all 
 styling from headers, blockquotes, list items, pre and code sections, 
 etc. [2] The effect of this is that Markdown features like headers, 
 quotes, etc. don't display correctly under the default theme. I'd like 
 to modify the default DDoc theme to avoid de-styling these elements.


 
 I'm the author of the current Ddoc default theme. As far as I know it's 
 common practice to reset the style for standard tags because the 
 browsers might render it differently otherwise. If any tags are not 
 render the way you like, just add the appropriate styling. But please, 
 do not remove the resetting of the style for the standard tags.


There are different opinions on the amount of resets; some CSS resets 
preserve some styles while normalizing others. However I don't have a 
strong opinion either way so I'll stick with your recommendation and add 
back the necessary styles. Once (if?) this makes it to a PR I'd 
appreciate your input on the chosen styles, to help them to better fit 
the theme.
Jan 21 2018
prev sibling parent Kagamin <spam here.lot> writes:
On Sunday, 21 January 2018 at 17:55:20 UTC, Jacob Carlborg wrote:

 I'm the author of the current Ddoc default theme. As far as I 
 know it's common practice to reset the style for standard tags 
 because the browsers might render it differently otherwise.


AFAIK, w3c and gcc docs don't do that. I'd say browsers must 
render it as they like, why not?
Jan 23 2018
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/20/2018 10:33 AM, David Gileadi wrote:

 I've been trying to make my changes non-breaking, in that existing DDoc files 
 should be rendered the same as they currently are. However I've realized that 
 there may need to be some breakage, and I'd like to discuss it.
 
 First of all, the CSS styles in the default DDoc theme remove all styling from 
 headers, blockquotes, list items, pre and code sections, etc. [2] The effect
of 
 this is that Markdown features like headers, quotes, etc. don't display 
 correctly under the default theme. I'd like to modify the default DDoc theme
to 
 avoid de-styling these elements.


I don't understand how styling enters into this. Markdown, such as:

    * cat
    * dog
    * horse

should be simply rewritten by the highlighter to:

    $(UL
    $(LI cat)
    $(LI dog)
    $(LI horse)
    )

and then let the macros do their thing.



 Secondly, HTML is treated differently under Markdown and DDoc. In DDoc
anything 
 directly within an HTML tag (i.e. between < and >) is passed through
unchanged. 
 However if it's not part of the tag it is treated as regular DDoc text and 
 subject to regular processing. In Markdown all text between a given set of
HTML 
 tags is passed through unchanged [3]. I'd prefer to switch to the CommonMark 
 handling, allowing whole blocks of HTML to be passed through directly.


When Markdown and Ddoc conflict, do it the Ddoc way.



 Thirdly and most controversially is how DDoc handles blank lines between 
 paragraphs. Markdown wraps paragraphs in <p></p> tags and collapses multiple 
 blank lines between paragraphs (i.e. ten blank lines is treated the same as 
 one). DDoc inserts $(DDOC_BLANKLINE) macros on each blank line it encounters, 
 which in the default theme inserts <br><br>, in the Phobos theme inserts <div 
 class="blankline"/>, and for LaTeX inserts /par. Phobos docs also have the 
 common practice of hand-wrapping paragraphs in $(P) macros which inserts
<p></p> 
 tags.
 
 The result of DDoc's handling is that under the default theme if you have,
say, 
 three blank lines between paragraphs then it results in five blank lines being 
 rendered between them.
 
 I'm not yet sure it's feasible, but I'd really like to change DDoc to treat 
 paragraphs the way Markdown does--wrap them in, say, a $(P) macro and collapse 
 multiple blank lines. Any desired extra blank lines could be inserted by hand, 
 via the $(BR) macro or similar. However this could change the rendering of 
 existing documentation which depends on the $(DDOC_BLANKLINE) macro.
 
 What do you think about the proposed changes? Should I try to implement any or 
 all of them?


There is so much written in Ddoc these days that fundamentally changing how it 
works would be very upsetting to a lot of people who have their documents 
broken, for very little benefit.

What I mean by this is there is no existing Markdown text that will be imported 
into Ddoc, so this does not have to be accommodated. Supporting some Markdown 
features is for convenience for Ddoc users. Where Ddoc and Markdown conflict, 
Ddoc takes precedence because there is a lot of legacy Ddoc text and zero
legacy 
Markdown text.




 Appendix A: Planned Markdown support
 ------------------------------------
 
 I plan to support the following Markdown features, as specified by CommonMark
[4]:
 
 - Headings, both the ATX and setext kind


No reason to support both. ATX looks simpler to me, so go with that.


 - Styling text with * (but not with _)


Yes, just stick with one. The * is good.


 - Quoting text blocks with >


Ok.


 - Code blocks fenced with ```, ~~~ and DDoc's existing ---


The existing --- is good, no need to add more options.


 - Backtick-quoted code as it currently works in DDoc


Ok. (This has been a big success in Ddoc.)


 - Nesting lists


    1. cat
    2. dog
    3. box

is fine for ordered lists, using * for unordered ones is plenty good enough.


 - Thematic breaks (<hr>)


Just pick one style for that, I suggest the 3 underscores.


 - Inline links, although I'm not sure if I will support reference links [5]


Ddoc already recognizes URLs and creates links for them


 - Markdown escapes with \


Ok


 - HTML, although that's a discussion item (see above)


I wouldn't change the HTML behavior.


 I don't plan to support the following Markdown features:
 
 - Indented code blocks because existing DDoc comments are often already
indented
 - Using --- for HRs and second-level headings because DDoc uses it as a code 
 block fence
 - Tables
 - Backtick-quoting code across multiple lines
 - Github-specific features like task lists, mentioning people and teams, 
 referencing issues and pull requests, and emoji


Yes, don't waste time on those things.

And thank you for doing this! This will be a nice addition to Ddoc.
Jan 21 2018
next sibling parent reply John Gabriele <jgabriele fastmail.fm> writes:
On Monday, 22 January 2018 at 05:03:41 UTC, Walter Bright wrote:

 On 1/20/2018 10:33 AM, David Gileadi wrote:
 {snip}


 Appendix A: Planned Markdown support
 ------------------------------------
 
 I plan to support the following Markdown features, as 
 specified by CommonMark [4]:
 
 {snip}



 - Nesting lists


    1. cat
    2. dog
    3. box

 is fine for ordered lists, using * for unordered ones is plenty 
 good enough.


For maximum ease of reading, nesting, and for consistency, I 
suggest following the 4-space rule. That is, list item *content* 
(not the markers) always starts at multiples of 4 columns in; 
additional paragraphs of a list item are also indented 4 spaces; 
code blocks indented 4 spaces. For example: 
<https://gist.github.com/uvtc/379bca988270ba8b44a30efabbfa9d6b>.
Jan 22 2018
parent reply David Gileadi <gileadisNOSPM gmail.com> writes:
On 1/22/18 10:37 AM, John Gabriele wrote:

 On Monday, 22 January 2018 at 05:03:41 UTC, Walter Bright wrote:

 On 1/20/2018 10:33 AM, David Gileadi wrote:
 {snip}


 Appendix A: Planned Markdown support
 ------------------------------------

 I plan to support the following Markdown features, as specified by 
 CommonMark [4]:

 {snip}



 - Nesting lists


    1. cat
    2. dog
    3. box

 is fine for ordered lists, using * for unordered ones is plenty good 
 enough.


 
 For maximum ease of reading, nesting, and for consistency, I suggest 
 following the 4-space rule. That is, list item *content* (not the 
 markers) always starts at multiples of 4 columns in; additional 
 paragraphs of a list item are also indented 4 spaces; code blocks 
 indented 4 spaces. For example: 
 <https://gist.github.com/uvtc/379bca988270ba8b44a30efabbfa9d6b>.


Thanks for the suggestion. However, since I'm trying to stick to the 
CommonMark spec I've implemented their rules for determining when 
indented text continues a list item, etc.

I'm currently not supporting indented code blocks, however. I could 
potentially support them within list items, but right now I don't intend to.
Jan 22 2018
next sibling parent reply John Gabriele <jgabriele fastmail.fm> writes:
On Monday, 22 January 2018 at 18:17:37 UTC, David Gileadi wrote:

 On 1/22/18 10:37 AM, John Gabriele wrote:

 On Monday, 22 January 2018 at 05:03:41 UTC, Walter Bright 
 wrote:

 On 1/20/2018 10:33 AM, David Gileadi wrote:
 {snip}


 Appendix A: Planned Markdown support
 ------------------------------------

 I plan to support the following Markdown features, as 
 specified by CommonMark [4]:

 {snip}



 - Nesting lists


    1. cat
    2. dog
    3. box

 is fine for ordered lists, using * for unordered ones is 
 plenty good enough.


 
 For maximum ease of reading, nesting, and for consistency, I 
 suggest following the 4-space rule. That is, list item 
 *content* (not the markers) always starts at multiples of 4 
 columns in; additional paragraphs of a list item are also 
 indented 4 spaces; code blocks indented 4 spaces. For example: 
 <https://gist.github.com/uvtc/379bca988270ba8b44a30efabbfa9d6b>.


 Thanks for the suggestion. However, since I'm trying to stick 
 to the CommonMark spec I've implemented their rules for 
 determining when indented text continues a list item, etc.


AFAICT, CommonMark does indeed follow the 4-space rule, though 
may also provide the extra flexibility of leaving out some 
prefixing spaces. Try copying and pasting my gist example (above) 
into [the CommonMark dingus](http://spec.commonmark.org/dingus/). 
It works as expected.

Note also, for multiple lines in a list item, the continued lines 
line up vertically with the first (same indentation throughout). 
It's really a nifty consistency across all list types that makes 
nesting clear cut.
Jan 22 2018
parent David Gileadi <gileadisNOSPM gmail.com> writes:
On 1/22/18 11:35 AM, John Gabriele wrote:

 On Monday, 22 January 2018 at 18:17:37 UTC, David Gileadi wrote:

 On 1/22/18 10:37 AM, John Gabriele wrote:

 On Monday, 22 January 2018 at 05:03:41 UTC, Walter Bright wrote:

 On 1/20/2018 10:33 AM, David Gileadi wrote:
 {snip}


 Appendix A: Planned Markdown support
 ------------------------------------

 I plan to support the following Markdown features, as specified by 
 CommonMark [4]:

 {snip}



 - Nesting lists


    1. cat
    2. dog
    3. box

 is fine for ordered lists, using * for unordered ones is plenty good 
 enough.


 For maximum ease of reading, nesting, and for consistency, I suggest 
 following the 4-space rule. That is, list item *content* (not the 
 markers) always starts at multiples of 4 columns in; additional 
 paragraphs of a list item are also indented 4 spaces; code blocks 
 indented 4 spaces. For example: 
 <https://gist.github.com/uvtc/379bca988270ba8b44a30efabbfa9d6b>.


 Thanks for the suggestion. However, since I'm trying to stick to the 
 CommonMark spec I've implemented their rules for determining when 
 indented text continues a list item, etc.


 
 AFAICT, CommonMark does indeed follow the 4-space rule, though may also 
 provide the extra flexibility of leaving out some prefixing spaces. Try 
 copying and pasting my gist example (above) into [the CommonMark 
 dingus](http://spec.commonmark.org/dingus/). It works as expected.
 
 Note also, for multiple lines in a list item, the continued lines line 
 up vertically with the first (same indentation throughout). It's really 
 a nifty consistency across all list types that makes nesting clear cut.


It does indeed support 4-space formatting if you write your Markdown 
that way, but its actual rule is a bit different. See example 218 and 
subsequent examples for details, plus the preceding discussion [1].

[1] http://spec.commonmark.org/0.28/#example-218
Jan 22 2018
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/22/2018 10:17 AM, David Gileadi wrote:

 However, since I'm trying to stick to the CommonMark 
 spec I've implemented their rules for determining when indented text continues
a 
 list item, etc.


Good!



 I'm currently not supporting indented code blocks, however. I could
potentially 
 support them within list items, but right now I don't intend to.


Also a good decision.
Jan 22 2018
prev sibling next sibling parent reply David Gileadi <gileadisNOSPM gmail.com> writes:
Thanks for looking this over.

On 1/21/18 10:03 PM, Walter Bright wrote:

 On 1/20/2018 10:33 AM, David Gileadi wrote:

 I've been trying to make my changes non-breaking, in that existing 
 DDoc files should be rendered the same as they currently are. However 
 I've realized that there may need to be some breakage, and I'd like to 
 discuss it.

 First of all, the CSS styles in the default DDoc theme remove all 
 styling from headers, blockquotes, list items, pre and code sections, 
 etc. [2] The effect of this is that Markdown features like headers, 
 quotes, etc. don't display correctly under the default theme. I'd like 
 to modify the default DDoc theme to avoid de-styling these elements.


 
 I don't understand how styling enters into this. Markdown, such as:
 
     * cat
     * dog
     * horse
 
 should be simply rewritten by the highlighter to:
 
     $(UL
     $(LI cat)
     $(LI dog)
     $(LI horse)
     )
 
 and then let the macros do their thing.


Right, and that's exactly what I'm doing. Styling enters into it because 
Ddoc ships with a default theme, and that theme doesn't render things 
like headings properly. With Markdown support Ddoc will now generate 
headings, so the default theme needs to support them too. I'll follow 
Jacob's suggestion for adding styles for headings and other elements.

I'll leave HTML and blank line handling as-is.

[snip]


 

 Appendix A: Planned Markdown support
 ------------------------------------

 I plan to support the following Markdown features, as specified by 
 CommonMark [4]:

 - Headings, both the ATX and setext kind


 
 No reason to support both. ATX looks simpler to me, so go with that.
 

 - Styling text with * (but not with _)


 
 Yes, just stick with one. The * is good.
 

 - Quoting text blocks with >


 
 Ok.
 

 - Code blocks fenced with ```, ~~~ and DDoc's existing ---


 
 The existing --- is good, no need to add more options.
 

 - Backtick-quoted code as it currently works in DDoc


 
 Ok. (This has been a big success in Ddoc.)
 

 - Nesting lists


 
     1. cat
     2. dog
     3. box
 
 is fine for ordered lists, using * for unordered ones is plenty good 
 enough.
 

 - Thematic breaks (<hr>)


 
 Just pick one style for that, I suggest the 3 underscores.
 

 - Inline links, although I'm not sure if I will support reference 
 links [5]


 
 Ddoc already recognizes URLs and creates links for them
 

 - Markdown escapes with \


 
 Ok
 

 - HTML, although that's a discussion item (see above)


 
 I wouldn't change the HTML behavior.
 

 I don't plan to support the following Markdown features:

 - Indented code blocks because existing DDoc comments are often 
 already indented
 - Using --- for HRs and second-level headings because DDoc uses it as 
 a code block fence
 - Tables
 - Backtick-quoting code across multiple lines
 - Github-specific features like task lists, mentioning people and 
 teams, referencing issues and pull requests, and emoji


 
 Yes, don't waste time on those things.
 
 And thank you for doing this! This will be a nice addition to Ddoc.


You're very welcome.

Regarding multiple ways of doing headings, thematic breaks, unordered 
lists, etc. I'd really like to support as much standard Markdown as 
possible. My rationale is:

1. It's easier for people who already know Markdown to use when the list 
of unsupported features is small.
2. There is already harboured-mod which melds Ddoc and Markdown, and 
I've been trying to match its list of differences from vanilla Markdown 
[1] so that it's straightforward to switch between DDoc generators.
3. The additional code for implementing the same feature with different 
triggering characters is small.

Incidentally, to me help stick to the CommonMark spec I'm using its test 
suite to generate a Ddoc page that tests the Markdown support [2]. I'm 
currently at about 40% compliance. When I'm done I plan to make the 
generated page part of DDoc's test suite.


[1] 
https://github.com/dlang-community/harbored-mod#differences-from-vanilla-markdown
[2] https://github.com/dgileadi/ddoc-commonmark-spec
Jan 22 2018
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/22/2018 10:14 AM, David Gileadi wrote:

 Regarding multiple ways of doing headings, thematic breaks, unordered lists, 
 etc. I'd really like to support as much standard Markdown as possible. My 
 rationale is:


Thanks for the reasonable approach, but I disagree and will try to explain.

Markdown supports multiple ways of doing things because (I presume) it's trying 
to accommodate multiple diverse existing markdown schemes. We do not have that 
issue. When there are multiple ways to do X, then inevitably the arguments come 
as to which is the "approved" method. I do not wish to expend any time debating 
potayto-potahto. I've had plenty enough of them in my career :-)

Having multiple equivalent ways to do something increases the size of the 
documentation, the size of the test suite, the potential for bugs, and the 
cognitive load on the user.

Just pick the most reasonable method, and do that.

(For example, using --- or ``` or ~~~ or 4 space indent for code blocks. Gahhh. 
Just use ---, because it works and looks fine, and we already do it.)



 1. It's easier for people who already know Markdown to use when the list of
 unsupported features is small.


Markdown is so trivial I can't believe this is an issue. (I myself use
different 
markdowns on different systems. The differences are simply not a problem.)



 2. There is already harboured-mod which melds Ddoc and Markdown, and I've been
 trying to match its list of differences from vanilla Markdown [1] so that it's
 straightforward to switch between DDoc generators.


What is proposed here shouldn't make life difficult for Habored-mod.



 3. The additional code for implementing the same feature with different
 triggering characters is small.


Features should not be added because of "why not" or "they're easy to 
implement". They need to be justified. Ideally, a language should be a very 
small set of orthogonal building blocks from which anything can be constructed, 
not a panoply of marginally different overlapping features.
Jan 22 2018
next sibling parent John Gabriele <jgabriele fastmail.fm> writes:
On Tuesday, 23 January 2018 at 03:23:37 UTC, Walter Bright wrote:

 On 1/22/2018 10:14 AM, David Gileadi wrote:

 Regarding multiple ways of doing headings, thematic breaks, 
 unordered lists, etc. I'd really like to support as much 
 standard Markdown as possible. My rationale is:


 Thanks for the reasonable approach, but I disagree and will try 
 to explain.

 Markdown supports multiple ways of doing things because (I 
 presume) it's trying to accommodate multiple diverse existing 
 markdown schemes. We do not have that issue. When there are 
 multiple ways to do X, then inevitably the arguments come as to 
 which is the "approved" method. I do not wish to expend any 
 time debating potayto-potahto. I've had plenty enough of them 
 in my career :-)

 Having multiple equivalent ways to do something increases the 
 size of the documentation, the size of the test suite, the 
 potential for bugs, and the cognitive load on the user.

 Just pick the most reasonable method, and do that.

 (For example, using --- or ``` or ~~~ or 4 space indent for 
 code blocks. Gahhh. Just use ---, because it works and looks 
 fine, and we already do it.)


The prime directive of markdown is to be easy to *read* --- not 
necessarily easy to write. That is: to look good/natural, without 
looking hardly at all like markup. For example, you go through 
the extra trouble to indent subsequent lines in list items:

   * Some multi-line list item
     with subsequent lines indented
     to look good.

   * Second item. Lorem ipsum dolor
     sit amet, consectetur adipiscing
     elit,sed do eiusmod tempor.

instead of:

* Some multi-line list item
with subsequent lines not indented.
Doesn't look great.

* Second item. Lorem ipsum dolor
sit amet, consectetur adipiscing
elit,sed do eiusmod tempor.

because the former is much easier on the eyes. Though, both do 
indeed work...

There are two distinct reasons why Markdown supports multiple 
ways of doing some things:

  1. Because markdown also tries to be accomodating. For example, 
it allows some sloppy input and things still work; like allowing 
list items to have *up to* that 4 space indent, rather than 
*requiring* 4. Or allowing two _types_ of *italics*, **and** 
__bold__. Or all the different ways to make an `<hr/>`, or the 
three different list item markers. These accomodations are more 
for the occasional or even accidental markdown user, or for users 
with _olde_ typing habits that die hard. (Admittedly, some of 
them *do* sometimes make special cases look a little prettier.)

  2. As a compromise, to make things more convenient for people 
who use it all the time, but at the expense of it not looking 
*quite* as good. For example, allowing fenced (delimited) code 
blocks instead of only the indented code blocks (fenced are 
easier to copy/paste into). ATX-style headers are faster to type 
(and easier to autogenerate as well). "Lazy-style" list items (as 
the 2nd example above shows (and this current list item too) --- 
where subsequent lines aren't indented) and blockquotes.

It sounds like the kind of multiple-ways-of-doing-things you want 
to avoid is the former (the accomodations), not the latter (the 
conveniences).
Jan 23 2018
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2018-01-23 04:23, Walter Bright wrote:


 (For example, using --- or ``` or ~~~ or 4 space indent for code blocks. 
 Gahhh. Just use ---, because it works and looks fine, and we already do 
 it.)


The fenced code blocks (```) supports specifying a language as well:

```c++
class Foo {}
```

The already existing "---" only supports D. Although, it might not be 
that important to support.

-- 
/Jacob Carlborg
Jan 23 2018
next sibling parent reply David Gileadi <gileadisNOSPM gmail.com> writes:
On 1/23/18 10:30 AM, Jacob Carlborg wrote:

 On 2018-01-23 04:23, Walter Bright wrote:
 

 (For example, using --- or ``` or ~~~ or 4 space indent for code 
 blocks. Gahhh. Just use ---, because it works and looks fine, and we 
 already do it.)


 
 The fenced code blocks (```) supports specifying a language as well:
 
 ```c++
 class Foo {}
 ```
 
 The already existing "---" only supports D. Although, it might not be 
 that important to support.


My current code does support it for ``` and ~~~ fences, but not for --- 
fences because that could change existing behavior. When there's a 
language it outputs an $(OTHER_CODE language, code) macro instead of a 
$(D_CODE code) macro.
Jan 23 2018
parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/23/2018 9:35 AM, David Gileadi wrote:

 but not for --- fences 
 because that could change existing behavior.


Current Ddoc behavior is to not recognize --- followed by other characters as 
part of a fence. A language suffix could be added - I don't see that it is more 
of a breaking change than adding ~~~.

Please, don't support ``` or ~~~.


 When there's a language it outputs an $(OTHER_CODE language, code) macro 


instead of a $(D_CODE code) macro.

Clearly, you'll need to write a specification for what you're doing.
Jan 23 2018
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/23/2018 9:30 AM, Jacob Carlborg wrote:

 The fenced code blocks (```) supports specifying a language as well:


Remember, Ddoc macros are not being removed. For the cases of using some other 
language,

    $(SOME_OTHER_LANGUAGE
        ...
    )

will still work, and will of course work with the inevitable languages that do 
not have syntax highlighting builtin to Ddoc.
Jan 23 2018
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 23 January 2018 at 21:11:26 UTC, Walter Bright wrote:

 Remember, Ddoc macros are not being removed. For the cases of 
 using some other language,

    $(SOME_OTHER_LANGUAGE
        ...
    )

 will still work, and will of course work with the inevitable 
 languages that do not have syntax highlighting builtin to Ddoc.


Have you ever actually tried this? If not, please do. Spoiler 
alert! It won't work right. Behold:

Given this D:

---
/++
         $(MY_LANG
                 if(a<foo> < 10) {
                         printf("<i>not</i>\n");
                 }
         );

         Macros:
                 MY_LANG=<pre>$0</pre>
+/
module a;
---


dmd -D a.d will spit out something like this:

```html

     <pre>               if(<code class="code">a</code><foo> &lt; 
10) {
                         printf("<i>not</i>\n");
                 }
         </pre>

```


That won't render anything remotely close to what it should. The 
printf contents will come out as italic instead of showing the 
original code. It will incorrectly highlight the variable `a` 
because dmd thought it was referring to the module name `a` (this 
is ddoc's biggest misfeature PLEASE REMOVE IT FOR THE LOVE OF ALL 
THAT IS GOOD!!!).

And most damningly: the C++ template argument, <foo>, gets output 
without any character escaping, causing a browser to think it is 
an html tag, breaking the output entirely.


Of course, you could write:

---
         $(MY_LANG
                 if(a$(LT)foo$(RT) $(LT) 10) {
                         printf("$(LT)i$(RT)not$(LT)/i$(RT)\n");
                 }
         );
---

but pleeeease, be realistic, that will not be used.


At least the ``` block, like the --- and existing `code` 
sections, could automatically escape those characters and 
suppress other ddoc processing inside.



As I constantly remind people, `foo` is NOT equivalent to $(D 
foo) and does NOT simply rewrite one into the other. It also 
performs output character replacement (like ---) and THAT is why 
I added that code in the first place. The shorthand syntax of 
`foo` vs $(D foo) was a side-effect; the real goal was this 
escaping. (of course, everyone loves the syntax too...)

The same reasoning would apply to the ``` blocks.
Feb 02 2018
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/2/2018 9:16 PM, Adam D. Ruppe wrote:

 Have you ever actually tried this?


Yes, I use it now and then. You can see it in the CCODE macro used on the dlang 
web site source. There's also CPPCODE, CONSOLE, and GRAMMAR (for the BNF
grammar 
code).
Feb 03 2018
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 3 February 2018 at 10:57:58 UTC, Walter Bright wrote:

 You can see it in the CCODE macro used on the dlang web site 
 source.


Hmm, indeed. I'll let a copy/pasted segment out of 
templates-revisited.dd speak for itself:

$(CCODE
template&lt;class T, class U&gt; class Bar { ... };

template&lt;class T, class U&gt; T foo(T t, U u) { ... }

template&lt;class T, class U&gt; static T abc;
)
Feb 03 2018
parent reply Kagamin <spam here.lot> writes:
Indeed, can't the compiler just replace every <>& with $(LT) 
$(GT) $(AMP) and the template would decide what to translate them 
to?
Feb 03 2018
parent reply Kagamin <spam here.lot> writes:
Looks like ddoc does support escapes:
ESCAPES=
   /&/$(AMP)/
   /</$(LT)/
   />/$(GT)/
Feb 03 2018
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 3 February 2018 at 15:54:03 UTC, Kagamin wrote:

 Looks like ddoc does support escapes:


Right, but it is only used in code blocks, not plain text/macros. 
(I've tried to change that in the past too, but "embedded html" 
is a feature not a bug.)

The --- code --- and the `code` both use ddoc's existing escape 
system. I'm saying ```code``` ought to use it too and THAT is 
reason for it to exist. It is more than just syntax - it does 
something macros cannot do.
Feb 03 2018
prev sibling parent reply David Gileadi <gileadisNOSPM gmail.com> writes:
On 1/22/18 8:23 PM, Walter Bright wrote:

 On 1/22/2018 10:14 AM, David Gileadi wrote:

 Regarding multiple ways of doing headings, thematic breaks, unordered 
 lists, etc. I'd really like to support as much standard Markdown as 
 possible. My rationale is:


 
 Thanks for the reasonable approach, but I disagree and will try to explain.
 
 Markdown supports multiple ways of doing things because (I presume) it's 
 trying to accommodate multiple diverse existing markdown schemes. We do 
 not have that issue. When there are multiple ways to do X, then 
 inevitably the arguments come as to which is the "approved" method. I do 
 not wish to expend any time debating potayto-potahto. I've had plenty 
 enough of them in my career :-)


Markdown doesn't support multiple ways of doing things because of trying 
to accommodate multiple Markdown schemes--from the beginning the John 
Gruber designed the format with multiple ways of doing things [1]. As he 
says in his philosophy section, "Markdown is intended to be as 
easy-to-read and easy-to-write as is feasible" and "the single biggest 
source of inspiration for Markdown’s syntax is the format of plain text 
email." He intended it to support things that people already do, like 
start bullet lists with - or * or +, and have it just work.

We can pick out parts of Markdown to support, but it won't be true to 
the intended vision of Markdown and it will have the effect of annoying 
people who know Markdown because when they try to write it the way 
they're used to it won't do what they expect ("wait, I can only use + 
for lists? I always use dashes! And why aren't my headers working?") [2].

If we go this route then we shouldn't say we support Markdown.


 Having multiple equivalent ways to do something increases the size of 
 the documentation, the size of the test suite, the potential for bugs, 
 and the cognitive load on the user.


These things are all true.

I haven't yet made the corresponding changes to the documentation, but 
my intent is to show a few examples of Markdown like CommonMark's (very 
short) help page does [3], provide a link for more information about 
Markdown, and a (preferably short) list of items DDoc Markdown doesn't 
support, along with the reasons why.

If instead we support just few Markdown features, I believe we should 
avoid saying we support Markdown at all. This would probably result in 
somewhat longer documentation describing each feature.

As for the test suite, as I mentioned I'm generating it from 
CommonMark's test suite. This allows it to be comprehensive and accurate 
without building it from scratch. It also reduces the potential for 
bugs, although I can't argue with the fact that more code == more bugs.

The cognitive load on users depends on what kind of users we're talking 
about. For those who don't know Markdown the load is indeed greater, 
although Markdown's focus on supporting things people already do with 
plain text reduces its impact.

However for people who already know Markdown (and I suspect that's most 
developers who have used things like Github), the fewer standard 
Markdown features we support the greater the cognitive load is. Those 
developers will have to keep in mind both the standard features of 
Markdown and the list of things DDoc's Markdown doesn't support. The 
shorter we make that second list, the smaller the cognitive load.


  > 2. There is already harboured-mod which melds Ddoc and Markdown, and 
 I've been
  > trying to match its list of differences from vanilla Markdown [1] so 
 that it's
  > straightforward to switch between DDoc generators.
 
 What is proposed here shouldn't make life difficult for Habored-mod.


No, but it would make life more difficult for someone who currently 
generates their documents with harbored-mod and would like to switch to 
built-in DDoc instead. However this is probably not a large audience.


  > 3. The additional code for implementing the same feature with different
  > triggering characters is small.
 
 Features should not be added because of "why not" or "they're easy to 
 implement". They need to be justified. Ideally, a language should be a 
 very small set of orthogonal building blocks from which anything can be 
 constructed, not a panoply of marginally different overlapping features.


I agree. I mentioned it because I feel strongly about trying to support 
as much standard Markdown as we can, for the reasons I mentioned 
earlier. One argument against supporting many Markdown features is the 
additional code and its associated burdens, and I was attempting to 
argue in advance that it doesn't add large amounts of additional code.


[1] https://daringfireball.net/projects/markdown/syntax, see the two 
different styles of headers, two styles of links, and different 
triggering characters for list items and horizontal rules.

[2] I say this while fully aware that there are parts of Markdown that 
we won't support. However not supporting a feature because it conflicts 
with existing features is different than not supporting it for 
philosophical reasons.

[3] http://commonmark.org/help/
Jan 24 2018
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
I agree completely with your reasoning that what we are doing should not be 
called "supporting Markdown". The farthest we should go is saying "inspired by 
Markdown." I also agree that we should fully document how this will work in
Ddoc 
rather than referring to the Markdown spec.

Please be careful in copying any text, tests, or examples from the Markdown
spec 
to ensure we fully comply with any licensing terms on their site. It may be
more 
practical to just do our own tests and document how ours works in our own
words. 
Frankly, I'd prefer doing the latter anyway so the D spec is 100% our work.
Jan 24 2018
prev sibling next sibling parent reply rjframe <dlang ryanjframe.com> writes:
On Wed, 24 Jan 2018 16:55:12 -0700, David Gileadi wrote:


 The cognitive load on users depends on what kind of users we're talking
 about. For those who don't know Markdown the load is indeed greater,
 although Markdown's focus on supporting things people already do with
 plain text reduces its impact.


I actually think supporting multiple methods means the cognitive load may 
be reduced for people that don't already know Markdown. When I first read 
some Markdown spec, I quickly realized I could ignore the fact that there 
are three ways to do a bulleted list and just do what I already do - if 
there was only one way to do it, I'd have to remember that one way (which 
would probably not be _my_ way). Reading someone else's Markdown is still 
easy - '+' for bulleted lists is still obviously a list, even though I use 
'*'.


 However for people who already know Markdown (and I suspect that's most
 developers who have used things like Github), the fewer standard
 Markdown features we support the greater the cognitive load is. Those
 developers will have to keep in mind both the standard features of
 Markdown and the list of things DDoc's Markdown doesn't support. The
 shorter we make that second list, the smaller the cognitive load.


Agreed. It would also be easy to go back and add the alternative syntax 
later though, right? Start with one way to do it, and if/when people 
complain, add the alternative methods as needed.


  > 2. There is already harboured-mod which melds Ddoc and Markdown, and
 I've been
  > trying to match its list of differences from vanilla Markdown [1] so
 that it's
  > straightforward to switch between DDoc generators.
 
 What is proposed here shouldn't make life difficult for Habored-mod.


 
 No, but it would make life more difficult for someone who currently
 generates their documents with harbored-mod and would like to switch to
 built-in DDoc instead. However this is probably not a large audience.


 
Another advantage to supporting a variety of features is that support for 
making README.md the generated index.html becomes possible (like Doxygen 
can do). Supporting a limited subset greatly restricts that ability, as 
the README would have to limit itself to the intersection of [GH-]Markdown 
and DDocDown(?).
Jan 25 2018
parent David Gileadi <gileadisNOSPM gmail.com> writes:
On 1/25/18 5:34 AM, rjframe wrote:

 Another advantage to supporting a variety of features is that support for
 making README.md the generated index.html becomes possible (like Doxygen
 can do). Supporting a limited subset greatly restricts that ability, as
 the README would have to limit itself to the intersection of [GH-]Markdown
 and DDocDown(?).


I quite like the idea of making README.md the index page, since people 
are used to it from Github and et al. It's something to keep in mind for 
a future enhancement.
Jan 26 2018
prev sibling parent reply John Gabriele <jgabriele fastmail.fm> writes:
On Wednesday, 24 January 2018 at 23:55:12 UTC, David Gileadi 
wrote:

 We can pick out parts of Markdown to support, but it won't be 
 true to the intended vision of Markdown and it will have the 
 effect of annoying people who know Markdown because when they 
 try to write it the way they're used to it won't do what they 
 expect ("wait, I can only use + for lists? I always use dashes! 
 And why aren't my headers working?") [2].

 If we go this route then we shouldn't say we support Markdown.


What do you think about supporting two modes of operation:

   * one with ddoc markup plus a conservative limited
     subset of CommonMark to enhance ddoc, and

   * one which just assumes doc comments are exclusively
     CommonMark Markdown

?

Not sure if this discussion is strictly limited to documenting D 
and phobos.

New users coming to D and creating modules for code.dlang.org 
will (do?) balk at having to learn yet another custom 
markup/macro format for their doc comments. It's hard enough to 
get people to write any docs at all, let alone ones in a 
language-specific format (see Racket's Scribble, Perl's POD, 
Python's ReST, Ruby's RDoc), *let alone* good docs. Would be 
great if they/we could just use common Markdown.

Another thing to note, since D is often compared to Rust: Rust 
doc comments are in markdown, and that has worked out very well 
for them.
Jan 25 2018
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/25/2018 7:59 AM, John Gabriele wrote:

 What do you think about supporting two modes of operation:


Multiple modes makes things even worse :-(
Jan 25 2018
parent reply Mike Parker <aldacron gmail.com> writes:
On Friday, 26 January 2018 at 07:11:33 UTC, Walter Bright wrote:

 On 1/25/2018 7:59 AM, John Gabriele wrote:

 What do you think about supporting two modes of operation:


 Multiple modes makes things even worse :-(


As a matter of practicality, and from a maintenance cost 
perspective, I agree with everything you've said in this thread. 
 From a "first five minutes" point of view, I have to disagree.

Supporting a half-assed "Markdownish" syntax, where some Markdown 
bits work and others don't, where fenced code blocks require 
"---" and backticks are unsupported, is bound to be a point of 
frustration to someone giving D a legitimate look.

This is one of those ecosystem things like IDE support that maybe 
by itself won't be enough to turn people away, but on a list of 
cons it will carry weight. If we're going to support Markdown in 
Ddoc, then we ought to really support Markdown. Failure to go 
whole hog on this is just going to create one more pain point for 
people motivated to look, but not motivated to leave their 
comfort zone.

And here we've got someone who's not only willing to do it, but 
actually implementing it! We ought to run with it.
Jan 27 2018
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/27/2018 12:08 AM, Mike Parker wrote:

 And here we've got someone who's not only willing to do it, but actually 
 implementing it! We ought to run with it.



I agreed with David to not call it Markdown support, so we do not raise false 
expectations. We simply implement what we need, and document it.
Jan 27 2018
parent reply David Gileadi <gileadisNOSPM gmail.com> writes:
On 1/27/18 1:54 AM, Walter Bright wrote:

 On 1/27/2018 12:08 AM, Mike Parker wrote:

 And here we've got someone who's not only willing to do it, but 
 actually implementing it! We ought to run with it.


 
 
 I agreed with David to not call it Markdown support, so we do not raise 
 false expectations. We simply implement what we need, and document it.


To make myself plain (in case I didn't before) I strongly disagree with 
this approach.

I agree with Mike et al that DDoc should support as much standard 
Markdown as is possible modulo conflicts with DDoc features. I'm willing 
to do the work (and indeed have done most of it already), and as I've 
explained before I believe the downsides are few and well mitigated.
Jan 27 2018
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Saturday, 27 January 2018 at 18:15:04 UTC, David Gileadi wrote:

 On 1/27/18 1:54 AM, Walter Bright wrote:

 On 1/27/2018 12:08 AM, Mike Parker wrote:

 And here we've got someone who's not only willing to do it, 
 but actually implementing it! We ought to run with it.


 
 
 I agreed with David to not call it Markdown support, so we do 
 not raise false expectations. We simply implement what we 
 need, and document it.


 To make myself plain (in case I didn't before) I strongly 
 disagree with this approach.

 I agree with Mike et al that DDoc should support as much 
 standard Markdown as is possible modulo conflicts with DDoc 
 features. I'm willing to do the work (and indeed have done most 
 of it already), and as I've explained before I believe the 
 downsides are few and well mitigated.


+1
Jan 28 2018
parent Marco Leise <Marco.Leise gmx.de> writes:
Am Sun, 28 Jan 2018 10:25:33 +0000
schrieb Paolo Invernizzi <paolo.invernizzi gmail.com>:


 On Saturday, 27 January 2018 at 18:15:04 UTC, David Gileadi wrote:

 On 1/27/18 1:54 AM, Walter Bright wrote:  

 On 1/27/2018 12:08 AM, Mike Parker wrote:  

 And here we've got someone who's not only willing to do it, 
 but actually implementing it! We ought to run with it.  


 
 
 I agreed with David to not call it Markdown support, so we do 
 not raise false expectations. We simply implement what we 
 need, and document it.  


 To make myself plain (in case I didn't before) I strongly 
 disagree with this approach.

 I agree with Mike et al that DDoc should support as much 
 standard Markdown as is possible modulo conflicts with DDoc 
 features. I'm willing to do the work (and indeed have done most 
 of it already), and as I've explained before I believe the 
 downsides are few and well mitigated.  


 
 +1


Mildly +1 from me too. Seems a little better to make it work
as expected most of the time than to reduce the number of ways
to do the same thing.

-- 
Marco
Feb 21 2018
prev sibling parent reply Sebastian Wilzbach <seb wilzba.ch> writes:
On 2018-01-27 09:08, Mike Parker via Dlang-internal wrote:

 On Friday, 26 January 2018 at 07:11:33 UTC, Walter Bright wrote:

 On 1/25/2018 7:59 AM, John Gabriele wrote:

 What do you think about supporting two modes of operation:


 
 Multiple modes makes things even worse :-(


 
 As a matter of practicality, and from a maintenance cost perspective,
 I agree with everything you've said in this thread. From a "first five
 minutes" point of view, I have to disagree.
 
 Supporting a half-assed "Markdownish" syntax, where some Markdown bits
 work and others don't, where fenced code blocks require "---" and
 backticks are unsupported, is bound to be a point of frustration to
 someone giving D a legitimate look.
 
 This is one of those ecosystem things like IDE support that maybe by
 itself won't be enough to turn people away, but on a list of cons it
 will carry weight. If we're going to support Markdown in Ddoc, then we
 ought to really support Markdown. Failure to go whole hog on this is
 just going to create one more pain point for people motivated to look,
 but not motivated to leave their comfort zone.
 
 And here we've got someone who's not only willing to do it, but
 actually implementing it! We ought to run with it.


I can't agree more with your words, Mike.
DDoc is already super-hard for people to get used to as it's very 
un-intuitive and relies on many under-documented features.
My vote is fully on "implement as much Markdown as possible without 
brekage". Everything else will be annoying.

A good example from the past is Vibe'd Markdown parser. It's used e.g. 
on code.dlang.org, but as you can see it doesn't support the full 
MarkDown spec:

- It doesn't support all variations of inline HTML elements

http://code.dlang.org/packages/mir-algorithm

- It doesn't auto-highlight URLs

http://code.dlang.org/packages/buffer

- It doesn't support all variations of tables

http://code.dlang.org/packages/dubbio

- It treats code blocks with backticks differently to indented blocks 
(see "Dub" on "Compile & Run")

http://code.dlang.org/packages/dlang-tour

...

(for a long time it didn't even support advanced features like tables 
which lead to even more broken pages)

The problem is that people are used to all things Markdown supports and 
they just expect it to work as it does on GitHub.
How should they know that Vibe.d's MarkDown parser doesn't support these 
features?
Even if they do know that Vibe.d is used for the registry (and that 
already filters out a majority), they would end up here:

http://vibed.org/api/vibe.textfilter.markdown

Which helps them ... not at all. I don't want DMD's documentation to 
have the same issues.
Shouldn't we learn from the mistakes of the past?

Or in other words: it's great that run.dlang.io now supports -D (e.g. 
https://run.dlang.io/is/5F3rOX),
but a developer shouldn't need to use run.dlang.io to learn the 
documentation system of a language.
It should have a minimal cognitive overhead.
Jan 27 2018
parent aliak <something something.com> writes:
On Saturday, 27 January 2018 at 13:42:39 UTC, Sebastian Wilzbach 
wrote:

 On 2018-01-27 09:08, Mike Parker via Dlang-internal wrote:

 And here we've got someone who's not only willing to do it, but
 actually implementing it! We ought to run with it.




+1


 I can't agree more with your words, Mike.
 DDoc is already super-hard for people to get used to as it's 
 very un-intuitive and relies on many under-documented features.
 My vote is fully on "implement as much Markdown as possible 
 without brekage". Everything else will be annoying.


+1

I do remember getting a bit frustrated because I just wanted to 
document and continue coding, but I couldn't "just". I mean I 
don't mind having to learn yet another doc language, but I'd 
prefer not to, is all I'm saying.

Of course there'll be ddoc specific things (standard sections for 
eg, ditto is a big help), but it really is quite ugly to see all 
those macros when markdown is just so much easier on the eyes 
(and aesthetics do matter a lot),  and on ---/```, my instinct is 
to write ``` for code blocks - because everyone else does, github 
does, bitbucket, slack, and many apps just follow the same, 
because people are used to it. New comers will just know how to 
write docs without having to dig through ddoc docs, it just seems 
a win/win to just support markdown, and call it that.
Feb 01 2018
prev sibling next sibling parent kinke <noone nowhere.com> writes:
On Monday, 22 January 2018 at 05:03:41 UTC, Walter Bright wrote:

 On 1/20/2018 10:33 AM, David Gileadi wrote:

 [...]


 And thank you for doing this! This will be a nice addition to 
 Ddoc.


+1, thanks a lot!
Jan 22 2018
prev sibling parent Sebastian Wilzbach <seb wilzba.ch> writes:
I think there were a few misunderstandings - see below:

On 2018-01-22 06:03, Walter Bright via Dlang-internal wrote:

 - Nesting lists


 
    1. cat
    2. dog
    3. box
 
 is fine for ordered lists, using * for unordered ones is plenty good 
 enough.


I think David was talking about _nested_ lists, e.g.

1. cat
1.1. dog
1.2. box

or

* cat
   - dog
   - box

http://spec.commonmark.org/0.28/#lists

-> Yes, would be awesome. The current way is quite verbose and forces 
you to be consistent with indentation to avoid losing the overview:

$(UL
   $(LI cat)
   $(LI
     $(UL
       $(LI dog)
       $(LI box)
     )
   )
)

If you don't indent consistently (and many people


 - Inline links, although I'm not sure if I will support reference 
 links [5]


 
 Ddoc already recognizes URLs and creates links for them


Inline links are something different, e.g. [foo](https:...)
See also: http://spec.commonmark.org/0.28/#links

The equivalent in today's Ddoc is the ugly $(LINK2 link, text) macro.
-> I would love to have inline links in Ddoc!


 - Code blocks fenced with ```, ~~~ and DDoc's existing ---


 The existing --- is good, no need to add more options.


Imho ``` is almost ubiquitously known, so according to the rule of least 
astonishment [1] three backticks should be supported
(and I doubt it's hard to do so). This will also allow to copy/paste 
better between GitHub, Ddoc and other Markdown documentations/wikis.

[1] https://en.wikipedia.org/wiki/Principle_of_least_astonishment



 I don't plan to support the following Markdown features:


 - Tables


I know that this might be hard to do, but if you have time after your 
initial round this one would be cool to have too.
At Phobos and dlang.org there are many of these nasty tables:

https://github.com/dmd/phobos/blob/617f6b0e592a079de134317bee180f76200f4b6c/std/math.d#L6560
Jan 22 2018
prev sibling parent reply Sebastian Wilzbach <seb wilzba.ch> writes:
David, thanks a lot for championing this project and pushing it forward!
I saw that you already joined Slack and #dmd, so please feel free to ask 
questions there.

There are two advises I can give you that as they will save you a lot of 
frustration and work:


 Open your PR early


Most discussions happen either on GitHub or Slack.
As not everyone is active on Slack, a GitHub PR is usually a good place 
for a in-depth conversation.
Also looking at your work allows us to give you feedback early on and 
avoid unnecessary work / it drifting in the wrong direction.


 Split your changes into minimal chunks


Ideally each point on your TODO list translates to one small PR.
This allows fast review and as a bonus avoids rebasing conflicts.
This point is usually underestimated, but it's vastly easier to pull a 
30 lines change (+tests) than a 1K diff.

On 2018-01-20 19:33, David Gileadi via Dlang-internal wrote:

 (I don't know if this is the right forum for this post, so if not
 please let me know.)
 
 I'm working on adding Markdown support to DDoc. I'm doing it by
 modifying the highlightText method in doc.d [1].
 
 I've been trying to make my changes non-breaking, in that existing
 DDoc files should be rendered the same as they currently are. However
 I've realized that there may need to be some breakage, and I'd like to
 discuss it.
 
 First of all, the CSS styles in the default DDoc theme remove all
 styling from headers, blockquotes, list items, pre and code sections,
 etc. [2] The effect of this is that Markdown features like headers,
 quotes, etc. don't display correctly under the default theme. I'd like
 to modify the default DDoc theme to avoid de-styling these elements.
 
 Secondly, HTML is treated differently under Markdown and DDoc. In DDoc
 anything directly within an HTML tag (i.e. between < and >) is passed
 through unchanged. However if it's not part of the tag it is treated
 as regular DDoc text and subject to regular processing. In Markdown
 all text between a given set of HTML tags is passed through unchanged
 [3]. I'd prefer to switch to the CommonMark handling, allowing whole
 blocks of HTML to be passed through directly.
 
 Thirdly and most controversially is how DDoc handles blank lines
 between paragraphs. Markdown wraps paragraphs in <p></p> tags and
 collapses multiple blank lines between paragraphs (i.e. ten blank
 lines is treated the same as one). DDoc inserts $(DDOC_BLANKLINE)
 macros on each blank line it encounters, which in the default theme
 inserts <br><br>, in the Phobos theme inserts <div
 class="blankline"/>, and for LaTeX inserts /par. Phobos docs also have
 the common practice of hand-wrapping paragraphs in $(P) macros which
 inserts <p></p> tags.
 
 The result of DDoc's handling is that under the default theme if you
 have, say, three blank lines between paragraphs then it results in
 five blank lines being rendered between them.
 
 I'm not yet sure it's feasible, but I'd really like to change DDoc to
 treat paragraphs the way Markdown does--wrap them in, say, a $(P)
 macro and collapse multiple blank lines. Any desired extra blank lines
 could be inserted by hand, via the $(BR) macro or similar. However
 this could change the rendering of existing documentation which
 depends on the $(DDOC_BLANKLINE) macro.
 
 What do you think about the proposed changes? Should I try to
 implement any or all of them?
 
 -Dave
 
 
 Appendix A: Planned Markdown support
 ------------------------------------
 
 I plan to support the following Markdown features, as specified by
 CommonMark [4]:
 
 - Headings, both the ATX and setext kind
 - Styling text with * (but not with _)
 - Quoting text blocks with >
 - Code blocks fenced with ```, ~~~ and DDoc's existing ---
 - Backtick-quoted code as it currently works in DDoc
 - Nesting lists
 - Thematic breaks (<hr>)
 - Inline links, although I'm not sure if I will support reference links 
 [5]
 - Markdown escapes with \
 - HTML, although that's a discussion item (see above)
 
 I don't plan to support the following Markdown features:
 
 - Indented code blocks because existing DDoc comments are often already 
 indented
 - Using --- for HRs and second-level headings because DDoc uses it as
 a code block fence
 - Tables
 - Backtick-quoting code across multiple lines
 - Github-specific features like task lists, mentioning people and
 teams, referencing issues and pull requests, and emoji
 
 
 [1] https://github.com/dgileadi/dmd/tree/ddoc-markdown
 [2] 
 https://github.com/dlang/dmd/blob/master/res/default_ddoc_theme.ddoc#L70
 [3] http://spec.commonmark.org/0.28/#html-blocks
 [4] http://spec.commonmark.org/0.28/
 [5] http://spec.commonmark.org/0.28/#reference-link
Jan 22 2018
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/22/2018 5:50 PM, Sebastian Wilzbach wrote:

 Split your changes into minimal chunks


 
 Ideally each point on your TODO list translates to one small PR.
 This allows fast review and as a bonus avoids rebasing conflicts.
 This point is usually underestimated, but it's vastly easier to pull a 30
lines 
 change (+tests) than a 1K diff.


I totally agree with "chunking" it into multiple independent PRs.
Jan 22 2018
prev sibling parent reply David Gileadi <gileadisNOSPM gmail.com> writes:
On 1/22/18 6:50 PM, Sebastian Wilzbach wrote:

 David, thanks a lot for championing this project and pushing it forward!
 I saw that you already joined Slack and #dmd, so please feel free to ask 
 questions there.
 
 There are two advises I can give you that as they will save you a lot of 
 frustration and work:
 

 Open your PR early


 
 Most discussions happen either on GitHub or Slack.
 As not everyone is active on Slack, a GitHub PR is usually a good place 
 for a in-depth conversation.
 Also looking at your work allows us to give you feedback early on and 
 avoid unnecessary work / it drifting in the wrong direction.


Thanks for the advice. I went ahead and create a PR [1].

I'd been avoiding a PR because I've been rebasing my changes on master 
rather than merging, and I didn't want to cause confusion by rewriting 
history. Should I switch to merging changes from master now that I 
created a PR?


 Split your changes into minimal chunks


 
 Ideally each point on your TODO list translates to one small PR.
 This allows fast review and as a bonus avoids rebasing conflicts.
 This point is usually underestimated, but it's vastly easier to pull a 
 30 lines change (+tests) than a 1K diff.


I haven't managed to do that so far, but it may be possible to do so 
after the fact.


[1] https://github.com/dlang/dmd/pull/7764
Jan 22 2018
next sibling parent David Gileadi <gileadisNOSPM gmail.com> writes:
On 1/22/18 9:56 PM, David Gileadi wrote:

 ...I went ahead and create a PR [1].


This is far from the first typo I've made today. I think I need more sleep.
Jan 22 2018
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2018-01-23 05:56, David Gileadi wrote:


 I'd been avoiding a PR because I've been rebasing my changes on master 
 rather than merging, and I didn't want to cause confusion by rewriting 
 history. Should I switch to merging changes from master now that I 
 created a PR?


No, you should always rebase from master to get the latest changes from 
upstream. Branches for PRs are only temporary. Any changes that are 
performed in the PR after it's opened should be amended on the commit 
that introduced the code change.

-- 
/Jacob Carlborg
Jan 23 2018