www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - docs.json is incomplete

reply Anonymouse <zorael gmail.com> writes:
I'm trying to get docs for my project hosted, and everything 
works, except there are entries missing. Looking at the generated 
docs.json there's simply a lot of omitted stuff.

Example:

http://kameloso.dpldocs.info/kameloso.plugins.sedreplace.html
http://kameloso.dpldocs.info/source/kameloso.plugins.sedreplace.d.html
https://zorael.github.io/kameloso/kameloso/plugins/sedreplace/SedReplacePlugin.html

(TL;DR: the module contains one public class and (private) two 
structs, three free functions and one module-level mixin, but 
only the class is shown. I have more elaborate examples where 
some free functions make it and some don't, showing that 
public/private visibility is not the cause.)

```json
	{
		"kind": "module",
		"file": "source/kameloso/plugins/sedreplace.d",
		"members": [
			{
				"line": 449,
				"kind": "class",
				"char": 7,
				"members": [],
				"name": "SedReplacePlugin",
				"interfaces": [
					"kameloso.plugins.ircplugin.IRCPlugin"
				],
				"comment": "[...]"
			},
			{
				"line": 449,
				"kind": "class",
				"char": 7,
				"members": [],
				"name": "SedReplacePlugin",
				"interfaces": [
					"kameloso.plugins.ircplugin.IRCPlugin"
				],
				"comment": "[...]"
			},
			{
				"line": 449,
				"kind": "class",
				"char": 7,
				"members": [],
				"name": "SedReplacePlugin",
				"interfaces": [
					"kameloso.plugins.ircplugin.IRCPlugin"
				],
				"comment": "[...]"
			},
			{
				"line": 449,
				"kind": "class",
				"char": 7,
				"members": [],
				"name": "SedReplacePlugin",
				"interfaces": [
					"kameloso.plugins.ircplugin.IRCPlugin"
				],
				"comment": "[...]"
			}
		],
		"comment": "[module ddoc ...]",
		"name": "kameloso.plugins.sedreplace"
	},

```

The fact that it is listed four times seems to have to do with 
the use of `version(Something):` at the top of the module, to 
allow for opt-in enabling it based on dub build configuration. 
Commenting them out does not bring the missing stuff back.

Is there a syntax error somewhere? I can't see it. Is the 
docs.json generator at fault?
Apr 10 2020
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 11 April 2020 at 06:49:21 UTC, Anonymouse wrote:

 I'm trying to get docs for my project hosted, and everything 
 works, except there are entries missing. Looking at the 
 generated docs.json there's simply a lot of omitted stuff.


My dpldocs.info site doesn't use docs.json, it goes directly from 
the source with a custom parser. So always a chance I bugged 
things up there.


 https://zorael.github.io/kameloso/kameloso/plugins/sedreplace/SedReplacePlugin.html


this is a wholly different generator though, so my bugs won't 
affect it.


 (TL;DR: the module contains one public class and (private) two 
 structs, three free functions and one module-level mixin, but 
 only the class is shown.


It is all under a `private:` thing for this file.

http://kameloso.dpldocs.info/source/kameloso.plugins.sedreplace.d.html#L34

Then there's a `public:` directly above the class. My doc 
generator skips private members unless you run the generator 
yourself and pass `-p` to the command line. This is impossible on 
the hosted version, but you can `dub run adrdox -- args...` 
yourself to generate the files.

I would assume the json thing does the same basically.



 I have more elaborate examples where some free functions make 
 it and some don't, showing that public/private visibility is 
 not the cause.)


Have an example there? My parser has been known to get off track 
before on things and just give up on the rest of the file. The D 
parser I have is pretty hacky.


 The fact that it is listed four times seems to have to do with 
 the use of `version(Something):` at the top of the module, to 
 allow for opt-in enabling it based on dub build configuration. 
 Commenting them out does not bring the missing stuff back.

 Is there a syntax error somewhere? I can't see it. Is the 
 docs.json generator at fault?


That repetition sounds like a bug in the other thing. Their weird 
handling of `version` is why I actually decided to write my own 
parser!

my version at least shows it kinda sanely:

version(WithSedReplacePlugin)
class SedReplacePlugin : IRCPlugin {}
Apr 11 2020
parent reply Anonymouse <zorael gmail.com> writes:
On Saturday, 11 April 2020 at 13:13:45 UTC, Adam D. Ruppe wrote:

 On Saturday, 11 April 2020 at 06:49:21 UTC, Anonymouse wrote:

 (TL;DR: the module contains one public class and (private) two 
 structs, three free functions and one module-level mixin, but 
 only the class is shown.


 It is all under a `private:` thing for this file.

 http://kameloso.dpldocs.info/source/kameloso.plugins.sedreplace.d.html#L34

 Then there's a `public:` directly above the class. My doc 
 generator skips private members unless you run the generator 
 yourself and pass `-p` to the command line.


This makes sense.


 I have more elaborate examples where some free functions make 
 it and some don't, showing that public/private visibility is 
 not the cause.)


 Have an example there? My parser has been known to get off 
 track before on things and just give up on the rest of the 
 file. The D parser I have is pretty hacky.


Yes, http://kameloso.dpldocs.info/kameloso.plugins.seen.html. 
onBusMessage is private like the other functions, but it shows 
up. (Additionally the public imports are private.)

In http://kameloso.dpldocs.info/kameloso.plugins.admin.html, 
http://kameloso.dpldocs.info/kameloso.plugins.automode.html, 
http://kameloso.dpldocs.info/kameloso.plugins.chatbot.html, 
http://kameloso.dpldocs.info/kameloso.plugins.connect.html, 
http://kameloso.dpldocs.info/kameloso.plugins.notes.html, ...; in 
nearly all modules listed in 
http://kameloso.dpldocs.info/kameloso.plugins.html private 
members are partly included.

The idea is for every plugin module to only expose the public 
IRCPlugin class and hide the rest. Some, correctly then, only 
show the plugin (e.g. 
http://kameloso.dpldocs.info/kameloso.plugins.ctcp.html).


 The fact that it is listed four times seems to have to do with 
 the use of `version(Something):` at the top of the module, to 
 allow for opt-in enabling it based on dub build configuration. 
 Commenting them out does not bring the missing stuff back.

 Is there a syntax error somewhere? I can't see it. Is the 
 docs.json generator at fault?


 That repetition sounds like a bug in the other thing. Their 
 weird handling of `version` is why I actually decided to write 
 my own parser!

 my version at least shows it kinda sanely:

 version(WithSedReplacePlugin)
 class SedReplacePlugin : IRCPlugin {}


Yeah, I like yours more, I just imagined it used docs.json too.
Apr 11 2020
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 11 April 2020 at 14:49:17 UTC, Anonymouse wrote:

 Yes, http://kameloso.dpldocs.info/kameloso.plugins.seen.html. 
 onBusMessage is private like the other functions, but it shows 
 up. (Additionally the public imports are private.)


Oh, I see, my colon attribute code is bad, it doesn't realize it 
descended into a function so still thinks the public applies to 
those imports.

Then it got dropped when the version was attached to the other 
function.

I just spent two hours fixing other bugs in this so I think I'll 
come back to it later tonight or something, but yeah definitely 
bugs on my end there, so I'll take care of it.


 Yeah, I like yours more, I just imagined it used docs.json too.


I sometimes wish I did, keeping up with parser changes can eat 
some time (my last few hours were spent fixing the new contract 
syntax and enum member UDA related bugs), and then you know my 
own crappy bugs. I started with a fork of libdparse which is good 
but not perfect (and omg it builds slowly but meh) and now I've 
been doing it myself for years so lots of divergence.

The json simply didn't have the information I needed though. Even 
dmd frontend parser as a lib (which wasn't a thing when I 
started) doesn't actually do everything I need here, it discards 
ddoc comments on ... IIRC anonymous enum groupings and version 
statements, so even if the version weirdness works it still isn't 
quite right.

DIY gives me a lot more flexibility but also a bunch of bugs 
keeping up with all the syntax changes in the language.
Apr 11 2020
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 11 April 2020 at 15:14:21 UTC, Adam D. Ruppe wrote:

 I'll come back to it later tonight or something, but yeah 
 definitely bugs on my end there, so I'll take care of it.


Just rebuilt your page with the new stuff...

* fixed a bunch of bugs in private: detection. Most your modules 
now only show one class.
* fixed a parser bug with new contract syntax that led to it 
thinking the inside of a function was module-level
* slightly reorganized the display of version statements and 
UDAs. UDAs get their own line (though I can't find any to demo 
this now... prolly all private... but my home test page is 
http://dpldocs.info/experimental-docs/bug.cool.html ) and version 
gets boolean style appearance like you can see here 
http://kameloso.dpldocs.info/kameloso.plugins.webtitles.WebtitlesPlugin.html to
make it a little more compact.


lemme know if you see any more problems or have requests. Your 
code style is pretty different than mine and I spend most my test 
itme on my stuff and Phobos so this revealed a decent number of 
bugs!
Apr 11 2020