www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - code.dlang.org package list

reply void <void main.d> writes:
How do I get a list of all packages (Github URL) available at 
code.dlang.org?

I could download individual pages with wget --recursive 
code.dlang.org but I wonder if there is a better solution.
Sep 08 2018
parent reply Neia Neutuladh <neia ikeran.org> writes:
On Sunday, 9 September 2018 at 00:20:04 UTC, void wrote:
 How do I get a list of all packages (Github URL) available at 
 code.dlang.org?

 I could download individual pages with wget --recursive 
 code.dlang.org but I wonder if there is a better solution.
https://code.dlang.org/packages/index.json https://code.dlang.org/api/packages/[package name]/info
Sep 08 2018
parent reply void <void main.d> writes:
On Sunday, 9 September 2018 at 01:30:14 UTC, Neia Neutuladh wrote:
 On Sunday, 9 September 2018 at 00:20:04 UTC, void wrote:
 How do I get a list of all packages (Github URL) available at 
 code.dlang.org?

 I could download individual pages with wget --recursive 
 code.dlang.org but I wonder if there is a better solution.
https://code.dlang.org/packages/index.json https://code.dlang.org/api/packages/[package name]/info
Thanks. I downloaded 100 packages from code.dlang.org and search for "*doc*" and "*example*" The results: 13 packages with "*doc*" 41 packages with "*example*" 55 packages with neither 8 packages with both
Sep 08 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Saturday, September 8, 2018 8:36:26 PM MDT void via Digitalmars-d wrote:
 On Sunday, 9 September 2018 at 01:30:14 UTC, Neia Neutuladh wrote:
 On Sunday, 9 September 2018 at 00:20:04 UTC, void wrote:
 How do I get a list of all packages (Github URL) available at
 code.dlang.org?

 I could download individual pages with wget --recursive
 code.dlang.org but I wonder if there is a better solution.
https://code.dlang.org/packages/index.json https://code.dlang.org/api/packages/[package name]/info
Thanks. I downloaded 100 packages from code.dlang.org and search for "*doc*" and "*example*" The results: 13 packages with "*doc*" 41 packages with "*example*" 55 packages with neither 8 packages with both
What would you expect that to tell you? ddoc doesn't require any kind of doc folder (though some projects would have one for custom ddoc files that then affect how the documentation looks), and usually, the best way to handle examples is to use ddoc-ed unittest blocks. So, while in some cases, it would be appropriate to have some sort of additional documentation or examples separate from the source code, in general, there isn't much need for it. So, I don't know what information anyone would expect to get from whether or not a project has any folders with doc or example in their name. - Jonathan M Davis
Sep 08 2018
next sibling parent Neia Neutuladh <neia ikeran.org> writes:
On Sunday, 9 September 2018 at 06:32:39 UTC, Jonathan M Davis 
wrote:
 What would you expect that to tell you? ddoc doesn't require 
 any kind of doc folder (though some projects would have one for 
 custom ddoc files that then affect how the documentation 
 looks), and usually, the best way to handle examples is to use 
 ddoc-ed unittest blocks. So, while in some cases, it would be 
 appropriate to have some sort of additional documentation or 
 examples separate from the source code, in general, there isn't 
 much need for it. So, I don't know what information anyone 
 would expect to get from whether or not a project has any 
 folders with doc or example in their name.

 - Jonathan M Davis
It would tell you who thought their projects were complex enough to need separate prose documentation and separate examples, and used the most obvious directory names. Which is potentially interesting. It's a rather narrow result, though.
Sep 09 2018
prev sibling parent reply void <void main.d> writes:
On Sunday, 9 September 2018 at 06:32:39 UTC, Jonathan M Davis 
wrote:
 On Saturday, September 8, 2018 8:36:26 PM MDT void via 
 Digitalmars-d wrote:
 On Sunday, 9 September 2018 at 01:30:14 UTC, Neia Neutuladh 
 wrote:
 On Sunday, 9 September 2018 at 00:20:04 UTC, void wrote:
 [...]
https://code.dlang.org/packages/index.json https://code.dlang.org/api/packages/[package name]/info
Thanks. I downloaded 100 packages from code.dlang.org and search for "*doc*" and "*example*" The results: 13 packages with "*doc*" 41 packages with "*example*" 55 packages with neither 8 packages with both
What would you expect that to tell you? ddoc doesn't require any kind of doc folder (though some projects would have one for custom ddoc files that then affect how the documentation looks), and usually, the best way to handle examples is to use ddoc-ed unittest blocks. So, while in some cases, it would be appropriate to have some sort of additional documentation or examples separate from the source code, in general, there isn't much need for it. So, I don't know what information anyone would expect to get from whether or not a project has any folders with doc or example in their name. - Jonathan M Davis
Script updated now searches for the string "unittest" on package directory (*.d files only). Result: 48 packages with "unittest" That means there is ~= 50% chance an user should read the library source in order to use it. To be fair I should manually check if README.md is decent enough.
Sep 09 2018
next sibling parent Dominikus Dittes Scherkl <dominikus.scherkl continental-corporation.com> writes:
On Monday, 10 September 2018 at 00:56:37 UTC, void wrote:
 On Sunday, 9 September 2018 at 06:32:39 UTC, Jonathan M Davis 
 wrote:
 I downloaded 100 packages from code.dlang.org and search for 
 "*doc*" and "*example*"
Script updated now searches for the string "unittest" on package directory (*.d files only). Result: 48 packages with "unittest"
Ok, this is really a scary result. Is really more than half the packages around not using unittests?!? I can't believe it...
Sep 10 2018
prev sibling parent reply bauss <jj_1337 live.dk> writes:
On Monday, 10 September 2018 at 00:56:37 UTC, void wrote:
 On Sunday, 9 September 2018 at 06:32:39 UTC, Jonathan M Davis 
 wrote:
 On Saturday, September 8, 2018 8:36:26 PM MDT void via 
 Digitalmars-d wrote:
 On Sunday, 9 September 2018 at 01:30:14 UTC, Neia Neutuladh 
 wrote:
 On Sunday, 9 September 2018 at 00:20:04 UTC, void wrote:
 [...]
https://code.dlang.org/packages/index.json https://code.dlang.org/api/packages/[package name]/info
Thanks. I downloaded 100 packages from code.dlang.org and search for "*doc*" and "*example*" The results: 13 packages with "*doc*" 41 packages with "*example*" 55 packages with neither 8 packages with both
What would you expect that to tell you? ddoc doesn't require any kind of doc folder (though some projects would have one for custom ddoc files that then affect how the documentation looks), and usually, the best way to handle examples is to use ddoc-ed unittest blocks. So, while in some cases, it would be appropriate to have some sort of additional documentation or examples separate from the source code, in general, there isn't much need for it. So, I don't know what information anyone would expect to get from whether or not a project has any folders with doc or example in their name. - Jonathan M Davis
Script updated now searches for the string "unittest" on package directory (*.d files only). Result: 48 packages with "unittest" That means there is ~= 50% chance an user should read the library source in order to use it. To be fair I should manually check if README.md is decent enough.
I'm sorry, but it's not even close to accurate, because some libraries has documents on additional websites that has examples and no examples directly in the source code using "standard unittests" - This is true for most big libraries / frameworks. You can't really measure such a thing accurately.
Sep 10 2018
parent Neia Neutuladh <neia ikeran.org> writes:
On Monday, 10 September 2018 at 11:20:55 UTC, bauss wrote:
 I'm sorry, but it's not even close to accurate, because some 
 libraries has documents on additional websites that has 
 examples and no examples directly in the source code using 
 "standard unittests" - This is true for most big libraries / 
 frameworks.
Having checked out recent versions of all dub projects... 1300 dub projects with at least one release. 631 projects have example or test files / folders. 723 have unittests. (Or at least mention unittests somewhere in their source code. A `version (unittest) static assert(0)` would have counted. Or a comment saying that this project really should have at least one unittest. It's the thought that counts, right?) 452 have a file or directory with 'test' in the name. 352 have a file or directory with 'example' in the name. 953 have unittest or tests or examples. That's about 19 out of every 26 packages.
 You can't really measure such a thing accurately.
It's quite possible that every package that lacks an example or test project instead has an external test suite. That's very unlikely, though. It's rather likely that about 25% of dub packages with at least one release have no tests. Probably quite a lot of the untested packages have only one release and no readme.
Sep 10 2018