www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Druntime and phobos with dub

reply Rikki Cattermole <alphaglosined gmail.com> writes:
I'm currently working on getting dub package files for Druntime and phobos.
It has been reasonably easy getting code to compile atleast for 2.066 
branch.

Problems:
- Druntime contains assembly + c files requiring external compilers.
To play nicely with dub repos and having them dependencies, they could 
be stored in git like I have it right now.
But the real problem here is that atleast one of these assembly files 
can't even be compiled by pretty much anyone but Walter.
- Phobos has zlib which is in the same situation as druntime for c files.
We seriously should consider rewriting or porting that.
*wink* GSOC.
That would only leave curl but that's manual anyway.
- Documentation building.
This is separated out into dlang.org and druntime/phobos.
   - Dlang.org repo contains the base website and formatting.
   Dub doesn't really understand past sourceLibrary that certain files 
are ddoc macro files and should include them in all packages that depend 
upon it.
   That is okay. The problem is both the formatting but also 
images/css/js are required as part of this.
   - With druntime/phobos they merely need resources from dlang.org. 
Which are detailed above.
   - Dlang.org IS A MESS
   I'm not joking here. Before tackling getting a dub file for e.g. html 
output I moved everything into a new folder.
- std.c.linux and std.c.osx need guard version statements. Other then 
that only issue is std.metastrings doesn't compile so is excluded.

TLDR: Turning druntime/phobos into dub repos are easy. Getting 
documentation to play nicely does not.



For reference here are the files:
Druntime's dub.json:
{
     "name": "druntime",
     "description": "This is DRuntime. It is the low-level runtime 
library backing the D programming language.",
     "homepage": "http://dlang.org",
     "authors": ["Walter Bright", "Andrei Alexandrescu"],
     "license": "Boost",

     "targetType": "none",
     "targetPath": "bin",
     "sourcePaths": [],
     "dependencies": {
         "druntime:rt": "*",
         "druntime:core": "*",
         "druntime:gc": "*",
         "druntime:etc": "*"
     },

     "subPackages": [
         {
             "name": "gcstub",
             "targetType": "staticLibrary",
             "targetPath": "bin",
             "sourcePaths": [],
             "sourceFiles": ["src/gcstub/gc.d"]
         },
         {
             "name": "rt",
             "targetType": "staticLibrary",
             "targetPath": "bin",
             "sourcePaths": ["src/rt"],
             "sourceFiles": ["src/object_.d"],
             "sourceFiles-windows": ["bin/rt_minit.obj"],
             "sourceFiles-posix": ["bin/rt_bss_section.obj"]
         },
         {
             "name": "gc",
             "targetType": "staticLibrary",
             "targetPath": "bin",
             "sourcePaths": ["src/gc"]
         },
         {
             "name": "core",
             "targetType": "staticLibrary",
             "targetPath": "bin",
             "sourcePaths": ["src/core"],
             "sourceFiles-windows": ["bin/stdc_errno_c.obj"],
             "sourceFiles-posix": ["bin/stdc_errno_c.obj", 
"bin/core_threadasm.obj"]
         },
         {
             "name": "etc",
             "targetType": "staticLibrary",
             "targetPath": "bin",
             "sourcePaths": ["src/etc"]
         }
     ]
}

Phobos's dub.json:
{
     "name": "phobos",
     "description": "Phobos is the standard library that comes with the 
D Programming Language Compiler.",
     "homepage": "http://dlang.org",
     "authors": ["Walter Bright", "Andrei Alexandrescu"],
     "license": "Boost",

     "targetPath": "bin",
     "sourcePaths": ["std", "etc"],
     "excludedSourceFiles": ["std/metastrings.d"],
     "sourceFiles-windows-x86": ["bin/zlib.lib"],
     "sourceFiles-windows-x86_64": ["bin/zlib64.lib"],
     "sourceFiles-osx": ["bin/zlib_osx.a"],
     "sourceFiles-linux": ["bin/zlib_linux.a"],
     "dependencies": {
         "druntime": "*"
     },

     "configurations": [
         {
             "name": "staticLibrary",
             "targetType": "staticLibrary"
         },
         {
             "name": "dynamicLibrary",
             "targetType": "dynamicLibrary",
             "dflags": ["-main"]
         }
     ]
}

This is not about debating who should be listed as authors FYI.
Jan 14 2015
parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 15/01/2015 4:35 p.m., Rikki Cattermole wrote:

 I'm currently working on getting dub package files for Druntime and phobos.
 It has been reasonably easy getting code to compile atleast for 2.066
 branch.

 Problems:
 - Druntime contains assembly + c files requiring external compilers.
 To play nicely with dub repos and having them dependencies, they could
 be stored in git like I have it right now.
 But the real problem here is that atleast one of these assembly files
 can't even be compiled by pretty much anyone but Walter.
 - Phobos has zlib which is in the same situation as druntime for c files.
 We seriously should consider rewriting or porting that.
 *wink* GSOC.
 That would only leave curl but that's manual anyway.
 - Documentation building.
 This is separated out into dlang.org and druntime/phobos.
    - Dlang.org repo contains the base website and formatting.
    Dub doesn't really understand past sourceLibrary that certain files
 are ddoc macro files and should include them in all packages that depend
 upon it.
    That is okay. The problem is both the formatting but also
 images/css/js are required as part of this.
    - With druntime/phobos they merely need resources from dlang.org.
 Which are detailed above.
    - Dlang.org IS A MESS
    I'm not joking here. Before tackling getting a dub file for e.g. html
 output I moved everything into a new folder.
 - std.c.linux and std.c.osx need guard version statements. Other then
 that only issue is std.metastrings doesn't compile so is excluded.

 TLDR: Turning druntime/phobos into dub repos are easy. Getting
 documentation to play nicely does not.



 For reference here are the files:
 Druntime's dub.json:
 {
      "name": "druntime",
      "description": "This is DRuntime. It is the low-level runtime
 library backing the D programming language.",
      "homepage": "http://dlang.org",
      "authors": ["Walter Bright", "Andrei Alexandrescu"],
      "license": "Boost",

      "targetType": "none",
      "targetPath": "bin",
      "sourcePaths": [],
      "dependencies": {
          "druntime:rt": "*",
          "druntime:core": "*",
          "druntime:gc": "*",
          "druntime:etc": "*"
      },

      "subPackages": [
          {
              "name": "gcstub",
              "targetType": "staticLibrary",
              "targetPath": "bin",
              "sourcePaths": [],
              "sourceFiles": ["src/gcstub/gc.d"]
          },
          {
              "name": "rt",
              "targetType": "staticLibrary",
              "targetPath": "bin",
              "sourcePaths": ["src/rt"],
              "sourceFiles": ["src/object_.d"],
              "sourceFiles-windows": ["bin/rt_minit.obj"],
              "sourceFiles-posix": ["bin/rt_bss_section.obj"]
          },
          {
              "name": "gc",
              "targetType": "staticLibrary",
              "targetPath": "bin",
              "sourcePaths": ["src/gc"]
          },
          {
              "name": "core",
              "targetType": "staticLibrary",
              "targetPath": "bin",
              "sourcePaths": ["src/core"],
              "sourceFiles-windows": ["bin/stdc_errno_c.obj"],
              "sourceFiles-posix": ["bin/stdc_errno_c.obj",
 "bin/core_threadasm.obj"]
          },
          {
              "name": "etc",
              "targetType": "staticLibrary",
              "targetPath": "bin",
              "sourcePaths": ["src/etc"]
          }
      ]
 }

 Phobos's dub.json:
 {
      "name": "phobos",
      "description": "Phobos is the standard library that comes with the
 D Programming Language Compiler.",
      "homepage": "http://dlang.org",
      "authors": ["Walter Bright", "Andrei Alexandrescu"],
      "license": "Boost",

      "targetPath": "bin",
      "sourcePaths": ["std", "etc"],
      "excludedSourceFiles": ["std/metastrings.d"],
      "sourceFiles-windows-x86": ["bin/zlib.lib"],
      "sourceFiles-windows-x86_64": ["bin/zlib64.lib"],
      "sourceFiles-osx": ["bin/zlib_osx.a"],
      "sourceFiles-linux": ["bin/zlib_linux.a"],
      "dependencies": {
          "druntime": "*"
      },

      "configurations": [
          {
              "name": "staticLibrary",
              "targetType": "staticLibrary"
          },
          {
              "name": "dynamicLibrary",
              "targetType": "dynamicLibrary",
              "dflags": ["-main"]
          }
      ]
 }

 This is not about debating who should be listed as authors FYI.


Personally I believe we should move towards a model where every library 
is represented by JSON (maybe with help of e.g. markdown) that gets e.g. 
parsed by client side technologies such as json.

Benefits:
  - We can have central documentation as part of dub registry for all 
libraries registered
  - Same look and feel for every piece of documentation
  - Less dependencies between druntime/phobos docs and dlang.org
Negatives:
  - Loss of control over end formatting excluding what e.g. markdown support
  - If javascript is disabled, nothing gets displayed
Jan 14 2015
parent reply "NVolcz" <niklas.volcz gmail.com> writes:
On Thursday, 15 January 2015 at 03:42:39 UTC, Rikki Cattermole 
wrote:

 Personally I believe we should move towards a model where every 
 library is represented by JSON (maybe with help of e.g. 
 markdown) that gets e.g. parsed by client side technologies 
 such as json.

 Benefits:
  - We can have central documentation as part of dub registry 
 for all libraries registered
  - Same look and feel for every piece of documentation
  - Less dependencies between druntime/phobos docs and dlang.org
 Negatives:
  - Loss of control over end formatting excluding what e.g. 
 markdown support
  - If javascript is disabled, nothing gets displayed


Nice work!
Central searchable documentation for third party libraries 
registered on code.dlang.org could be a killer :-).
Jan 14 2015
next sibling parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 15/01/2015 7:34 p.m., NVolcz wrote:

 On Thursday, 15 January 2015 at 03:42:39 UTC, Rikki Cattermole wrote:

 Personally I believe we should move towards a model where every
 library is represented by JSON (maybe with help of e.g. markdown) that
 gets e.g. parsed by client side technologies such as json.

 Benefits:
  - We can have central documentation as part of dub registry for all
 libraries registered
  - Same look and feel for every piece of documentation
  - Less dependencies between druntime/phobos docs and dlang.org
 Negatives:
  - Loss of control over end formatting excluding what e.g. markdown
 support
  - If javascript is disabled, nothing gets displayed


 Nice work!
 Central searchable documentation for third party libraries registered on
 code.dlang.org could be a killer :-).


Thanks, and I agree it could very well be.
But it will be quite a lot of work, even if everyone agrees this is the 
right direction to go.

Eventually this could allow for making e.g. std.algorithm as a subpackage.
But atleast we are getting round to sorting out our documentation now.
Jan 14 2015
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Thursday, 15 January 2015 at 06:34:59 UTC, NVolcz wrote:

 Central searchable documentation for third party libraries 
 registered on code.dlang.org could be a killer :-).


My dpldocs.info predates code.dlang.org... but it actually does 
that, in its own kinda quirky way:

http://dpldocs.info/querySelector

Of course, the only packages added right now are phobos, 
druntime, and my misc stuff, but it could take more here: 
http://dpldocs.info/search/add-project

It takes the json made with -X -D over *.d of your project, which 
includes all the ddoc so it can search it.
Jan 15 2015
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-01-15 07:34, NVolcz wrote:


 Nice work!
 Central searchable documentation for third party libraries registered on
 code.dlang.org could be a killer :-).


Yeah, I asked Sönke about that [1].

[1] 
http://forum.rejectedsoftware.com/groups/rejectedsoftware.dub/thread/3528/

-- 
/Jacob Carlborg
Jan 15 2015