www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - dub: how to reference a compiled package

reply mahdi <xemailpro yahoo.co.uk> writes:
Hi,

Suppose I have a package `mypack` in `~/mypack`. I run `dub` 
command on this package and have the compiled `mypack` file (OS 
is Linux).

Now I am working on my project. I know how to use the source-code 
of `mypack` package in the project but what if I only have the 
compiled binary? How can I reference and use the modules of the 
compiled `mypack`?

(I looked into the DUB homepage and it's Getting Started page but 
could not find anything).
Feb 25 2016
next sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Thu, 25 Feb 2016 12:15:42 +0000, mahdi wrote:


 Hi,
 
 Suppose I have a package `mypack` in `~/mypack`. I run `dub` command on
 this package and have the compiled `mypack` file (OS is Linux).
 
 Now I am working on my project. I know how to use the source-code of
 `mypack` package in the project but what if I only have the compiled
 binary? How can I reference and use the modules of the compiled
 `mypack`?
 
 (I looked into the DUB homepage and it's Getting Started page but could
 not find anything).


First thing, you need D interface files. See:
https://dlang.org/dmd-linux.html#interface-files

Note that D interface files still require you to include the full body of 
anything in a template, so you might have to modify your code somewhat 
for improved secrecy.

Copy the *.di files and binaries into another dub project and add a 
postBuildCommand to copy that binary to the output location.

I think that will work. dub might complain that it doesn't have any 
source files, in which case you'll have to provide at least one D file 
for it to compile, but that can be empty.
Feb 25 2016
parent reply mahdi <xemailpro yahoo.co.uk> writes:
On Thursday, 25 February 2016 at 16:45:46 UTC, Chris Wright wrote:

 On Thu, 25 Feb 2016 12:15:42 +0000, mahdi wrote:


 Hi,
 
 Suppose I have a package `mypack` in `~/mypack`. I run `dub` 
 command on this package and have the compiled `mypack` file 
 (OS is Linux).
 
 Now I am working on my project. I know how to use the 
 source-code of `mypack` package in the project but what if I 
 only have the compiled binary? How can I reference and use the 
 modules of the compiled `mypack`?
 
 (I looked into the DUB homepage and it's Getting Started page 
 but could
 not find anything).


 First thing, you need D interface files. See: 
 https://dlang.org/dmd-linux.html#interface-files

 Note that D interface files still require you to include the 
 full body of anything in a template, so you might have to 
 modify your code somewhat for improved secrecy.

 Copy the *.di files and binaries into another dub project and 
 add a postBuildCommand to copy that binary to the output 
 location.

 I think that will work. dub might complain that it doesn't have 
 any source files, in which case you'll have to provide at least 
 one D file for it to compile, but that can be empty.


Thanks. Is there a way to use a D library without having access 
to it's source code? I tried `dmd -lib abcd.d` which creates a 
static library. But still I need to specify path to library's 
source files using -I option when compiling the code that uses 
that library.

So if we have just access to the library file, is it possible to 
use it in the code?
Feb 25 2016
parent reply Mike Parker <aldacron gmail.com> writes:
On Thursday, 25 February 2016 at 21:06:59 UTC, mahdi wrote:

 On Thursday, 25 February 2016 at 16:45:46 UTC, Chris Wright



 Thanks. Is there a way to use a D library without having access 
 to it's source code? I tried `dmd -lib abcd.d` which creates a 
 static library. But still I need to specify path to library's 
 source files using -I option when compiling the code that uses 
 that library.

 So if we have just access to the library file, is it possible 
 to use it in the code?


The compiler needs to know what symbols are available from any 
imports you use in your source. .di files exist to allow closed 
source projects to be distributed as binary. They are analagous 
to C or C++ header files. You could create them by hand like so:

// foo.d
struct S { int x, y; }
void addTwo(S s) { s.x += 2; s.y += 2; }

// foo.di
struct S { int x, y; }
void addTwo(S s);

The compiler needs to know about S and its types, and it needs to 
know the signature of addTwo. The .di file allows you to provide 
that while keeping the implementation of addTwo closed. When foo 
is imported in client code, the compiler will find foo.di and use 
that instead of foo.d.

However, the compiler must have the source for templates, as they 
are instantiated when they are used, not when the library is 
compiled. The same is true for any functions you want inlined. In 
the example above, addTwo can only be inlined when foo.d is used, 
since the compiler will not have the implementation with foo.di.
Feb 25 2016
next sibling parent Mike Parker <aldacron gmail.com> writes:
On Friday, 26 February 2016 at 02:49:20 UTC, Mike Parker wrote:


 The compiler needs to know about S and its types, and it needs


S and its *members*
Feb 25 2016
prev sibling parent reply mahdi <xemailpro yahoo.co.uk> writes:
On Friday, 26 February 2016 at 02:49:20 UTC, Mike Parker wrote:

 On Thursday, 25 February 2016 at 21:06:59 UTC, mahdi wrote:

 On Thursday, 25 February 2016 at 16:45:46 UTC, Chris Wright



 Thanks. Is there a way to use a D library without having 
 access to it's source code? I tried `dmd -lib abcd.d` which 
 creates a static library. But still I need to specify path to 
 library's source files using -I option when compiling the code 
 that uses that library.

 So if we have just access to the library file, is it possible 
 to use it in the code?


 The compiler needs to know what symbols are available from any 
 imports you use in your source. .di files exist to allow closed 
 source projects to be distributed as binary. They are analagous 
 to C or C++ header files. You could create them by hand like so:

 // foo.d
 struct S { int x, y; }
 void addTwo(S s) { s.x += 2; s.y += 2; }

 // foo.di
 struct S { int x, y; }
 void addTwo(S s);

 The compiler needs to know about S and its types, and it needs 
 to know the signature of addTwo. The .di file allows you to 
 provide that while keeping the implementation of addTwo closed. 
 When foo is imported in client code, the compiler will find 
 foo.di and use that instead of foo.d.

 However, the compiler must have the source for templates, as 
 they are instantiated when they are used, not when the library 
 is compiled. The same is true for any functions you want 
 inlined. In the example above, addTwo can only be inlined when 
 foo.d is used, since the compiler will not have the 
 implementation with foo.di.


Great! Thanks.

I was looking for a feature like `jar` files in Java or 

are stored together inside a single binary file.

I think same can be implemented for D language and it won't break 
any code because it is not touching the language itself, but the 
compiler.

Anyway, thanks.
Feb 25 2016
next sibling parent Mike Parker <aldacron gmail.com> writes:
On Friday, 26 February 2016 at 03:19:26 UTC, mahdi wrote:


 Great! Thanks.

 I was looking for a feature like `jar` files in Java or 

 are stored together inside a single binary file.

 I think same can be implemented for D language and it won't 
 break any code because it is not touching the language itself, 
 but the compiler.


The magic in Java isn't from Jar files, but from the fact that 
class binaries are distributed as byte code. All of the 
information from the source is still there. The compiler can 
learn everything it needs to know about imported classes simply 
by reading the byte code instead of the source.

In the model that D uses, I don't know enough about object file 
formats to know how much information would still be available if 
they were used at compile time, but I'm fairly certain it 
wouldn't allow the use of templates. There was a project several 
years ago that allowed loading D classes dynamically from object 
files, but I suspect if it were practical to use object files at 
compile time in the same way Java class files are used, someone 
would have implemented such a thing already (in C and C++ as 
well).

That said, it might be an interesting enhancement for the 
compiler to support reading source or interface files from zip 
archives (which is I know is what you're getting at), allowing 
the required imports and the binary to be located in one place so 
that only a single path need be passed on the command line to 
find everything.
Feb 25 2016
prev sibling parent Chris Wright <dhasenan gmail.com> writes:
On Fri, 26 Feb 2016 03:19:26 +0000, mahdi wrote:


 Great! Thanks.
 
 I was looking for a feature like `jar` files in Java or `assemblies` in

 inside a single binary file.



formats. One aspect of that was including enough metadata to reconstruct 
the source code from the binary in a convenient and performant manner.

DMD could store an extra symbol in each file it output including the 
autogenerated .di file for each source file. However, that would require 
the frontend to be able to parse arbitrary library and object formats.

It's much easier to solve this in an external tool like Dub.
Feb 26 2016
prev sibling parent reply BBasile <b2.temp gmx.com> writes:
On Thursday, 25 February 2016 at 12:15:42 UTC, mahdi wrote:

 Hi,

 Suppose I have a package `mypack` in `~/mypack`. I run `dub` 
 command on this package and have the compiled `mypack` file (OS 
 is Linux).

 Now I am working on my project. I know how to use the 
 source-code of `mypack` package in the project but what if I 
 only have the compiled binary? How can I reference and use the 
 modules of the compiled `mypack`?

 (I looked into the DUB homepage and it's Getting Started page 
 but could not find anything).


As you've been told previous you need a D interface file. But 
additionally:

The D interface file must be specified to DUB using

"sourceFiles" : ["folder/interface.di"],

either in a config or in the globals.

The binary, so either a .lib | .a or .obj | .o must be specified 
to DUB using

"DFlags" : ["folder/binary.a"],

Here again also accepted in a config or the globals.

This is because DUB doesn't consider such binary as source file 
but DMD or LDMD2 will accept them as source in the command line.

e.g the DMD equivalent for the two previous example is

DMD "sourceThis.d" "folder/interface.di" "folder/binary.a" 
-ofbin/thesoft
Feb 25 2016
next sibling parent Mike Parker <aldacron gmail.com> writes:
On Friday, 26 February 2016 at 04:03:15 UTC, BBasile wrote:


 The D interface file must be specified to DUB using

 "sourceFiles" : ["folder/interface.di"],

 either in a config or in the globals.

 The binary, so either a .lib | .a or .obj | .o must be 
 specified to DUB using

 "DFlags" : ["folder/binary.a"],

 Here again also accepted in a config or the globals.

 This is because DUB doesn't consider such binary as source file 
 but DMD or LDMD2 will accept them as source in the command line.

 e.g the DMD equivalent for the two previous example is

 DMD "sourceThis.d" "folder/interface.di" "folder/binary.a" 
 -ofbin/thesoft


Why? As long as the interface file is on the import path, it 
shouldn't need to be passed along to DMD. When you import 
foo.bar, the compiler will look for both foo/bar.di and foo/bar.d.
Feb 25 2016
prev sibling parent asdf <a b.c> writes:
On Friday, 26 February 2016 at 04:03:15 UTC, BBasile wrote:

 e.g the DMD equivalent for the two previous example is

 DMD "sourceThis.d" "folder/interface.di" "folder/binary.a" 
 -ofbin/thesoft


You can mix unlinked binaries and text-editor source files on 
commandline? Didn't know that when I tried this. 
http://imgur.com/AIgZGmW I ought to try these interface *.di 
files.

Just tried `ldc2 -H <dlang-file>` but it wouldn't do a C file 
also.
Feb 25 2016