www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Can I create static c callable library?

reply John Burton <john.burton jbmail.com> writes:
I need to write a library to statically link into a c program.
Can I write this library in D?
Will I be able to use proper D abilities like gc? Obviously the 
public interface will need to be basic c callable functions...

I 'main' is a c program will this work?
Sep 25 2018
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, September 25, 2018 5:03:11 AM MDT John Burton via Digitalmars-d-
learn wrote:

 I need to write a library to statically link into a c program.
 Can I write this library in D?
 Will I be able to use proper D abilities like gc? Obviously the
 public interface will need to be basic c callable functions...

 I 'main' is a c program will this work?


If you use -betterC, then it's trivial, because your D program is restricted
to extern(C) functions and features which don't require druntime. It can
also be done without -betterC (and thus with druntime), but it gets to be
_way_ more of a pain, because it requires that you manually initialize
druntime - either by forcing whatever is using your "C" library to call a
specific function to initialize druntime before using any of its normal
functions or by having every function in the library check whether druntime
has been initialized yet and initialize it if it hasn't been before it does
whatever it's supposed to do. And of course, if you pass any GC-allocated
memory out of the library, you have to worry about calling all of the
appropriate GC functions so that it knows not to free it and knowing when to
tell the GC that that memory can be freed. It's all very feasible and all
very annoying. In general, it's far, far easier to write D programs that
call into C code than to write C programs that call into D code. That's part
of why some folks are so excited about -betterC. It makes it _way_ easier to
write D libraries that can be called from C or C++ - though because you lose
out on so many D features in the process (like the GC), whether it's even
worth it is highly debatable.

- Jonathan M Davis
Sep 25 2018
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On Tuesday, 25 September 2018 at 12:05:21 UTC, Jonathan M Davis 
wrote:


 If you use -betterC, then it's trivial, because your D program 
 is restricted to extern(C) functions and features which don't 
 require druntime. It can also be done without -betterC (and 
 thus with druntime), but it gets to be _way_ more of a pain, 
 because it requires that you manually initialize druntime - 
 either by forcing whatever is using your "C" library to call a 
 specific function to initialize druntime before using any of 
 its normal functions or by having every function in the library 
 check whether druntime has been initialized yet and initialize 
 it if it hasn't been before it does whatever it's supposed to 
 do.


Shouldn't it be possible to use a C initialization function, i.e. 
pragma(crt_constructor) to initialize druntime? Then it only 
needs to be initialized once and it's not required to check if 
it's initialized all the time.

--
/Jacob Carlborg
Sep 25 2018
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 9/25/18 10:13 AM, Jacob Carlborg wrote:

 On Tuesday, 25 September 2018 at 12:05:21 UTC, Jonathan M Davis wrote:
 

 If you use -betterC, then it's trivial, because your D program is 
 restricted to extern(C) functions and features which don't require 
 druntime. It can also be done without -betterC (and thus with 
 druntime), but it gets to be _way_ more of a pain, because it requires 
 that you manually initialize druntime - either by forcing whatever is 
 using your "C" library to call a specific function to initialize 
 druntime before using any of its normal functions or by having every 
 function in the library check whether druntime has been initialized 
 yet and initialize it if it hasn't been before it does whatever it's 
 supposed to do.


 
 Shouldn't it be possible to use a C initialization function, i.e. 
 pragma(crt_constructor) to initialize druntime? Then it only needs to be 
 initialized once and it's not required to check if it's initialized all 
 the time.


I don't know if that's the right call. pragma(crt_constructor) tasks run 
*before* C main. If you are initializing the runtime, it means you are 
running D static ctors.

Since C initialization functions have no order to them, it's possible 
that some initialization functions in the D runtime are using 
uninitialized pieces of the C runtime. But of course, I'm not sure. I 
just wouldn't trust it if it were me. We don't test initializing the D 
runtime before C main is started (in which case, the C runtime is 
guaranteed to be set up).

Does anyone know if the C runtime is set up before these functions are 
run? Is it guaranteed?

-Steve
Sep 25 2018
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 25 September 2018 at 14:13:50 UTC, Jacob Carlborg 
wrote:

 On Tuesday, 25 September 2018 at 12:05:21 UTC, Jonathan M Davis 
 wrote:


 If you use -betterC, then it's trivial, because your D program 
 is restricted to extern(C) functions and features which don't 
 require druntime. It can also be done without -betterC (and 
 thus with druntime), but it gets to be _way_ more of a pain, 
 because it requires that you manually initialize druntime - 
 either by forcing whatever is using your "C" library to call a 
 specific function to initialize druntime before using any of 
 its normal functions or by having every function in the 
 library check whether druntime has been initialized yet and 
 initialize it if it hasn't been before it does whatever it's 
 supposed to do.


 Shouldn't it be possible to use a C initialization function, 
 i.e. pragma(crt_constructor) to initialize druntime? Then it 
 only needs to be initialized once and it's not required to 
 check if it's initialized all the time.

 --
 /Jacob Carlborg


Even easier, compile this C file and add the resulting object 
file to your (now mostly) D static library:

-----------------------
extern int rt_init(void);
extern int rt_term(void);

__attribute__((__constructor__)) void dinit(void) {
     rt_init();
}
__attribute__((__destructor__)) void dterm(void) {
     rt_term();
}
-----------------------

The C runtime will initialise the D runtime for you.
Sep 27 2018
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Thursday, September 27, 2018 6:16:13 AM MDT Atila Neves via Digitalmars-
d-learn wrote:

 On Tuesday, 25 September 2018 at 14:13:50 UTC, Jacob Carlborg

 wrote:

 On Tuesday, 25 September 2018 at 12:05:21 UTC, Jonathan M Davis

 wrote:

 If you use -betterC, then it's trivial, because your D program
 is restricted to extern(C) functions and features which don't
 require druntime. It can also be done without -betterC (and
 thus with druntime), but it gets to be _way_ more of a pain,
 because it requires that you manually initialize druntime -
 either by forcing whatever is using your "C" library to call a
 specific function to initialize druntime before using any of
 its normal functions or by having every function in the
 library check whether druntime has been initialized yet and
 initialize it if it hasn't been before it does whatever it's
 supposed to do.


 Shouldn't it be possible to use a C initialization function,
 i.e. pragma(crt_constructor) to initialize druntime? Then it
 only needs to be initialized once and it's not required to
 check if it's initialized all the time.

 --
 /Jacob Carlborg


 Even easier, compile this C file and add the resulting object
 file to your (now mostly) D static library:

 -----------------------
 extern int rt_init(void);
 extern int rt_term(void);

 __attribute__((__constructor__)) void dinit(void) {
      rt_init();
 }
 __attribute__((__destructor__)) void dterm(void) {
      rt_term();
 }
 -----------------------

 The C runtime will initialise the D runtime for you.


That's a neat trick.

- Jonathan M Davis
Sep 27 2018
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Sep 27, 2018 at 03:11:26PM -0600, Jonathan M Davis via
Digitalmars-d-learn wrote:

 On Thursday, September 27, 2018 6:16:13 AM MDT Atila Neves via Digitalmars-
 d-learn wrote:


[...]

 Even easier, compile this C file and add the resulting object
 file to your (now mostly) D static library:

 -----------------------
 extern int rt_init(void);
 extern int rt_term(void);

 __attribute__((__constructor__)) void dinit(void) {
      rt_init();
 }
 __attribute__((__destructor__)) void dterm(void) {
      rt_term();
 }
 -----------------------

 The C runtime will initialise the D runtime for you.


 
 That's a neat trick.


[...]

Indeed!

Though I'm not sure what will happen if your C program tries loading two
or more D libraries that use this trick... is rt_init() idempotent? If
not, it could lead to a fun mess on startup... :-D  It also doesn't
address the very thorny issue of how to make multiple D libraries work
nicely with each other's copy of druntime, or how to make them share a
single druntime.


T

-- 
Без труда не выловишь и рыбку из пруда.
Sep 27 2018
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 27 September 2018 at 21:41:31 UTC, H. S. Teoh wrote:

 Though I'm not sure what will happen if your C program tries 
 loading two or more D libraries that use this trick... is 
 rt_init() idempotent?


It just refcounts itself.
Sep 27 2018
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Sep 27, 2018 at 09:48:50PM +0000, Adam D. Ruppe via Digitalmars-d-learn
wrote:

 On Thursday, 27 September 2018 at 21:41:31 UTC, H. S. Teoh wrote:

 Though I'm not sure what will happen if your C program tries loading
 two or more D libraries that use this trick... is rt_init()
 idempotent?


 
 It just refcounts itself.


Does that mean we could potentially make this "trick" the standard
druntime initialization?  Then we could make things work by default
whether you compile a standalone executable or a shared library.

Though I'm not sure what happens if multiple libraries each ship with
their own copy of druntime...


T

-- 
Bomb technician: If I'm running, try to keep up.
Sep 27 2018
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 9/27/18 8:16 AM, Atila Neves wrote:

 On Tuesday, 25 September 2018 at 14:13:50 UTC, Jacob Carlborg wrote:

 On Tuesday, 25 September 2018 at 12:05:21 UTC, Jonathan M Davis wrote:


 If you use -betterC, then it's trivial, because your D program is 
 restricted to extern(C) functions and features which don't require 
 druntime. It can also be done without -betterC (and thus with 
 druntime), but it gets to be _way_ more of a pain, because it 
 requires that you manually initialize druntime - either by forcing 
 whatever is using your "C" library to call a specific function to 
 initialize druntime before using any of its normal functions or by 
 having every function in the library check whether druntime has been 
 initialized yet and initialize it if it hasn't been before it does 
 whatever it's supposed to do.


 Shouldn't it be possible to use a C initialization function, i.e. 
 pragma(crt_constructor) to initialize druntime? Then it only needs to 
 be initialized once and it's not required to check if it's initialized 
 all the time.

 -- 
 /Jacob Carlborg


 
 Even easier, compile this C file and add the resulting object file to 
 your (now mostly) D static library:
 
 -----------------------
 extern int rt_init(void);
 extern int rt_term(void);
 
 __attribute__((__constructor__)) void dinit(void) {
      rt_init();
 }
 __attribute__((__destructor__)) void dterm(void) {
      rt_term();
 }
 -----------------------
 
 The C runtime will initialise the D runtime for you.
 


I will point out that this is EXACTLY what pragma(crt_constructor) does.

And my comments still aren't answered -- I'm not sure whether this works 
correctly or not, as we don't test initializing druntime before C main runs.

-Steve
Sep 27 2018
parent reply Atila Neves <atila.neves gmail.com> writes:
On Thursday, 27 September 2018 at 23:53:50 UTC, Steven 
Schveighoffer wrote:

 On 9/27/18 8:16 AM, Atila Neves wrote:

 On Tuesday, 25 September 2018 at 14:13:50 UTC, Jacob Carlborg 
 wrote:

 On Tuesday, 25 September 2018 at 12:05:21 UTC, Jonathan M 
 Davis wrote:


 If you use -betterC, then it's trivial, because your D 
 program is restricted to extern(C) functions and features 
 which don't require druntime. It can also be done without 
 -betterC (and thus with druntime), but it gets to be _way_ 
 more of a pain, because it requires that you manually 
 initialize druntime - either by forcing whatever is using 
 your "C" library to call a specific function to initialize 
 druntime before using any of its normal functions or by 
 having every function in the library check whether druntime 
 has been initialized yet and initialize it if it hasn't been 
 before it does whatever it's supposed to do.


 Shouldn't it be possible to use a C initialization function, 
 i.e. pragma(crt_constructor) to initialize druntime? Then it 
 only needs to be initialized once and it's not required to 
 check if it's initialized all the time.

 --
 /Jacob Carlborg


 
 Even easier, compile this C file and add the resulting object 
 file to your (now mostly) D static library:
 
 -----------------------
 extern int rt_init(void);
 extern int rt_term(void);
 
 __attribute__((__constructor__)) void dinit(void) {
      rt_init();
 }
 __attribute__((__destructor__)) void dterm(void) {
      rt_term();
 }
 -----------------------
 
 The C runtime will initialise the D runtime for you.
 


 I will point out that this is EXACTLY what 
 pragma(crt_constructor) does.


Really? Huh. You live, you learn. I didn't even know that pragma 
existed - it's not listed here at all:

https://dlang.org/spec/pragma.html


 And my comments still aren't answered -- I'm not sure whether 
 this works correctly or not, as we don't test initializing 
 druntime before C main runs.


It's worked for me in practice.


 Since C initialization functions have no order to them, it's 
 possible that some  initialization functions in the D runtime 
 are using uninitialized pieces of the C runtime


No, that can't happen. The C runtime is initialised no matter 
what you do (unless you write `_start` yourself), _then_ the 
global constructors are run. The code I wrote isn't standard C - 
it's just that gcc/clang/cl are all also C++ compilers so they 
chose to extend the already existing functionality to C.
Sep 28 2018
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Sep 28, 2018 at 02:08:25PM +0000, Atila Neves via Digitalmars-d-learn
wrote:

 On Thursday, 27 September 2018 at 23:53:50 UTC, Steven Schveighoffer wrote:


[...]

 Since C initialization functions have no order to them, it's
 possible that some  initialization functions in the D runtime are
 using uninitialized pieces of the C runtime


 
 No, that can't happen. The C runtime is initialised no matter what you
 do (unless you write `_start` yourself), _then_ the global
 constructors are run. The code I wrote isn't standard C - it's just
 that gcc/clang/cl are all also C++ compilers so they chose to extend
 the already existing functionality to C.


[...]

Potentially some C libraries have not yet been initialized, though.  I
don't know if the druntime init depends on any of them -- it's doubtful,
but if it does, it may cause problems depending on which order library
ctors are called.


T

-- 
In a world without fences, who needs Windows and Gates? -- Christian Surchi
Sep 28 2018
prev sibling parent reply John Burton <john.burton jbmail.com> writes:
On Tuesday, 25 September 2018 at 12:05:21 UTC, Jonathan M Davis 
wrote:

 [...]


Thanks everyone.

Is there any documentation anywhere that deals with calling D 
from C? I could find plenty the other way round. I think I'll 
give up on the idea though, and rewrite the whole thing in D :)
Sep 26 2018
parent Mike Parker <aldacron gmail.com> writes:
On Wednesday, 26 September 2018 at 09:54:22 UTC, John Burton 
wrote:

 Is there any documentation anywhere that deals with calling D 
 from C? I could find plenty the other way round. I think I'll 
 give up on the idea though, and rewrite the whole thing in D :)


Rewriting it in D is a great idea ;) But for the record, much of 
what you've read about calling C from D applies the other way, 
too. You just need to write your D functions as extern(C), 
following the same approach to function signatures and types as 
you've read about, then declare the crossover functions and types 
in C.

One gotcha to look out for is when you call a D function that 
allocates GC memory. In that case, you need to keep a reference 
to it alive on the D side. If it's only being allocated for use 
in C, you can call GC.addRoot [1] when you allocate the memory on 
the D side and GC.removeRoot [2] when you no longer need it.

Also keep in mind that D variables thread-local, and if you need 
to access any of them in C from multiple threads they would 
better be declared as __gshared in D.

[1] https://dlang.org/phobos/core_memory.html#.GC.addRoot
[2] https://dlang.org/phobos/core_memory.html#.GC.removeRoot
Sep 26 2018
prev sibling parent 9il <ilyayaroshenko gmail.com> writes:
On Tuesday, 25 September 2018 at 11:03:11 UTC, John Burton wrote:

 I need to write a library to statically link into a c program.
 Can I write this library in D?
 Will I be able to use proper D abilities like gc? Obviously the 
 public interface will need to be basic c callable functions...

 I 'main' is a c program will this work?


Yes, for example https://github.com/libmir/mir-optim
It has *.cpp example.
Sep 25 2018