digitalmars.D.learn - ImportC in a Dub project

Carsten Schlote (37/37) Oct 28 2022 Hi,

Imperatorn (2/6) Oct 28 2022 Are you using DMD?

Carsten Schlote (10/19) Oct 28 2022 I'm using

Steven Schveighoffer (5/56) Oct 28 2022 Are you passing the c file to the compiler? Also, you must be using dmd

Carsten Schlote (61/64) Oct 28 2022 ```

Imperatorn (3/21) Oct 28 2022 Like schveiguy said, what's your build line? dmd should pick up

Carsten Schlote (8/10) Oct 28 2022 What do you mean with 'buildline'? The project is build with dub.

Imperatorn (3/13) Oct 28 2022 Have you read this?

Steven Schveighoffer (7/73) Oct 28 2022 By default dub does not build C files (as evidenced by your command

Carsten Schlote (7/12) Oct 30 2022 Ok, this works. So Dub is not picking any C files by default.
Carsten Schlote (68/68) Oct 30 2022 It turned out, that the required changes to add support for C

Imperatorn (4/10) Oct 30 2022 Agreed, but don't forget about the .i extension.

Carsten Schlote (10/22) Oct 30 2022 I will merge the two PRs into a new patchset, and also add

Imperatorn (3/16) Oct 30 2022 Yeah maybe something like that. Because I agree we need to update

mw (18/28) Jun 21 ...

Carsten Schlote <carsten.schlote gmx.net> writes:

Hi,

I created a Dub project containing two files: app.d and 
zstd_binding.c

```
$ cat source/zstd_binding.c

#include <zstd.h>
#include <zstd_errors.h>

#include <stdio.h>

void relatedCode(void)
{
	printf("Hallo! This is some output from C code!\n");
}
```
and
```
$ cat source/app.d
import std.conv;
import std.stdio;

import zstd_binding;

void main()
{
	auto versionNr = ZSTD_versionNumber();
	auto versionStr = ZSTD_versionString();
	writefln("Version Info: Numeric %d String %s", versionNr, 
versionStr.to!string);

	/** more code stripped */

	relatedCode(); // The linker can't find it....
}

```

Accessing ZSTD works perfect. For linking you must give "-lzstd". 
So far, so good. The problem started, when I try to access the 
```relatedCode()``` C function. There is simply no code generated 
for this C function. I need to compile the C separately into an 
object file and pass it as extra linker argument.

I expected that ImportC also allows to import C functions and 
creates the required code on the fly.

Is this a bug? What's wrong?

Oct 28 2022

Imperatorn <johan_forsberg_86 hotmail.com> writes:

On Friday, 28 October 2022 at 17:45:59 UTC, Carsten Schlote wrote:
 Hi,

 I created a Dub project containing two files: app.d and 
 zstd_binding.c

 [...]

Are you using DMD?

Oct 28 2022

Carsten Schlote <carsten.schlote gmx.net> writes:

On Friday, 28 October 2022 at 17:56:57 UTC, Imperatorn wrote:
 On Friday, 28 October 2022 at 17:45:59 UTC, Carsten Schlote 
 wrote:
 Hi,

 I created a Dub project containing two files: app.d and 
 zstd_binding.c

 [...]

 Are you using DMD?

I'm using
```
$ dub --version
DUB version 1.29.2, built on Sep 29 2022
$ dmd-beta --version
DMD64 D Compiler v2.101.0-beta.1-36-ga2865d74fb
Copyright (C) 1999-2022 by The D Language Foundation, All Rights 
Reserved written by Walter Bright
```

Oct 28 2022

Steven Schveighoffer <schveiguy gmail.com> writes:

On 10/28/22 1:45 PM, Carsten Schlote wrote:
 Hi,
 
 I created a Dub project containing two files: app.d and zstd_binding.c
 
 ```
 $ cat source/zstd_binding.c
 
 #include <zstd.h>
 #include <zstd_errors.h>
 
 #include <stdio.h>
 
 void relatedCode(void)
 {
      printf("Hallo! This is some output from C code!\n");
 }
 ```
 and
 ```
 $ cat source/app.d
 import std.conv;
 import std.stdio;
 
 import zstd_binding;
 
 void main()
 {
      auto versionNr = ZSTD_versionNumber();
      auto versionStr = ZSTD_versionString();
      writefln("Version Info: Numeric %d String %s", versionNr, 
 versionStr.to!string);
 
      /** more code stripped */
 
      relatedCode(); // The linker can't find it....
 }
 
 ```
 
 Accessing ZSTD works perfect. For linking you must give "-lzstd". So 
 far, so good. The problem started, when I try to access the 
 ```relatedCode()``` C function. There is simply no code generated for 
 this C function. I need to compile the C separately into an object file 
 and pass it as extra linker argument.
 
 I expected that ImportC also allows to import C functions and creates 
 the required code on the fly.
 
 Is this a bug? What's wrong?
 
 

Are you passing the c file to the compiler? Also, you must be using dmd 
for ImportC currently.

What is your build line?

-Steve

Oct 28 2022

Carsten Schlote <carsten.schlote gmx.net> writes:

On Friday, 28 October 2022 at 18:31:25 UTC, Steven Schveighoffer 
wrote:

 Are you passing the c file to the compiler? Also, you must be 
 using dmd for ImportC currently.

 What is your build line?

```
$ cat dub.json
{
	"authors": [
		"Carsten Schlote"
	],
	"copyright": "Copyright © 2022, Carsten Schlote",
	"description": "A minimal D application.",
	"license": "proprietary",
	"name": "importc-app",
	"lflags": [ "-lzstd", "zstd_binding.o" ],
	"preBuildCommands": [ "gcc -g -O0 -c -o zstd_binding.o 
source/zstd_binding.c"]
```

```
$ dub build --compiler=dmd-beta -v
Using dub registry url 'https://code.dlang.org/'
Refreshing local packages (refresh existing: true)...
Looking for local package map at 
/var/lib/dub/packages/local-packages.json
Looking for local package map at 
/home/cschlote/.dub/packages/local-packages.json
Looking for local package map at 
/home2/cschlote/build/dlang/importc-tests/dubbed/.dub/packages/local-packages.json
Note: Failed to determine version of package importc-app at .. 
Assuming ~master.
Refreshing local packages (refresh existing: false)...
Looking for local package map at 
/var/lib/dub/packages/local-packages.json
Looking for local package map at 
/home/cschlote/.dub/packages/local-packages.json
Looking for local package map at 
/home2/cschlote/build/dlang/importc-tests/dubbed/.dub/packages/local-packages.json
Refreshing local packages (refresh existing: false)...
Looking for local package map at 
/var/lib/dub/packages/local-packages.json
Looking for local package map at 
/home/cschlote/.dub/packages/local-packages.json
Looking for local package map at 
/home2/cschlote/build/dlang/importc-tests/dubbed/.dub/packages/local-packages.json
Generating using build
Configuring dependent importc-app, deps:
Performing "debug" build using dmd-beta for x86_64.
Target 
'/home2/cschlote/build/dlang/importc-tests/dubbed/.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C68
C85F85/importc-app' doesn't exist, need rebuild.
importc-app ~master: building configuration "application"...
Running pre-build commands...
Running gcc -g -O0 -c -o zstd_binding.o source/zstd_binding.c
dmd-beta -c 
-of.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C68B
85F85/importc-app.o -debug -g -w -version=Have_importc_app -Isource/
source/app.d -vcolumns
Linking...
dmd-beta 
-of.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C6
BC85F85/importc-app .dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C68B
85F85/importc-app.o -L--no-as-needed -L-lzstd -Lzstd_binding.o -g
Copying target from 
/home2/cschlote/build/dlang/importc-tests/dubbed/.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C6
BC85F85/importc-app to /home2/cschlote/build/dlang/importc-tests/dubbed
```

Without the ```preBuildCommands``` and without passing the 
resulting object file to the linker, the ```relatedCode()``` 
doesn't exist. And linking is failing.

Oct 28 2022

Imperatorn <johan_forsberg_86 hotmail.com> writes:

On Friday, 28 October 2022 at 18:43:21 UTC, Carsten Schlote wrote:
 On Friday, 28 October 2022 at 18:31:25 UTC, Steven 
 Schveighoffer wrote:

 [...]

 ```
 $ cat dub.json
 {
 	"authors": [
 		"Carsten Schlote"
 	],
 	"copyright": "Copyright © 2022, Carsten Schlote",
 	"description": "A minimal D application.",
 	"license": "proprietary",
 	"name": "importc-app",
 	"lflags": [ "-lzstd", "zstd_binding.o" ],
 	"preBuildCommands": [ "gcc -g -O0 -c -o zstd_binding.o 
 source/zstd_binding.c"]
 ```

 [...]

Like schveiguy said, what's your build line? dmd should pick up 
the file if the import has the same name as the file.

Oct 28 2022

Carsten Schlote <carsten.schlote gmx.net> writes:

On Friday, 28 October 2022 at 18:56:03 UTC, Imperatorn wrote:
 Like schveiguy said, what's your build line? dmd should pick up 
 the file if the import has the same name as the file.

What do you mean with 'buildline'? The project is build with dub. 
See previous posts with outputs from commandline.

DMD does find and import the C file. But it doesn't create code 
for ```relatedCode()``` in the C file. The only way to fix this, 
is to compile it with a C compiler and use the object as 
additional linker output.

IMHO this is a bug.

Oct 28 2022

Imperatorn <johan_forsberg_86 hotmail.com> writes:

On Friday, 28 October 2022 at 19:04:21 UTC, Carsten Schlote wrote:
 On Friday, 28 October 2022 at 18:56:03 UTC, Imperatorn wrote:
 Like schveiguy said, what's your build line? dmd should pick 
 up the file if the import has the same name as the file.

 What do you mean with 'buildline'? The project is build with 
 dub. See previous posts with outputs from commandline.

 DMD does find and import the C file. But it doesn't create code 
 for ```relatedCode()``` in the C file. The only way to fix 
 this, is to compile it with a C compiler and use the object as 
 additional linker output.

 IMHO this is a bug.

Have you read this?

https://dlang.org/spec/importc.html

Oct 28 2022

Steven Schveighoffer <schveiguy gmail.com> writes:

On 10/28/22 2:43 PM, Carsten Schlote wrote:
 On Friday, 28 October 2022 at 18:31:25 UTC, Steven Schveighoffer wrote:
 
 Are you passing the c file to the compiler? Also, you must be using 
 dmd for ImportC currently.

 What is your build line?

 
 ```
 $ cat dub.json
 {
      "authors": [
          "Carsten Schlote"
      ],
      "copyright": "Copyright © 2022, Carsten Schlote",
      "description": "A minimal D application.",
      "license": "proprietary",
      "name": "importc-app",
      "lflags": [ "-lzstd", "zstd_binding.o" ],
      "preBuildCommands": [ "gcc -g -O0 -c -o zstd_binding.o 
 source/zstd_binding.c"]
 ```
 
 ```
 $ dub build --compiler=dmd-beta -v
 Using dub registry url 'https://code.dlang.org/'
 Refreshing local packages (refresh existing: true)...
 Looking for local package map at /var/lib/dub/packages/local-packages.json
 Looking for local package map at 
 /home/cschlote/.dub/packages/local-packages.json
 Looking for local package map at 
 /home2/cschlote/build/dlang/importc-tests/dubbed/.dub/packages/local-packages.json
 Note: Failed to determine version of package importc-app at .. Assuming 
 ~master.
 Refreshing local packages (refresh existing: false)...
 Looking for local package map at /var/lib/dub/packages/local-packages.json
 Looking for local package map at 
 /home/cschlote/.dub/packages/local-packages.json
 Looking for local package map at 
 /home2/cschlote/build/dlang/importc-tests/dubbed/.dub/packages/local-packages.json
 Refreshing local packages (refresh existing: false)...
 Looking for local package map at /var/lib/dub/packages/local-packages.json
 Looking for local package map at 
 /home/cschlote/.dub/packages/local-packages.json
 Looking for local package map at 
 /home2/cschlote/build/dlang/importc-tests/dubbed/.dub/packages/local-packages.json
 Generating using build
 Configuring dependent importc-app, deps:
 Performing "debug" build using dmd-beta for x86_64.
 Target 
 '/home2/cschlote/build/dlang/importc-tests/dubbed/.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C68
C85F85/importc-app' doesn't exist, need rebuild.
 importc-app ~master: building configuration "application"...
 Running pre-build commands...
 Running gcc -g -O0 -c -o zstd_binding.o source/zstd_binding.c
 dmd-beta -c 
 -of.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C68B
85F85/importc-app.o -debug -g -w -version=Have_importc_app -Isource/
source/app.d -vcolumns
 Linking...
 dmd-beta 
 -of.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C6
BC85F85/importc-app .dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C68B
85F85/importc-app.o -L--no-as-needed -L-lzstd -Lzstd_binding.o -g
 Copying target from 
 /home2/cschlote/build/dlang/importc-tests/dubbed/.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C6
BC85F85/importc-app to /home2/cschlote/build/dlang/importc-tests/dubbed
 ```
 
 Without the ```preBuildCommands``` and without passing the resulting 
 object file to the linker, the ```relatedCode()``` doesn't exist. And 
 linking is failing.
 

By default dub does not build C files (as evidenced by your command 
line). It may not even let you I don't know, but try:

```json
   "sourceFiles" : ["source/zstdc_binding.c"]
```

-Steve

Oct 28 2022

Carsten Schlote <carsten.schlote gmx.net> writes:

On Friday, 28 October 2022 at 19:08:47 UTC, Steven Schveighoffer 
wrote:
 By default dub does not build C files (as evidenced by your 
 command line). It may not even let you I don't know, but try:

 ```json
   "sourceFiles" : ["source/zstdc_binding.c"]
 ```

Ok, this works. So Dub is not picking any C files by default. 
Thanks for the hint.

I will spent some time to prepare a fix for Dub, so that C files 
(only '.c' files, no headers) in detected/configured source 
directories are picked up as well.

Oct 30 2022

Carsten Schlote <carsten.schlote gmx.net> writes:

It turned out, that the required changes to add support for C
files in Dub are really small. So I added a PR
(https://github.com/dlang/dub/pull/2521). There is also some
other PR (https://github.com/dlang/dub/pull/2270) releated to C
file support in Dub.

With my PR (2521) applied I can now compile my ImportC test
projects
(https://gitlab.vahanus.net:dlang/examples/importc-tests.git)
without any additional hacks. Just a current copy of DUB with the
patch and a current beta build of DMD is needed.

```
$ ../../dub/bin/dub build -v --compiler=dmd-beta -f
Using dub registry url 'https://code.dlang.org/'
Note: Failed to determine version of package importc-app at ..
Assuming ~master.
Generating using build
Configuring dependent importc-app, deps:
Starting Performing "debug" build using dmd-beta for x86_64.
Building importc-app ~master: building configuration
[application]
dmd-beta -c
-of.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-38-gfabd06214e-B8D88D35889281A0F16BEA64448915871A8820982AFB76E0D5F2AD9849
F80E5/importc-app.o -debug -g -w -version=Have_importc_app -Isource/
source/app.d source/some_d_file.d source/zstd_binding.c -vcolumns
Linking importc-app
dmd-beta
-of.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-38-gfabd06214e-B8D88D35889281A0F16BEA64448915871A8820982AFB76E0D5F2AD98
9AF80E5/importc-app .dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-38-gfabd06214e-B8D88D35889281A0F16BEA64448915871A8820982AFB76E0D5F2AD9849
F80E5/importc-app.o -L--no-as-needed -L-lzstd -g
Copying target from
/home2/cschlote/build/dlang/importc-tests/dubbed/.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-38-gfabd06214e-B8D88D35889281A0F16BEA64448915871A8820982AFB76E0D5F2AD98
9AF80E5/importc-app to /home2/cschlote/build/dlang/importc-tests/dubbed

```

However, the maintainers of Dub seem to have different ideas
resulting into much more work to be done. Their ideas are surely
good and reasonable, but it sounds like a lot of changes needed
to be applied before we see support for direct and easy C file
support in Dub.

In the meantime (weeks, months, years?) you are forced to either
pass an explicite list of C sourcefiles in dub.json (see previous
forum posts), or use some ```preBuild/Generate``` commands to
compile your C source file(s) to object file(s) and add these
object files to other explicite list as linker input. Both ways
are ugly and cumbersome.

What we need in dub instead is that C files are just picked up
like any other D file as a valid source for the compiler. And
without any explicite lists of source files in ```dub.json```.

The C header files are intentionally left out here, because it is
bad style to have real functions in a C header. Yes, I know,
people are doing such things. It is one of the bad habbits of
C/C++. If you want to include a C header, just wrap it into a C
file, which is then picked up by Dub and passed to the D
compiler. See the mentioned example project for wrapping the ZSTD
C header. This is easy and straight forward.

I'm pretty sure that any additional code to support the simple
usecases for C header files will also be easy to implement.

So if 80% or more of the general usecases with C source/header
files in Dub can be implemented with simple and small changes
like with the PR mentioned above, we should do it now.

A better and more general implementation covering 100%+ of all
thinkable usecases and edge conditions can be done later after
having a working solution *now*, instead of discussing a *proper*
solutions for several months.

IMHO **straight-forward** and **easy** support for using C files
in Dub projects is a must for the up-coming next DMD release with
its much improved ImportC feature.

It will also help a lot when we want to promote D as a suitable
language alternative for C/C++ programmers. As long as D needs
hand-crafted binding files or other hacky workarounds instead of
just ```ìmport your.c.file;``` and a C source wrapper it will be
hard to convince them.

So I strongly recommend to add easy to use support for C files in
Dub projects **ASAP**.

Oct 30 2022

Imperatorn <johan_forsberg_86 hotmail.com> writes:

On Sunday, 30 October 2022 at 12:46:10 UTC, Carsten Schlote wrote:
 It turned out, that the required changes to add support for C 
 files in Dub are really small. So I added a PR 
 (https://github.com/dlang/dub/pull/2521). There is also some 
 other PR (https://github.com/dlang/dub/pull/2270) releated to C 
 file support in Dub.

 [...]

Agreed, but don't forget about the .i extension.

Imo we could add a switch which enables something like your PR 
(include .c and .i) until we know what the default should be.

Oct 30 2022

Carsten Schlote <carsten.schlote gmx.net> writes:

On Sunday, 30 October 2022 at 13:38:23 UTC, Imperatorn wrote:
 On Sunday, 30 October 2022 at 12:46:10 UTC, Carsten Schlote 
 wrote:
 It turned out, that the required changes to add support for C 
 files in Dub are really small. So I added a PR 
 (https://github.com/dlang/dub/pull/2521). There is also some 
 other PR (https://github.com/dlang/dub/pull/2270) releated to 
 C file support in Dub.

 [...]

 Agreed, but don't forget about the .i extension.

I will merge the two PRs into a new patchset, and also add 
appropriate code
to search für {c|i|h} files at the right places. The patchset 
should be as minimal as possible, so that the planned better 
solution is not blocked/messed up.

 Imo we could add a switch which enables something like your PR 
 (include .c and .i) until we know what the default should be.

Maybe a switch/flag should be added to the dub.json/sdl files to 
turn on the new behaviour? This allows to enable the new search 
logic for C files without breaking something old. It would make 
it more clear, that also a new 2.101+ D compiler is needed.

Oct 30 2022

Imperatorn <johan_forsberg_86 hotmail.com> writes:

On Sunday, 30 October 2022 at 14:42:55 UTC, Carsten Schlote wrote:
 On Sunday, 30 October 2022 at 13:38:23 UTC, Imperatorn wrote:
 [...]

 I will merge the two PRs into a new patchset, and also add 
 appropriate code
 to search für {c|i|h} files at the right places. The patchset 
 should be as minimal as possible, so that the planned better 
 solution is not blocked/messed up.

 [...]

 Maybe a switch/flag should be added to the dub.json/sdl files 
 to turn on the new behaviour? This allows to enable the new 
 search logic for C files without breaking something old. It 
 would make it more clear, that also a new 2.101+ D compiler is 
 needed.

Yeah maybe something like that. Because I agree we need to update 
it since we now have ImportC.

Oct 30 2022

mw <mingwu gmail.com> writes:

On Friday, 28 October 2022 at 19:08:47 UTC, Steven Schveighoffer 
wrote:
 On 10/28/22 2:43 PM, Carsten Schlote wrote:
 On Friday, 28 October 2022 at 18:31:25 UTC, Steven 
 Schveighoffer wrote:
 
 Are you passing the c file to the compiler? Also, you must be



...
 By default dub does not build C files (as evidenced by your 
 command line). It may not even let you I don't know, but try:

 ```json
   "sourceFiles" : ["source/zstdc_binding.c"]
 ```

What if the dub.json target is a staticLibrary?

Right now I have this error:

```
gdc: fatal error: cannot specify ‘-o’ with ‘-c’, ‘-S’ or ‘-E’ 
with multiple files
```

Since both (a single) .d file and the single line .c (include 
".h") file is on the command line:

e.g.

```
gdc -dip25 -Wno-error=builtin-declaration-mismatch 
-fall-instantiations -c -o 
/.dub/cache/liblfdsd/0.3.4/build/library-unittest-swOTDhNpx8l9Y
K1BPSAow/liblfdsd.a -fdebug -g -Werror -Wall -Wno-deprecated
/.dub/packages/liblfdsd/0.3.4/liblfdsd/source/liblfdsd.d
/.dub/packages/liblfdsd/0.3.4/liblfdsd/source/liblfdsd_ImportC.c -Xlinker
-Xlinker -llfdsd -Xlinker -llfdsdc -Xlinker -llfds711

```

Thanks.

Jun 21

D Programming

C/C++ Programming

Other

digitalmars.D.learn - ImportC in a Dub project