digitalmars.D.learn - ImportC in a Dub project
- Carsten Schlote (37/37) Oct 28 2022 Hi,
- Imperatorn (2/6) Oct 28 2022 Are you using DMD?
- Carsten Schlote (10/19) Oct 28 2022 I'm using
- Steven Schveighoffer (5/56) Oct 28 2022 Are you passing the c file to the compiler? Also, you must be using dmd
- Carsten Schlote (61/64) Oct 28 2022 ```
- Imperatorn (3/21) Oct 28 2022 Like schveiguy said, what's your build line? dmd should pick up
- Carsten Schlote (8/10) Oct 28 2022 What do you mean with 'buildline'? The project is build with dub.
- Imperatorn (3/13) Oct 28 2022 Have you read this?
- Steven Schveighoffer (7/73) Oct 28 2022 By default dub does not build C files (as evidenced by your command
- Carsten Schlote (7/12) Oct 30 2022 Ok, this works. So Dub is not picking any C files by default.
- Carsten Schlote (68/68) Oct 30 2022 It turned out, that the required changes to add support for C
- Imperatorn (4/10) Oct 30 2022 Agreed, but don't forget about the .i extension.
- Carsten Schlote (10/22) Oct 30 2022 I will merge the two PRs into a new patchset, and also add
- Imperatorn (3/16) Oct 30 2022 Yeah maybe something like that. Because I agree we need to update
Hi, I created a Dub project containing two files: app.d and zstd_binding.c ``` $ cat source/zstd_binding.c #include <zstd.h> #include <zstd_errors.h> #include <stdio.h> void relatedCode(void) { printf("Hallo! This is some output from C code!\n"); } ``` and ``` $ cat source/app.d import std.conv; import std.stdio; import zstd_binding; void main() { auto versionNr = ZSTD_versionNumber(); auto versionStr = ZSTD_versionString(); writefln("Version Info: Numeric %d String %s", versionNr, versionStr.to!string); /** more code stripped */ relatedCode(); // The linker can't find it.... } ``` Accessing ZSTD works perfect. For linking you must give "-lzstd". So far, so good. The problem started, when I try to access the ```relatedCode()``` C function. There is simply no code generated for this C function. I need to compile the C separately into an object file and pass it as extra linker argument. I expected that ImportC also allows to import C functions and creates the required code on the fly. Is this a bug? What's wrong?
Oct 28 2022
On Friday, 28 October 2022 at 17:45:59 UTC, Carsten Schlote wrote:Hi, I created a Dub project containing two files: app.d and zstd_binding.c [...]Are you using DMD?
Oct 28 2022
Carsten Schlote <carsten.schlote gmx.net> On Friday, 28 October 2022 at 17:56:57 UTC, Imperatorn wrote:On Friday, 28 October 2022 at 17:45:59 UTC, Carsten Schlote wrote:I'm using ``` $ dub --version DUB version 1.29.2, built on Sep 29 2022 $ dmd-beta --version DMD64 D Compiler v2.101.0-beta.1-36-ga2865d74fb Copyright (C) 1999-2022 by The D Language Foundation, All Rights Reserved written by Walter Bright ```Hi, I created a Dub project containing two files: app.d and zstd_binding.c [...]Are you using DMD?
Oct 28 2022
On 10/28/22 1:45 PM, Carsten Schlote wrote:Hi, I created a Dub project containing two files: app.d and zstd_binding.c ``` $ cat source/zstd_binding.c #include <zstd.h> #include <zstd_errors.h> #include <stdio.h> void relatedCode(void) { printf("Hallo! This is some output from C code!\n"); } ``` and ``` $ cat source/app.d import std.conv; import std.stdio; import zstd_binding; void main() { auto versionNr = ZSTD_versionNumber(); auto versionStr = ZSTD_versionString(); writefln("Version Info: Numeric %d String %s", versionNr, versionStr.to!string); /** more code stripped */ relatedCode(); // The linker can't find it.... } ``` Accessing ZSTD works perfect. For linking you must give "-lzstd". So far, so good. The problem started, when I try to access the ```relatedCode()``` C function. There is simply no code generated for this C function. I need to compile the C separately into an object file and pass it as extra linker argument. I expected that ImportC also allows to import C functions and creates the required code on the fly. Is this a bug? What's wrong?Are you passing the c file to the compiler? Also, you must be using dmd for ImportC currently. What is your build line? -Steve
Oct 28 2022
On Friday, 28 October 2022 at 18:31:25 UTC, Steven Schveighoffer wrote:Are you passing the c file to the compiler? Also, you must be using dmd for ImportC currently. What is your build line?``` $ cat dub.json { "authors": [ "Carsten Schlote" ], "copyright": "Copyright © 2022, Carsten Schlote", "description": "A minimal D application.", "license": "proprietary", "name": "importc-app", "lflags": [ "-lzstd", "zstd_binding.o" ], "preBuildCommands": [ "gcc -g -O0 -c -o zstd_binding.o source/zstd_binding.c"] ``` ``` $ dub build --compiler=dmd-beta -v Using dub registry url 'https://code.dlang.org/' Refreshing local packages (refresh existing: true)... Looking for local package map at /var/lib/dub/packages/local-packages.json Looking for local package map at /home/cschlote/.dub/packages/local-packages.json Looking for local package map at /home2/cschlote/build/dlang/importc-tests/dubbed/.dub/packages/local-packages.json Note: Failed to determine version of package importc-app at .. Assuming ~master. Refreshing local packages (refresh existing: false)... Looking for local package map at /var/lib/dub/packages/local-packages.json Looking for local package map at /home/cschlote/.dub/packages/local-packages.json Looking for local package map at /home2/cschlote/build/dlang/importc-tests/dubbed/.dub/packages/local-packages.json Refreshing local packages (refresh existing: false)... Looking for local package map at /var/lib/dub/packages/local-packages.json Looking for local package map at /home/cschlote/.dub/packages/local-packages.json Looking for local package map at /home2/cschlote/build/dlang/importc-tests/dubbed/.dub/packages/local-packages.json Generating using build Configuring dependent importc-app, deps: Performing "debug" build using dmd-beta for x86_64. Target '/home2/cschlote/build/dlang/importc-tests/dubbed/.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C68 C85F85/importc-app' doesn't exist, need rebuild. importc-app ~master: building configuration "application"... Running pre-build commands... Running gcc -g -O0 -c -o zstd_binding.o source/zstd_binding.c dmd-beta -c -of.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C68B 85F85/importc-app.o -debug -g -w -version=Have_importc_app -Isource/ source/app.d -vcolumns Linking... dmd-beta -of.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C6 BC85F85/importc-app .dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C68B 85F85/importc-app.o -L--no-as-needed -L-lzstd -Lzstd_binding.o -g Copying target from /home2/cschlote/build/dlang/importc-tests/dubbed/.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C6 BC85F85/importc-app to /home2/cschlote/build/dlang/importc-tests/dubbed ``` Without the ```preBuildCommands``` and without passing the resulting object file to the linker, the ```relatedCode()``` doesn't exist. And linking is failing.
Oct 28 2022
On Friday, 28 October 2022 at 18:43:21 UTC, Carsten Schlote wrote:On Friday, 28 October 2022 at 18:31:25 UTC, Steven Schveighoffer wrote:Like schveiguy said, what's your build line? dmd should pick up the file if the import has the same name as the file.[...]``` $ cat dub.json { "authors": [ "Carsten Schlote" ], "copyright": "Copyright © 2022, Carsten Schlote", "description": "A minimal D application.", "license": "proprietary", "name": "importc-app", "lflags": [ "-lzstd", "zstd_binding.o" ], "preBuildCommands": [ "gcc -g -O0 -c -o zstd_binding.o source/zstd_binding.c"] ``` [...]
Oct 28 2022
On Friday, 28 October 2022 at 18:56:03 UTC, Imperatorn wrote:Like schveiguy said, what's your build line? dmd should pick up the file if the import has the same name as the file.What do you mean with 'buildline'? The project is build with dub. See previous posts with outputs from commandline. DMD does find and import the C file. But it doesn't create code for ```relatedCode()``` in the C file. The only way to fix this, is to compile it with a C compiler and use the object as additional linker output. IMHO this is a bug.
Oct 28 2022
On Friday, 28 October 2022 at 19:04:21 UTC, Carsten Schlote wrote:On Friday, 28 October 2022 at 18:56:03 UTC, Imperatorn wrote:Have you read this? https://dlang.org/spec/importc.htmlLike schveiguy said, what's your build line? dmd should pick up the file if the import has the same name as the file.What do you mean with 'buildline'? The project is build with dub. See previous posts with outputs from commandline. DMD does find and import the C file. But it doesn't create code for ```relatedCode()``` in the C file. The only way to fix this, is to compile it with a C compiler and use the object as additional linker output. IMHO this is a bug.
Oct 28 2022
On 10/28/22 2:43 PM, Carsten Schlote wrote:On Friday, 28 October 2022 at 18:31:25 UTC, Steven Schveighoffer wrote:By default dub does not build C files (as evidenced by your command line). It may not even let you I don't know, but try: ```json "sourceFiles" : ["source/zstdc_binding.c"] ``` -SteveAre you passing the c file to the compiler? Also, you must be using dmd for ImportC currently. What is your build line?``` $ cat dub.json { "authors": [ "Carsten Schlote" ], "copyright": "Copyright © 2022, Carsten Schlote", "description": "A minimal D application.", "license": "proprietary", "name": "importc-app", "lflags": [ "-lzstd", "zstd_binding.o" ], "preBuildCommands": [ "gcc -g -O0 -c -o zstd_binding.o source/zstd_binding.c"] ``` ``` $ dub build --compiler=dmd-beta -v Using dub registry url 'https://code.dlang.org/' Refreshing local packages (refresh existing: true)... Looking for local package map at /var/lib/dub/packages/local-packages.json Looking for local package map at /home/cschlote/.dub/packages/local-packages.json Looking for local package map at /home2/cschlote/build/dlang/importc-tests/dubbed/.dub/packages/local-packages.json Note: Failed to determine version of package importc-app at .. Assuming ~master. Refreshing local packages (refresh existing: false)... Looking for local package map at /var/lib/dub/packages/local-packages.json Looking for local package map at /home/cschlote/.dub/packages/local-packages.json Looking for local package map at /home2/cschlote/build/dlang/importc-tests/dubbed/.dub/packages/local-packages.json Refreshing local packages (refresh existing: false)... Looking for local package map at /var/lib/dub/packages/local-packages.json Looking for local package map at /home/cschlote/.dub/packages/local-packages.json Looking for local package map at /home2/cschlote/build/dlang/importc-tests/dubbed/.dub/packages/local-packages.json Generating using build Configuring dependent importc-app, deps: Performing "debug" build using dmd-beta for x86_64. Target '/home2/cschlote/build/dlang/importc-tests/dubbed/.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C68 C85F85/importc-app' doesn't exist, need rebuild. importc-app ~master: building configuration "application"... Running pre-build commands... Running gcc -g -O0 -c -o zstd_binding.o source/zstd_binding.c dmd-beta -c -of.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C68B 85F85/importc-app.o -debug -g -w -version=Have_importc_app -Isource/ source/app.d -vcolumns Linking... dmd-beta -of.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C6 BC85F85/importc-app .dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C68B 85F85/importc-app.o -L--no-as-needed -L-lzstd -Lzstd_binding.o -g Copying target from /home2/cschlote/build/dlang/importc-tests/dubbed/.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-36-ga2865d74fb-9DB8925B9A720899BDD003C6 BC85F85/importc-app to /home2/cschlote/build/dlang/importc-tests/dubbed ``` Without the ```preBuildCommands``` and without passing the resulting object file to the linker, the ```relatedCode()``` doesn't exist. And linking is failing.
Oct 28 2022
On Friday, 28 October 2022 at 19:08:47 UTC, Steven Schveighoffer wrote:By default dub does not build C files (as evidenced by your command line). It may not even let you I don't know, but try: ```json "sourceFiles" : ["source/zstdc_binding.c"] ```Ok, this works. So Dub is not picking any C files by default. Thanks for the hint. I will spent some time to prepare a fix for Dub, so that C files (only '.c' files, no headers) in detected/configured source directories are picked up as well.
Oct 30 2022
It turned out, that the required changes to add support for C files in Dub are really small. So I added a PR (https://github.com/dlang/dub/pull/2521). There is also some other PR (https://github.com/dlang/dub/pull/2270) releated to C file support in Dub. With my PR (2521) applied I can now compile my ImportC test projects (https://gitlab.vahanus.net:dlang/examples/importc-tests.git) without any additional hacks. Just a current copy of DUB with the patch and a current beta build of DMD is needed. ``` $ ../../dub/bin/dub build -v --compiler=dmd-beta -f Using dub registry url 'https://code.dlang.org/' Note: Failed to determine version of package importc-app at .. Assuming ~master. Generating using build Configuring dependent importc-app, deps: Starting Performing "debug" build using dmd-beta for x86_64. Building importc-app ~master: building configuration [application] dmd-beta -c -of.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-38-gfabd06214e-B8D88D35889281A0F16BEA64448915871A8820982AFB76E0D5F2AD9849 F80E5/importc-app.o -debug -g -w -version=Have_importc_app -Isource/ source/app.d source/some_d_file.d source/zstd_binding.c -vcolumns Linking importc-app dmd-beta -of.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-38-gfabd06214e-B8D88D35889281A0F16BEA64448915871A8820982AFB76E0D5F2AD98 9AF80E5/importc-app .dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-38-gfabd06214e-B8D88D35889281A0F16BEA64448915871A8820982AFB76E0D5F2AD9849 F80E5/importc-app.o -L--no-as-needed -L-lzstd -g Copying target from /home2/cschlote/build/dlang/importc-tests/dubbed/.dub/build/application-debug-linux.posix-x86_64-dmd_v2.101.0-beta.1-38-gfabd06214e-B8D88D35889281A0F16BEA64448915871A8820982AFB76E0D5F2AD98 9AF80E5/importc-app to /home2/cschlote/build/dlang/importc-tests/dubbed ``` However, the maintainers of Dub seem to have different ideas resulting into much more work to be done. Their ideas are surely good and reasonable, but it sounds like a lot of changes needed to be applied before we see support for direct and easy C file support in Dub. In the meantime (weeks, months, years?) you are forced to either pass an explicite list of C sourcefiles in dub.json (see previous forum posts), or use some ```preBuild/Generate``` commands to compile your C source file(s) to object file(s) and add these object files to other explicite list as linker input. Both ways are ugly and cumbersome. What we need in dub instead is that C files are just picked up like any other D file as a valid source for the compiler. And without any explicite lists of source files in ```dub.json```. The C header files are intentionally left out here, because it is bad style to have real functions in a C header. Yes, I know, people are doing such things. It is one of the bad habbits of C/C++. If you want to include a C header, just wrap it into a C file, which is then picked up by Dub and passed to the D compiler. See the mentioned example project for wrapping the ZSTD C header. This is easy and straight forward. I'm pretty sure that any additional code to support the simple usecases for C header files will also be easy to implement. So if 80% or more of the general usecases with C source/header files in Dub can be implemented with simple and small changes like with the PR mentioned above, we should do it now. A better and more general implementation covering 100%+ of all thinkable usecases and edge conditions can be done later after having a working solution *now*, instead of discussing a *proper* solutions for several months. IMHO **straight-forward** and **easy** support for using C files in Dub projects is a must for the up-coming next DMD release with its much improved ImportC feature. It will also help a lot when we want to promote D as a suitable language alternative for C/C++ programmers. As long as D needs hand-crafted binding files or other hacky workarounds instead of just ```ìmport your.c.file;``` and a C source wrapper it will be hard to convince them. So I strongly recommend to add easy to use support for C files in Dub projects **ASAP**.
Oct 30 2022
On Sunday, 30 October 2022 at 12:46:10 UTC, Carsten Schlote wrote:It turned out, that the required changes to add support for C files in Dub are really small. So I added a PR (https://github.com/dlang/dub/pull/2521). There is also some other PR (https://github.com/dlang/dub/pull/2270) releated to C file support in Dub. [...]Agreed, but don't forget about the .i extension. Imo we could add a switch which enables something like your PR (include .c and .i) until we know what the default should be.
Oct 30 2022
On Sunday, 30 October 2022 at 13:38:23 UTC, Imperatorn wrote:On Sunday, 30 October 2022 at 12:46:10 UTC, Carsten Schlote wrote:I will merge the two PRs into a new patchset, and also add appropriate code to search für {c|i|h} files at the right places. The patchset should be as minimal as possible, so that the planned better solution is not blocked/messed up.It turned out, that the required changes to add support for C files in Dub are really small. So I added a PR (https://github.com/dlang/dub/pull/2521). There is also some other PR (https://github.com/dlang/dub/pull/2270) releated to C file support in Dub. [...]Agreed, but don't forget about the .i extension.Imo we could add a switch which enables something like your PR (include .c and .i) until we know what the default should be.Maybe a switch/flag should be added to the dub.json/sdl files to turn on the new behaviour? This allows to enable the new search logic for C files without breaking something old. It would make it more clear, that also a new 2.101+ D compiler is needed.
Oct 30 2022
On Sunday, 30 October 2022 at 14:42:55 UTC, Carsten Schlote wrote:On Sunday, 30 October 2022 at 13:38:23 UTC, Imperatorn wrote:Yeah maybe something like that. Because I agree we need to update it since we now have ImportC.[...]I will merge the two PRs into a new patchset, and also add appropriate code to search für {c|i|h} files at the right places. The patchset should be as minimal as possible, so that the planned better solution is not blocked/messed up.[...]Maybe a switch/flag should be added to the dub.json/sdl files to turn on the new behaviour? This allows to enable the new search logic for C files without breaking something old. It would make it more clear, that also a new 2.101+ D compiler is needed.
Oct 30 2022