• Johannes Pfau (118/118) Feb 25 2012 As recently discussed in a pull request it would be great if we could
• Kagamin (17/17) Feb 26 2012 deimos
• Johannes Pfau (5/23) Feb 26 2012 Keeping the C libraries in own repositories is an interesting idea. We'd
• Kagamin (4/5) Feb 26 2012 That's an issue. Given the c libraries incompatibilities, what
• Johannes Pfau (47/53) Feb 26 2012 Fortunately backtrace is not part of core.stdc. core.stdc only includes
• Kagamin (9/13) Feb 27 2012 The structure seems good, but I still suggest to go through
• Kagamin (2/3) Feb 27 2012 I mean, should not be a problem.
• Leandro Lucarella (18/24) Feb 26 2012 Backtrace is not stdc :)
• Johannes Pfau (12/34) Feb 26 2012 I mostly agree. The API of the C libraries is the same, what's

Johannes Pfau <nospam example.com> writes:

As recently discussed in a pull request it would be great if we could
make it easier to port druntime to different architectures. Special
casing every C library, every architecture and every OS using version
blocks could lead to difficult to maintain code, so we need a better
solution.

As far as I can see, the biggest differences are caused by different C
libraries, not by different architectures (except 32bit vs 64 bit
differences). For example glibc headers seem to be very similar for arm
and x86, but bionic vs. glibc brings more differences. Bionic initially
didn't support anything wchar_t related, so core.stdc.wchar_ and
core.stdc.wctype couldn't work with android at all. And C libraries
vary even more in the subset of posix functionality they support.

We need a directory scheme to support
* Different C libraries
  * on different OS (glibc/bsd glibc/linux)
  * on the same OS (glibc on linux, bionic on linux, uclibc on
    linux, ...)
* Different architectures (ARM, X86, MIPS, PPC, SH4, ...)

I propose the following directory layout, but of course
better solutions are appreciated:

druntime
|-core
|---stdc
|---sync
|---sys
|-----posix
|-gc
|-gcstub
|-rt
|-gcc
|---arch
|-----glibc
|-------core
|---------stdc
|---------sync
|---------sys
|-----------posix
|-----bionic
|-------stdc
|-------sync
|-------core
|---------sys
|-----------posix
|-----newlib
|-------stdc
|-------sync
|-------core
|---------sys
|-----------posix

Code in the gc, gcstub and rt directories shouldn't depend on special C
libraries, small differences can be solved using version() blocks.

Every C library get's it's  own bindings in the gcc.arch package. I'm
not sure if those bindings should only provide functionality included
in core, or whether these bindings should include additional, C library
specific bindings.

The files in core.* would reference the corresponding files in
gcc.arch.core.*

The downside of that approach is that we'll have some code duplication,
especially for similar C libraries. And merging changes is therefore
not that easy as well. However, as the C and Posix headers don't change
often, that's probably neglectable.

Here are some examples for that directory scheme:

core/stdc/config.d (Should be shared with dmd druntime?)
-----------------------------------------------
enum LibC
{
    GlibC,
    uClibc,
    dietlibc,
    newlib,
    dmc, //digitalmars, windows
    msvcrt,
    ...,
    unknown
}

//Need some way to define this correctly
enum libC = LibC.GlibC;
enum string libCVersion = ""; //Free form version number
-----------------------------------------------

core/stdc/math.d
-----------------------------------------------
import core.stdc.config;

static if(libC == LibC.GlibC)
{
    public import gcc.arch.glibc.core.stdc.math;
}
else
{
    static assert(false, "C library '" ~ libC ~ "' not supported in " ~
    __FILE__);
}

//Maybe put generic parts here?
-----------------------------------------------

core/sys/posix/dirent.d
-----------------------------------------------
import core.stdc.config;

static if(libC == LibC.GlibC)
{
    /*
     * I admit this looks horrible. We can probably remove the core
     * part, maybe even gcc, so arch.glibc.sys.posix.dirent?
     * or glibc[.core].sys.posix.dirent?
     */
    public import gcc.arch.glibc.core.sys.posix.dirent;
}
else static if(libC == LibC.GlibC)
{
    public import gcc.arch.glibc.core.sys.posix.dirent;
}
else
{
    static assert(false, "C library '" ~ libC ~ "' not supported in " ~
    __FILE__);
}
-----------------------------------------------

Maybe we could use pragma(msg) warnings instead of static asserts, so
it'd be possible to import the modules in any case and use
traits(__compiles) to check for specific symbols/functions.

"Kagamin" <spam here.lot> writes:

deimos
|-glibc
|---stdio.d
|---math.d
|---sys
|-----posix
|-bionic
|---stdio.d
|---math.d
|---sys
|-----posix
|-dietlibc
|---stdio.d
|---math.d
|---sys
|-----posix

Maybe better replicate c library structure?

Johannes Pfau <nospam example.com> writes:

Am Sun, 26 Feb 2012 12:47:30 +0100
schrieb "Kagamin" <spam here.lot>:

 deimos
 |-glibc
 |---stdio.d
 |---math.d
 |---sys
 |-----posix
 |-bionic
 |---stdio.d
 |---math.d
 |---sys
 |-----posix
 |-dietlibc
 |---stdio.d
 |---math.d
 |---sys
 |-----posix
 
 Maybe better replicate c library structure?

Keeping the C libraries in own repositories is an interesting idea. We'd
still have to make sure core.* works. We could use public imports
in the core.* files for that as described in the previous message.

"Kagamin" <spam here.lot> writes:

On Sunday, 26 February 2012 at 13:06:58 UTC, Johannes Pfau wrote:
 still have to make sure core.* works.

That's an issue. Given the c libraries incompatibilities, what 
functionality the core.stdc should provide? What to do with 
functions like backtrace?

Johannes Pfau <nospam example.com> writes:

Am Sun, 26 Feb 2012 19:30:03 +0100
schrieb "Kagamin" <spam here.lot>:

 On Sunday, 26 February 2012 at 13:06:58 UTC, Johannes Pfau wrote:
 still have to make sure core.* works.

 
 That's an issue. Given the c libraries incompatibilities, what 
 functionality the core.stdc should provide? What to do with 
 functions like backtrace?

Fortunately backtrace is not part of core.stdc. core.stdc only includes
functions which are described by the C(99?) standard and C libraries
usually provide most of that functionality. It's more difficult for
core.sys.posix as 'lightweight' libcs often decide to not implement
some posix modules. If some of the basic functionality isn't supported,
there's probably nothing we can do.

For the special backtrace & backtrace_symbols case we should get rid of
backtrace and use gcc functionality (We already have access to that in
gcc.unwind). backtrace_symbols can be implemented using dladdr. dladdr
isn't standard C/Posix either, but it's supported by more C libraries.
In case dl_addr isn't available, we can always just print the addresses
instead of the function names. addr2line works better than
backtrace_symbols / dladdr anyway.

I had a look at glibc and bionic though and those C libraries provide
much more functionality than what's currently included in druntime.
Creating complete bindings is probably a waste of time as those 'extra'
functions are not portable anyway.

Maybe the simplest solution is to add a 'sublibrary' to druntime, libdc
or something, then store the normal core.stdc/core.sys modules in a
subdir for every C library and compile in only the correct modules using
the configure script:

druntime
|-core
|---atomic.d (generic files)
|---bitop.d
|---[...]
|-gc
|-gcstub
|-rt
|-libdc
|---glibc
|-----core
|-------sys
|-------stdc
|---bionic
|-----core
|-------sys
|-------stdc
|---newlib
|-----core
|-------sys
|-------stdc

The files in libc/glibc/core/stdc would be in the core.stdc package
(not glib.core.stdc), so we just have to compile the correct
subdirectory into druntime.

"Kagamin" <spam here.lot> writes:

On Sunday, 26 February 2012 at 20:54:27 UTC, Johannes Pfau wrote:
 The files in libc/glibc/core/stdc would be in the core.stdc 
 package
 (not glib.core.stdc), so we just have to compile the correct
 subdirectory into druntime.

The structure seems good, but I still suggest to go through 
deimos project. One can start with partial bindings, which get 
druntime working, but eventually they should become complete, if 
someone will have time for it. Does deimos accept partial 
bindings? Should be a problem, I believe. If the deimos bindings 
will be worked on later, we'll get them in core* for free. 
libc/glibc/core/stdc will just reimport the deimos code in a 
structure adopted for druntime.

"Kagamin" <spam here.lot> writes:

On Monday, 27 February 2012 at 11:10:03 UTC, Kagamin wrote:
 Should be a problem, I believe.

I mean, should not be a problem.

Leandro Lucarella <luca llucax.com.ar> writes:

Kagamin, el 26 de febrero a las 19:30 me escribiste:
 On Sunday, 26 February 2012 at 13:06:58 UTC, Johannes Pfau wrote:
still have to make sure core.* works.

 
 That's an issue. Given the c libraries incompatibilities, what
 functionality the core.stdc should provide? What to do with
 functions like backtrace?

Backtrace is not stdc :)

I'm sorry to interject when I'm not very involved, but libraries
implementing the C standard library should not be too different from
each other. Maybe some miss some features, but the things they have,
should be pretty much the same.

I think core.stdc should publicly import the local compiler's stdc. Any
incompatibilities among different libraries should be handled by the
users of core.stdc, not core.stdc itself, which is supposed to be just
a thin layer to get what the current C environment have. People should
be using std stuff, not core.stdc, most of the time, specially if they
intend to write portable code.

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
Si por el chancho fuera, se autocomería con chimichurri Worshestershire!

Johannes Pfau <nospam example.com> writes:

Am Sun, 26 Feb 2012 18:00:49 -0300
schrieb Leandro Lucarella <luca llucax.com.ar>:

 Kagamin, el 26 de febrero a las 19:30 me escribiste:
 On Sunday, 26 February 2012 at 13:06:58 UTC, Johannes Pfau wrote:
still have to make sure core.* works.

 
 That's an issue. Given the c libraries incompatibilities, what
 functionality the core.stdc should provide? What to do with
 functions like backtrace?

 
 Backtrace is not stdc :)
 
 I'm sorry to interject when I'm not very involved, but libraries
 implementing the C standard library should not be too different from
 each other. Maybe some miss some features, but the things they have,
 should be pretty much the same.

I mostly agree. The API of the C libraries is the same, what's
different is ABI: The C/Posix standards don't define layouts for some
structs, for example the 'stat' struct is different between most C
libraries. That's the reason we need different bindings for different C
libraries.
 
 I think core.stdc should publicly import the local compiler's stdc.
 Any incompatibilities among different libraries should be handled by
 the users of core.stdc, not core.stdc itself, which is supposed to be
 just a thin layer to get what the current C environment have. People
 should be using std stuff, not core.stdc, most of the time, specially
 if they intend to write portable code.
 

I agree. The question is how that 'publicly import' should actually
work. Using real 'public import' statements with static if/version
blocks adds some additional clutter. Placing bindings for different C
libraries in different directories and just compiling in the correct
folder seems to be the easiest solution.