www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - version(Confused) researched... (long)

reply Alan West <alan alanz.com> writes:
When I started getting into D I was confused by the versions used for 
different operating systems. Since looking at a lot of D code in phobos 
and other projects, I've spent some time working out what we attempt to 
accomplish with the versions Windows, Linux, Posix, as to me they seemed 
to be missing the target.

In the language spec I believe it says that D uses the standard C 
runtime library, so...

These libraries aim to implement, in whole, in part, or extended, a
standard C library relevant to their target kernel/environment (using 
possible version identifiers):

     dietlibc (An embedded libc)
     freebsd  (FreeBSD's own libc)
     glibc    (Linux Standard C Library, but used by others too)
     minix    (Minix libc)
     msvcrt   (Windows Standard C Library)
     netbsd   (NetBSD libc)
     newlib   (An embedded libc)
     openbsd  (OpenBSD libc)
     darwin   (Apple libc)
     opendarwin (OpenDarwin libc)
     uclibc   (An embedded libc)
     ...

They may or may not conform with these particular standards created
to allow cross platform software development in the C language:

     c89      <--- oldest
     posix92 or posix1
     unix95
     posix96 or posix2
     unix98
     c99
     susv3    <--- newest

Of those standards created; c89 and c99 appear to be the only standards
which solely target the standard C library. The others, as well as 
extensions to the C library, also define a Unix System standard, which 
can include around 160 utility programs such as cd, rmdir, ps, sed, 
grep, mailx, man, vi.

Therefore my logical conclusion is:

   std.c (modules implementing bindings for the c99 standard)
     assert
     complex
     ctype
     errno
     fenv
     float
     inttypes
     iso646
     limits
     locale
     math
     setjmp
     signal
     stdarg
     stdbool
     stddef
     stdint
     stdio
     stdlib
     string
     tgmath
     time
     wchar
     wctype

   std.c.unix (modules impl. bindings for the SUSv3 C extensions)
     aio
     cpio
     dirent
     dlfcn
     fcntl
     fmtmsg
     fnmatch
     ftw
     glob
     grp
     iconv
     langinfo
     libgen
     monetary
     mqueue
     ndbm
     netdb
     nl_types
     poll
     pthread
     pwd
     regex
     sched
     search
     semaphore
     spawn
     strings
     stropts
     syslog
     tar
     termios
     trace
     ucontext
     ulimit
     unistd
     utime
     utmpx
     wordexp
	
     std.c.unix.arpa
       inet
					
     std.c.unix.net
       if

     std.c.unix.netinet
       in
       tcp
	
     std.c.unix.sys
       ipc
       mman
       msg
       resource
       select
       sem
       shm
       socket
       stat
       statvfs
       time
       timeb
       times
       types
       uio
       un
       utsname
       wait

Obviously some of those items aren't actually required such as the 
assert module - but they are named one-to-one to standard C header files 
(as phobos does such as std.c.stdio). There is also obviously no need to 
deal with parts of unix and posix standards which specify utility programs.

In all of the above packages and modules, because they implement 
bindings to a C standard library, I believe the version()s available 
should be named based on the the possible C library being linked, rather 
than a standard or operating system. Therefore typically those I 
mentioned in the first list above as versions for starters.

I called the Standard C Library for Windows "version(msvcrt)" because 
that is the library linked to, but also because "version(windows)" is 
ambiguous due to the option of linking with other Standard C Library 
implementations on Windows such as mingw, cygwin, and maybe others.

FreeBSD, NetBSD, OpenBSD, Darwin, and others..., seem to call their 
standard C library as simply libc. I believe the standard C library 
commonly used on Linux (glibc) may also be used on other systems 
including the different *BSDs.

But also due to the possible quantity in versions it may be the case 
that modules in std.c and std.c.unix simply alias import from modules in 
packages with names like:

     std.c.msvcrt (Microsoft standard C library)
     std.c.glibc  (for glibc specific modules)
     std.c.glibc.linux (for Linux specific parts of glibc)

     std.c.unix.msvcrt (May have some support for unix features)
     std.c.unix.mingw
     std.c.unix.glibc  (for glibc implementation of unix extensions)
     std.c.unix.uclibc

Unix here meaning the standard C library extensions required for unix 
systems.

A lot should, but not always, overlap, so it would be good
to be able to do:
     version( msvcrt | glibc | darwin ) {}
     else version( mingw | cygwin ) {}
     else version( freebsd | netbsd | openbsd ) {}
     else static assert(0); // feature not implemented by C lib


This allows the modules in std.c and std.c.unix to expose the current 
cross platform C library APIs. Programs using modules just from these 
two standard packages would run on all supported systems and wouldn't 
need any particular version() conditions.

Which brings me on to system specific APIs

I think the following package hierarchy should be made available 
containing modules that bind to APIs that are operating system specific, 
as delivered by their vendor or creator, which relate more with 
version()s currently being used:

     sys.windows  (sys specific modules like: )
       kernel
       user
       gdi
       winspool
       comdlg
       advapi
       comctl
       shell
       ...more
     sys.linux  (for direct syscalls not exposed in standard C lib)
     sys.mach   (modules for the mach micro-kernel in macosx etc)
     sys.macos  (sys specific modules such as Carbon)
     sys.solaris
     sys.tru64
     ...etc

But of course those systems also have sub-versions (which may or may not 
use packages for their specific versions):

     sys.windows.win32 (Win32 API available on many versions)
     sys.windows.win64 (Win64 API only on newer NT variants)
     sys.windows.winfx (Microsoft's new Longhorn API)

     sys.windows.95    (95-ME variants)
     sys.windows.ce    (CE embedded variants)
     sys.windows.nt    (NT-2000-XP-2003 variants)
     sys.windows.vista (Supposedly its the new name)

     sys.linux.v2_4
     sys.linux.v2_6
     sys.linux.v2_6_11

     sys.mach.v3
     sys.mach.v4
     sys.mach.apple (Apple's API may have extensions)

     sys.macos.v9
     sys.macos.v10_2_8
     sys.macos.v10_3_9
     sys.macos.v10_4

To import all modules in a package, would this work? (imagine):

     sys/windows.d   (containing imports for kernel, user, gdi, etc)

     sys/windows/kernel.d
     sys/windows/user.d
     sys/windows/gdi.d

So someone writing a program could use "import sys.windows;" to import 
all the common modules relevant. As the windows.h C file mainly includes 
all the other headers, maybe even "version = windows_lean_and_mean; 
import sys.windows;" as you can do now in C using a "#define".

Conclusion

We really need an excellent h2d tool.

What do you think?

Oh I also did this list of architectures and their processor variants:

module archs;

version(ia32)
{
     version(i386) {}
     version(i486) {}
     version(pentium) {}
     version(pentium3) {}
     version(pentium4) {}
     version(athlon) {}
     version(cyrix) {}
	
     version(mmx) {}
     version(simd) {}
}

version(ia64) {} // I believe these two are
version(amd64) {}// not actually the same???

version(powerpc)
{
     version(ppc7450) {}
     version(altivec) {}
}

version(powerpc64)
{
     version(ppc970) {}
     version(power4) {}
     version(altivec) {}
}

version(alpha)
{
     version(ev6) {}
     version(ev7) {}
     version(ev7z) {}
}

version(arm)
{
     version(arm6) {}
     version(arm7) {}
     version(strongarm) {}
     version(xscale) {}
}

version(mips)
{
     version(mips3) {}
     version(r10000) {}
}

version(sparc) {}
version(sparc64) {}

                           -- THE END --
Aug 10 2005
next sibling parent reply =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Alan West wrote:

 When I started getting into D I was confused by the versions used for 
 different operating systems. Since looking at a lot of D code in phobos 
 and other projects, I've spent some time working out what we attempt to 
 accomplish with the versions Windows, Linux, Posix, as to me they seemed 
 to be missing the target.
This has been debated for ages, and the surviving ones are: version(Windows) and version(Unix). DMD still has the limited "linux" only, just as GDC does have a specific version saying what the current platform `uname` is. It seems you were proposing some brand new hierarchy, but anyway. Cleaning up the std.c portion (as in Ares) would be a good start ? http://www.dsource.org/forums/viewforum.php?f=31 (C99 changes) [...]
 But of course those systems also have sub-versions (which may or may not 
 use packages for their specific versions):
[...]
     sys.macos.v9
     sys.macos.v10_2_8
     sys.macos.v10_3_9
     sys.macos.v10_4
Just a pet peeve, but those are really *not* the same operating system. And I don't really think it needs to separate out the update versions ? Thus: sys.macos.v7 sys.macos.v8 sys.macos.v9 sys.macosx.v10_1 sys.macosx.v10_2 sys.macosx.v10_3 sys.macosx.v10_4 Not that it has much practical value, as D won't run on Mac OS 9 anyway. For Mac OS X, it would probably be a more useful start to have "carbon"?
 Oh I also did this list of architectures and their processor variants:
I assume you know that there are already such versions, and that they are using upper case, but that you had some special reason for changing? e.g. version(X86) version(PPC) Not sure if you wanted Phobos/D to be a) updated or b) rewritten ? --anders
Aug 11 2005
next sibling parent reply Alan West <alan alanz.com> writes:
Anders F Björklund wrote:
 Alan West wrote:
 Not that it has much practical value, as D won't run on Mac OS 9 anyway.
 For Mac OS X, it would probably be a more useful start to have "carbon"?
Yeah I understand, I was trying to show how the namespace could grow, once things started opening up to a much wider array of different systems especially GDC. I've seen quite a few messages on this versions subject, and probably because there are so many discussions about it shows that the naming is off target. Maybe it hasn't been resolved yet because it was mainly a discussion around Windows and Linux or Posix, but I attempted to take a bigger step back and look at things with a fresher perspective than had done before. I tried to to keep the post to more factual though hypothetical examples rather than jabbering on endlessly. I hoped that setting things out, showing a kind of picture of how things could be, by including many systems and their standard C libraries it would click more with people. I used all lowercase in the names so versions could be more logically related to version specific packages/modules. When looking at the possibilities try not to think about current limitations, but envisage the D language being right up there at the number one spot, used by many in the industry, taught in college/universities etc... Widely available on many different platforms. It would be cool to be able to set in stone now early the structure for people to follow in implementing bindings to those systems, whether it is a mobile phone, or super computer. It would allow Ds evolution to run much more smoothly, with happier users and developers. I hope I don't come across rude in anyway, that's the last thing I intend, and if anyone wants me to clarify further on things I don't mind, mail me directly if you wish.
 I assume you know that there are already such versions, and that they
 are using upper case, but that you had some special reason for changing?
 e.g. version(X86) version(PPC)

 Not sure if you wanted Phobos/D to be a) updated or b) rewritten ?
I wants don't get ;-), but it would be nice for it to be re-arranged to a more carefully organised structure/namespace. I don't necessarily see dramatic changes either. Actually Darwin was first on my list... OK, I'll write that awesome h2d tool then so we can create the bindings with ease. -- Alan West
Aug 11 2005
next sibling parent reply =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Alan West wrote:

 When looking at the possibilities try not to think about current 
 limitations, but envisage the D language being right up there at the 
 number one spot, used by many in the industry, taught in 
 college/universities etc... Widely available on many different 
 platforms. It would be cool to be able to set in stone now early the 
 structure for people to follow in implementing bindings to those 
 systems, whether it is a mobile phone, or super computer. It would allow 
 Ds evolution to run much more smoothly, with happier users and developers.
I'm all for improvements, just hope that D will be released ever. :-P So my thoughts were based on what is currently available or at least with minor tweaks, didn't mean to hold back any speculative discussion on what could happen in the future or by re-writing... As usual around here, any major changes will be up to Walter anyway. --anders
Aug 11 2005
parent Alan West <alan alanz.com> writes:
Anders F Björklund wrote:

 I'm all for improvements, just hope that D will be released ever. :-P
 
 So my thoughts were based on what is currently available or at
 least with minor tweaks, didn't mean to hold back any speculative
 discussion on what could happen in the future or by re-writing...
 
 As usual around here, any major changes will be up to Walter anyway.
Yeah I had heard of D and read over the specification quite a few years ago when it was Windows only. Walter's Linux port didn't really grab me because there are still many other systems and architectures out there. Then recently I discovered David Friedman's D front end for GCC, which did grab me. I'm now in a position to devote about 20 solid hours a week on helping to standardize the D library for many platforms. Though first, I want to automate as much of that task as possible with tools written in D.
Aug 13 2005
prev sibling parent Alan West <alan alanz.com> writes:
Alan West wrote:
 Actually Darwin was first on my list... OK, I'll write that awesome h2d 
 tool then so we can create the bindings with ease.
Since that statement, I've nearly completed D language output support in The Ragel State Machine Compiler (http://www.elude.ca/ragel/), the maintainer of that project has agreed to include it as a standard part of the project. I hope to use this to aid in creating that awesome h2d tool. It could also help produce a tool for generating D specific source documentation in XML maybe even XMI for output via XSLT in any format.
Aug 13 2005
prev sibling parent reply Sean Kelly <sean f4.ca> writes:
In article <ddfkcr$3ct$1 digitaldaemon.com>,
=?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= says...
Alan West wrote:

 When I started getting into D I was confused by the versions used for 
 different operating systems. Since looking at a lot of D code in phobos 
 and other projects, I've spent some time working out what we attempt to 
 accomplish with the versions Windows, Linux, Posix, as to me they seemed 
 to be missing the target.
This has been debated for ages, and the surviving ones are: version(Windows) and version(Unix). DMD still has the limited "linux" only, just as GDC does have a specific version saying what the current platform `uname` is. It seems you were proposing some brand new hierarchy, but anyway. Cleaning up the std.c portion (as in Ares) would be a good start ? http://www.dsource.org/forums/viewforum.php?f=31 (C99 changes)
Outside of std.c, I think the only surviving OS versions in Ares are 'Windows' and 'Posix'. The std.c.posix modules are quite incomplete, but this is one of the few locations where versions such as 'linux' or 'Darwin' might have to persist, since each OS can implement most of the Posix data types however it wants to. If someone wants to work on the std.c.posix headers, I would be very grateful. Or if someone is willing to zip up the necessary C headers and just email them to me. These modules have been largely ignored mostly because I don't have a Linux install to refer to. Sean
Aug 11 2005
parent reply =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Sean Kelly wrote:

 Outside of std.c, I think the only surviving OS versions in Ares are 'Windows'
 and 'Posix'.  The std.c.posix modules are quite incomplete, but this is one of
 the few locations where versions such as 'linux' or 'Darwin' might have to
 persist, since each OS can implement most of the Posix data types however it
 wants to.  If someone wants to work on the std.c.posix headers, I would be very
 grateful.  Or if someone is willing to zip up the necessary C headers and just
 email them to me.  These modules have been largely ignored mostly because I
 don't have a Linux install to refer to.
Oh, so Ares picked "Posix" ? That's too bad, since GDC has "Unix". And yes there are a couple of places when version(linux) and version(darwin) et. al. are still needed, like where there are actual differences between the platforms. But for the user code, most of the time it's enough to differ between Windows and Unix. It would be nice if those two could be kept to a minimum as well, but then again D isn't Java and doesn't invent a "virtual" target. --anders
Aug 11 2005
parent reply Sean Kelly <sean f4.ca> writes:
In article <ddg86l$rjd$1 digitaldaemon.com>,
=?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= says...
Oh, so Ares picked "Posix" ? That's too bad, since GDC has "Unix".
I just checked, and I use both "Posix" and "linux" in the headers. I would change "Posix" to "Unix" if that's the accepted identifier. And it looks like some of the version blocks may need to be restructured anyway, as there are a bunch of places in "linux" blocks that should probably be Posix/Unix. Next on my list was to revisit the threading code, so I'll need to be messing with the Posix/Unix headers soon anyway.
And yes there are a couple of places when version(linux) and 
version(darwin) et. al. are still needed, like where there are
actual differences between the platforms. But for the user code,
most of the time it's enough to differ between Windows and Unix.

It would be nice if those two could be kept to a minimum as well,
but then again D isn't Java and doesn't invent a "virtual" target.
If we target library names (for data strucutres like pthread_t) then there should be few if any version blocks which need to target a specific OS--really just the stuff in sys.whatever, to use the suggested layout. Sean
Aug 11 2005
parent reply =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Sean Kelly wrote:

 I just checked, and I use both "Posix" and "linux" in the headers.  I would
 change "Posix" to "Unix" if that's the accepted identifier. 
I don't think there ever was any universally accepted identifier (consensus), but version(Unix) is pre-defined in the GDC compiler. I think Mango uses version(Posix) as well and that Build defines it. version(linux) is also available, but only on the Linux platforms. http://www.digitalmars.com/d/version.html still ignores the issue. (see http://www.prowiki.org/wiki4d/wiki.cgi?DocComments/Version) --anders
Aug 12 2005
parent reply Sean Kelly <sean f4.ca> writes:
In article <ddhtk8$ne$1 digitaldaemon.com>,
=?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= says...
Sean Kelly wrote:

 I just checked, and I use both "Posix" and "linux" in the headers.  I would
 change "Posix" to "Unix" if that's the accepted identifier. 
I don't think there ever was any universally accepted identifier (consensus), but version(Unix) is pre-defined in the GDC compiler. I think Mango uses version(Posix) as well and that Build defines it. version(linux) is also available, but only on the Linux platforms.
I've given it some thought, and I'm going to keep 'Posix' as it's the name of the API rather than a class of operating systems. Switching to 'Unix' would imply to me that the files should be in sys.unix rather than std.c.unix, and I think this stuff belongs in std.c. I'll add 'Posix' as a version identifier in the makefiles. Also, I'm cleaning up the headers (and moving some stuff from std.c.signal to std.c.posix.signal in the process). Sean
Aug 12 2005
parent reply Alan West <alan alanz.com> writes:
Sean Kelly wrote:
 
 I've given it some thought, and I'm going to keep 'Posix' as it's the name of
 the API rather than a class of operating systems.  Switching to 'Unix' would
 imply to me that the files should be in sys.unix rather than std.c.unix, and I
 think this stuff belongs in std.c.  I'll add 'Posix' as a version identifier in
 the makefiles.  Also, I'm cleaning up the headers (and moving some stuff from
 std.c.signal to std.c.posix.signal in the process).
Posix is the name of older APIs, of which there are a number of versions. I did at first name that section myself after posix, but after looking into it more at http://www.unix.org/ I felt unix was more appropriate as the Single Unix Specification v3 is the latest standard. Quote from http://www.unix.org/what_is_unix.html: "Today, the definition of UNIX ® takes the form of the worldwide Single UNIX Specification integrating X/Open Company's XPG4, IEEE's POSIX Standards and ISO C. Through continual evolution, the Single UNIX Specification is the defacto and dejure standard definition for the UNIX system application programming interfaces." I also found that the documentation for older IEEE POSIX standard was very expensive to purchase. So if D was to correctly implement POSIX interfaces someone would need that documentation. Whereas the Single Unix Specification is freely available from http://www.unix.org/ .
Aug 13 2005
parent John Reimer <terminal.node gmail.com> writes:
Alan West wrote:
 Sean Kelly wrote:
 
 I've given it some thought, and I'm going to keep 'Posix' as it's the 
 name of
 the API rather than a class of operating systems.  Switching to 'Unix' 
 would
 imply to me that the files should be in sys.unix rather than 
 std.c.unix, and I
 think this stuff belongs in std.c.  I'll add 'Posix' as a version 
 identifier in
 the makefiles.  Also, I'm cleaning up the headers (and moving some 
 stuff from
 std.c.signal to std.c.posix.signal in the process).
Posix is the name of older APIs, of which there are a number of versions. I did at first name that section myself after posix, but after looking into it more at http://www.unix.org/ I felt unix was more appropriate as the Single Unix Specification v3 is the latest standard. Quote from http://www.unix.org/what_is_unix.html: "Today, the definition of UNIX ® takes the form of the worldwide Single UNIX Specification integrating X/Open Company's XPG4, IEEE's POSIX Standards and ISO C. Through continual evolution, the Single UNIX Specification is the defacto and dejure standard definition for the UNIX system application programming interfaces." I also found that the documentation for older IEEE POSIX standard was very expensive to purchase. So if D was to correctly implement POSIX interfaces someone would need that documentation. Whereas the Single Unix Specification is freely available from http://www.unix.org/ .
I have to admit: your evidence is convincing. I was originally convinced that "Posix" was the way to go. But perhaps "Unix" is the safer bet afterall. -JJR
Aug 14 2005
prev sibling next sibling parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
[snip]
 I think the following package hierarchy should be made available 
 containing modules that bind to APIs that are operating system specific, 
 as delivered by their vendor or creator, which relate more with version()s 
 currently being used:

     sys.windows  (sys specific modules like: )
       kernel
       user
       gdi
       winspool
       comdlg
       advapi
       comctl
       shell
       ...more
     sys.linux  (for direct syscalls not exposed in standard C lib)
     sys.mach   (modules for the mach micro-kernel in macosx etc)
[snip] I like your suggested "sys" prefix instead of the "std.c" prefix for Windows APIs since those APIs aren't part of the standard C or D API. Why should D consider the platform-specific API as part of the same "std" namespace as its platform-independent APIs. It would be like putting the Windows API inside the C++ std namespace in C++ - which would be wierd. A D compiler on a non-Windows OS shouldn't have to include the std.c.windows module but that would break the implicit rule that "the std package is part of all D implemantations".
Aug 11 2005
parent Vathix <chris dprogramming.com> writes:
On Thu, 11 Aug 2005 11:27:29 -0400, Ben Hinkle <bhinkle mathworks.com>  
wrote:

 [snip]
 I think the following package hierarchy should be made available
 containing modules that bind to APIs that are operating system specific,
 as delivered by their vendor or creator, which relate more with  
 version()s
 currently being used:

     sys.windows  (sys specific modules like: )
       kernel
       user
       gdi
       winspool
       comdlg
       advapi
       comctl
       shell
       ...more
     sys.linux  (for direct syscalls not exposed in standard C lib)
     sys.mach   (modules for the mach micro-kernel in macosx etc)
[snip] I like your suggested "sys" prefix instead of the "std.c" prefix for Windows APIs since those APIs aren't part of the standard C or D API. Why should D consider the platform-specific API as part of the same "std" namespace as its platform-independent APIs. It would be like putting the Windows API inside the C++ std namespace in C++ - which would be wierd. A D compiler on a non-Windows OS shouldn't have to include the std.c.windows module but that would break the implicit rule that "the std package is part of all D implemantations".
I was thinking the same thing, but sys.c in case a sys ever decides to have D API ;)
Aug 11 2005
prev sibling next sibling parent Sean Kelly <sean f4.ca> writes:
In article <ddeeu6$1rmn$1 digitaldaemon.com>, Alan West says...
When I started getting into D I was confused by the versions used for 
different operating systems. Since looking at a lot of D code in phobos 
and other projects, I've spent some time working out what we attempt to 
accomplish with the versions Windows, Linux, Posix, as to me they seemed 
to be missing the target.
You make a lot of good points. I think some of this may be a bit more granular than D needs, but it's on the right track for the most part. If you have any specific suggestions or want to submit changes, please do so :) Sean
Aug 11 2005
prev sibling parent Sean Kelly <sean f4.ca> writes:
In article <ddeeu6$1rmn$1 digitaldaemon.com>, Alan West says...
When I started getting into D I was confused by the versions used for 
different operating systems. Since looking at a lot of D code in phobos 
and other projects, I've spent some time working out what we attempt to 
accomplish with the versions Windows, Linux, Posix, as to me they seemed 
to be missing the target.
By the way, the Ares C headers are here: http://svn.dsource.org/projects/ares/trunk/src/ares/std/c/ The Posix stuff is very much unfinished (mostly because I don't have a Linux install to reference), and I've left std.c.linux as it is in Phobos for now--it was going to be changes to OS-specific stuff only once the Posix headers are sufficiently complete. If you want to submit changes or just email me C header files I'd very much appreciate it. My main interest with the Posix stuff is currently to just take care of popular utility functionality plus everything needed for threading/synchronization, but I'll take what I can get :) And as Ben said, the sys prefix for OS headers makes a lot of sense. I'll definately consider changing the file layout in the near future. Sean
Aug 11 2005