www.digitalmars.com         C & C++   DMDScript  

D - Dernel: Protected mode far pointers in D?

reply "=?iso-8859-1?Q?Robert_M._M=FCnch?=" <robert.muench robertmuench.de> writes:
Hi, I found this quote:

	A disadvantage of all the mainstream IA-32 compilers is that they assume  
a flat 32-bit address space, with CS, DS, ES and SS each having the same  
base address. Because they don't use far (48-bit seg16:ofs32) pointers  
they make it a lot harder to write code for a segmented OS. Programs on a  
segmented OS written in C would have to rely on a lot of assembly code and  
would be a lot less efficient than programs written for a flat OS.  
However, at the time of writing, Watcom are still promising to release  
their OpenWatcom compiler, which will supposedly support far pointers in  
32-bit protected mode.

Can the D compiler use protected mode far pointers? Is it an issue to care  
about at all?

-- 
Robert M. Münch
Management & IT Freelancer
http://www.robertmuench.de
Mar 02 2004
next sibling parent reply "Jeroen van Bemmel" <someone somewhere.com> writes:
 Can the D compiler use protected mode far pointers? Is it an issue to care
 about at all?


How many "segmented OSs" can you list?
Mar 03 2004
parent reply "=?iso-8859-1?Q?Robert_M._M=FCnch?=" <robert.muench robertmuench.de> writes:
On Wed, 3 Mar 2004 21:06:25 +0100, Jeroen van Bemmel  
<someone somewhere.com> wrote:


 How many "segmented OSs" can you list?


Hmm... I don't know. I tried google but didn't found anything. However,  
I'm not that a big fan of x86 (this is mostly due being an old Amigian  
with 68x assembler know-how) but x86 is pretty advanced these days and  
using segmentation seems to be a valuable option.

-- 
Robert M. Münch
Management & IT Freelancer
http://www.robertmuench.de
Mar 05 2004
parent reply "Walter" <walter digitalmars.com> writes:
"Robert M. Münch" <robert.muench robertmuench.de> wrote in message
news:opr4d2mbseheztw6 news.digitalmars.com...

 On Wed, 3 Mar 2004 21:06:25 +0100, Jeroen van Bemmel
 <someone somewhere.com> wrote:


 How many "segmented OSs" can you list?


 Hmm... I don't know. I tried google but didn't found anything. However,
 I'm not that a big fan of x86 (this is mostly due being an old Amigian
 with 68x assembler know-how) but x86 is pretty advanced these days and
 using segmentation seems to be a valuable option.


It would be a differentiating feature! But it'd be crippled by the lack of
tools for writing segmented 32 bit code. Instead, how about these features:

1) The x86 CPU's have 4 privilege levels. These are unused by windows, and I
think linux too, which runs everything at the max privilege level (or they
only use 2 of the 4 levels, I forgot the details). This is why security is
so difficult on windows/linux, as everything is essentially root. Using the
4 levels enables you to create a *hardware* sandbox for running, for
example, internet apps. This is inherently much more robust than software
sandboxes such as Java's vm or .net.

2) Expose the full functionality of the virtual paging hardware to
application programming. This will enable better/faster garbage collectors
to be written. For example, the gc could read the 'dirty' bit on a page to
see if it needs to be scanned. In fact, gc should be part of the os core
functionality, so that all gc'd languages can have access to it.
Mar 05 2004
next sibling parent "=?iso-8859-1?Q?Robert_M._M=FCnch?=" <robert.muench robertmuench.de> writes:
On Fri, 5 Mar 2004 18:52:17 -0800, Walter <walter digitalmars.com> wrote:


 It would be a differentiating feature! But it'd be crippled by the lack  
 of tools for writing segmented 32 bit code.


Hi, yes right, that's the impression I get too. IMO for user applications  
it doesn't make any sense to muck around with this x86 feature. But using  
such features only on the kernel level and making it transparent to  
applications might be an option. I'm going to investigate this further.

\justification

Why to care anyhow? As I undestand current approaches, he problem is that  
there is no 4GB of memory available to user applications. That's mostly  
because the kernel needs to map in the address space as well. Hence the  
well known 1GB/3GB or 2GB/2GB slice. So, getting as much memory as  
possible in the fastest way is something that applications will benefit  
from.

/justification


 1) The x86 CPU's have 4 privilege levels.


Yes, that's a nice feature being available in hardware.


 These are unused by windows, and I think linux too,which runs everything  
 at the max privilege level (or they only use 2 of the 4 levels, I forgot  
 the details).


Aha, I didn't knew this. Interesting... I expected that this feature is  
used.


 This is why security is
 so difficult on windows/linux,
 as everything is essentially root.


Do these use the x86 protection features for segments or are those checks  
done within a some kernel internal code?


 Using the 4 levels enables you to create a *hardware* sandbox for  
 running, for
 example, internet apps. This is inherently much more robust than software
 sandboxes such as Java's vm or .net.


Yes, good point.


 2) Expose the full functionality of the virtual paging hardware to
 application programming. This will enable better/faster garbage  
 collectors to be written. For example, the gc could read the 'dirty' bit  
 on a page to see if it needs to be scanned. In fact, gc should be part  
 of the os core
 functionality, so that all gc'd languages can have access to it.


The latter option is the one I'm interested in. With the OSs I know, you  
now have several layers of memory-management, starting from the virtual  
paging hardware, thru the OS, up to application level. My idea is to get  
the GC as much down to the metal as possible and than provide very simple,  
fast and lean interface to it. Even C/C++ code can have an implementation  
of memory functions using the GC than.

Thanks for the ideas and the information.

-- 
Robert M. Münch
Management & IT Freelancer
http://www.robertmuench.de
Mar 06 2004
prev sibling parent Scott Wood <scott buserror.net> writes:
On Fri, 5 Mar 2004 18:52:17 -0800, Walter <walter digitalmars.com> wrote:

 1) The x86 CPU's have 4 privilege levels. These are unused by
 windows, and I think linux too, which runs everything at the max
 privilege level (or they only use 2 of the 4 levels, I forgot the
 details).


They use two levels (which is all that paging provides on x86), 0 and
3.


 This is why security is so difficult on windows/linux, as
 everything is essentially root. Using the 4 levels enables you to
 create a *hardware* sandbox for running, for example, internet
 apps. 


You can do that with two levels as well.  The problem on Linux is not
that everything's root (as that's not the case), but that the
security model it inherited from Unix is rather primitive, causing a
lot of things to have more privilege than they need.

I'm not familiar enough with Windows to comment on its security model
(for those versions that have one (i.e. NT and derivatives)).


 2) Expose the full functionality of the virtual paging hardware to
 application programming. This will enable better/faster garbage collectors
 to be written. For example, the gc could read the 'dirty' bit on a page to
 see if it needs to be scanned. 


That'd be nice.  The ability to request arbitrary copy-on-write
regions (as opposed to only being able to use it with fork() and
inter-process mmap()) would be great as well, both for doing GC in
the background and for non-GC-related things.


 In fact, gc should be part of the os core
 functionality, so that all gc'd languages can have access to it.


Though there should be a way of hooking language-specific GC code
into that, which can know more about how the data is laid out.

-Scott
Mar 06 2004
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Robert M. Münch" <robert.muench robertmuench.de> wrote in message
news:opr39xslc8heztw6 news.digitalmars.com...

 Hi, I found this quote:

 A disadvantage of all the mainstream IA-32 compilers is that they assume
 a flat 32-bit address space, with CS, DS, ES and SS each having the same
 base address. Because they don't use far (48-bit seg16:ofs32) pointers
 they make it a lot harder to write code for a segmented OS. Programs on a
 segmented OS written in C would have to rely on a lot of assembly code and
 would be a lot less efficient than programs written for a flat OS.
 However, at the time of writing, Watcom are still promising to release
 their OpenWatcom compiler, which will supposedly support far pointers in
 32-bit protected mode.

 Can the D compiler use protected mode far pointers?


No. I know all about protected mode far pointers in the 16 bit compiler and
the grief it causes.


 Is it an issue to care
 about at all?


No. <g> Nobody has ever asked for this capability for 32 bit code generation
for 15 years. Writing one would imply one needed a bigger memory space for a
program than 2 gigs. But these days, if you need that, you'd be better off
programming for the new 64 bit chips.
Mar 04 2004
parent reply "=?iso-8859-1?Q?Robert_M._M=FCnch?=" <robert.muench robertmuench.de> writes:
On Thu, 4 Mar 2004 10:54:10 -0800, Walter <walter digitalmars.com> wrote:


 No. I know all about protected mode far pointers in the 16 bit compiler  
 and the grief it causes.


Hi, I can remember some parts too.


 No. <g> Nobody has ever asked for this capability for 32 bit code  
 generation for 15 years. Writing one would imply one needed a bigger  
 memory space for a program than 2 gigs.


Which happens these days quite fast... at least in enterprise applications.


 But these days, if you need that, you'd be better off
 programming for the new 64 bit chips.


Are these than supported by the D compiler? ;-)

-- 
Robert M. Münch
Management & IT Freelancer
http://www.robertmuench.de
Mar 05 2004
next sibling parent Scott Wood <scott buserror.net> writes:
On Fri, 05 Mar 2004 13:43:15 +0100, Robert M. Münch
<robert.muench robertmuench.de> wrote:

 On Thu, 4 Mar 2004 10:54:10 -0800, Walter <walter digitalmars.com> wrote:

 No. <g> Nobody has ever asked for this capability for 32 bit code  
 generation for 15 years. Writing one would imply one needed a bigger  
 memory space for a program than 2 gigs.


 
 Which happens these days quite fast... at least in enterprise
 applications.


Segmentation won't help there, though.  The result of a base+offset
calculation is still a 32-bit virtual address.

32-bit segmentation (beyond basic fixed values for CS and DS/ES/SS)
is mainly useful for the transparent relocation/growth of pointers,
but it's really not worth it IMHO.  Even with compiler support,
manipulating segments (and storing the larger pointers) is going to
cause significant overhead and lots of programmer headaches.

-Scott
Mar 05 2004
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Robert M. Münch" <robert.muench robertmuench.de> wrote in message
news:opr4d2ydf0heztw6 news.digitalmars.com...

 But these days, if you need that, you'd be better off
 programming for the new 64 bit chips.


 Are these than supported by the D compiler? ;-)


Not yet, I don't even have a 64 bit machine. But that's where things are
going.
Mar 05 2004
parent Mark T <Mark_member pathlink.com> writes:
In article <c2be3f$ul1$2 digitaldaemon.com>, Walter says...

"Robert M. Münch" <robert.muench robertmuench.de> wrote in message
news:opr4d2ydf0heztw6 news.digitalmars.com...

 But these days, if you need that, you'd be better off
 programming for the new 64 bit chips.


 Are these than supported by the D compiler? ;-)


Not yet, I don't even have a 64 bit machine. But that's where things are
going.


not in the embedded world, we just moved to 32 bits :)
Mar 06 2004