www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - HANDLE and GC

reply pinzo gmail.com writes:
Hello all.
I have just read through the docs and found this statements regarding th=
e  =

GC:
  * Do not store into pointers values that may point into the garbage  =

collected heap.
  * Do not store magic values into pointers, other than null.

But now, looking at std.c.windows.windows I noticed that Win32 HANDLEs  =

(that is HANDLE, HWND, HFILE, etc) are defined:

alias void* HANDLE;

Since HANDLEs are just arbitrary values, and usually don't even point to=
  =

user memory, won't that raise undefined behavior.

If it's true, I think that just changing the definition to something lik=
e:

typedef size_t HANDLE;

would do the trick, since size_t should have the same size and alignment=
  =

requirements that a pointer.

What do you think?
Sep 01 2005
next sibling parent reply Chris Sauls <ibisbasenji gmail.com> writes:
pinzo gmail.com wrote:

 Hello all.
 I have just read through the docs and found this statements regarding 
 the  GC:
  * Do not store into pointers values that may point into the garbage  
 collected heap.
  * Do not store magic values into pointers, other than null.
 
 But now, looking at std.c.windows.windows I noticed that Win32 HANDLEs  
 (that is HANDLE, HWND, HFILE, etc) are defined:
 
 alias void* HANDLE;
 
 Since HANDLEs are just arbitrary values, and usually don't even point 
 to  user memory, won't that raise undefined behavior.


Presumably, going by the spec docs you quoted.


 If it's true, I think that just changing the definition to something like:
 
 typedef size_t HANDLE;
 
 would do the trick, since size_t should have the same size and 
 alignment  requirements that a pointer.
 
 What do you think?


I think you've got the right idea, except I might debate the merits of using
'typedef' 
over 'alias' in this case, and I'd probably use 'ptrdiff_t' rather than
'size_t'.

-- Chris Sauls
Sep 01 2005
parent pinzo gmail.com writes:
On Thu, 01 Sep 2005 21:22:30 +0200, Chris Sauls <ibisbasenji gmail.com> =
 =

wrote:


 I think you've got the right idea, except I might debate the merits of=


  =


 using 'typedef' over 'alias' in this case, and I'd probably use  =



 'ptrdiff_t' rather than 'size_t'.


Well, I said size_t just because is more familiar. Anyway it's guarantee=
d  =

that they are
the same size, isn't it? And the size is what matters...

About 'typedef' or 'alias', errr well, the whole point of typedef is to =
 =

get type-safeness,
so the following get a compiler error:

typedef size_t HWND;
int n;
ShowWindow(n, SW_SHOW) // error cannot convert int to HANDLE.

Curiously enough, using a constant compiles fine:

ShowWindow(1729, SW_SHOW) // OK (?)

That's great for the 0, and not so great for other numbers.

But the true advantage of typedef over alias comes when you have 20  =

different types of handles,
all of them with a lot of functions incompatibles between them (HWND,  =

HANDLE, HGLOBAL, HINSTANCE, HGDIOBJ, HKEY...)
It's too easy to get messed up, and call DestroyObject with a HANDLE, fo=
r  =

example. typedef would
mark this as an error.
Sep 01 2005
prev sibling parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
Storing pointers to non-GC data is fine (ie malloc'ed data or HANDLES). What 
you don't want to do is store an arbitrary value into a pointer type since 
it might accidentally point to some GC region.

<pinzo gmail.com> wrote in message news:op.swfmpvdn0r3bey pinzopc2...
Hello all.
I have just read through the docs and found this statements regarding the
GC:
  * Do not store into pointers values that may point into the garbage
collected heap.
  * Do not store magic values into pointers, other than null.
Sep 01 2005
parent reply pinzo gmail.com writes:
On Thu, 01 Sep 2005 22:37:56 +0200, Ben Hinkle <bhinkle mathworks.com>  =

wrote:


 Storing pointers to non-GC data is fine (ie malloc'ed data or HANDLES)=


.  =


 What
 you don't want to do is store an arbitrary value into a pointer type  =



 since
 it might accidentally point to some GC region.


The problem here is that HANDLE values, in general, are not pointers! Th=
ey  =

are arbitrary values
cast to (void*) (well, maybe pointers from kernel space or from CSRSS  =

space, etc.)
So they could point just anywhere, including GC-data.
Sep 01 2005
next sibling parent reply Derek Parnell <derek psych.ward> writes:
On Thu, 01 Sep 2005 23:26:16 +0200, pinzo gmail.com wrote:


 On Thu, 01 Sep 2005 22:37:56 +0200, Ben Hinkle <bhinkle mathworks.com>  
 wrote:
 

 Storing pointers to non-GC data is fine (ie malloc'ed data or HANDLES).  
 What
 you don't want to do is store an arbitrary value into a pointer type  
 since
 it might accidentally point to some GC region.


 
 The problem here is that HANDLE values, in general, are not pointers! They  
 are arbitrary values
 cast to (void*) (well, maybe pointers from kernel space or from CSRSS  
 space, etc.)
 So they could point just anywhere, including GC-data.


Actually it turns out that the term "handle" can refer to at least two
types of data items. One, as you say, is a theoretically arbitrary value
but is usually an index into any array, and the other is a pointer to a
pointer.

In many Windows API and Macintosh contexts, a handle is a pointer to a
pointer. The idea being that applications use the handle to deference the
target data and that the operating system is free to move the target data
about. However, in any Unix contexts, a handle is an array index. In both
cases though, a handle is an indirect reference to data.

-- 
Derek Parnell
Melbourne, Australia
2/09/2005 8:08:27 AM
Sep 01 2005
parent reply pinzo <pinzo_member pathlink.com> writes:
In article <1ri252j8s1xuw.1in47u3z42esx.dlg 40tude.net>, Derek Parnell says...



Actually it turns out that the term "handle" can refer to at least two
types of data items. One, as you say, is a theoretically arbitrary value
but is usually an index into any array, and the other is a pointer to a
pointer.


I think your definition of "handle" is too restrictive. A "handle" is just an
'opaque' value used to reference an object, in the context of a library.
A handle can be 'implemented' as a pointer, a pointer to a pointer, an index
into an array, or any other way you can imagine. A different thing is how the
handle value is seen by the programmer.
Win32 handles are declared as (void*), but the fact is that it isn't documented
how exactly they are implemented.
Of course, we can guess, I have done some investigation :-)
* Some are direct pointers: HMENU, HACCEL, HINSTANCE, HANDLE from FindFirstFile
* Some are pointers to pointers: HGLOBAL
* Some are indexes into an array: SOCKET?
* Some are pointers into the address space of 'another' process: HWND
* Some are pointers into the kernel space: HANDLE of kernel objects

What I am trying to say is that the last two of them are *not* valid pointers in
the current address space, even when they are cast to void*. And so they can
theoretically trash the GC.
Of course, don't trust the list above if you want to be minimally portable,
because this is totally undocumented. 
So, to be in the safe side, just say no to pointer handles.



 Sep 01 2005



 next sibling parent  "Ben Hinkle" <ben.hinkle gmail.com>  writes:


 * Some are direct pointers: HMENU, HACCEL, HINSTANCE, HANDLE from 
 FindFirstFile
 * Some are pointers to pointers: HGLOBAL
 * Some are indexes into an array: SOCKET?
 * Some are pointers into the address space of 'another' process: HWND
 * Some are pointers into the kernel space: HANDLE of kernel objects

 What I am trying to say is that the last two of them are *not* valid 
 pointers in
 the current address space, even when they are cast to void*. And so they 
 can
 theoretically trash the GC.


Only invalid pointers into the GC regions can "trash" the GC (I put quotes 
around trash because the GC won't get trashed but your fake pointer might). 
The GC knows what memory regions it owns and if a pointer points to 
something outside its control it ignores it. So pointing out of the address 
space is ok - as are values like (HANDLE)-1 that will never point to a valid 
GC region.


 Of course, don't trust the list above if you want to be minimally 
 portable,
 because this is totally undocumented.
 So, to be in the safe side, just say no to pointer handles.





 Sep 02 2005



prev sibling next sibling parent reply Mike Parker <aldacron71 yahoo.com>  writes:


pinzo wrote:



 What I am trying to say is that the last two of them are *not* valid pointers
in
 the current address space, even when they are cast to void*. And so they can
 theoretically trash the GC.


Since they are not GC-allocated resources, how are they going to trash 
the DC? From the GC docs:

"Pointers in D can be broadly divided into two categories: those that 
point to garbage collected memory, and those that do not. Examples of 
the latter are pointers created by calls to C's malloc(), pointers 
received from C library routines, pointers to static data, pointers to 
objects on the stack, etc. For those pointers, anything that is legal in 
C can be done with them."

The two points you began the thread with apply to GCed pointers, which 
Win32 HANDLES are not.



 Sep 02 2005



  parent reply Sean Kelly <sean f4.ca>  writes:


In article <df9e92$e1a$1 digitaldaemon.com>, Mike Parker says...

pinzo wrote:


 What I am trying to say is that the last two of them are *not* valid pointers
in
 the current address space, even when they are cast to void*. And so they can
 theoretically trash the GC.


Since they are not GC-allocated resources, how are they going to trash 
the DC? From the GC docs:

"Pointers in D can be broadly divided into two categories: those that 
point to garbage collected memory, and those that do not. Examples of 
the latter are pointers created by calls to C's malloc(), pointers 
received from C library routines, pointers to static data, pointers to 
objects on the stack, etc. For those pointers, anything that is legal in 
C can be done with them."

The two points you began the thread with apply to GCed pointers, which 
Win32 HANDLES are not.


With the current mark/scan implementation, the worst that could happen is memory
not being recycled when it should be (because the GC mistakes a handle value for
a pointer into GC memory).  With a generational (ie. copying) GC, the handle
value could be changed if it happens to coincide with the address of a valid GC
memory block.


Sean



 Sep 02 2005



  parent reply rodrigorivascosta gmail.com  writes:


On Sat, 03 Sep 2005 02:03:43 +0200, Sean Kelly <sean f4.ca> wrote:


 In article <df9e92$e1a$1 digitaldaemon.com>, Mike Parker says...
 With the current mark/scan implementation, the worst that could happen=


  =


 is memory
 not being recycled when it should be (because the GC mistakes a handle=


  =


 value for
 a pointer into GC memory).  With a generational (ie. copying) GC, the =


 =


 handle
 value could be changed if it happens to coincide with the address of a=


  =


 valid GC
 memory block.


Well, to be on the safe side of the life, I strongly suggest to change a=
ll  =

handle
types to size_t or similar, the cost of this change is practically zero,=
  =

and
the potential problems and incompatibilities that can arise with this or=
  =

future
GC implementations are too great.



 Sep 04 2005



  parent  pinzo correo.nu  writes:


On Sun, 04 Sep 2005 23:30:42 +0200, <rodrigorivascosta gmail.com> wrote:=



 Well, to be on the safe side of the life, I strongly suggest to change=


  =


 all handle
 types to size_t or similar, the cost of this change is practically zer=


o,  =


 and
 the potential problems and incompatibilities that can arise with this =


or  =


 future
 GC implementations are too great.


I have just messed up my newsreader program. That was myself, of course.=


---

Pinzo.



 Sep 04 2005



prev sibling  parent  "Walter Bright" <newshound digitalmars.com>  writes:


"pinzo" <pinzo_member pathlink.com> wrote in message
news:df8pve$2nfe$1 digitaldaemon.com...

 What I am trying to say is that the last two of them are *not* valid


pointers in

 the current address space, even when they are cast to void*.


That's true.


 And so they can
 theoretically trash the GC.


In the case of Windows handles, that's not likely because the only values
that can trash the GC are values that happen to fall within range of the
GC's allocated memory pool. The non-pointer values I've seen in Windows
handles are all less than 64K, which is never, by design, memory allocated
to a process, hence cannot point into the GC pool.


 Of course, don't trust the list above if you want to be minimally


portable,

 because this is totally undocumented.
 So, to be in the safe side, just say no to pointer handles.


For any new design, that is correct. You can do a legacy special case
exception for Windows handles, though.



 Sep 08 2005



prev sibling  parent reply Shammah Chancellor <Shammah_member pathlink.com>  writes:


In article <op.swf0h2q40r3bey pinzopc2>, pinzo gmail.com says...

On Thu, 01 Sep 2005 22:37:56 +0200, Ben Hinkle <bhinkle mathworks.com>  =

wrote:


 Storing pointers to non-GC data is fine (ie malloc'ed data or HANDLES)=


.  =


 What
 you don't want to do is store an arbitrary value into a pointer type  =



 since
 it might accidentally point to some GC region.


The problem here is that HANDLE values, in general, are not pointers! Th=
ey  =

are arbitrary values
cast to (void*) (well, maybe pointers from kernel space or from CSRSS  =

space, etc.)
So they could point just anywhere, including GC-data.


The definition of a "handle" is a pointer to a pointer.

-Sha



 Sep 01 2005



  parent reply "Ben Hinkle" <ben.hinkle gmail.com>  writes:


"Shammah Chancellor" <Shammah_member pathlink.com> wrote in message 
news:df878o$251n$1 digitaldaemon.com...

 In article <op.swf0h2q40r3bey pinzopc2>, pinzo gmail.com says...

On Thu, 01 Sep 2005 22:37:56 +0200, Ben Hinkle <bhinkle mathworks.com>  =

wrote:


 Storing pointers to non-GC data is fine (ie malloc'ed data or HANDLES)=


.  =


 What
 you don't want to do is store an arbitrary value into a pointer type  =



 since
 it might accidentally point to some GC region.


The problem here is that HANDLE values, in general, are not pointers! Th=
ey  =

are arbitrary values
cast to (void*) (well, maybe pointers from kernel space or from CSRSS  =

space, etc.)
So they could point just anywhere, including GC-data.


 The definition of a "handle" is a pointer to a pointer.


In Win32 API HANDLE is void* but there are places where things like -1 are 
cast to HANDLE. I'm not aware of what non-pointers are cast to HANDLES but 
I've never looked into it. The "pointer to pointer" is what Macs used ages 
ago and from what I understand the Mac OS X doesn't use handles anymore. In 
MATLAB, ironically, a handle is a double :-) That's because back when MATLAB 
first got graphics objects (called Handle Graphics) the double was the only 
datatype in MATLAB. It has advantages and disadvantages but I'm mentioning 
it just for those trivia-minded folks out there who are interested in uses 
of the word "handle".


 -Sha

 





 Sep 01 2005



 next sibling parent reply Derek Parnell <derek psych.ward>  writes:


On Thu, 1 Sep 2005 20:56:51 -0400, Ben Hinkle wrote:

[snip]
 

 In Win32 API HANDLE is void* but there are places where things like -1 are 
 cast to HANDLE. I'm not aware of what non-pointers are cast to HANDLES but 
 I've never looked into it.


<trivia>
Windows API assumes that no valid pointer will ever point anywhere in the
range of addresses 0x00000000 to 0x0000001F, and 0xFFFFFFFD to 0xFFFFFFFF.
And thus it allows Handles to contain integer values in the range -3 to 31
inclusive. These are usually reserved values for various things, such as
system defined Brushes, Fonts, etc... and error codes.
</trivia>

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
2/09/2005 11:35:49 AM



 Sep 01 2005



  parent  pinzo <pinzo_member pathlink.com>  writes:


In article <keoxw1cdrwjs.2hi280n7hudh.dlg 40tude.net>, Derek Parnell says...


Windows API assumes that no valid pointer will ever point anywhere in the
range of addresses 0x00000000 to 0x0000001F, and 0xFFFFFFFD to 0xFFFFFFFF.


<more_trivia>
Actually, Win32 assumes that the range 0x00000000 to 0x0000FFFF is never valid
memory, and thus is never referenced. This is what enables the MAKEINTRESOURCE
hack.
</more_trivia>



 Sep 01 2005



prev sibling next sibling parent reply Derek Parnell <derek psych.ward>  writes:


On Thu, 1 Sep 2005 20:56:51 -0400, Ben Hinkle wrote:


 The definition of a "handle" is a pointer to a pointer.




This usage was extensively used in the Amiga system too. In the Amiga,
there was only one fixed address in the entire system. The address
0x00000004 contained the address of where the core library was loaded into
RAM. From that knowledge, you could locate everything else in the system.
They chose that address instead of zero because a common mistake for coders
was to write something to RAM using a null pointer and doing that would
have wiped out the operating system :D

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
2/09/2005 11:43:36 AM



 Sep 01 2005



  parent  "Walter Bright" <newshound digitalmars.com>  writes:


"Derek Parnell" <derek psych.ward> wrote in message
news:akshp04hduz4$.16z4tpc4g2lzg$.dlg 40tude.net...

 This usage was extensively used in the Amiga system too. In the Amiga,
 there was only one fixed address in the entire system. The address
 0x00000004 contained the address of where the core library was loaded into
 RAM. From that knowledge, you could locate everything else in the system.
 They chose that address instead of zero because a common mistake for


coders

 was to write something to RAM using a null pointer and doing that would
 have wiped out the operating system :D


The biggest mistake in the Intel 8088 design was to put the interrupt table
in the bottom 64Kb of memory, guaranteeing that any NULL or offset-from-NULL
pointer write would trash the operating system. This means that EVERY time
your program crashed, the only thing to do was reboot regardless, otherwise
you risked scrambling your hard disk.

The system PROM should have been mapped to the bottom 64Kb (instead of the
top 64Kb).

The biggest productivity booster for the old DOS days was when OS/2 1.1 came
out, which ran 16 bit programs in protected mode. Oh heaven, a program could
crash and I didn't have to reboot! I never did develop on DOS after that -
all the programs were built, debugged, and tested on OS/2 1.1, and only the
last step was getting them running under DOS.



 Sep 08 2005



prev sibling  parent  "Walter Bright" <newshound digitalmars.com>  writes:


"Ben Hinkle" <ben.hinkle gmail.com> wrote in message
news:df880n$25p2$1 digitaldaemon.com...

 In Win32 API HANDLE is void* but there are places where things like -1 are
 cast to HANDLE. I'm not aware of what non-pointers are cast to HANDLES but
 I've never looked into it. The "pointer to pointer" is what Macs used ages
 ago and from what I understand the Mac OS X doesn't use handles anymore.


In

 MATLAB, ironically, a handle is a double :-) That's because back when


MATLAB

 first got graphics objects (called Handle Graphics) the double was the


only

 datatype in MATLAB. It has advantages and disadvantages but I'm mentioning
 it just for those trivia-minded folks out there who are interested in uses
 of the word "handle".


16 bit DMC++ supports a "handle" pointer type, which is the combination of
an EMM page number, and EMM page offset. A runtime conversion would be done
automagically to map the EMM page into memory and then compute an actual
pointer into it.

char __handle *p;

Ah, those were the daze <g>.



 Sep 08 2005