www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - size_t vs uintptr_t

reply Walter Bright <newshound2 digitalmars.com> writes:
I recently remembered something I'd half-forgotten. A size_t is not guaranteed 
to be the same size as a pointer. A uintptr_t is.

size_t and uintptr_t are the same for all platforms that D currently supports. 
But this may not always hold true, and besides, it is better self-documenting 
when using uintptr_t for "holds a pointer" as opposed to size_t as "holds an 
offset to a pointer".

uintptr_t may be found in:

     import core.stdc.stdint : uintptr_t;

as it is part of Standard C.

When are they not the same?

1. In 16 bit code that supports the 'far' memory model, where pointers are a 
segment/offset pair and uintptr_t would be 32 bits and size_t 16. Of course, D 
does not support 16 bit code so this is irrelevant for D.

2. There is a memory model in the 32 bit x86 where segment/offset pairs are 
used, and uintptr_t would be >=48 bits and size_t 32. This is used sometimes in 
kernel programming.

3. Any system that would have some sort of banked switched memory scheme.

Ok, I admit these are not likely to emerge. But I'd like our code to be 
pedantically, nitpickingly correct, as well as self-documenting.

druntime/phobos pervasively misuse size_t. I've put in a PR to address some:

     https://github.com/dlang/phobos/pull/4430

=========================================================

A related issue is I see much code of the form:

     cast(size_t)ptr & 3

to check alignment. A better (possibly faster) method is:

     cast(uint)ptr & 3

because even 64 bit CPUs often operate faster with 32 bit operations (thanks to 
some research by Andrei).

=========================================================

Call to action: do things like:

     grep cast(size_t) *.d

to look for misuse, and submit PRs to fix!
Jun 14 2016
next sibling parent reply Guillaume Boucher <guillaume.boucher.d gmail.com> writes:
On Tuesday, 14 June 2016 at 21:59:32 UTC, Walter Bright wrote:

 A related issue is I see much code of the form:

     cast(size_t)ptr & 3

 to check alignment. A better (possibly faster) method is:

     cast(uint)ptr & 3

 because even 64 bit CPUs often operate faster with 32 bit 
 operations (thanks to some research by Andrei).


Isn't it guaranteed that x.sizeof >= x.alignof?  (At least it is 
in C and C++.)  So the alignment should be of type size_t and not 
of type uintptr_t.

Also in general cast(uint)ptr%alignment is wrong since alignment 
does not need to be 32 bit.  However, cast(size_t)ptr%alignment 
is be correct in any case.
Jun 14 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/14/2016 3:38 PM, Guillaume Boucher wrote:

 Isn't it guaranteed that x.sizeof >= x.alignof?  (At least it is in C and C++.)
 So the alignment should be of type size_t and not of type uintptr_t.

 Also in general cast(uint)ptr%alignment is wrong since alignment does not need
 to be 32 bit.  However, cast(size_t)ptr%alignment is be correct in any case.


There is no conceivable case where alignment will be > 32 bits, nor not being a 
power of 2.
Jun 14 2016
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 14 June 2016 at 23:19:12 UTC, Walter Bright wrote:

 On 6/14/2016 3:38 PM, Guillaume Boucher wrote:

 Isn't it guaranteed that x.sizeof >= x.alignof?  (At least it 
 is in C and C++.)
 So the alignment should be of type size_t and not of type 
 uintptr_t.

 Also in general cast(uint)ptr%alignment is wrong since 
 alignment does not need
 to be 32 bit.  However, cast(size_t)ptr%alignment is be 
 correct in any case.


 There is no conceivable case where alignment will be > 32 bits, 
 nor not being a power of 2.


There are many cases in which alignment in 64bits.

There is nothing that says that x.sizeof >= x.alignof must be 
respected, but really, if you don't respect that, you are simply 
wasting space and cache without getting any benefit (the purpose 
of alignment is to avoid store/load across cache lines).
Jun 14 2016
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/14/2016 7:04 PM, deadalnix wrote:

 There are many cases in which alignment in 64bits.


Not any that have anything to do with what we use alignment for.
Jun 15 2016
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/14/16 10:04 PM, deadalnix wrote:

 On Tuesday, 14 June 2016 at 23:19:12 UTC, Walter Bright wrote:

 On 6/14/2016 3:38 PM, Guillaume Boucher wrote:

 Isn't it guaranteed that x.sizeof >= x.alignof?  (At least it is in C
 and C++.)
 So the alignment should be of type size_t and not of type uintptr_t.

 Also in general cast(uint)ptr%alignment is wrong since alignment does
 not need
 to be 32 bit.  However, cast(size_t)ptr%alignment is be correct in
 any case.


 There is no conceivable case where alignment will be > 32 bits, nor
 not being a power of 2.


 There are many cases in which alignment in 64bits.

 There is nothing that says that x.sizeof >= x.alignof must be respected,
 but really, if you don't respect that, you are simply wasting space and
 cache without getting any benefit (the purpose of alignment is to avoid
 store/load across cache lines).


I think you guys are missing the point.

I can check for 64-bit alignment by casting to ubyte:

((cast(ubyte)ptr) & 0x07) == 0

Walter: I don't see why the optimizer doesn't figure out the fastest way 
to do this, even with size_t. You are logic-anding a constant.

-Steve
Jun 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/15/2016 9:31 AM, Steven Schveighoffer wrote:

 I think you guys are missing the point.

 I can check for 64-bit alignment by casting to ubyte:

 ((cast(ubyte)ptr) & 0x07) == 0

 Walter: I don't see why the optimizer doesn't figure out the fastest way to do
 this, even with size_t. You are logic-anding a constant.


You're right.
Jun 15 2016
parent reply Guillaume Boucher <guillaume.boucher.d gmail.com> writes:
I was referring to this diff in the pull linked request:

-private size_t _alignUp(size_t alignment)(size_t n)
+private uintptr_t _alignUp(uintptr_t alignment)(uintptr_t n)

size_t is the correct type.  There is no reason to change it.
Jun 17 2016
next sibling parent Observer <here inter.net> writes:
On Friday, 17 June 2016 at 15:50:41 UTC, Guillaume Boucher wrote:

 I was referring to this diff in the pull linked request:

 -private size_t _alignUp(size_t alignment)(size_t n)
 +private uintptr_t _alignUp(uintptr_t alignment)(uintptr_t n)

 size_t is the correct type.  There is no reason to change it.


I dislike the conceptually privileged status of "uint" in Walter's
proposal.  size_t is generic; uint is not.  I like the idea in C++
of the ptrdiff_t type being generic across pointer types.  If we
want to have the code be conceptually cleaner, I think a type such
as voidptr_t, which is more obviously intended to represent an
arbitrary pointer, would be a better choice.
Jun 18 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/17/2016 8:50 AM, Guillaume Boucher wrote:

 I was referring to this diff in the pull linked request:

 -private size_t _alignUp(size_t alignment)(size_t n)
 +private uintptr_t _alignUp(uintptr_t alignment)(uintptr_t n)

 size_t is the correct type.  There is no reason to change it.


Since _alignUp returns a pointer-sized integer, uintptr_t is the correct type 
for it, not size_t.
Jun 18 2016
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 14 June 2016 at 23:19:12 UTC, Walter Bright wrote:

 There is no conceivable case where alignment will be > 32 bits, 
 nor not being a power of 2.


Are you talking about hardware alignment or programmer specified 
alignment?
Jun 15 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 15 June 2016 at 07:29:05 UTC, Ola Fosheim Grøstad 
wrote:

 On Tuesday, 14 June 2016 at 23:19:12 UTC, Walter Bright wrote:

 There is no conceivable case where alignment will be > 32 
 bits, nor not being a power of 2.


 Are you talking about hardware alignment or programmer 
 specified alignment?


When I come to think of it, there probably are some DSPs with 3 
byte alignment -> 24 bit load/store. When loaded into a register 
the upper 8 bits are fixed to zero.
Jun 15 2016
prev sibling next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
size_t is defined in object.d. If we want to migrate fully, we will need 
to add a public import for uintptr_t. Otherwise it is not very 
consistent (and adds one more import across the board, when previously 
there was none.).
Jun 14 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/14/2016 8:43 PM, rikki cattermole wrote:

 size_t is defined in object.d. If we want to migrate fully, we will need to add
 a public import for uintptr_t. Otherwise it is not very consistent (and adds
one
 more import across the board, when previously there was none.).


I won't say you're wrong, but I'd prefer to see how things go a bit before 
making such a change. Let things perc a bit.
Jun 15 2016
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-06-14 23:59, Walter Bright wrote:

 I recently remembered something I'd half-forgotten. A size_t is not
 guaranteed to be the same size as a pointer. A uintptr_t is.

 size_t and uintptr_t are the same for all platforms that D currently
 supports. But this may not always hold true, and besides, it is better
 self-documenting when using uintptr_t for "holds a pointer" as opposed
 to size_t as "holds an offset to a pointer".



 Ok, I admit these are not likely to emerge. But I'd like our code to be
 pedantically, nitpickingly correct, as well as self-documenting.


I'd like that too, but as you said it's not an issue on any supported 
platforms. Therefore I think we have much more important stuff to do 
than worry about than fixing this.

As Andrei has said:

"Let's keep the eyes on the ball".

-- 
/Jacob Carlborg
Jun 14 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/14/2016 11:31 PM, Jacob Carlborg wrote:

 Ok, I admit these are not likely to emerge. But I'd like our code to be
 pedantically, nitpickingly correct, as well as self-documenting.


 I'd like that too, but as you said it's not an issue on any supported
platforms.
 Therefore I think we have much more important stuff to do than worry about than
 fixing this.

 As Andrei has said:

 "Let's keep the eyes on the ball".


1. I'm not saying this is a priority, more like something to keep in mind.
2. Lots of people say they are looking for simple ways to get into
contributing. 
This qualifies.
3. Sweating the details is what makes for great software.


Some anecdotes:

I once toured a new house for sale. My agent pointed out that the screw slots
on 
the electric wall plates were all lined up. I said who cares. The agent said he 
looked for that, because it was indicative of the general contractor caring 
about the details, and if he'd do right things that didn't matter, it was a
good 
clue he'd done the right things on things that did matter.

I once looked at a high end Sony projector a long time ago. The salesman said 
that Sony put their better techs on building this product, and you could tell
by 
the way the resistors were installed - they all were oriented the same way 
(resistors work exactly the same way no matter which way they were installed). 
This wasn't true of the low end products.

Back in college I had a side job assembling electronics boards. I learned how
to 
assemble them perfectly neatly, everything lined up just so. I got paid well 
because my boards worked first try.

In EE91 lab, where students designed and built a project for the semester, I 
could just glance at them and tell which students' projects were going to work 
and which were not. The ones that were going to work were neatly done, the ones 
that were not were a snarl of wires and components. I can't remember ever being 
wrong about my predictions.

I don't believe that software is any different.
Jun 15 2016
prev sibling parent Wyatt <wyatt.epp gmail.com> writes:
On Tuesday, 14 June 2016 at 21:59:32 UTC, Walter Bright wrote:

 Ok, I admit these are not likely to emerge.


Not in desktop, server, or modern mobile phones, but I think 
there are some embedded platforms that have this concern.  I know 
that's not a huge priority, but it's nice to be mindful of it.

-Wyatt
Jun 15 2016