digitalmars.D.learn - convert ubyte[k..k + 1] to int

ref2401 (6/6) May 16 2012 i have an array of ubytes. how can i convert two adjacent ubytes

Regan Heath (24/30) May 16 2012 You don't need to "copy" the data, just tell the compiler to "pretend"

Jonathan M Davis (5/41) May 16 2012 As long as you're going from big endian to little endian,

Robert DaSilva (4/10) May 16 2012 Except they don't take slices. You need a helper function.

Andrew Wiley (9/40) May 16 2012 Unfortunately, this is undefined behavior because you're breaking alignm...
H. S. Teoh (18/70) May 16 2012 Do unions suffer from this problem? Could this prevent alignment
Andrew Wiley (4/76) May 16 2012 As I understand it, this should be fine because the compiler will guaran...

Roman D. Boiko (17/45) May 17 2012 And what about the following code:

Roman D. Boiko (4/20) May 17 2012 I mean, is it safe (assuming that we are allowed to mutate blob,

Artur Skawina (5/26) May 17 2012 Only if C.ptr ends up properly aligned. There are also aliasing

Roman D. Boiko (5/15) May 17 2012 Is it possible to ensure? In my case blob is created as

Luis Panadero =?UTF-8?B?R3VhcmRlw7Fv?= (10/18) May 21 2012 Try to use littleEndianToNative or bigEndianToNative, but you should che...

"ref2401" <refactor24 gmail.com> writes:

i have an array of ubytes. how can i convert two adjacent ubytes 
from the array to an integer?

pseudocode example:
ubyte[5] array = createArray();
int value = array[2..3];

is there any 'memcpy' method or something else to do this?

May 16 2012

"Regan Heath" <regan netmail.co.nz> writes:

On Wed, 16 May 2012 15:24:33 +0100, ref2401 <refactor24 gmail.com> wrote:

 i have an array of ubytes. how can i convert two adjacent ubytes from  
 the array to an integer?

 pseudocode example:
 ubyte[5] array = createArray();
 int value = array[2..3];

 is there any 'memcpy' method or something else to do this?

You don't need to "copy" the data, just tell the compiler to "pretend"  
it's a short (in this case, for 2 bytes) then copy the value/assign to an  
int. e.g.

import std.stdio;

void main()
{
	ubyte[5] array = [ 0xFF, 0xFF, 0x01, 0x00, 0xFF ];
	int value = *cast(short*)array[2..3].ptr;
	writefln("Result = %s", value);
}

The line:
   int value = *cast(short*)array[2..3].ptr;

1. slices 2 bytes from the array.
2. obtains the ptr to them
3. casts the ptr to short*
4. copies the value pointed at by the short* ptr to an int

You may need to worry about little/big endian issues, see:
http://en.wikipedia.org/wiki/Endianness

The above code outputs "Result = 1" on my little-endian x86 desktop  
machine but would output "Result = 256" on a big-endian machine.

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

May 16 2012

"Jonathan M Davis" <jmdavisProg gmx.com> writes:

On Wednesday, May 16, 2012 17:03:44 Regan Heath wrote:
 On Wed, 16 May 2012 15:24:33 +0100, ref2401 <refactor24 gmail.com> wrote:
 i have an array of ubytes. how can i convert two adjacent ubytes from
 the array to an integer?
 
 pseudocode example:
 ubyte[5] array = createArray();
 int value = array[2..3];
 
 is there any 'memcpy' method or something else to do this?

 
 You don't need to "copy" the data, just tell the compiler to "pretend"
 it's a short (in this case, for 2 bytes) then copy the value/assign to an
 int. e.g.
 
 import std.stdio;
 
 void main()
 {
 ubyte[5] array = [ 0xFF, 0xFF, 0x01, 0x00, 0xFF ];
 int value = *cast(short*)array[2..3].ptr;
 writefln("Result = %s", value);
 }
 
 The line:
 int value = *cast(short*)array[2..3].ptr;
 
 1. slices 2 bytes from the array.
 2. obtains the ptr to them
 3. casts the ptr to short*
 4. copies the value pointed at by the short* ptr to an int
 
 You may need to worry about little/big endian issues, see:
 http://en.wikipedia.org/wiki/Endianness
 
 The above code outputs "Result = 1" on my little-endian x86 desktop
 machine but would output "Result = 256" on a big-endian machine.

As long as you're going from big endian to little endian, 
std.bitmanip.bigEndianToNative will do the conversion fairly easily, but if 
they're in little endian, then the nasty casting is the way to go.

- Jonathan M Davis

May 16 2012

"Robert DaSilva" <spunit262 yahoo.com> writes:

On Wednesday, 16 May 2012 at 18:47:55 UTC, Jonathan M Davis wrote:
 As long as you're going from big endian to little endian,
 std.bitmanip.bigEndianToNative will do the conversion fairly 
 easily, but if
 they're in little endian, then the nasty casting is the way to 
 go.

 - Jonathan M Davis

Except they don't take slices. You need a helper function.

ubyte[2] _2(ubyte[] a){ubyte[2] b; assert(a.length==2); b[]=a; 
return b;}

May 16 2012

Andrew Wiley <wiley.andrew.j gmail.com> writes:

On Wed, May 16, 2012 at 11:03 AM, Regan Heath <regan netmail.co.nz> wrote:

 On Wed, 16 May 2012 15:24:33 +0100, ref2401 <refactor24 gmail.com> wrote:

  i have an array of ubytes. how can i convert two adjacent ubytes from the
 array to an integer?

 pseudocode example:
 ubyte[5] array = createArray();
 int value = array[2..3];

 is there any 'memcpy' method or something else to do this?

 You don't need to "copy" the data, just tell the compiler to "pretend"
 it's a short (in this case, for 2 bytes) then copy the value/assign to an
 int. e.g.

 import std.stdio;

 void main()
 {
        ubyte[5] array = [ 0xFF, 0xFF, 0x01, 0x00, 0xFF ];
        int value = *cast(short*)array[2..3].ptr;
        writefln("Result = %s", value);
 }

 The line:
  int value = *cast(short*)array[2..3].ptr;

 1. slices 2 bytes from the array.
 2. obtains the ptr to them
 3. casts the ptr to short*
 4. copies the value pointed at by the short* ptr to an int

 You may need to worry about little/big endian issues, see:
 http://en.wikipedia.org/wiki/**Endianness<http://en.wikipedia.org/wiki/Endianness>

 The above code outputs "Result = 1" on my little-endian x86 desktop
 machine but would output "Result = 256" on a big-endian machine.

 R

Unfortunately, this is undefined behavior because you're breaking alignment
rules. On x86, this will just cause a slow load from memory. On ARM, this
will either crash your program with a bus error on newer hardware or give
you a gibberish value on ARMv6 and older.
Declaring a short, getting a pointer to it, and casting that pointer to a
ubyte* to copy into it is fine, but casting a ubyte* to a short* will cause
a 2-byte load from a 1-byte aligned address, which leads down the yellow
brick road to pain.

May 16 2012

"H. S. Teoh" <hsteoh quickfur.ath.cx> writes:

On Wed, May 16, 2012 at 10:25:51PM -0500, Andrew Wiley wrote:
 On Wed, May 16, 2012 at 11:03 AM, Regan Heath <regan netmail.co.nz> wrote:
 
 On Wed, 16 May 2012 15:24:33 +0100, ref2401 <refactor24 gmail.com> wrote:

  i have an array of ubytes. how can i convert two adjacent ubytes from the
 array to an integer?

 pseudocode example:
 ubyte[5] array = createArray();
 int value = array[2..3];

 is there any 'memcpy' method or something else to do this?

 You don't need to "copy" the data, just tell the compiler to "pretend"
 it's a short (in this case, for 2 bytes) then copy the value/assign to an
 int. e.g.

 import std.stdio;

 void main()
 {
        ubyte[5] array = [ 0xFF, 0xFF, 0x01, 0x00, 0xFF ];
        int value = *cast(short*)array[2..3].ptr;
        writefln("Result = %s", value);
 }

 The line:
  int value = *cast(short*)array[2..3].ptr;

 1. slices 2 bytes from the array.
 2. obtains the ptr to them
 3. casts the ptr to short*
 4. copies the value pointed at by the short* ptr to an int

 You may need to worry about little/big endian issues, see:
 http://en.wikipedia.org/wiki/**Endianness<http://en.wikipedia.org/wiki/Endianness>

 The above code outputs "Result = 1" on my little-endian x86 desktop
 machine but would output "Result = 256" on a big-endian machine.

 R

 Unfortunately, this is undefined behavior because you're breaking
 alignment rules. On x86, this will just cause a slow load from memory.
 On ARM, this will either crash your program with a bus error on newer
 hardware or give you a gibberish value on ARMv6 and older.
 Declaring a short, getting a pointer to it, and casting that pointer
 to a ubyte* to copy into it is fine, but casting a ubyte* to a short*
 will cause a 2-byte load from a 1-byte aligned address, which leads
 down the yellow brick road to pain.

Do unions suffer from this problem? Could this prevent alignment
problems:

	short bytesToShort(ubyte[] b)
	in { assert(b.length==2); }
	body {
		union U {
			short val;
			ubyte[2] b;
		}
		U u;

		u.b[] = b[];
		return u.val;
	}

?


T

-- 
Береги платье снову, а здоровье смолоду.

May 16 2012

Andrew Wiley <wiley.andrew.j gmail.com> writes:

On Wed, May 16, 2012 at 11:07 PM, H. S. Teoh <hsteoh quickfur.ath.cx> wrote:

 On Wed, May 16, 2012 at 10:25:51PM -0500, Andrew Wiley wrote:
 On Wed, May 16, 2012 at 11:03 AM, Regan Heath <regan netmail.co.nz>

 wrote:
 On Wed, 16 May 2012 15:24:33 +0100, ref2401 <refactor24 gmail.com>


 wrote:
  i have an array of ubytes. how can i convert two adjacent ubytes from


 the
 array to an integer?

 pseudocode example:
 ubyte[5] array = createArray();
 int value = array[2..3];

 is there any 'memcpy' method or something else to do this?

 You don't need to "copy" the data, just tell the compiler to "pretend"
 it's a short (in this case, for 2 bytes) then copy the value/assign to


 an
 int. e.g.

 import std.stdio;

 void main()
 {
        ubyte[5] array = [ 0xFF, 0xFF, 0x01, 0x00, 0xFF ];
        int value = *cast(short*)array[2..3].ptr;
        writefln("Result = %s", value);
 }

 The line:
  int value = *cast(short*)array[2..3].ptr;

 1. slices 2 bytes from the array.
 2. obtains the ptr to them
 3. casts the ptr to short*
 4. copies the value pointed at by the short* ptr to an int

 You may need to worry about little/big endian issues, see:
 http://en.wikipedia.org/wiki/**Endianness<


 http://en.wikipedia.org/wiki/Endianness>
 The above code outputs "Result = 1" on my little-endian x86 desktop
 machine but would output "Result = 256" on a big-endian machine.

 R

 Unfortunately, this is undefined behavior because you're breaking
 alignment rules. On x86, this will just cause a slow load from memory.
 On ARM, this will either crash your program with a bus error on newer
 hardware or give you a gibberish value on ARMv6 and older.
 Declaring a short, getting a pointer to it, and casting that pointer
 to a ubyte* to copy into it is fine, but casting a ubyte* to a short*
 will cause a 2-byte load from a 1-byte aligned address, which leads
 down the yellow brick road to pain.

 Do unions suffer from this problem? Could this prevent alignment
 problems:

        short bytesToShort(ubyte[] b)
        in { assert(b.length==2); }
        body {
                union U {
                        short val;
                        ubyte[2] b;
                }
                U u;

                u.b[] = b[];
                return u.val;
        }

 ?

As I understand it, this should be fine because the compiler will guarantee
that the union is aligned to the maximum alignment required by one of its
members, which is the short. This is probably the safest solution.

May 16 2012

"Roman D. Boiko" <rb d-coding.com> writes:

On Thursday, 17 May 2012 at 04:16:10 UTC, Andrew Wiley wrote:
 On Wed, May 16, 2012 at 11:07 PM, H. S. Teoh 
 <hsteoh quickfur.ath.cx> wrote:
 Do unions suffer from this problem? Could this prevent 
 alignment
 problems:

        short bytesToShort(ubyte[] b)
        in { assert(b.length==2); }
        body {
                union U {
                        short val;
                        ubyte[2] b;
                }
                U u;

                u.b[] = b[];
                return u.val;
        }

 ?

 As I understand it, this should be fine because the compiler 
 will guarantee
 that the union is aligned to the maximum alignment required by 
 one of its
 members, which is the short. This is probably the safest 
 solution.

And what about the following code:

// This implementation is optimized for speed via swapping
endianness in-place
pure immutable(C)[] fixEndian(C, Endian blobEndian =
endian)(ubyte[] blob) if(is(CharTypeOf!C))
{
      import std.bitmanip, std.system;
      auto data = cast(C[]) blob;
      static if(blobEndian != endian)
      {
          static assert(!is(typeof(C) == char)); // UTF-8 doesn't
have endianness
          foreach(ref ch; data) ch = swapEndian(ch);
      }
      return cast(immutable) data;
}

May 17 2012

"Roman D. Boiko" <rb d-coding.com> writes:

On Thursday, 17 May 2012 at 07:07:58 UTC, Roman D. Boiko wrote:
 And what about the following code:

 // This implementation is optimized for speed via swapping
 endianness in-place
 pure immutable(C)[] fixEndian(C, Endian blobEndian =
 endian)(ubyte[] blob) if(is(CharTypeOf!C))
 {
      import std.bitmanip, std.system;
      auto data = cast(C[]) blob;
      static if(blobEndian != endian)
      {
          static assert(!is(typeof(C) == char)); // UTF-8 doesn't
 have endianness
          foreach(ref ch; data) ch = swapEndian(ch);
      }
      return cast(immutable) data;
 }

I mean, is it safe (assuming that we are allowed to mutate blob, 
and its length is a multiple of C.sizeof)?

I do casting from ubyte[] to C[].

May 17 2012

Artur Skawina <art.08.09 gmail.com> writes:

On 05/17/12 10:15, Roman D. Boiko wrote:
 On Thursday, 17 May 2012 at 07:07:58 UTC, Roman D. Boiko wrote:
 And what about the following code:

 // This implementation is optimized for speed via swapping
 endianness in-place
 pure immutable(C)[] fixEndian(C, Endian blobEndian =
 endian)(ubyte[] blob) if(is(CharTypeOf!C))
 {
      import std.bitmanip, std.system;
      auto data = cast(C[]) blob;
      static if(blobEndian != endian)
      {
          static assert(!is(typeof(C) == char)); // UTF-8 doesn't
 have endianness
          foreach(ref ch; data) ch = swapEndian(ch);
      }
      return cast(immutable) data;
 }

 I mean, is it safe (assuming that we are allowed to mutate blob, and its
length is a multiple of C.sizeof)?
 
 I do casting from ubyte[] to C[].

Only if C.ptr ends up properly aligned. There are also aliasing
issues, which i don't think are sufficiently defined for D (for
C, it would be legal only because char* is allowed to alias anything).

artur

May 17 2012

"Roman D. Boiko" <rb d-coding.com> writes:

On Thursday, 17 May 2012 at 08:39:21 UTC, Artur Skawina wrote:
 On 05/17/12 10:15, Roman D. Boiko wrote:
 I mean, is it safe (assuming that we are allowed to mutate 
 blob, and its length is a multiple of C.sizeof)?
 
 I do casting from ubyte[] to C[].

 Only if C.ptr ends up properly aligned. There are also aliasing
 issues, which i don't think are sufficiently defined for D (for
 C, it would be legal only because char* is allowed to alias 
 anything).

 artur

Is it possible to ensure? In my case blob is created as
     auto blob = cast(ubyte[]) read(fileName);
I assume that alignment is safe.

But what should I do to be safe in a general case?

May 17 2012

Luis Panadero =?UTF-8?B?R3VhcmRlw7Fv?= <luis.panadero gmail.com> writes:

ref2401 wrote:

 i have an array of ubytes. how can i convert two adjacent ubytes
 from the array to an integer?
 
 pseudocode example:
 ubyte[5] array = createArray();
 int value = array[2..3];
 
 is there any 'memcpy' method or something else to do this?

Try to use littleEndianToNative or bigEndianToNative, but you should check 
the endianes of the data that you desire convert to a int.
For example:
  ubyte[5] array = createArray();
  int value = littleEndianToNative!int(array);
  // or:
  // int value = bigEndianToNative!int(array);

-- 
I'm afraid that I have a blog: http://zardoz.es

May 21 2012

D Programming

C/C++ Programming

Other

digitalmars.D.learn - convert ubyte[k..k + 1] to int