www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Integer promotion issue

reply Evan <maskedmonkyman gmail.com> writes:
I am writing an 8 bit emulator that uses a lot of small numbers 
and running into a problem where the compiler will automatically 
promote smaller operands up to an int which causes the result of 
math calculations to not fit into smaller types.

//compiled with dmd 2.090.0
void main()
{
     ubyte a = 0x01;
     ubyte b = 0x01;
     ubyte c = a + b; //Error: cannot implicitly convert 
expression cast(int)a + cast(int)b of type int to ubyte
     writeln(c);
}

Is there a way to stop the compiler from up casting 
automatically? or have it down cast it back once it's done?
Jan 25 2020
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/25/20 6:46 PM, Evan wrote:

 I am writing an 8 bit emulator that uses a lot of small numbers and 
 running into a problem where the compiler will automatically promote 
 smaller operands up to an int which causes the result of math 
 calculations to not fit into smaller types.
 
 //compiled with dmd 2.090.0
 void main()
 {
      ubyte a = 0x01;
      ubyte b = 0x01;
      ubyte c = a + b; //Error: cannot implicitly convert expression 
 cast(int)a + cast(int)b of type int to ubyte
      writeln(c);
 }
 
 Is there a way to stop the compiler from up casting automatically? or 
 have it down cast it back once it's done?


Not really. You can cast yourself, write your own type that does it, or 
use masking to have the compiler accept the result (i.e. (a + b) & 0xff).

I'd recommend your own type, as it will look a lot cleaner.

-Steve
Jan 25 2020
parent reply Evan <maskedmonkyman gmail.com> writes:
On Saturday, 25 January 2020 at 23:58:00 UTC, Steven 
Schveighoffer wrote:

 On 1/25/20 6:46 PM, Evan wrote:

 [...]


 Not really. You can cast yourself, write your own type that 
 does it, or use masking to have the compiler accept the result 
 (i.e. (a + b) & 0xff).

 I'd recommend your own type, as it will look a lot cleaner.

 -Steve


If I make my own type then I have to write more casts from my 
type to built in types. I want to create less work for myself not 
more.
Jan 25 2020
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 26 January 2020 at 00:07:47 UTC, Evan wrote:

 I want to create less work for myself not more.


Yeah, there's basically no winning :(

D inherited a silly rule from C (all arithmetic yields int or 
above) then it combined poorly with the (decent in isolation) 
idea to make lossy conversions stand out.

It can sometimes help to try sticking to += operators which 
aren't as picky.
Jan 25 2020
parent reply Guillaume Piolat <first.last gmail.com> writes:
On Sunday, 26 January 2020 at 00:22:14 UTC, Adam D. Ruppe wrote:

 D inherited a silly rule from C (all arithmetic yields int or 
 above)


It seems silly however in the wild this C int promotion rule make 
stuff work without the author knowledge.


Example:

--------------------------------------------------------


void computeMean(short* )
{
     // let's compute the mean of two arrays
     foreach(n; 0..N)
     {
         short a = input[n];
         short b = input2[n];
         int meanOfTwo = (a + b) >> 1; // BOOM
     }
}

--------------------------------------------------------

Pretty sure a LOT of code would break if C++ or D would abandon 
this rule (well, except you introduce traps for integer overflow).

Doing it without the rule introduce a lot of casts...
Jan 26 2020
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 26 January 2020 at 13:01:58 UTC, Guillaume Piolat 
wrote:

 It seems silly however in the wild this C int promotion rule 
 make stuff work without the author knowledge.


indeed.

Though if that rule wasn't there we'd probably have more author 
knowledge anyway. But it is too late now.

For D though maybe we could change our truncating rule to make it 
more relaxed if all the inputs are literals or the same type as 
the assignment. So it still promotes, just allows implicit cast 
back to the original.

short a;
short b = a + 1; // implicitly does cast(short)(cast(int) a + 1)

int a;
short b;
short c = b + a; // error: cannot implicitly cast because the 
`int a` triggers more strictness

ubyte a;
ubyte b;
ubyte c = a + b; // allowed

ushort d;
ubyte c = a + d; // prohibited



So it will follow the C promotion rules and then implicitly cast 
back to the largest integral type in the original expression.

ubyte a = 255 + 5; // literals are excluded from the new implicit 
rule loosening, so this does what it does now = int(int(255) + 
int(5)) => VRP -> too large for ubyte, error.


Cuz I just think if we are explicitly specifying a particular 
smaller type throughout it indicates a pretty clear intent 
already.
Jan 26 2020
parent NaN <divide by.zero> writes:
On Sunday, 26 January 2020 at 14:14:20 UTC, Adam D. Ruppe wrote:

 On Sunday, 26 January 2020 at 13:01:58 UTC, Guillaume Piolat 
 wrote:


 Cuz I just think if we are explicitly specifying a particular 
 smaller type throughout it indicates a pretty clear intent 
 already.


Would a truncated assign operator be feasible? Something like...

byte b := a*b-c;
Jan 28 2020
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/26/2020 5:01 AM, Guillaume Piolat wrote:

 Pretty sure a LOT of code would break if C++ or D would abandon this rule


Worst of all, the breakage would be silent.
Jan 26 2020
parent reply HuskyNator <HuskyNator protonmail.ch> writes:
On Sunday, 26 January 2020 at 23:16:40 UTC, Walter Bright wrote:

 On 1/26/2020 5:01 AM, Guillaume Piolat wrote:

 Pretty sure a LOT of code would break if C++ or D would 
 abandon this rule


 Worst of all, the breakage would be silent.


Could this be added as a compiler flag?
This would probably fix a lot of annoyances, without causing 
issues in older codebases.
Sep 08 2021
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/8/2021 5:19 AM, HuskyNator wrote:

 On Sunday, 26 January 2020 at 23:16:40 UTC, Walter Bright wrote:

 On 1/26/2020 5:01 AM, Guillaume Piolat wrote:

 Pretty sure a LOT of code would break if C++ or D would abandon this rule


 Worst of all, the breakage would be silent.


 
 Could this be added as a compiler flag?
 This would probably fix a lot of annoyances, without causing issues in older 
 codebases.


Every such flag doubles the amount of testing required. And global behavior 
flags like this will have essentially unpredictable results on the code.
Sep 08 2021
parent Basile B. <b2.temp gmx.com> writes:
On Wednesday, 8 September 2021 at 20:15:36 UTC, Walter Bright 
wrote:

 Every such flag doubles the amount of testing required. And 
 global behavior flags like this will have essentially 
 unpredictable results on the code.


such flags could hypothetically be created in the source code at 
the local scope and be included in the compiler `Scope`. But 
unfortuanatly this is not the D way

```d
void v()
{
    /* code here does not obey to FLAG*/
    {$PUSH FLAG} // in dmd : sc = sc.push(...)
    /* code here obey to FLAG*/
    {$POP} // in dmd : sc = sc.pop()
    /* code here does not obey to FLAG*/
}
```

as this looks much as preprocessor things (also they are not).
This also solves the problem of reusing small samples without 
knowing that a special compiler flag is required.
Sep 09 2021
prev sibling parent bauss <jj_1337 live.dk> writes:
On Wednesday, 8 September 2021 at 12:19:11 UTC, HuskyNator wrote:

 On Sunday, 26 January 2020 at 23:16:40 UTC, Walter Bright wrote:

 On 1/26/2020 5:01 AM, Guillaume Piolat wrote:

 Pretty sure a LOT of code would break if C++ or D would 
 abandon this rule


 Worst of all, the breakage would be silent.


 Could this be added as a compiler flag?
 This would probably fix a lot of annoyances, without causing 
 issues in older codebases.


A lot of tests would fail because they rely on the current 
implementation, which means a lot of tests would have to be 
changed.

You'll essentially have to do double tests where you have all 
tests without the flag and all tests with the flag.

The annoyance isn't that much if you just understand it in the 
first place.

Personally I've never had to deal with it more than maybe once or 
twice.
Sep 09 2021
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Jan 26, 2020 at 12:07:47AM +0000, Evan via Digitalmars-d wrote:

 On Saturday, 25 January 2020 at 23:58:00 UTC, Steven Schveighoffer wrote:

 On 1/25/20 6:46 PM, Evan wrote:

 [...]


 
 Not really. You can cast yourself, write your own type that does it,
 or use masking to have the compiler accept the result (i.e. (a + b)
 & 0xff).
 
 I'd recommend your own type, as it will look a lot cleaner.
 
 -Steve


 
 If I make my own type then I have to write more casts from my type to
 built in types. I want to create less work for myself not more.


No you don't.  Check this out (put it in a file called "nopromote.d" and
import it):

	module nopromote;

	enum isNarrowInt(T) = is(T : int) || is(T : uint);

	/**
	 * A wrapper around a built-in narrow int that truncates the
	 * result of arithmetic operations to the narrow type,
	 * overriding built-in int promotion rules.
	 */
	struct Np(T)
	    if (isNarrowInt!T)
	{
	    T impl;
	    alias impl this;

	    /**
	     * Truncating binary operator.
	     */
	    Np opBinary(string op, U)(U u)
		if (is(typeof((T x, U y) => mixin("x " ~ op ~ " y"))))
	    {
		return Np(cast(T) mixin("this.impl " ~ op ~ " u"));
	    }

	    /**
	     * Truncating unary operator.
	     */
	    Np opUnary(string op)()
		if (is(typeof((T x) => mixin(op ~ "cast(int) x"))))
	    {
		return Np(cast(T) mixin(op ~ " cast(int) this.impl"));
	    }

	    /**
	     * Infectiousness: any expression containing Np should
	     * automatically use Np operator semantics.
	     */
	    Np opBinaryRight(string op, U)(U u)
		if (is(typeof((T x, U y) => mixin("x " ~ op ~ " y"))))
	    {
		return Np(cast(T) mixin("u " ~ op ~ " this.impl"));
	    }
	}

	/**
	 * Returns: A lightweight wrapped type that overrides built-in
	 * arithmetic operators to always truncate to the given type
	 * without promoting to int or uint.
	 */
	auto np(T)(T t)
	    if (isNarrowInt!T)
	{
	    return Np!T(t);
	}

	// Test binary ops
	 safe unittest
	{
	    ubyte x = 1;
	    ubyte y = 2;
	    auto z = x.np + y;
	    static assert(is(typeof(z) : ubyte));
	    assert(z == 3);

	    byte zz = x.np + y;
	    assert(zz == 3);

	    x = 255;
	    z = x.np + y;
	    assert(z == 1);
	}

	 safe unittest
	{
	    byte x = 123;
	    byte y = 5;
	    auto z = x.np + y;
	    static assert(is(typeof(z) : byte));
	    assert(z == byte.min);

	    byte zz = x.np + y;
	    assert(zz == byte.min);
	}

	 safe unittest
	{
	    import std.random;
	    short x = cast(short) uniform(0, 10);
	    short y = 10;
	    auto z = x.np + y;
	    static assert(is(typeof(z) : short));
	    assert(z == x + 10);

	    short s = x.np + y;
	    assert(s == x + 10);
	}

	// Test unary ops
	 safe unittest
	{
	    byte b = 10;
	    auto c = -b.np;
	    static assert(is(typeof(c) : byte));
	    assert(c == -10);

	    ubyte ub = 16;
	    auto uc = -ub.np;
	    static assert(is(typeof(uc) : ubyte));
	    assert(uc == 0xF0);
	}

	version(unittest)
	{
	    // These tests are put here as actual module functions, to
	    // force optimizer not to discard calls to these functions,
	    // so that we can see the actual generated code.
	    byte byteNegate(byte b) { return -b.np; }
	    ubyte ubyteNegate(ubyte b) { return -b.np; }

	    byte byteTest1(int choice, byte a, byte b)
	    {
		if (choice == 1)
		    return a.np + b;
		if (choice == 2)
		    return a.np / b;
		assert(0);
	    }

	    short shortAdd(short a, short b) { return a.np + b; }

	    // Test opBinaryRight
	    byte byteRightTest(byte a, byte c)
	    {
		auto result = a + c.np;
		static assert(is(typeof(result) : byte));
		return result;
	    }

	    unittest
	    {
		assert(byteRightTest(127, 1) == byte.min);
	    }

	    short multiTest1(short x, short y)
	    {
		return short(2) + 2*(x - y.np);
	    }

	    unittest
	    {
		// Test wraparound semantics.
		assert(multiTest1(32767, 16384) == short.min);
	    }

	    short multiTest2(short a, short b)
	    {
		short x = a;
		short y = b;
		return (2*x + 1) * (y.np/2 - 1);
	    }

	    unittest
	    {
		assert(multiTest2(1, 4) == 3);
	    }
	}

As the unittests show, all you need to do is to insert .np somewhere in
your arithmetic expression, and it automatically "taints" the expression
to be the given narrow type and returns a narrow type as result. No
casts are needed in user code.

I use this module to work around silly archaic C/C++ integer promotion
rules that Walter insists D must have.

Hope this helps.


T

-- 
Дерево держится корнями, а человек -
друзьями.
Jan 25 2020
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/25/20 7:07 PM, Evan wrote:

 On Saturday, 25 January 2020 at 23:58:00 UTC, Steven Schveighoffer wrote:

 On 1/25/20 6:46 PM, Evan wrote:

 [...]


 Not really. You can cast yourself, write your own type that does it, 
 or use masking to have the compiler accept the result (i.e. (a + b) & 
 0xff).

 I'd recommend your own type, as it will look a lot cleaner.


 
 If I make my own type then I have to write more casts from my type to 
 built in types. I want to create less work for myself not more.


I'm not sure what you mean. You should be able to alias-this the type to 
reduce the casting. Whipped up in a couple minutes:

struct U8
{
     ubyte _val;
     alias _val this;
     this(ubyte x)
     {
         _val = x;
     }

     U8 opBinary(string op, T)(T other)
             if ((is(T == U8) || is(T == ubyte)) && (op == "+" || op == 
"-" /* || op == ... */))
     {
         return mixin("U8(cast(ubyte)(_val " ~ op ~ " cast(ubyte)other))");
     }
     // opBinaryRight etc...
}

void main()
{
     import std.stdio;

     U8 a = 0x01;
     U8 b = 0x01;
     ubyte c = a + b; //Error: cannot implicitly convert expression 
cast(int)a + cast(int)b of type int to ubyte
     writeln(c); // 2
     ubyte d = b - a;
     writeln(d); // 0
     writeln(a); // 1 (uses alias this)
}

-Steve
Jan 25 2020
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/25/20 7:31 PM, Steven Schveighoffer wrote:

 //Error: cannot implicitly convert expression cast(int)a + cast(int)b of 
 type int to u


Ugh, I keep forgetting to adjust the comments. This error doesn't happen.

-Steve
Jan 25 2020
prev sibling parent reply Shachar Shemesh <shachar weka.io> writes:
On 26/01/2020 01:46, Evan wrote:

 I am writing an 8 bit emulator that uses a lot of small numbers and
 running into a problem where the compiler will automatically promote
 smaller operands up to an int which causes the result of math
 calculations to not fit into smaller types.
 
 //compiled with dmd 2.090.0
 void main()
 {
     ubyte a = 0x01;
     ubyte b = 0x01;
     ubyte c = a + b; //Error: cannot implicitly convert expression
 cast(int)a + cast(int)b of type int to ubyte
     writeln(c);
 }
 
 Is there a way to stop the compiler from up casting automatically? or
 have it down cast it back once it's done?


It was a recurring pet-peeve for me while working with D, and for pretty
much the same reason as you (for me it was writing the RAID algorithm
for Weka.io).

I have since decided to write my own programming language, and,
predictably, this was a problem I decided to tackle.

I am posting this here in the hope that someone will be able to steal
any good ideas I have. I am working on it, but I do so alone, and in my
spare time (such that there is). I would much rather have good ideas be
used than be given credit for them, but see them go unused.

So if there is anything you think is worth stealing, please do. I don't
think it is possible, as Practical's integer promotion rules are _very_
different, and I don't see retrofitting them as plausible, but on the
off chance I'm wrong:

https://github.com/Practical/practical-sa/wiki/Implicit-casts
Feb 01 2020
parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
n Saturday, February 1, 2020 5:50:21 AM MST Shachar Shemesh via Digitalmars-
d wrote:

 On 26/01/2020 01:46, Evan wrote:

 I am writing an 8 bit emulator that uses a lot of small numbers and
 running into a problem where the compiler will automatically promote
 smaller operands up to an int which causes the result of math
 calculations to not fit into smaller types.

 //compiled with dmd 2.090.0
 void main()
 {
     ubyte a = 0x01;
     ubyte b = 0x01;
     ubyte c = a + b; //Error: cannot implicitly convert expression
 cast(int)a + cast(int)b of type int to ubyte
     writeln(c);
 }

 Is there a way to stop the compiler from up casting automatically? or
 have it down cast it back once it's done?


 It was a recurring pet-peeve for me while working with D, and for pretty
 much the same reason as you (for me it was writing the RAID algorithm
 for Weka.io).

 I have since decided to write my own programming language, and,
 predictably, this was a problem I decided to tackle.

 I am posting this here in the hope that someone will be able to steal
 any good ideas I have. I am working on it, but I do so alone, and in my
 spare time (such that there is). I would much rather have good ideas be
 used than be given credit for them, but see them go unused.

 So if there is anything you think is worth stealing, please do. I don't
 think it is possible, as Practical's integer promotion rules are _very_
 different, and I don't see retrofitting them as plausible, but on the
 off chance I'm wrong:

 https://github.com/Practical/practical-sa/wiki/Implicit-casts


So, basically, narrowing conversions are stricter in that converting between
signed and unsigned types is treated as a narrowing conversion when it isn't
guaranteed that the target type can hold the original value, and integer
types smaller than int don't get changed to int when doing arinthmetic. It
seems like a better approach to me, though I don't know how easy it would be
to change D to function that way at this point even if we wanted to. Either
way, for a language starting from scratch, I'm inclined to think that it's a
more sensible approach than what we have in D.

- Jonathan M Davis
Feb 05 2020