www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - malloc and buffer overflow attacks

reply Walter Bright <newshound2 digitalmars.com> writes:
While D offers buffer overflow detection, it does not protect against buffer 
overflows resulting from an array size calculation overflow:

     T* p = cast(T*)malloc(len * T.sizeof);

What if `len*T.sizeof` overflows? malloc() will succeed, but the result will be 
too small for the data.

I decided to grep dmd for such allocations:

https://github.com/dlang/dmd/pull/13479/files

and fix them with overflow checks. I recommend everyone check their own
projects 
and eliminate such vulnerabilities.

I post this as I've recently seen reports on malware injection being enabled by 
presenting specially crafted input data to a program that causes an overflow on 
the allocation, then overwrites the data beyond the truncated allocated memory.
Dec 30 2021
next sibling parent reply Adam Ruppe <destructionator gmail.com> writes:
On Friday, 31 December 2021 at 00:13:56 UTC, Walter Bright wrote:

 While D offers buffer overflow detection, it does not protect 
 against buffer overflows resulting from an array size 
 calculation overflow:

     T* p = cast(T*)malloc(len * T.sizeof);


What I do in D is always slice the malloc to the given size 
immediately;

T[] p = (cast(T*)malloc(len * T.sizeof))[0 .. len * T.sizepf];

Then you'd get the protection of bounds checking and if you need 
the ptr, there's still that property.

I'd suggest everyone always do that.
Dec 30 2021
next sibling parent reply Adam Ruppe <destructionator gmail.com> writes:
On Friday, 31 December 2021 at 00:15:48 UTC, Adam Ruppe wrote:

 T[] p = (cast(T*)malloc(len * T.sizeof))[0 .. len * T.sizepf];


eeeek, I did it wrong!

Should be either [0 .. len] on the slice or do the cast on the 
outside instead of inside of parens.

This is why just using `new` is the best of all!
Dec 30 2021
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 31, 2021 at 12:34:46AM +0000, Adam Ruppe via Digitalmars-d wrote:

 On Friday, 31 December 2021 at 00:15:48 UTC, Adam Ruppe wrote:

 T[] p = (cast(T*)malloc(len * T.sizeof))[0 .. len * T.sizepf];


 
 eeeek, I did it wrong!
 
 Should be either [0 .. len] on the slice or do the cast on the outside
 instead of inside of parens.


[...]

Actually, if (len * T.sizeof) overflows, then neither [0 .. len] nor
[0 .. len * T.sizeof)] would be safe from buffer overruns.

E.g., if len = size_t.max / 4 and T.sizeof = 8, then (len * T.sizeof)
would wrap around to a much smaller value than expected, which is the
problem Walter is trying to point out.


T

-- 
That's not a bug; that's a feature!
Dec 30 2021
parent Adam Ruppe <destructionator gmail.com> writes:
On Friday, 31 December 2021 at 01:02:36 UTC, H. S. Teoh wrote:

 [0 .. len * T.sizeof)] would be safe from buffer overruns.


If you do this correctly on the byte before casting you SHOULD be 
ok because if the malloc overflows, so will the slice 
calculation, meaning you get the RangeError when you try to use 
it, which is what we want.

But since I screwed it up twice already i say this is evidence to 
just use `new` and find joy :P
Dec 30 2021
prev sibling parent reply sarn <sarn theartofmachinery.com> writes:
On Friday, 31 December 2021 at 00:15:48 UTC, Adam Ruppe wrote:

 On Friday, 31 December 2021 at 00:13:56 UTC, Walter Bright 
 wrote:

 While D offers buffer overflow detection, it does not protect 
 against buffer overflows resulting from an array size 
 calculation overflow:

     T* p = cast(T*)malloc(len * T.sizeof);


 What I do in D is always slice the malloc to the given size 
 immediately;

 T[] p = (cast(T*)malloc(len * T.sizeof))[0 .. len * T.sizepf];

 Then you'd get the protection of bounds checking and if you 
 need the ptr, there's still that property.

 I'd suggest everyone always do that.


Good thing to do, but Walter's talking about integer overflow 
with the `len * T.sizeof` calculation itself.

calloc() doesn't have this problem.
Dec 30 2021
next sibling parent Elronnd <elronnd elronnd.net> writes:
On Friday, 31 December 2021 at 00:37:20 UTC, sarn wrote:

 Walter's talking about integer overflow with the `len * 
 T.sizeof` calculation itself.


Slicing should have its own overflow check, meaning it doesn't 
matter if the other calculation overflows.  (I don't know if it 
does, but should.)
Dec 30 2021
prev sibling next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 31/12/2021 1:37 PM, sarn wrote:

 On Friday, 31 December 2021 at 00:15:48 UTC, Adam Ruppe wrote:

 On Friday, 31 December 2021 at 00:13:56 UTC, Walter Bright wrote:

 While D offers buffer overflow detection, it does not protect against 
 buffer overflows resulting from an array size calculation overflow:

     T* p = cast(T*)malloc(len * T.sizeof);


 What I do in D is always slice the malloc to the given size immediately;

 T[] p = (cast(T*)malloc(len * T.sizeof))[0 .. len * T.sizepf];

 Then you'd get the protection of bounds checking and if you need the 
 ptr, there's still that property.

 I'd suggest everyone always do that.


 
 Good thing to do, but Walter's talking about integer overflow with the 
 `len * T.sizeof` calculation itself.
 
 calloc() doesn't have this problem.


I would argue any usage of malloc and even calloc is itself a bug in 
your code.

Ideally we would deprecate malloc and calloc from being called directly 
and force people to use a callocSlice version of it instead.

This solves most of these issues out right and allows for D's memory 
safety features to come into play.
Dec 30 2021
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/30/2021 4:37 PM, sarn wrote:

 Good thing to do, but Walter's talking about integer overflow with the `len * 
 T.sizeof` calculation itself.
 
 calloc() doesn't have this problem.


The calculation of `len` can also have overflow problems. `calloc` is not 
sufficient. The provenance of `len` needs to be carefully checked.
Dec 31 2021
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 31, 2021 at 12:12:45PM -0800, Walter Bright via Digitalmars-d wrote:

 On 12/30/2021 4:37 PM, sarn wrote:

 Good thing to do, but Walter's talking about integer overflow with
 the `len * T.sizeof` calculation itself.
 
 calloc() doesn't have this problem.


 
 The calculation of `len` can also have overflow problems. `calloc` is
 not sufficient. The provenance of `len` needs to be carefully checked.


At my day job we use Coverity to identify potential issues with our C
codebase.  One of the issues it reports is using external inputs as the
length of a memory allocation, the typical case being reading an int or
long from a file/socket and then passing that to malloc, et al
(potentially with a sizeof multipler).  So imagine a carefully-crafted
malicious input designed to overflow a 64-bit integer just a little --
the malloc call would end up allocating just a tiny amount of memory
while the rest of the code thinks that the buffer has more memory than
can be addressed.

Of course, that tempts the following check (obviously wrong, but I've
seen this scarily often in "professional" code):

	size_t n = ... /* read from file/socket */;
	size_t nbytes = n * sizeof(Element); // oops
	if (nbytes > INT64_MAX)	// never true
		error();	// dead code
	void *p = malloc(n * sizeof(Element)); // uh-oh
	for (i=0; i < n; i++) {
		... /* use p: kaboom */
	}


T

-- 
INTEL = Only half of "intelligence".
Dec 31 2021
parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Friday, 31 December 2021 at 22:10:10 UTC, H. S. Teoh wrote:

 On Fri, Dec 31, 2021 at 12:12:45PM -0800, Walter Bright via 
 Digitalmars-d wrote:

 On 12/30/2021 4:37 PM, sarn wrote:

 Good thing to do, but Walter's talking about integer 
 overflow with the `len * T.sizeof` calculation itself.
 
 calloc() doesn't have this problem.


 
 The calculation of `len` can also have overflow problems. 
 `calloc` is not sufficient. The provenance of `len` needs to 
 be carefully checked.


 At my day job we use Coverity to identify potential issues with 
 our C codebase.  One of the issues it reports is using external 
 inputs as the length of a memory allocation, the typical case 
 being reading an int or long from a file/socket and then 
 passing that to malloc, et al (potentially with a sizeof 
 multipler).  So imagine a carefully-crafted malicious input 
 designed to overflow a 64-bit integer just a little -- the 
 malloc call would end up allocating just a tiny amount of 
 memory while the rest of the code thinks that the buffer has 
 more memory than can be addressed.

 Of course, that tempts the following check (obviously wrong, 
 but I've seen this scarily often in "professional" code):

 	size_t n = ... /* read from file/socket */;
 	size_t nbytes = n * sizeof(Element); // oops
 	if (nbytes > INT64_MAX)	// never true


    if(__builtin_clz(nbytes)+__builtin_clz(sizeof Element)<64)

this is the best way to find overflow as it avoids all the 
undefined behaviour shenanigans of the compilers optimizers. The 
inconvenience is that not all CPU implement CLZ and some do but 
quite slowly.


 		error();	// dead code
 	void *p = malloc(n * sizeof(Element)); // uh-oh
 	for (i=0; i < n; i++) {
 		... /* use p: kaboom */
Jan 01 2022
prev sibling next sibling parent reply claptrap <clap trap.com> writes:
On Friday, 31 December 2021 at 00:13:56 UTC, Walter Bright wrote:

 While D offers buffer overflow detection, it does not protect 
 against buffer overflows resulting from an array size 
 calculation overflow:

     T* p = cast(T*)malloc(len * T.sizeof);

 What if `len*T.sizeof` overflows? malloc() will succeed, but 
 the result will be too small for the data.

 I decided to grep dmd for such allocations:

 https://github.com/dlang/dmd/pull/13479/files

 and fix them with overflow checks. I recommend everyone check 
 their own projects and eliminate such vulnerabilities.

 I post this as I've recently seen reports on malware injection 
 being enabled by presenting specially crafted input data to a 
 program that causes an overflow on the allocation, then 
 overwrites the data beyond the truncated allocated memory.


are the asserts not taken out in release mode?
Dec 31 2021
parent reply max haughton <maxhaton gmail.com> writes:
On Friday, 31 December 2021 at 08:46:59 UTC, claptrap wrote:

 On Friday, 31 December 2021 at 00:13:56 UTC, Walter Bright 
 wrote:

 [...]


 are the asserts not taken out in release mode?


The compiler ships with assertions turned on.
Dec 31 2021
next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 12/31/21 5:05 AM, max haughton wrote:

 On Friday, 31 December 2021 at 08:46:59 UTC, claptrap wrote:

 On Friday, 31 December 2021 at 00:13:56 UTC, Walter Bright wrote:

 [...]


 are the asserts not taken out in release mode?


 
 The compiler ships with assertions turned on.


And to remind, dmd's -release picks 'safeonly' for the following command 
line switch, which can separately be selected to override the default:

   -boundscheck=[on|safeonly|off]
                     bounds checks on, in  safe only, or off

---
 safe: // Assertions are checked by-default for  safe code

int main(string[] args) {
   int[] arr;
   return arr[args.length];
}
---

Ali
Dec 31 2021
prev sibling parent reply russhy <russhy__ gmail.com> writes:
Allocators should be first class citizen anyways, malloc/free 
should not be used directly, not only they are unsafe, they are 
most of the time not what people need

And i hope what will come out of `std.experimental.allocator` 
won't be using classes..something light, value, extensible and 
made with betterC in mind

It should be the root of everything that comes out of D, not 
something lightly thought

A base to move forward and be future proof
Dec 31 2021
parent reply russhy <russhy__ gmail.com> writes:
On Friday, 31 December 2021 at 21:37:56 UTC, russhy wrote:

 Allocators should be first class citizen anyways, malloc/free 
 should not be used directly, not only they are unsafe, they are 
 most of the time not what people need

 And i hope what will come out of `std.experimental.allocator` 
 won't be using classes..something light, value, extensible and 
 made with betterC in mind

 It should be the root of everything that comes out of D, not 
 something lightly thought

 A base to move forward and be future proof


Follow up, it's roten: 
https://github.com/dlang/phobos/search?q=path%3Astd%2Fexperimental%2Fallocator+exception

There are words, and facts, unfortunately..

According to DConf, Atila is working to get that mess out of 
experimental, hopefully he won't repeat same mistakes as Andrei, 
and step up to remove all of that crap that holds the language 
back

I can offer a hand if he wants to, i personally would hold them 
for a D3, if that effect happen, making a bigger impact is better 
than a few in an ocean full of mistakes
Dec 31 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Friday, 31 December 2021 at 21:58:14 UTC, russhy wrote:

 On Friday, 31 December 2021 at 21:37:56 UTC, russhy wrote:

 Allocators should be first class citizen anyways, malloc/free 
 should not be used directly, not only they are unsafe, they 
 are most of the time not what people need

 And i hope what will come out of `std.experimental.allocator` 
 won't be using classes..something light, value, extensible and 
 made with betterC in mind

 It should be the root of everything that comes out of D, not 
 something lightly thought

 A base to move forward and be future proof


 Follow up, it's roten: 
 https://github.com/dlang/phobos/search?q=path%3Astd%2Fexperimental%2Fallocator+exception

 There are words, and facts, unfortunately..


As far as I can tell, the "uses" of exceptions found by that 
search consist of

1. Code to handle exceptions thrown by copy constructors and 
postblits.
2. Optional features that must be explicitly enabled by the 
programmer at compile-time.
3. A try/catch block in GCAllocator.
3. Unit-test code.

So, nothing here should block compatibility with BetterC, 
although it might require some `version` blocks to disable the 
runtime-dependent stuff.
Dec 31 2021
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
What I can confirm as working right now is make, makeArray and dispose.

What I don't think works right now and that needs to work is 
RCISharedAllocator.

In saying all this, I'm planning on writing my own allocators anyway, so 
it may be all moot for me.
Dec 31 2021
prev sibling parent russhy <russhy__ gmail.com> writes:
On Friday, 31 December 2021 at 22:13:15 UTC, Paul Backus wrote:

 On Friday, 31 December 2021 at 21:58:14 UTC, russhy wrote:

 On Friday, 31 December 2021 at 21:37:56 UTC, russhy wrote:

 Allocators should be first class citizen anyways, malloc/free 
 should not be used directly, not only they are unsafe, they 
 are most of the time not what people need

 And i hope what will come out of `std.experimental.allocator` 
 won't be using classes..something light, value, extensible 
 and made with betterC in mind

 It should be the root of everything that comes out of D, not 
 something lightly thought

 A base to move forward and be future proof


 Follow up, it's roten: 
 https://github.com/dlang/phobos/search?q=path%3Astd%2Fexperimental%2Fallocator+exception

 There are words, and facts, unfortunately..


 As far as I can tell, the "uses" of exceptions found by that 
 search consist of

 1. Code to handle exceptions thrown by copy constructors and 
 postblits.
 2. Optional features that must be explicitly enabled by the 
 programmer at compile-time.
 3. A try/catch block in GCAllocator.
 3. Unit-test code.

 So, nothing here should block compatibility with BetterC, 
 although it might require some `version` blocks to disable the 
 runtime-dependent stuff.


Ok great, sounds manageable, i'm starting the work already, we'll 
see where that'll go
Dec 31 2021
prev sibling next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Friday, 31 December 2021 at 00:13:56 UTC, Walter Bright wrote:

 While D offers buffer overflow detection, it does not protect 
 against buffer overflows resulting from an array size 
 calculation overflow:

     T* p = cast(T*)malloc(len * T.sizeof);

 What if `len*T.sizeof` overflows? malloc() will succeed, but 
 the result will be too small for the data.


For projects using Phobos, an easy way to avoid this is to use 
[`Mallocator`][1] and [`makeArray`][2] from the 
`std.experimental.allocator` package.

     T[] array = Mallocator.instance.makeArray!T(len);

`makeArray` will perform an overflow check internally and return 
`null` if the check fails.

[1]: 
https://dlang.org/library/std/experimental/allocator/mallocator/mallocator.html
[2]: 
https://dlang.org/library/std/experimental/allocator/make_array.html
Dec 31 2021
next sibling parent Nick Treleaven <nick geany.org> writes:
On Friday, 31 December 2021 at 13:52:26 UTC, Paul Backus wrote:

 For projects using Phobos, an easy way to avoid this is to use 
 [`Mallocator`][1] and [`makeArray`][2] from the 
 `std.experimental.allocator` package.

     T[] array = Mallocator.instance.makeArray!T(len);

 `makeArray` will perform an overflow check internally and 
 return `null` if the check fails.


This. D code should not keep calling C malloc when we can do 
better. It's unfortunate that the import and the call above are 
quite awkward to remember and type. It's a shame 
core.memory.pureMalloc repeats this vulnerable design. Perhaps 
add an overload for ease of use?

```d
import core.memory;
T[] array = pureMalloc!T(len);
```
Dec 31 2021
prev sibling parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Friday, 31 December 2021 at 13:52:26 UTC, Paul Backus wrote:

 On Friday, 31 December 2021 at 00:13:56 UTC, Walter Bright 
 wrote:

 While D offers buffer overflow detection, it does not protect 
 against buffer overflows resulting from an array size 
 calculation overflow:

     T* p = cast(T*)malloc(len * T.sizeof);

 What if `len*T.sizeof` overflows? malloc() will succeed, but 
 the result will be too small for the data.


 For projects using Phobos, an easy way to avoid this is to use 
 [`Mallocator`][1] and [`makeArray`][2] from the 
 `std.experimental.allocator` package.

     T[] array = Mallocator.instance.makeArray!T(len);

 `makeArray` will perform an overflow check internally and 
 return `null` if the check fails.

 [1]: 
 https://dlang.org/library/std/experimental/allocator/mallocator/mallocator.html
 [2]: 
 https://dlang.org/library/std/experimental/allocator/make_array.html


In the vulnerability described in the article, the 'len' 
parameter is the result of a sum overflowing in a previous for 
loop, so the problem actually is _outside_ of the allocator.

Anyway, apart from the vulnerability, the described exploit 
details are fascinating and terrific at the same time ..."My 
other compression format is turing-complete!"
Jan 03 2022
parent reply forkit <forkit gmail.com> writes:
On Monday, 3 January 2022 at 12:58:33 UTC, Paolo Invernizzi wrote:

 In the vulnerability described in the article, the 'len' 
 parameter is the result of a sum overflowing in a previous for 
 loop, so the problem actually is _outside_ of the allocator.


That is not entirely correct, and could mislead one into 
implementing a less than optimal solution to the problem.

The overflow and the allocater 'together', provide the attack 
surface.
Jan 03 2022
parent Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Monday, 3 January 2022 at 21:00:38 UTC, forkit wrote:

 On Monday, 3 January 2022 at 12:58:33 UTC, Paolo Invernizzi 
 wrote:

 In the vulnerability described in the article, the 'len' 
 parameter is the result of a sum overflowing in a previous for 
 loop, so the problem actually is _outside_ of the allocator.


 That is not entirely correct, and could mislead one into 
 implementing a less than optimal solution to the problem.

 The overflow and the allocater 'together', provide the attack 
 surface.


I agree that the _surface_ is the couple, but the vulnerability 
to patch is how the needed amount is calculated, and that can be 
an arbitrary complex piece of code. A super-duper-safe allocator 
does not help here, if we are talking about a system language and 
write to memory via pointers arithmetic.
Jan 04 2022
prev sibling next sibling parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Friday, 31 December 2021 at 00:13:56 UTC, Walter Bright wrote:

 What if `len*T.sizeof` overflows? malloc() will succeed, but 
 the result will be too small for the data.


  Makes me wish access to the raw multiplication result and carry 
flags may have been useful here in some way, although the 
language might not have liked it.

  As a quick overview for non-ASM savvy types, x86 will take 2 
arguments, you put one argument in AX and the other you pass to 
mul. So...

```
mov AX, 0x155;
mov BX, 0x123;
mul BX; //result: AX=839F, DX=0001
```

The result of which is going to be in AX:DX, where the overflow 
is put in DX (*if any*), letting you get a 32bit result from 
16bit registers (*or in 64 bit machines you'd get a 128bit 
result*).

Most modern languages just ignore the upper result though, which 
brings us to the following topic. In the 16bit example above 
you'd be short 64k.

Too bad we don't have the cent type yet. Otherwise I'd think 
using ulong and calculating the result and passing that would be 
the safest, (*assuming malloc would take a 64bit result*), 
otherwise checking the upper bits for if it's too big.
Dec 31 2021
next sibling parent reply max haughton <maxhaton gmail.com> writes:
On Saturday, 1 January 2022 at 01:44:42 UTC, Era Scarecrow wrote:

 On Friday, 31 December 2021 at 00:13:56 UTC, Walter Bright 
 wrote:

 What if `len*T.sizeof` overflows? malloc() will succeed, but 
 the result will be too small for the data.


  Makes me wish access to the raw multiplication result and 
 carry flags may have been useful here in some way, although the 
 language might not have liked it.

  As a quick overview for non-ASM savvy types, x86 will take 2 
 arguments, you put one argument in AX and the other you pass to 
 mul. So...

 ```
 mov AX, 0x155;
 mov BX, 0x123;
 mul BX; //result: AX=839F, DX=0001
 ```

 The result of which is going to be in AX:DX, where the overflow 
 is put in DX (*if any*), letting you get a 32bit result from 
 16bit registers (*or in 64 bit machines you'd get a 128bit 
 result*).

 Most modern languages just ignore the upper result though, 
 which brings us to the following topic. In the 16bit example 
 above you'd be short 64k.

 Too bad we don't have the cent type yet. Otherwise I'd think 
 using ulong and calculating the result and passing that would 
 be the safest, (*assuming malloc would take a 64bit result*), 
 otherwise checking the upper bits for if it's too big.


Cleanly expressing access to the flags sounds quite hard, also 
note that some architectures don't have flags, and some other 
architectures make writing to the flags optional.

ARM chose optional, it might be one of a few things that could 
lead them beating RISC-V in the long run (both as a design 
decision but also as a point about embracing pragmatism)
Dec 31 2021
parent Era Scarecrow <rtcvb32 yahoo.com> writes:
On Saturday, 1 January 2022 at 02:07:36 UTC, max haughton wrote:

 Cleanly expressing access to the flags sounds quite hard, also 
 note that some architectures don't have flags, and some other 
 architectures make writing to the flags optional.


  Ultimately it would mostly be a function of saving the flags. 
After that a struct or other to check specific flags based on 
architecture. Setting said flags doesn't seem useful with how the 
languages are set up, unless you were doing some low-level work 
with loops immediately following.
Dec 31 2021
prev sibling parent Elronnd <elronnd elronnd.net> writes:
On Saturday, 1 January 2022 at 01:44:42 UTC, Era Scarecrow wrote:

 Makes me wish access to the raw multiplication result and carry 
 flags may have been useful here in some way, although the 
 language might not have liked it.


You can check carry using core.checkedint 
(https://dlang.org/phobos/core_checkedint.html).
Dec 31 2021
prev sibling next sibling parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Friday, 31 December 2021 at 00:13:56 UTC, Walter Bright wrote:


 I post this as I've recently seen reports on malware injection 
 being enabled by presenting specially crafted input data to a 
 program that causes an overflow on the allocation, then 
 overwrites the data beyond the truncated allocated memory.


I guess you are talking about this [1] ...

[1] 
https://googleprojectzero.blogspot.com/2021/12/a-deep-dive-into-nso-zero-click.html
Jan 01 2022
parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/1/2022 9:29 AM, Paolo Invernizzi wrote:

 I guess you are talking about this [1] ...
 
 [1] 
 https://googleprojectzero.blogspot.com/2021/12/a-deep-dive-into-nso-zero-click.html


Yes, thank you, that was it. The overflow in the computation of numSyms.
Jan 02 2022
prev sibling parent reply forkit <forkit gmail.com> writes:
On Friday, 31 December 2021 at 00:13:56 UTC, Walter Bright wrote:

 While D offers buffer overflow detection, it does not protect 
 against buffer overflows resulting from an array size 
 calculation overflow:

     T* p = cast(T*)malloc(len * T.sizeof);

 What if `len*T.sizeof` overflows? malloc() will succeed, but 
 the result will be too small for the data.


but the idea that code is only as safe as the functions it calls, 
is not a new idea...right?
Jan 02 2022
parent reply Brian Callahan <bcallah openbsd.org> writes:
OpenBSD has had a function for a long time to deal with this 
exact problem. It's called reallocarray: 
https://man.openbsd.org/reallocarray

Don't let the name fool you--it handles both the initial 
allocation and reallocation.

Perhaps D should provide a similar function (not saying it has to 
be reallocarray). Asking people to fix their own code is a recipe 
for everyone creating different, subtly different and potentially 
incorrect, versions of a problem that should be solved once and 
then used by everyone.

~Brian
Jan 02 2022
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
That sounds similar to my idea of callocSlice but applied to realloc as 
well.

Which absolutely is the solution here I think. Using malloc directly 
should be a red flag.
Jan 02 2022
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Monday, 3 January 2022 at 00:57:55 UTC, rikki cattermole wrote:

 That sounds similar to my idea of callocSlice but applied to 
 realloc as well.

 Which absolutely is the solution here I think. Using malloc 
 directly should be a red flag.


  mhmm.. i think we need a helper function, which is malloc(size, 
count) much like calloc, the test/protections are done in the 
helper. If a slice is returned then bounds checking is 
incorporated when we cast it to something else. So i agree with 
that idea, though using malloc/calloc slice vs malloc/calloc...

  It does make me wonder if my ScaledInt will be incorporated so 
cent/ucent can be used natively (*division being the only real 
complicated and slow part of the code*). Reminds me, i gotta get 
back to the optional crypto instruction set to make it faster.
Jan 02 2022
parent Brian Callahan <bcallah openbsd.org> writes:
On Monday, 3 January 2022 at 01:29:14 UTC, Era Scarecrow wrote:

  mhmm.. i think we need a helper function, which is 
 malloc(size, count) much like calloc, the test/protections are 
 done in the helper.


This is exactly what reallocarray does.
Jan 03 2022
prev sibling next sibling parent reply Adam Ruppe <destructionator gmail.com> writes:
On Monday, 3 January 2022 at 00:52:38 UTC, Brian Callahan wrote:

 Perhaps D should provide a similar function


int[] array;

array.length = 5;
Jan 02 2022
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Monday, 3 January 2022 at 01:29:11 UTC, Adam Ruppe wrote:

 On Monday, 3 January 2022 at 00:52:38 UTC, Brian Callahan wrote:

 Perhaps D should provide a similar function


 ```d
 int[] array;

 array.length = 5;
 ```


I don't think that helps...

```d
long cnt=60, itemsize=100;

array.length = cnt*itemsize; //cannot pass argument `cnt * 
itemsize` of type `long` to parameter `uint newlength`
```

Even if you do your math there and it overflows, it's the same 
problem as using malloc.
Jan 02 2022
parent Era Scarecrow <rtcvb32 yahoo.com> writes:
On Monday, 3 January 2022 at 01:34:38 UTC, Era Scarecrow wrote:

 Even if you do your math there and it overflows, it's the same 
 problem as using malloc.


Except maybe the item size (*in this case*) is already known.
Jan 02 2022
prev sibling parent Elronnd <elronnd elronnd.net> writes:
On Monday, 3 January 2022 at 00:52:38 UTC, Brian Callahan wrote:

 Perhaps D should provide a similar function


What is wrong with Mallocator, mentioned else-thread?
Jan 03 2022