www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - foreach counter now must be size_t ?

reply Johan Engelen <j j.nl> writes:
This code now gives a deprecation message (>= 2.084):
```
void foo(int[] arr) {
     foreach (uint i, ref elem; arr) { }
}
```
Deprecation: foreach: loop index implicitly converted from 
`size_t` to `uint`

This is in contrast to the spec that says that "The index must be 
of int, uint, long or ulong type, it cannot be ref, and it is set 
to be the index of the array element."
https://dlang.org/spec/statement.html#foreach_over_arrays

Did someone forget to update the spec? Or is it a frontend bug?

-Johan
Feb 04
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Feb 04, 2019 at 10:26:08PM +0000, Johan Engelen via Digitalmars-d wrote:
 This code now gives a deprecation message (>= 2.084):
 ```
 void foo(int[] arr) {
     foreach (uint i, ref elem; arr) { }
 }
 ```
 Deprecation: foreach: loop index implicitly converted from `size_t` to
 `uint`
 
 This is in contrast to the spec that says that "The index must be of
 int, uint, long or ulong type, it cannot be ref, and it is set to be
 the index of the array element."
 https://dlang.org/spec/statement.html#foreach_over_arrays
 
 Did someone forget to update the spec? Or is it a frontend bug?
[...] I'd expect the loop index should be of a type that can accomodate the largest possible array index. On 64-bit systems, that would be ulong, which size_t is aliased to when compiling for 64-bit. On 32-bit, size_t would be uint, so the above code should be compilable with -m32. T -- He who does not appreciate the beauty of language is not worthy to bemoan its flaws.
Feb 04
prev sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Monday, 4 February 2019 at 22:26:08 UTC, Johan Engelen wrote:
 This code now gives a deprecation message (>= 2.084):
 ```
 void foo(int[] arr) {
     foreach (uint i, ref elem; arr) { }
 }
 ```
 Deprecation: foreach: loop index implicitly converted from 
 `size_t` to `uint`

 This is in contrast to the spec that says that "The index must 
 be of int, uint, long or ulong type, it cannot be ref, and it 
 is set to be the index of the array element."
 https://dlang.org/spec/statement.html#foreach_over_arrays

 Did someone forget to update the spec? Or is it a frontend bug?

 -Johan
Thats a spec problem. The behaviour is very much deliberate.
Feb 04
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/4/2019 3:01 PM, Nicholas Wilson wrote:
 Thats a spec problem. The behaviour is very much deliberate.
Which PR was it?
Feb 04
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Monday, 4 February 2019 at 23:30:54 UTC, Walter Bright wrote:
 On 2/4/2019 3:01 PM, Nicholas Wilson wrote:
 Thats a spec problem. The behaviour is very much deliberate.
Which PR was it?
https://github.com/dlang/dmd/pull/8941
Feb 04
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/4/2019 4:54 PM, Nicholas Wilson wrote:
 https://github.com/dlang/dmd/pull/8941
Thank you, that's most helpful.
Feb 04
prev sibling next sibling parent Rubn <where is.this> writes:
On Monday, 4 February 2019 at 23:01:06 UTC, Nicholas Wilson wrote:
 On Monday, 4 February 2019 at 22:26:08 UTC, Johan Engelen wrote:
 This code now gives a deprecation message (>= 2.084):
 ```
 void foo(int[] arr) {
     foreach (uint i, ref elem; arr) { }
 }
 ```
 Deprecation: foreach: loop index implicitly converted from 
 `size_t` to `uint`

 This is in contrast to the spec that says that "The index must 
 be of int, uint, long or ulong type, it cannot be ref, and it 
 is set to be the index of the array element."
 https://dlang.org/spec/statement.html#foreach_over_arrays

 Did someone forget to update the spec? Or is it a frontend bug?

 -Johan
Thats a spec problem. The behaviour is very much deliberate.
I remember someone pointing out that foreach_reverse didn't implicitly cast to int, but foreach did. Kind of sad they went to opposite way. Very rarely are you going to have an array that will fill a uint. If your type is only 1 byte that's 4 GB of memory. That's not a very common use case. If it's 4 bytes it grows to 16 GB which is more than the average consumer device has. Having to deal with length array being different for 32 and 64-bit is a pain. I ended up creating a function "length32" to avoid that garbage. If I ever have an array that can fill 32-bit then something has gone very wrong (in my use case).
Feb 04
prev sibling next sibling parent Rubn <where is.this> writes:
On Monday, 4 February 2019 at 23:01:06 UTC, Nicholas Wilson wrote:
 On Monday, 4 February 2019 at 22:26:08 UTC, Johan Engelen wrote:
 This code now gives a deprecation message (>= 2.084):
 ```
 void foo(int[] arr) {
     foreach (uint i, ref elem; arr) { }
 }
 ```
 Deprecation: foreach: loop index implicitly converted from 
 `size_t` to `uint`

 This is in contrast to the spec that says that "The index must 
 be of int, uint, long or ulong type, it cannot be ref, and it 
 is set to be the index of the array element."
 https://dlang.org/spec/statement.html#foreach_over_arrays

 Did someone forget to update the spec? Or is it a frontend bug?

 -Johan
Thats a spec problem. The behaviour is very much deliberate.
Same goes for "sizeof32" which is especially annoying when sometimes it evaluates to int because the value is known at the time and other times it evaluates to size_t cause it can't determine the value right away or is being used at runtime. If you a struct that can fill a uint something has gone extremely wrong and your executable file is going to be 4 GB minimum to accomodate the .init data.
Feb 04
prev sibling next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Monday, 4 February 2019 at 23:01:06 UTC, Nicholas Wilson wrote:
 On Monday, 4 February 2019 at 22:26:08 UTC, Johan Engelen wrote:
 This code now gives a deprecation message (>= 2.084):
 ```
 void foo(int[] arr) {
     foreach (uint i, ref elem; arr) { }
 }
 ```
 Deprecation: foreach: loop index implicitly converted from 
 `size_t` to `uint`

 This is in contrast to the spec that says that "The index must 
 be of int, uint, long or ulong type, it cannot be ref, and it 
 is set to be the index of the array element."
 https://dlang.org/spec/statement.html#foreach_over_arrays

 Did someone forget to update the spec? Or is it a frontend bug?

 -Johan
Thats a spec problem. The behaviour is very much deliberate.
https://github.com/dlang/dlang.org/pull/2565/
Feb 04
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 05.02.19 00:01, Nicholas Wilson wrote:

 
 Thats a spec problem. The behaviour is very much deliberate.
I wish it wasn't. It seems this doesn't do anything useful. If I'm going through the trouble of explicitly specifying the counter type to be what I need, why annoy me with this error message?
Feb 04
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 5 February 2019 at 03:15:03 UTC, Timon Gehr wrote:
 I wish it wasn't. It seems this doesn't do anything useful. If 
 I'm going through the trouble of explicitly specifying the 
 counter type to be what I need, why annoy me with this error 
 message?
Ditto. This change is bewildering to me.
Feb 04
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/4/2019 7:15 PM, Timon Gehr wrote:
 I wish it wasn't. It seems this doesn't do anything useful. If I'm going
through 
 the trouble of explicitly specifying the counter type to be what I need, why 
 annoy me with this error message?
It is the same thing as: long l; uint i = l; // error D doesn't allow implicit narrowing conversions that lose information. If the foreach was written like this: void foo(int[3] arr) { foreach (uint i, ref elem; arr) { } } I.e. where the array bounds are known at compile time, then it works, because: enum long l = 3; uint i = l; also works. As for "who would ever have an array that big": 1. It's irrelevant, as the semantics of the language have to be sound across the uses it supports. 2. I noticed with Windows MovieMaker that it produces corrupt output files if the movie is bigger than 4Gb. Evidently a 32 bit counter isn't enough.
Feb 04
parent reply Rubn <where is.this> writes:
On Tuesday, 5 February 2019 at 04:53:00 UTC, Walter Bright wrote:
 On 2/4/2019 7:15 PM, Timon Gehr wrote:
 I wish it wasn't. It seems this doesn't do anything useful. If 
 I'm going through the trouble of explicitly specifying the 
 counter type to be what I need, why annoy me with this error 
 message?
It is the same thing as: long l; uint i = l; // error D doesn't allow implicit narrowing conversions that lose information. If the foreach was written like this: void foo(int[3] arr) { foreach (uint i, ref elem; arr) { } } I.e. where the array bounds are known at compile time, then it works, because: enum long l = 3; uint i = l; also works.
WHAT. WHAT. Where do you see an assignment from size_t to int here? int[] arr = someRandomArray(); for( int i = 0; i < arr.length; ++i ) { dg( i, arr[i] ); } ??? There is only a comparison between `int` and `size_t` which is completely valid. And as far as I remember you strictly disapprove of even allowing a warning to be displayed in that case.
 As for "who would ever have an array that big":

 1. It's irrelevant, as the semantics of the language have to be 
 sound across the uses it supports.
If by semantics you are referring to the above, then see the above. Otherwise your misguided attempt to try and illustrate that a size_t can't be assigned to an int. Foreach is a special case. foreach(i, ref int v; someArray) { // i is size_t } Can you do the following? i = 10; No you can't, cause they aren't the same thing! If you want to stick with that, I can do a case without creating another variable. How do I do that with foreach so I don't have to do this: foreach( i, ref int value; arr ) { int i = cast(int)i; } foreach( int i, ref int value; arr ) { // better }
 2. I noticed with Windows MovieMaker that it produces corrupt 
 output files if the movie is bigger than 4Gb. Evidently a 32 
 bit counter isn't enough.
1. That's not even remotely related. The error in Movie Maker is caused by the C api (on Windows) uses 32-bit integers. 2. If they were using D and for some reason they wanted to create the entire file in memory first before writing it to a file. There still would have been the same problem. If you build on 32-bit size_t is int, so they would be limited to 4 GB files. Not to mention how horrible it would be on memory, but I know you and DMD love your memory leaks (never running the garbage collector nor freeing the memory used) and not distributing 64-bit binaries, but I'll leave it at that maybe you would fill an entire array before writing it to a file. 3. Creating such a huge array in the first place is just a bad idea. You get incompatibilities with 32-bit and 64-bit, and most of the time you don't need to do it. It's a code smell honestly. You should probably have a structure that specifically deals with gigantic arrays, to disable things like copying the array. Though again you'll almost never need it except for very special cases. 4. I never said to NEVER use a 64-bit integer, merely that you almost never need an array that would fill a 32-bit length (your Movie Maker example has not disproved that). Hell you can make DMD use 32-bit integers for all it's arrays and it will still function even though it runs out of memory (not because it allocates an array that fills 32-bit counter though) because it is one gigantic memory leak. I know it still works with 32-bit cause there is no 64-bit version for Windows! Classic.
Feb 05
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/5/2019 1:14 PM, Rubn wrote:
 WHAT. WHAT. Where do you see an assignment from size_t to int here?
 
 int[] arr = someRandomArray();
 for( int i = 0; i < arr.length; ++i ) {
     dg( i, arr[i] );
 }
 
 ???
The ++i overflows and the loop never terminates.
Feb 05
next sibling parent Rubn <where is.this> writes:
On Tuesday, 5 February 2019 at 23:04:16 UTC, Walter Bright wrote:
 On 2/5/2019 1:14 PM, Rubn wrote:
 WHAT. WHAT. Where do you see an assignment from size_t to int 
 here?
 
 int[] arr = someRandomArray();
 for( int i = 0; i < arr.length; ++i ) {
     dg( i, arr[i] );
 }
 
 ???
The ++i overflows and the loop never terminates.
I'm saying SEMANTICALLY it is correct.
Feb 05
prev sibling next sibling parent Rubn <where is.this> writes:
On Tuesday, 5 February 2019 at 23:04:16 UTC, Walter Bright wrote:
 On 2/5/2019 1:14 PM, Rubn wrote:
 WHAT. WHAT. Where do you see an assignment from size_t to int 
 here?
 
 int[] arr = someRandomArray();
 for( int i = 0; i < arr.length; ++i ) {
     dg( i, arr[i] );
 }
 
 ???
The ++i overflows and the loop never terminates.
Just as it is safe. safe void foo(int[] a) { for(int i = 0; i < a.length; ++i ) { writeln( i ); } }
Feb 05
prev sibling parent Rubn <where is.this> writes:
On Tuesday, 5 February 2019 at 23:04:16 UTC, Walter Bright wrote:
 On 2/5/2019 1:14 PM, Rubn wrote:
 WHAT. WHAT. Where do you see an assignment from size_t to int 
 here?
 
 int[] arr = someRandomArray();
 for( int i = 0; i < arr.length; ++i ) {
     dg( i, arr[i] );
 }
 
 ???
The ++i overflows and the loop never terminates.
In case you forgot what you said:
 It's irrelevant, as the semantics of the language have to be 
 sound across the uses it supports.
This change to not allow int goes against the semantics of the language.
Feb 05
prev sibling next sibling parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Tuesday, 5 February 2019 at 21:14:17 UTC, Rubn wrote:
 On Tuesday, 5 February 2019 at 04:53:00 UTC, Walter Bright 
 wrote:
 [...]
WHAT. WHAT. Where do you see an assignment from size_t to int here? [...]
Memory mapping archives and video files can easily get beyond 32 bits. It's not because you never noticed that there aren't people out there doing that every day.
 [...]
Feb 05
parent reply Rubn <where is.this> writes:
On Wednesday, 6 February 2019 at 06:27:27 UTC, Patrick Schluter 
wrote:
 On Tuesday, 5 February 2019 at 21:14:17 UTC, Rubn wrote:
 On Tuesday, 5 February 2019 at 04:53:00 UTC, Walter Bright 
 wrote:
 [...]
WHAT. WHAT. Where do you see an assignment from size_t to int here? [...]
Memory mapping archives and video files can easily get beyond 32 bits. It's not because you never noticed that there aren't people out there doing that every day.
 [...]
I never knew arrays were files, thanks for the tip.
Feb 06
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Feb 06, 2019 at 08:59:14PM +0000, Rubn via Digitalmars-d wrote:
 On Wednesday, 6 February 2019 at 06:27:27 UTC, Patrick Schluter wrote:
[...]
 Memory mapping archives and video files can easily get beyond 32
 bits.  It's not because you never noticed that there aren't people
 out there doing that every day.
[...]
 I never knew arrays were files, thanks for the tip.
As you may know, mmap() turns files into arrays (see std.mmfile). You access it just like an array, and the OS maps it to the file for you. You wouldn't want somebody to pass the array returned by an mmap() call to your code that assumes arrays cannot have length larger than 32-bits -- you'd run into wraparound bugs. Just because the array has a huge size doesn't necessarily mean you need that much RAM to hold it. The OS pages parts of it in/out as you access it. T -- If blunt statements had a point, they wouldn't be blunt...
Feb 06
next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 6 February 2019 at 21:23:20 UTC, H. S. Teoh wrote:
 [snip]

 As you may know, mmap() turns files into arrays (see 
 std.mmfile).  You
 access it just like an array, and the OS maps it to the file 
 for you.

 You wouldn't want somebody to pass the array returned by an 
 mmap() call to your code that assumes arrays cannot have length 
 larger than 32-bits -- you'd run into wraparound bugs.

 Just because the array has a huge size doesn't necessarily mean 
 you need that much RAM to hold it. The OS pages parts of it 
 in/out as you access it.


 T
mmap is something that I never really played around much with, but it seems like it would be useful for me in some cases. I've done some Monte Carlo simulations that end up causing me to run out of memory. Doing it as an mmap instead might be slower but at least I wouldn't run out of memory.
Feb 06
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Feb 06, 2019 at 10:00:35PM +0000, jmh530 via Digitalmars-d wrote:
[...]
 mmap is something that I never really played around much with, but it
 seems like it would be useful for me in some cases. I've done some
 Monte Carlo simulations that end up causing me to run out of memory.
 Doing it as an mmap instead might be slower but at least I wouldn't
 run out of memory.
It depends on how big your resident set size is, and what your memory access patterns are. If you're using the mmap'd memory in a random-access pattern, be prepared for your OS to slow down to a crawl as it begins thrashing on I/O swapping pages from disk only to evict them milliseconds later because there's no more memory to load in the page for the next random access. There's no free lunch. :-P That's why in this day and age people are moving away from hashtables, once hailed as the holy grail of data structure design, to structures based on linear access that have "worse" big-O complexity but better real-world performance, because linear access plays nicer with the cache hierarchy. T -- "How are you doing?" "Doing what?"
Feb 06
parent jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 6 February 2019 at 22:29:11 UTC, H. S. Teoh wrote:
 [snip]
Very informative. Thanks.
Feb 06
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/6/2019 2:00 PM, jmh530 wrote:
 mmap is something that I never really played around much with, but it seems
like 
 it would be useful for me in some cases. I've done some Monte Carlo
simulations 
 that end up causing me to run out of memory. Doing it as an mmap instead might 
 be slower but at least I wouldn't run out of memory.
Executables are not loaded into memory before running them, they are mmap'ed into memory. Digital Mars C++ used mmap for precompiled headers. There are all kinds of uses, it's a very effective tool.
Feb 06
parent reply Rubn <where is.this> writes:
On Thursday, 7 February 2019 at 03:20:03 UTC, Walter Bright wrote:
 On 2/6/2019 2:00 PM, jmh530 wrote:
 mmap is something that I never really played around much with, 
 but it seems like it would be useful for me in some cases. 
 I've done some Monte Carlo simulations that end up causing me 
 to run out of memory. Doing it as an mmap instead might be 
 slower but at least I wouldn't run out of memory.
Executables are not loaded into memory before running them, they are mmap'ed into memory. Digital Mars C++ used mmap for precompiled headers. There are all kinds of uses, it's a very effective tool.
I'll take that to mean you don't think this is a bug: safe void foo(int[] a) { for(int i = 0; i < a.length; ++i ) { writeln( i ); } } Fucking classic.
Feb 07
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Thursday, February 7, 2019 4:02:14 PM MST Rubn via Digitalmars-d wrote:
 On Thursday, 7 February 2019 at 03:20:03 UTC, Walter Bright wrote:
 On 2/6/2019 2:00 PM, jmh530 wrote:
 mmap is something that I never really played around much with,
 but it seems like it would be useful for me in some cases.
 I've done some Monte Carlo simulations that end up causing me
 to run out of memory. Doing it as an mmap instead might be
 slower but at least I wouldn't run out of memory.
Executables are not loaded into memory before running them, they are mmap'ed into memory. Digital Mars C++ used mmap for precompiled headers. There are all kinds of uses, it's a very effective tool.
I'll take that to mean you don't think this is a bug: safe void foo(int[] a) { for(int i = 0; i < a.length; ++i ) { writeln( i ); } } Fucking classic.
If you're trying to say that you think that that's a compiler bug, then you misunderstand safe. safe code cannot access invalid memory. It cannot have undefined behavior. But it can have plenty of logic bugs, and what that code has is a logic bug. It's at zero risk of accessing invalid memory. It's just going to do the wrong thing if the array is sufficiently large. The only way that array size issues are an safe-related bug is if you have code that somehow manages to access the array out-of-bounds in spite of the code being safe. It's been argued before that comparing integer types of different size or signedness should not be legal, because it's a source of bugs, and maybe it shouldn't be legal, but even if that's true, it's still not an issue with safe. - Jonathan M Davis
Feb 07
parent reply Rubn <where is.this> writes:
On Thursday, 7 February 2019 at 23:37:06 UTC, Jonathan M Davis 
wrote:
 On Thursday, February 7, 2019 4:02:14 PM MST Rubn via 
 Digitalmars-d wrote:
 On Thursday, 7 February 2019 at 03:20:03 UTC, Walter Bright 
 wrote:
 On 2/6/2019 2:00 PM, jmh530 wrote:
 mmap is something that I never really played around much 
 with, but it seems like it would be useful for me in some 
 cases. I've done some Monte Carlo simulations that end up 
 causing me to run out of memory. Doing it as an mmap 
 instead might be slower but at least I wouldn't run out of 
 memory.
Executables are not loaded into memory before running them, they are mmap'ed into memory. Digital Mars C++ used mmap for precompiled headers. There are all kinds of uses, it's a very effective tool.
I'll take that to mean you don't think this is a bug: safe void foo(int[] a) { for(int i = 0; i < a.length; ++i ) { writeln( i ); } } Fucking classic.
If you're trying to say that you think that that's a compiler bug, then you misunderstand safe. safe code cannot access invalid memory. It cannot have undefined behavior. But it can have plenty of logic bugs, and what that code has is a logic bug. It's at zero risk of accessing invalid memory. It's just going to do the wrong thing if the array is sufficiently large. The only way that array size issues are an safe-related bug is if you have code that somehow manages to access the array out-of-bounds in spite of the code being safe. It's been argued before that comparing integer types of different size or signedness should not be legal, because it's a source of bugs, and maybe it shouldn't be legal, but even if that's true, it's still not an issue with safe. - Jonathan M Davis
Good you agree it isn't a bug. Then we should remove this deprecation right?
Feb 08
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, February 8, 2019 5:05:09 PM MST Rubn via Digitalmars-d wrote:
 On Thursday, 7 February 2019 at 23:37:06 UTC, Jonathan M Davis

 wrote:
 On Thursday, February 7, 2019 4:02:14 PM MST Rubn via

 Digitalmars-d wrote:
 On Thursday, 7 February 2019 at 03:20:03 UTC, Walter Bright

 wrote:
 On 2/6/2019 2:00 PM, jmh530 wrote:
 mmap is something that I never really played around much
 with, but it seems like it would be useful for me in some
 cases. I've done some Monte Carlo simulations that end up
 causing me to run out of memory. Doing it as an mmap
 instead might be slower but at least I wouldn't run out of
 memory.
Executables are not loaded into memory before running them, they are mmap'ed into memory. Digital Mars C++ used mmap for precompiled headers. There are all kinds of uses, it's a very effective tool.
I'll take that to mean you don't think this is a bug: safe void foo(int[] a) { for(int i = 0; i < a.length; ++i ) { writeln( i ); } } Fucking classic.
If you're trying to say that you think that that's a compiler bug, then you misunderstand safe. safe code cannot access invalid memory. It cannot have undefined behavior. But it can have plenty of logic bugs, and what that code has is a logic bug. It's at zero risk of accessing invalid memory. It's just going to do the wrong thing if the array is sufficiently large. The only way that array size issues are an safe-related bug is if you have code that somehow manages to access the array out-of-bounds in spite of the code being safe. It's been argued before that comparing integer types of different size or signedness should not be legal, because it's a source of bugs, and maybe it shouldn't be legal, but even if that's true, it's still not an issue with safe. - Jonathan M Davis
Good you agree it isn't a bug. Then we should remove this deprecation right?
It's not a compiler bug, but the code is still buggy. Using int for indexing is wrong in the general case. It works for smaller arrays, but it's just a bug waiting to happen. I see no problem with the deprecation. Quite the opposite. Using int when size_t should be used is an incredibly common bug, and this helps combat that. - Jonathan M Davis
Feb 08
next sibling parent Rubn <where is.this> writes:
On Saturday, 9 February 2019 at 02:13:13 UTC, Jonathan M Davis 
wrote:
 On Friday, February 8, 2019 5:05:09 PM MST Rubn via 
 Digitalmars-d wrote:
 On Thursday, 7 February 2019 at 23:37:06 UTC, Jonathan M Davis

 wrote:
 On Thursday, February 7, 2019 4:02:14 PM MST Rubn via

 Digitalmars-d wrote:
 On Thursday, 7 February 2019 at 03:20:03 UTC, Walter Bright

 wrote:
 On 2/6/2019 2:00 PM, jmh530 wrote:
 mmap is something that I never really played around much 
 with, but it seems like it would be useful for me in 
 some cases. I've done some Monte Carlo simulations that 
 end up causing me to run out of memory. Doing it as an 
 mmap instead might be slower but at least I wouldn't run 
 out of memory.
Executables are not loaded into memory before running them, they are mmap'ed into memory. Digital Mars C++ used mmap for precompiled headers. There are all kinds of uses, it's a very effective tool.
I'll take that to mean you don't think this is a bug: safe void foo(int[] a) { for(int i = 0; i < a.length; ++i ) { writeln( i ); } } Fucking classic.
If you're trying to say that you think that that's a compiler bug, then you misunderstand safe. safe code cannot access invalid memory. It cannot have undefined behavior. But it can have plenty of logic bugs, and what that code has is a logic bug. It's at zero risk of accessing invalid memory. It's just going to do the wrong thing if the array is sufficiently large. The only way that array size issues are an safe-related bug is if you have code that somehow manages to access the array out-of-bounds in spite of the code being safe. It's been argued before that comparing integer types of different size or signedness should not be legal, because it's a source of bugs, and maybe it shouldn't be legal, but even if that's true, it's still not an issue with safe. - Jonathan M Davis
Good you agree it isn't a bug. Then we should remove this deprecation right?
It's not a compiler bug, but the code is still buggy. Using int for indexing is wrong in the general case. It works for smaller arrays, but it's just a bug waiting to happen. I see no problem with the deprecation. Quite the opposite. Using int when size_t should be used is an incredibly common bug, and this helps combat that. - Jonathan M Davis
It's a bandaid, the larger problem is allowing the comparison between int and size_t. Which my bet is how it is implemented internally, which is why foreach_reverse doesn't work cause it actually has to assign the counter a size_t.
Feb 12
prev sibling parent reply Rubn <where is.this> writes:
On Saturday, 9 February 2019 at 02:13:13 UTC, Jonathan M Davis 
wrote:
 On Friday, February 8, 2019 5:05:09 PM MST Rubn via 
 Digitalmars-d wrote:
 On Thursday, 7 February 2019 at 23:37:06 UTC, Jonathan M Davis

 wrote:
 On Thursday, February 7, 2019 4:02:14 PM MST Rubn via

 Digitalmars-d wrote:
 On Thursday, 7 February 2019 at 03:20:03 UTC, Walter Bright

 wrote:
 On 2/6/2019 2:00 PM, jmh530 wrote:
 mmap is something that I never really played around much 
 with, but it seems like it would be useful for me in 
 some cases. I've done some Monte Carlo simulations that 
 end up causing me to run out of memory. Doing it as an 
 mmap instead might be slower but at least I wouldn't run 
 out of memory.
Executables are not loaded into memory before running them, they are mmap'ed into memory. Digital Mars C++ used mmap for precompiled headers. There are all kinds of uses, it's a very effective tool.
I'll take that to mean you don't think this is a bug: safe void foo(int[] a) { for(int i = 0; i < a.length; ++i ) { writeln( i ); } } Fucking classic.
If you're trying to say that you think that that's a compiler bug, then you misunderstand safe. safe code cannot access invalid memory. It cannot have undefined behavior. But it can have plenty of logic bugs, and what that code has is a logic bug. It's at zero risk of accessing invalid memory. It's just going to do the wrong thing if the array is sufficiently large. The only way that array size issues are an safe-related bug is if you have code that somehow manages to access the array out-of-bounds in spite of the code being safe. It's been argued before that comparing integer types of different size or signedness should not be legal, because it's a source of bugs, and maybe it shouldn't be legal, but even if that's true, it's still not an issue with safe. - Jonathan M Davis
Good you agree it isn't a bug. Then we should remove this deprecation right?
It's not a compiler bug, but the code is still buggy. Using int for indexing is wrong in the general case. It works for smaller arrays, but it's just a bug waiting to happen. I see no problem with the deprecation. Quite the opposite. Using int when size_t should be used is an incredibly common bug, and this helps combat that. - Jonathan M Davis
The point was, if you don't see the for() being an issue that has to be fixed. Then there's no reason to see foreach() as an issue to fix. You either fix them both or don't, it doesn't make sense to fix one but not the other. foreach()'s implementation is literally just a for() statement which is where the bug originates from.
Feb 12
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 13 February 2019 at 01:54:35 UTC, Rubn wrote:
 On Saturday, 9 February 2019 at 02:13:13 UTC, Jonathan M Davis
 It's not a compiler bug, but the code is still buggy. Using 
 int for indexing is wrong in the general case. It works for 
 smaller arrays, but it's just a bug waiting to happen. I see 
 no problem with the deprecation. Quite the opposite. Using int 
 when size_t should be used is an incredibly common bug, and 
 this helps combat that.

 - Jonathan M Davis
It's a bandaid, the larger problem is allowing the comparison between int and size_t. Which my bet is how it is implemented internally, which is why foreach_reverse doesn't work cause it actually has to assign the counter a size_t. The point was, if you don't see the for() being an issue that has to be fixed. Then there's no reason to see foreach() as an issue to fix. You either fix them both or don't, it doesn't make sense to fix one but not the other. foreach()'s implementation is literally just a for() statement which is where the bug originates from.
The `for` is an issue, but one that is much harder to reason about. With `foreach` over arrays, the index is going to be used for indexing. With for, the counter (if any) is assigned separately and _may_ be used in an arbitrary stopping condition and the indexing (if any) is in the loop.
Feb 12
prev sibling parent reply Kagamin <spam here.lot> writes:
On Saturday, 9 February 2019 at 00:05:09 UTC, Rubn wrote:
 Good you agree it isn't a bug. Then we should remove this 
 deprecation right?
The entire ecosystem is locked on unsigned integers, you can't just pull them out.
Feb 18
parent Rubn <where is.this> writes:
On Monday, 18 February 2019 at 11:43:11 UTC, Kagamin wrote:
 On Saturday, 9 February 2019 at 00:05:09 UTC, Rubn wrote:
 Good you agree it isn't a bug. Then we should remove this 
 deprecation right?
The entire ecosystem is locked on unsigned integers, you can't just pull them out.
This feature is currently in DMD, and they are breaking code to change the behavior to not include it. I have no idea what you are talking about.
Feb 18
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/7/2019 3:02 PM, Rubn wrote:
 [...] classic.
Use of such words will result in deletion of the posts. Please use professional demeanor on these forums.
Feb 07
parent Rubn <where is.this> writes:
On Friday, 8 February 2019 at 06:33:00 UTC, Walter Bright wrote:
 On 2/7/2019 3:02 PM, Rubn wrote:
 [...] classic.
Use of such words will result in deletion of the posts. Please use professional demeanor on these forums.
You can go ahead and delete the post if you want. It'd be no different than not answer a valid question :), thought maybe mildly better.
Feb 08
prev sibling next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 7 February 2019 at 23:02:14 UTC, Rubn wrote:
 I'll take that to mean you don't think this is a bug:

  safe void foo(int[] a) {
     for(int i = 0; i < a.length; ++i ) {
        writeln( i );
     }
 }

 Fucking classic.
Its not a bug. In the case that a.length > int.max, the loop will not terminate and only print indices. If the loop was for(int i = 0; i < a.length; ++i ) { writeln( a[i] ); } it would still be safe, but the program would crash because the index would (eventually) be OOB. If a.length > uint.max and the loop was for(uint i = 0; i < a.length; ++i ) { writeln( a[i] ); } then the loop fail to terminate, and it would still be safe. All the above problems are avoided using size_t as the index.
Feb 08
parent reply Rubn <where is.this> writes:
On Friday, 8 February 2019 at 08:06:39 UTC, Nicholas Wilson wrote:
 On Thursday, 7 February 2019 at 23:02:14 UTC, Rubn wrote:
 I'll take that to mean you don't think this is a bug:

  safe void foo(int[] a) {
     for(int i = 0; i < a.length; ++i ) {
        writeln( i );
     }
 }

 Fucking classic.
Its not a bug. In the case that a.length > int.max, the loop will not terminate and only print indices. If the loop was for(int i = 0; i < a.length; ++i ) { writeln( a[i] ); } it would still be safe, but the program would crash because the index would (eventually) be OOB. If a.length > uint.max and the loop was for(uint i = 0; i < a.length; ++i ) { writeln( a[i] ); } then the loop fail to terminate, and it would still be safe. All the above problems are avoided using size_t as the index.
TIL infinite loop that freezes program isn't a bug.
Feb 08
parent Norm <norm.rowtree gmail.com> writes:
On Saturday, 9 February 2019 at 00:15:41 UTC, Rubn wrote:
 On Friday, 8 February 2019 at 08:06:39 UTC, Nicholas Wilson 
 wrote:
 On Thursday, 7 February 2019 at 23:02:14 UTC, Rubn wrote:
 [...]
Its not a bug. In the case that a.length > int.max, the loop will not terminate and only print indices. If the loop was for(int i = 0; i < a.length; ++i ) { writeln( a[i] ); } it would still be safe, but the program would crash because the index would (eventually) be OOB. If a.length > uint.max and the loop was for(uint i = 0; i < a.length; ++i ) { writeln( a[i] ); } then the loop fail to terminate, and it would still be safe. All the above problems are avoided using size_t as the index.
TIL infinite loop that freezes program isn't a bug.
It is a bug in your code, not the safe.
Feb 08
prev sibling parent reply Olivier FAURE <couteaubleu gmail.com> writes:
On Thursday, 7 February 2019 at 23:02:14 UTC, Rubn wrote:
 Fucking classic.
Specific language aside, there's a proper way to bring bugs to a maintainer's attention, and telling them "Oh, so you don't think [this bug I hate] is important enough for you? Typical..." out of nowhere is not it. The maintainers have a responsibility to the community, not to you personally. Acting passive-aggressive because they haven't yet fixed your pet issue is incredibly entitled and unprofessional. Please don't. That kind of behavior is common, but it's *not* acceptable.
Feb 08
parent reply Rubn <where is.this> writes:
On Friday, 8 February 2019 at 20:22:29 UTC, Olivier FAURE wrote:
 The maintainers have a responsibility to the community, not to 
 you personally.
Guess you weren't around, Walter's opinion on the matter is they have responsibility to ducking no one.
Feb 08
parent Olivier FAURE <couteaubleu gmail.com> writes:
On Saturday, 9 February 2019 at 00:12:47 UTC, Rubn wrote:
 On Friday, 8 February 2019 at 20:22:29 UTC, Olivier FAURE wrote:
 The maintainers have a responsibility to the community, not to 
 you personally.
Guess you weren't around, Walter's opinion on the matter is they have responsibility to ducking no one.
Regardless. Your answers in this thread don't need to be this aggressive. Just because people don't agree with you on how to address an issue doesn't mean you have to patronize them.
Feb 14
prev sibling parent Rubn <where is.this> writes:
On Wednesday, 6 February 2019 at 21:23:20 UTC, H. S. Teoh wrote:
 On Wed, Feb 06, 2019 at 08:59:14PM +0000, Rubn via 
 Digitalmars-d wrote:
 On Wednesday, 6 February 2019 at 06:27:27 UTC, Patrick 
 Schluter wrote:
[...]
 Memory mapping archives and video files can easily get 
 beyond 32 bits.  It's not because you never noticed that 
 there aren't people out there doing that every day.
[...]
 I never knew arrays were files, thanks for the tip.
As you may know, mmap() turns files into arrays (see std.mmfile). You access it just like an array, and the OS maps it to the file for you. You wouldn't want somebody to pass the array returned by an mmap() call to your code that assumes arrays cannot have length larger than 32-bits -- you'd run into wraparound bugs.
Not everyone is writing generic code. Especially in performance critical code. Like I was saying, this is one those very rare use cases anyways. And AGAIN if you have a memory mapped file in a 32-bit executable, this will not function. Lol man the implementation of mmfile. The interface all uses ulong but then it casts everything to size_t, that's a minefield on a 32-bit system. Good luck accessing a > 4GB file with that.
 Just because the array has a huge size doesn't necessarily mean 
 you need that much RAM to hold it. The OS pages parts of it 
 in/out as you access it.
Which will be extremely slow. That might be fine for testing something but you'll be better off writing your own code to handle processing that much data in a way that is most efficient for whatever it is you are doing.
Feb 06
prev sibling parent Tony <tonytdominguez aol.com> writes:
On Tuesday, 5 February 2019 at 21:14:17 UTC, Rubn wrote:
 On Tuesday, 5 February 2019 at 04:53:00 UTC, Walter Bright 
 wrote:
 I.e. where the array bounds are known at compile time, then it 
 works, because:

     enum long l = 3;
     uint i = l;

 also works.
WHAT. WHAT. Where do you see an assignment from size_t to int here? int[] arr = someRandomArray(); for( int i = 0; i < arr.length; ++i ) { dg( i, arr[i] ); } ???
Seems that the size_t would have to be a compiler variable we don't see, but would be nice to have this answered.
Feb 06
prev sibling parent reply Dukc <ajieskola gmail.com> writes:
On Tuesday, 5 February 2019 at 03:15:03 UTC, Timon Gehr wrote:
 I wish it wasn't. It seems this doesn't do anything useful
It does! Even I, who contribute only occasionally, have more than once submitted a PR to Phobos that passes my local 32-bit tests, but fails on the GitHub autotester. And the reason is precisely because I tend to accidently assign size_t to int or uint. With this change, I can now immediately detect when I make that error, and I don't think I'm the only 32-bit user who makes them.
Feb 05
next sibling parent 0xEAB <desisma heidel.beer> writes:
On Tuesday, 5 February 2019 at 11:53:13 UTC, Dukc wrote:
 With this change, I can now immediately detect when I make that 
 error, and I don't think I'm the only 32-bit user who makes 
 them.
In one of my first D program I used `int` to store some index or something. First time trying -m64 broke everything then... - Elias
Feb 05
prev sibling parent reply Rubn <where is.this> writes:
On Tuesday, 5 February 2019 at 11:53:13 UTC, Dukc wrote:
 On Tuesday, 5 February 2019 at 03:15:03 UTC, Timon Gehr wrote:
 I wish it wasn't. It seems this doesn't do anything useful
It does! Even I, who contribute only occasionally, have more than once submitted a PR to Phobos that passes my local 32-bit tests, but fails on the GitHub autotester. And the reason is precisely because I tend to accidently assign size_t to int or uint. With this change, I can now immediately detect when I make that error, and I don't think I'm the only 32-bit user who makes them.
Nope this will still work on 32-bit: foreach(uint i, ref int value; arr ) { } size_t is just an alias for uint on 32-bit and ulong on 64-bit.
Feb 05
parent reply Dukc <ajieskola gmail.com> writes:
On Tuesday, 5 February 2019 at 21:16:12 UTC, Rubn wrote:
 Nope this will still work on 32-bit:

    foreach(uint i, ref int value; arr )
    {
    }

 size_t is just an alias for uint on 32-bit and ulong on 64-bit.
If so, too bad. But it'll still help, because if I try to compile to 64-bit, the compiler will catch the error. Without the deprecation, there would be a bug (due to 32-bit overflow) that likely never manifests, but bites all the harder if it still does.
Feb 05
parent Kagamin <spam here.lot> writes:
On Tuesday, 5 February 2019 at 21:51:41 UTC, Dukc wrote:
 If so, too bad. But it'll still help, because if I try to 
 compile to 64-bit, the compiler will catch the error. Without 
 the deprecation, there would be a bug (due to 32-bit overflow) 
 that likely never manifests, but bites all the harder if it 
 still does.
There's no limit to error checks, do you compile with warnings enabled? See https://dlang.org/articles/warnings.html
Feb 06