www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - std.stdio.File.seek error

reply stunaep <admin pea2nuts.com> writes:
I have a very large file I need to read data from at certain 
positions, but I have run into this error

 std.conv.ConvOverflowException std\conv.d(1328): Conversion 
 positive overflow


when seeking to 6346890680. Seeking to smaller values such as 
3580720 work with no problem. The file is well over 8GB, so it 
must be able to read data at positions that high.
Mar 13 2016
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 13 March 2016 at 10:32:41 UTC, stunaep wrote:

 I have a very large file I need to read data from at certain 
 positions, but I have run into this error

 std.conv.ConvOverflowException std\conv.d(1328): Conversion 
 positive overflow


 when seeking to 6346890680. Seeking to smaller values such as 
 3580720 work with no problem. The file is well over 8GB, so it 
 must be able to read data at positions that high.


are you on a 32 or 64 bit system?
You could try 2 or more consecutive relative seeks in place of an 
absolute seek.
i.e.
File f = ... ;
f.seek(173445340 , SEEK_SET);
f.seek(173445340 , SEEK_REL);
also what does
f.seek(0,SEEK_END);
writeln(f.tell());
print?
Mar 13 2016
next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
 f.seek(173445340 , SEEK_SET);
 f.seek(173445340 , SEEK_REL);


oops that should be 3173445340.
Mar 13 2016
prev sibling parent reply stunaep <admin pea2nuts.com> writes:
On Sunday, 13 March 2016 at 12:21:11 UTC, Nicholas Wilson wrote:

 On Sunday, 13 March 2016 at 10:32:41 UTC, stunaep wrote:

 I have a very large file I need to read data from at certain 
 positions, but I have run into this error

 std.conv.ConvOverflowException std\conv.d(1328): Conversion 
 positive overflow


 when seeking to 6346890680. Seeking to smaller values such as 
 3580720 work with no problem. The file is well over 8GB, so it 
 must be able to read data at positions that high.


 are you on a 32 or 64 bit system?
 You could try 2 or more consecutive relative seeks in place of 
 an absolute seek.
 i.e.
 File f = ... ;
 f.seek(173445340 , SEEK_SET);
 f.seek(173445340 , SEEK_REL);
 also what does
 f.seek(0,SEEK_END);
 writeln(f.tell());
 print?


I'm on 64 bit but it needs to work on both. It works for anything 
between 0 and 2147483647.


 f.seek(0,SEEK_END);
 writeln(f.tell());


that throws an error before reaching f.tell()


 std.exception.ErrnoException std\stdio.d(920): Could not seek 
 in file `./file.dat' (Invalid argument)
 ----------------
 0x000000013FE67868 in  safe bool 
 std.exception.errnoEnforce!(bool, "std\stdio.d", 
 920uL).errnoEnforce(bool, lazy immutable(char)[])
 0x000000013FE4E7D3 in  trusted void std.stdio.File.seek(long, 
 int)


Also, seeking relative to the current position throws the same 
error as in the original post if it's over max signed int.


 f.seek(2147483647, SEEK_SET);
 writeln(f.tell()); // prints 2147483647
 f.seek(4, SEEK_CUR); // throws error
Mar 13 2016
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Monday, 14 March 2016 at 00:12:46 UTC, stunaep wrote:

 On Sunday, 13 March 2016 at 12:21:11 UTC, Nicholas Wilson wrote:

 On Sunday, 13 March 2016 at 10:32:41 UTC, stunaep wrote:

 I have a very large file I need to read data from at certain 
 positions, but I have run into this error

 [...]


 when seeking to 6346890680. Seeking to smaller values such as 
 3580720 work with no problem. The file is well over 8GB, so 
 it must be able to read data at positions that high.


 are you on a 32 or 64 bit system?
 You could try 2 or more consecutive relative seeks in place of 
 an absolute seek.
 i.e.
 File f = ... ;
 f.seek(173445340 , SEEK_SET);
 f.seek(173445340 , SEEK_REL);
 also what does
 f.seek(0,SEEK_END);
 writeln(f.tell());
 print?


 I'm on 64 bit but it needs to work on both. It works for 
 anything between 0 and 2147483647.


 f.seek(0,SEEK_END);
 writeln(f.tell());


 that throws an error before reaching f.tell()


 std.exception.ErrnoException std\stdio.d(920): Could not seek 
 in file `./file.dat' (Invalid argument)
 ----------------
 0x000000013FE67868 in  safe bool 
 std.exception.errnoEnforce!(bool, "std\stdio.d", 
 920uL).errnoEnforce(bool, lazy immutable(char)[])
 0x000000013FE4E7D3 in  trusted void std.stdio.File.seek(long, 
 int)


 Also, seeking relative to the current position throws the same 
 error as in the original post if it's over max signed int.


 f.seek(2147483647, SEEK_SET);
 writeln(f.tell()); // prints 2147483647
 f.seek(4, SEEK_CUR); // throws error




Hmm. If you're getting an errno exception ( as opposed to a conv) 
I really don't
think that theres anything you can do about it, as its a problem 
with the C standard
lib. What OS/version are you running?

what does the equivalent in C give you.
i.e.

FILE* f = fopen(...,"r");
fseek(f,0,SEEK_END);
printf("%ld",ftell(f));
printf("%d",errno);
Mar 13 2016
parent reply stunaep <admin pea2nuts.com> writes:
On Monday, 14 March 2016 at 03:07:05 UTC, Nicholas Wilson wrote:

 On Monday, 14 March 2016 at 00:12:46 UTC, stunaep wrote:

 On Sunday, 13 March 2016 at 12:21:11 UTC, Nicholas Wilson 
 wrote:

 [...]


 I'm on 64 bit but it needs to work on both. It works for 
 anything between 0 and 2147483647.


 [...]


 that throws an error before reaching f.tell()


 [...]


 Also, seeking relative to the current position throws the same 
 error as in the original post if it's over max signed int.


 [...]




 Hmm. If you're getting an errno exception ( as opposed to a 
 conv) I really don't
 think that theres anything you can do about it, as its a 
 problem with the C standard
 lib. What OS/version are you running?

 what does the equivalent in C give you.
 i.e.

 FILE* f = fopen(...,"r");
 fseek(f,0,SEEK_END);
 printf("%ld",ftell(f));
 printf("%d",errno);


I'm currently on windows 7. The code you gave me prints 022. It's 
weird because it always tries to convert longs to ints and I 
think that is weird because the function uses a c_long.

 int fseek(FILE* stream, c_long offset, int whence)


It wont even compile if I try giving it a long

 Error: function core.stdc.stdio.fseek (shared(_iobuf)* stream, 
 int offset, int whence) is not callable using argument types 
 (shared(_iobuf)*, long, int)


I also tried giving it a c_long, but this line

c_long test = 2147483649;


gives this error for some reason...

 Error: cannot implicitly convert expression (2147483649L) of 
 type long to int


It's like c_longs are not longs at all...
Mar 13 2016
next sibling parent reply stunaep <admin pea2nuts.com> writes:
It seems my lacking knowledge of C has gotten the best of me and 
longs in C only have a signed int range? It looks like in C, 
fseeko() is needed on linux and _fseeki64() is needed on windows, 
but I dont see either of these in stdc.stdio.
Mar 13 2016
parent stunaep <admin pea2nuts.com> writes:
Just tested it on arch linux 64 bit and it works with no problem 
seeking to positions over 2^31-1
Mar 13 2016
prev sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Monday, 14 March 2016 at 05:24:48 UTC, stunaep wrote:

 On Monday, 14 March 2016 at 03:07:05 UTC, Nicholas Wilson wrote:

 [...]


 I'm currently on windows 7. The code you gave me prints 022. 
 It's weird because it always tries to convert longs to ints and 
 I think that is weird because the function uses a c_long.

 [...]


 It wont even compile if I try giving it a long

 [...]


 I also tried giving it a c_long, but this line

[...]


 gives this error for some reason...

 [...]


 It's like c_longs are not longs at all...


c_long maps to the target long which evidently is not 64-bit on 
win7
meaning you can't seek longer than int.max.
  if you cant't find _fseeki64 try looking in the windows section.
Glad you got it working on arch.

Nic
Mar 14 2016
parent reply stunaep <admin pea2nuts.com> writes:
On Monday, 14 March 2016 at 07:15:01 UTC, Nicholas Wilson wrote:

 On Monday, 14 March 2016 at 05:24:48 UTC, stunaep wrote:

 On Monday, 14 March 2016 at 03:07:05 UTC, Nicholas Wilson 
 wrote:

 [...]


 I'm currently on windows 7. The code you gave me prints 022. 
 It's weird because it always tries to convert longs to ints 
 and I think that is weird because the function uses a c_long.

 [...]


 It wont even compile if I try giving it a long

 [...]


 I also tried giving it a c_long, but this line

[...]


 gives this error for some reason...

 [...]


 It's like c_longs are not longs at all...


 c_long maps to the target long which evidently is not 64-bit on 
 win7
 meaning you can't seek longer than int.max.
  if you cant't find _fseeki64 try looking in the windows 
 section.
 Glad you got it working on arch.

 Nic


It looks like _fseeki64 is in the nightly build but not dmd 
2.070.2; However, the nightly build says std.stdio and std.conv 
are deprecated and I cant use them.
Mar 14 2016
parent reply Mike Parker <aldacron gmail.com> writes:
On Monday, 14 March 2016 at 09:57:19 UTC, stunaep wrote:


 It looks like _fseeki64 is in the nightly build but not dmd 
 2.070.2; However, the nightly build says std.stdio and std.conv 
 are deprecated and I cant use them.


I think you may be misinterpreting the error message. There was a 
change recently in how imports are handled in certain cases and 
that may be what you're seeing. What exactly is the error message?
Mar 14 2016
parent reply stunaep <admin pea2nuts.com> writes:
On Monday, 14 March 2016 at 13:33:36 UTC, Mike Parker wrote:

 On Monday, 14 March 2016 at 09:57:19 UTC, stunaep wrote:


 It looks like _fseeki64 is in the nightly build but not dmd 
 2.070.2; However, the nightly build says std.stdio and 
 std.conv are deprecated and I cant use them.


 I think you may be misinterpreting the error message. There was 
 a change recently in how imports are handled in certain cases 
 and that may be what you're seeing. What exactly is the error 
 message?


I'm on my phone but I think It said something like
Deprecation: module std.stdio not accessible from here. Try 
import static std.stdio
Mar 14 2016
next sibling parent Kagamin <spam here.lot> writes:
On Monday, 14 March 2016 at 14:19:27 UTC, stunaep wrote:

 I'm on my phone but I think It said something like
 Deprecation: module std.stdio not accessible from here. Try 
 import static std.stdio


That's fix for bug https://issues.dlang.org/show_bug.cgi?id=313
See the code where std.stdio is not accessible from.
Mar 14 2016
prev sibling parent Mike Parker <aldacron gmail.com> writes:
On Monday, 14 March 2016 at 14:19:27 UTC, stunaep wrote:


 I'm on my phone but I think It said something like
 Deprecation: module std.stdio not accessible from here. Try 
 import static std.stdio


Deprecation: module std.stdio is not accessible here, perhaps add 
'static import std.stdio;'

The solution is right there in the error message. Change your 
import from whatever it is now to a static one. See [1].

[1] http://forum.dlang.org/thread/1835132.hmAZsairJU lyonel
Mar 14 2016