www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 3409] New: stdio.File.seek() doesn't work for files >2GB

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3409

           Summary: stdio.File.seek() doesn't work for files >2GB
           Product: D
           Version: 2.035
          Platform: Other
        OS/Version: Windows
            Status: NEW
          Keywords: patch
          Severity: major
          Priority: P2
         Component: Phobos
        AssignedTo: nobody puremagic.com
        ReportedBy: dsimcha yahoo.com



stdio.File.seek() doesn't work for large files.  This is because fseek() is
using an int to represent the offset because it's crufty old C code from the
Stone Age.  Instead, it throws a ConvOverflowError exception on trying to seek
a long distance (> int.max).  The proper fix would be to use the C function
that takes a long instead, but this is apparently not available on all
platforms and not part of the C standard.  

I've attached a file with an implementation of a kludge to work around this
issue by seeking incrementally.  It's ugly but it works.  It's implemented in
terms of the current seek() implementation, and I haven't integrated it into
the File struct, but the basic code is there.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 16 2009
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3409




Oops, looks like I forgot to attach the attachment.  That's ok, on further
testing, it only solved the problem up to 4GB anyhow.  I guess what *really*
needs to happen is LFS support.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 16 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3409


David Simcha <dsimcha yahoo.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|patch                       |



Note that std.stream gets this right on Win32 but not Linux by using the Win32
API directly.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 17 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3409


Andrei Alexandrescu <andrei metalanguage.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
                 CC|                            |andrei metalanguage.com
         AssignedTo|nobody puremagic.com        |andrei metalanguage.com



09:00:27 PDT ---
The bug is in druntime. It defines fseek and ftell in core.stdc.stdio like
this:

int    fseek(FILE* stream, long offset, int whence);
c_long ftell(FILE* stream);

And it defines c_long in core.stdc.config like this:


version( Windows )
{
    alias int   c_long;
    alias uint  c_ulong;
}
else
{
  static if( (void*).sizeof > int.sizeof )
  {
    alias long  c_long;
    alias ulong c_ulong;
  }
  else
  {
    alias int   c_long;
    alias uint  c_ulong;
  }
} 

I don't know why under Windows c_long is actually 32 bit.

Then, in my code I had this note:

    void seek(long offset, int origin = SEEK_SET)
    {
        enforce(p && p.handle,
                "Attempting to seek() in an unopened file");
        //     Dubious: why is fseek in std.c.stdio taking an int???
        errnoEnforce(core.stdc.stdio.fseek(
                    p.handle, to!int(offset), origin) == 0,
                "Could not seek in file `"~p.name~"'");
    }

So it seeme like there was something fishy going on.

I emailed Sean about the problem. He needs to fix druntime, then I will fix
stdio, and then we can close this.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 17 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3409




Could be wrong, but I think the way druntime defines these functions is because
that's the way they're defined in the C std lib API.  I think to do better you
need to use OS-specific APIs directly.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 17 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3409




02:08:08 PDT ---
 I don't know why under Windows c_long is actually 32 bit.
because msvc compiler is LLP64. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Oct 19 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3409




02:13:29 PDT ---

 Could be wrong, but I think the way druntime defines these functions is because
 that's the way they're defined in the C std lib API.
Yes, but that's *C* API, so the type should be c_long. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Oct 19 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3409


Sobirari Muhomori <dfj1esp02 sneakemail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|major                       |regression



02:20:19 PDT ---
Strange, in 2.020 it was

int    fseek(FILE* stream, c_long offset, int whence);

so this is regression.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 19 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3409


Sobirari Muhomori <dfj1esp02 sneakemail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|regression                  |major



02:25:26 PDT ---
No, headers are correct, see
http://www.dsource.org/projects/druntime/browser/trunk/import/core/stdc/stdio.d
http://www.kernel.org/doc/man-pages/online/pages/man3/fseek.3.html

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 19 2009
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3409


Andrei Alexandrescu <andrei metalanguage.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|andrei metalanguage.com     |nobody puremagic.com



08:13:04 PDT ---
David or Sobirari, any chance you could take a second look at this? I'd
appreciate it - thanks!

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jun 05 2011