digitalmars.D.bugs - [Issue 3425] New: std.stdio.File does not properly handle EOF from stdin on Windows
- d-bugmail puremagic.com (35/35) Oct 20 2009 http://d.puremagic.com/issues/show_bug.cgi?id=3425
- d-bugmail puremagic.com (39/39) Mar 09 2011 http://d.puremagic.com/issues/show_bug.cgi?id=3425
- d-bugmail puremagic.com (15/15) Oct 12 2011 http://d.puremagic.com/issues/show_bug.cgi?id=3425
- d-bugmail puremagic.com (26/33) Dec 28 2011 http://d.puremagic.com/issues/show_bug.cgi?id=3425
- d-bugmail puremagic.com (277/277) Dec 28 2011 http://d.puremagic.com/issues/show_bug.cgi?id=3425
http://d.puremagic.com/issues/show_bug.cgi?id=3425 Summary: std.stdio.File does not properly handle EOF from stdin on Windows Product: D Version: 2.035 Platform: Other OS/Version: Windows Status: NEW Severity: normal Priority: P2 Component: Phobos AssignedTo: nobody puremagic.com ReportedBy: dsimcha yahoo.com Code: import std.stdio, std.algorithm; void main(string[] args) { auto foo = stdin.byLine(); foreach(elem; foo) { writeln(elem); } } Input file (foo.txt) : foo bar Result: F:\>cat foo.txt | test2.exe foo bar std.stdio.StdioException: Bad file descriptor Works perfectly on Linux. Haven't tested on Mac, BSD or whatever the heck else DMD runs on lately. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Oct 20 2009
http://d.puremagic.com/issues/show_bug.cgi?id=3425 David Simcha <dsimcha yahoo.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|std.stdio.File does not |StdioException on end of |properly handle EOF from |stdin on Windows |stdin on Windows | Severity|normal |major I've managed to figure out where this exception is coming from, but I don't know how to fix it because I don't know why errno is getting set. It's on line 2271 in stdio.d: if (fp._flag & _IONBF) { /* Use this for unbuffered I/O, when running * across buffer boundaries, or for any but the common * cases. */ L1: auto app = appender(buf); app.clear(); if(app.capacity == 0) app.reserve(128); // get at least 128 bytes available int c; while((c = FGETC(fp)) != -1) { app.put(cast(char) c); if(c == terminator) { buf = app.data; return buf.length; } } if (ferror(fps)) StdioException(); Can we **PLEASE** fix this one ASAP? It's been in Bugzilla for over a year and a half and makes it impossible to write simple text filter programs on Windows without ugly and/or unsafe workarounds. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Mar 09 2011
http://d.puremagic.com/issues/show_bug.cgi?id=3425 Jay Norwood <jayn prismnet.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jayn prismnet.com You can work around the issue by testing for eof on the first line in the loop. This works with no error. foreach(elem; stdin.byLine()) { if (stdin.eof()) break; writeln(elem); } -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Oct 12 2011
http://d.puremagic.com/issues/show_bug.cgi?id=3425 Max Vilimpoc <max vilimpoc.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |max vilimpoc.orgYou can work around the issue by testing for eof on the first line in the loop. This works with no error. foreach(elem; stdin.byLine()) { if (stdin.eof()) break; writeln(elem); }This didn't work for me, when using the example program on the D homepage: import std.stdio; void main() { ulong lines = 0; double sumLength = 0; foreach(line; stdin.byLine()) { if (stdin.eof()) break; ++lines; sumLength += line.length; } writeln("Average line length = ", lines ? sumLength / lines : 0); } I still get: std.stdio.StdioException std\stdio.d(2159): Bad file descriptor -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Dec 28 2011
http://d.puremagic.com/issues/show_bug.cgi?id=3425 However, I did find that if I modify stdio.d in Phobos to the following under the L1 label around line 2280, then the other form of piping, i.e. "type filename | executable", will work on Windows. if (ferror(fps) && EPIPE != ferror(fps)) StdioException(); In the "|" pipe case, it could be that perhaps the "type" command already closed down its end by the time ferror() was called on the receiving side. On Windows, "executable < filename" piping worked fine, but is not ideal for stringing filters together. Here's the writeln-debugged block of stdio.d code that works: // Private implementation of readln version (DIGITAL_MARS_STDIO) private size_t readlnImpl(FILE* fps, ref char[] buf, dchar terminator = '\n') { FLOCK(fps); scope(exit) FUNLOCK(fps); writeln("Entered readlnImpl()"); scope(exit) writeln("Exited readlnImpl()"); /* Since fps is now locked, we can create an "unshared" version * of fp. */ auto fp = cast(_iobuf*)fps; if (__fhnd_info[fp._file] & FHND_WCHAR) { /* Stream is in wide characters. * Read them and convert to chars. */ writeln("Entered if (__fhnd_info[fp._file] & FHND_WCHAR)"); static assert(wchar_t.sizeof == 2); auto app = appender(buf); app.clear(); for (int c = void; (c = FGETWC(fp)) != -1; ) { if ((c & ~0x7F) == 0) { app.put(cast(char) c); if (c == terminator) break; } else { if (c >= 0xD800 && c <= 0xDBFF) { int c2 = void; if ((c2 = FGETWC(fp)) != -1 || c2 < 0xDC00 && c2 > 0xDFFF) { StdioException("unpaired UTF-16 surrogate"); } c = ((c - 0xD7C0) << 10) + (c2 - 0xDC00); } //std.utf.encode(buf, c); app.put(cast(dchar)c); } } if (ferror(fps)) StdioException(); buf = app.data; return buf.length; } auto sz = GC.sizeOf(buf.ptr); //auto sz = buf.length; buf = buf.ptr[0 .. sz]; if (fp._flag & _IONBF) { writeln("Entered if (fp._flag & _IONBF)"); /* Use this for unbuffered I/O, when running * across buffer boundaries, or for any but the common * cases. */ L1: auto app = appender(buf); app.clear(); if(app.capacity == 0) app.reserve(128); // get at least 128 bytes available int c; writeln("fp._cnt: ", fp._cnt); while((c = FGETC(fp)) != -1) { writeln("chars: ", cast(char) c); app.put(cast(char) c); if(c == terminator) { writeln("hit terminator"); buf = app.data; return buf.length; } } writeln("feof(fps): ", feof(fps)); writeln("ferror(fps): ", ferror(fps)); // If EPIPE is seen then probably the other side has closed // already. This is the case when using, for example: // // "type filename | D-program" syntax on Windows. if (ferror(fps) && EPIPE != ferror(fps)) StdioException(); buf = app.data; return buf.length; } else { writeln("Entered if (!(fp._flag & _IONBF))"); int u = fp._cnt; char* p = fp._ptr; int i; writeln("length of stdin fp: ", u); if (fp._flag & _IOTRAN) { /* Translated mode ignores \r and treats ^Z as end-of-file */ writeln("Entered if (fp._flag & _IOTRAN)"); char c; while (1) { if (i == u) // if end of buffer goto L1; // give up c = p[i]; i++; if (c != '\r') { if (c == terminator) break; if (c != 0x1A) continue; goto L1; } else { if (i != u && p[i] == terminator) break; goto L1; } } if (i > sz) { buf = uninitializedArray!(char[])(i); } if (i - 1) memcpy(buf.ptr, p, i - 1); buf[i - 1] = cast(char)terminator; buf = buf[0 .. i]; if (terminator == '\n' && c == '\r') i++; } else { writeln("Entered if !(fp._flag & _IOTRAN)"); while (1) { if (i == u) // if end of buffer goto L1; // give up auto c = p[i]; i++; if (c == terminator) break; } if (i > sz) { buf = uninitializedArray!(char[])(i); } memcpy(buf.ptr, p, i); buf = buf[0 .. i]; } fp._cnt -= i; fp._ptr += i; return i; } } And here's what it outputs: c:\>avg < index.html Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 0 Entered if !(fp._flag & _IOTRAN) fp._cnt: 0 chars: < chars: h chars: t chars: m chars: l chars: > chars: hit terminator Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 106 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 97 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 37 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 27 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 18 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 8 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 0 Entered if !(fp._flag & _IOTRAN) fp._cnt: 0 feof(fps): 16 ferror(fps): 0 Exited readlnImpl() Average line length = 15.1429 c:\>type index.html | avg Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 0 Entered if !(fp._flag & _IOTRAN) fp._cnt: 0 chars: < chars: h chars: t chars: m chars: l chars: > chars: hit terminator Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 106 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 97 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 37 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 27 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 18 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 8 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 0 Entered if !(fp._flag & _IOTRAN) fp._cnt: 0 feof(fps): 0 ferror(fps): 32 Exited readlnImpl() Average line length = 15.1429 -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Dec 28 2011