www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - std.utf.toUTF16z and windows unicode path limits

reply "Jay Norwood" <jayn prismnet.com> writes:
I ran into a problem with the std.file.remove() operation being 
limited by the windows ascii maxpath of around 260 characters, 
even though the low level code is calling the unicode version of 
windows delete, which has the capability to go up to 32k.   The 
trick appears to be that the unicode api calls still need "\\?\" 
prepended to the string in order to extend the path limit to 32K.

There is some discussion of it at this link.
http://msdn.microsoft.com/en-us/library/windows/desktop/aa363915(v=vs.85).aspx

So, I wonder if it would be appropriate to update this 
std.utf.toUTF16z, or else create a new version that does the 
prepending correctly if the path provided exceeds the relatively 
small MAX_PATH limit.

We might also want to re-examine MAX_PATH use elsewhere in the 
code, since windows can supposedly support the 32K paths through 
use of the unicode versions of the calls, which the std.file 
methods are using.

void remove(in char[] name)
{
     version(Windows)
     {
         cenforce(DeleteFileW(std.utf.toUTF16z(name)), name);
     }
Mar 17 2014
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
I didn't know this. Please file a bugzilla enhancement request for it, and
we'll 
see about fixing it.

On 3/17/2014 6:34 PM, Jay Norwood wrote:

 I ran into a problem with the std.file.remove() operation being limited by the
 windows ascii maxpath of around 260 characters, even though the low level code
 is calling the unicode version of windows delete, which has the capability to
go
 up to 32k.   The trick appears to be that the unicode api calls still need
 "\\?\" prepended to the string in order to extend the path limit to 32K.

 There is some discussion of it at this link.
 http://msdn.microsoft.com/en-us/library/windows/desktop/aa363915(v=vs.85).aspx

 So, I wonder if it would be appropriate to update this std.utf.toUTF16z, or
else
 create a new version that does the prepending correctly if the path provided
 exceeds the relatively small MAX_PATH limit.

 We might also want to re-examine MAX_PATH use elsewhere in the code, since
 windows can supposedly support the 32K paths through use of the unicode
versions
 of the calls, which the std.file methods are using.

 void remove(in char[] name)
 {
      version(Windows)
      {
          cenforce(DeleteFileW(std.utf.toUTF16z(name)), name);
      }
Mar 17 2014
prev sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Tuesday, 18 March 2014 at 01:34:44 UTC, Jay Norwood wrote:

 I ran into a problem with the std.file.remove() operation being 
 limited by the windows ascii maxpath of around 260 characters,


http://d.puremagic.com/issues/show_bug.cgi?id=8967
Mar 17 2014
parent reply "Jay Norwood" <jayn prismnet.com> writes:
On Tuesday, 18 March 2014 at 02:02:04 UTC, Vladimir Panteleev 
wrote:


 http://d.puremagic.com/issues/show_bug.cgi?id=8967


ok, thanks.  I was able to work around my issues

The basic solution, as you indicated, is to prefix any long 
paths.   Also the prefix is only usable on an absolute path.

nm = r"\\?\" ~ absolutePath(e.name);

One more issue I ran into was that our remove() and rmdir() don't 
override the read-only attribute for you.   You have to check and 
clear this.  If you are doing these getAttributes and 
setAttributes calls on long paths, you'll run into the same 
issues and need the above expansion.

uint att = getAttributes(fn);
att ^= FILE_ATTRIBUTE_READONLY;
setAttributes(fn, att);

So, I think the issue has enough notes in it to create a fix, and 
it would be a general solution that would make all of the windows 
unicode path calls support the 32K unicode path limit.

An option to handle  the read-only attribute would be nice also...
Mar 18 2014
parent "Jay Norwood" <jayn prismnet.com> writes:
How funny ...

The win7 gui copy and paste failed to copy a folder with a deep 
directory structure ... putting up a message dialog about the 
pathnmames being too long!

A d parallel copy rewrite works.
Mar 19 2014