digitalmars.D - Bug in -J
- Mr. Pib (22/22) Aug 11 2017 I have -J added to the command line like
- rikki cattermole (3/36) Aug 11 2017 Most likely related or is the issue[0].
- Vladimir Panteleev (12/47) Aug 11 2017 No, it just prefixes the given import() path will all those given
- Mr. Pib (15/63) Aug 11 2017 You are making assumptions about me making assumptions... please
- Vladimir Panteleev (16/29) Aug 11 2017 Sorry, maybe I don't understand the question. Maybe you could
- Mr. Pib (24/54) Aug 11 2017 I'm pretty sure that on no OS does the same location mean
- Vladimir Panteleev (21/44) Aug 11 2017 I understand what you mean, but just to clarify on the .. thing:
- Vladimir Panteleev (11/13) Aug 11 2017 There is a potential ambiguity here:
- Mr. Pib (39/52) Aug 12 2017 What is the inconsistency? Absolute paths are specific and well
I have -J added to the command line like -JC:\Temp I then use import(r"C:\Temp\a.dat"); and I get an error about the file not existing in the main.d: Error: file "C:\\Temp\\a.dat" cannot be found or not in a path specified with -J It seems dmd does internally compare the paths to see if they are identical. Of course, removing the C:\Temp\ part of import works fine. The problem with that approach is it then is not consistent with other code. I need to specify the full path because sometimes it is used. essentially version(X) import(path) else load(path); while I could do something like import(baseName(path)) it seems kinda clunky, in any case. It's pretty obvious that one excepts the same behavior so it could create bugs in code that except the behavior to work correctly.
Aug 11 2017
On 12/08/2017 3:34 AM, Mr. Pib wrote:I have -J added to the command line like -JC:\Temp I then use import(r"C:\Temp\a.dat"); and I get an error about the file not existing in the main.d: Error: file "C:\\Temp\\a.dat" cannot be found or not in a path specified with -J It seems dmd does internally compare the paths to see if they are identical. Of course, removing the C:\Temp\ part of import works fine. The problem with that approach is it then is not consistent with other code. I need to specify the full path because sometimes it is used. essentially version(X) import(path) else load(path); while I could do something like import(baseName(path)) it seems kinda clunky, in any case. It's pretty obvious that one excepts the same behavior so it could create bugs in code that except the behavior to work correctly.Most likely related or is the issue[0]. [0] https://issues.dlang.org/show_bug.cgi?id=3420
Aug 11 2017
On Saturday, 12 August 2017 at 02:45:30 UTC, rikki cattermole wrote:On 12/08/2017 3:34 AM, Mr. Pib wrote:No, it just prefixes the given import() path will all those given by -J.I have -J added to the command line like -JC:\Temp I then use import(r"C:\Temp\a.dat"); and I get an error about the file not existing in the main.d: Error: file "C:\\Temp\\a.dat" cannot be found or not in a path specified with -J It seems dmd does internally compare the paths to see if they are identical.I don't understand why you would want to do that. Using absolute paths makes assumptions about the system your software is being built on. Even if you never plan to build the software on any computer but your own, it still seems like a questionable design decision.Of course, removing the C:\Temp\ part of import works fine. The problem with that approach is it then is not consistent with other code. I need to specify the full path because sometimes it is used. essentially version(X) import(path) else load(path);You could also use -J/ and import(path[1..$]) (or the Windows equivalent). Still a bad idea.while I could do something like import(baseName(path)) it seems kinda clunky, in any case. It's pretty obvious that one excepts the same behavior so it could create bugs in code that except the behavior to work correctly.Most likely related or is the issue[0]. [0] https://issues.dlang.org/show_bug.cgi?id=3420Nope.
Aug 11 2017
On Saturday, 12 August 2017 at 03:02:31 UTC, Vladimir Panteleev wrote:On Saturday, 12 August 2017 at 02:45:30 UTC, rikki cattermole wrote:You are making assumptions about me making assumptions... please don't make any more assumptions or we will be in an infinite regression ;/On 12/08/2017 3:34 AM, Mr. Pib wrote:No, it just prefixes the given import() path will all those given by -J.I have -J added to the command line like -JC:\Temp I then use import(r"C:\Temp\a.dat"); and I get an error about the file not existing in the main.d: Error: file "C:\\Temp\\a.dat" cannot be found or not in a path specified with -J It seems dmd does internally compare the paths to see if they are identical.I don't understand why you would want to do that. Using absolute paths makes assumptions about the system your software is being built on. Even if you never plan to build the software on any computer but your own, it still seems like a questionable design decision.Of course, removing the C:\Temp\ part of import works fine. The problem with that approach is it then is not consistent with other code. I need to specify the full path because sometimes it is used. essentially version(X) import(path) else load(path);If are aware, it has nothing to do with absolute paths as the bug should also be exhibited with relative paths. It is a comparison issue of the strings rather than checking to see if they represent the same physical location. It would be like saying that \x\y is different from \x\..\x\y. Dmd does a blind comparison, I bet, rather than what it should be doing. Obviously \x\y are different strings but they are not different file locations. If that is not the case, then a better error message should be given.You could also use -J/ and import(path[1..$]) (or the Windows equivalent). Still a bad idea.while I could do something like import(baseName(path)) it seems kinda clunky, in any case. It's pretty obvious that one excepts the same behavior so it could create bugs in code that except the behavior to work correctly.Most likely related or is the issue[0]. [0] https://issues.dlang.org/show_bug.cgi?id=3420Nope.
Aug 11 2017
On Saturday, 12 August 2017 at 03:52:25 UTC, Mr. Pib wrote:You are making assumptions about me making assumptions... please don't make any more assumptions or we will be in an infinite regression ;/Sorry, maybe I don't understand the question. Maybe you could explain in broader terms the higher-level goal or problem you're trying to solve, and maybe we can recommend a better way?If are aware, it has nothing to do with absolute paths as the bug should also be exhibited with relative paths. It is a comparison issue of the strings rather than checking to see if they represent the same physical location. It would be like saying that \x\y is different from \x\..\x\y. Dmd does a blind comparison, I bet, rather than what it should be doing. Obviously \x\y are different strings but they are not different file locations.Fairly sure we forbid .. out of security considerations. I don't think this applies to Windows, but on POSIX, depending on how .. is interpreted, \x\y x/y actually can mean a different file from x/../x/y. It has taken a lot of consideration and research until we even allowed path separators in import paths. For a very long time, they were completely forbidden, and a long time after that, they were forbidden on Windows (because on Windows things can be more complicated due to the various kinds of reparse points and things such as short filenames).If that is not the case, then a better error message should be given.Feel free to file a diagnostic enhancement request if you have specific suggestions.
Aug 11 2017
On Saturday, 12 August 2017 at 03:58:30 UTC, Vladimir Panteleev wrote:On Saturday, 12 August 2017 at 03:52:25 UTC, Mr. Pib wrote:I'm pretty sure that on no OS does the same location mean different things? I am not talking about strange stuff but simple stuff. I have code that loads a file at runtime and requires the absolute path. This is only for debugging purposes. When built in release, everything is switched over to use imports and embed the files in the binary. The same path is used for other things like caching/uniqueID but are never actually read from. You see this sort of stuff a lot when you open an executable and see hard coded paths but obviously never used for file system purposes. The files and paths are all the same but import doens't seem to think so. Adding baseName solves the problem immediately but that is a hack. import should know that the path is the same as the one specified by -J. The whole point of -J is to specify the path for security purposes, right? So why does it matter if I use path\filename or baseName(filename)? Both point to the same location and both are consistent with -J, import should understand that. It is an obvious oversight. But there is an obvious programmatic difference between the two versions. Luckily, using baseName does fix the problem so it is not a huge deal but it is still a bug/issue with import for being ignorant of what it is actually doing.You are making assumptions about me making assumptions... please don't make any more assumptions or we will be in an infinite regression ;/Sorry, maybe I don't understand the question. Maybe you could explain in broader terms the higher-level goal or problem you're trying to solve, and maybe we can recommend a better way?If are aware, it has nothing to do with absolute paths as the bug should also be exhibited with relative paths. It is a comparison issue of the strings rather than checking to see if they represent the same physical location. It would be like saying that \x\y is different from \x\..\x\y. Dmd does a blind comparison, I bet, rather than what it should be doing. Obviously \x\y are different strings but they are not different file locations.Fairly sure we forbid .. out of security considerations. I don't think this applies to Windows, but on POSIX, depending on how .. is interpreted, \x\y x/y actually can mean a different file from x/../x/y. It has taken a lot of consideration and research until we even allowed path separators in import paths. For a very long time, they were completely forbidden, and a long time after that, they were forbidden on Windows (because on Windows things can be more complicated due to the various kinds of reparse points and things such as short filenames).If that is not the case, then a better error message should be given.Feel free to file a diagnostic enhancement request if you have specific suggestions.
Aug 11 2017
On Saturday, 12 August 2017 at 05:42:21 UTC, Mr. Pib wrote:I'm pretty sure that on no OS does the same location mean different things?I understand what you mean, but just to clarify on the .. thing: $ mkdir d d/x d/z $ ln -s d/z x $ echo foo > d/z/y $ echo bar > d/x/y $ cat x/y foo + cat x/../x/y barI am not talking about strange stuff but simple stuff. I have code that loads a file at runtime and requires the absolute path. This is only for debugging purposes. When built in release, everything is switched over to use imports and embed the files in the binary. The same path is used for other things like caching/uniqueID but are never actually read from. You see this sort of stuff a lot when you open an executable and see hard coded paths but obviously never used for file system purposes.Sounds like you can also work around it using e.g. enum appRoot = `C:\Temp\`; debug string data = readText(appRoot ~ "a.dat"); else enum data = import("a.dat"); (and build with -JC:\Temp). That should now also work if a.dat is in a subdirectory relative to the -J path.The files and paths are all the same but import doens't seem to think so. Adding baseName solves the problem immediately but that is a hack. import should know that the path is the same as the one specified by -J. The whole point of -J is to specify the path for security purposes, right? So why does it matter if I use path\filename or baseName(filename)? Both point to the same location and both are consistent with -J, import should understand that. It is an obvious oversight. But there is an obvious programmatic difference between the two versions. Luckily, using baseName does fix the problem so it is not a huge deal but it is still a bug/issue with import for being ignorant of what it is actually doing.Sounds reasonable, the compiler could check if paths start with a -J path.
Aug 11 2017
On Saturday, 12 August 2017 at 06:02:57 UTC, Vladimir Panteleev wrote:Sounds reasonable, the compiler could check if paths start with a -J path.There is a potential ambiguity here: dmd -Jsomedir test.d test.d: import("somedir/file.txt"); Does the user mean to import "somedir/file.txt" or "somedir/somedir/file.txt"? Currently the latter is understood. Simply checking for path prefix would break this case. I guess this could be done only with absolute paths, but that introduces an inconsistency with relative paths. I'm not sure it's worth it, considering it's easy to work around.
Aug 11 2017
Mr. Pib <Af343 MS.com> On Saturday, 12 August 2017 at 06:06:33 UTC, Vladimir Panteleev wrote:On Saturday, 12 August 2017 at 06:02:57 UTC, Vladimir Panteleev wrote:What is the inconsistency? Absolute paths are specific and well defined and so are relatives. Relatives are always relative to J, are they not? so, import("somedir/file.txt"); obviously means somedir/somedir/file.txt. If you wanted somedir/file.txt you would do import(`file.txt`). I see no problem here. With absolute paths, one simply checks if the absolute base path matches the absolute base path specified by J. If it matches, then it passes and can be used directly or removed internally. If not then an error is given. e.g., -JC:/basepath/somedir C:/basepath/somedir/file.txt import('C:/basepath/somedir/file.txt') import('file.txt') import('../somedir/file.txt') import('C:/basepath/../basepath/somedir/file.txt') import('C:/path/../basepath/file.txt') should pass all pass. import('../file.txt') import('C:/basepath/../basepath/file.txt') import('C:/path/file.txt') should all fail. (I'm only using .. for reference, not sure if they should actually be supported) It seems quite simple to me Any relative paths specified by import are prepended with the path specified by -J. Any absolute paths specified by an import are first matched to the path -J simply by example: if (J_path.length < basepath.length) assert("Import path must be located in the same path that is specified by -J"); if (basepath[0..J_path.length] != J_path) ditto; // makes sure absolute path is in the -J path. This is the check that is missing from dmd. (this doesn't resolve ..'s, that could be done before and assumes basepath is an absolute path)Sounds reasonable, the compiler could check if paths start with a -J path.There is a potential ambiguity here: dmd -Jsomedir test.d test.d: import("somedir/file.txt"); Does the user mean to import "somedir/file.txt" or "somedir/somedir/file.txt"? Currently the latter is understood. Simply checking for path prefix would break this case. I guess this could be done only with absolute paths, but that introduces an inconsistency with relative paths. I'm not sure it's worth it, considering it's easy to work around.
Aug 12 2017