www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Hardcoded filepaths in compiled exe

reply Raikia <raikiasec gmail.com> writes:
Hey all,

I'm trying to compile a release-level binary but it looks like 
the resulting executable has metadata in it that I would like to 
avoid.  I've tried using both LDC and DMD with the below commands 
(I've tried many variations of them, you can see the switches are 
in an attempt to remove this problem):

dmd -m64 -O -J. -release -inline -boundscheck=off 
-L/SUBSYSTEM:WINDOWS -L/ENTRY:mainCRTStartup .\program.d

and

ldc2.exe --link-internally -O --boundscheck=off --o- --release 
--relocation-model=pic --static .\program.d


However, after doing those, if you look at the strings in the 
compiled exe, you'll see things like:

$ strings -a program.exe | grep 'dmd2'
C:\D\dmd2\windows\bin\..\..\src\phobos\std\file.d
C:\D\dmd2\windows\bin\..\..\src\phobos\std\utf.d
C:\D\dmd2\windows\bin\..\..\src\phobos\std\base64.d


This problem is more egregious when I am using a home folder, 
like "C:\Users\<my name>\" instead of "C:\D\".  Am I missing 
something?  Is there a way to compile D without leaking metadata 
like this in a production release binary?
Dec 29 2020
parent reply Basile B. <b2.temp gmx.com> writes:
On Tuesday, 29 December 2020 at 16:13:53 UTC, Raikia wrote:

 Hey all,

 [...]
 $ strings -a program.exe | grep 'dmd2'
 C:\D\dmd2\windows\bin\..\..\src\phobos\std\file.d
 C:\D\dmd2\windows\bin\..\..\src\phobos\std\utf.d
 C:\D\dmd2\windows\bin\..\..\src\phobos\std\base64.d


 This problem is more egregious when I am using a home folder, 
 like "C:\Users\<my name>\" instead of "C:\D\".  Am I missing 
 something?  Is there a way to compile D without leaking 
 metadata like this in a production release binary?


I believe those are because of Exception constructors that use 
the __FILE__ special keyword. You might patch the final 
executable and replace the string content with spaces or 'x's.
Dec 29 2020
parent reply Raikia <raikiasec gmail.com> writes:
On Tuesday, 29 December 2020 at 19:30:53 UTC, Basile B. wrote:

 On Tuesday, 29 December 2020 at 16:13:53 UTC, Raikia wrote:

 Hey all,

 [...]
 $ strings -a program.exe | grep 'dmd2'
 C:\D\dmd2\windows\bin\..\..\src\phobos\std\file.d
 C:\D\dmd2\windows\bin\..\..\src\phobos\std\utf.d
 C:\D\dmd2\windows\bin\..\..\src\phobos\std\base64.d


 This problem is more egregious when I am using a home folder, 
 like "C:\Users\<my name>\" instead of "C:\D\".  Am I missing 
 something?  Is there a way to compile D without leaking 
 metadata like this in a production release binary?


 I believe those are because of Exception constructors that use 
 the __FILE__ special keyword. You might patch the final 
 executable and replace the string content with spaces or 'x's.


Interesting.  I was able to clobber it with bbe with no issues.  
I'm surprised the compiler doesn't strip out this potentially 
sensitive metadata, but I guess I'll just patch it out as part of 
my build process.  Thanks!
Dec 29 2020
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/29/20 4:27 PM, Raikia wrote:

 On Tuesday, 29 December 2020 at 19:30:53 UTC, Basile B. wrote:

 On Tuesday, 29 December 2020 at 16:13:53 UTC, Raikia wrote:

 Hey all,

 [...]
 $ strings -a program.exe | grep 'dmd2'
 C:\D\dmd2\windows\bin\..\..\src\phobos\std\file.d
 C:\D\dmd2\windows\bin\..\..\src\phobos\std\utf.d
 C:\D\dmd2\windows\bin\..\..\src\phobos\std\base64.d


 This problem is more egregious when I am using a home folder, like 
 "C:\Users\<my name>\" instead of "C:\D\".  Am I missing something?  
 Is there a way to compile D without leaking metadata like this in a 
 production release binary?


 I believe those are because of Exception constructors that use the 
 __FILE__ special keyword. You might patch the final executable and 
 replace the string content with spaces or 'x's.


 
 Interesting.  I was able to clobber it with bbe with no issues. I'm 
 surprised the compiler doesn't strip out this potentially sensitive 
 metadata, but I guess I'll just patch it out as part of my build 
 process.  Thanks!


Yeah, that's a bit surprising. I think it's for use in debuggers so they 
can pull up the exact file where the exception was thrown.

But I would think a feature should exist that masks the base directory 
of exception file names.

Probably worth an enhancement request.

-Steve
Dec 29 2020
parent reply Basile B. <b2.temp gmx.com> writes:
On Tuesday, 29 December 2020 at 23:11:25 UTC, Steven 
Schveighoffer wrote:

 But I would think a feature should exist that masks the base 
 directory of exception file names.

 Probably worth an enhancement request.

 -Steve


Also aren't dmd output binaries supposed to be "reproducible" ?
Dec 29 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/29/20 7:46 PM, Basile B. wrote:

 On Tuesday, 29 December 2020 at 23:11:25 UTC, Steven Schveighoffer wrote:

 But I would think a feature should exist that masks the base directory 
 of exception file names.

 Probably worth an enhancement request.


 
 Also aren't dmd output binaries supposed to be "reproducible" ?


If you had an option to change __FILE__ to be canonical, then it could be.

i.e. instead of:

C:\D\dmd2\windows\bin\..\..\src\phobos\std\file.d

you stored:

(imports)\std\file.d

where (imports) (or maybe some other token) was substituted in for the 
base of every import. Files passed on the command line would just store 
the __FILE__ as it was passed.

-Steve
Dec 29 2020
parent reply Basile B. <b2.temp gmx.com> writes:
On Wednesday, 30 December 2020 at 01:21:37 UTC, Steven 
Schveighoffer wrote:

 On 12/29/20 7:46 PM, Basile B. wrote:

 On Tuesday, 29 December 2020 at 23:11:25 UTC, Steven 
 Schveighoffer wrote:

 But I would think a feature should exist that masks the base 
 directory of exception file names.

 Probably worth an enhancement request.


 
 Also aren't dmd output binaries supposed to be "reproducible" ?


 If you had an option to change __FILE__ to be canonical, then 
 it could be.

 i.e. instead of:

 C:\D\dmd2\windows\bin\..\..\src\phobos\std\file.d

 you stored:

 (imports)\std\file.d

 where (imports) (or maybe some other token) was substituted in 
 for the base of every import. Files passed on the command line 
 would just store the __FILE__ as it was passed.

 -Steve


actually the string 
"C:\D\dmd2\windows\bin\..\..\src\phobos\std\file.d"
looks like it comes from how the ini file variables are expanded.

__FILE__ is not supposed to represent an absolute file name 
unless the compiler get passed absolute file names. That why 
__FILE_FULL_PATH__ was added at some point.
Dec 29 2020
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/29/20 11:43 PM, Basile B. wrote:

 On Wednesday, 30 December 2020 at 01:21:37 UTC, Steven Schveighoffer wrote:

 On 12/29/20 7:46 PM, Basile B. wrote:

 On Tuesday, 29 December 2020 at 23:11:25 UTC, Steven Schveighoffer 
 wrote:

 But I would think a feature should exist that masks the base 
 directory of exception file names.

 Probably worth an enhancement request.


 Also aren't dmd output binaries supposed to be "reproducible" ?


 If you had an option to change __FILE__ to be canonical, then it could 
 be.

 i.e. instead of:

 C:\D\dmd2\windows\bin\..\..\src\phobos\std\file.d

 you stored:

 (imports)\std\file.d

 where (imports) (or maybe some other token) was substituted in for the 
 base of every import. Files passed on the command line would just 
 store the __FILE__ as it was passed.


 
 actually the string "C:\D\dmd2\windows\bin\..\..\src\phobos\std\file.d"
 looks like it comes from how the ini file variables are expanded.
 
 __FILE__ is not supposed to represent an absolute file name unless the 
 compiler get passed absolute file names. That why __FILE_FULL_PATH__ was 
 added at some point.


When I tested this code, it spit out a full path:

import std.stdio;
import std.range;

void main()
{
     char[] str = [0xff, 0xff];
     writeln(str.front);
}

------

std.utf.UTFException /Users/steves/.dvm/compilers/dmd-2.094.2/osx/bin/../../src/pho
os/std/utf.d(1508): 
Invalid UTF-8 sequence (at index 1)

The path comes from the ini file to generate the import directory. I 
believe the ini configuration gives a full path (the path to the ini 
file, and then the relative path to the import).

I think if the import was in the project and not the dmd compiler 
directory, then full paths wouldn't be happening.

But that doesn't mean you can't FORCE the compiler to behave better via 
an option switch when importing from elsewhere.

-Steve
Dec 30 2020
prev sibling parent Basile B. <b2.temp gmx.com> writes:
On Tuesday, 29 December 2020 at 21:27:07 UTC, Raikia wrote:

 Interesting.  I was able to clobber it with bbe with no issues.
  I'm surprised the compiler doesn't strip out this potentially 
 sensitive metadata, but I guess I'll just patch it out as part 
 of my build process.  Thanks!


Other super safe options are to use either a VM or a CI service 
to build the releases.
Dec 29 2020