www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - extend pragma(msg) with writing to file

reply a11e99z <black80 bk.ru> writes:
for example
for generating sources for other languages (interop)
or for viewing/checking auto generated code
```d
version (CS) pragma("interop.file.cs", ...);
pragma("just.to.check.generated.code.d", ...);
```

1) I know that exists security issues with it but IDK which one. 
so explain please.

2) allow writing file only to folder (compiler choice) **"g"** or 
**"gen"** for current source file or for current project.
disallow any changing folder symbols like '/', '\\', "..", and 
less than ASCII SPACE etc
Oct 06 2022
next sibling parent reply bauss <jacobbauss gmail.com> writes:
On Thursday, 6 October 2022 at 09:55:34 UTC, a11e99z wrote:

 for example
 for generating sources for other languages (interop)
 or for viewing/checking auto generated code
 ```d
 version (CS) pragma("interop.file.cs", ...);
 pragma("just.to.check.generated.code.d", ...);
 ```

 1) I know that exists security issues with it but IDK which 
 one. so explain please.

 2) allow writing file only to folder (compiler choice) **"g"** 
 or **"gen"** for current source file or for current project.
 disallow any changing folder symbols like '/', '\\', "..", and 
 less than ASCII SPACE etc


That is never going to be implemented, has been suggested before 
that compile-time file writing should be supported.

It's a very controversial topic because there's a huge security 
risk in allowing third-parties access to your filesystem, 
especially without executing an application or anything and 
simply from building source code or it being a dependency of some 
other package.

The risk is much greater when I/O happens at compile-time because 
it's very difficult for ex. an AV to pick up on malicious intend 
when it can't really scan for malicious executable code, as AVs 
typically aren't scanning source codes.

So while you could argue that someone could create malicious code 
and compile it with your program and the result would be the 
same, then the difference in it is that if the malicious code 
runs at compile-time then your system most likely won't pick up 
on it and protected, while if the same code ran at runtime then 
your system will most likely do so.
Oct 06 2022
next sibling parent reply Andrey Zherikov <andrey.zherikov gmail.com> writes:
On Thursday, 6 October 2022 at 12:08:16 UTC, bauss wrote:

 It's a very controversial topic because there's a huge security 
 risk in allowing third-parties access to your filesystem, 
 especially without executing an application or anything and 
 simply from building source code or it being a dependency of 
 some other package.

 The risk is much greater when I/O happens at compile-time 
 because it's very difficult for ex. an AV to pick up on 
 malicious intend when it can't really scan for malicious 
 executable code, as AVs typically aren't scanning source codes.


We already have read compile-time access to a system by 
[`import("foo.txt")`](https://dlang.org/spec/expression.html#
mport_expressions). How does write access change the risk in principle?


 So while you could argue that someone could create malicious 
 code and compile it with your program and the result would be 
 the same, then the difference in it is that if the malicious 
 code runs at compile-time then your system most likely won't 
 pick up on it and protected, while if the same code ran at 
 runtime then your system will most likely do so.


It's possible to prevent inclusion of generated file into the 
compilation. For example, generated file might not be in any 
-I/-J (sub)dirs.
Oct 07 2022
parent reply kdevel <kdevel vogtner.de> writes:
On Friday, 7 October 2022 at 16:22:22 UTC, Andrey Zherikov wrote:
[...]

 We already have read compile-time access to a system by 
 [`import("foo.txt")`](https://dlang.org/spec/expression.html#import_expressions).


Quote: "Note that by default an import expression will not 
compile unless one or more paths are passed via the -J switch. 
This tells the compiler where it should look for the files to 
import. This is a security feature."


 How does write access change the risk in principle?


Nice try.
Oct 08 2022
parent reply Andrey Zherikov <andrey.zherikov gmail.com> writes:
On Saturday, 8 October 2022 at 10:48:49 UTC, kdevel wrote:

 On Friday, 7 October 2022 at 16:22:22 UTC, Andrey Zherikov 
 wrote:
 [...]

 We already have read compile-time access to a system by 
 [`import("foo.txt")`](https://dlang.org/spec/expression.html#import_expressions).


 Quote: "Note that by default an import expression will not 
 compile unless one or more paths are passed via the -J switch. 
 This tells the compiler where it should look for the files to 
 import. This is a security feature."


Writing can be implemented the same way: it works only when a 
specific command line parameter is provided, or error out 
otherwise.


 How does write access change the risk in principle?


 Nice try.


Tis is not an answer to the question how does writing make 
security risks higher comparing to existing reading.
Oct 08 2022
parent reply kdevel <kdevel vogtner.de> writes:
On Saturday, 8 October 2022 at 13:28:54 UTC, Andrey Zherikov 
wrote:
[...]

 How does write access change the risk in principle?


 Nice try.


 Tis is not an answer to the question how does writing make 
 security risks higher comparing to existing reading.


What about

```
    ~/.ssh/authorized_keys?
```

or

```
    ~/.profile
    ~/.bashrc
```

Now assume someone coaxes you to compile his code under your 
account. See the problem which does not exist if the compiler 
could only read those files?
Oct 08 2022
next sibling parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Saturday, 8 October 2022 at 19:57:35 UTC, kdevel wrote:

 What about

 ```
    ~/.ssh/authorized_keys?
 ```

 or

 ```
    ~/.profile
    ~/.bashrc
 ```

 Now assume someone coaxes you to compile his code under your 
 account. See the problem which does not exist if the compiler 
 could only read those files?


https://xkcd.com/1958/

"Except... the program could also just write those files when you 
run it. What's stopping people **now?**"

"Yeah, messing with people's home folder isn't hard."

"I guess it's just that most people aren't criminals?"

"Oh right, I always forget."

"An underappreciated component of our computer security system."
Oct 08 2022
parent reply kdevel <kdevel vogtner.de> writes:
On Saturday, 8 October 2022 at 20:14:24 UTC, FeepingCreature 
wrote:

 https://xkcd.com/1958/

 "Except... the program could also just write those files when 
 you run it. What's stopping people **now?**"


Andrey asked for compilation only. Of course the program is ran 
under a separate user.


 "Yeah, messing with people's home folder isn't hard."


True if you set the file perms to 0777. Uh sorry. 
std.conv.octal!"777".to!int.


 "I guess it's just that most people aren't criminals?"


An open door may tempt a saint.
Oct 08 2022
next sibling parent Max Samukha <maxsamukha gmail.com> writes:
On Saturday, 8 October 2022 at 20:44:15 UTC, kdevel wrote:

 An open door may tempt a saint.


A better analogy would be "the safest house is one that doesn't 
exist". Closing the door doesn't impair the capabilities of your 
house. The ban on compile-time output severely impairs the 
capabilities of the compiler.
Oct 08 2022
prev sibling parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Saturday, 8 October 2022 at 20:44:15 UTC, kdevel wrote:

 On Saturday, 8 October 2022 at 20:14:24 UTC, FeepingCreature 
 wrote:

 https://xkcd.com/1958/

 "Except... the program could also just write those files when 
 you run it. What's stopping people **now?**"


 Andrey asked for compilation only. Of course the program is ran 
 under a separate user.


Under what circumstances would you compile a program on an 
account with *more* sensitive data than the one you run it on?
Oct 09 2022
parent reply kdevel <kdevel vogtner.de> writes:
On Sunday, 9 October 2022 at 08:00:58 UTC, FeepingCreature wrote:

 Andrey asked for compilation only. Of course the program is 
 ran under a separate user.


 Under what circumstances would you compile a program on an 
 account with *more* sensitive data than the one you run it on?


The D program P is compiled before another program Q is compiled, 
both on the same account. After compilation the programs are 
deployed to the production machine/account where the "more 
sensitive data" reside. By mere compilation P can alter the 
source code of Q. Though P is never executed in the production 
environment its code is run via Q.

Note that for pure scripting languages (without separate 
compilation stage) this attack vector does not exist.
Oct 09 2022
parent TheGag96 <thegag96 gmail.com> writes:
On Sunday, 9 October 2022 at 12:33:59 UTC, kdevel wrote:

 The D program P is compiled before another program Q is 
 compiled, both on the same account. After compilation the 
 programs are deployed to the production machine/account where 
 the "more sensitive data" reside. By mere compilation P can 
 alter the source code of Q. Though P is never executed in the 
 production environment its code is run via Q.

 Note that for pure scripting languages (without separate 
 compilation stage) this attack vector does not exist.


Again though, refer to my last post. We *already* have this 
problem with basically all of our other build systems that way 
have today. The world hasn't ended yet because of that, so we may 
as well at the very least reduce the layers of tooling complexity 
at the same time.
Oct 09 2022
prev sibling parent reply Andrey Zherikov <andrey.zherikov gmail.com> writes:
On Saturday, 8 October 2022 at 19:57:35 UTC, kdevel wrote:

 On Saturday, 8 October 2022 at 13:28:54 UTC, Andrey Zherikov 
 wrote:
 [...]

 How does write access change the risk in principle?


 Nice try.


 Tis is not an answer to the question how does writing make 
 security risks higher comparing to existing reading.


 What about

 ```
    ~/.ssh/authorized_keys?
 ```

 or

 ```
    ~/.profile
    ~/.bashrc
 ```

 Now assume someone coaxes you to compile his code under your 
 account. See the problem which does not exist if the compiler 
 could only read those files?


I see what you mean and I agree with you that unrestricted 
writing should not be allowed.

But I see that my question was a bit unclear so let me re-phrase 
it. Right now D has restricted compile-time reading: you can read 
a file only in a subdir of `-J` CLI parameter (if there is no 
`-J` then DMD will error out). What additional security risks do 
you see if writing has the same restrictions as reading: requires 
command line option (error out otherwise) and be allowed to 
create files under specified (by CLI) folder only?
Oct 09 2022
parent reply kdevel <kdevel vogtner.de> writes:
On Monday, 10 October 2022 at 01:10:42 UTC, Andrey Zherikov wrote:

 What additional security risks do you see if writing has the 
 same restrictions as reading: requires command line option 
 (error out otherwise) and be allowed to create files under 
 specified (by CLI) folder only?


If the context which invokes `dmd -J` already has write 
permission and the source code of the context and the source code 
handed over to dmd is of same origin then none.
Oct 10 2022
parent Andrey Zherikov <andrey.zherikov gmail.com> writes:
On Monday, 10 October 2022 at 12:59:30 UTC, kdevel wrote:

 On Monday, 10 October 2022 at 01:10:42 UTC, Andrey Zherikov 
 wrote:

 What additional security risks do you see if writing has the 
 same restrictions as reading: requires command line option 
 (error out otherwise) and be allowed to create files under 
 specified (by CLI) folder only?


 If the context which invokes `dmd -J` already has write 
 permission and the source code of the context and the source 
 code handed over to dmd is of same origin then none.


I don't think that `-J` should support writing so let's introduce 
new CLI option, say `-pragma-out=<dir>`. Now, if context that 
invokes `dmd -pragma-out` has no write permissions then 
compilation should error out when it tries to write to file.

Regarding "source code of the context", I'm not sure I get what 
you mean here. If you mean something that calls dmd then is it 
makefile or build tool? The first one is usually written by 
owners of the code you are compiling, the second one is not.
Oct 10 2022
prev sibling parent reply TheGag96 <thegag96 gmail.com> writes:
On Thursday, 6 October 2022 at 12:08:16 UTC, bauss wrote:

 It's a very controversial topic because there's a huge security 
 risk in allowing third-parties access to your filesystem, 
 especially without executing an application or anything and 
 simply from building source code or it being a dependency of 
 some other package.


But we've already basically been there for decades through:

* `configure` scripts from autotools
* Makefiles
* Ad-hoc shell/Python build scripts
* `dub.json`/`dub.sdl` calling out to shell

If you're willing to trust any of these, I don't see why you 
shouldn't be able to trust code from the program you're building.

D has always been my true language love, but I'm honestly very 
excited for Jai simply because it has [unrestricted compile-time 
execution](https://youtu.be/De0Am_QcZiQ?t=1582) that it uses to 
replace the need for external build scripts (and cases like OP 
desires). Hope he gets it out of private beta soon because I'd 
really like to see how the "Write your build script in the same 
language your program is in" idea pays off. Zig does something 
similar with its `build.zig` stuff, but this seems more 
comprehensive.
Oct 07 2022
parent reply Araq <rumpf_a web.de> writes:
On Friday, 7 October 2022 at 21:45:25 UTC, TheGag96 wrote:

 On Thursday, 6 October 2022 at 12:08:16 UTC, bauss wrote:

 [...]


 But we've already basically been there for decades through:

 * `configure` scripts from autotools
 * Makefiles
 * Ad-hoc shell/Python build scripts
 * `dub.json`/`dub.sdl` calling out to shell

 [...]


Fwiw Nim supports that since about a decade now... It's of course 
harder to be excited about something that already actually exists 
and works. ;-)
Oct 08 2022
parent TheGag96 <thegag96 gmail.com> writes:
On Saturday, 8 October 2022 at 07:33:42 UTC, Araq wrote:

 Fwiw Nim supports that since about a decade now... It's of 
 course harder to be excited about something that already 
 actually exists and works. ;-)


Wow - I've kept my eye on Nim a bit over the years, but I didn't 
realize that it already does do things like reading files at 
compile-time out of the box. Very cool. However, there are some 
limitations still:

```nim
import std/httpclient

proc doCrazyStuff(): string =
   var client = newHttpClient()
   let content = 
client.getContent("https://forum.dlang.org/thread/ijvehaanfbuhywaziqns forum.dlang.org")
   echo content

const stuff = doCrazyStuff()
```

```
/usr/lib/nim/pure/httpclient.nim(335, 12) Error: cannot evaluate 
at compile time: defaultSslContext
```

Jai should already be capable of this, seeing as how it can load 
a window, play sounds, etc. Hopefully competition ramps up in 
this area. :o)
Oct 08 2022
prev sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 6 October 2022 at 09:55:34 UTC, a11e99z wrote:

 for example
 for generating sources for other languages (interop)


are there other languages you would like to be supported?
C/C++ are available with `-HC` family of switches.


 or for viewing/checking auto generated code


mixin has this option already:
`-mixin=<filename>` expand and save mixins to file specified by 
`<filename>`
Oct 07 2022
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 08/10/2022 2:00 PM, Nicholas Wilson wrote:

 are there other languages you would like to be supported?


I've also wanted it in the past:

- Lua
- Javascript
- HTML

There is a ton of things especially in web you can use it with.
Oct 07 2022