www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - building Windows kernel-mode drivers in D

reply Cauterite <cauterite gmail.com> writes:
The prospect of writing kernel-mode drivers in D doesn't seem to 
get mentioned very often. I know there's been some attempts in 
the past at running D in the kernel, such as XOmB, but no mention 
of doing so on Windows.
I gave it a shot, and it seems to be perfectly feasible, so I 
thought I should share my findings for the benefit of anyone else 
who decides to explore this route.

(Note: I have no idea how any of this applies to 64-bit Windows. 
I'm using Windows XP SP3 32-bit, DMD v2.067.1, OPTLINK 8.00.17.)

DMD seems to have no trouble generating code to run in the kernel 
(if you avoid some language features that rely on the 
D-runtime/GC). The main difficulties are with the linker 
(Optlink) — it's really not obvious how to make it generate a 
valid driver image, and linking with the kernel runtime can be a 
bit of a struggle too.

I approached this problem by first determining what defines a 
Windows driver image (usually a .sys file):
- it follows the same portable-executable format as any other 
EXE/DLL.
- the subsystem attribute in the file header is set to 'native', 
instead of the usual values of 'console' or 'windows' found in 
user-mode programs.
- the entrypoint points to a function commonly known as 
'DriverEntry', instead of something along the lines of 
'MainCRTStartup' for most user-mode programs.
- it does not import user-mode libraries such as kernel32.dll or 
ntdll.dll.
- (IMPORTANT:) it must have a valid checksum in the file header, 
or windows will refuse to load it. User-mode programs often just 
leave the checksum field as 0.

To make optlink output this kind of PE file, you need to compile 
using:
`dmd -L/exetype:nt -L/subsystem:native -ofmydriver.sys mydriver.d`
The /subsystem:native flag alone is not enough, because when you 
specify it, optlink changes the output format to COM or 
something. /exetype:nt fixes that.
However, optlink doesn't set the checksum field, even if you use 
the /checksum flag. You have to use a separate tool to set the 
checksum after optlink finishes - ARTeam CheckSum Fixer ( 
http://www.accessroot.com/arteam/site/download.php?view.239 ) 
seems to do the job with:
`checksum.exe mydriver.sys -fix`

So that gets us a mostly valid-looking driver image, though it 
still may be linked to user-mode DLLs since I haven't mentioned 
the contents of 'mydriver.d'. If you write the program as

     pragma(startaddress, DriverEntry);
     extern(Windows) int DriverEntry(void*, void*) {return 0;};

then the linker should not include the D runtime or any win32 
imports at all. My .sys file had a completely empty import table 
with this code. DriverEntry is the function which is called when 
the driver is first started; for the sake of brevity I haven't 
specified the parameter types here, but you can read all about 
DriverEntry in Microsoft's documentation. Note that you can use 
any name instead of 'DriverEntry'; it's Microsoft's naming 
convention.

At this point I was able to successfully run the compiled image 
as a driver on my system. I used Process Hacker ( 
http://processhacker.sourceforge.net/ ) to quickly install it: 
"Tools" -> "Create Service" -> "Type: Driver". Obviously the 
driver can't do much yet, since it has no direct access to 
external functions, but at least we have D code running in kernel 
mode (and at it didn't BSOD).

Most real drivers import functions from ntoskrnl.exe for runtime 
support and system APIs. I'm going to cover this in another post, 
since I honestly can't remember the exact procedure I used to get 
ntoskrnl imported. So for now I'll summarise:

mydriver.d :
     pragma(startaddress, DriverEntry);
     extern(Windows) int DriverEntry(void*, void*) {return 0;};

build.cmd :
     dmd -L/exetype:nt -L/subsystem:native -ofmydriver.sys 
mydriver.d
     checksum.exe mydriver.sys -fix

I'll continue this guide as soon as I have time.
Sep 25 2015
next sibling parent Dominikus Dittes Scherkl <Dominikus.Scherkl continental-corporation.com> writes:
On Friday, 25 September 2015 at 15:17:02 UTC, Cauterite wrote:
[...]
 mydriver.d :
     pragma(startaddress, DriverEntry);
     extern(Windows) int DriverEntry(void*, void*) {return 0;};

 build.cmd :
     dmd -L/exetype:nt -L/subsystem:native -ofmydriver.sys 
 mydriver.d
     checksum.exe mydriver.sys -fix

 I'll continue this guide as soon as I have time.
Cool. This is really helpful, especially the checksum part.
Sep 25 2015
prev sibling parent anon <anon anon.ne> writes:
On Friday, 25 September 2015 at 15:17:02 UTC, Cauterite wrote:
 The prospect of writing kernel-mode drivers in D doesn't seem 
 to get mentioned very often. I know there's been some attempts 
 in the past at running D in the kernel, such as XOmB, but no 
 mention of doing so on Windows.
 I gave it a shot, and it seems to be perfectly feasible, so I 
 thought I should share my findings for the benefit of anyone 
 else who decides to explore this route.

 [...]
Interesting. You could put something in the wiki if you succeed, e.g 'how to make a windows driver' with a small description for each step.
Sep 25 2015