digitalmars.D - Higher abstraction level for calling C functions
- Denis Shelomovskij (62/62) Jan 29 2012 D has complete (IMHO) compiler support for calling C functions (using
- Marco Leise (14/77) Jan 29 2012 This idea came to me the other day when working with Windows and Posix
- Timon Gehr (26/89) Jan 29 2012 Looks like very cool stuff.
- Denis Shelomovskij (6/21) Jan 30 2012 Yes, this restriction isn't necessary. I just was too lazy to add new
- Kagamin (1/2) Jan 29 2012 hmm... What declaration of HANDLE do you use?
- Denis Shelomovskij (5/7) Jan 30 2012 Wrapper knows nothing about non-standard types, but in `CreateFileW` the...
D has complete (IMHO) compiler support for calling C functions (using
extern(C)). But there is a lack of library support.
Readers from this NG probably know this, but... Microsoft .NET Framework
has such support. Look briefly at source example in:
http://msdn.microsoft.com/en-gb/library/s97shtze.aspx
And at System.IO.Directory.GetCurrentDirectory source:
http://typedescriptor.net/name/members/5540086E017CD13896E80A0CAEA6E517-System.IO.Directory.GetCurrentDirectory%28%29
where Win32Native.GetCurrentDirectory is defined as:
---
[DllImport("kernel32.dll", CharSet=CharSet.Auto, SetLastError=true)]
internal static extern int GetCurrentDirectory(int nBufferLength,
StringBuilder lpBuffer);
---
Looks good? But it's not. It tries to provide both low level support
(it's needed to just call C function, like D extern(C), but do
everything at runtime (yes, in .NET Framework even Regex can be compiled
at runtime from a string)) and high level support (it converts UTF-16 to
ANSI in the given MSDN example). As a result it even doesn't allow you
to specify a (scalar) parameter as a length of another (array) parameter
or throw on native function failure.
What can be done in that direction in D? The whole my point is expressed
in my small cwrap library:
http://deoma-cmd.ru/d/docs/src/cwrap.html
"Examples" is the most informative section of the documentation. Other
examples can be found in (original functions, all in one file):
https://bitbucket.org/denis_sh/cwrap/src/tip/examples/c.d
and generated wrappers:
https://bitbucket.org/denis_sh/cwrap/src/tip/out
Library state: almost every internal function is unittested, generates
compilable code which looks right but not unittested yet.
NOTE: You need head dmd to CTFE it or you can just change `enum data` to
`auto data` in examples/c.d, this will also reduce compilation time more
than x10.
Once original function is properly described in IDL, such library gives
the fallowing advantages:
* [memory-corruption-safe] User works with arrays, not separate pointers
and lengths.
* [memory-leak-safe] Original function allocated memory will be freed on
any failure (NOTE: see "Original function restrictions" in documentation).
* [fail-safe] An exception will be thrown if original function fails.
* [usability] Much easier to work with arrays/preferred string types (it
converts UTF-8/16/32 back and forth, see docs).
* [speed] No unnecessary GC (or other) allocations/copying/reading.
* [garbage-free] Look at, e.g. std.file - it leaves lots of garbage
because of calling native platform functions. Generated wrappers don't
even allocate GC memory (except for output parameters).
Looks like this functionality is usable for auto-generated bindings, at
least a useful GtkD still leaks memory almost everywhere because it
never frees (OK, not sure, I just haven't seen) allocated memory marked
by "free with g_free".
So should such stuff be ever included in Phobos? IMHO yes because it's a
general stuff. But if not, can generated wrappers only be included and
used to improve current situation "a system language with a library
which leaves garbage"?
By the way, std.file.getcwd and std.windows.charset.toMBSz are examples
of hand-written wrappers (the first is mine and strictly speaking forced
me to do this lib).
P.S.
Not sure if this library usable at all because I'm as sure in its
usability as in tabs indention superiority over spaces indention. )
So if it's not (usable), tell me without scruple.
Thanks for reading!
Jan 29 2012
"Marco Leise" <Marco.Leise gmx.de>
Am 29.01.2012, 21:48 Uhr, schrieb Denis Shelomovskij
<verylonglogin.reg gmail.com>:
D has complete (IMHO) compiler support for calling C functions (using
extern(C)). But there is a lack of library support.
Readers from this NG probably know this, but... Microsoft .NET Framework
has such support. Look briefly at source example in:
http://msdn.microsoft.com/en-gb/library/s97shtze.aspx
And at System.IO.Directory.GetCurrentDirectory source:
http://typedescriptor.net/name/members/5540086E017CD13896E80A0CAEA6E517-System.IO.Directory.GetCurrentDirectory%28%29
where Win32Native.GetCurrentDirectory is defined as:
---
[DllImport("kernel32.dll", CharSet=CharSet.Auto, SetLastError=true)]
internal static extern int GetCurrentDirectory(int nBufferLength,
StringBuilder lpBuffer);
---
Looks good? But it's not. It tries to provide both low level support
(it's needed to just call C function, like D extern(C), but do
everything at runtime (yes, in .NET Framework even Regex can be compiled
at runtime from a string)) and high level support (it converts UTF-16 to
ANSI in the given MSDN example). As a result it even doesn't allow you
to specify a (scalar) parameter as a length of another (array) parameter
or throw on native function failure.
What can be done in that direction in D? The whole my point is expressed
in my small cwrap library:
http://deoma-cmd.ru/d/docs/src/cwrap.html
"Examples" is the most informative section of the documentation. Other
examples can be found in (original functions, all in one file):
https://bitbucket.org/denis_sh/cwrap/src/tip/examples/c.d
and generated wrappers:
https://bitbucket.org/denis_sh/cwrap/src/tip/out
Library state: almost every internal function is unittested, generates
compilable code which looks right but not unittested yet.
NOTE: You need head dmd to CTFE it or you can just change `enum data` to
`auto data` in examples/c.d, this will also reduce compilation time more
than x10.
Once original function is properly described in IDL, such library gives
the fallowing advantages:
* [memory-corruption-safe] User works with arrays, not separate pointers
and lengths.
* [memory-leak-safe] Original function allocated memory will be freed on
any failure (NOTE: see "Original function restrictions" in
documentation).
* [fail-safe] An exception will be thrown if original function fails.
* [usability] Much easier to work with arrays/preferred string types (it
converts UTF-8/16/32 back and forth, see docs).
* [speed] No unnecessary GC (or other) allocations/copying/reading.
* [garbage-free] Look at, e.g. std.file - it leaves lots of garbage
because of calling native platform functions. Generated wrappers don't
even allocate GC memory (except for output parameters).
Looks like this functionality is usable for auto-generated bindings, at
least a useful GtkD still leaks memory almost everywhere because it
never frees (OK, not sure, I just haven't seen) allocated memory marked
by "free with g_free".
So should such stuff be ever included in Phobos? IMHO yes because it's a
general stuff. But if not, can generated wrappers only be included and
used to improve current situation "a system language with a library
which leaves garbage"?
By the way, std.file.getcwd and std.windows.charset.toMBSz are examples
of hand-written wrappers (the first is mine and strictly speaking forced
me to do this lib).
P.S.
Not sure if this library usable at all because I'm as sure in its
usability as in tabs indention superiority over spaces indention. )
So if it's not (usable), tell me without scruple.
Thanks for reading!
This idea came to me the other day when working with Windows and Posix
API. Out of the enhancements you describe I find most useful:
- string / array conversion
- memory management
- errno / return value to exception (already implemented as a function in
exception.d)
The only reason not to use your wrappers, is if they don't expose all
features of the original or if you don't need D types because you pass the
result of function A to function B directly. This could happen with some
char* returning functions where you don't want to slow down your
application by converting forth and back from Windows charset to UTF-8. It
is definitely something for the "modern convenience" bullet point.
Jan 29 2012
On 01/29/2012 09:48 PM, Denis Shelomovskij wrote:D has complete (IMHO) compiler support for calling C functions (using extern(C)). But there is a lack of library support. Readers from this NG probably know this, but... Microsoft .NET Framework has such support. Look briefly at source example in: http://msdn.microsoft.com/en-gb/library/s97shtze.aspx And at System.IO.Directory.GetCurrentDirectory source: http://typedescriptor.net/name/members/5540086E017CD13896E80A0CAEA6E517-System.IO.Directory.GetCurrentDirectory%28%29 where Win32Native.GetCurrentDirectory is defined as: --- [DllImport("kernel32.dll", CharSet=CharSet.Auto, SetLastError=true)] internal static extern int GetCurrentDirectory(int nBufferLength, StringBuilder lpBuffer); --- Looks good? But it's not. It tries to provide both low level support (it's needed to just call C function, like D extern(C), but do everything at runtime (yes, in .NET Framework even Regex can be compiled at runtime from a string)) and high level support (it converts UTF-16 to ANSI in the given MSDN example). As a result it even doesn't allow you to specify a (scalar) parameter as a length of another (array) parameter or throw on native function failure. What can be done in that direction in D? The whole my point is expressed in my small cwrap library: http://deoma-cmd.ru/d/docs/src/cwrap.html "Examples" is the most informative section of the documentation. Other examples can be found in (original functions, all in one file): https://bitbucket.org/denis_sh/cwrap/src/tip/examples/c.d and generated wrappers: https://bitbucket.org/denis_sh/cwrap/src/tip/out Library state: almost every internal function is unittested, generates compilable code which looks right but not unittested yet. NOTE: You need head dmd to CTFE it or you can just change `enum data` to `auto data` in examples/c.d, this will also reduce compilation time more than x10. Once original function is properly described in IDL, such library gives the fallowing advantages: * [memory-corruption-safe] User works with arrays, not separate pointers and lengths. * [memory-leak-safe] Original function allocated memory will be freed on any failure (NOTE: see "Original function restrictions" in documentation). * [fail-safe] An exception will be thrown if original function fails. * [usability] Much easier to work with arrays/preferred string types (it converts UTF-8/16/32 back and forth, see docs). * [speed] No unnecessary GC (or other) allocations/copying/reading. * [garbage-free] Look at, e.g. std.file - it leaves lots of garbage because of calling native platform functions. Generated wrappers don't even allocate GC memory (except for output parameters). Looks like this functionality is usable for auto-generated bindings, at least a useful GtkD still leaks memory almost everywhere because it never frees (OK, not sure, I just haven't seen) allocated memory marked by "free with g_free". So should such stuff be ever included in Phobos? IMHO yes because it's a general stuff.+1.But if not, can generated wrappers only be included and used to improve current situation "a system language with a library which leaves garbage"? By the way, std.file.getcwd and std.windows.charset.toMBSz are examples of hand-written wrappers (the first is mine and strictly speaking forced me to do this lib). P.S. Not sure if this library usable at all because I'm as sure in its usability as in tabs indention superiority over spaces indention. ) So if it's not (usable), tell me without scruple. Thanks for reading!Looks like very cool stuff. In this example: // from cstring.h __(out) int memcmp( __(in[num] ) void* ptr1, __(in[+num]) void* ptr2, __(in ) size_t num ) __( success: true ); I suppose the unary + before the second 'num' is required to disambiguate from which array length 'num' will be deduced? Shouldn't this restriction be lifted? (Obviously, if the lengths have to match, both are fine.) In this example: // from WinBase.h __(out) DWORD GetCurrentDirectoryW( __(in) DWORD nBufferLength, __(out""[nBufferLength]->""[return]) LPWSTR lpBuffer ) __( repeat: return > nBufferLength ) __( success: return ); // NOTE: current directory can't have zero length Maybe the generated bindings should rather look like char[] winapi_GetCurrentDirectoryW();
Jan 29 2012
30.01.2012 4:42, Timon Gehr пишет:I suppose the unary + before the second 'num' is required to disambiguate from which array length 'num' will be deduced? Shouldn't this restriction be lifted? (Obviously, if the lengths have to match, both are fine.)Yes, this restriction isn't necessary. I just was too lazy to add new asserts in such cases. Fill be fixed.In this example: // from WinBase.h __(out) DWORD GetCurrentDirectoryW( __(in) DWORD nBufferLength, __(out""[nBufferLength]->""[return]) LPWSTR lpBuffer ) __( repeat: return > nBufferLength ) __( success: return ); // NOTE: current directory can't have zero length Maybe the generated bindings should rather look like char[] winapi_GetCurrentDirectoryW();Maybe. Looks like "if return value is a length, corresponding array should be returned instead" rule is good. If I will not find anything major against this rule will be added.
Jan 30 2012
static assert(__traits(compiles, HANDLE.init is null));hmm... What declaration of HANDLE do you use?
Jan 29 2012
30.01.2012 8:59, Kagamin пишет:Wrapper knows nothing about non-standard types, but in `CreateFileW` the last argument is `__(in?) HANDLE hTemplateFile`, so it is an optional parameter (can be `null`) and wrapper ensures that it isn't a mistake and `HANDLE` is nullable.static assert(__traits(compiles, HANDLE.init is null));hmm... What declaration of HANDLE do you use?
Jan 30 2012
On Monday, 30 January 2012 at 10:53:17 UTC, Denis Shelomovskij wrote:30.01.2012 8:59, Kagamin пишет:Will it compile for alias int HANDLE; ?Wrapper knows nothing about non-standard types, but in `CreateFileW` the last argument is `__(in?) HANDLE hTemplateFile`, so it is an optional parameter (can be `null`) and wrapper ensures that it isn't a mistake and `HANDLE` is nullable.static assert(__traits(compiles, HANDLE.init is null));hmm... What declaration of HANDLE do you use?
Jan 30 2012
On Monday, 30 January 2012 at 15:51:40 UTC, Kagamin wrote:On Monday, 30 January 2012 at 10:53:17 UTC, Denis Shelomovskij wrote:Ah, got it.30.01.2012 8:59, Kagamin пишет:Will it compile for alias int HANDLE; ?Wrapper knows nothing about non-standard types, but in `CreateFileW` the last argument is `__(in?) HANDLE hTemplateFile`, so it is an optional parameter (can be `null`) and wrapper ensures that it isn't a mistake and `HANDLE` is nullable.static assert(__traits(compiles, HANDLE.init is null));hmm... What declaration of HANDLE do you use?
Jan 30 2012