www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - mutable string

reply Adam Sansier <Adam.Sansier gmail.com> writes:
Is it possible to turn temporary char/wchar buffer in to a string 
to be used by string functions rather than having to convert?

I'm working with win32 and have to use char*'s. This requires a 
lot of in place case conversions and comparisons and such. I want 
to avoid the gc too.

I could use C strcmp etc but then I might as well just write the 
code in C.
Jul 10 2016
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 10 July 2016 at 19:19:57 UTC, Adam Sansier wrote:

 Is it possible to turn temporary char/wchar buffer in to a 
 string to be used by string functions rather than having to 
 convert?


What string functions in particular? If they are written 
correctly, it should just work.... mutable char[] can be passed 
as a char* or const char* with the .ptr property.



 I could use C strcmp etc but then I might as well just write 
 the code in C.


meh D rox over C even if you just use C functions
Jul 10 2016
parent reply Adam Sansier <Adam.Sansier gmail.com> writes:
On Sunday, 10 July 2016 at 19:44:01 UTC, Adam D. Ruppe wrote:

 On Sunday, 10 July 2016 at 19:19:57 UTC, Adam Sansier wrote:

 Is it possible to turn temporary char/wchar buffer in to a 
 string to be used by string functions rather than having to 
 convert?


 What string functions in particular? If they are written 
 correctly, it should just work.... mutable char[] can be passed 
 as a char* or const char* with the .ptr property.



 I could use C strcmp etc but then I might as well just write 
 the code in C.


 meh D rox over C even if you just use C functions


I'm trying to use windows registry functions. This requires 
passing a char(sometimes a TCHAR) buffer.

But I need to compare the results, case insensitive, to a 
string(since that is what D uses). It's being a real pain in the 
butt to deal with the mixture.

I tried using slices, which works for char but not wchar as 
string is not a immutable(wchar). I don't want to use wide 
strings because that doesn't seem to be a "thing" in d.

This whole char vs wchar vs dchar is a big confusing, not in what 
they are but how they all interrelate between D and windows and 
their doesn't seem to be an easy way to convert between them all 
unless one wants to litter their code with to!'s.
Jul 10 2016
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 10 July 2016 at 19:50:28 UTC, Adam Sansier wrote:

 But I need to compare the results, case insensitive, to a 
 string(since that is what D uses). It's being a real pain in 
 the butt to deal with the mixture.


Are your strings literals or generated somewhere else?

You can stick a w at the end of a string literal to make it a 
wchar string: `"this is wchar"w`

Most library functions in Phobos at least will work the same with 
wstrings as string.
Jul 10 2016
prev sibling parent reply Lodovico Giaretta <lodovico giaretart.net> writes:
On Sunday, 10 July 2016 at 19:50:28 UTC, Adam Sansier wrote:

 On Sunday, 10 July 2016 at 19:44:01 UTC, Adam D. Ruppe wrote:

 On Sunday, 10 July 2016 at 19:19:57 UTC, Adam Sansier wrote:

 Is it possible to turn temporary char/wchar buffer in to a 
 string to be used by string functions rather than having to 
 convert?


 What string functions in particular? If they are written 
 correctly, it should just work.... mutable char[] can be 
 passed as a char* or const char* with the .ptr property.



 I could use C strcmp etc but then I might as well just write 
 the code in C.


 meh D rox over C even if you just use C functions


 I'm trying to use windows registry functions. This requires 
 passing a char(sometimes a TCHAR) buffer.

 But I need to compare the results, case insensitive, to a 
 string(since that is what D uses). It's being a real pain in 
 the butt to deal with the mixture.

 I tried using slices, which works for char but not wchar as 
 string is not a immutable(wchar). I don't want to use wide 
 strings because that doesn't seem to be a "thing" in d.

 This whole char vs wchar vs dchar is a big confusing, not in 
 what they are but how they all interrelate between D and 
 windows and their doesn't seem to be an easy way to convert 
 between them all unless one wants to litter their code with 
 to!'s.


Well, the main problem I always faced is Windows having quite a 
bad string handling...

That said, if you want char[] -> string or wchar[] -> wstring you 
can use assumeUnique, which casts a mutable array to an immutable 
one.
If you have an array of ubyte, ushort or int, and would like to 
use it as a string, you can use assumeUTF, which casts it 
respectively to string, wstring or dstring.
But both functions are in fact just casts.

Also keep in mind that in D a string literal will behave as a 
string, wstring or dstring based on the type of variable to which 
it is assigned.

This are just some hints, as I don't really understand what is 
not working for you. Do you mind showing a snippet?
Jul 10 2016
next sibling parent ag0aep6g <anonymous example.com> writes:
On 07/10/2016 10:31 PM, Lodovico Giaretta wrote:

 That said, if you want char[] -> string or wchar[] -> wstring you can
 use assumeUnique, which casts a mutable array to an immutable one.


After having ensured that the array is actually unique. That is, there 
must not be any other references to it or into it, or you're going to 
run into undefined behavior.
Jul 10 2016
prev sibling parent reply Adam Sansier <Adam.Sansier gmail.com> writes:
On Sunday, 10 July 2016 at 20:31:34 UTC, Lodovico Giaretta wrote:

 On Sunday, 10 July 2016 at 19:50:28 UTC, Adam Sansier wrote:

 On Sunday, 10 July 2016 at 19:44:01 UTC, Adam D. Ruppe wrote:

 On Sunday, 10 July 2016 at 19:19:57 UTC, Adam Sansier wrote:

 Is it possible to turn temporary char/wchar buffer in to a 
 string to be used by string functions rather than having to 
 convert?


 What string functions in particular? If they are written 
 correctly, it should just work.... mutable char[] can be 
 passed as a char* or const char* with the .ptr property.



 I could use C strcmp etc but then I might as well just write 
 the code in C.


 meh D rox over C even if you just use C functions


 I'm trying to use windows registry functions. This requires 
 passing a char(sometimes a TCHAR) buffer.

 But I need to compare the results, case insensitive, to a 
 string(since that is what D uses). It's being a real pain in 
 the butt to deal with the mixture.

 I tried using slices, which works for char but not wchar as 
 string is not a immutable(wchar). I don't want to use wide 
 strings because that doesn't seem to be a "thing" in d.

 This whole char vs wchar vs dchar is a big confusing, not in 
 what they are but how they all interrelate between D and 
 windows and their doesn't seem to be an easy way to convert 
 between them all unless one wants to litter their code with 
 to!'s.


 Well, the main problem I always faced is Windows having quite a 
 bad string handling...

 That said, if you want char[] -> string or wchar[] -> wstring 
 you can use assumeUnique, which casts a mutable array to an 
 immutable one.
 If you have an array of ubyte, ushort or int, and would like to 
 use it as a string, you can use assumeUTF, which casts it 
 respectively to string, wstring or dstring.
 But both functions are in fact just casts.

 Also keep in mind that in D a string literal will behave as a 
 string, wstring or dstring based on the type of variable to 
 which it is assigned.

 This are just some hints, as I don't really understand what is 
 not working for you. Do you mind showing a snippet?


The problem is things like

https://msdn.microsoft.com/en-us/library/windows/desktop/ms724902(v=vs.85).aspx

require data buffers to be used. I can't just plug in a wstring 
to it, can I?

`First-chance exception: core.exception.UnicodeException illegal 
UTF-16 value at src\rt\util\utf.d(400)`

This happens when I call toLower on a wstring trying to lower all 
the characters.


Also, if I clear the buffer before it is used, to!wstring returns 
an empty string. Even though right before the call to to!wstring 
the buffer contains the string. If I just cast instead of call 
to!wstring, it works but the length is 1024, the original size of 
the buffer.

I'm slowing debugging the code and most things work but it's just 
a mess trying to sort out all the different things versions of 
chars and strings to make sure that things works and hopefully 
are robust.
Jul 10 2016
next sibling parent reply Adam Sansier <Adam.Sansier gmail.com> writes:
For example, I'm trying to compare a wchar buffer with a wstring 
using slices:

x[0..$] == y[0..$]

It fails. I think because x has length 1024. If do

x[0..y.length] == str[0..y.length]

it fails, also because y has length 1024(since it was generated 
from a buffer and the length wasn't set correctly).

If I fix the length to the size of y, it passes, but isn't 
correct since it is a partial match.

also, to lower fails but CharLowerBuff passes(the error in the 
previous post)

	//str = str.toLower();	
	CharLowerBuff(cast(LPTSTR)str, str.length);
Jul 10 2016
next sibling parent reply Adam Sansier <Adam.Sansier gmail.com> writes:
On Sunday, 10 July 2016 at 21:26:29 UTC, Adam Sansier wrote:

 For example, I'm trying to compare a wchar buffer with a 
 wstring using slices:

 x[0..$] == y[0..$]

 It fails. I think because x has length 1024. If do

 x[0..y.length] == str[0..y.length]

 it fails, also because y has length 1024(since it was generated 
 from a buffer and the length wasn't set correctly).

 If I fix the length to the size of y, it passes, but isn't 
 correct since it is a partial match.

 also, to lower fails but CharLowerBuff passes(the error in the 
 previous post)

 	//str = str.toLower();	
 	CharLowerBuff(cast(LPTSTR)str, str.length);


Using this code

import core.stdc.wchar_; // For wcslen.

wstring toWstring(wchar[] value)
{
      return value ? cast(wstring) 
value[0..wcslen(cast(wchar*)value)] : null;
}


works and sets the length. Must be a bug in to!wstring!
Jul 10 2016
parent ag0aep6g <anonymous example.com> writes:
On 07/10/2016 11:38 PM, Adam Sansier wrote:

 Using this code

 import core.stdc.wchar_; // For wcslen.


Hint: D has special syntax for importing only specific parts of a module:

     import core.std.wchar_: wcslen;


 wstring toWstring(wchar[] value)
 {
       return value ? cast(wstring) value[0..wcslen(cast(wchar*)value)] :
 null;
 }


You're casting mutable data to immutable. That's only valid if you 
ensure that the data in value is never going to be mutated ever again. 
Better just return the `value[0 .. wcslen(...)]` slice without casting. 
Or use .idup if you really need a wstring, but you haven't indicated 
that you need a wstring.

Instead of `cast(wchar*)value` you can use `value.ptr`. Means exactly 
the same but looks nicer and is less bug-prone.


 works and sets the length. Must be a bug in to!wstring!


I doubt it. But it's possible, of course. Show code that demonstrates 
the bug and we can check.
Jul 10 2016
prev sibling parent reply ag0aep6g <anonymous example.com> writes:
On 07/10/2016 11:26 PM, Adam Sansier wrote:

 For example, I'm trying to compare a wchar buffer with a wstring using
 slices:

 x[0..$] == y[0..$]

 It fails. I think because x has length 1024. If do

 x[0..y.length] == str[0..y.length]

 it fails, also because y has length 1024(since it was generated from a
 buffer and the length wasn't set correctly).


So you fill the buffer with a call to some Windows function. The Windows 
function either returns the amount of data it wrote, and/or it 
zero-terminates the string which it writes.

If it returns the length, you use that when slicing the buffer. Then you 
can compare.

If it zero-terminates, you search for zero in the buffer and count along 
to determine the length. I wanted to say that there's a function for 
this in phobos, but fromStringz [1] seems to be for char arrays only.


[1] https://dlang.org/phobos/std_string.html#.fromStringz
Jul 10 2016
parent Jonathan M Davis via Digitalmars-d-learn writes:
On Sunday, July 10, 2016 23:38:26 ag0aep6g via Digitalmars-d-learn wrote:

 On 07/10/2016 11:26 PM, Adam Sansier wrote:

 For example, I'm trying to compare a wchar buffer with a wstring using
 slices:

 x[0..$] == y[0..$]

 It fails. I think because x has length 1024. If do

 x[0..y.length] == str[0..y.length]

 it fails, also because y has length 1024(since it was generated from a
 buffer and the length wasn't set correctly).


 So you fill the buffer with a call to some Windows function. The Windows
 function either returns the amount of data it wrote, and/or it
 zero-terminates the string which it writes.

 If it returns the length, you use that when slicing the buffer. Then you
 can compare.

 If it zero-terminates, you search for zero in the buffer and count along
 to determine the length. I wanted to say that there's a function for
 this in phobos, but fromStringz [1] seems to be for char arrays only.


 [1] https://dlang.org/phobos/std_string.html#.fromStringz


There's std.utf.toUTFz for converting to arbitrary, null-terminated,
strings, but I don't think that there's a fromUTFz. fromStringz is a more
recent addition, I believe, and whoever added it, didn't add the more
general version.

However, there _is_ a function to get the length of a zero-terminated
wchar*/wchar[]. You can use core.stdc.wchar_.wcslen and pass it a pointer to
the buffer - e.g.

auto length = wcslen(buffer.ptr);

So, when you call the Windows function, you end up with something like

wchar[1024] buffer;
auto result = someWindowsFunc(buffer.ptr, someArg, someOtherArg);
if(/+ result is error +/)
    throw new Exception("some error message");
buffer[$ - 1] = '/0'; // just in case
wstring str = buffer[0 .. wcslen(buffer)].idup;

- Jonathan M Davis
Jul 10 2016
prev sibling parent ag0aep6g <anonymous example.com> writes:
On 07/10/2016 11:17 PM, Adam Sansier wrote:

 The problem is things like

 https://msdn.microsoft.com/en-us/library/windows/desktop/ms724902(v=vs.85).aspx


 require data buffers to be used. I can't just plug in a wstring to it,
 can I?


You can't.


 `First-chance exception: core.exception.UnicodeException illegal UTF-16
 value at src\rt\util\utf.d(400)`

 This happens when I call toLower on a wstring trying to lower all the
 characters.


Looks like you've got bad data. How did you make the wstring?
Jul 10 2016