www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - wmemchar for unix

reply "monarch_dodra" <monarchdodra gmail.com> writes:
For performance reasons, I need a "w" version of memchr.

C defines wmemchr as:
wchar_t * wmemchr ( const wchar_t *, wchar_t, size_t );

Unfortunatly, on unix, "wchar_t" is defined a *4* bytes long,
making wmemchr, effectivelly, "dmemchr".

Are there any "2 byte" alternatives for wmemchr on unix?
Aug 26 2013
next sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
On Aug 26, 2013, at 11:57 PM, monarch_dodra <monarchdodra gmail.com> =
wrote:


 For performance reasons, I need a "w" version of memchr.
=20
 C defines wmemchr as:
 wchar_t * wmemchr ( const wchar_t *, wchar_t, size_t );
=20
 Unfortunatly, on unix, "wchar_t" is defined a *4* bytes long,
 making wmemchr, effectivelly, "dmemchr".
=20
 Are there any "2 byte" alternatives for wmemchr on unix?


Why not cast the array to ushort[] and do a find()?  Or is that too slow =
as well?=
Aug 27 2013
parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Tuesday, 27 August 2013 at 14:37:14 UTC, Sean Kelly wrote:

 On Aug 26, 2013, at 11:57 PM, monarch_dodra 
 <monarchdodra gmail.com> wrote:


 For performance reasons, I need a "w" version of memchr.
 
 C defines wmemchr as:
 wchar_t * wmemchr ( const wchar_t *, wchar_t, size_t );
 
 Unfortunatly, on unix, "wchar_t" is defined a *4* bytes long,
 making wmemchr, effectivelly, "dmemchr".
 
 Are there any "2 byte" alternatives for wmemchr on unix?


 Why not cast the array to ushort[] and do a find()?  Or is that 
 too slow as well?


Because it's specifically to speed up find's implementation ^^
Aug 27 2013
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Aug 27, 2013 at 07:37:02AM -0700, Sean Kelly wrote:

 On Aug 26, 2013, at 11:57 PM, monarch_dodra <monarchdodra gmail.com> wrote:
 

 For performance reasons, I need a "w" version of memchr.
 
 C defines wmemchr as:
 wchar_t * wmemchr ( const wchar_t *, wchar_t, size_t );
 
 Unfortunatly, on unix, "wchar_t" is defined a *4* bytes long,
 making wmemchr, effectivelly, "dmemchr".
 
 Are there any "2 byte" alternatives for wmemchr on unix?


 
 Why not cast the array to ushort[] and do a find()?  Or is that too
 slow as well?


Optimized searches of this kind ideally translate to the various rep*
instructions on x86. I'm not sure if dmd does that optimization. If you
really feel inclined, you could do static if (X86) and throw in an asm
block (but that would break purity,  safety, etc., so probably not a
good idea).

You *might* be able to coax GDC (or LDC) to do loop unrolling and/or
substitution with rep* instructions with just plain D code, though.
Can't really say without trying it out.


T

-- 
They pretend to pay us, and we pretend to work. -- Russian saying
Aug 27 2013
next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Tuesday, 27 August 2013 at 14:43:10 UTC, H. S. Teoh wrote:

 On Tue, Aug 27, 2013 at 07:37:02AM -0700, Sean Kelly wrote:

 On Aug 26, 2013, at 11:57 PM, monarch_dodra 
 <monarchdodra gmail.com> wrote:
 

 For performance reasons, I need a "w" version of memchr.
 
 C defines wmemchr as:
 wchar_t * wmemchr ( const wchar_t *, wchar_t, size_t );
 
 Unfortunatly, on unix, "wchar_t" is defined a *4* bytes long,
 making wmemchr, effectivelly, "dmemchr".
 
 Are there any "2 byte" alternatives for wmemchr on unix?


 
 Why not cast the array to ushort[] and do a find()?  Or is 
 that too
 slow as well?


 Optimized searches of this kind ideally translate to the 
 various rep*
 instructions on x86. I'm not sure if dmd does that 
 optimization. If you
 really feel inclined, you could do static if (X86) and throw in 
 an asm
 block (but that would break purity,  safety, etc., so probably 
 not a
 good idea).

 You *might* be able to coax GDC (or LDC) to do loop unrolling 
 and/or
 substitution with rep* instructions with just plain D code, 
 though.
 Can't really say without trying it out.


 T


Hum... I think that is too complicated for what I'm trying to do. 
I don't know assembly enough. Ideally, I was hoping for a 
pre-existing C function to do the work for me :)

For now, I can settle for a simple:
version (windows)
     //use fast wmemchr
else
     //use standard code

But It feels weird, in the sense that there is no reason for 
"2byte" search to be more specific to windows than for unix...
Aug 27 2013
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
27-Aug-2013 18:41, H. S. Teoh пишет:

 On Tue, Aug 27, 2013 at 07:37:02AM -0700, Sean Kelly wrote:

 On Aug 26, 2013, at 11:57 PM, monarch_dodra <monarchdodra gmail.com> wrote:


 For performance reasons, I need a "w" version of memchr.

 C defines wmemchr as:
 wchar_t * wmemchr ( const wchar_t *, wchar_t, size_t );

 Unfortunatly, on unix, "wchar_t" is defined a *4* bytes long,
 making wmemchr, effectivelly, "dmemchr".

 Are there any "2 byte" alternatives for wmemchr on unix?


 Why not cast the array to ushort[] and do a find()?  Or is that too
 slow as well?


 Optimized searches of this kind ideally translate to the various rep*
 instructions on x86.


Rather a loop with XMM moves. What's best though is always a moving target.


 I'm not sure if dmd does that optimization. If you
 really feel inclined, you could do static if (X86) and throw in an asm
 block (but that would break purity,  safety, etc., so probably not a
 good idea).


It would be awesome to have pure D analogs for memchr, memcpy and its 
ilk that won't be so limiting (as in types used) but would guarantee top 
performance.

-- 
Dmitry Olshansky
Aug 27 2013
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Aug 27, 2013 at 11:18:50PM +0400, Dmitry Olshansky wrote:

 27-Aug-2013 18:41, H. S. Teoh пишет:

On Tue, Aug 27, 2013 at 07:37:02AM -0700, Sean Kelly wrote:

On Aug 26, 2013, at 11:57 PM, monarch_dodra <monarchdodra gmail.com> wrote:


For performance reasons, I need a "w" version of memchr.

C defines wmemchr as:
wchar_t * wmemchr ( const wchar_t *, wchar_t, size_t );

Unfortunatly, on unix, "wchar_t" is defined a *4* bytes long,
making wmemchr, effectivelly, "dmemchr".

Are there any "2 byte" alternatives for wmemchr on unix?


Why not cast the array to ushort[] and do a find()?  Or is that too
slow as well?


Optimized searches of this kind ideally translate to the various rep*
instructions on x86.


 
 Rather a loop with XMM moves. What's best though is always a moving
 target.
 

I'm not sure if dmd does that optimization. If you really feel
inclined, you could do static if (X86) and throw in an asm block (but
that would break purity,  safety, etc., so probably not a good idea).


 
 It would be awesome to have pure D analogs for memchr, memcpy and its
 ilk that won't be so limiting (as in types used) but would guarantee
 top performance.


[...]

Those would have to be compiler intrinsics, since they are CPU-dependent
optimizations. Plus they could improve dmd code generation. :)


T

-- 
The only difference between male factor and malefactor is just a little
emptiness inside.
Aug 27 2013
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
27-Aug-2013 23:31, H. S. Teoh пишет:

 On Tue, Aug 27, 2013 at 11:18:50PM +0400, Dmitry Olshansky wrote:

 27-Aug-2013 18:41, H. S. Teoh пишет:




[snip]

 I'm not sure if dmd does that optimization. If you really feel
 inclined, you could do static if (X86) and throw in an asm block (but
 that would break purity,  safety, etc., so probably not a good idea).


 It would be awesome to have pure D analogs for memchr, memcpy and its
 ilk that won't be so limiting (as in types used) but would guarantee
 top performance.


 [...]

 Those would have to be compiler intrinsics, since they are CPU-dependent
 optimizations. Plus they could improve dmd code generation. :)


It would be golden to finally see a time where compilers have D-specific 
intrinsics! :)


-- 
Dmitry Olshansky
Aug 27 2013