www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Encouraging preliminary results implementing memcpy in D

reply Mike Franklin <slavo5150 yahoo.com> writes:
I had a little fun today kicking the crap out of C's memcpy with 
a D implementation.

https://github.com/JinShil/memcpyD

Request for help: I don't have a Linux system running on real 
hardware at this time, nor do I have a wide range of platforms 
and machines to test with.  If you'd like to help me with this 
potentially foolish endeavor, please run the program on your 
hardware and send me the results.

Feedback, advise, and pull requests to improve the implementation 
are most welcome.

Mike
Jun 12 2018
next sibling parent Antonio Corbi <acrb ggmail.com> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:

 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.


Hi Mike,
These are my results running your program under archlinux x86_64 
with the zen-kernel 4.17.1, the hardware is powered by an ancient 
"Intel(R) Core(TM)2 CPU 6600    2.40GHz" with 4GB of ram:

size memcpyC memcpyD
1 67607 56810
2 68105 57638
4 66760 58949
8 66943 61262
16 71937 43821
32 70955 48392
64 111473 54226
128 144784 77165
256 183504 113597
512 289039 180930
1024 450526 1314835
2048 782029 1890236
4096 1627622 3165319
8192 2751701 5614202
16384 6361074 11484517
32768 30931212 42805529
65536 61878379 86000892

size memcpyC memcpyD
1 66796 44745
1 66773 44343
1 66780 44157
2 66769 44370
2 66792 44529
4 66776 44298
4 66775 44412
8 66766 44409
8 70945 44359
4 66804 44367
8 71007 44432
16 75210 50656
Jun 13 2018
prev sibling next sibling parent reply drug <drug2004 bk.ru> writes:
Ubuntu 18.04 Linux 4.15.0-23-generic
AMD® Fx(tm)-8350 eight-core processor × 8

size memcpyC memcpyD
1 51089 36921
2 45896 35733
4 46079 36200
8 48443 37509
16 48669 24925
32 52917 27787
64 55631 44928
128 84282 47795
256 107350 66009
512 159310 126795
1024 247683 452560
2048 440687 673211
4096 1129135 1304085
8192 4740910 4095254
16384 8389579 8874273
32768 16630336 17370310
65536 33032013 42904705

size memcpyC memcpyD
1 52354 28365
1 48407 28445
1 50264 30273
2 51312 27708
2 46138 28973
4 52753 28535
4 52150 27418
8 52220 27276
8 49625 27804
4 49356 33510
8 48529 27668
16 52662 135357

second run

size memcpyC memcpyD
1 47248 36964
2 45624 35627
4 45535 35596
8 47920 37012
16 47960 25107
32 52798 27394
64 55444 44282
128 76819 41055
256 105852 66429
512 157629 126243
1024 253841 448974
2048 438973 667101
4096 1144280 1337549
8192 3647558 4141162
16384 8301059 8722185
32768 16413116 17506957
65536 32958933 40381270

size memcpyC memcpyD
1 48513 26288
1 46080 26842
1 48526 26989
2 48634 26419
2 43522 27150
4 48229 25737
4 52841 28117
8 49632 25913
8 46325 25487
4 40267 32343
8 45990 25220
16 46509 124042
Jun 13 2018
parent Mike Franklin <slavo5150 yahoo.com> writes:
On Wednesday, 13 June 2018 at 08:55:40 UTC, drug wrote:

 Ubuntu 18.04 Linux 4.15.0-23-generic
 AMD® Fx(tm)-8350 eight-core processor × 8

 size memcpyC memcpyD
 1 51089 36921
 2 45896 35733
 4 46079 36200
 8 48443 37509
 16 48669 24925
 32 52917 27787
 64 55631 44928
 128 84282 47795
 256 107350 66009
 512 159310 126795
 1024 247683 452560
 2048 440687 673211
 4096 1129135 1304085
 8192 4740910 4095254
 16384 8389579 8874273
 32768 16630336 17370310
 65536 33032013 42904705

 size memcpyC memcpyD
 1 52354 28365
 1 48407 28445
 1 50264 30273
 2 51312 27708
 2 46138 28973
 4 52753 28535
 4 52150 27418
 8 52220 27276
 8 49625 27804
 4 49356 33510
 8 48529 27668
 16 52662 135357


Interesting! I have an AMD 8370 running Windows 8, and I get more 
favorable results in Windows:

size memcpyC memcpyD
1 45361 43626
2 55091 43791
4 70507 43714
8 50910 42854
16 63328 28831
32 72817 30790
64 76307 45823
128 97180 55368
256 164935 68362
512 230508 132100
1024 502189 490590
2048 892968 823070
4096 1896480 1456353
8192 4530645 4516681
16384 10886602 9921215
32768 21717080 19116839
65536 59787610 43549445

size memcpyC memcpyD
1 48770 30084
1 49169 30921
1 43370 30144
2 51404 27571
2 56002 29729
4 69588 29804
4 63743 29510
8 55492 29002
8 46752 31793
4 72673 28858
8 48989 27547
10 55527 121628

In your results, I see that for sizes 1024 and higher (that's 
when is dispatches to the REP MOVSB algorithm), the performance 
begins to degrade for Linux.  I'm going to install Linux soon and 
see if I can fix that.

Thanks for the data,

Mike
Jun 13 2018
prev sibling next sibling parent reply ToRuSer <ToRuSer ToRuSer.com> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:

 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.

 Mike


All Tor users now apparently have their posts subjected to 
'moderation'.

(i.e. someone, will, perhaps, at some point, get around to 
reviewing their posts, and then, perhaps, it might reach the 
forum, or not.)

So... maybe..you'll get those posts...or maybe not...and when.. 
is anyones guess.

So well done to the D community, for discriminating against all 
the Tor users out there. You've done yourself proud.
Jun 13 2018
next sibling parent Basile B. <b2.b2.b2.b2.b2.temp.temp.temp gmx.gmx.gmx.com.com> writes:
On Wednesday, 13 June 2018 at 09:07:21 UTC, ToRuSer wrote:

 On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:

 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.

 Mike


 All Tor users now apparently have their posts subjected to 
 'moderation'.

 (i.e. someone, will, perhaps, at some point, get around to 
 reviewing their posts, and then, perhaps, it might reach the 
 forum, or not.)

 So... maybe..you'll get those posts...or maybe not...and when.. 
 is anyones guess.

 So well done to the D community, for discriminating against all 
 the Tor users out there. You've done yourself proud.


The problem is likely more that someone has used Tor to troll 
here and then the enpoint used was blacklisted.
Jun 13 2018
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 06/13/2018 02:07 AM, ToRuSer wrote:


 So well done to the D community, for discriminating against all the Tor
 users out there. You've done yourself proud.


I am part of the D community. I haven't discriminated against anyone. I 
don't know what a Tor user is.

I've just searched: So Tor is an old idea of mine, implemented. :o)

Ali
Jun 13 2018
parent reply errExit <err error.com> writes:
On Wednesday, 13 June 2018 at 17:04:11 UTC, Ali Çehreli wrote:

 I am part of the D community. I haven't discriminated against 
 anyone. I don't know what a Tor user is.

 I've just searched: So Tor is an old idea of mine, implemented. 
 :o)

 Ali


Tor is our last line of defence against an Orson Wells future, 
where everyones actions are scrutinized by big brother, so that 
big brother can use that knowledge to put fear into, control and 
manipulate, those that don't conform.

assert("bad tor user" != "all tor users are bad");

(actually there are more bad non-tor users)

Unfortunately, it's becoming increasingly, the norm, to 
discriminate against tor users (no doubt those doing that 
discrimination are those that are happy to conform, of which 
there will be many, sadly).

https://people.torproject.org/~lunar/20160331-CloudFlare_Fact_Sheet.pdf
Jun 13 2018
next sibling parent reply Cym13 <cpicard openmailbox.org> writes:
On Thursday, 14 June 2018 at 02:32:51 UTC, errExit wrote:

 Tor is our last line of defence against an Orson Wells future, 
 where everyones actions are scrutinized by big brother, so that 
 big brother can use that knowledge to put fear into, control 
 and manipulate, those that don't conform.

 assert("bad tor user" != "all tor users are bad");

 (actually there are more bad non-tor users)

 Unfortunately, it's becoming increasingly, the norm, to 
 discriminate against tor users (no doubt those doing that 
 discrimination are those that are happy to conform, of which 
 there will be many, sadly).

 https://people.torproject.org/~lunar/20160331-CloudFlare_Fact_Sheet.pdf


Don't mistake spammer management with discrimination. I share 
your frustration that TOR isn't more usable than it is today with 
CloudFlare etc, but coming with political reasons holds no water 
if the reason why it was blacklisted wasn't political in the 
first place. It's not false, it just won't work.

Hopefully once that particular user gets discouraged or we find a 
way to actually avoid user impersonification, then things will be 
able to come back to normal.
Jun 13 2018
next sibling parent reply errExit <errExit err.com> writes:
On Thursday, 14 June 2018 at 03:59:47 UTC, Cym13 wrote:

 Don't mistake spammer management with discrimination. I share 
 your frustration that TOR isn't more usable than it is today 
 with CloudFlare etc, but coming with political reasons holds no 
 water if the reason why it was blacklisted wasn't political in 
 the first place. It's not false, it just won't work.

 Hopefully once that particular user gets discouraged or we find 
 a way to actually avoid user impersonification, then things 
 will be able to come back to normal.


The D Foundation now subjects all users having an ip originating 
from a tor exit node, to having their posts moderated (but by 
whom, when, how, criteria ?? etc).

Literally millions of people could, and probably would, be using 
that exit node.

So that is plain discrimination. It's not spammer management.

Forcing people to identify themselves, is also not about spammer 
management either.

The D Foundation IS now discimantory against those that want that 
believe that freedom and privacy is some to be protected.

This becomes problematic for those of us who work in 'certain 
organisations', that insist on tracking it's employees online 
activities (even outside of the workplace).

It's a shame the D Foundation has finally succumed to the big 
brother mentality - under the guise of protecting you from spam.

https://blog.torproject.org/dont-let-facebook-or-any-tracker-follow-you-web
Jun 13 2018
parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 06/14/2018 02:34 AM, errExit wrote:

 
 The D Foundation now subjects all users having an ip originating from a 
 tor exit node, to having their posts moderated (but by whom, when, how, 
 criteria ?? etc).
 
 Literally millions of people could, and probably would, be using that 
 exit node.
 
 So that is plain discrimination. It's not spammer management.
 
 Forcing people to identify themselves, is also not about spammer 
 management either.
 
 The D Foundation IS now discimantory against those that want that 
 believe that freedom and privacy is some to be protected.
 
 This becomes problematic for those of us who work in 'certain 
 organisations', that insist on tracking it's employees online activities 
 (even outside of the workplace).
 
 It's a shame the D Foundation has finally succumed to the big brother 
 mentality - under the guise of protecting you from spam.
 
 https://blog.torproject.org/dont-let-facebook-or-any-tracker-follow-you-web
 


I'm with you on a lot of that, however, this part troubles me:

"This becomes problematic for those of us who work in 'certain 
organisations', that insist on tracking it's employees online activities 
(even outside of the workplace)."

If I worked in such an organization that tracked its employees 
activities *outside the workplace*, I'd LEAVE it ASAP, and I'd strongly 
suggest anyone else do the same. I mean what insane workplace is that, 
the 1920's mob? Apple?

Honestly, if you believe strongly enough in Tor to use it, why in the 
world would you willfully aid and abed an organization that does that 
sort of thing? It doesn't make any sense at all. It's EXTREMELY 
self-contradictory and completely erodes your entire stance. If you're 
going to preach for personal freedom and privacy, at least have the 
basic integrity to LIVE the basic ideals you're preaching even when 
doing so ISN'T so trivial as installing a mere web browser.
Jun 14 2018
next sibling parent errExit <errExit err.com> writes:
On Thursday, 14 June 2018 at 07:19:31 UTC, Nick Sabalausky 
(Abscissa) wrote:

 I'm with you on a lot of that, however, this part troubles me:

 "This becomes problematic for those of us who work in 'certain 
 organisations', that insist on tracking it's employees online 
 activities (even outside of the workplace)."

 If I worked in such an organization that tracked its employees 
 activities *outside the workplace*, I'd LEAVE it ASAP, and I'd 
 strongly suggest anyone else do the same. I mean what insane 
 workplace is that, the 1920's mob? Apple?

 Honestly, if you believe strongly enough in Tor to use it, why 
 in the world would you willfully aid and abed an organization 
 that does that sort of thing? It doesn't make any sense at all. 
 It's EXTREMELY self-contradictory and completely erodes your 
 entire stance. If you're going to preach for personal freedom 
 and privacy, at least have the basic integrity to LIVE the 
 basic ideals you're preaching even when doing so ISN'T so 
 trivial as installing a mere web browser.



Sadly, it's increasingly 'standard practice' in HR to do just 
that. Monitor what employees, and potential employees have done 
or made available on the internet.

I don't support that approach, which is exactly why I use Tor.

Now, their attempts are moot, and therefore I am in no way 
supporting those actions.

And so, your comments about 'self-contradictory' are moot also.

In addition, HR data is increasingly becoming a valuable target 
for attack, due to the 'profiles' they keep on people. So now the 
situation gets even worse. Cause not only does HR have this info, 
so will others... eventually.

The only real world option, is to prevent them from gathering 
data on you in the first place.

btw. Some people in my team (those that use D), might have 
contributed to this post, but now, likely will not.
Jun 14 2018
prev sibling parent reply AnotherTorUser <me me.com> writes:
On Thursday, 14 June 2018 at 07:19:31 UTC, Nick Sabalausky 
(Abscissa) wrote:

 I'm with you on a lot of that, however, this part troubles me:

 "This becomes problematic for those of us who work in 'certain 
 organisations', that insist on tracking it's employees online 
 activities (even outside of the workplace)."

 If I worked in such an organization that tracked its employees 
 activities *outside the workplace*, I'd LEAVE it ASAP, and I'd 
 strongly suggest anyone else do the same. I mean what insane 
 workplace is that, the 1920's mob? Apple?

 Honestly, if you believe strongly enough in Tor to use it, why 
 in the world would you willfully aid and abed an organization 
 that does that sort of thing? It doesn't make any sense at all. 
 It's EXTREMELY self-contradictory and completely erodes your 
 entire stance. If you're going to preach for personal freedom 
 and privacy, at least have the basic integrity to LIVE the 
 basic ideals you're preaching even when doing so ISN'T so 
 trivial as installing a mere web browser.


sorry. but what world do you live in?

If all such people stopped working for such companies, what do 
you think the economic impact would be?

Tor prevents tracking, and therefore it is not contradictory to 
work for such companies - because they can't track you (or at 
least, it become much more difficult to do so).
Jun 14 2018
parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 06/14/2018 05:01 AM, AnotherTorUser wrote:

 
 sorry. but what world do you live in?


Note this part of what I said:


 If I worked in such an organization that tracked its employees
 activities *outside the workplace*




That part is KEY, but it sounds like you an errExit have chosen to 
ignore that part.


 If all such people stopped working for such companies, what do you think 
 the economic impact would be?


What do you think is the social impact if they don't? And don't even try 
to pretend the companies can't trivially solve the "economic" issues for 
themselves in an instant by knocking off the behaviour that causes loss 
of talent.
Jun 14 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Thursday, June 14, 2018 16:04:32 Nick Sabalausky  via Digitalmars-d-
announce wrote:

 On 06/14/2018 05:01 AM, AnotherTorUser wrote:

 If all such people stopped working for such companies, what do you think
 the economic impact would be?


 What do you think is the social impact if they don't? And don't even try
 to pretend the companies can't trivially solve the "economic" issues for
 themselves in an instant by knocking off the behaviour that causes loss
 of talent.


But that would imply that they have a frontal lobe. :)

In all seriousness, it is surprising how frequently companies seem to be
incapable of making decisions that would fix a lot of their problems, and
they seem to be incredibly prone to thinking about things in a shortsighted
manner.

I'm reminded of an article by Joel Spoelskey where he talks about how one of
the key things that a source control software solution can do to make it
more likely for folks to be willing to try it is to make it easy to get your
source code and history back out again and into another source control
system. However, companies typically freak out at the idea of making it easy
to switch from their product to another product. They're quite willing to
make it easy to switch _to_ their product so that they can start making
money off of you, but the idea that making it low cost to leave could
actually improve the odds of someone trying their product - and thus
increase their profits - seems to be beyond them.

Another case which is closer to the exact topic at hand is that many
companies seem to forget how much it costs to hire someone when they
consider what they should do to make it so that their employees are willing
- or even eager - to stay. Spending more money on current employees (be that
on salary or something else to make the workplace desirable) or avoiding
practices that tick employees off so that they leave can often save money in
the long run, but companies frequently ignore that fact. They're usually
more interested in saving on the bottom line right now than making decisions
that save money over time.

So, while I completely agree that companies can technically make decisions
that solve some of their problems with things like retaining talent, it
seems like it's frequently the case that they're simply incapable of doing
it in practice - though YMMV; some companies are better about it than
others.

- Jonathan M Davis
Jun 14 2018
parent Joakim <dlang joakim.fea.st> writes:
On Thursday, 14 June 2018 at 20:59:06 UTC, Jonathan M Davis wrote:

 On Thursday, June 14, 2018 16:04:32 Nick Sabalausky  via 
 Digitalmars-d- announce wrote:

 On 06/14/2018 05:01 AM, AnotherTorUser wrote:

 If all such people stopped working for such companies, what 
 do you think the economic impact would be?


 What do you think is the social impact if they don't? And 
 don't even try to pretend the companies can't trivially solve 
 the "economic" issues for themselves in an instant by knocking 
 off the behaviour that causes loss of talent.


 But that would imply that they have a frontal lobe. :)

 In all seriousness, it is surprising how frequently companies 
 seem to be incapable of making decisions that would fix a lot 
 of their problems, and they seem to be incredibly prone to 
 thinking about things in a shortsighted manner.

 I'm reminded of an article by Joel Spoelskey where he talks 
 about how one of the key things that a source control software 
 solution can do to make it more likely for folks to be willing 
 to try it is to make it easy to get your source code and 
 history back out again and into another source control system. 
 However, companies typically freak out at the idea of making it 
 easy to switch from their product to another product. They're 
 quite willing to make it easy to switch _to_ their product so 
 that they can start making money off of you, but the idea that 
 making it low cost to leave could actually improve the odds of 
 someone trying their product - and thus increase their profits 
 - seems to be beyond them.

 Another case which is closer to the exact topic at hand is that 
 many companies seem to forget how much it costs to hire someone 
 when they consider what they should do to make it so that their 
 employees are willing - or even eager - to stay. Spending more 
 money on current employees (be that on salary or something else 
 to make the workplace desirable) or avoiding practices that 
 tick employees off so that they leave can often save money in 
 the long run, but companies frequently ignore that fact. 
 They're usually more interested in saving on the bottom line 
 right now than making decisions that save money over time.

 So, while I completely agree that companies can technically 
 make decisions that solve some of their problems with things 
 like retaining talent, it seems like it's frequently the case 
 that they're simply incapable of doing it in practice - though 
 YMMV; some companies are better about it than others.


This was an interesting read on that topic, which I've linked on 
this forum before, where an engineer points out that companies 
would be better off not chasing "rockstars" with hot keywords on 
their resumes but improving their training, processes, and 
culture so that even average programmers can be productive, 
including mentioning using source control and the Joel test that 
you just referenced:

https://danluu.com/programmer-moneyball/

Of course, the reason companies mostly don't do it is they're 
prone to the same cognitive failings as anybody else: it's easier 
to chase a quick fix than doing the hard work of putting in a 
system like this.
Jun 14 2018
prev sibling parent Arafel <er.krali gmail.com> writes:
Well, e-mail was never meant to be reliable or secure... BTW, there are 
already solutions to prevent impersonation: sign the messages with 
either PGP or S/MIME... the former is more decentralised, and the later 
usually comes together with stronger verifications, like personal 
identification of the sender (I mean, whoever issues the certificate for 
S/MIME should ideally check your name, etc.) :-)

Although admittedly that'd work only for the NNTP / mailing list, I 
don't know how that could be adapted to the web "forum" interface...

On 06/14/2018 05:59 AM, Cym13 wrote:

 
 Hopefully once that particular user gets discouraged or we find a way to 
 actually avoid user impersonification, then things will be able to come 
 back to normal.
Jun 15 2018
prev sibling next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 14 June 2018 at 02:32:51 UTC, errExit wrote:

 On Wednesday, 13 June 2018 at 17:04:11 UTC, Ali Çehreli wrote:

 I am part of the D community. I haven't discriminated against 
 anyone. I don't know what a Tor user is.

 I've just searched: So Tor is an old idea of mine, 
 implemented. :o)

 Ali


 Tor is our last line of defence against an Orson Wells future, 
 where everyones actions are scrutinized by big brother, so that 
 big brother can use that knowledge to put fear into, control 
 and manipulate, those that don't conform.

 assert("bad tor user" != "all tor users are bad");

 (actually there are more bad non-tor users)

 Unfortunately, it's becoming increasingly, the norm, to 
 discriminate against tor users (no doubt those doing that 
 discrimination are those that are happy to conform, of which 
 there will be many, sadly).

 https://people.torproject.org/~lunar/20160331-CloudFlare_Fact_Sheet.pdf


Tor is merely one tool used to route around those building 
centralized systems on top of the internet. The real solution is 
that as more and more decentralized tech does well, like git or 
cryptocurrencies, to get rid of these obsolete centralized 
systems altogether.
Jun 14 2018
parent YetAnotherTorUser <YetAnotherTorUser YetAnotherTorUser.com> writes:
On Thursday, 14 June 2018 at 07:45:15 UTC, Joakim wrote:

 Tor is merely one tool used to route around those building 
 centralized systems on top of the internet. The real solution 
 is that as more and more decentralized tech does well, like git 
 or cryptocurrencies, to get rid of these obsolete centralized 
 systems altogether.


The problem with the D Foundation's 'new approach' is, that there 
is simply no visibility as to what is being moderated, how long 
it takes, whom does it, and for what the filtering criteria is. 
We're just being told that it is too avoid 'spammers'??

If that is not a tool for abuse, then I don't know what is.
Jun 14 2018
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/13/18 10:32 PM, errExit wrote:

 On Wednesday, 13 June 2018 at 17:04:11 UTC, Ali Çehreli wrote:

 I am part of the D community. I haven't discriminated against anyone. 
 I don't know what a Tor user is.

 I've just searched: So Tor is an old idea of mine, implemented. :o)

 Ali


 
 Tor is our last line of defence against an Orson Wells future, where 
 everyones actions are scrutinized by big brother, so that big brother 
 can use that knowledge to put fear into, control and manipulate, those 
 that don't conform.


Sorry KingJoffrey/psychoticRabbit/etc, you literally only started using 
Tor when your steady IP was banned. This argument kind of falls down 
when your past behavior is examined.

It's really the impersonation that is the problem, not the anonymity. If 
you just would stick to one persona, and especially not impersonate 
people who actually post on this forum, you would not have to use Tor at 
all, and the forum moderators wouldn't have to worry about active 
moderation.

-Steve
Jun 14 2018
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 15/06/2018 1:22 AM, Steven Schveighoffer wrote:

 It's really the impersonation that is the problem, not the anonymity. If 
 you just would stick to one persona, and especially not impersonate 
 people who actually post on this forum, you would not have to use Tor at 
 all, and the forum moderators wouldn't have to worry about active 
 moderation.


Not quite. It was the attacking of the D community at large which 
convinced those with the power to actively ban, to ban "him".

We have in essence decided that this person will never be willing to 
have positive experiences within our community and have out right 
decided that we do not want them here under any circumstance.

A lot of work has gone into getting rid of "him". As far as I am 
concerned the original IP address that was used from Australia was in 
fact a proxy and had the single desire for this person, was to attack us.

Once they are gone, we can deactivate the extra protections that have 
been put in place, because until this person came along, we were quite 
happy moderating ourselves. It lasted nearly 20 years that peace...
Jun 14 2018
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/14/18 9:31 AM, rikki cattermole wrote:

 On 15/06/2018 1:22 AM, Steven Schveighoffer wrote:

 It's really the impersonation that is the problem, not the anonymity. 
 If you just would stick to one persona, and especially not impersonate 
 people who actually post on this forum, you would not have to use Tor 
 at all, and the forum moderators wouldn't have to worry about active 
 moderation.


 
 Not quite. It was the attacking of the D community at large which 
 convinced those with the power to actively ban, to ban "him".


Well, for me, it wasn't attacks. There have been quite a few people in 
this community who were rude, insulting, and IMO, that is not worth 
moderation. In fact some of them have even came around and become quite 
good D contributors. Those problems usually work themselves out because 
we have a great community which does not give trolls the attention they 
desire.

But when you start impersonating people, especially ones that post to 
this forum, you have crossed the line and are literally putting words 
into other's mouths. We've seen this before, and generally they go away, 
but this guy does not want to.

Now he's claiming "discrimination" ;)


 Once they are gone, we can deactivate the extra protections that have 
 been put in place, because until this person came along, we were quite 
 happy moderating ourselves. It lasted nearly 20 years that peace...


I agree, this will be a blip in our forum experience, and next month 
we'll barely remember him.

-Steve
Jun 14 2018
parent bachmeier <no spam.net> writes:
On Thursday, 14 June 2018 at 13:42:14 UTC, Steven Schveighoffer 
wrote:


 Well, for me, it wasn't attacks. There have been quite a few 
 people in this community who were rude, insulting, and IMO, 
 that is not worth moderation. In fact some of them have even 
 came around and become quite good D contributors. Those 
 problems usually work themselves out because we have a great 
 community which does not give trolls the attention they desire.


That seems to be the opinion of most. The problem I have is that 
this forum is the main way to communicate about the language, and 
thus it is how others form their opinion of the language.
Jun 14 2018
prev sibling next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:

 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.


If I read your benchmark graphs right, they claimed that 
allocating 16 kilobytes takes over 10^^6 usecs, with both 
mallocs. Doesn't that mean over a second, 16 kilobytes? Can't be! 
Are you confusing usecs with nsecs?
Jun 13 2018
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Wednesday, 13 June 2018 at 09:40:05 UTC, Dukc wrote:


 If I read your benchmark graphs right, they claimed that 
 allocating 16 kilobytes takes over 10^^6 usecs, with both 
 mallocs. Doesn't that mean over a second, 16 kilobytes? Can't 
 be! Are you confusing usecs with nsecs?


The benchmark doesn't allocate any data; it's just copying data.  
Each benchmark is run 10,000,000 times to smooth out some of the 
entropy in the results:  
https://github.com/JinShil/memcpyD/blob/2e0d3c33ea876a25a04358a3ae505b2eba
f99cb/memcpyd.d#L78  The usecs in the graph is the time it takes to run the
benchmark 10,000,000 times.

Mike
Jun 13 2018
parent reply Dukc <ajieskola gmail.com> writes:
On Wednesday, 13 June 2018 at 09:59:52 UTC, Mike Franklin wrote:

 The benchmark doesn't allocate any data; it's just copying data.

 Mike


Ah of course. I was thinking other stuff while writing.
Jun 13 2018
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Wednesday, 13 June 2018 at 10:13:13 UTC, Dukc wrote:

 On Wednesday, 13 June 2018 at 09:59:52 UTC, Mike Franklin wrote:

 The benchmark doesn't allocate any data; it's just copying 
 data.

 Mike


 Ah of course. I was thinking other stuff while writing.


Well, actually, I probably should divide that time by 10,000,000 
to make a more accurate representation.

Thanks for the feedback,

Mike
Jun 13 2018
parent Arredondo <arm.plus gmail.com> writes:
On Wednesday, 13 June 2018 at 10:17:10 UTC, Mike Franklin wrote:

 Well, actually, I probably should divide that time by 
 10,000,000 to make a more accurate representation.



For rigorous benchmarking, check out the first part of Andrei's 
Writing Fast Code:

https://www.youtube.com/watch?v=vrfYLlR8X8k

One takeaway is that taking the average of many runtimes is not 
the best use of your dataset.
Jun 13 2018
prev sibling next sibling parent reply Fra Mecca <me francescomecca.eu> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:

 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.

 Mike


I get this on Linux 4.16.3-gentoo, AMD FX(tm)-6100 Six-Core 
Processor, 8GiB ram,

using ldc2 -O3L:
size memcpyC memcpyD
1 5 0
2 0 0
4 0 0
8 0 0
16 1519 0
32 1833 0
64 3816 0
128 7543 0
256 146500 0
512 194818 0
1024 329593 846142
2048 714945 1117132
4096 1596170 1803621
8192 5899818 6110026
16384 12338384 14141850
32768 24971315 26771484
65536 49806637 63260128

size memcpyC memcpyD
1 0 0
1 0 0
1 0 0
2 0 0
2 0 0
4 0 0
4 0 0
8 0 0
8 0 0
4 0 0
8 0 0
core.exception.AssertError memcpyd.d(9): Assertion failure
----------------
??:? _d_assert [0xcaf056d5]
??:? [0xa015e7fe]
??:? [0xa0158cb0]
??:? void rt.dmain2._d_run_main(int, char**, extern (C) int 
function(char[][])*).runAll() [0xcaf29daf]
??:? _d_run_main [0xcaf29c7b]
??:? __libc_start_main [0xca160eeb]
??:? [0xa0158b29]
Jun 13 2018
parent Mike Franklin <slavo5150 yahoo.com> writes:
On Wednesday, 13 June 2018 at 12:45:26 UTC, Fra Mecca wrote:

 I get this on Linux 4.16.3-gentoo, AMD FX(tm)-6100 Six-Core 
 Processor, 8GiB ram,

 using ldc2 -O3L:
 size memcpyC memcpyD
 1 5 0
 2 0 0
 4 0 0
 8 0 0
 16 1519 0
 32 1833 0
 64 3816 0
 128 7543 0
 256 146500 0
 512 194818 0
 1024 329593 846142
 2048 714945 1117132
 4096 1596170 1803621
 8192 5899818 6110026
 16384 12338384 14141850
 32768 24971315 26771484
 65536 49806637 63260128

 size memcpyC memcpyD
 1 0 0
 1 0 0
 1 0 0
 2 0 0
 2 0 0
 4 0 0
 4 0 0
 8 0 0
 8 0 0
 4 0 0
 8 0 0
 core.exception.AssertError memcpyd.d(9): Assertion failure
 ----------------
 ??:? _d_assert [0xcaf056d5]
 ??:? [0xa015e7fe]
 ??:? [0xa0158cb0]
 ??:? void rt.dmain2._d_run_main(int, char**, extern (C) int 
 function(char[][])*).runAll() [0xcaf29daf]
 ??:? _d_run_main [0xcaf29c7b]
 ??:? __libc_start_main [0xca160eeb]
 ??:? [0xa0158b29]


Yes, optimizations are throwing code away.  I'm working on it.

Mike
Jun 13 2018
prev sibling next sibling parent reply Uknown <sireeshkodali1 gmail.com> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:

 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.

 Mike


I'm glad to see this kind of improvement! I noticed there wasn't 
any macOS testing though, so heres the results on my machine:

size memcpyC memcpyD
1 38613 5185
2 42475 7259
4 54272 3450
8 29391 3457
16 32947 3626
32 33761 7253
64 44152 14510
128 51985 27711
256 65700 58044
512 95315 119977
1024 150726 387021
2048 265975 667858
4096 521565 1113945
8192 965322 2039064
16384 3923818 3997524
32768 15586237 15669232
65536 33195890 31640205

size memcpyC memcpyD
1 40057 5180
1 36664 5226
1 37625 5280
2 36584 5178
2 36956 5252
4 51086 3448
4 49952 3456
8 29497 3455
8 29710 3987
4 51539 3453
8 29715 3449
16 29747 23535

sys info : Intel Core i5 4258u (28w), 8GB DDR3 1600MHz RAM

Looks very promising. One question though, why not use 
std.datetime.stopwatch.benchmark? I think it has some protection 
against optimizing compilers (I may be wrong though). Also, LDC 
has attributes to control optimizations on a per function 
basis.see : https://wiki.dlang.org/LDC-specific_language_changes

My linux install got messed up, but I'll post benchmarks as soon 
as its up
Jun 13 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, June 13, 2018 14:34:28 Uknown via Digitalmars-d-announce 
wrote:

 Looks very promising. One question though, why not use
 std.datetime.stopwatch.benchmark? I think it has some protection
 against optimizing compilers (I may be wrong though). Also, LDC
 has attributes to control optimizations on a per function
 basis.see : https://wiki.dlang.org/LDC-specific_language_changes


Unfortunately, std.datetime.stopwatch.benchmark does not yet have such
protections. It has been discussed, but there were issues with it that still
need to be sorted out.

In any case, what he has implemented is pretty much what's in Phobos except
for the fact that he set up his to take arguments, whereas Phobos' solution
just takes the function(s) to call, so anything that it does has to be
self-contained.

- Jonathan M Davis
Jun 14 2018
parent Uknown <sireeshkodali1 gmail.com> writes:
On Thursday, 14 June 2018 at 07:54:20 UTC, Jonathan M Davis wrote:

 On Wednesday, June 13, 2018 14:34:28 Uknown via 
 Digitalmars-d-announce wrote:

 Looks very promising. One question though, why not use 
 std.datetime.stopwatch.benchmark? I think it has some 
 protection against optimizing compilers (I may be wrong 
 though). Also, LDC has attributes to control optimizations on 
 a per function basis.see : 
 https://wiki.dlang.org/LDC-specific_language_changes


 Unfortunately, std.datetime.stopwatch.benchmark does not yet 
 have such protections. It has been discussed, but there were 
 issues with it that still need to be sorted out.

 In any case, what he has implemented is pretty much what's in 
 Phobos except for the fact that he set up his to take 
 arguments, whereas Phobos' solution just takes the function(s) 
 to call, so anything that it does has to be self-contained.

 - Jonathan M Davis


huh. I saw the PR and assumed it was accepted. Anyway, for DMD 
putting `asm` blocks seems to still disable optimizations, and 
for LDC, the pragma is perfect. GDC is the only unknown.
Jun 14 2018
prev sibling next sibling parent UnpaidTester <UnpaidTester UnpaidTester.com> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:

 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.

 Mike


specs
.........................................

(note: tried DMD v2.078.3) > dmd -O  (and got strange assertion 
errors).

(used LDC 1.8.0 instead) > ldc2 -O2

Intel(R) Core(TM) i7 CPU 920    2.67GHz
24GB Mem (Speed: 1066 MT/s)

Kubuntu 18.04 LTS - 4.15.0-23-generic
........................................


results
........................................
size memcpyC memcpyD
1 0 0
2 0 0
4 0 0
8 0 0
16 1247 0
32 1889 0
64 3055 0
128 5040 0
256 91407 0
512 158527 0
1024 253191 870780
2048 474243 1170349
4096 932151 1933045
8192 1894059 3284901
16384 4447945 6122015
32768 18572704 20772417
65536 37470068 40211203

size memcpyC memcpyD
1 0 0
1 0 0
1 0 0
2 0 0
2 0 0
4 0 0
4 0 0
8 0 0
8 0 0
4 0 0
8 0 0
16 0 0

..........................
Jun 13 2018
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/13/18 2:46 AM, Mike Franklin wrote:

 I had a little fun today kicking the crap out of C's memcpy with a D 
 implementation.
 
 https://github.com/JinShil/memcpyD
 
 Request for help: I don't have a Linux system running on real hardware 
 at this time, nor do I have a wide range of platforms and machines to 
 test with.  If you'd like to help me with this potentially foolish 
 endeavor, please run the program on your hardware and send me the results.


Just FYI, Linux results on a VM are still valid -- many people 
(including myself) only use Linux on a VM, so even though it's not a 
definitive win against glibc memcpy on Linux, the end result is, it's 
still faster for those users :)

I won't add much, since I'm using a Mac, and those numbers have already 
been posted.

But I wanted to say don't discount those findings off-hand.

-Steve
Jun 13 2018
parent Cym13 <cpicard openmailbox.org> writes:
On Wednesday, 13 June 2018 at 16:21:53 UTC, Steven Schveighoffer 
wrote:

 I won't add much, since I'm using a Mac, and those numbers have 
 already been posted.


Reproduction is an important part of the scientific process, 
please post away ;)

Also: memcpyD commit f5fb0fda0dbacc960ced59d7171ff76a95cc2abf on 
Archlinux native x86_64 4.16.13-2 with Intel(R) Core(TM) i7-3520M 
CPU   2.90GHz and 7681MB of RAM, with dmd and ldc, no flag and 
optimized.

DMD:

size memcpyC memcpyD
1 50392 30745
2 50189 33515
4 47493 33456
8 50282 33428
16 36385 36061
32 36212 32142
64 50141 31137
128 52947 52785
256 86422 76346
512 131191 128344
1024 227875 291414
2048 444865 449210
4096 826391 823613
8192 1542429 1545051
16384 3264964 3228990
32768 11644816 11902418
65536 23658237 24026304

size memcpyC memcpyD
1 43209 37462
1 42412 38043
1 42079 38002
2 56503 39923
2 50544 38033
4 47760 37239
4 48910 36976
8 51110 38065
8 53761 36297
4 48201 35548
8 54820 38036
16 39360 35409



DMD -O:

core.exception.AssertError memcpyd.d(9): Assertion failure
----------------
??:? _d_assertp [0xc4e79cc5]
??:? pure nothrow  nogc void 
memcpyd.verify!(real).verify(const(real), const(real)) 
[0xc4e77c2c]
??:? void memcpyd.test!(real).test() [0xc4e76d60]
??:? _Dmain [0xc4e63ecd]

1 42014 26518
2 46086 26486
4 45984 29272
8 48813 26484
16 34866 18126
32 32036 18107
64 46073 20892
128 45964 27993
256 82344 50250
512 133484 94766
1024 216402 270462
2048 436465 443122
4096 815875 801596
8192 1524872 1530453
16384 3721245 3584620
32768 11776185 11739396
65536 23702074 23589480

size memcpyC memcpyD
1 37755 15424
1 40486 15319
1 40505 15352
2 47097 15653
2 46160 15319
4 43180 18110
4 46041 15321
8 46014 15342
8 46066 15341
4 43277 18105
8 45962 15321


LDC:

size memcpyC memcpyD
1 56378 48873
2 59461 49025
4 50481 45299
8 53786 49517
16 46103 39122
32 48100 46139
64 83864 67401
128 83849 90426
256 122471 138309
512 169668 228472
1024 260461 276878
2048 444937 472365
4096 860962 825468
8192 1578929 1567154
16384 3429235 3284611
32768 11941494 11868947
65536 24052536 24112980

size memcpyC memcpyD
1 47853 33403
1 47623 32924
1 48190 33100
2 59752 33146
2 59574 34371
4 53928 35042
4 54131 31991
8 57929 35864
8 57372 32174
4 54901 33253
8 62537 34535
16 52487 38358


LDC -O2:

core.exception.AssertError memcpyd.d(9): Assertion failure
----------------
??:? _d_assert [0x7fba47d79b35]
??:? void memcpyd.test!(real).test() [0x5638cb483d6e]
??:? _Dmain [0x5638cb47ea30]
size memcpyC memcpyD
1 0 0
2 0 0
4 0 0
8 0 0
16 385 0
32 792 0
64 1542 0
128 2981 0
256 90108 0
512 126316 0
1024 217881 391419
2048 415182 620853
4096 1244805 1240074
8192 2428417 2414095
16384 4863280 4971193
32768 12968909 12966676
65536 26393408 26395410

size memcpyC memcpyD
1 0 0
1 0 0
1 0 0
2 0 0
2 0 0
4 0 0
4 0 0
8 0 0
8 0 0
4 0 0
8 0 0
Jun 13 2018
prev sibling next sibling parent Arun Chandrasekaran <aruncxy gmail.com> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:

 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.

 Mike


On 8 core, 16 GB Intel Skull Candy box running Ubuntu 18.04 64 
bit.

https://gist.githubusercontent.com/carun/f7c2c200b1be20d0a9489296d6601332/raw/db01bb8bc909c6048288fccc500bd15e5ee491b2/memcpyd-output.log

Hope this helps.
Jun 13 2018
prev sibling next sibling parent Diego <diego example.com> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:

 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.

 Mike


Hello Mike,

These are my results:

Ubuntu 16.04.4 amd64 Linux 4.16.0-ck1+
Intel Xeon E5-2620 0   2.00GHz - 12 cores
32 GB RAM



size memcpyC memcpyD
1 125349 47658
2 155014 50492
4 173099 52669
8 228236 52676
16 107897 32621
32 128039 32604
64 163644 37658
128 223840 50420
256 338769 90300
512 584772 171038
1024 878093 995813
2048 1346958 1254141
4096 2439378 2101284
8192 5631202 3554307
16384 9873090 6496635
32768 22489302 21328288
65536 50522961 45748356

size memcpyC memcpyD
1 123241 27631
1 130758 28165
1 123247 32748
2 142964 27587
2 140914 28103
4 168084 32616
4 171166 27590
8 228274 27604
8 233249 27605
4 168624 27597
8 238049 29435
16 103956 52730



(I think these are strange results)

size memcpyC memcpyD
1 0 0
2 0 0
4 0 0
8 0 0
16 559 0
32 1003 0
64 0 0
128 0 0
256 0 0
512 0 0
1024 460182 1519048
2048 739148 1973641
4096 1533047 3168472
8192 2913463 5560106
16384 6385370 10353178
32768 20889322 21487968
65536 44920382 48339716

size memcpyC memcpyD
1 0 0
1 0 0
1 0 0
2 0 0
2 0 0
4 0 0
4 0 0
8 0 0
8 0 0
4 0 0
8 0 0
16 0 0
Jun 14 2018
prev sibling next sibling parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:

 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.



Just a little remark. Alignment has also an extremely heavy 
impact on the performance of memcpy(). Does your test include it?
Jun 14 2018
parent reply baz <b2.b2.b2.temp.temp.temp gmx.gmx.gmx.com.com> writes:
On Thursday, 14 June 2018 at 17:27:27 UTC, Patrick Schluter wrote:

 On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:

 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.



 Just a little remark. Alignment has also an extremely heavy 
 impact on the performance of memcpy(). Does your test include 
 it?


I asked on IRC yesterday and actually tHose memcpy are not the 
memcpy we use to copy wide chunks, apparently it's rather for an 
internal druntime thing, i.e cpy type to type so likely always 
aligned.
Jun 14 2018
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Thursday, 14 June 2018 at 20:35:23 UTC, baz wrote:


 I asked on IRC yesterday and actually tHose memcpy are not the 
 memcpy we use to copy wide chunks, apparently it's rather for 
 an internal druntime thing, i.e cpy type to type so likely 
 always aligned.


Correct! D already has features like `a[] = b[]` so there is no 
reason to call `memcpy` directly; that is a job for the druntime. 
  `memcpyD` is intended to be an internal druntime utility.

However, we should still be good D citizens when we write our low 
level druntime code, so the interface will be `memcpy(T)(T a, T 
b)` and `memcpy(T)(T[] a, T[] b)` instead of `memcpy(void* a, 
void* b, size_t size)`.  This will ensure the code, at 
compile-time, adheres to D's guarantees such as ` safe`, 
`nothrow`, and `pure`.  And, given that it won't require 
`TypeInfo` it will be usable in -betterC.

Although rare, I believe it is still possible to have misaligned 
memory in D.  My understanding is that misaligned copies usually 
involve copying smaller chunks of the memory until they are 
aligned, and then copying the aligned memory in larger chunks.  I 
suspect that will work well with the D implementation.

TL;DR, Unaligned memory will be handled after optimized aligned 
memory copies are implemented.

Mike
Jun 14 2018
parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Friday, 15 June 2018 at 00:15:35 UTC, Mike Franklin wrote:

 On Thursday, 14 June 2018 at 20:35:23 UTC, baz wrote:


 [...]


 Correct! D already has features like `a[] = b[]` so there is no 
 reason to call `memcpy` directly; that is a job for the 
 druntime.
  `memcpyD` is intended to be an internal druntime utility.

 [...]


Ok, thanks for the explanation.
Jun 15 2018
prev sibling parent reply David Nadlinger <code klickverbot.at> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:

 https://github.com/JinShil/memcpyD

 […]

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.


The memcpyD implementation is buggy; it assumes that all 
arguments are aligned to their size. This isn't necessarily true. 
For example, `ubyte[1024].alignof == 1`, and struct alignment can 
also be set explicitly using align(N).

On x86, you can get away with this in a lot of cases even though 
it's undefined behaviour [1], but this is not necessarily the 
case for SSE/AVX instructions. In fact, that's probably a pretty 
good guess as to where those weird crashes you mentioned come 
from.

On other architectures, unaligned accesses might be outright 
unsupported.

For loading into vector registers, you can use 
core.simd.loadUnaligned instead (ldc.simd.loadUnaligned for LDC – 
LDC's druntime has not been updated yet after {load, 
store}Unaligned were added upstream as well).

  — David



[1] This is applying C rules to D, where creation of unaligned 
pointers is undefined behaviour. The D specification doesn't 
mention
Jun 17 2018
next sibling parent David Nadlinger <code klickverbot.at> writes:
On Sunday, 17 June 2018 at 17:00:00 UTC, David Nadlinger wrote:

 core.simd.loadUnaligned instead


Ah, well… https://issues.dlang.org/show_bug.cgi?id=19001

  — David
Jun 17 2018
prev sibling parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Sunday, 17 June 2018 at 17:00:00 UTC, David Nadlinger wrote:

 On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:

 https://github.com/JinShil/memcpyD

 […]

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.


 The memcpyD implementation is buggy; it assumes that all 
 arguments are aligned to their size. This isn't necessarily 
 true. For example, `ubyte[1024].alignof == 1`, and struct 
 alignment can also be set explicitly using align(N).


Yes, I'm already aware of that.  My plan is to create optimized 
implementations for aligned data, and then handled unaligned data 
as compositions of the various aligned implementations.  For 
example a 3 byte copy would be a short copy plus a byte copy.  
That may not be appropriate for all cases.  I'll have to measure, 
and adapt.


 On x86, you can get away with this in a lot of cases even 
 though it's undefined behaviour [1], but this is not 
 necessarily the case for SSE/AVX instructions. In fact, that's 
 probably a pretty good guess as to where those weird crashes 
 you mentioned come from.


Thanks! I think you're right.


 For loading into vector registers, you can use 
 core.simd.loadUnaligned instead (ldc.simd.loadUnaligned for LDC 
 – LDC's druntime has not been updated yet after {load, 
 store}Unaligned were added upstream as well).


Unfortunately the code gen is quite a bit worse:

Exibit A:
https://run.dlang.io/is/jIuHRG
*(cast(void16*)(&s2)) = *(cast(const void16*)(&s1));

_Dmain:
		push	RBP
		mov	RBP,RSP
		sub	RSP,020h
		lea	RAX,-020h[RBP]
		xor	ECX,ECX
		mov	[RAX],RCX
		mov	8[RAX],RCX
		lea	RDX,-010h[RBP]
		mov	[RDX],RCX
		mov	8[RDX],RCX
		movdqa	XMM0,-020h[RBP]
		movdqa	-010h[RBP],XMM0
		xor	EAX,EAX
		leave
		ret
		add	[RAX],AL
.text._Dmain	ends


Exhibit B:
https://run.dlang.io/is/PLRfhW
storeUnaligned(cast(void16*)(&s2), loadUnaligned(cast(const 
void16*)(&s1)));

_Dmain:
		push	RBP
		mov	RBP,RSP
		sub	RSP,050h
		lea	RAX,-050h[RBP]
		xor	ECX,ECX
		mov	[RAX],RCX
		mov	8[RAX],RCX
		lea	RDX,-040h[RBP]
		mov	[RDX],RCX
		mov	8[RDX],RCX
		mov	-030h[RBP],RDX
		mov	-010h[RBP],RAX
		movdqu	XMM0,[RAX]
		movdqa	-020h[RBP],XMM0
		movdqa	XMM1,-020h[RBP]
		movdqu	[RDX],XMM1
		xor	EAX,EAX
		leave
		ret
		add	[RAX],AL
.text._Dmain	ends


If the code gen was better, that would definitely be the way to 
go; to have unaligned and aligned share the same implementation.  
Maybe I can fix the DMD code gen, or implement a `copyUnaligned` 
intrinsic.

Also, there doesn't seem to be any equivalent 32-byte 
implementations in `core.simd`.  Is that just because noone's 
bother to implement them yet?  And with AVX512, we should 
probably have 64-byte implementations as well.

Mike
Jun 17 2018
parent Mike Franklin <slavo5150 yahoo.com> writes:
On Monday, 18 June 2018 at 02:31:25 UTC, Mike Franklin wrote:


 Unfortunately the code gen is quite a bit worse:


Scratch that.  If compiling with -O it seems to do the right 
thing.

Mike
Jun 17 2018