www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Developing blockchain software with D, not C++

reply aberba <karabutaworld gmail.com> writes:
In this video[1] from 2016, developer talks about C++ memory 
safety features, meta-programming, maturity and few others as 
main reasons they choose it for developing their blockchain 
software (the way I got it from a quick view).

Besides, D maturity (which I can't confirm or deny), what else 
does D miss to be considered a better alternative for blockchain 
in 2018?

D is also more productive, has safety and unittest built-in.


1. https://www.youtube.com/watch?v=w4jq4frE5v4
Dec 30 2017
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Saturday, 30 December 2017 at 16:59:41 UTC, aberba wrote:

 Besides, D maturity (which I can't confirm or deny), what else 
 does D miss to be considered a better alternative for 
 blockchain in 2018?


You can write blockchain software in any language you want. The 


I don't think "better" in this context is a technical issue, more 
of a cultural one. Well, portability could be an issue.
Dec 31 2017
parent reply aberba <karabutaworld gmail.com> writes:
On Sunday, 31 December 2017 at 09:45:52 UTC, Ola Fosheim Grøstad 
wrote:

 On Saturday, 30 December 2017 at 16:59:41 UTC, aberba wrote:

 Besides, D maturity (which I can't confirm or deny), what else 
 does D miss to be considered a better alternative for 
 blockchain in 2018?


 You can write blockchain software in any language you want. The 
 reference implementation for OpenChain seems to be written in 



That's not the point. JavaScript can also be used (technically), 
but you don't want to use it (for their use case). Seems you 
haven't watched the video so you are speaking out of context.

 what else does D miss





 I don't think "better" in this context is a technical issue, 
 more of a cultural one. Well, portability could be an issue.


There is is better when you consider things in context. The talk 
was focused on a particular context.
Jan 01 2018
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 1 January 2018 at 10:46:23 UTC, aberba wrote:

 On Sunday, 31 December 2017 at 09:45:52 UTC, Ola Fosheim 
 Grøstad wrote:

 On Saturday, 30 December 2017 at 16:59:41 UTC, aberba wrote:

 Besides, D maturity (which I can't confirm or deny), what 
 else does D miss to be considered a better alternative for 
 blockchain in 2018?


 You can write blockchain software in any language you want. 
 The reference implementation for OpenChain seems to be written 



 That's not the point. JavaScript can also be used 
 (technically), but you don't want to use it (for their use 
 case). Seems you haven't watched the video so you are speaking 
 out of context.


Well, the video seems to be mostly about building a large 
key-value store without caching… So whatever language has been 
used for that before would be suitable, basically any language 
that can interface well with C/Posix…

But you want static typing or verification when security is an 
issue, so Javascript is obviously out.


 There is is better when you consider things in context. The 
 talk was focused on a particular context.


Then you need to be a bit more clear on what you mean. You can 
make a lot of assumptions to shoehorn a project to something 
specific, e.g. database will not fit in main memory, but that 
might not hold if you control the nodes and can afford large 
amounts of memory etc. "Blockchain software" is a very vague 
requirement… What kind of load do you expect, what are the 
real-time requirements (nanoseconds vs minutes).

Remember that even Javascript can run at close to 25-50% of the 
performance of portable C… So it really depends on what you do, 
what architecture you design and what the long term development 
process looks like. Mixing languages is often the better choice 
for many projects.

I am not arguing with their choice, but it was for a C++ 
conference, so obviously they would praise C++…
Jan 01 2018
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 1 January 2018 at 11:24:45 UTC, Ola Fosheim Grøstad 
wrote:

 I am not arguing with their choice, but it was for a C++ 
 conference, so obviously they would praise C++…


Btw, I think one should be very sceptical of such presentations 
in general:

1.  They are there to market their project, so they have a strong 
incentive to appeal to the audience.  Offending the audience 
would be a bad strategy.

2.  Developers often have technological preferences in advance 
and will create arguments to defend their choice on rather 
subjective terms using "objective" claims.

3. The design and scope of a project tend to grow in the 
direction that the chosen platform is most suited for.

FWIW, I think the best technical choice for security related 
infrastructure would be to write a library in C or Ada or some 
other toolset where proven verification capabilities are 
available, then tie it together using whatever is most convenient 
on the platform you target.
Jan 01 2018
parent reply aberba <karabutaworld gmail.com> writes:
On Monday, 1 January 2018 at 12:24:29 UTC, Ola Fosheim Grøstad 
wrote:

 On Monday, 1 January 2018 at 11:24:45 UTC, Ola Fosheim Grøstad 
 wrote:

 [...]


 Btw, I think one should be very sceptical of such presentations 
 in general:

 [...]


Come on! Ada?
Jan 01 2018
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 1 January 2018 at 13:34:39 UTC, aberba wrote:

 On Monday, 1 January 2018 at 12:24:29 UTC, Ola Fosheim Grøstad 
 wrote:

 On Monday, 1 January 2018 at 11:24:45 UTC, Ola Fosheim Grøstad 
 wrote:

 [...]


 Btw, I think one should be very sceptical of such 
 presentations in general:

 [...]


 Come on! Ada?


Ada was designed for DoD projects…

https://www.adacore.com/sparkpro
Jan 01 2018
prev sibling next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Saturday, 30 December 2017 at 16:59:41 UTC, aberba wrote:

 In this video[1] from 2016, developer talks about C++ memory 
 safety features, meta-programming, maturity and few others as 
 main reasons they choose it for developing their blockchain 
 software (the way I got it from a quick view).

 Besides, D maturity (which I can't confirm or deny), what else 
 does D miss to be considered a better alternative for 
 blockchain in 2018?

 D is also more productive, has safety and unittest built-in.


 1. https://www.youtube.com/watch?v=w4jq4frE5v4


The better question is whether blockchain is worth implementing 
in D? ;) It wastes a ton of energy with the most common 
proof-of-work approach and, as I've pointed out here before, 
there are better ways to do it:

https://arstechnica.com/tech-policy/2017/12/bitcoins-insane-energy-consumption-explained/
http://forum.dlang.org/post/xzuzvykrqouqlsjmkvug forum.dlang.org

But some decentralized approach, if not blockchain, is 
undoubtedly going to be a huge winner, and of course D would be a 
great language with which to build it.  The lead developer of 
Bitcoin Cash, currently the fourth-most valuable cryptocurrency 
(https://coinmarketcap.com), gave a talk on doing crypto in D at 
the last DConf:

http://dconf.org/2017/talks/sechet.html

I don't think he's using D since they forked the Bitcoin source 
though.
Dec 31 2017
parent aberba <karabutaworld gmail.com> writes:
On Sunday, 31 December 2017 at 10:58:09 UTC, Joakim wrote:

 On Saturday, 30 December 2017 at 16:59:41 UTC, aberba wrote:

 In this video[1] from 2016, developer talks about C++ memory 
 safety features, meta-programming, maturity and few others as 
 main reasons they choose it for developing their blockchain 
 software (the way I got it from a quick view).

 Besides, D maturity (which I can't confirm or deny), what else 
 does D miss to be considered a better alternative for 
 blockchain in 2018?

 D is also more productive, has safety and unittest built-in.


 1. https://www.youtube.com/watch?v=w4jq4frE5v4


 The better question is whether blockchain is worth implementing 
 in D? ;) It wastes a ton of energy with the most common 
 proof-of-work approach and, as I've pointed out here before, 
 there are better ways to do it:


Maybe, maybe not. Let the developer decide for none of us know 
the future of such technology with certainty.

 https://arstechnica.com/tech-policy/2017/12/bitcoins-insane-energy-consumption-explained/
 http://forum.dlang.org/post/xzuzvykrqouqlsjmkvug forum.dlang.org

 But some decentralized approach, if not blockchain, is 
 undoubtedly going to be a huge winner, and of course D would be 
 a great language with which to build it.  The lead developer of 
 Bitcoin Cash, currently the fourth-most valuable cryptocurrency 
 (https://coinmarketcap.com), gave a talk on doing crypto in D 
 at the last DConf:

 http://dconf.org/2017/talks/sechet.html


That talk was more about cryptography/performance/security IMO. I 
don't recall him talking about blockchain software with D. Seems 
he is in a good position to give such a talk now that 
blockchain/cryptocurrency is getting loud.


 I don't think he's using D since they forked the Bitcoin source 
 though.
Jan 01 2018
prev sibling next sibling parent Tromos <tromosx gmail.com> writes:
On Saturday, 30 December 2017 at 16:59:41 UTC, aberba wrote:

 In this video[1] from 2016, developer talks about C++ memory 
 safety features, meta-programming, maturity and few others as 
 main reasons they choose it for developing their blockchain 
 software (the way I got it from a quick view).

 Besides, D maturity (which I can't confirm or deny), what else 
 does D miss to be considered a better alternative for 
 blockchain in 2018?

 D is also more productive, has safety and unittest built-in.


 1. https://www.youtube.com/watch?v=w4jq4frE5v4


We can have a full implementation of bitcoin protocol in D, if 
anyone is interested we can start it..
Jan 04 2018
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Saturday, 30 December 2017 at 16:59:41 UTC, aberba wrote:

 In this video[1] from 2016, developer talks about C++ memory 
 safety features, meta-programming, maturity and few others as 
 main reasons they choose it for developing their blockchain 
 software (the way I got it from a quick view).

 Besides, D maturity (which I can't confirm or deny), what else 
 does D miss to be considered a better alternative for 
 blockchain in 2018?

 D is also more productive, has safety and unittest built-in.


 1. https://www.youtube.com/watch?v=w4jq4frE5v4


I can talk about this first hand as I have a project running in 
D. However, I would sadly not recommend D ATM for such a project 
for 2 reasons:
1/ It is practically not possible to write efficient crypto 
routines without cent/ucent, short of writing them in asm.
2/ The network layer become very tedious very quick because of 
how broken shared is and because there is no ownership mechanism. 
While none of this is present in C++ it doesn't get in your way 
either.

I would *LOVE* to be able to use more D on a day to day basis, 
but these 2 problems make it very hard. It is especially sad 
considering 1/ could be solved very easily. It literally took me 
less than 1h to add support for it in SDC.
Jan 04 2018
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2018-01-04 14:42, deadalnix wrote:


 I would *LOVE* to be able to use more D on a day to day basis, but these 
 2 problems make it very hard. It is especially sad considering 1/ could 
 be solved very easily. It literally took me less than 1h to add support 
 for it in SDC.


There have been commits from the LDC developers related to cent/ucent 
fairly recently.

-- 
/Jacob Carlborg
Jan 04 2018
prev sibling next sibling parent Tromos <tromosx gmail.com> writes:
On Thursday, 4 January 2018 at 13:42:10 UTC, deadalnix wrote:

 On Saturday, 30 December 2017 at 16:59:41 UTC, aberba wrote:

 In this video[1] from 2016, developer talks about C++ memory 
 safety features, meta-programming, maturity and few others as 
 main reasons they choose it for developing their blockchain 
 software (the way I got it from a quick view).

 Besides, D maturity (which I can't confirm or deny), what else 
 does D miss to be considered a better alternative for 
 blockchain in 2018?

 D is also more productive, has safety and unittest built-in.


 1. https://www.youtube.com/watch?v=w4jq4frE5v4


 I can talk about this first hand as I have a project running in 
 D. However, I would sadly not recommend D ATM for such a 
 project for 2 reasons:
 1/ It is practically not possible to write efficient crypto 
 routines without cent/ucent, short of writing them in asm.
 2/ The network layer become very tedious very quick because of 
 how broken shared is and because there is no ownership 
 mechanism. While none of this is present in C++ it doesn't get 
 in your way either.

 I would *LOVE* to be able to use more D on a day to day basis, 
 but these 2 problems make it very hard. It is especially sad 
 considering 1/ could be solved very easily. It literally took 
 me less than 1h to add support for it in SDC.


Not a drama, can be solved, problems are opportunities
Jan 04 2018
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/4/18 8:42 AM, deadalnix wrote:

 On Saturday, 30 December 2017 at 16:59:41 UTC, aberba wrote:

 In this video[1] from 2016, developer talks about C++ memory safety 
 features, meta-programming, maturity and few others as main reasons 
 they choose it for developing their blockchain software (the way I got 
 it from a quick view).

 Besides, D maturity (which I can't confirm or deny), what else does D 
 miss to be considered a better alternative for blockchain in 2018?

 D is also more productive, has safety and unittest built-in.


 1. https://www.youtube.com/watch?v=w4jq4frE5v4


 
 I can talk about this first hand as I have a project running in D. 
 However, I would sadly not recommend D ATM for such a project for 2 
 reasons:
 1/ It is practically not possible to write efficient crypto routines 
 without cent/ucent, short of writing them in asm.
 2/ The network layer become very tedious very quick because of how 
 broken shared is and because there is no ownership mechanism. While none 
 of this is present in C++ it doesn't get in your way either.
 
 I would *LOVE* to be able to use more D on a day to day basis, but these 
 2 problems make it very hard. It is especially sad considering 1/ could 
 be solved very easily. It literally took me less than 1h to add support 
 for it in SDC.


Thanks for these thoughts!

* (u)cent support
* fixes for the shared qualifier
* ownership mechanism

These took less than 1h to add support for? That would be awesome... but 
realistically only the (u)cent sounds like that size of effort.

I've always wondered why we can't implement struct LargeInt(uint bytes) 
as a library mechanism for larged fixed-size integers, with asm 
specialization for LargeInt!8. Is adding the type to the compiler 
necessary, and if so, why?


Andrei
Jan 14 2018
next sibling parent rjframe <dlang ryanjframe.com> writes:
On Sun, 14 Jan 2018 18:03:27 -0500, Andrei Alexandrescu wrote:


 On 1/4/18 8:42 AM, deadalnix wrote:

 I would *LOVE* to be able to use more D on a day to day basis, but
 these 2 problems make it very hard. It is especially sad considering 1/
 could be solved very easily. It literally took me less than 1h to add
 support for it in SDC.


 
 Thanks for these thoughts!
 
 * (u)cent support * fixes for the shared qualifier * ownership mechanism
 
 These took less than 1h to add support for? That would be awesome... but
 realistically only the (u)cent sounds like that size of effort.
 


That's what he said: "... considering 1/ could be solved very easily. It 
literally took me less than 1h to add support for it in SDC."
Jan 14 2018
prev sibling next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 14 January 2018 at 23:03:27 UTC, Andrei Alexandrescu 
wrote:

 Thanks for these thoughts!

 * (u)cent support
 * fixes for the shared qualifier
 * ownership mechanism

 These took less than 1h to add support for? That would be 
 awesome... but realistically only the (u)cent sounds like that 
 size of effort.

 I've always wondered why we can't implement struct 
 LargeInt(uint bytes) as a library mechanism for larged 
 fixed-size integers, with asm specialization for LargeInt!8. Is 
 adding the type to the compiler necessary, and if so, why?


 Andrei


That is the proposed path for bootstrapping the compiler.

See
https://github.com/d-gamedev-team/gfm/blob/master/integers/gfm/integers/wideint.d
https://github.com/dlang/dmd/pull/7699
and recent effort to have struct as the main integral type in the 
compiler.
Jan 14 2018
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 14 January 2018 at 23:03:27 UTC, Andrei Alexandrescu 
wrote:

 Thanks for these thoughts!

 * (u)cent support
 * fixes for the shared qualifier
 * ownership mechanism

 These took less than 1h to add support for? That would be 
 awesome... but realistically only the (u)cent sounds like that 
 size of effort.


Agreed. That would be already a plus, as it would allow to do all 
the crypto in D.


 I've always wondered why we can't implement struct 
 LargeInt(uint bytes) as a library mechanism for larged 
 fixed-size integers, with asm specialization for LargeInt!8. Is 
 adding the type to the compiler necessary, and if so, why?


Asm specialization would not be ideal. Compilers have a pass 
called legalization where they break down the operation of types 
larger than the largest the plateform support into a series of 
operations on smaller types. It can generate specific pattern 
that the rest of the compiler is able to understand and optimize 
for.

This result in the use of instruction that would be very 
difficult for the compiler to reconstruct from the use of smaller 
integer types, such as mulhi.

Using asm is not ideal, unless the whole routine is written in 
asm, because the compiler find itself to optimize it, for 
instance after inlining. So even if it can inline - modern 
compiler can inline asm under specific circumstances, it finds 
itself unable to optimize operations at a higher level such as 
doing (a + b) + (c + d) instead of ((a + b) + c) + d.

Having larger types than 128 bits is not really necessary as you 
can leverage 128bits integers and do the magic yourself. For 
instance, to add two 256 bits integers represented as ulong[4], 
you can do:

ucent acc = 0;
ulong result[4];
for (i; 0 .. 4) {
     acc += a[i];
     acc += b[i];
     result[i] = cast(ulong) acc;
     acc >>= 64;
}

This will generate the ideal code on a modern optimizing 
compiler. Doing the same thing using only ulong will not generate 
good code as the compiler would have to understand from the 
techniques you used that you are indeed making addition porting 
the carry over. The problem gets even more hairy for 
multiplications.
Jan 17 2018
parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 18 January 2018 at 03:19:57 UTC, deadalnix wrote:

 On Sunday, 14 January 2018 at 23:03:27 UTC, Andrei Alexandrescu 
 wrote:

 Thanks for these thoughts!

 * (u)cent support
 * fixes for the shared qualifier
 * ownership mechanism

 These took less than 1h to add support for? That would be 
 awesome... but realistically only the (u)cent sounds like that 
 size of effort.


 Agreed. That would be already a plus, as it would allow to do 
 all the crypto in D.


Reading this again, I think there is a bit of a misunderstanding. 
Only cent/ucent took me ~1h to implement. The rest is more 
complex. That being said, having cent/ucent would unlock a great 
deal of performance for crypto libraries, and this is where the 
bottleneck is as far as CPU is concerned in this type of 
application.
Jan 17 2018
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/17/2018 7:22 PM, deadalnix wrote:

 Reading this again, I think there is a bit of a misunderstanding. Only 
 cent/ucent took me ~1h to implement. The rest is more complex. That being
said, 
 having cent/ucent would unlock a great deal of performance for crypto
libraries, 
 and this is where the bottleneck is as far as CPU is concerned in this type of 
 application.
 


I don't remember how long, but it took me a fair while to do the divide:

   https://github.com/dlang/druntime/blob/master/src/rt/llmath.d

It could be upscaled by rote to 128 bits, but even that would take me much 
longer than an hour. And it would still leave the issue of making ucent work 
with 32 bit code gen.

It could also be translated to D, but I doubt the generated code would be as
good.

Nevertheless, we do have the technology, we just need someone to put it
together.
Jan 18 2018
parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 18 January 2018 at 09:02:38 UTC, Walter Bright wrote:

 I don't remember how long, but it took me a fair while to do 
 the divide:

   https://github.com/dlang/druntime/blob/master/src/rt/llmath.d

 It could be upscaled by rote to 128 bits, but even that would 
 take me much longer than an hour. And it would still leave the 
 issue of making ucent work with 32 bit code gen.

 It could also be translated to D, but I doubt the generated 
 code would be as good.

 Nevertheless, we do have the technology, we just need someone 
 to put it together.


All the code to split 64 bits into 32 bits was generic and could 
be reused.
Jan 23 2018