digitalmars.D.learn - Persistent key-value-store for D?
- krylon (31/31) Apr 26 2017 Hello everyone,
- bachmeier (15/21) Apr 26 2017 Welcome to the D community.
- krylon (2/15) Apr 27 2017 Thank you! I will look into that!
- Matthias Klumpp (15/22) Apr 26 2017 I can recommend using LMDB[1] which likely does all you want.
- krylon (3/20) Apr 27 2017 Thank you very much, that sounds exactly like what I was looking
- yawniek (7/20) Apr 28 2017 i recommend leveldb
- krylon (6/27) Apr 28 2017 Thank you for the suggestion!
Hello everyone, After several unsuccessful attempts over the last couple of years, I made a new attempt at learning D last weekend and this time ... something clicked. So first of all, let me say that I like it very much so far! Over the years, I have fallen into the habit when learning a new programming language to rewrite a certain program I first started working on about 12 years ago. It's not so much that this program is super-useful (it really isn't), but that it is a good way to learn how a given programming language handles things like concurrency, data structures, networking, and user interfaces. One part of this program requires a DBM-like database library, preferably one that handles concurrent read-write-access as transparently as possible. In prior incarnations of this program I have used Berkeley DB and Tokyocabinet. I looked at the DUB package registry and asked Google quite a bit now, but I did not found such a package for D. So my first question is - did I not look hard enough? I found a reimplentation of QDBM [1] (the spiritual ancestor of Tokyocabinet), but it does not seem to handle concurrency at all. Are there other options along those lines? (If there was one that also provides transactions, that would be awesome!) If I understand what I have read so far correctly, it is possible to access libraries written in C or C++ from D - in that case, I could just use Tokyocabinet directly, but I have not found any pointers on how to do this. Is this a feasible option, and if so, where can I find documentation on how to do this? [1] http://forum.dlang.org/thread/mjinam$31cp$2 digitalmars.com Thank you very much for any insights you feel like sharing with me, Benjamin
Apr 26 2017
On Wednesday, 26 April 2017 at 17:06:52 UTC, krylon wrote:If I understand what I have read so far correctly, it is possible to access libraries written in C or C++ from D - in that case, I could just use Tokyocabinet directly, but I have not found any pointers on how to do this. Is this a feasible option, and if so, where can I find documentation on how to do this?Welcome to the D community. I've never heard of Tokyocabinet, but I did a search and the header files look like they'd be a straightforward translation to D, for instance http://bazaar.launchpad.net/~tokyocabinet/tokyocabinet/c99-posix-trunk/view/head:/tcutil.h You can call C libraries directly. I'd suggest trying dstep on the header files as a first step. https://github.com/jacob-carlborg/dstep These resources might help: https://wiki.dlang.org/Bind_D_to_C http://dlang.org/spec/interfaceToC.html You only need to write a binding for functions that you actually want to call, which might explain why nobody has made bindings available.
Apr 26 2017
krylon <anonymous example.com> On Wednesday, 26 April 2017 at 19:07:22 UTC, bachmeier wrote:On Wednesday, 26 April 2017 at 17:06:52 UTC, krylon wrote:Thank you! I will look into that![...]Welcome to the D community. [...] You can call C libraries directly. I'd suggest trying dstep on the header files as a first step. https://github.com/jacob-carlborg/dstep These resources might help: https://wiki.dlang.org/Bind_D_to_C http://dlang.org/spec/interfaceToC.html You only need to write a binding for functions that you actually want to call, which might explain why nobody has made bindings available.
Apr 27 2017
On Wednesday, 26 April 2017 at 17:06:52 UTC, krylon wrote:[...] If I understand what I have read so far correctly, it is possible to access libraries written in C or C++ from D - in that case, I could just use Tokyocabinet directly, but I have not found any pointers on how to do this. Is this a feasible option, and if so, where can I find documentation on how to do this?I can recommend using LMDB[1] which likely does all you want. It also has D bindings[2]. We use it in the AppStream generator with multithreaded parallelization and it is working great so far (although for the usecase of asgen it's not an ideal database choice, but one which makes deployments of the generator really simple). I explored other solutions including Tokyokabinet and LevelDB and LMDB was by far the fastest. Cheers, Matthias [1]: https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database [2]: https://github.com/ximion/appstream-generator/blob/master/src/asgen/bindings/lmdb.d
Apr 26 2017
On Thursday, 27 April 2017 at 01:56:12 UTC, Matthias Klumpp wrote:On Wednesday, 26 April 2017 at 17:06:52 UTC, krylon wrote:Thank you very much, that sounds exactly like what I was looking for![...] If I understand what I have read so far correctly, it is possible to access libraries written in C or C++ from D - in that case, I could just use Tokyocabinet directly, but I have not found any pointers on how to do this. Is this a feasible option, and if so, where can I find documentation on how to do this?I can recommend using LMDB[1] which likely does all you want. It also has D bindings[2]. [...] Cheers, Matthias [1]: https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database [2]: https://github.com/ximion/appstream-generator/blob/master/src/asgen/bindings/lmdb.d
Apr 27 2017
On Wednesday, 26 April 2017 at 17:06:52 UTC, krylon wrote:I looked at the DUB package registry and asked Google quite a bit now, but I did not found such a package for D. So my first question is - did I not look hard enough? I found a reimplentation of QDBM [1] (the spiritual ancestor of Tokyocabinet), but it does not seem to handle concurrency at all. Are there other options along those lines? (If there was one that also provides transactions, that would be awesome!) If I understand what I have read so far correctly, it is possible to access libraries written in C or C++ from D - in that case, I could just use Tokyocabinet directly, but I have not found any pointers on how to do this. Is this a feasible option, and if so, where can I find documentation on how to do this?i recommend leveldb http://code.dlang.org/packages/d-leveldb its easy to use and mostly faster than tokyocabinet ( only very specifically tuned tokyo btrees outperform leveldb) i used above library with great success. it also shows you how to do c bindings.
Apr 28 2017
On Friday, 28 April 2017 at 16:01:58 UTC, yawniek wrote:On Wednesday, 26 April 2017 at 17:06:52 UTC, krylon wrote:Thank you for the suggestion! For the moment I am using LMDB, as suggested in another reply, and it works well. LevelDB seems to have a cleaner interface, though, at least in D. I'll keep that in mind for future projects.I looked at the DUB package registry and asked Google quite a bit now, but I did not found such a package for D. So my first question is - did I not look hard enough? I found a reimplentation of QDBM [1] (the spiritual ancestor of Tokyocabinet), but it does not seem to handle concurrency at all. Are there other options along those lines? (If there was one that also provides transactions, that would be awesome!) If I understand what I have read so far correctly, it is possible to access libraries written in C or C++ from D - in that case, I could just use Tokyocabinet directly, but I have not found any pointers on how to do this. Is this a feasible option, and if so, where can I find documentation on how to do this?i recommend leveldb http://code.dlang.org/packages/d-leveldb its easy to use and mostly faster than tokyocabinet ( only very specifically tuned tokyo btrees outperform leveldb) i used above library with great success. it also shows you how to do c bindings.
Apr 28 2017