www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - asdf json library moved to libmir

reply yannick <dlang srtnwz.com> writes:
Hi All,

Since i'll be leaving Tamedia (who sponsored development of Asdf) 
and it's not under active use anymore here we decided to donate 
it to libmir.

Asdf has a few nifty features that make it perfect for reading 
large amounts of json lines and transforming data. For an example 
use case see je, the json to csv transformer app ( 
https://github.com/tamediadigital/je )

I still think that it is by far the easiest to use and fastest 
production grade Json Library for D.
Maybe it should be renamed to reflect that, proposals welcome.

cheers,
y
Jul 11 2018
next sibling parent reply WebFreak001 <d.forum webfreak.org> writes:
On Wednesday, 11 July 2018 at 21:42:45 UTC, yannick wrote:

 Hi All,

 Since i'll be leaving Tamedia (who sponsored development of 
 Asdf) and it's not under active use anymore here we decided to 
 donate it to libmir.

 Asdf has a few nifty features that make it perfect for reading 
 large amounts of json lines and transforming data. For an 
 example use case see je, the json to csv transformer app ( 
 https://github.com/tamediadigital/je )

 I still think that it is by far the easiest to use and fastest 
 production grade Json Library for D.
 Maybe it should be renamed to reflect that, proposals welcome.

 cheers,
 y


cool to hear that you don't let that library die. I like asdf but 
I never really had a usecase for it. When using vibe.d I just 
parse my json using their serializer. Additionally I never have 
the case of my input being line separated json, for example when 
reading a dub registry dump it's a 40MB json array where I don't 
really see application for asdf, or can it also be used with 
generic arrays that aren't line separated?

Anyway pretty cool library, would definitely use it if I ever see 
line separated json data in the wild.
Jul 11 2018
parent yawniek <yawniek srtnwz.com> writes:
On Wednesday, 11 July 2018 at 21:58:29 UTC, WebFreak001 wrote:

 cool to hear that you don't let that library die. I like asdf 
 but I never really had a usecase for it. When using vibe.d I 
 just parse my json using their serializer. Additionally I never 
 have the case of my input being line separated json, for 
 example when reading a dub registry dump it's a 40MB json array 
 where I don't really see application for asdf, or can it also 
 be used with generic arrays that aren't line separated?


i never understood why people do this. why would you ever 
generate such a big json, it requires a lot of memory to generate 
it too.
that being said asdf also works in this case and i think you 
should even be able to stream non th e less (not 100% sure, if 
not it could/should be implemented).


 Anyway pretty cool library, would definitely use it if I ever 
 see line separated json data in the wild.


die Hoffnung stirbt zuletzt...

anyway, really happy to see that asdf found a few uses and i can 
really recommend to try it out, even to replace the standard json 
libs in e.g. vibe.d, which is what i do too.
Its not only faster but also easier, since you can use your 
structs/classes and have granular control over what properties 
are (de-)serialized.
Jul 13 2018
prev sibling next sibling parent reply Daniel Kozak <kozzi11 gmail.com> writes:
On Wed, Jul 11, 2018 at 11:45 PM yannick via Digitalmars-d-announce <
digitalmars-d-announce puremagic.com> wrote:


 Hi All,

 Since i'll be leaving Tamedia (who sponsored development of Asdf)
 and it's not under active use anymore here we decided to donate
 it to libmir.

 Asdf has a few nifty features that make it perfect for reading
 large amounts of json lines and transforming data. For an example
 use case see je, the json to csv transformer app (
 https://github.com/tamediadigital/je )

 I still think that it is by far the easiest to use and fastest
 production grade Json Library for D.
 Maybe it should be renamed to reflect that, proposals welcome.

 cheers,
 y


I have tried it few days ago and it does not work for me at  first. Because
I always read only "Simple Example".
One of my problem was that I have a struct with little different name that
was attribute name used in
json (positions and types has been correct), but it does not work and the
error was really wierd so it took
me a while until I have found out what was wrong.

Other problem was with types when one json attribute can be a number or null
Jul 11 2018
parent yawniek <yawniek srtnwz.com> writes:
On Wednesday, 11 July 2018 at 22:21:32 UTC, Daniel Kozak wrote:


 I have tried it few days ago and it does not work for me at  
 first. Because
 I always read only "Simple Example".
 One of my problem was that I have a struct with little 
 different name that
 was attribute name used in
 json (positions and types has been correct), but it does not 
 work and the
 error was really wierd so it took
 me a while until I have found out what was wrong.

 Other problem was with types when one json attribute can be a 
 number or null


Asdf was actually exactly created for this use case.
You can do transformations when reading and writing.
Also the null case is handled via different tools, please read 
the Readme and look into the API.

If there is an use case that you seem not to be able to pull of, 
please file a ticket.
Jul 13 2018
prev sibling next sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Wednesday, 11 July 2018 at 21:42:45 UTC, yannick wrote:

 Hi All,

 Since i'll be leaving Tamedia (who sponsored development of 
 Asdf) and it's not under active use anymore here we decided to 
 donate it to libmir.

 Asdf has a few nifty features that make it perfect for reading 
 large amounts of json lines and transforming data. For an 
 example use case see je, the json to csv transformer app ( 
 https://github.com/tamediadigital/je )

 I still think that it is by far the easiest to use and fastest 
 production grade Json Library for D.
 Maybe it should be renamed to reflect that, proposals welcome.

 cheers,
 y


Great! Thanks for keeping it alive.

I use the library specifically for reading json files stored on 
s3 that were generated by  aws' kinesis service. (The files have 
multiple json roots in them, but aren't separated by newlines; 
just one after another. I created a small custom parseJsonByLine 
for that).
Jul 12 2018
prev sibling parent Rene Zwanenburg <renezwanenburg gmail.com> writes:
On Wednesday, 11 July 2018 at 21:42:45 UTC, yannick wrote:

 Hi All,

 Since i'll be leaving Tamedia (who sponsored development of 
 Asdf) and it's not under active use anymore here we decided to 
 donate it to libmir.

 ...

 cheers,
 y


Thanks for keeping it alive. I've used asdf at work last week to 
build a json plugin for logparser:

https://en.wikipedia.org/wiki/Logparser

I have yet to ask permission to open source it, but I'll make an 
announcement if the boss agrees.
Jul 12 2018