digitalmars.D - add uda (e.g. csvIgnore) support in std.csv?
- mw (24/24) Sep 07 2020 Hi,
- Jacob Carlborg (7/38) Sep 10 2020 How about a more general attribute in `core.attribute`, like
- Paul Backus (11/49) Sep 10 2020 Existing serialization libraries [1][2] already define their own
- Steven Schveighoffer (8/23) Sep 10 2020 To add to this, I have code that serializes the same types for both
- Sebastiaan Koppe (11/19) Oct 09 2020 I have sometimes used a dedicated struct just to let the
- 9il (4/11) Oct 10 2020 Ready to use module for this purpose.
- mw (6/30) Oct 09 2020 I do run into this problem today, i.e. my struct need some extra
- mw (9/49) Sep 24 2022 Run into this problem yet again:
Hi, I'm trying this example: https://dlang.org/phobos/std_csv.html ``` struct Layout { string name; int value; double other; csvIgnore Layout* parent; // add extra pointer } auto records = text.csvReader!Layout(';'); ``` Right now, if we add this extra pointer, the compiler will error out: https://run.dlang.io/is/EvU6Ee /dlang/dmd/linux/bin64/../../src/phobos/std/conv.d(223): Error: template std.conv.toImpl cannot deduce function from argument types !(Layout*)(string), candidates are: ... I think if we can add add uda (e.g. csvIgnore) support in std.csv, it will make the library more flexible to use. Thoughts?
Sep 07 2020
On 2020-09-08 06:53, mw wrote:Hi, I'm trying this example: https://dlang.org/phobos/std_csv.html ``` struct Layout { string name; int value; double other; csvIgnore Layout* parent; // add extra pointer } auto records = text.csvReader!Layout(';'); ``` Right now, if we add this extra pointer, the compiler will error out: https://run.dlang.io/is/EvU6Ee /dlang/dmd/linux/bin64/../../src/phobos/std/conv.d(223): Error: template std.conv.toImpl cannot deduce function from argument types !(Layout*)(string), candidates are: ... I think if we can add add uda (e.g. csvIgnore) support in std.csv, it will make the library more flexible to use.How about a more general attribute in `core.attribute`, like ` nonSerialized`? This could be used by third party serialization libraries. Then it could also be used by druntime and Phobos to ignore things that should not be serializable. Like threads, processes and sockets. -- /Jacob Carlborg
Sep 10 2020
On Thursday, 10 September 2020 at 14:57:11 UTC, Jacob Carlborg wrote:On 2020-09-08 06:53, mw wrote:Existing serialization libraries [1][2] already define their own attributes for this sort of thing. I don't think there's anything to be gained by attempting to force standardization here. If Phobos and/or the D ecosystem does someday decide to standardize on a shared ignore attribute, std.csv can be easily upgraded for compatibility by adding `alias csvIgnore = core.attribute.ignore`. [1] https://code.dlang.org/packages/asdf [2] https://vibed.org/api/vibe.data.serialization/Hi, I'm trying this example: https://dlang.org/phobos/std_csv.html ``` struct Layout { string name; int value; double other; csvIgnore Layout* parent; // add extra pointer } auto records = text.csvReader!Layout(';'); ``` Right now, if we add this extra pointer, the compiler will error out: https://run.dlang.io/is/EvU6Ee /dlang/dmd/linux/bin64/../../src/phobos/std/conv.d(223): Error: template std.conv.toImpl cannot deduce function from argument types !(Layout*)(string), candidates are: ... I think if we can add add uda (e.g. csvIgnore) support in std.csv, it will make the library more flexible to use.How about a more general attribute in `core.attribute`, like ` nonSerialized`? This could be used by third party serialization libraries. Then it could also be used by druntime and Phobos to ignore things that should not be serializable. Like threads, processes and sockets.
Sep 10 2020
On 9/10/20 1:57 PM, Paul Backus wrote:On Thursday, 10 September 2020 at 14:57:11 UTC, Jacob Carlborg wrote:To add to this, I have code that serializes the same types for both vibe.d JSON for REST interfaces, AND to a database. In one case, I want certain fields to be ignored (e.g. no reason to send internal database ids to the browser), and in another case I want other fields to be ignored. I'd rather have libraries define their own UDAs, and not expect Phobos to design a UDA system that pleases all parties. -SteveHow about a more general attribute in `core.attribute`, like ` nonSerialized`? This could be used by third party serialization libraries. Then it could also be used by druntime and Phobos to ignore things that should not be serializable. Like threads, processes and sockets.Existing serialization libraries already define their own attributes for this sort of thing. I don't think there's anything to be gained by attempting to force standardization here. If Phobos and/or the D ecosystem does someday decide to standardize on a shared ignore attribute, std.csv can be easily upgraded for compatibility by adding `alias csvIgnore = core.attribute.ignore`.
Sep 10 2020
Sebastiaan Koppe <mail skoppe.eu> On Thursday, 10 September 2020 at 19:11:03 UTC, Steven Schveighoffer wrote:To add to this, I have code that serializes the same types for both vibe.d JSON for REST interfaces, AND to a database. In one case, I want certain fields to be ignored (e.g. no reason to send internal database ids to the browser), and in another case I want other fields to be ignored. I'd rather have libraries define their own UDAs, and not expect Phobos to design a UDA system that pleases all parties. -SteveI have sometimes used a dedicated struct just to let the serialiser know how to deal with the fields. It would mostly be a copy of the original struct but with some udas here and there. It allows me to have different shapes even when targeting the same format. Wrt to csv, you could serialise into an intermediate struct first, and from that into your target struct. It wouldn't be hard to convert struct a { string timestamp, name; double value; } into struct ab { Datetime timestamp; string name; double value; }
Oct 09 2020
On Thursday, 10 September 2020 at 14:57:11 UTC, Jacob Carlborg wrote:On 2020-09-08 06:53, mw wrote:Ready to use module for this purpose. http://mir-algorithm.libmir.org/mir_serde.html[...]How about a more general attribute in `core.attribute`, like ` nonSerialized`? This could be used by third party serialization libraries. Then it could also be used by druntime and Phobos to ignore things that should not be serializable. Like threads, processes and sockets.
Oct 10 2020
On Saturday, 10 October 2020 at 10:13:55 UTC, 9il wrote:Ready to use module for this purpose. http://mir-algorithm.libmir.org/mir_serde.htmlNice, but this is in a 3rd-party library, and std.csv is the DLang standard library. Maybe can you create a PR just contribute this module to std? (I've read there are some issues regarding licensing to contribute the whole mir to stb lib).
Oct 10 2020
On Saturday, 10 October 2020 at 16:52:32 UTC, mw wrote:On Saturday, 10 October 2020 at 10:13:55 UTC, 9il wrote:We are making Mir the real standard D library.Ready to use module for this purpose. http://mir-algorithm.libmir.org/mir_serde.htmlNice, but this is in a 3rd-party library, and std.csv is the DLang standard library. Maybe can you create a PR just contribute this module to std? (I've read there are some issues regarding licensing to contribute the whole mir to stb lib).
Oct 11 2020
On Sunday, 11 October 2020 at 14:04:47 UTC, 9il wrote:On Saturday, 10 October 2020 at 16:52:32 UTC, mw wrote:This is excellent news. BTW, what namespace it will use? std.???? From a purely marketing / promotional perspective, may I suggest numD? People will immediately recognize numPy => numD, and if we can even make the interface the same, people may adopt it more quickly. I know there is numir project, but there are several repos under https://github.com/libmir, as a user I sometimes got confused which one to use, can they *all* be merged into a single library numD?On Saturday, 10 October 2020 at 10:13:55 UTC, 9il wrote:We are making Mir the real standard D library.Ready to use module for this purpose. http://mir-algorithm.libmir.org/mir_serde.htmlNice, but this is in a 3rd-party library, and std.csv is the DLang standard library. Maybe can you create a PR just contribute this module to std? (I've read there are some issues regarding licensing to contribute the whole mir to stb lib).
Oct 11 2020
On Sunday, 11 October 2020 at 16:15:30 UTC, mw wrote:On Sunday, 11 October 2020 at 14:04:47 UTC, 9il wrote:mir.* std.* is used by Phobos and it is fine. It isn't the same as it was in D1 with two standard libraries. Mir is compatible with Phobos.On Saturday, 10 October 2020 at 16:52:32 UTC, mw wrote:This is excellent news. BTW, what namespace it will use? std.????On Saturday, 10 October 2020 at 10:13:55 UTC, 9il wrote:We are making Mir the real standard D library.Ready to use module for this purpose. http://mir-algorithm.libmir.org/mir_serde.htmlNice, but this is in a 3rd-party library, and std.csv is the DLang standard library. Maybe can you create a PR just contribute this module to std? (I've read there are some issues regarding licensing to contribute the whole mir to stb lib).From a purely marketing / promotional perspective, may I suggest numD?Mir isn't limited by numeric code. For example, mir.rc (mir-algorithm) provides a production-ready full-featured ref-counted type system.People will immediately recognize numPy => numD, and if we can even make the interface the same, people may adopt it more quickly. I know there is numir project, but there are several repos under https://github.com/libmir, as a user I sometimes got confused which one to use, can they *all* be merged into a single library numD?Different code of different purpose needs different maintaining rules and coding standards.
Oct 11 2020
On Tuesday, 8 September 2020 at 04:53:26 UTC, mw wrote:Hi, I'm trying this example: https://dlang.org/phobos/std_csv.html ``` struct Layout { string name; int value; double other; csvIgnore Layout* parent; // add extra pointer } auto records = text.csvReader!Layout(';'); ``` Right now, if we add this extra pointer, the compiler will error out: https://run.dlang.io/is/EvU6Ee /dlang/dmd/linux/bin64/../../src/phobos/std/conv.d(223): Error: template std.conv.toImpl cannot deduce function from argument types !(Layout*)(string), candidates are: ... I think if we can add add uda (e.g. csvIgnore) support in std.csv, it will make the library more flexible to use. Thoughts?I do run into this problem today, i.e. my struct need some extra non-simple-scalar fields (e.g. SysTime) other than those defined in the csv file, I'm wondering what kind of work-around people use in such situation? Thanks.
Oct 09 2020
On Saturday, 10 October 2020 at 04:31:20 UTC, mw wrote:On Tuesday, 8 September 2020 at 04:53:26 UTC, mw wrote:Run into this problem yet again: ``` class Foo { Foo[] foos; // how to mark this field to be csv ignored? } ``` Any work around? Thanks.Hi, I'm trying this example: https://dlang.org/phobos/std_csv.html ``` struct Layout { string name; int value; double other; csvIgnore Layout* parent; // add extra pointer } auto records = text.csvReader!Layout(';'); ``` Right now, if we add this extra pointer, the compiler will error out: https://run.dlang.io/is/EvU6Ee /dlang/dmd/linux/bin64/../../src/phobos/std/conv.d(223): Error: template std.conv.toImpl cannot deduce function from argument types !(Layout*)(string), candidates are: ... I think if we can add add uda (e.g. csvIgnore) support in std.csv, it will make the library more flexible to use. Thoughts?I do run into this problem today, i.e. my struct need some extra non-simple-scalar fields (e.g. SysTime) other than those defined in the csv file, I'm wondering what kind of work-around people use in such situation? Thanks.
Sep 24 2022