digitalmars.D.announce - $750 Bounty: Issue 16416 - Phobos std.uni out of date (should be updated to latest Unicode standard)
- =?iso-8859-1?Q?Robert_M._M=FCnch?= (25/25) May 04 2020 Following my "Is it time for a Unicode update of std.uni?" post in D
- Arine (5/10) May 04 2020 I feel like this is going to be the biggest obstacle. I worked on
- =?iso-8859-1?Q?Robert_M._M=FCnch?= (8/12) May 04 2020 Thanks for the feedback. Was the PR eventually merged? Did you get any
- welkam (3/10) May 04 2020 If changes to phobos do not bring breaking changes then I dont
- =?iso-8859-1?Q?Robert_M._M=FCnch?= (9/11) May 05 2020 Well, but that's a weak statement for an invest.
- notna (7/18) May 04 2020 Never used std.uni as far as I know ;) _BUT_ I think this is a
- notna (3/24) May 04 2020 sorry, think this is what you want with 2) :O So just great, BIG
- rikki cattermole (4/8) May 04 2020 It wouldn't help.
- =?iso-8859-1?Q?Robert_M._M=FCnch?= (11/19) May 05 2020 Sure, nevertheless, I think it makes sense to have the reproducibility
- rikki cattermole (4/19) May 05 2020 Strange, I thought they were in the repo.
- Dmitry Olshansky (11/33) May 05 2020 Anyhow if anyone wants easy money - shoot me an email, or reply
- ag0aep6g (3/4) May 05 2020 Why wouldn't you be eligible? If it's an easy fix for you, that's just
- =?iso-8859-1?Q?Robert_M._M=FCnch?= (20/48) May 05 2020 Well, as I wrote, since I don't have a real good understanding about
- Dmitry Olshansky (34/80) May 05 2020 Felt a bit like cheating. Russian traditions preclude taking
- Dmitry Olshansky (7/14) May 05 2020 So here goes, indeed about 4.5 hours so far.
- =?iso-8859-1?Q?Robert_M._M=FCnch?= (17/42) May 06 2020 Great and deal.
- Bastiaan Veelo (3/5) May 12 2020 That's great! Glad to hear that.
- Dmitry Olshansky (3/8) May 12 2020 Bastian! Great to see you still around.
- Bastiaan Veelo (9/17) May 12 2020 First real application is running: a program for the numerical
- Dmitry Olshansky (2/14) May 12 2020
- Petar Kirov [ZombineDev] (51/52) May 05 2020 I believe this is an excellent initiative, thank you for starting
Following my "Is it time for a Unicode update of std.uni?" post in D group, I would like to try out to sponsor this effort for "Issue 16416 - Phobos std.uni out of date (should be updated to latest Unicode standard)" [1] For me, this, too, is an experiment to find out if it's possible to move specific issues/topics forward. And maybe even find people that are open to contact work too. For me, all these things are pretty related. So, not knowing how much work it is, nor knowing what a good amount would be, I took the other route and asked me, what is it worth for me likely, I could even live with the current state of std.uni. On the other hand, std.uni is a very fundamental building block, and having it up to date and maybe even extended should be much value to D. So, I'm offering $750 to get it done. Besides getting the work done, there is one constraint: The work needs to get into Phobos. It doesn't make sense to have it sit around, because it's not being merged. I don't have any clue who is in charge, who decides this. Or if there need to be some conditions full-filled so that the result gets merged. [1] https://issues.dlang.org/show_bug.cgi?id=16416 -- Robert M. Münch http://www.saphirion.com smarter | better | faster
May 04 2020
On Monday, 4 May 2020 at 17:01:01 UTC, Robert M. Münch wrote:Besides getting the work done, there is one constraint: The work needs to get into Phobos. It doesn't make sense to have it sit around, because it's not being merged. I don't have any clue who is in charge, who decides this. Or if there need to be some conditions full-filled so that the result gets merged.I feel like this is going to be the biggest obstacle. I worked on a bug bounty in the past, made a pull request, and it just sat there for months. It's a waste of time to try and get anything merged. Especially on the scale that this would be at.
May 04 2020
On 2020-05-04 17:30:41 +0000, Arine said:I feel like this is going to be the biggest obstacle. I worked on a bug bounty in the past, made a pull request, and it just sat there for months. It's a waste of time to try and get anything merged. Especially on the scale that this would be at.Thanks for the feedback. Was the PR eventually merged? Did you get any feedback why it wasn't merged, what needs to be done so that it gets merged, who decides this, etc.? -- Robert M. Münch http://www.saphirion.com smarter | better | faster
May 04 2020
On Monday, 4 May 2020 at 17:30:41 UTC, Arine wrote:On Monday, 4 May 2020 at 17:01:01 UTC, Robert M. Münch wrote:If changes to phobos do not bring breaking changes then I dont see how update to std.uni might not be mergedBesides getting the work done, there is one constraint: The work needs to get into Phobos. It doesn't make sense to have it sit around, because it's not being merged. I don't have any clue who is in charge, who decides this. Or if there need to be some conditions full-filled so that the result gets merged.I feel like this is going to be the biggest obstacle.
May 04 2020
On 2020-05-04 21:14:49 +0000, welkam said:If changes to phobos do not bring breaking changes then I dont see how update to std.uni might not be mergedWell, but that's a weak statement for an invest. If unicode is developing in a way that results in breaking changes, what to do? Never update? Doesn't make sense... So, breaking-changes because unicode requires these, have to be taken IMO. -- Robert M. Münch http://www.saphirion.com smarter | better | faster
May 05 2020
On Monday, 4 May 2020 at 17:01:01 UTC, Robert M. Münch wrote:Following my "Is it time for a Unicode update of std.uni?" post in D group, I would like to try out to sponsor this effort for "Issue 16416 - Phobos std.uni out of date (should be updated to latest Unicode standard)" [1]So, I'm offering $750 to get it done. Besides getting the work done, there is one constraint: The work needs to get into Phobos. It doesn't make sense to have it sit around, because it's not being merged. I don't have any clue who is in charge, who decides this. Or if there need to be some conditions full-filled so that the result gets merged. [1] https://issues.dlang.org/show_bug.cgi?id=16416Never used std.uni as far as I know ;) _BUT_ I think this is a great initiative, thanks! Maybe you want to add an additional constraint... It would be great if this would result in a tool, scripts or at least a simple-to-follow to-do (say Wiki?!)... so best case we could use this also for the next updates / releases in the future?!
May 04 2020
On Monday, 4 May 2020 at 19:26:28 UTC, notna wrote:On Monday, 4 May 2020 at 17:01:01 UTC, Robert M. Münch wrote:sorry, think this is what you want with 2) :O So just great, BIG thanks!Following my "Is it time for a Unicode update of std.uni?" post in D group, I would like to try out to sponsor this effort for "Issue 16416 - Phobos std.uni out of date (should be updated to latest Unicode standard)" [1]So, I'm offering $750 to get it done. Besides getting the work done, there is one constraint: The work needs to get into Phobos. It doesn't make sense to have it sit around, because it's not being merged. I don't have any clue who is in charge, who decides this. Or if there need to be some conditions full-filled so that the result gets merged. [1] https://issues.dlang.org/show_bug.cgi?id=16416Never used std.uni as far as I know ;) _BUT_ I think this is a great initiative, thanks! Maybe you want to add an additional constraint... It would be great if this would result in a tool, scripts or at least a simple-to-follow to-do (say Wiki?!)... so best case we could use this also for the next updates / releases in the future?!
May 04 2020
On 05/05/2020 7:26 AM, notna wrote:Maybe you want to add an additional constraint... It would be great if this would result in a tool, scripts or at least a simple-to-follow to-do (say Wiki?!)... so best case we could use this also for the next updates / releases in the future?!It wouldn't help. The reason we can't just grab a newer copy of the unicode database and throw it into Phobos is because the format was changed.
May 04 2020
On 2020-05-04 21:34:27 +0000, rikki cattermole said:On 05/05/2020 7:26 AM, notna wrote:Sure, nevertheless, I think it makes sense to have the reproducibility of the process in mind. Maybe not with a script that lasts for 10 years. But the process, some tools for a specific version, which can be used as inspiration for upcoming changes. IMO it makes a lot of sense for D to keep up close with the unicode development. -- Robert M. Münch http://www.saphirion.com smarter | better | fasterMaybe you want to add an additional constraint... It would be great if this would result in a tool, scripts or at least a simple-to-follow to-do (say Wiki?!)... so best case we could use this also for the next updates / releases in the future?!The reason we can't just grab a newer copy of the unicode database and throw it into Phobos is because the format was changed.
May 05 2020
On 05/05/2020 9:05 PM, Robert M. Münch wrote:On 2020-05-04 21:34:27 +0000, rikki cattermole said:Strange, I thought they were in the repo. Okay after looking through his fork of Phobos to see if its lying around somewhere, it looks like we need to hear from Dmitry Olshansky.On 05/05/2020 7:26 AM, notna wrote:Sure, nevertheless, I think it makes sense to have the reproducibility of the process in mind. Maybe not with a script that lasts for 10 years. But the process, some tools for a specific version, which can be used as inspiration for upcoming changes.Maybe you want to add an additional constraint... It would be great if this would result in a tool, scripts or at least a simple-to-follow to-do (say Wiki?!)... so best case we could use this also for the next updates / releases in the future?!The reason we can't just grab a newer copy of the unicode database and throw it into Phobos is because the format was changed.
May 05 2020
On Monday, 4 May 2020 at 17:01:01 UTC, Robert M. Münch wrote:Following my "Is it time for a Unicode update of std.uni?" post in D group, I would like to try out to sponsor this effort for "Issue 16416 - Phobos std.uni out of date (should be updated to latest Unicode standard)" [1] For me, this, too, is an experiment to find out if it's possible to move specific issues/topics forward. And maybe even find people that are open to contact work too. For me, all these things are pretty related. So, not knowing how much work it is, nor knowing what a good amount would be, I took the other route and asked me, what is not critical for me; most likely, I could even live with the current state of std.uni. On the other hand, std.uni is a very fundamental building block, and having it up to date and maybe even extended should be much value to D. So, I'm offering $750 to get it done.I'm guess I'm not eligible for the bounty ;)Besides getting the work done, there is one constraint: The work needs to get into Phobos. It doesn't make sense to have it sit around, because it's not being merged. I don't have any clue who is in charge, who decides this. Or if there need to be some conditions full-filled so that the result gets merged.Anyhow if anyone wants easy money - shoot me an email, or reply in this thread. Spoiler is - the whole thing is code generated and there is only one table that I forgot about (i.e. I have no idea what is the source table for it in Unicode standard).[1] https://issues.dlang.org/show_bug.cgi?id=16416P.S. I'm kind of back, but very busy and my health is mostly great despite the COVID outrage out there. --- Dmitry Olshansky
May 05 2020
On 05.05.20 17:39, Dmitry Olshansky wrote:I'm guess I'm not eligible for the bounty ;)Why wouldn't you be eligible? If it's an easy fix for you, that's just because you've got the needed expert knowledge. And that's valuable.
May 05 2020
On 2020-05-05 15:39:12 +0000, Dmitry Olshansky said:On Monday, 4 May 2020 at 17:01:01 UTC, Robert M. Münch wrote:Why not?Following my "Is it time for a Unicode update of std.uni?" post in D group, I would like to try out to sponsor this effort for "Issue 16416 - Phobos std.uni out of date (should be updated to latest Unicode standard)" [1] For me, this, too, is an experiment to find out if it's possible to move specific issues/topics forward. And maybe even find people that are open to contact work too. For me, all these things are pretty related. So, not knowing how much work it is, nor knowing what a good amount would be, I took the other route and asked me, what is it worth for me likely, I could even live with the current state of std.uni. On the other hand, std.uni is a very fundamental building block, and having it up to date and maybe even extended should be much value to D. So, I'm offering $750 to get it done.I'm guess I'm not eligible for the bounty ;)Anyhow if anyone wants easy money - shoot me an email, or reply in this thread.Well, as I wrote, since I don't have a real good understanding about the necessary effort I started from "what is it worth for me in $ to get it done?". So, if it's a simple script-change and a re-run and you are the only one knowing this and keeping it for yourself... yes, it's easy money. On the other hand, if you can help someone to get started and it's a couple of hours, I would expect people to be fair enough and state: Hey, $400 (or whatever) is OK, let's take the rest to sponsor something else. That's what I would do.Spoiler is - the whole thing is code generated and there is only one table that I forgot about (i.e. I have no idea what is the source table for it in Unicode standard).With "forgot" you mean, you can't remember, or it's missing at all in your prior work?P.S. I'm kind of back, but very busy and my health is mostly great despite the COVID outrage out there.That's great to hear... and maybe std.uni support/coaching for someone stepping up is possible. That would be great too. If, maybe even I can try to do it... -- Robert M. Münch http://www.saphirion.com smarter | better | faster
May 05 2020
On Tuesday, 5 May 2020 at 20:11:44 UTC, Robert M. Münch wrote:On 2020-05-05 15:39:12 +0000, Dmitry Olshansky said:Felt a bit like cheating. Russian traditions preclude taking money for things you (think you) wanted to do anyway.On Monday, 4 May 2020 at 17:01:01 UTC, Robert M. Münch wrote:Why not?Following my "Is it time for a Unicode update of std.uni?" post in D group, I would like to try out to sponsor this effort for "Issue 16416 - Phobos std.uni out of date (should be updated to latest Unicode standard)" [1] For me, this, too, is an experiment to find out if it's possible to move specific issues/topics forward. And maybe even find people that are open to contact work too. For me, all these things are pretty related. So, not knowing how much work it is, nor knowing what a good amount would be, I took the other route and asked me, what is not critical for me; most likely, I could even live with the current state of std.uni. On the other hand, std.uni is a very fundamental building block, and having it up to date and maybe even extended should be much value to D. So, I'm offering $750 to get it done.I'm guess I'm not eligible for the bounty ;)I started on it, and it turned out a bit more then I hope for + I'm doing it on simple Windows workstation without much of my usual power tools. LDC for Windows works like a charm though. It seems Unicode 13.0.0 pulled a plug on a couple of "derived" tables, that is data files that can be reconsturcted from other primary ones. Took at least half an hour to figure that out and rebuild the missing bits. If you don't mind I'll go with 100$ per hour estimate which is basically my usual contract rate. It took me about 2 hours for now, and I think I'd be done in a one or two more. Merging this into Phobos though is the otehr 90% of the legwork, I hope somebody will help me with that and maybe we'll just split your generous bounty this way.Anyhow if anyone wants easy money - shoot me an email, or reply in this thread.Well, as I wrote, since I don't have a real good understanding about the necessary effort I started from "what is it worth for me in $ to get it done?". So, if it's a simple script-change and a re-run and you are the only one knowing this and keeping it for yourself... yes, it's easy money. On the other hand, if you can help someone to get started and it's a couple of hours, I would expect people to be fair enough and state: Hey, $400 (or whatever) is OK, let's take the rest to sponsor something else. That's what I would do.I mean I know what this table does by its usage but the codegen part is missing, likely a classic missing commit problem of being a single maintainer of the codegen tool (and the fact that it's not in the main dlang repos).Spoiler is - the whole thing is code generated and there is only one table that I forgot about (i.e. I have no idea what is the source table for it in Unicode standard).With "forgot" you mean, you can't remember, or it's missing at all in your prior work?Absolutely. I mean I'm in no shape to do the heavy lifting of day in day out maintanance of std.* stuff but I'd love to coach somebody to learn how std.regex and std.uni work. I can also share my vision for improvement, and will gladly help with refactoring. Both of modules predate many of the good things in DLang and std.allocator in particular. Boy, I'd love to have allocators back in the day. -- Dmitry OlshanskyP.S. I'm kind of back, but very busy and my health is mostly great despite the COVID outrage out there.That's great to hear... and maybe std.uni support/coaching for someone stepping up is possible. That would be great too. If, maybe even I can try to do it...
May 05 2020
On Tuesday, 5 May 2020 at 21:41:39 UTC, Dmitry Olshansky wrote:On Tuesday, 5 May 2020 at 20:11:44 UTC, Robert M. Münch wrote:So here goes, indeed about 4.5 hours so far. The generator is untangled from the old crap from that GSOC 2012 repo: https://github.com/DmitryOlshansky/gen-uni-dlang PR: https://github.com/dlang/phobos/pull/7469 Let's see if the CI loves it or not.On 2020-05-05 15:39:12 +0000, Dmitry Olshansky said: On the other hand, if you can help someone to get started and it's a couple of hours, I would expect people to be fair enough and state: Hey, $400 (or whatever) is OK, let's take the rest to sponsor something else. That's what I would do.
May 05 2020
On 2020-05-05 21:41:39 +0000, Dmitry Olshansky said:Felt a bit like cheating. Russian traditions preclude taking money for things you (think you) wanted to do anyway.Well, that's a good habit and still IMO it's OK to offer and take an incentive.I started on it, and it turned out a bit more then I hope for + I'm doing it on simple Windows workstation without much of my usual power tools. LDC for Windows works like a charm though. It seems Unicode 13.0.0 pulled a plug on a couple of "derived" tables, that is data files that can be reconsturcted from other primary ones. Took at least half an hour to figure that out and rebuild the missing bits. If you don't mind I'll go with 100$ per hour estimate which is basically my usual contract rate. It took me about 2 hours for now, and I think I'd be done in a one or two more.Great and deal.Merging this into Phobos though is the otehr 90% of the legwork, I hope somebody will help me with that and maybe we'll just split your generous bounty this way.Sure. As said, I'm not totally sure how this code-merging process works, who can do it, who approves things (if at all) or if it's enough of the automated tests don't fail.I mean I know what this table does by its usage but the codegen part is missing, likely a classic missing commit problem of being a single maintainer of the codegen tool (and the fact that it's not in the main dlang repos).Got it.Absolutely. I mean I'm in no shape to do the heavy lifting of day in day out maintanance of std.* stuff but I'd love to coach somebody to learn how std.regex and std.uni work. I can also share my vision for improvement, and will gladly help with refactoring.With focus on std.uni I think what would help is a short description of the whole process. A context setting chapter "unicode pitfalls, important things to know, general process" and a "step-by-step" description/log of what needs to be done, what the step does and where it fits into the overall picture. Just found out that std.regex is from you too... nice. -- Robert M. Münch http://www.saphirion.com smarter | better | faster
May 06 2020
On Tuesday, 5 May 2020 at 15:39:12 UTC, Dmitry Olshansky wrote:P.S. I'm kind of back, but very busy and my health is mostly great despite the COVID outrage out there.That's great! Glad to hear that. -- Bastiaan.
May 12 2020
On Tuesday, 12 May 2020 at 07:21:43 UTC, Bastiaan Veelo wrote:On Tuesday, 5 May 2020 at 15:39:12 UTC, Dmitry Olshansky wrote:Bastian! Great to see you still around. How your D stuff is going at that naval company?P.S. I'm kind of back, but very busy and my health is mostly great despite the COVID outrage out there.That's great! Glad to hear that.-- Bastiaan.
May 12 2020
On Tuesday, 12 May 2020 at 07:48:46 UTC, Dmitry Olshansky wrote:On Tuesday, 12 May 2020 at 07:21:43 UTC, Bastiaan Veelo wrote:First real application is running: a program for the numerical analysis of a ship launch at the yard. Currently testing and debugging. Pain points typically revolve around low level tricks in Pascal using arrays starting at 1 (these usually translate without problems, except where they don't)... Or passing strings to/from win32. Still committed to translate all other programs in our suite to D, busy times as usual. -- Bastiaan.On Tuesday, 5 May 2020 at 15:39:12 UTC, Dmitry Olshansky wrote:Bastian! Great to see you still around. How your D stuff is going at that naval company?P.S. I'm kind of back, but very busy and my health is mostly great despite the COVID outrage out there.That's great! Glad to hear that.
May 12 2020
On Tuesday, 12 May 2020 at 08:11:03 UTC, Bastiaan Veelo wrote:On Tuesday, 12 May 2020 at 07:48:46 UTC, Dmitry Olshansky wrote:Cool stuff. Keep it rolling ;)Bastian! Great to see you still around. How your D stuff is going at that naval company?First real application is running: a program for the numerical analysis of a ship launch at the yard. Currently testing and debugging. Pain points typically revolve around low level tricks in Pascal using arrays starting at 1 (these usually translate without problems, except where they don't)... Or passing strings to/from win32. Still committed to translate all other programs in our suite to D, busy times as usual.-- Bastiaan.
May 12 2020
On Monday, 4 May 2020 at 17:01:01 UTC, Robert M. Münch wrote:...I believe this is an excellent initiative, thank you for starting it! Perhaps this script, along with repository that is part of can help those wishing to update std.uni to the latest version: https://github.com/DmitryOlshansky/gsoc-bench-2012/blob/master/gen_uni.d With regard to the rate of pull requests being merged into the core repositories, I would say that it is highly contextually dependent. I strongly advise either: a) subscribing for notifications from the core dlang repositories (dmd, druntime, phobos, dub, etc.) for an extended period of time (3 months min) - you'll be able to observe the group dynamics (e.g. which contributors have experience with which part of the codebase, why some things are merged quickly and others take a while, etc.) - this way you can really draw conclusions for yourself (b) looking at the statistics: - https://github.com/dlang/dmd/pulse/monthly - https://github.com/dlang/druntime/pulse/monthly - https://github.com/dlang/phobos/pulse/monthly - https://github.com/dlang/dub/pulse/monthly as opposed to drawing conclusions from single data points of anecdotal evidence. From my several years of experience, I can say the following: - small, less complex pull requests are generally easy to get merged - it depends on the part of the codebase - if you open a pull request for a part whose maintainers are currently active, you can expect a speedy review. If it's a part (e.g. std.regex) that is both highly complex and with a small number of maintainers, then it may take a while) - teamwork and communication - since all of us are living in different time zones, rather than working in the same office, you should be prepared that communication (which is a prerequisite of merging) will be with high-latency. Changes that are described well, for which the benefit is clear and doesn't look like they may introduce regressions are of course received well. Discussion prior to opening a merge request can help to guide the implementation in the right direction and save time later in the review process. Many contributors are active on the dlang Slack [1] which makes it a good place to ping people for feedback, or just to have a near real-time conversation. In the past 1-3 years, I have noticed a trend that many active contributors are mostly active on GitHub and Slack, rather than the newsgroup. If you see that pull request has fallen through the cracks (no new replies from maintainers), don't hesitate to ping us either there or here on the newsgroup. [1]: https://dlang.slack.com/
May 05 2020