www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Voting/Scoring and final decision discussion

reply "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
There is little documentation on how to handle the situation the 
Review Manager is currently facing. I would like to open this 
discussion to point out why, and to poll for if we should have 
any.

The reason is that our process comes from the the Boost review, 
and there is no such definition[1].

The way we have it makes it appear that the module is accepted as 
a majority vote. However that isn't the intention of the Boost 
process. The Review Manager wields a lot of power during the vote 
tally. The key line in our documentation:

"Tallies votes and decides if there is consensus to accept the 
library and under what conditions."

The Review Manager is trying to judge based on the input how well 
this library fits with the goals of Phobos and the D community. 
That doesn't mean a landslide victory is needed.

So with that I, and probably Dicebot, would like to hear feedback.

Dicebot, consider what information may help make your decision. 
Would yes votes including positive feedback help (it is easier to 
side with those providing an argument)?

1. http://www.boost.org/community/reviews.html#Introduction
"The final "accept" or "reject" decision is made by the Review 
Manager, based on the review comments received from boost mailing 
list members."

Boost doesn't do a review then vote separation.
Oct 09 2013
next sibling parent "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
On Thursday, 10 October 2013 at 03:14:30 UTC, Jesse Phillips 
wrote:

 There is little documentation on how to handle the situation 
 the Review Manager is currently facing. I would like to open 
 this discussion to point out why, and to poll for if we should 
 have any.

 The reason is that our process comes from the the Boost review, 
 and there is no such definition[1].

 The way we have it makes it appear that the module is accepted 
 as a majority vote. However that isn't the intention of the 
 Boost process. The Review Manager wields a lot of power during 
 the vote tally. The key line in our documentation:

 "Tallies votes and decides if there is consensus to accept the 
 library and under what conditions."

 The Review Manager is trying to judge based on the input how 
 well this library fits with the goals of Phobos and the D 
 community. That doesn't mean a landslide victory is needed.

 So with that I, and probably Dicebot, would like to hear 
 feedback.

 Dicebot, consider what information may help make your decision. 
 Would yes votes including positive feedback help (it is easier 
 to side with those providing an argument)?

 1. http://www.boost.org/community/reviews.html#Introduction
 "The final "accept" or "reject" decision is made by the Review 
 Manager, based on the review comments received from boost 
 mailing list members."

 Boost doesn't do a review then vote separation.


I think that we should use consensus model, not a majority vote. 
It's useful for small groups. For example, Wikipedia use it:
This page in a nutshell: Consensus is Wikipedia's fundamental 
model for editorial decision-making.
http://en.wikipedia.org/wiki/Wikipedia:Consensus
Oct 10 2013
prev sibling next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Thursday, 10 October 2013 at 03:14:30 UTC, Jesse Phillips 
wrote:

 Dicebot, consider what information may help make your decision. 
 Would yes votes including positive feedback help (it is easier 
 to side with those providing an argument)?


Yes, that will help. If anyone who has voted "Yes" will comment 
in this topic with more detailed explanation, it will be taken 
into consideration.

Right now I am more leaning towards declining the module 
adoption, mostly because "No" vote amount is rather high among 
Phobos developers, which is a really bad sign.

It is pretty clear though that this voting has shown certain weak 
spot in out adoption process for more controversial proposals. 
Once person (review manager) simply should not have that much 
decision power in such situation.
Oct 10 2013
next sibling parent "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Thursday, 10 October 2013 at 11:57:37 UTC, Dicebot wrote:

 On Thursday, 10 October 2013 at 03:14:30 UTC, Jesse Phillips 
 wrote:

 Dicebot, consider what information may help make your 
 decision. Would yes votes including positive feedback help (it 
 is easier to side with those providing an argument)?


 Yes, that will help. If anyone who has voted "Yes" will comment 
 in this topic with more detailed explanation, it will be taken 
 into consideration.


I think the question is, do you think there is positive feedback
which you think would be able to sway your choice? Since boost
mixes the review/vote they list these items:


1. What is your evaluation of the design?
2. What is your evaluation of the implementation?
3. What is your evaluation of the documentation?
4. What is your evaluation of the potential usefulness of the
library?
5. Did you try to use the library? With what compiler? Did you
have any problems?
6. How much effort did you put into your evaluation? A glance? A
quick reading? In-depth study?
7. Are you knowledgeable about the problem domain?

Of which I think 5-7 may be good for inclusion with voting. 1-3
get covered with the review and condition, 4 is somewhat handled
by people stepping up to vote.


 Right now I am more leaning towards declining the module 
 adoption, mostly because "No" vote amount is rather high among 
 Phobos developers, which is a really bad sign.

 It is pretty clear though that this voting has shown certain 
 weak spot in out adoption process for more controversial 
 proposals. Once person (review manager) simply should not have 
 that much decision power in such situation.


In my opinion it must be one person, reaching out for help is
perfectly reasonable. It was clear to me this wasn't going to be
an easy choice. Which is why I started this to give you an idea
of how you should be evaluating these votes. I've certainly been
on either side, here some things I've been thinking about.

* Clearly the library function is desired
* The library is well constructed with decent performance
* Performance improvements may be needed, but I couldn't see
reason this would change the API
* There are some changes which could be made to the
implementation and API which several D members feel would make
the library more "D" like

With the first three points I was thinking passing the library
was the "correct" choice. I do prefer the tk!"<<" style, but it
didn't seem a large enough issue (others didn't like it).

However Andrei and Walter's call out to take advantage of D's
strengths to do things other languages couldn't dream of has
pushed me toward the other side. Don't get me wrong I think
Andrei's suggestion could support the API. And do talk with Brian
to see if he feels he should look at changing the implementation.

Yes, the suggestion of a major implementation change during
voting was unfortunate, but the goal is to get good idiomatic D
libraries and I wouldn't want anyone to feel they can't introduce
information during the vote just because they couldn't review or
didn't have the idea until the voting actually started.

Anyway, that is probably my $5 worth.
Oct 10 2013
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, October 10, 2013 13:57:36 Dicebot wrote:

 It is pretty clear though that this voting has shown certain weak
 spot in out adoption process for more controversial proposals.
 Once person (review manager) simply should not have that much
 decision power in such situation.


I agree. I don' think that it makes any sense for the review manager to have 
the final say if it's questionable whether the module should be included. That 
essentially reduces the vote down to one person if the vote isn't clear. 
Rather, I think that it should be clear that the consensus is for inclusion 
before a module can be included. IMHO, if it's at all unclear, the vote should 
fail. And if that means that controversial stuff doesn't make it in, then it 
doesn't make it in. Better that then put subpar modules into Phobos.

- Jonathan M Davis
Oct 10 2013
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, October 10, 2013 05:14:26 Jesse Phillips wrote:

 There is little documentation on how to handle the situation the
 Review Manager is currently facing. I would like to open this
 discussion to point out why, and to poll for if we should have
 any.
 
 The reason is that our process comes from the the Boost review,
 and there is no such definition[1].
 
 The way we have it makes it appear that the module is accepted as
 a majority vote. However that isn't the intention of the Boost
 process. The Review Manager wields a lot of power during the vote
 tally. The key line in our documentation:
 
 "Tallies votes and decides if there is consensus to accept the
 library and under what conditions."
 
 The Review Manager is trying to judge based on the input how well
 this library fits with the goals of Phobos and the D community.
 That doesn't mean a landslide victory is needed.
 
 So with that I, and probably Dicebot, would like to hear feedback.
 
 Dicebot, consider what information may help make your decision.
 Would yes votes including positive feedback help (it is easier to
 side with those providing an argument)?
 
 1. http://www.boost.org/community/reviews.html#Introduction
 "The final "accept" or "reject" decision is made by the Review
 Manager, based on the review comments received from boost mailing
 list members."
 
 Boost doesn't do a review then vote separation.


I think that the vote needs to be clearly overwhelmingly yes for it to pass. 
Not everyone has to vote yes, and on some level it is the review manager's 
call as to whether the vote is heavily in favor of inclusion, but if the 
percentage of yes-es is high enough for inclusion, then that really should be 
clear to everyone involved that almost everyone is in favor. If it's at all 
unclear that almost everyone is in favor of inclusion, then I think that the 
module should fail the vote and have to be improved such that it will get a 
yes from almost everyone.

Clearly, we can't require a unanimous vote, but I think that it should be at 
least reasonably close to unanimous and that anything closer to 50% than 100% 
should definitely fail. Given that this is the standard library that we're 
talking about, we need to maintain a high standard, and anything that can't 
get at least a two thirds vote is suspect IMHO. And given how the reviews and 
voting have gone thus far, I'd be worried if it was even as low as two-thirds 
in favor of inclusion.

- Jonathan M Davis
Oct 10 2013